I am going to be asking a lot of questions in the upcoming months. For my ninth grade science fair project I would like to create a traffic simulator in order to test whether or not interconnected communicating traffic lights can increase traffic flow. I have a couple of generic questions that I need help with...
How would I represent roads?
How would I make car follow a road?
How would I make a car switch lanes or roads?
I am not looking for specific code, just good pointers and resources to help me get started. Any help is appreciated, C.Ruhl.
PS I am only in high school so no advanced math notations please :)
One possible approach which is taken quite often is to use a discrete model for roads and cars' positions.
Each position on the road can either be occupied by a car (blue dot) or be empty. Cars move at discrete time steps by exactly one position (if the target position is empty) along the given arrows. Thus a car can even switch lanes if it would otherwise have to slow down or stop.
You can further improve it by using separate timesteps per car (simulating faster/slower cars) or in many other ways.
After you've defined your roads (i.e. the positions and their follow-up positions) by an appropriate data structure this model is relatively easy to simulate but already shows interesting effects.
Forget about the UI.
Represent each object in its base form --only put object properties in it. Example, a car will have a size and ability to move. But it won't have the logic to make it move. Similarly a traffic light will have states such as green, amber and red. But it won't have the logic to switch between these states. Similar classes for roads, lanes etc.
Build a different class for the driver. This class will contain all methods such as lane shifting, stopping, turning, moving forward etc. More technically, this will be your "actor" and will act on the veichle. A similar actor would be for traffic light control which will act on a network of traffic lights. Make it an interface and have two implementations to it --one that takes advantage of interconnectedness and other that operates on static times.
Optional add a UI on top of this object model. Don't go fancy, have simple dots to begin with. Once you get all simple stuff working, adding more fancy features should be easy and impact free (relatively).
This will be a very challenging project.
But if your objective is a proof of concept, I have a simpler suggestion. You can go user generated here and get all the complexity of simulation out and all the accuracy in. Start with 15-20 remote controlled cars, a cardboard model of a fictional town, some bulbs to simulate traffic lights and some volunteers who know how to drive. Have a preprogrammed sequence of on and offs written on paper and assign some of the volunteers to control those lights. Have another set of volunteers control the cars. If you have hands on experience in basic electronics you can build a timer controlled circuit to control the lights.
All the very best!
You could try the SIM.JS discrete event simulation library in Javascript. They have a very simple example for traffic at road intersection simulation here.
Ooh, Conner, you've found an interesting question indeed -- and one which is the subject of research even today. Here's a suggestion: before you fret about how to do it in JavaScript, spend some time thinking just how to do it at all.
Here's a suggestion: think about the objects invovled first. You have Cars, and they travel along Roads. Start with a square grid of roads, so your cars go from intersection to intersection.
Pick a fixed speed for the cars, so it takes a constant time to travel from intersection to intersection.
Each intersection has a traffic light, which can be red or green. If it's red, of course cars can't go through; they have to wait.
Now, your basic program will look like
time = 0
while time < end-time:
for each car:
update the car's location
add time consumed to time
when you update the cars location, what happens? (Hint: the car moves; can it go through an intersection or not?)
That will give you a start.
For my Bachelor's degree exam I developed a traffic control web-app that tracked the vehicles in my town in real-time and I used google maps api.
I suggest you use a map service such as maps.google.com , yahoo.maps.com...
They have an api for everything... you can use markers to represent anything on the map (cars,street lights,even pedestrians :)) ) and you can use their api to calculate distances and paths.
It may seem a bit more complex then the average div-implementation, but, trust me, it's a big plus to use a service with a well-organised api.
+it would have a more professional look ;).
Related
I am making a neural network with reinforcement.
The model looks like this:
63 input neurons (environment state) - 21 neurons in the hidden layer - 4 outputs. The output neurons contain the probability of going up, down, left, right. ([0,0,0,1])
The neural network gives the result of the move, the agent performs an action.
Each new move, after the agent has performed actions, I give him a reward or a penalty.
How to do backpropagation in tensor flow js? I need not an error to propagate back, but a reward or a penalty. And only from a certain output neuron.
enter image description here
Example:
The neural network predicted the move to the right
the agent went to the right and left the playing field.
This is a bad action. The agent is charged a fine of -0.02
In the current model of the neural network, we determine the output neuron that responds to the move to the right.
We backpropagate from this neuron back with a coefficient of -0.02. If it is not a fine, but a reward, then the coefficient will be positive.
How to do step 5 in code?
UDP:I initially thought that the task is simple and does not require additional clarification. Therefore, I formulated the question briefly. I think it's worth giving more information :) The game consists of 10 squares in width and 10 in height, a total of 100. There are 20 chicken legs in a static position on the playing field. The agent's task is to collect as many chicken legs as possible. I started my research with a genetic algorithm. I created a tensor flow model, in which I submitted the state of the game to the input. I didn't teach the model. In tensors, it's just a set of random weights. After each pass of the game, we choose the winners, cross them and mutate a little. The crossing and mutation itself occurs directly in the neural network that is attached to each agent. I do not teach the system, I take weights from the neural network (brain) of the agent and perform mutation and crossing directly, I change the coefficients in tensors. Result = 10 chicken legs. This is not bad, the agents are really trained, but I am not satisfied with the result. I now want to use reinforcement learning. I'm new to this field and I can't find examples anywhere of exactly how to praise or fine a neuron for wrong actions. It is in the form of a code. I understand the concept of the award, but how to implement it...I can't think without the order of actions. For example, the agent walked across the playing field 1 time. He made 4 moves to the left and went outside the playing field. On the 2nd move, he hit the cage with a chicken leg. The experiment is over. Every move, I saved the state of the game (the game environment for input neurons) to an array and saved the rewards to another array [0,1,0,-1] => 1 - reward for a chicken leg, -1 for going beyond.
How do I now teach the system with this data?
(I assumed that it was necessary to reduce the weights of y along the branch from the wrong output neuron to the incoming data by a gradient. Not training the neural network at all, but working purely with weights)
Consider that RL agents do not learn from what is a good or bad action, but they do learn from rewards.
An action is never explicitly tagged as good or bad, just a reward is given back. and usually rewards comes as consequence of several actions.
Consider a simple agent that learns to play tic tac toe. Usually reward is 1 for winning, -1 when looses and 0 for ties. For any other action (i.e. intermediate actions while playing is going on) reward is 0.
Now the agent makes it first move: whatever it is, it will get 0 as reward. Was it a good or a bad move? You don't know until the end of the game. And more formally, you don't even know at the end: the agent may win a game even with a very bad initial movement.
This is also valid for any other action, not only the initial. To guess how good was an action in RL (and in any trial-error framework) is known as the credit assignment problem, and is not explicitly solved.
This is what makes RL very different from supervised learning: in supervised learning, you have pairs of inputs and expected outputs, that tell how to solve you task. In RL you just know what to do (win a tic tac toe game) but you don't know how to do it, so you just pay the agent once he does it ok.
That said, there are several formulations on how to solve an RL problem, from your question I assume you're trying to do Q-learning: the agent learns a function called Q which takes a state and an action and tell how good is the action for the state. When the function is modeled as a neural network, it usually takes as input a state and outputs the Q for each action: exactly as the model you share in the image.
And now we arrive to your question: how do you update the function? Well, first you need to take an action from your current state. Which to take? The best? Well, since your agent is learning, taking always the action he think is the best won't help: he will be wrong most of the time and will only try bad movements. A random one? This may help, but it is also problematic since randomly playing is hard to achieve the task and get paid, so it will not succeed and never learns. What I introduced here is the exploration-exploitation dilema: what to do, expoit agent's knowledge taking the action supposed to be the best or explore trying something different to guess what happens? You need to do both, and an appropriate balance among them is crucial. A simple, yet effective way to achieve this is the epsilon-greedy strategy: with a probability of epsilon take a greedy action (best up to agent's knowledge) otherwise take a random action (explore).
Once you take an action, you receive a reward, and it's time to learn from this reward and update your function. For this, you have your current Q function at time t and the brand-new reward, at time t, and want to update the Q function in time t+1.
There are some small details not covered here to make it work, just wanted to clarify some points. For a keras based implementation, take a look at this.
Sorry for my bad English.
I do a little game in JavaScript using Box2D (although I think the language is not important, because the engine is the same, as is well known, and has been exported to JavaScript).
Connecting several bodies via Distance Joint. When creating a joint pointing frequencyHz = 11 and dampingRatio = 0 (as I understand it, the less dampingRatio, the faster the item returns to its original position, that is, the spring will be tougher that I need). For each body to use force:
body.ApplyForce(new b2Vec2(0, 5800), body.GetWorldCenter())
That I needed to have these bodies was greater gravity than others.
After creating several elements, they begin to twitch, to vibrate as if they were something haunted, stop and engage in a certain position.
I tried to "play" with parameters frequencyHz and dampingRatio have joint, and sought to design not twitch, but then the construction seems less harsh and smoother.
There is some sort of solution?
Thanks.
I think that the structure is vibrating due to the fact that the weight of several elements, and the power applicable to them, the object is pulled down by increasing the length of the joint, the joint is aimed szhimatsya back to its original position, due to this vibration is obtained, because the element moves quickly in different directions
I'm relatively new to HTML and Javascript, but I'm knee deep in the Udacity interactive 3D course and have gotten my hands dirty with some three.js + WebGL. And I've been able to make and somewhat understand this:
http://goo.gl/UPWKKL
So far.(having a hard time understanding the API and getting cannon.js and really any interesting mechanics to work, any advice for learning APIs like threejs?)
I was wondering if anyone could provide any input for someone whose end goal is to make a game that is somewhat like a demi-version of: REZ, Exteel, Armored Core or Zone of The Enders versus mode.
My goal is implementing: rail shooting(w/ cannon.js?), health bars, NPC boss battles with different stages, animated movements, a cross-hair, level bounds, concepts of upgrades to a character.
To be really specific, a 5 level game with PointerLockControl + shooting interface, where each level pass requires bringing a boss' health bar down to zero. The enemy would have a vulnerable mesh area where if bullet objects hit it, it'd trigger a collision event where its health decreased. If health<= 25 it speeds up and becomes harder to kill. After its death the screen blacks out and restarts with a new boss and so on. I'd want to put in victory screens, failure screens and if possible, cut scenes where I guess I'd disable user control and enable some kind of path cinematic camera. And preferrably for this to all be in the browser like Quake, BUT if something like this isn't possible, I'd try something else.
Sorry if this question is too broad or weird, I want to work on video games for a living, I will appreciate any feedback I get, I just want to know if someone more experienced can look at what kind of game I want to make and recommend some up to date material or helpful sites.
Currently I'm working with webGL and threejs, I've looked into Unity3D but I can't develop that on my Linux machine. Far FARR down the line I'd like make full blown games in C++.
Design as specifically as possible, because then you will have lots of small tasks whose role in the greater whole is known. Then if you don't know what to do on any given day, just look at your design, your map, and pick a piece that you can do that day.
Sorry if this answer is not specific to WebGL but you have asked broadly.
I was looking into implementing an Inertial Navigation System for an Android phone, which I realise is hard given the accelerometer accuracy, and constant fluctuation of readings.
To start with, I set the phone on a flat surface and sampled 1000 accelerometer readings in the X and Y directions (parallel to the table, so no gravity acting in these directions). I then averaged these readings and used this value to calibrate the phone (subtracting this value from each subsequent reading).
I then tested the system by again placing it on the table and sampling 5000 accelerometer readings in the X and Y directions. I would expect, given the calibration, that these accelerations should add up to 0 (roughly) in each direction. However, this is not the case, and the total acceleration over 5000 iterations is nowhere near 0 (averaging around 10 on each axis).
I realise without seeing my code this might be difficult to answer but in a more general sense...
Is this simply an example of how inaccurate the accelerometer readings are on a mobile phone (HTC Desire S), or is it more likely that I've made some errors in my coding?
You get position by integrating the linear acceleration twice but the error is horrible. It is useless in practice.
Here is an explanation why (Google Tech Talk) at 23:20. I highly recommend this video.
It is not the accelerometer noise that causes the problem but the gyro white noise, see subsection 6.2.3 Propagation of Errors. (By the way, you will need the gyroscopes too.)
As for indoor positioning, I have found these useful:
RSSI-Based Indoor Localization and Tracking Using Sigma-Point Kalman Smoothers
Pedestrian Tracking with Shoe-Mounted Inertial Sensors
Enhancing the Performance of Pedometers Using a Single Accelerometer
I have no idea how these methods would perform in real-life applications or how to turn them into a nice Android app.
A similar question is this.
UPDATE:
Apparently there is a newer version than the above Oliver J. Woodman, "An introduction to inertial navigation", his PhD thesis:
Pedestrian Localisation for Indoor Environments
I am just thinking out loud, and I haven't played with an android accelerometer API yet, so bear with me.
First of all, traditionally, to get navigation from accelerometers you would need a 6-axis accelerometer. You need accelerations in X, Y, and Z, but also rotations Xr, Yr, and Zr. Without the rotation data, you don't have enough data to establish a vector unless you assume the device never changes it's attitude, which would be pretty limiting. No one reads the TOS anyway.
Oh, and you know that INS drifts with the rotation of the earth, right? So there's that too. One hour later and you're mysteriously climbing on a 15° slope into space. That's assuming you had an INS capable of maintaining location that long, which a phone can't do yet.
A better way to utilize accelerometers -even with a 3-axis accelerometer- for navigation would be to tie into GPS to calibrate the INS whenever possible. Where GPS falls short, INS compliments nicely. GPS can suddenly shoot you off 3 blocks away because you got too close to a tree. INS isn't great, but at least it knows you weren't hit by a meteor.
What you could do is log the phones accelerometer data, and a lot of it. Like weeks worth. Compare it with good (I mean really good) GPS data and use datamining to establish correlation of trends between accelerometer data and known GPS data. (Pro tip: You'll want to check the GPS almanac for days with good geometry and a lot of satellites. Some days you may only have 4 satellites and that's not enough) What you might be able to do is find that when a person is walking with their phone in their pocket, the accelerometer data logs a very specific pattern. Based on the datamining, you establish a profile for that device, with that user, and what sort of velocity that pattern represents when it had GPS data to go along with it. You should be able to detect turns, climbing stairs, sitting down (calibration to 0 velocity time!) and various other tasks. How the phone is being held would need to be treated as separate data inputs entirely. I smell a neural network being used to do the data mining. Something blind to what the inputs mean, in other words. The algorithm would only look for trends in the patterns, and not really paying attention to the actual measurements of the INS. All it would know is historically, when this pattern occurs, the device is traveling and 2.72 m/s X, 0.17m/s Y, 0.01m/s Z, so the device must be doing that now. And it would move the piece forward accordingly. It's important that it's completely blind, because just putting a phone in your pocket might be oriented in one of 4 different orientations, and 8 if you switch pockets. And there's many ways to hold your phone, as well. We're talking a lot of data here.
You'll obviously still have a lot of drift, but I think you'd have better luck this way because the device will know when you stopped walking, and the positional drift will not be a perpetuating. It knows that you're standing still based on historical data. Traditional INS systems don't have this feature. The drift perpetuates to all future measurements and compounds exponentially. Ungodly accuracy, or having a secondary navigation to check with at regular intervals, is absolutely vital with traditional INS.
Each device, and each person would have to have their own profile. It's a lot of data and a lot of calculations. Everyone walks different speeds, with different steps, and puts their phones in different pockets, etc. Surely to implement this in the real world would require number-crunching to be handled server-side.
If you did use GPS for the initial baseline, part of the problem there is GPS tends to have it's own migrations over time, but they are non-perpetuating errors. Sit a receiver in one location and log the data. If there's no WAAS corrections, you can easily get location fixes drifting in random directions 100 feet around you. With WAAS, maybe down to 6 feet. You might actually have better luck with a sub-meter RTK system on a backpack to at least get the ANN's algorithm down.
You will still have angular drift with the INS using my method. This is a problem. But, if you went so far to build an ANN to pour over weeks worth of GPS and INS data among n users, and actually got it working to this point, you obviously don't mind big data so far. Keep going down that path and use more data to help resolve the angular drift: People are creatures of habit. We pretty much do the same things like walk on sidewalks, through doors, up stairs, and don't do crazy things like walk across freeways, through walls, or off balconies.
So let's say you are taking a page from Big Brother and start storing data on where people are going. You can start mapping where people would be expected to walk. It's a pretty sure bet that if the user starts walking up stairs, she's at the same base of stairs that the person before her walked up. After 1000 iterations and some least-squares adjustments, your database pretty much knows where those stairs are with great accuracy. Now you can correct angular drift and location as the person starts walking. When she hits those stairs, or turns down that hall, or travels down a sidewalk, any drift can be corrected. Your database would contain sectors that are weighted by the likelihood that a person would walk there, or that this user has walked there in the past. Spatial databases are optimized for this using divide and conquer to only allocate sectors that are meaningful. It would be sort of like those MIT projects where the laser-equipped robot starts off with a black image, and paints the maze in memory by taking every turn, illuminating where all the walls are.
Areas of high traffic would get higher weights, and areas where no one has ever been get 0 weight. Higher traffic areas are have higher resolution. You would essentially end up with a map of everywhere anyone has been and use it as a prediction model.
I wouldn't be surprised if you could determine what seat a person took in a theater using this method. Given enough users going to the theater, and enough resolution, you would have data mapping each row of the theater, and how wide each row is. The more people visit a location, the higher fidelity with which you could predict that that person is located.
Also, I highly recommend you get a (free) subscription to GPS World magazine if you're interested in the current research into this sort of stuff. Every month I geek out with it.
I'm not sure how great your offset is, because you forgot to include units. ("Around 10 on each axis" doesn't say much. :P) That said, it's still likely due to inaccuracy in the hardware.
The accelerometer is fine for things like determining the phone's orientation relative to gravity, or detecting gestures (shaking or bumping the phone, etc.)
However, trying to do dead reckoning using the accelerometer is going to subject you to a lot of compound error. The accelerometer would need to be insanely accurate otherwise, and this isn't a common use case, so I doubt hardware manufacturers are optimizing for it.
Android accelerometer is digital, it samples acceleration using the same number of "buckets", lets say there are 256 buckets and the accelerometer is capable of sensing from -2g to +2g. This means that your output would be quantized in terms of these "buckets" and would be jumping around some set of values.
To calibrate an android accelerometer, you need to sample a lot more than 1000 points and find the "mode" around which the accelerometer is fluctuating. Then find the number of digital points by how much the output fluctuates and use that for your filtering.
I recommend Kalman filtering once you get the mode and +/- fluctuation.
I realise this is quite old, but the issue at hand is not addressed in ANY of the answers given.
What you are seeing is the linear acceleration of the device including the effect of gravity. If you lay the phone on a flat surface the sensor will report the acceleration due to gravity which is approximately 9.80665 m/s2, hence giving the 10 you are seeing. The sensors are inaccurate, but they are not THAT inaccurate! See here for some useful links and information about the sensor you may be after.
You are making the assumption that the accelerometer readings in the X and Y directions, which in this case is entirely hardware noise, would form a normal distribution around your average. Apparently that is not the case.
One thing you can try is to plot these values on a graph and see whether any pattern emerges. If not then the noise is statistically random and cannot be calibrated against--at least for your particular phone hardware.
I've been trying to create something (much more basic) as in the video linked below:
http://blog.theclinic.eu/?p=653 (Should start at 14:30 ish)
It's a talk by John Maeda & he demonstrates his application called Powershop. I've been trying to recreate something that had the same effect, but I can't figure out how I could make the circles stay at a distance like that without collision, any suggestions on how to approach it?
I've tried to make the constraints have a min & max distance so that they'd follow around, I tried to let circles orbit, but none seems to look like his.
I'm trying this with javascript/ HTML5, but I would just like some tips or suggestions on the approach! (My physics/math knowledge is limited, so I might be missing out on some obvious concept to apply?)
Thanks in advance!
If you are looking for a nice canned solution, instead of building everything from the ground up, I suggest you look at Box2D.
The system you see in the Powershop graphics is a simple physical model where each "ball" or "node" or whathaveyou is a charged object that repels all the other balls/nodes. If you model the system as a collection of points connected by freely-rotating lines, all you have to do is understand a little physics to get this effect working with Box2D. Namely, like charges repel.
It should be that easy. You will constrain a set of points so that each has to stay within a length L from a center, and this will be your model of the arms all the balls rotate on. Then you will give each point the exact same charge and they will repel each other and spread out evenly with nice bouncy effects.
(The part where you get creative is how you want to add a whole new collection of points, like when the speaker in your Powershop talk clicks on a node.)