I am making a neural network with reinforcement.
The model looks like this:
63 input neurons (environment state) - 21 neurons in the hidden layer - 4 outputs. The output neurons contain the probability of going up, down, left, right. ([0,0,0,1])
The neural network gives the result of the move, the agent performs an action.
Each new move, after the agent has performed actions, I give him a reward or a penalty.
How to do backpropagation in tensor flow js? I need not an error to propagate back, but a reward or a penalty. And only from a certain output neuron.
enter image description here
Example:
The neural network predicted the move to the right
the agent went to the right and left the playing field.
This is a bad action. The agent is charged a fine of -0.02
In the current model of the neural network, we determine the output neuron that responds to the move to the right.
We backpropagate from this neuron back with a coefficient of -0.02. If it is not a fine, but a reward, then the coefficient will be positive.
How to do step 5 in code?
UDP:I initially thought that the task is simple and does not require additional clarification. Therefore, I formulated the question briefly. I think it's worth giving more information :) The game consists of 10 squares in width and 10 in height, a total of 100. There are 20 chicken legs in a static position on the playing field. The agent's task is to collect as many chicken legs as possible. I started my research with a genetic algorithm. I created a tensor flow model, in which I submitted the state of the game to the input. I didn't teach the model. In tensors, it's just a set of random weights. After each pass of the game, we choose the winners, cross them and mutate a little. The crossing and mutation itself occurs directly in the neural network that is attached to each agent. I do not teach the system, I take weights from the neural network (brain) of the agent and perform mutation and crossing directly, I change the coefficients in tensors. Result = 10 chicken legs. This is not bad, the agents are really trained, but I am not satisfied with the result. I now want to use reinforcement learning. I'm new to this field and I can't find examples anywhere of exactly how to praise or fine a neuron for wrong actions. It is in the form of a code. I understand the concept of the award, but how to implement it...I can't think without the order of actions. For example, the agent walked across the playing field 1 time. He made 4 moves to the left and went outside the playing field. On the 2nd move, he hit the cage with a chicken leg. The experiment is over. Every move, I saved the state of the game (the game environment for input neurons) to an array and saved the rewards to another array [0,1,0,-1] => 1 - reward for a chicken leg, -1 for going beyond.
How do I now teach the system with this data?
(I assumed that it was necessary to reduce the weights of y along the branch from the wrong output neuron to the incoming data by a gradient. Not training the neural network at all, but working purely with weights)
Consider that RL agents do not learn from what is a good or bad action, but they do learn from rewards.
An action is never explicitly tagged as good or bad, just a reward is given back. and usually rewards comes as consequence of several actions.
Consider a simple agent that learns to play tic tac toe. Usually reward is 1 for winning, -1 when looses and 0 for ties. For any other action (i.e. intermediate actions while playing is going on) reward is 0.
Now the agent makes it first move: whatever it is, it will get 0 as reward. Was it a good or a bad move? You don't know until the end of the game. And more formally, you don't even know at the end: the agent may win a game even with a very bad initial movement.
This is also valid for any other action, not only the initial. To guess how good was an action in RL (and in any trial-error framework) is known as the credit assignment problem, and is not explicitly solved.
This is what makes RL very different from supervised learning: in supervised learning, you have pairs of inputs and expected outputs, that tell how to solve you task. In RL you just know what to do (win a tic tac toe game) but you don't know how to do it, so you just pay the agent once he does it ok.
That said, there are several formulations on how to solve an RL problem, from your question I assume you're trying to do Q-learning: the agent learns a function called Q which takes a state and an action and tell how good is the action for the state. When the function is modeled as a neural network, it usually takes as input a state and outputs the Q for each action: exactly as the model you share in the image.
And now we arrive to your question: how do you update the function? Well, first you need to take an action from your current state. Which to take? The best? Well, since your agent is learning, taking always the action he think is the best won't help: he will be wrong most of the time and will only try bad movements. A random one? This may help, but it is also problematic since randomly playing is hard to achieve the task and get paid, so it will not succeed and never learns. What I introduced here is the exploration-exploitation dilema: what to do, expoit agent's knowledge taking the action supposed to be the best or explore trying something different to guess what happens? You need to do both, and an appropriate balance among them is crucial. A simple, yet effective way to achieve this is the epsilon-greedy strategy: with a probability of epsilon take a greedy action (best up to agent's knowledge) otherwise take a random action (explore).
Once you take an action, you receive a reward, and it's time to learn from this reward and update your function. For this, you have your current Q function at time t and the brand-new reward, at time t, and want to update the Q function in time t+1.
There are some small details not covered here to make it work, just wanted to clarify some points. For a keras based implementation, take a look at this.
Related
I have a javascript canvas game with pixi.js that requires a player to press a certain combination of buttons to complete a level. They basically have to press the button that matches a certain color.
It turns out that players are writing bots in python to do this task and they are getting the maxium score each time. The game is already live and users enjoy playing it so I can't really change anything gameplay wise.
So I thought about a few possible solutions but I have some concerns
Captcha between each level
Check the speed of the input
Check how consistent the input is
The captcha will hurt user experience, and there are tons of video's how to bypass it. 2 and 3 will fail after the creators of the bots understand what is happening. So I am really stuck on what I could do.
I would consider a random grace period before allowing the buttons to be clicked. this may stump some bots, but is circumventable.
Besides that, I would profile the timing of the clicks/interactions. Every time next level is requested, compare to the profile, and if they are consistently the same introduce a randomized button id, button shape (circle, oval, square, etc.), button placement (swap buttons) to avoid easy scripting. Also the font and the actual text could be varied.
I would also change the input element to <input type="image"> since it will give you the exact coordinates (if possible - I'm not familiar with pixi.js) and this will aid in the profiling.
You could also implement some sort of mouse position tracker, but people on touchscreens will not produce data for this. You could supplement with additional check if the user input is touch, but a bot would easily be able to circumvent it.
EDIT
I don't know if some library to detect other JavaScript imports and thereby detecting potential bots would be applicable. Might be one avenue to consider.
Doing something like this: Check whether user has a Chrome extension installed to verify that you are running in a browser and not in a python environment could be another avenue. It would mean that you restrict your users to certain browsers, and as a lot of other code, could be circumvented. Cost/benefit should be kept in mind here.
If everything is being run though the actual browser with some sort of headless interface it is not going to be useful at all.
EDIT 2
A quick googling of python automate browser game brings up a tutorial of how to automate browser games with python. based on a cursory glance, making your buttons move around and changing font would be effective, and even resizing the playing area "randomly" (even if you have a full screen function) may be a viable defense. Again, following the tutorial and trying to automate it using that, and seeing how to block it would be a good exercise.
You could also consider asking some students for help. This could be a good project idea for many computer studies courses that offer project based courses. It could also be a student job type deal - if you want to ensure that you get a result and a "report".
I think your approach is valid. It seems a bit excessive to add Capcha between each level, perhaps add it before the game starts.
It might be a good idea to check interval between individual clicks, and define some threshold when you can safely assume that it was a bot who clicked the button.
Another approach you could take is to make it more complicated to look up the correct buttons. Approaches like randomizing the element IDs, not rendering the label inside the buttons but as separate elements (I assume it is a game with some fixed window size and you don't care about mobile that much).
I am not familiar with Pixi.js, but that could be an approach to consider.
----------------------- Edit -----------------------
What if you run your game in an iframe ?
I am currently building a web application that acts as a storage tank level dashboard. It parses incoming data from a number of sensors in tanks and stores these values in a database. The application is built using express / node.js. The data is sampled every 5 minutes but is sent to the server every hour (12 samples per transmission).
I am currently trying to expand the application's capabilities to detect changes in the tank level due to filling or emptying. The end goal is to have a daily report that generates a summary of the filling / emptying events with the duration of time and quantity added or removed. This image shows a screenshot of tank capacity during one day - https://imgur.com/a/kZ50N.
My questions are:
What algorithms / functions are available that detects the changes in tank level? How would I implement them into my application?
When should the data handling take place? As the data is parsed and saved into the server? At the end of the day with a function that goes through all the data for that day?
Is it worth considering some sort of data cleaning during the parsing stage? I have noticed times when there are random spikes in the data due to noise.
How should I handle events when they immediately start emptying the tank immediately after completing a delivery? I will need the algorithm to be robust enough that it detects a change in the direction of the slope to be the end of an event. Example of this is in the provided image.
I realise that it may difficult to put together a robust solution. There are times when the tank is being emptied at the same time that it is being filled. This makes it difficult to measure these reductions. The only was to know that this took place is the slope of during the delivery flatlines for approximately 15 minutes and the delivery is a fixed amount less than the usual delivery total.
This has been a fun project to put together. Thanks for any assistance.
You should be able to develop an algorithm that specifies what you mean by a fill or en emptying (a change in tank level). A good place to start is X% in Y seconds. You then calibrate to avoid false positives or false negatives (e.g. showing a fill when there was none vs. missing a fill when it occurs. One potential approach is to average the fuel level over a period of time (say 10 minutes) and compare it with the average for the next 10 minutes. If there is a difference above a threshold (say 5%), you can call this a change.
When you process the data depends on when you need it, so if the users need to be constantly informed of changes, this could be done on querying of the data. Processing the data into changes in level on write to your datastore might be more efficient (you only do it once), however you lose the ability to tweak your algorithm. It could well depend on performance, e.g. if someone wants to pull a years worth of data, is the system able to deal with this?
You will almost certainly need to do something like a low pass filter on the incoming data. You don't want to show a tank fill based on a temporary spike in level. This is easy to do with an array of values. As mentioned above, a moving average, say of the last 10 minutes of levels is another way of smoothing the data. You may never get a 0% false positive rate or a 0% false negative rate, you can only aim for values as low as possible.
In this case it looks like a fill followed by an emptying of the tank. If you consider these to be two separate events then you can simply detect changes on the incoming data. I'd suggest you create a graph marking fills as a symbol on the graph as well as emptying. This way you can eyeball the data to ensure you are detecting changes. I would also say you could add some very useful unit tests for your calculations using perhaps jasmin.js or cucumber.js.
I was looking into implementing an Inertial Navigation System for an Android phone, which I realise is hard given the accelerometer accuracy, and constant fluctuation of readings.
To start with, I set the phone on a flat surface and sampled 1000 accelerometer readings in the X and Y directions (parallel to the table, so no gravity acting in these directions). I then averaged these readings and used this value to calibrate the phone (subtracting this value from each subsequent reading).
I then tested the system by again placing it on the table and sampling 5000 accelerometer readings in the X and Y directions. I would expect, given the calibration, that these accelerations should add up to 0 (roughly) in each direction. However, this is not the case, and the total acceleration over 5000 iterations is nowhere near 0 (averaging around 10 on each axis).
I realise without seeing my code this might be difficult to answer but in a more general sense...
Is this simply an example of how inaccurate the accelerometer readings are on a mobile phone (HTC Desire S), or is it more likely that I've made some errors in my coding?
You get position by integrating the linear acceleration twice but the error is horrible. It is useless in practice.
Here is an explanation why (Google Tech Talk) at 23:20. I highly recommend this video.
It is not the accelerometer noise that causes the problem but the gyro white noise, see subsection 6.2.3 Propagation of Errors. (By the way, you will need the gyroscopes too.)
As for indoor positioning, I have found these useful:
RSSI-Based Indoor Localization and Tracking Using Sigma-Point Kalman Smoothers
Pedestrian Tracking with Shoe-Mounted Inertial Sensors
Enhancing the Performance of Pedometers Using a Single Accelerometer
I have no idea how these methods would perform in real-life applications or how to turn them into a nice Android app.
A similar question is this.
UPDATE:
Apparently there is a newer version than the above Oliver J. Woodman, "An introduction to inertial navigation", his PhD thesis:
Pedestrian Localisation for Indoor Environments
I am just thinking out loud, and I haven't played with an android accelerometer API yet, so bear with me.
First of all, traditionally, to get navigation from accelerometers you would need a 6-axis accelerometer. You need accelerations in X, Y, and Z, but also rotations Xr, Yr, and Zr. Without the rotation data, you don't have enough data to establish a vector unless you assume the device never changes it's attitude, which would be pretty limiting. No one reads the TOS anyway.
Oh, and you know that INS drifts with the rotation of the earth, right? So there's that too. One hour later and you're mysteriously climbing on a 15° slope into space. That's assuming you had an INS capable of maintaining location that long, which a phone can't do yet.
A better way to utilize accelerometers -even with a 3-axis accelerometer- for navigation would be to tie into GPS to calibrate the INS whenever possible. Where GPS falls short, INS compliments nicely. GPS can suddenly shoot you off 3 blocks away because you got too close to a tree. INS isn't great, but at least it knows you weren't hit by a meteor.
What you could do is log the phones accelerometer data, and a lot of it. Like weeks worth. Compare it with good (I mean really good) GPS data and use datamining to establish correlation of trends between accelerometer data and known GPS data. (Pro tip: You'll want to check the GPS almanac for days with good geometry and a lot of satellites. Some days you may only have 4 satellites and that's not enough) What you might be able to do is find that when a person is walking with their phone in their pocket, the accelerometer data logs a very specific pattern. Based on the datamining, you establish a profile for that device, with that user, and what sort of velocity that pattern represents when it had GPS data to go along with it. You should be able to detect turns, climbing stairs, sitting down (calibration to 0 velocity time!) and various other tasks. How the phone is being held would need to be treated as separate data inputs entirely. I smell a neural network being used to do the data mining. Something blind to what the inputs mean, in other words. The algorithm would only look for trends in the patterns, and not really paying attention to the actual measurements of the INS. All it would know is historically, when this pattern occurs, the device is traveling and 2.72 m/s X, 0.17m/s Y, 0.01m/s Z, so the device must be doing that now. And it would move the piece forward accordingly. It's important that it's completely blind, because just putting a phone in your pocket might be oriented in one of 4 different orientations, and 8 if you switch pockets. And there's many ways to hold your phone, as well. We're talking a lot of data here.
You'll obviously still have a lot of drift, but I think you'd have better luck this way because the device will know when you stopped walking, and the positional drift will not be a perpetuating. It knows that you're standing still based on historical data. Traditional INS systems don't have this feature. The drift perpetuates to all future measurements and compounds exponentially. Ungodly accuracy, or having a secondary navigation to check with at regular intervals, is absolutely vital with traditional INS.
Each device, and each person would have to have their own profile. It's a lot of data and a lot of calculations. Everyone walks different speeds, with different steps, and puts their phones in different pockets, etc. Surely to implement this in the real world would require number-crunching to be handled server-side.
If you did use GPS for the initial baseline, part of the problem there is GPS tends to have it's own migrations over time, but they are non-perpetuating errors. Sit a receiver in one location and log the data. If there's no WAAS corrections, you can easily get location fixes drifting in random directions 100 feet around you. With WAAS, maybe down to 6 feet. You might actually have better luck with a sub-meter RTK system on a backpack to at least get the ANN's algorithm down.
You will still have angular drift with the INS using my method. This is a problem. But, if you went so far to build an ANN to pour over weeks worth of GPS and INS data among n users, and actually got it working to this point, you obviously don't mind big data so far. Keep going down that path and use more data to help resolve the angular drift: People are creatures of habit. We pretty much do the same things like walk on sidewalks, through doors, up stairs, and don't do crazy things like walk across freeways, through walls, or off balconies.
So let's say you are taking a page from Big Brother and start storing data on where people are going. You can start mapping where people would be expected to walk. It's a pretty sure bet that if the user starts walking up stairs, she's at the same base of stairs that the person before her walked up. After 1000 iterations and some least-squares adjustments, your database pretty much knows where those stairs are with great accuracy. Now you can correct angular drift and location as the person starts walking. When she hits those stairs, or turns down that hall, or travels down a sidewalk, any drift can be corrected. Your database would contain sectors that are weighted by the likelihood that a person would walk there, or that this user has walked there in the past. Spatial databases are optimized for this using divide and conquer to only allocate sectors that are meaningful. It would be sort of like those MIT projects where the laser-equipped robot starts off with a black image, and paints the maze in memory by taking every turn, illuminating where all the walls are.
Areas of high traffic would get higher weights, and areas where no one has ever been get 0 weight. Higher traffic areas are have higher resolution. You would essentially end up with a map of everywhere anyone has been and use it as a prediction model.
I wouldn't be surprised if you could determine what seat a person took in a theater using this method. Given enough users going to the theater, and enough resolution, you would have data mapping each row of the theater, and how wide each row is. The more people visit a location, the higher fidelity with which you could predict that that person is located.
Also, I highly recommend you get a (free) subscription to GPS World magazine if you're interested in the current research into this sort of stuff. Every month I geek out with it.
I'm not sure how great your offset is, because you forgot to include units. ("Around 10 on each axis" doesn't say much. :P) That said, it's still likely due to inaccuracy in the hardware.
The accelerometer is fine for things like determining the phone's orientation relative to gravity, or detecting gestures (shaking or bumping the phone, etc.)
However, trying to do dead reckoning using the accelerometer is going to subject you to a lot of compound error. The accelerometer would need to be insanely accurate otherwise, and this isn't a common use case, so I doubt hardware manufacturers are optimizing for it.
Android accelerometer is digital, it samples acceleration using the same number of "buckets", lets say there are 256 buckets and the accelerometer is capable of sensing from -2g to +2g. This means that your output would be quantized in terms of these "buckets" and would be jumping around some set of values.
To calibrate an android accelerometer, you need to sample a lot more than 1000 points and find the "mode" around which the accelerometer is fluctuating. Then find the number of digital points by how much the output fluctuates and use that for your filtering.
I recommend Kalman filtering once you get the mode and +/- fluctuation.
I realise this is quite old, but the issue at hand is not addressed in ANY of the answers given.
What you are seeing is the linear acceleration of the device including the effect of gravity. If you lay the phone on a flat surface the sensor will report the acceleration due to gravity which is approximately 9.80665 m/s2, hence giving the 10 you are seeing. The sensors are inaccurate, but they are not THAT inaccurate! See here for some useful links and information about the sensor you may be after.
You are making the assumption that the accelerometer readings in the X and Y directions, which in this case is entirely hardware noise, would form a normal distribution around your average. Apparently that is not the case.
One thing you can try is to plot these values on a graph and see whether any pattern emerges. If not then the noise is statistically random and cannot be calibrated against--at least for your particular phone hardware.
I am building a Sudoku game for fun, written in Javascript.
Everything works fine, board is generated completely with a single solution each time.
My only problem is, and this is what's keeping me from having my project released to public
is that I don't know how to grade my boards for difficulty levels. I've looked EVERYWHERE,
posted on forums, etc. I don't want to write the algorithms myself, thats not the point of this
project,and beside, they are too complex for me, as i am no mathematician.
The only thing i came close to was is this website that does grading via JSbut the problem is, the code is written in such a lousy undocumented, very ad-hoc manner,therefor cannot be borrowed...
I'll come to the point -Can anyone please point me to a place which offers a source code for Sudoku grading/rating?
Thanks
Update 22.6.11:
This is my Sudoku game, and I've implemented my own grading system which relies
on basic human logic solving techniques, so check it out.
I have considered this problem myself and the best I can do is to decide how difficult the puzzle is to solve by actually solving it and analyzing the game tree.
Initially:
Implement your solver using "human rules", not with algorithms unlikely to be used by human players. (An interesting problem in its own right.) Score each logical rule in your solver according to its difficulty for humans to use. Use values in the hundreds or larger so you have freedom to adjust the scores relative to each other.
Solve the puzzle. At each position:
Enumerate all new cells which can be logically deduced at the current game position.
The score of each deduction (completely solving one cell) is the score of the easiest rule that suffices to make that deduction.
EDIT: If more than one rule must be applied together, or one rule multiple times, to make a single deduction, track it as a single "compound" rule application. To score a compound, maybe use the minimum number of individual rule applications to solve a cell times the sum of the scores of each. (Considerably more mental effort is required for such deductions.) Calculating that minimum number of applications could be a CPU-intensive effort depending on your rules set. Any rule application that completely solves one or more cells should be rolled back before continuing to explore the position.
Exclude all deductions with a score higher than the minimum among all deductions. (The logic here is that the player will not perceive the harder ones, having perceived an easier one and taken it; and also, this promises to prune a lot of computation out of the decision process.)
The minimum score at the current position, divided by the number of "easiest" deductions (if many exist, finding one is easier) is the difficulty of that position. So if rule A is the easiest applicable rule with score 20 and can be applied in 4 cells, the position has score 5.
Choose one of the "easiest" deductions at random as your play and advance to the next game position. I suggest retaining only completely solved cells for the next position, passing no other state. This is wasteful of CPU of course, repeating computations already done, but the goal is to simulate human play.
The puzzle's overall difficulty is the sum of the scores of the positions in your path through the game tree.
EDIT: Alternative position score: Instead of completely excluding deductions using harder rules, calculate overall difficulty of each rule (or compound application) and choose the minimum. (The logic here is that if rule A has score 50 and rule B has score 400, and rule A can be applied in one cell but rule B can be applied in ten, then the position score is 40 because the player is more likely to spot one of the ten harder plays than the single easier one. But this would require you to compute all possibilities.)
EDIT: Alternative suggested by Briguy37: Include all deductions in the position score. Score each position as 1 / (1/d1 + 1/d2 + ...) where d1, d2, etc. are the individual deductions. (This basically computes "resistance to making any deduction" at a position given individual "deduction resistances" d1, d2, etc. But this would require you to compute all possibilities.)
Hopefully this scoring strategy will produce a metric for puzzles that increases as your subjective appraisal of difficulty increases. If it does not, then adjusting the scores of your rules (or your choice of heuristic from the above options) may achieve the desired correlation. Once you have achieved a consistent correlation between score and subjective experience, you should be able to judge what the numeric thresholds of "easy", "hard", etc. should be. And then you're done!
Donald Knuth studied the problem and came up with the Dancing Links algorithm for solving sudoku, and then rating the difficulty of them.
Google around, there are several implementations of the Dancing Links engine.
Perhaps you could grade the general "constrainedness" of a puzzle? Consider that a new puzzle (with only hints) might have a certain number of cells which can be determined simply by eliminating the values which it cannot contain. We could say these cells are "constrained" to a smaller number of possible values than the typical cell and the more highly constrained cells that exist the more progress one can make on the puzzle without guessing. (Here we consider the requirement for "guessing" to be what makes a puzzle hard.)
At some point, however, the player must start guessing and, again, the constrainedness of a cell is important because with fewer values to choose between for a given cell the easier it is to find the correct value (and increase the constrainedness of other cells).
Of course, I don't actually play Sudoku (I just enjoy writing games and solvers for it), so I have no idea if this is a valid metric, just thinking out loud =)
I have a simple solver that looks for only unique possibilities in rows, columns and squares. When it has solved the few cells solvable by this method, it then picks a remaining candidate tries it and sees if the simple solver then leads to either a solution or a cell empty of possibilities. In the first case the puzzle is solved, in the second, one possibility has shown to be infeasible and thus eliminated. In the third case, which leads to neither a final solution nor an infeasibility, no
deduction can be reached.
The primary result of cycling through this procedure is to eliminate possiblities until picking
a correct cell entry leads to a solution. So far this procedure has solved even the hardest
puzzles without fail. It solves without difficulty puzzles with multiple solutions. If the
trial candidates are picked a random, it will generate all possilbe solutions.
I then generate a difficulty for the puzzle based on the number of illegal candidates that must
be eliminated before the simple solver can find a solution.
I know that this is like guessing, but if simple logic can eliminated a possible candidate, then one
is closer to the final solution.
Mike
I've done this in the past.
The key is that you have to figure out which rules to use from a human logic perspective. The example you provide details a number of different human logic patterns as a list on the right-risde.
You actually need to solve the puzzle using these rules instead of computer rules (which can solve it in milliseconds using simple pattern replacement). Every time you change the board, you can start over from the 'easiest' pattern (say, single open boxes in a cell or row), and move down the chain until you find one the next logical 'rule' to use.
When scoring the sodoku, each methodology is assigned some point value, which you would add up for every field you needed to fill out. While 'single empty cell' might get a 0, 'XY Chain' might get 100. You tabulate all of the methods needed (and frequency) and you wind up with a final weighting. There are plenty of places that list expected values for those weightings, but they are all fairly empirical. You're trying to model human logic, so feel free to come up with your own weightings or enhance the system (if you really only use XY chains, the puzzle is probably easier than if it requires more advanced mechanisms).
You may also find that even though you have a unique sodoku, that it is unsolvable through human logic.
And also note that this is all far more CPU intensive than solving it in a standard, patterned way. Some years ago when I wrote my code, it was taking multiple (I forget exactly, but maybe even up to 15) seconds to solve some of the generated puzzles I'd created.
Assuming difficulty is directly proportional to the time it takes a user to solve the puzzle, here is an Artificially Intelligent solution that approaches the results of the ideal algorithm over time.
Randomly generate a fixed number of starting puzzle layouts, say 100.
Initially, offer a random difficulty section that let's a user play random puzzles from the available layouts.
Keep an average random solution time for each user. I would probably make a top 10/top X leaderboard for this to generate interest in playing random puzzles.
Keep an average solution time multiplier for each puzzle solution (if the user normally solves the puzzle in 5 minutes and solves it in 20 minutes, 4 should be figured in to the puzzles average solution time multiplier)
Once a puzzle has been played enough times to get a base difficulty for the puzzle, say 5 times, add that puzzle to your list of rated puzzles and add another randomly generated puzzle to your available puzzle layouts.
Note: You should keep the first puzzle in your random puzzles list so that you can get better and better statistics on it.
Once you have enough base-rated puzzles, say 50, allow users to access the "Rated Difficulty" portion of your application. The difficulty for each puzzle will be the average time multiplier for that puzzle.
Note: When users choose to play puzzles with rated difficulty, this should NOT affect the average random solution time or average solution time multiplier, unless you want to get into calculating weighted averages (otherwise if a user plays a lot of harder puzzles, their average time and time multipliers will be skewed).
Using the method above, a solution would be rated from 0 (already solved/no time to solve) to 1 (users will probably solve this puzzle in their average time) to 2 (users will probably take twice as long to solve this puzzle than their average time) to infinity (users will take forever to find a solution to this puzzle).
I am going to be asking a lot of questions in the upcoming months. For my ninth grade science fair project I would like to create a traffic simulator in order to test whether or not interconnected communicating traffic lights can increase traffic flow. I have a couple of generic questions that I need help with...
How would I represent roads?
How would I make car follow a road?
How would I make a car switch lanes or roads?
I am not looking for specific code, just good pointers and resources to help me get started. Any help is appreciated, C.Ruhl.
PS I am only in high school so no advanced math notations please :)
One possible approach which is taken quite often is to use a discrete model for roads and cars' positions.
Each position on the road can either be occupied by a car (blue dot) or be empty. Cars move at discrete time steps by exactly one position (if the target position is empty) along the given arrows. Thus a car can even switch lanes if it would otherwise have to slow down or stop.
You can further improve it by using separate timesteps per car (simulating faster/slower cars) or in many other ways.
After you've defined your roads (i.e. the positions and their follow-up positions) by an appropriate data structure this model is relatively easy to simulate but already shows interesting effects.
Forget about the UI.
Represent each object in its base form --only put object properties in it. Example, a car will have a size and ability to move. But it won't have the logic to make it move. Similarly a traffic light will have states such as green, amber and red. But it won't have the logic to switch between these states. Similar classes for roads, lanes etc.
Build a different class for the driver. This class will contain all methods such as lane shifting, stopping, turning, moving forward etc. More technically, this will be your "actor" and will act on the veichle. A similar actor would be for traffic light control which will act on a network of traffic lights. Make it an interface and have two implementations to it --one that takes advantage of interconnectedness and other that operates on static times.
Optional add a UI on top of this object model. Don't go fancy, have simple dots to begin with. Once you get all simple stuff working, adding more fancy features should be easy and impact free (relatively).
This will be a very challenging project.
But if your objective is a proof of concept, I have a simpler suggestion. You can go user generated here and get all the complexity of simulation out and all the accuracy in. Start with 15-20 remote controlled cars, a cardboard model of a fictional town, some bulbs to simulate traffic lights and some volunteers who know how to drive. Have a preprogrammed sequence of on and offs written on paper and assign some of the volunteers to control those lights. Have another set of volunteers control the cars. If you have hands on experience in basic electronics you can build a timer controlled circuit to control the lights.
All the very best!
You could try the SIM.JS discrete event simulation library in Javascript. They have a very simple example for traffic at road intersection simulation here.
Ooh, Conner, you've found an interesting question indeed -- and one which is the subject of research even today. Here's a suggestion: before you fret about how to do it in JavaScript, spend some time thinking just how to do it at all.
Here's a suggestion: think about the objects invovled first. You have Cars, and they travel along Roads. Start with a square grid of roads, so your cars go from intersection to intersection.
Pick a fixed speed for the cars, so it takes a constant time to travel from intersection to intersection.
Each intersection has a traffic light, which can be red or green. If it's red, of course cars can't go through; they have to wait.
Now, your basic program will look like
time = 0
while time < end-time:
for each car:
update the car's location
add time consumed to time
when you update the cars location, what happens? (Hint: the car moves; can it go through an intersection or not?)
That will give you a start.
For my Bachelor's degree exam I developed a traffic control web-app that tracked the vehicles in my town in real-time and I used google maps api.
I suggest you use a map service such as maps.google.com , yahoo.maps.com...
They have an api for everything... you can use markers to represent anything on the map (cars,street lights,even pedestrians :)) ) and you can use their api to calculate distances and paths.
It may seem a bit more complex then the average div-implementation, but, trust me, it's a big plus to use a service with a well-organised api.
+it would have a more professional look ;).