Related
I am currently working on a project that involves recording the magnitude of a user's mobile device, and then analyzing this after recording to find the frequency of the user's movement. It's supposed to help the user with a specific kind of exercise that is done periodically. I need to find a way using javascript to find the frequency in the array of magnitudes recorded.
I already have some data of the movement and was able to process it a bit in python, the magnitude basically looks like following:
What I want is a method that can detect that frequency of the array, so I know to how many points should I cut the array and process it and verify it. It's an app to count repetitions of an exercise, and I am planning after getting the frequency, to divide the array based on this frequency, and then compare each period with a stored "ideal" repetition of the exercise and do dynamic time warping on it to compare how well it was performed.
I need to move my local project to a webserver and it is time to start saving things locally (users progress and history).
The main idea is that the webapp every 50ms or so will calculate 8 values that are related to the user which is using the webapp.
My questions are:
Should i use MySQL to store the data? At the moment im using a plain text file with a predefined format like:
Option1,Option2,Option3
Iteration 1
value1,value2,value3,value4,value5
Iteration 2
value1,value2,value3,value4,value5
Iteration 3
value1,value2,value3,value4,value5
...
If so, should i use 5 (or more in the future) columns (one for each value) and their ID as Iteration? Keep in mind i will have 5000+ Iterations per session (roughly 4mins)
Each users can have 10-20 sessions a day.
Will the DB become too big to be efficient?
Due to the sample speed a call to the DB every 50 ms seems a problem to me (especially since i have to animate the webpage heavily). I was wondering if it would be better to implement a Save button which populate all the DB with all the 5000+ values in one go. If so what could it be the best way?
Would it be better to save the *.txt directly in a folder in the webserver? Something like DB/usernameXYZ/dateZXY/filename_XZA.txt . To me yes, way less effort. If so which is the function that allows me to do so (possible JS/HTML).
The rules are simple, and are discussed in many Q&A here.
With rare exceptions...
Do not have multiple tables with the same schema. (Eg, one table per User)
Do not splay an array across columns. Use another table.
Do not put an array into a single column as a commalist. Exception: If you never use SQL to look at the individual items in the list, then it is ok for it to be an opaque text field.
Be wary of EAV schema.
Do batch INSERTs or use LOAD DATA. (10x speedup over one-row-per-INSERT)
Properly indexed, a billion-row table performs just fine. (Problem: It may not be possible to provide an adequate index.)
Images (a la your .txt files) could be stored in the filesystem or in a TEXT column in the database -- there is no universal answer of which to do. (That is, need more details to answer your question.)
"calculate 8 values that are related to the user" -- to vague. Some possibilities:
Dynamically computing a 'rank' is costly and time-consuming.
Summary data is best pre-computed
Huge numbers (eg, search hits) are best approximated
Calculating age from birth date - trivial
Data sitting in the table for that user is, of course, trivial to get
Counting number of 'friends' - it depends
etc.
Hi I have a website with hundreds of tv shows and tens of thousands of episodes. I want the user (who didn't log in) to be able to save his favorites shows and to mark the episodes he's watched and save these data using cookies.
So I've already implemented it both in PHP and JS. I just wanted to know what's the best choice when it comes to arrays of cookies.
Now I save the data in a cookie called 'favorites', which is a JSON encoded array which simply contains the IDs of the tv shows.
In the cookie 'watched' there are the IDs of the episodes that the user has marked as watched.
What's the best option for performance and memory? You must keep in mind this is also intended for mobiles so, if the user marks thousand of episodes as watched, I don't want this to slow down everything.
The code below is not real code, it's just to make you understand the differences between the options.
Option 1
one cookie called 'watched' which is a JSON encoded array containing the IDs. I would need to check if the array contains that value
$_COOKIE['watched'] = array(1,2,3,4,...)
With this option, in the tv-show page I would always have to pick the whole array and check if every single episode ID is a value in the array.
Option 2
one cookie for each ID, called for example "watched_[ID]". I would just check if it exists or not.
exists( $_COOKIE['watched_1'] ) ? watched : !watched
exists( $_COOKIE['watched_2'] ) ? watched : !watched
Option 3
one cookie called 'watched' which is an associative JSON encoded array containing an array for each tv show, called for example 'watched[show_ID]', and each one of these contain only the IDs of the episodes marked as watched belonging to that show. (so in the tv-show page I first check for 'watched[show_ID]' then I already have all the watched episodes, instead of comparing every single episode ID with the array 'watched' from Option 1
$_COOKIE['watched'] = array(
1 => array(1,2,3),
2 => array(4,5,6)
)
The Approach you select will purely depend on the type of users you have, if data size generally is large in your case then you should go with option 3 as associative arrays are a good approach when it comes to large data size as it will be somewhat easy to loop through them and get the results.
If your users data set is fairly small then option 2 is the best approach as we have different cookies for each shows and you could immediately check the corresponding values.
Further I would like you to go through the html5 api to use indexed db as a solution to your problem as it provides client side DB and further it is a persistent database.
Hope this helps
Something not addressed in the accepted answer is browser cookie limitations (on the client side of things). Option 2 will clearly not work due to the limits on the number of cookies allowed:
Browser Num Cookies Size Per Cookie Size Per Domain
------- ----------- --------------- ---------------
Chrome 8-25 180 4096 bytes
FireFox 3.6.13-19 150 4097 characters
IE 8/9/10 50 5117 characters 10234 characters
Safari 600 4093 bytes 4093 bytes
Android 2.1/2.3.4 50 4096 bytes
Safari Mobile 5.1 600 4093 bytes 4093 bytes
Some of those numbers may be outdated...[source]
However, you will still have to be careful when utilizing options 1 or 3 due to the size limitation per cookie/domain. This fiddle shows how big a strigified associative array is when it contains 1000 items with a key length of 7 characters each (it's about 12000 bytes).
If you're intent on storing thousands of items in cookies, you will probably have to write a cookie wrapper library that chunks your data across multiple cookies to avoid the per-cookie size limitations.
I'm writing a standalone javascript application with Spine, Node.js, etc.(Here is an earlier incarnation of it if you are interested). Basically, the application is an interactive 'number property' explorer. The idea being that you can select any number, and see what properties it possesses. Is it a prime, or triangular, etc? Where are other numbers that share the same properties? That kind of thing.
At the moment I can pretty easily show like numbers 1-10k, but I would like to show properties for numbers 1-million, or even better 1-billion.
I want my client to download a set of static data files, and then use them to present the information to the user. I don't want to write a server backend.
Currently I'm using JSON for the data files. For some data, I know a simple algorithm to derive the information I'm looking for on the client side, and I use that (ie, is it even?). For the harder numbers, I pre compute them, and then store the values in JSON parseable data files. I've kinda gone a little overboard with the whole thing - I implemented a pure javascript bloom filter and when that didn't scale to 1 million for primes, I tried using CONCISE bitmaps underneath (which didn't help). Eventually I realized that it doesn't matter too much how 'compressed' I get my data, if I'm representing it as JSON.
So the question is - I want to display 30 properties for each number, and I want to show a million numbers...thats like 30 million data points. I want the javascript app to download this data and present it to the user, but I don't want the user to have to download megabytes of information to use the app...
What options do I have for efficiently sending these large sets of data to my javascript only solution?
Can I convert to binary and then read binary on the client side? Examples, please!
How about just computing these data points on the client?
You'll save yourself a lot of headache. You can pre-compute the index chart and leave the rest of the data-points to be processed only when the user selects a particular number.
For the properties exhibited per number. Pure JavaScript on modern desktops is blindingly fast (if you stay away from DOM), I think you'll find processing speed differences are negligible between the algorithmic vs pre-computed JSON solution and you'll be saving yourself a lot of pain and unnecessary bandwith usage.
As for the initial index chart, this displays only the number of properties per number and can be transferred as an array:
'[18,12,9,11,9,7,8,2,6,1,4, ...]'
or in JSON:
{"i": [18,12,9,11,9,7,8,2,6,1,4, ...]}
Note that this works the same for a logarithmic scale since either way you can only attach a value to 1 point in the screen at any one time. You just have to cater the contents of the array accordingly (by returning logarithmic values sequentially on a 1-2K sized array).
You can even use a DEFLATE algorithm to compress it further, but since you can only display a limited amount of numbers on screen (<1-2K pixels on desktop), I would recommend you create your solution around this fact, for example by checking if you can calculate 2K *30 = 60K properties on the go with minimal impact, which will probably be faster than asking the server at this point to give you some JSON.
UPDATE 10-Jan-2012
I just saw your comment about users being able to click on a particular property and get a list of numbers that display that property.
I think the intial transfer of number of properties above can be jazzed up to include all properties in the initial payload, bearing in mind that you only want to transfer the values for numbers displayed in the initial logarithmic scale you wish to display (that means that you can skip numbers if they are not going to be represented on screen when a user first loads the page or clicks on a property). Anything beyond the initial payload can be calculated on the client.
{
"n": [18,12,9,11,9,7,8,2,6,1,4, ...] // number of properties x 1-2K
"p": [1,2,3,5,7,13,...] // prime numbers x 1-2K
"f": [1,2,6, ...] // factorials x 1-2K
}
My guess is that a JSON object like this will be around 30-60K, but you can further reduce this by removing properties whose algorithms are not recursive and letting the client calculate those locally.
If you want an alternative way to compress those arrays when you get to large numbers, you can format your array as a VECTOR instead of a list of numbers, storing differences between one number and the next, this will keep space down when you are dealing with large numbers (>1000). An example of the JSON above using vectors would be as follows:
{
"n": [18,-6,-3,2,-2,-2,1,-6,4,-5,-1, ...] // vectorised no of properties x 1-2K
"p": [1,1,2,2,2,6,...] // vectorised prime numbers x 1-2K
"f": [1,1,4, ...] // vectorised factorials x 1-2K
}
I would say the easiest way would be to break the dataset out into multiple data files. The "client" can then download the files as-needed based on what number(s) the user is looking for.
One advantage of this is that you can tune the size of the data files as you see fit, from one number per file up to all of the numbers in one file. The client only has to know how to pick the file it's numbers are in. This does require there to be some server, but all it needs to do is serve out the static data files.
To reduce the data load, you can also cache the data files using local storage within the browser.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
[I know there have been similar questions about preventing cheating on high score lists, but no answer didn't really help me for JavaScript based games, so please try to think about my question, before telling me about similar posts. I ask about best practices because the JavaScript is always visible for the user and therefore it is not possible to prevent cheating completly, I just want to make it harder.]
I'm developing a JavaScript based game that works in the browser. I want to make a high score list that contains the user name and the score of all users. To achieve that the browser sends the username and the score to my server (via AJAX).
Submitting fake scores to this list would be fairly easy: One could take a look at the AJAX requests and then make an own AJAX request with a faked score. Using something like a token that has to be send with the other data is pointless, as it will be easy to discover.
My only approach, that would prevent cheating, would be to send a description of every user action to the server and calculate the score there. But this is not really practicable as it would be too much for the server.
I accepted an answer, but in case anyone has other ideas about how to make cheating harder, please create another answer!
I like to play cheat the cheater - something like using a token to authenticate the score that changes every time the update is called... but I accept the cheat score that gets posted using a duplicate token. Then I display that cheat score to only the cheater, so it appears that it worked, but now the cheater is seeing his results in a sandbox.
You pretty much answered your own question. If you want to really make it harder for users to cheat, send game log to the server, where you'll calculate the score.
You don't have to send all the events, just ones that affect result score.
There are some techniques, though, that may help you:
include signature in your request. Something like MD5(secret_key + params). Although, "secret key" will have to be in JS source, it will effectively protect you from simple request interception (see Tamper Data and Charles)
if it's a multiplayer game, accept scores calculated by clients and compare them. Cheaters will be pretty visible (assuming that the majority of users are honest).
you can set a score cap, an "unreachable" result. Everyone who posts score higher than this is a cheater. For example, speed typing game: no one can type correct text at 1500 chars/minute, even 700 is pretty damn hard (though achievable).
On score submit:
Request some token from the server, this should be time based and only valid for about 2 seconds
Only accept submits that include a valid hash of this token, some salt and the score.
This prevents manual tampering with the request as it would timeout the score. If you want to account for high-latency give it a little more time until the timeout.
The hashing function:
Scramble the hashing function inside packed code (http://dean.edwards.name/packer/ really produces nasty to read code) if you use jQuery or some other library just drop the hashing functionality inside the library file and it gets pretty bad to find, escpecially if you use a function name like "h" :)
Handling the score-variable itself:
Well everybody with a debugging console can change the variable on runtime when doing this but if you encapsulate your whole Javascript inside a function and call it nothing is in the global namespace and it's much harder to get to the variables:
(function() {
//your js code here
})();
I have had lots of thoughts about it and, eventually, decided to only have local individual highscores, so cheating is not really beneficial for player and not harmful to others. Yet my game is just a simple minesweeper, and there were people who complained about the lack of competitive table.
Option 2, is approach taken by WebSudoku - to show your place "among the people of internet". So you will not see any other results, and people wont see your results - but you can compare yourself to crowd.
p.s: And seriously - any kid with Firebug/WebInspector can easily hack your JS game and, eventually, to be able to reach very high score.
If you are relying on the client to send the final score to the server, then there is no way (afaik) to prevent a genius from cheating. But I think you might be able to prevent stupid people (and honest people) from cheating, so that only geniuses and their friends will dominate your leaderboards.
There are two ways I can think of
1.) "security through obscurity."
Come up with an algorithm that transforms simple scores into something else (and to transform them back). Then obfuscate it. Complicate it. Write a function that multiplies it by q and divides it by ralph. Apply a bunch of functions to it, and among the 5-15 functions that do random stuff to it, include one that multiplies the number by 19 ( a prime number ). On your server, check to make sure every incoming number (or letter) is divisible by 19, and decode
You have to write a bunch of complex code that transforms simple scores into something crazy-looking. You have to write a series of functions in the least-efficient, most spaghetti-code fashion possible. Use
One thing you cold do is to have a set of disallowed values. I.e., perhaps all points awarded are even. If anyone tries to submit an odd number, they are obviously cheating (and very stupid).
2.) time
You should be able to know when the user started the game. You should have a session started and record when they requested the page. Then you should also be able to tell when they submitted their score. And you should also know what the time series is for max points. I.e. can you get 5 points per minute, 100 per minute, minute^3, etc... If user submits more points than are possible during that time, they are cheating.
You could also strike a balance between server and client processing and make the client should send progress update every x minutes by ajax. And if it fails to report, you assume it's been compromised (much like in Bond movies, when he's infiltrating the enemy's lair and he snaps some guard's neck. When the guard doesn't respond to his next 10-minutely check-in, the alarms will go off).
If you've ever played Zynga Poker, you've probably seen what happens when someone at the table has a slow internet connection.
Depending on the nature of the game, you could use other players to verify the results. In simple games this works great, on others you have to be clever and develop many aspects around this feature. E.g. sometimes is possible to replay and verify results based on logged actions. This trick works specially well for Human versus AI, as long as the model is deterministic.
Another option is redefining the score concept to be more user-centric, this is pretty easy to implement, but tends to be hard to devise, and only applies to a few categories of games.
Purely speculative approaches are also possible, it's sometimes pretty easy to know when some parameters don't fit. It would not avoid cheating, but would moderate it a lot.
The most complicated part is getting a small enough replay log, but since most data isn't random (except for player actions, which, actually aren't that random because depend on the game) it's essentially a matter of getting the design right.
Also, if gameplay is extended enough, for action games and the like you can get a lot of compression from doing some approximation, merging (e.g. motion vectors), and clipping uninteresting stuff.
Ideally you would send your entire event log to the server for checking. Perhaps you can implement a heuristic so you can easily determine if the score is within a set of bounds. For instance, if the total game time is 5 seconds you might expect a much lower score than with a much longer game time.
Alternatively, you could choose to manually check the event log for really high scores (the overall top-X, which should be fairly stable).
You will need a seeded random number generator if you're doing anything with randomness (like random events). Which might be tricky if you hadn't already thought of it.
You can find many more resources but it really just boils down to server-side checking. JavaScript is not unique in this, but likely easiest to exploit because you not only see the client-server communication but also the client-side source code!
HTML5 Multiplayer Game Security Solutions
http://lanyrd.com/2011/jsconf/sfggb/
Games like Starcraft only record the mouse clicks and key presses. The actual commands are then simulated. I expect 'Worms Armageddon' to do something similar but their random events (like the bounciness of bananas) aren't seeded properly so in the instant replay you might get a different result.
You could imagine something similar for MMORPGs. The server calculates your position based on the keypresses, the client merely tries to give a good early interpretation but you may warp around when you're lagging because the server will place you elsewhere on the map because it didn't get the keypress events timely.
If you attack something, the server will check if you're close enough and how much damage you can expect to deal with current stats and equipment.
Record key points in game, then score is submitted with these key points. When people look high scores, they can also see overview of played game, if it looks like it is impossible to play like that without cheating, then people can report these suspicious scores to admins.
I used a system using a time based request having 3 parameters
req number, curr time, score
The req number is returned from server in the response to the update score request , each time this is a new random value.
The curr time is calculated not from computer clock but from start of game and is synced with server using an ajax request.
Update score request is sent after short intervals (around 30 sec max).
Following checks are applied on the server
Time is within 10 seconds range from the server clock.
there has been not more than 40 seconds since the req number was sent.
the score change sent after 30 seconds is possible (within 2 x humanly possible range)
Score is updated only if the above checks are passed or the user gets a disconnection message :(
This is simpler than most methods and works out to eliminate all casual hackers (well, unless they read this and want to go to the trouble of updating score quickly or making a script of their own).
If not cheating is more important than the game itself, try to construct and present your game in a way that it looks like finding the solution to a math problem. So the server will give an instance of the problem to the client (example A: a chess board about to be won in 3 moves, example B: a geometry dash randomly generated level) and the user will have to solve it and post back a solution (example A: the winning moves, example b: the exact timestamps and intensity of jumps to avoid obstacles)
With this approach, it is key that the server doesn't send the same level twice, or else the cheater can plan and "design" his solution in advance. Also, the game information must be randomly generated in the server and not sent via seed, or else the cheater can fake the seed and design his solution with time.
The given time for valid submissions must be also tracked in the server so that they will only have "playing" time and no "designing" time. If the cheater is good enough to design a solution as fast as honest players can win the game, then they are talented enough to win the game honestly and deserve their points.
Back in the server, you will need to check that the submitted solution is valid for that instance.
Of course this approach requires lots of extra work: More instances of games (ideally infinite and non repeating), server side generation, server side validation of submissions, time caps, etc.
Note: I know these approach was already suggested in multiple solutions some years ago, I wanted to add my humble contribution.