I have a class Message which can be serialized when the data goes through the network, I currently use JSON, mostly because I use JSON for everything. (webservice, sockets).
I want to improve the serialization to make it as good as possible, I believe improvments are possible here.
The aim is to make the transport string lighter, especially when used by sockets (video game), because it will be used for everything, every response client/server or server/client and even inside the server or client methods, it's the usual way to provide data.
The Message is a complex object that can also contain other object instances, like a MessageLang, which will be responsable to translate a sentence on the client based on a code.
So far it works fine, here are the results:
Socket server emit with simple string:
verbose: websocket writing 5:::{"name":"user.newAuthenticated","args":["Respond to emitter"]}
Socket server emit with simple message instance:
verbose: websocket writing 5:::{"name":"user.newAuthenticated","args":["{\"m\":\"Respond to all clients\",\"d\":{},\"s\":1,\"t\":\"m\"}"]}
Socket server emit with complex message instance:
verbose: websocket writing 5:::{"name":"user.newAuthenticated","args":["{\"m\":{\"m\":\"__12\",\"a\":{\"field\":\"name\",\"min\":3,\"max\":20}},\"d\":{\"key\":\"fakeKey\"},\"s\":1,\"t\":\"m\"}"]}
The complexe message would render the following sentence:
The min length of name is 3. Max length is 20. and would contain the key: "fakeKey" in data. Just to explain how it works.
As you see, the message get bigger and bigger and it is normal, but I would like to know what I can do to make a better serialization here:
Delete the message itself when there aren't (empty)
Delete the data when it's empty as well
Delete the status when it's false (because it's the default value)
I see a lot of \ in the socket log because it is JSON, I believe that's a problem, because each time I'll add something I'll get extra characters that I do not want. Maybe the JSON isn't a good choice and I should serialize differently, first in JSON like the examples at the top, but then in something else, maybe kind of binary, if it takes less space.
What do you think?
And if it would be a good idea to encrypt somehow the message in another format, would the cost of the encryption be worth it? Because encrypt it would take a bit of time as well, so I'm just wondering if it wouldn't just move the issue, like it would take less time to send the message through socket because it would be lighter, but we would use more time to encrypt it. Just wondering.
My guess is that your message object has two fields (name and args).
The first stop to reduce the length of the message is to get rid of the (pretty useless) outer object and replace it with an array. So an empty message
{"name":"empty","args":[]}
would become
["empty",[]]
or even
["empty"]
The next thing is that you have a bug in the serialization of the arguments. Instead of sending JSON, you wrap the JSON data in a string. Example: In the authenticated case, you send
{"name":"user.newAuthenticated","args":["{\"m\":\"Respond to all clients\",\"d\":{},\"s\":1,\"t\":\"m\"}"]}
but you should send
{"name":"user.newAuthenticated","args":[{"m":"Respond to all clients","d":{},"s":1,"t":"m"}]}
instead. Now the question is whether args is a list of a single object. If it's always a single object, then you could get rid of the [] as well. With my suggested change from above, that would give you:
["user.newAuthenticated",{"m":"Respond to all clients","d":{},"s":1,"t":"m"}]
which is pretty good IMO. If you can make the (de-)serializer handle default values properly, you can reduce this to:
["user.newAuthenticated",{"m":"Respond to all clients","s":1,"t":"m"}]
(i.e. we can omit the empty d property).
For a MMO, I think a minimum of data must be sent to the client. If a socket is called 2xx/3xx by sec, you must reduce the size of the data sent through the socket as most as possible.
On another hand, it also consummes resource to encrypt the object on the server side to send a minified version of the object... Wouldn't it be better not to reduce it and to send an object not reduced so we don't spent resource to encrypt it?
Related
A common question is how to pass an object from Node.js/Express.js to the browser. It's possible to do that using JSON stringify, but if the object contains user-provided data, that can open the door to script-injection and possibly other attacks.
Is there a downside to the approach mentioned in this link using Base64?
https://stackoverflow.com/a/37920555/645715
Related links:
Passing an object to client in node/express + ejs?
How to pass a javascript object that contains strings with quotes from node.js to the browser?
Pass a NodeJS express object to AngularJS 1.6
Passing an object to client in node/express + ejs?
Using Base64 encoding does solve the immediate problem of passing back an injection attack, but it doesn't necessarily solve the issue of having a possible injection attack floating around out there. For example, this fiddle shows that it does prevent the immediate issue : https://jsfiddle.net/9prhkx74/
var test2 = JSON.parse(window.atob('PC9zY3JpcHQ+PHNjcmlwdD5hbGVydCgndGVzdDInKTwvc2NyaXB0PjxzY3JpcHQ+'));
This won't show an alert box, it'll just throw an error about invalid JSON. But if you change it to the literal string, it'll show the alert box (injection vulnerable)
var test2 = JSON.parse("</script><script>alert('test2')</script><script>")
Now if you are immediately parsing it to a JSON object, it'll blow up, and everything will be "safe". But if you assign it to a value because you are going to pass it around some more etc, you still have a potential issue out there.
Instead of putting a bandaid on the injection itself, I'd suggest fixing it in the first place and properly escaping data before passing it back to the client or processing it on the server side.
There are plenty of libraries that can help do this
https://www.npmjs.com/package/sanitize
https://www.npmjs.com/package/express-sanitizer
Here's a pretty good article that kind of highlights why it is important to sanitize and not just just patch over potentially malicious data : https://lockmedown.com/5-steps-handling-untrusted-node-js-data/
The purpose, game programming, as you may have guessed, why else right?
How is it actually possible to read in values from a text file, so that those values can be then on wards used in the game? I have searched for hours on this topic.
example: each text file line token, will be read and passed as the different arguments into the constructor of each object during its instantiation via for loop. A common practice. Its Too expensive to store that much data at any given time in an array I would suspect.
In java this is dead simple using the Scanner object.
Any suggestions are appreciated thanks. I guess all I am asking is, is it even possible?
As Roland Starke said, the array will probably take up less memory than the objects you construct from it... So it is perfectly fine to have all the information in a JSON file for instance, which you load from your server.
If you want to avoid transfering all the data every time, you would need to use the right caching headers so that the data can be cached by the browser.
This might seem like an odd questions; I know its possible to slice an array, but I was thinking, if I'm calling an array externally via a $http GET (using Angular) like so:
$http.get('/url')
is it possible for me to declare how much of the array I want to retrieve first before making the request, instead of retrieving the whole thing (and therefore saving on performance). I know its possible to do something similar using PHP but wasn't sure to the extent I had in Javascript? Or can I only slice it once the array is declared fully?
The only way to retrieve part of the array is for the server that you are requesting the data from to support that capability. You cannot do that entirely from the client unless the server has that capability.
The usual way that this would be done would be to add query parameters to the URL that specify how much of the data you want and for the server to look at those query parameters and to send only those pieces of the data.
For example:
$http.get('/url?start=0&end=20');
Arrays can only be sliced once they exist and have some data in them so I'm not sure what that part of the question was trying to ask about. Either the client or the server could slice the result array once they had built the initial array.
Let's say I'm making an HTML5 game using JavaScript and the <canvas> The varaibles are stored in the DOM such as level, exp, current_map, and the like.
Obviously, they can be edited client-side using Firebug. What would I have to do to maximize security, so it would be really hard to edit (and cheat)?
Don't store the variables in the DOM if you wish a reasonable level of security. JavaScript, even if obfuscated, can easily be reverse engineered. That defeats any local encryption mechanisms.
Store key variables server-side and use https to maximize security. Even so, the client code (JavaScript) is quite vulnerable to hacking.
You can use Object.freeze or a polyfill or a framework which does the hiding for you.
Check out http://netjs.codeplex.com/
You could also optionally implement some type of signing system but nothing is really impenetrable. For instance objects locked with Object.freeze or Object.watch can still be manually modified in memory.
What are you really trying to accomplish in the end?
What you could do is send a representation of the matrix of the game or the game itself or a special hash or a combination of both and tally the score at the server... causing the user to not only have to modify the score but to correctly modify the state of the game.
Server-side game logic
You need to keep the sensitive data on the server and a local copy on the browser for display purposes only. Then for every action that changes these values the server should be the one responsible for verifying them. For example if the player needs to solve a puzzle you should never verify the solution client side, but take for example the hash value of the ordered pieces represented as a string and send it to the server to verify that the hash value is correct. Then increase the xp/level of the player and send the information back to the client.
Anything that is living in the client can be modified. That is because in MMORPG the character's data is living on the server, so players can't hack their characters using any memory tools, hex editor, etc (they actually "can", but because the server keep the correct version of the character's data is useless).
A good example was Diablo 2: you have actually two different characters: one for single player (and Network playing with other players where one was the server), and one for Battle.net. In the first case, people could "hack" the character's level and points just editing the memory on the fly or the character file with an hex editor. But that wasn't possible with the character you was using on Battle.net.
Another simple example could be a quiz where you have a limited time to answer. If you handle everything on client side, players could hack it and modify the elapsed time and always get the best score: so you need to store the timestamp on the server as well, and use that value as comparison when you get the answer.
To sum up, it doesn't matter if it's JavaScript, C++ or Assembly: the rule is always "Don't rely on client". If you need security for you game data, you have to use something where the clients have no access: the server.
If my JSON data-file comes from a database result set and that result set is empty, how do I tell OFC2 to display an error message, instead of crashing because of a malformed JSON string?
Add tags for javascript and actionscript-3 to this question and you should get a load more views and useful responses than you currently are, with more precise details than I am giving. Post the actual JSON string that is causing you the problem and that you would like to be guarded against. That way people can suggest a regexp to catch it, treating it as a string rather than as JSON data at some point before JSON.decode() happens.
In more detail:
You can catch it in two places. One route is to switch over to using the javascript interface to OFC2 and use client side javascript to detect the bad string. This allows you to modify the JSON string client side. See http://teethgrinder.co.uk/open-flash-chart-2/tutorial-5.php for that approach. One downside is that the clients must have javascript enabled for this to work.
Alternatively, since OFC2 is LGPL, you or an actionscript developer can dive into the OFC2 source code and do the same thing there. I am not an actionscript developer so you are better off ensuring you get a reply from one.
The main thing is to add those two tags to this question. 22 Views is way too low for a question with a bounty of 500. Hope this helps.
Several solution avenues are possible, depending on your level of access to the server and your knowledge of JavaScript and/or any server-side platforms.
With access to database
Depending on the kind of data you are displaying, it might be possible to add dummy records for those queries that would otherwise have returned an empty set. If you have access to the query definition, you may check for the empty set in the DB-query. For example, if you're on MS SQL Server you could check the condition with some T-SQL statements.
With access to server
If you have access to the server side script generating the dataset, add a condition that returns some default value that OFC2 will handle correctly.
With access to another server or serverlocation
If you don't have access to the specific script, you may consider creating a new script at another location that queries the original script and replaces empty results with the default value.
Client-side only
You can add the JavaScript open_flash_chart_data function (see tutorial) to replace empty datasets. OFC2 can use that function as data source.
Hope this helps.