We Keep Coding

Safely pass object to client in Node.js/express

A common question is how to pass an object from Node.js/Express.js to the browser. It's possible to do that using JSON stringify, but if the object contains user-provided data, that can open the door to script-injection and possibly other attacks.
Is there a downside to the approach mentioned in this link using Base64?
https://stackoverflow.com/a/37920555/645715
Related links:
Passing an object to client in node/express + ejs?
How to pass a javascript object that contains strings with quotes from node.js to the browser?
Pass a NodeJS express object to AngularJS 1.6
Passing an object to client in node/express + ejs?

Using Base64 encoding does solve the immediate problem of passing back an injection attack, but it doesn't necessarily solve the issue of having a possible injection attack floating around out there. For example, this fiddle shows that it does prevent the immediate issue : https://jsfiddle.net/9prhkx74/
var test2 = JSON.parse(window.atob('PC9zY3JpcHQ+PHNjcmlwdD5hbGVydCgndGVzdDInKTwvc2NyaXB0PjxzY3JpcHQ+'));
This won't show an alert box, it'll just throw an error about invalid JSON. But if you change it to the literal string, it'll show the alert box (injection vulnerable)
var test2 = JSON.parse("</script><script>alert('test2')</script><script>")
Now if you are immediately parsing it to a JSON object, it'll blow up, and everything will be "safe". But if you assign it to a value because you are going to pass it around some more etc, you still have a potential issue out there.
Instead of putting a bandaid on the injection itself, I'd suggest fixing it in the first place and properly escaping data before passing it back to the client or processing it on the server side.
There are plenty of libraries that can help do this
https://www.npmjs.com/package/sanitize
https://www.npmjs.com/package/express-sanitizer
Here's a pretty good article that kind of highlights why it is important to sanitize and not just just patch over potentially malicious data : https://lockmedown.com/5-steps-handling-untrusted-node-js-data/

javascript - Why shouldn't the server respond with a JSON Array?

According to this Angular 2 guide:
Don't expect the decoded JSON to be the heroes array directly. This server always wraps JSON results in an object with a data property. You have to unwrap it to get the heroes. This is conventional web API behavior, driven by security concerns.
('Heroes' is an array of objects).
The link in the above paragraph says :
Always return JSON with an Object on the outside
Always have the outside primitive be an object for JSON strings:
Exploitable:
[{"object": "inside an array"}]
Not exploitable:
{"object": "not inside an array"}
Also not exploitable:
{"result": [{"object": "inside an array"}]}
My Question is: Why shouldn't the server return something like a JSON array?
Say :
[
"apples"
,"oranges"
, "peaches"
]
How is this a security concern ?

To avoid JSON Hijacking:
The fact that this is a JSON array is important. It turns out that a script that contains a JSON array is a valid JavaScript script and can thus be executed. A script that just contains a JSON object is not a valid JavaScript file.
For example, if you had a JavaScript file that contained the following JSON: {“Id”:1, “Balance”:3.14} And you had a script tag that referenced that file:
<script src="http://example.com/SomeJson"></script>
You would get a JavaScript error in your HTML page. However, through an unfortunate coincidence, if you have a script tag that references a file only containing a JSON array, that would be considered valid JavaScript and the array gets executed.
So allowing JSON to be returned as anything but an object would make it possible to return a JSON array that contained code that could be run on the client level (in a context where the client isn't expecting it to be runnable, could be malicious, etc). Only returning JSON objects prevents this from happening.

This was rather bad advice that has since been removed from the angular tutorial.
The linked OWASP Cheet Sheet lists three ways to defend against JSON Hijacking. The one the tutorial picked is the hardest to implement correctly, because one must educate every single developer, and audit every single REST resource, rather than writing a single HttpInterceptor to extend CSRF-defenses to GET requests.
JSON hijacking can only occur due to browser bugs, which tend to be fixed quickly (this does not imply such attacks are impossible, but the easy exploits no longer work in modern browsers)

Serialize object through network with Javascript - How to improve it?

I have a class Message which can be serialized when the data goes through the network, I currently use JSON, mostly because I use JSON for everything. (webservice, sockets).
I want to improve the serialization to make it as good as possible, I believe improvments are possible here.
The aim is to make the transport string lighter, especially when used by sockets (video game), because it will be used for everything, every response client/server or server/client and even inside the server or client methods, it's the usual way to provide data.
The Message is a complex object that can also contain other object instances, like a MessageLang, which will be responsable to translate a sentence on the client based on a code.
So far it works fine, here are the results:
Socket server emit with simple string:
verbose: websocket writing 5:::{"name":"user.newAuthenticated","args":["Respond to emitter"]}
Socket server emit with simple message instance:
verbose: websocket writing 5:::{"name":"user.newAuthenticated","args":["{\"m\":\"Respond to all clients\",\"d\":{},\"s\":1,\"t\":\"m\"}"]}
Socket server emit with complex message instance:
verbose: websocket writing 5:::{"name":"user.newAuthenticated","args":["{\"m\":{\"m\":\"__12\",\"a\":{\"field\":\"name\",\"min\":3,\"max\":20}},\"d\":{\"key\":\"fakeKey\"},\"s\":1,\"t\":\"m\"}"]}
The complexe message would render the following sentence:
The min length of name is 3. Max length is 20. and would contain the key: "fakeKey" in data. Just to explain how it works.
As you see, the message get bigger and bigger and it is normal, but I would like to know what I can do to make a better serialization here:
Delete the message itself when there aren't (empty)
Delete the data when it's empty as well
Delete the status when it's false (because it's the default value)
I see a lot of \ in the socket log because it is JSON, I believe that's a problem, because each time I'll add something I'll get extra characters that I do not want. Maybe the JSON isn't a good choice and I should serialize differently, first in JSON like the examples at the top, but then in something else, maybe kind of binary, if it takes less space.
What do you think?
And if it would be a good idea to encrypt somehow the message in another format, would the cost of the encryption be worth it? Because encrypt it would take a bit of time as well, so I'm just wondering if it wouldn't just move the issue, like it would take less time to send the message through socket because it would be lighter, but we would use more time to encrypt it. Just wondering.

My guess is that your message object has two fields (name and args).
The first stop to reduce the length of the message is to get rid of the (pretty useless) outer object and replace it with an array. So an empty message
{"name":"empty","args":[]}
would become
["empty",[]]
or even
["empty"]
The next thing is that you have a bug in the serialization of the arguments. Instead of sending JSON, you wrap the JSON data in a string. Example: In the authenticated case, you send
{"name":"user.newAuthenticated","args":["{\"m\":\"Respond to all clients\",\"d\":{},\"s\":1,\"t\":\"m\"}"]}
but you should send
{"name":"user.newAuthenticated","args":[{"m":"Respond to all clients","d":{},"s":1,"t":"m"}]}
instead. Now the question is whether args is a list of a single object. If it's always a single object, then you could get rid of the [] as well. With my suggested change from above, that would give you:
["user.newAuthenticated",{"m":"Respond to all clients","d":{},"s":1,"t":"m"}]
which is pretty good IMO. If you can make the (de-)serializer handle default values properly, you can reduce this to:
["user.newAuthenticated",{"m":"Respond to all clients","s":1,"t":"m"}]
(i.e. we can omit the empty d property).

For a MMO, I think a minimum of data must be sent to the client. If a socket is called 2xx/3xx by sec, you must reduce the size of the data sent through the socket as most as possible.
On another hand, it also consummes resource to encrypt the object on the server side to send a minified version of the object... Wouldn't it be better not to reduce it and to send an object not reduced so we don't spent resource to encrypt it?

avoid mismatch between server side and client side result

I use PHP and Javascript. In my website some results are processed server side some client side.
Using javascript only, prevents your website from being crawled correctly by search engines and using PHP only prevents correct real time response.
The problem is how to grantee both js functions and PHP functions give the same result? for example suppose there is a function which gives relative time:
JS:
function relative_time(timestamp)
{
...
}
PHP:
function relative_time($timestamp)
{
...
}
Keeping both functions matched with each other is not easy since I want to edit both. For example if both give us:
one year ago
And I change PHP only, to give me:
a year ago
Then JS is not updated too. Is there any standard way to ensure both will act in the same way?
unfortunately js function cannot be called on server side.

If there is some complicated logic, you should implement it server-side and just pull the results via AJAX. That way you only need to maintain the PHP code and provide a kind of AJAX API for access via JS.

I think you just need to make a decision where it is to be done, because if they do vary which is to be dominant ? (that's the one that should be doing it)
Also, how are you saving server load by doing it in both locations ?
Avoid this by making a decision for which code is to do it, failing that, Put a note in you code at both locations reminding yourself to update both locations ?

Best practices for returning and displaying data from AJAX calls

I have some jquery/php interaction set up on a page. It submits some data to the server and gets back records of data which are then to be aligned on the page for comparison and possible action beyond that.
My question is what is the best practice for returning the info and then displaying it?
Return JSON object and then create
html on the fly with js and display
the data?
Return JSON object and then
put that data in already created
containers for the data on the page?
Return pure html from the server and
just put that on the page?
I was rolling these through my head last night and couldn't really figure out if one way would be better for any particular reason.
I'm not a js guru so wasn't really sure the pros/cons and caveats to these different methods.

I think it ends up depending on your app.
Pure HTML is the easiest, you just drop in in place and viola. JQuery makes it a breeze to add the proper events and what not.
However, whenever I've used AJAX it's always evolved into returning JSON and building elements on the fly. If you are populating a tree for example, it may become messy to get the proper nesting. This forces you to have to do client side code anyway at which point simply using JSON from the start is cleaner.
Also, If you plan to use the data calls from other pages then using JSON is the way to go because the HTML will bee fixed.

This completely depends on the way you have things set up in your application. I, for one, prefer to return JSON (second approach in your list), because I have an 'error code' parameter that I check in onSuccess function before updating the content on the page, and if it's not zero then I notify the user of the error, including the error message that came back from the server (server-side validation, database timeout, etc.).

I think you're missing a perfectly valid option, one which I use often. This is my typical schema and it has yet to fail me... :-)
Here's the basic jQuery template I use:
$(function() {
$.getJSON('/some/page',{foo:bar,bar:foo},function(json) {
if(json.outcome == 'success') {
$('body').prepend(json.html);
} else {
// Somehow let the user know why it didn't work
alert(json.error);
}
});
});
Here's the basic backend (PHP in my case) structure I use:
<?php // Page: '/some/page'
/* Blah Blah Blah... do whatever needs to be done... */
// If everything turns out okay (assuming '$output' is the HTML
// you want to display...
echo json_encode(array('outcome'=>'success','html'=>$output));
// If something goes wrong... just do:
echo json_encode(array('outcome'=>'error','error'=>'Uh oh... something is broken'));
Naturally, you'll want to be more specific with your error by putting them into some variable or something. But, you should get the idea. Also, of course you can add more information to the json output. You can have some pre-made HTML and also some other information like a 'success notice' or a new class name for some element, I dunno... whatever... the possibilities are endless.
Anyways, the reason I choose this route is because it's usually faster (based on my experience) to append pre-made HTML to the DOM rather than looping over JSON and inserting the stuff unless it's just, like, a bit of text to replace into an element. But, the method I've shown is, IMO, the best of both worlds. You can attach HTML as a string to one of the JSON properties.
Happy jQuerying :-)

The "possible action beyond that" part of your question makes a big difference. If you need to do other things with the data besides display it, returning as JSON is a clearly better option because you can work with the data as a native JavaScript object instead of having to traverse the HTML DOM. If all you ever intend to do is display it, I don't see any reason to go through the trouble of building that display in JavaScript; just build the HTML in your presentation layer on the server.

This came up recently and possible a dupe: The AJAX response: Data (JSON, XML) or HTML snippet?.
If you are going to be creating HTML then you may as well be returning HTML directly and inject it into the DOM. However, there are times you need to work with objects which is where JSON comes in handy.
If you return a Person object for example then you can greet Person.Name and show Person.Preferences which is really handy. It depends on your design but the general considerations should be to keep HTML out of Javascript unless you're building RIA.

I have used all three and have come to the conclusion that returning HTML is better when introducing new elements to a page.
My experiance is that when building HTML with javascript I am usually replicating work that will have already have been done for the non javascript user journey.
I still prefer parsing json for updating existing elements or creating javascript only functionality. I tell myself this for bandwidth, but I think it just because I love javascript.
As a forth option, I read a great post about how Flickr deal with vast quantities of data with string concatination. Basically just parse a big o' string down the pipe and chop it up on the client. This significantly reduces the on the server, with only a marginal increase on the client.

Returning pure HTML is the best solution. For the most part gzip should neutralize any difference in bandwidth, and rendering via javascript on the client can be slow if the client is a crappy machine. Finally, writing javascript to render HTML is hard to work with compared to using something nice like a view if you use MVC.