Server overload ... with requests? How do I efficiently program server-client wise - javascript

SO, I have a server with MySQL database in it, and a client (browser) that retrieves data from the server and displays to the user.
I'm struggling over whether I should let client get all the data he needs from a MySQL server (using PHP), and let client to do all querying, adding, updating to the data with JavaScript or other related library, and send it back to the server for the server to update his data; OR
whether I should let client send requests (query, add, update, etc) to the server with relevant parameters for server to handle the user's data with, say, MySQL commands.
I think first way could relief the server because all the work is done by the clients' computer, and not by the server, but would be hard for me to learn or make a library that does all the querying and stuff that can otherwise be done with MySQL commands which I find easier to work with at this moment.
And I think the second way would be easier for me, because I can just use PHP and MySQL to perform whatever server needs to do for client, but it makes me think that it would load server with too many repetitive work for each client if there were too many clients.
Which method is better?
At this moment, I'm the only client and server is run on the same computer, so there won't be too much load of commands that server would need to run, but I want to know which method is most canonical and efficient security, efficiency, etc wise.

Both solutions have their pros and cons. If you have a huge set of data, you don't want to dump it all to the client, especially if they only need to view or modify a fraction of it. If any of your data needs to be protected against unwanted change (like a user increasing their access level, credit, etc) you can't place the logic on the client since that will be easy to hack. If neither is a concern, client-side logic may indeed take a lot of load off your database server.
There are client-side frameworks like Angular and React that make working with data easier, although they too have a learning curve. Check if they fit your needs.

Related

Safe Javascript/Sql connection

im trying to create a simple website with HTML/CSS and Javascript. Basically the user should be able to input a number into a textfield and "send it" with a button. When the button got pressed i want to run a Javascript function that searches the number in a sql database.
Creating all that stuff shouldnt be a big problem for me, but i have no clue how to create a safe connection between JS and SQL. I have read that a direct connection with javascript is very insecure.
Some people recommend to use java or c# to built an sql connection. How would that work? Basically just an Javascript code, that runs an java/c# application(which builds an sql connection) and returns the needed sql data?
Also heard that its possible to create a sql connection with node.js, is this safe? Or is another method more suitable?
Greetings
I have read that a direct connection with javascript is very insecure
The danger is in giving direct access to your database to the client. JavaScript is most commonly run client-side in web browsers, so for it to access the database you would have to give the browser (and thus the visitor) a username and password on your database server and let them run raw SQL.
There are many possible security risks with this and it just isn't worth it.
(Aside: You can't make arbitrary socket connections with browser-side JavaScript, so it's impossible to connect to most database servers from it anyway).
If you want to expose data to JavaScript running in the web browser, then the standard approach is to write a webservice.
You can write the webservice in any programming language you like (including JavaScript). It listens for HTTP requests, reads data out of them, possibly performs authn/authz, the queries the database (applying the well-documented defences against SQL Injection attacks) and returns the result (often formatted as JSON).
The client-side JavaScript, therefore, just has to make an HTTP request (e.g. with XMLHttpRequest or fetch) with parameters passed in the query string or request body, and process the data it gets back from it.
Connecting to a database using client side javascript is very insecure as the javascript will need to know the login details. And since the client side javascript is on the client side, any user will be able to see the login details in plain text.
The best way to do this is to make a webservice on a server. When the button is clicked it will make a GET/POST request to the webservice with the entered number as a parameter. The webservice, which can be made using any language pretty much, will create the connection with the database and insert the row itself.
Although I would advise going the webservice route since it will be much easier to make secure. Playing with javascript to database is extremely dangerous unless you have a really good system and understand exactly what you are doing; but if you really want to do it and have an application that requires it, then can use PouchDB connected with CouchDB.
PouchDB is run locally and can sync with CouchDB over HTTP.
https://pouchdb.com/
https://couchdb.apache.org/
There is an answer here discussing basic security with pouchDb synchronizing with couchDb. Basically, each person needs separate login credentials and credentials should never be stored in the page code.
PouchDB security
There are some neat uses for pouchDB: https://pouchdb.com/users.html

how to secure queries in meteor

The question is about duplicating queries in server/client in meteor.js.
here is a solution : https://www.discovermeteor.com/blog/query-constructors/. There , the author proposes a shared file between client and server to hold the queries.
I have readed the article and I find it interesting but I have a question. If you put your queries in a shared file, the client also has access and can modify them? The security problem is not solved?
Code on the client is by definition untrusted. Conversely, code on the server is trusted. Code that is used on both the client and the server (often by being placed under /lib but also by being imported into both) is untrusted when running from the client and trusted when running from the server. Remember that the client gets a copy of the code, the users don't actually have access to the original on disk or the other copy that is in server memory.
With Meteor's latency compensation, a shared method runs on the client first. The client state (in minimongo) immediately reflects the state achieved by running the method. Then the method runs again on the server. If the result is different in some way, then the client state is updated from the server with the correct data.
If you want to hide the method's logic from the server you can just not include it in your client code. You will forego latency compensation but you will keep your secrets secret! (ex: API keys, critical business logic). You can also have pure server code, such as startup scripts and cron jobs, that are never even invoked from the client.
In Meteor, nothing on the client can ever be trusted or considered safe. There is simply no way you can "hide" stuff on the client. If the browser can run it, a hacker can read it. And modify it.
Remember that queries on the client run on data on the client, and then the result of those queries is sent over a web-socket to/from the server. It is then the job of the server to do security/authorization/sanity checks on all data going out or coming in, to make sure only the data the client is authorized to view is sent, and only the modifications the client is allowed to do is actually done on the server DB.
The Discovermeteor blog you linked to is all about how to reduce code duplication between server and client, and still have flexibility between them. This has very little to do with security.
It does not really matter from a security point of view that the source code for the DB queries are readable on the client, because your server needs to do its security police job anyway. Otherwise you have an insecure app, even if the actual query source code is unknown.
An attacker can always look at the DDP protocol, it is almost as readable as a MongoDB query!
I think you're asking 2 different questions:
1) How do you ensure the security of a query?
2) How do you ensure the secrecy of a query?
WRT #1: Keeping a query in a shared lib file is secure because regardless of whether a client knows what query you're running, he won't be able to run it on the server and even if he changes it, that only alters the client copy, and doesn't affect the server's copy.
In the example you link to, note that the client is only able to change the limit field. He can't change the 'find' field. Even if he were to redefine the 'latestPost' function client-side to allow an additional parameter that gets added to the 'find' field, that function isn't redefined on the server-side so only the original definition will be used server-side (one point, however, is that in the example, the limit field isn't sanitized or checked for validity; a client could send text and cause an error).
So it would still be secure as you are limiting exactly which parts of the query constructor the client is allowed to change.
WRT #2: you're correct that this means the query won't be secret. The client will know exactly how you're querying, and with that info, may be able to deduce parts of your internal data structure.
Whether or not this is an issue is up to you, although I will say that in the security world, "security through obscurity" is considered bad practice: you should write your code such that even if all of your data structures, algorithms, and code is known, your data is still secure. That's why, for example, you can easily download the code for any encryption algorithm: the security doesn't depend on keeping the algorithm secret.

Which is more efficient to send WebSocket updates with a MySQL database change

I'm currently experimenting with WebSockets in a bid to reduce / remove the need for constant AJAX requests in a potentially low bandwidth environment. All devices are WebSocket compliant so there's no issue there, and I'm trying to keep it to native PHP WebSockets, no node.js or other frameworks / libraries (Which so far has been fine).
What I'm looking to do is to decide how to go about notifying connected clients about an update to a database by another Client. The use case in question is a person pressing a button on their device, which then alerts that persons manager(s) to that press. So the two options I have though of are as follows:
1. Looping a Database Query (PHP)
My first thought was to insert a query into the WebSocket server that is effectively saying "Has the alert field changed? If so, notify the manager(s)". Whilst this is the most straightforward and sensible approach (That I can think of), it seems wasteful to have a PHP script designed to reduce strain on the server, that is now running a query every second, however, at least this would ensure that when a Database update is detected, the update is sent.
2. Sending a notification from the Client
Another thought I had, was that when the client updates the Database, they could in fact send a WebSocket notification themself. This has the advantage of reducing any intensive and looped queries, but also means that I'd need to have a WebSocket message being sent every time I want to change any data, such as:
$.post("AttemptDatabaseUpdate.php", {Data}).function(Result) // Don't worry about the semantics of this, it's not actual code
{
if(Result == "Successful")
{
SendWebSocketNotification(OtherData);
}
}
Maybe this is the best option, as it is the most efficient, but I worry that there is a chance the connection may drop between updating the Database, and sending the WebSocket notification, which may create a need for a fallback check in the PHP file, much like the one in the first solution, albeit at a longer interval (Say every 30 seconds).
3. MySQL Trigger?
This is purely a guess, but perhaps another option is to create a MySQL trigger, which can somehow notify the server.php file directly? I've no idea how this would work, and would hazard a guess that this may end up with the same or similar Query requirements as solution #1, but it's just a though...
Thank you in advance for your help :)
EDIT: Solution possibility 4
Another thought has just popped into my head in fact, whereby the PHP file used to update the database could in fact have a WebSocket message built into it. So that when the PHP file updates the database, the WebSocket server is notified via PHP, is this possible?
If you use websockets, you should use notifications from client. That's one of their main use cases.
If you're worried about inconsistencies due to connection dropping or something changing in-between, you could implement a system similar to HTTP ETags, where client would send a hash code that you can respond on server side if there is a conflict in updating.
Update: I guess I understood your initial issue a bit wrong. If I understand your use case correctly: you are sending database updates from a client and after that all connected clients need to be updated. In that case, I think server should send the update messages after DB updates have been done, so I agree with solution 4. I am assuming here that your websocket server is the same server running PHP and doing the DB updates.
However, depending on your use case, client should still send a hash value on the next request identifying its "view of the world", so you would not be doing identical updates multiple times if a connection gets broken.
Update 2: so it was now understood that you indeed use a separate, standalone websocket server. Basically you have two different web servers on the server side and are having an issue on how to communicate between the two. This is a real issue, and I'd recommend only using one server at a time - either take a look at using Apache websocket support (experimental and not really recommended) or migrating your php scripts to the websocket instance.
Neither PHP or Apache was really build with websockets in mind. It is quite easy to set up a standalone websocket server using only PHP, but it might not be so easy then to migrate the rest of the PHP stack to it if the code is relying on Apache/web server on. Apache websocket support also is hardly optimal. For a real websocket solution, unfortunately, best practice would be using a technology that is built for it from the ground up.
The better answer is to send notification through Server side when database is updated by PHP script, so that script have to add options of web sockets to directly send notification to all web socket clients registered.
User send content->Php script process content and save data according to true condition->check database is updated by checking return of mysql_query/other alternative->if true than use web-socket and send notification to all users
now this is more easy/handy/bandwidth saver.

Why make the server push data?

Why make the server push data to get notifications, like using SingleR while it can be made client side?
Using a javascript timing event, that checks for recent updates at specified time intervals user can get notifications as long as he remains connected to the server.
So my question is why do more work at the server that the client can already do?
It's not more work to the server, it's less work. Suppose that you have 10000 clients (and the number could easily be in the 100K or even millions for popular web-sites) polling the server every X seconds to find out if there's new data available for them. The server would have to handle 10000 requests every X seconds even if there's no new data to return to the clients. That's huge overhead.
When the server pushes updates to the clients, the server knows when an update is available and it can send it to just the clients this data is relevant to. This reduces the network traffic significantly.
In addition it makes the client code much simpler, but I think the server is the critical concern here.
First if you didn't use server push you will not get instant update for example you can't do chat application, second why bothering the client to do job that it is not designed to do it? third you will have performance issue on the client cause like #Ash said server is a lot more powerful than a client computer.

How can i get the HTML5 Local storage values in server side

I am a .Net developer, I know that the HTM5 localstorage is client-side storage technique. I want to get the local storage data on the server-side.
For getting cookie value from server-side we have Request.Cookie in ASP.NET. Is there any solution like that to take the local storage value directly on the server-side? Please guide me. I am using the .net 4.0 framework
Thanks,
Jibu
You will need to pass this information from the client to the server using standard HTTP techniques. Using javascript you could fill:
Hidden fields
Query string parameters
POST
Ajax call to the server
...
It will all depend on how your application is organized, what kind of information is being stored, its volume, whether you want to redirect or not, ... But in all cases this should be done using javascript since that's the only way to access data stored in localStorage.
No. The whole point of local storage is that it is local. One of the advantages of it over cookies is that you can store lots of data in it. One of the advantages of cookies is that they are tiny so the overhead of including them in every HTTP request to a given host is small. There two advantages are incompatible so you won't want them in a single technology.
If you want to get the data on the server, then you need to get the client to send it explicitly (e.g. via Ajax).
This is a widescope question. (like the length of a piece of string), but Ill try to make this helpful:
If you have values in local store in webserver I assume your webserver is JSON? Or did you use the sql local storage option?
Regardless of type of storage, you need to build an interface that both handles:
a) Reading data from your local database -> its important to involve some kind of date or index value in here if you are aiming to sync databases... this is to make sure you send IN ORDER all transactions / updates which are in your database. For this to happen you must store your data not only as tables with inforamtion but also tables that contain events of when updates happened and what was updated. (change tables). This will help check in the server end that everything is sync and also means you dont send data to the server that is not needed and can be kept locally. ((otherwise what is the point of local store if you cant save yourself server database space by only syncing waht is necessary?)
b) A HTTP local server to send the data to your destination client server or database server, etc (however you have set your infrastructure) - I recommend using industry standards for your language and server, which is Ajax and JQuery. If you do a lot of streaming of data then i recommend looking into RXjs with Ajax to get a http interface built (interface in this sense just means a way to expose your client like an API and post http calls)
c) An event loop to handle how often and what triggers the synchronization so that you dont destroy your users machine with overdoing it (you dont want to do this too often, but also want to it to be meaninful rather than "every night" maybe user enabled whenever you detect an event which triggers wifi available again.) - i recommend using native wifi reading capabilities built into Apache Cordova and also industry standards for your server setup (for example Express.js for Node.JS).
Obviously the backend server needs to have its API set up and authentication / authorizations, etc.

Categories