Let's imagine the situation that we've sent two similar (almost similar) async ajax requests to server one by one. Because of lag in network, the second request was executed before first request.
Ajax request #1: /change/?object_id=1&position=5
Ajax request #2: /change/?object_id=1&position=6
In result, we have object_id=1 position set to position=5, but we want position=6 because Ajax request #2 was executed after Ajax request #1 by us.
What is the best practice to avoid this on server side and client side?
Are you worried about racing conditions from the same client or from multiple clients?
If from the same client, I would think the safest bet would be to include a unix timestamp in the ajax request and log this value on the server. If a request comes with a timestamp that is older than the last logged value, ignore the request (or send a warning back to the browser).
I'm not sure how you would handle multiple clients with unsynchronized clocks...
For situations like this, I usually put a check in my success handler to make sure that the value being returned is still the one that I want. This will require sending up the parameter you're searching across in the results object.
For example:
var query = $('input').val();
$.get('/search', { query: query }, function(res) {
if(res.query == $('input').val()) {
//show search results
}
});
I don't know the particulars of your use case, but this general pattern should help.
On the server :
Build a request table to map request id to timestamp
Log any request to the server, expect all requests come with timestamp
If any request comes out of order (e.g. position 6 comes before 5)
Check the request table, if it is an earlier request (timestamp) then do not process the request and send an ignore flag
If it comes in order
This is fine, proceed as usual and no need to send any ignore order
On the client:
When request comes back in, check the ignore flag. If it is there. Don't do anything to the client
Otherwise proceed as usual by processing the data
Note that this implementation that I suggested requires you to send back and forth data (such as JSON) and not the presentation code (such as HTML fragment) as you would need to check for the ignore flag on the client side.
This answer is similar to what #Farray suggestion of using timestamp.
Related
I have an array in memory (nodejs server side) that I am updating every 10s and a client that do a request every 10s also. The request parses the array to get it in a specific string format. Also, the update process is in a setInterval function.
I want to run a stress test to that endpoint. I thought that if I move the process of parsing the array to string to the place where I am updating the array, then service only will return a string (like a cache) and stress test will not be a problem to pass.
Here, my problem is: if the time required to update my array and parsing is so long until reach the assignation of a new value for the string cached, then client will receive a non correct value from the service because it continues updating. So my question is how can I be sure client will receive the correct value always. That is, how can I avoid race condition in this context.
The good news is; unless you have spawned a worker or another process Node is single threaded. So it is quite impossible for Node (under normal circumstances) to encounter race conditions.
However, from your description it sounds like your are concerned about the asynchronous nature of your http requests.
Client makes a request to the server
Server begins work
10 seconds pass (server is still working)
Client makes another request to the server using outdated information since server isn't done working.
Server returns old data but, at this point it is too late.
Fortunately, there is more good news. Javascript has a TON of built in support for asynchronous programming. Usually you would wrap your requests in a promise to avoid such pitfalls. Resulting in a process that looks like this:
Client makes a request to the server
Server begins work
Client waits until server comes back before continuing
Server finishes work and returns data to client
Client send another request to the server (ad-infinitum)
You can also make your Promises look like synchronous programming that you're used to via the new(-ish) async functions. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
Short question: assuming a non-idempotent post operation, how do you defend your post request handlers in node.js from being called multiple times before they can respond, and hence cause data corruption?
Specific case: I have a matching API, which takes about 2-3 seconds to return (due to having to run through a large userbase). There are a number of operations where user can simply double call this within the same second (this is a bug, but not under my control, and therefore answering this part does not constitue an answer to the root question). Under these conditions, multiple matches are selected for the user, which is not desirable. Desirable outcome for this would be for all of these rapid requests to have the same end result.
Specific constrains:
node.js / express / sequelize.
If we add a queue, every single user's request will be on top of all other users' request, which might have drastic implications during heavy traffic.
I propose a solution where the server gracefully responds to the same* request and hence no changes on the client are required.
* First we need to establish what constitutes a request to be considered as "same". When you plan this kind of graceful, sync, response you would put a increasing counter into the clients request and this counter is the unique attribute that defines a request as same. But since you might not have access to the client and have no such counter you could define that requests to be the same if their post-body + url are the same (and could throw in that the user needs to be the same too).
So for every user you immediately save the request when it reaches your server. An easy way would be to hash the url + post-body, say with SHA-256 and save it in an object like this:
requests[user][hashOfRequest] = null
That null will be replaced by a response object once your server has calculated it. Now you process the request.
If the client sends the same* request again you can easily find out by checking your requests[user][hashOfRequest].
If the server has finished the processing it will contain a response object which you just send back to the client. If its still empty you need to wait for the processing of your server (of the first request) to finish. Maybe using an event listener or other task sync patterns.
Once the server has finished the first request it will generate the response and save it in requests[user][hashOfRequest]=response and emit the event, so that potential waiting clients will get the response too.
No more double processing and connection drops from clients, where the response does not reach them, are also handled by this pattern.
Of course you should clean up the responses hash table after a time that fits your (client) scenario. Maybe 10 minutes after the request was put into the hash table.
You can push all your requests into a queue. In this case all your responses will have to wait for the preceding ones to finish.
The other solution is to use sequelize transactions, but that would cause lock_wait_timeout errors in DB.
Try to use a transaction.
If you put all of your SQL commands into a transaction, I think the requests will be separated.
The solution I've settled on eventually was a variation on #Festo's general approach, as follows:
adding a unique key constraint for the matches
Each parallel requests attempts to create a new match; all but the first one will fail to do so due to the constraint
if the constraint insert fails, app just pulls the match already in the database, adds it to the rest of the matches, and returns it
This makes it impossible to spam-create new matches via rapidly calling the API. However, I am not satisfied with this, on account of this approach not generalizing to eg: non-deterministic idempotent operations (eg if the matches would be generated randomly, consequent calls would return different matches, and therefore a simple constraint check would be insufficient).
All my internet points for an answer which does not requre client-side ticket management (#Sushant), nor queues, and can handle non-deterministic idempotent functions.
Under my APIs I use express-jwt and token based authentication for all REST authentication. I keep these tokens valid for only one request. Once used token will get blacklisted.
So even if your client issues multiple requests only first one will be accepted. Others can throw error 409 Conflict. Once first request is processed new API token will be send back along with response, may be in headers.
Your client will have to keep updating the token from each response. If you are using AngularJS at client thats pretty easy using interceptors
I have a mobile project where I have to send ajax-requests one after the other. The project is using a mobile internet connection (egde, 3G), so it can happen that I lost the connection and I have to cache the failed request (in the localStorage), check at intervals for a valid connection and try again the request.
At the same time other requests come in (from the Browser), so I have to cache the requests in a queue and send them whole in a row.
Sorry for my bad Englisch, I hope you can understand my problem.
Any suggestions? Are there any libraries for my problem?
May be you can use below logic.
1. Create a array which will hold status of your ajax request.
2. Once you make a request add particular request to array and it results(response recieved) to false.
3. Once you recieve response from that request update the array and its results(response recieved) as true.
4. Read this array after particular time and send request again for false once.
I want to build an application which will automatically broadcast notification(s) to a user when data on server is changed. So far, I just know one method of doing this i.e. using JQuery setInterval. Using this function, every client requests data through ajax to server, asking if something changed.
The weakness of this method is every client must send a packet every specific time interval, so my server receives huge data packet. It's so frustrating to manage the server. Are there any alternatives for this besides Jquery setInterval?
If Websockets is not an option for you, you could use one ajax request to the server. Than server side go into a infinite loop. Use the sleep function to not overload the memory. Than check each time if there is something changed. If so, break out the loop and return the data. On the client side send immediately the next request.
After a bit of research it's called "Ajax long-polling requests".
Here is a explanation.
The PHP code would look something like this:
$prevHash = $_GET['hash'];
while(true){
$currHash = GetHashFromTable('myTable');
if ($prevHash != $currRowCount) break;
sleep(3);
}
$response[0] = GetDataFromTable('myTable');
$response[1] = GetHashFromTable('myTable');
echo json_encode($response);
Update
Long polling is not the best option. Better to use web-sockets.
If you want to compare the differences, see this answer: https://stackoverflow.com/a/10029326/3269816
I need to send many requests to server 50-100 requests to load data, each response has at least 0.5KB and at most 7KB of data.
I send the requests using ajax as following: (code is simplified)
for (var i=0; i<elements.length; i++) {
var element = elements[i];
// make ajax call with element as parameter and update page to show data for element
}
This works for my needs, because I dont need data to come from server in order, and it works most of the time. But sometimes the last few elements dont get loaded and I get communication link failure error in my chrome javascript console.
I am assuming that the server got overloaded, how can I regulate sending requests to make sure I get a response for each request in the shortest time possible?
Notes:
I use Spring MVC in the backend
I use ExtJS Ajax to make the requests
Try using seperate loops for your data uploading process. Overloading is the only cause for that Communication failure.
I solved this by recursively calling the each request, this way only one request is sent to the server at the time, and overload is avoided.