I need to collect all pages from an API that uses cursor-based pagination. Each call returns (in addition to a page of data) a boolean 'hasNext' and a key 'after'. If 'hasNext' is true, the next request passes 'after' to indicate where the next result should start.
Since each call is dependent on the result of the previous one, it seems this must be done synchronously. I can probably use XMLHttpRequest to make synchronous calls, but I often see comments that say synchronous requests are being deprecated and fetch is for asynchronous requests only.
Any suggestions on how to handle synchronous requests (where each request is dependent on the result of the previous one and the results need to be in order)?
I've seen several methods for handling paginated APIs, but they all seem to apply to offset pagination.
Related
I currently have a few basic question with asynchronous calls in JS.
I have searched and found a similar question posted in the past here:
Parallel Ajax Calls in Javascript/jQuery
But I still have some doubts about asynchronous calls.
Let´s say that I make it asynchronous using a jQuery when to join both flows when both AJAX calls are finished. What would happen if one of the AJAX calls fail?
Some of the calls that I need to make should be asynchronous only in some cases. Is it a good practice to make the type of call (asynchronous true or false) variable? Or doesn't it really matter to have an asynchronous true call when there is only one JS flow?
If I need to do a few database calls with my asynchronous methods (different tables). Could I have a conflict if i make two database calls at the same time because make my AJAX asynchronous? I am quite new to asynchronous calls. (SQL database) (Spring Boot Java 8)
The basic flow would be something like this:
Ths answer is a compilation of everything told in the comments, not my own.
You handle it in your code.
Synchronous calls are deprecated, so never make a synchronous ajax call - the first A in AJAX actually stands for Asynchronous, so a Synchronous AJAX request makes no sense anyway (though of course it is a thing)
It is the same in Java as most other systems: the backend may process your calls in a different order or even truly in parallel (think a cluster, one call is processed from one node, the other call fro another node). So it depends on your semantics: if one depends on the outcome of the other, then you should make sure they are only called sequentially. If they both contribute independently to the final outcome, then you have to make sure to revert the effects of the one, if the other fails. There are many ways to do that - and maybe it should be a responsibility of the backend.
I have an object which acts as a client side API Client which exposes several functions which all return jQuery ajax() objects. Some of those ajax calls have .done() and .fail() calls chained directly onto them because they are actions which need to be taken every time the API responses come back before the rest of the js code is allowed to deal with the response. Pretty standard stuff.
I need to kick off a variable number of API requests, wait for all to fail or succeed, and then continue processing. For the examples I will create, I will simplify this down to just two ajax calls.
So I create an array to hold the returned ajax objects, and then use $.when().apply(null, deferreds).then(function(){//do something after both ajax requests complete}). When the calls complete successfully, everything works great. When calls fail (such as if the ajax call 404s), things are not so great.
The problem is that .then() doesn't seem to detect the fails, even though I thought then() was supposed to be fired regardless of success or failure of the underlying promise(s).
I can switch to .always(), which seems to work better (in that it detects the failures and still triggers the callback) but it seems to fire before some of the .fail() callbacks that are registered directly on the ajax calls, which doesn't make sense to me since I thought the callbacks for an ajax call were called in the order they were registered.
I'm sure I'm just missing something about the behavior of the ajax() when() then() combo.
Fiddle showing successful calls using .then(): https://jsfiddle.net/kwrLyw6q/5/
Fiddle using .then() with failed ajax calls (not working, would love to know why. Seems like this is the "right" way to do it, but I can't figure out where I'm going wrong): https://jsfiddle.net/kwrLyw6q/2/
Fiddle using .always() (working, but notice the out-of-order callback order. At least, out of order compared to the order I want them!): https://jsfiddle.net/kwrLyw6q/7/
It looks like deferred.then() takes three arguments:
success function (first argument).
fail function (second argument).
progress function (third)
updated fiddle
Recently, I have been developing web application and I realize that I am not making use of the asynchronous property at all. Hence I am ending up with a lot of nested callbacks.
For example, if the user want to get a file from the server through a particular API, I will have code similar to this,
db.query(<select list of permitted files_names>, function(err, filenames) {
async.each(file_names, function(name, next) {
//open each file to put into array
});
})
This code needs to query database to get a list of file names before looping asynchronously and putting each file content into an array. Finally it will return the finished array to the client.
With the nested callback, and async library, this code is behaving like a synchronous code.
names = db.querySync(//select list of permitted files_names);
for(name in names) {
//open each file to put into array
}
I am better off writing synchronous code like this since it is much neater. My use case might be a little strange but most of my api behaves in similar manner and that makes me think why do I even need asynchronous function?
Can someone please enlighten me if there are any differences between these two codes in term of performance? How do I make use of non-blocking property to enhance the performance in this use case?
If you're writing callback functions you're using by definition using async calls. The callback function fires only when the operation is complete or has errored out. You don't need a fancy library to use these, this is the backbone of how Node's event-loop driven subsystem operates.
Node strongly advises against using "Sync" calls. The Node core only includes a handful as a convenience, they're there as last-resort tools. Many libraries don't even support them so you absolutely must get used to writing async code. In the browser environment, for example, you simply cannot use blocking calls without jamming up the JavaScript runtime and stalling the page.
I prefer using Promises line Bluebird implements to keep code orderly. There are other ways, like the async library, which can help manage otherwise complicated nesting patterns.
Some of the perks include things like Promise.all method runs a series of promises to completion and then triggers a next step, and Promise.map which iterates over a list, running async code for each element, then advancing when the list is complete.
If you're disciplined about organizing your code it's not too bad. Node does require a lot more attention being paid to the order of operations than in a traditional sync-by-default language like Ruby, Python or Java, but you can get used to it. Once you start working with async code rather than fighting it you can often do a ton of work quickly, efficiently, and with a minimum of fuss, in many cases more effectively than in other languages where you must juggle threads plus locking and/or deal with IPC.
Yes, there is a difference in the two codes in terms of performance.
In synchronous code:
names = db.querySync(//select list of permitted files_names);
you are calling the database here to give list of names. Assume , this takes 10 sec. So for this time, nodeJS as it is single threaded gos into blocking state. After 10 sec, it executes the rest of the code . Assume this for loop takes 5 sec and some code takes 5 sec.
for(name in names) {
//open each file to put into array
}
//some code
Therefore it takes a total time of 20 sec.
whereas in Asynchronous code:
db.query(<select list of permitted files_names>, function(err, filenames) {
NodeJs will ask the database to give list of names to a callback. Assume that it takes 10 sec. And immediately it goes into the next step(some code), but not into the blocking state. Assume that some code takes 5 sec.
async.each(file_names, function(name, next) {
//open each file to put into array
});
})
//some code.
After 5 sec, it will check whether it has an i/o operations to be performed. Once the call back is returned. It will execute the function(name, next) {..} for the 5 sec.
So the total time here is 15sec.
In this manner the performance is improved.
If the asynchronous code should be clear and neat then make use of closures & promises.
For ex: Above asynchronous code can be written as
fun = function(err, filenames) {
async.each(file_names, function(name, next) {
//open each file to put into array
}
db.query(<select list of permitted files_names>, fun);
The benefit is simple: By using asynchronous code, the current thread (remember, Node.js is single-threaded) is able to handle other requests while the current request is waiting on something (like a database query) to return.
If you use synchronous code instead, the current thread will block while it waits, and it won't be able to handle other requests in the meantime. In other words, you lose concurrency.
To keep your asynchronous code clean, look into promises (to avoid deeply nested callbacks) and ES7 async/await (to avoid callbacks at all and write asynchronous code that looks just like synchronous code).
I'm looking for a tried and true way of handling asynchronous calls to API endpoints returning JSON (using polymer's latest rendition of the iron-ajax element). These API calls rely on etag matching, so it is important that the etag sent matches the one on the server. I have that part working, except in certain circumstances, where quick succession API calls may cause an out-of-sequence response (and therefore can get the etag out of sync). There are also multiple API endpoints (i.e. different URLs). So, sometimes if quick succession calls using different endpoints are initiated via an iron-ajax element, it can cause issues for the response handler function, as the response handler currently checks the URL of the ajax element to know how to handle the response appropriately. Therefore, if the 2nd call overwrites the URL of the ajax component before the 1st call response is received, when the 1st call does come back the responseHandler doesn't handle it appropriately. Perhaps there is a much better and reliable way of checking exactly which call has returned?
I know I'm not the first person to encounter this scenario, so I'm wondering if someone out there can show me the enlightened path? I'm thinking there is a simple strategy to handle this problem, perhaps implementing call queuing etc., but I'm not sure if iron-ajax has anything built-in that could help in that regard.
An example with some sample code would be absolutely stupendous!
If you depend on multiple API endpoints, I would have a separate iron-ajax element for each one so that they don't stomp on each other if the URLs change (through data-binding or something):
<iron-ajax id="cats" url="https://api.example.com/cats" handle-as="json"></iron-ajax>
<iron-ajax id="dogs" url="https://api.example.com/dogs" handle-as="json"></iron-ajax>
Generate the requests, and use Promise.all() to wait for both requests to complete:
<script>
Polymer({
is: 'my-element',
...,
fetchPets: function () {
var catsRequest = this.$.cats.generateRequest();
var dogsRequest = this.$.dogs.generateRequest();
Promise.all([catsRequest.completes, dogsRequest.completes])
.then(function (requests) {
var cats = requests[0].response;
var dogs = requests[1].response;
// do whatever you want from here...
alert(cats.concat(dogs));
});
}
})
</script>
Firstly iron-ajax uses iron-request under the skin and that returns a Promise.
I think what you are trying to achieve is that with a string of sequential calls, you can have the same order of responses, despite them all running, possibly in parallel.
This article has an example of doing this with a set of chapters from a story
http://www.html5rocks.com/en/tutorials/es6/promises/
The article gives an online example here:
http://www.html5rocks.com/en/tutorials/es6/promises/async-best-example.html
In essence it is doing a Promise.all(array of requests).then(...) to give you an ordered array of responses.
If you look at the source code for iron-ajax, you will see how it creates the iron-request, and then uses it. You could do the same but based on the example I have pointed to above.
If I have misunderstood, and you are trying to ensure the requests go out sequentially, then you can still do that with Promises, but instead of using Promise.all, you chain them with .then clauses one after the other.
I was just reading another question about jQuery's synchronous ajax call, and I got to wondering:
What circumstances make a synchronous version of an ajax call beneficial/necessary?
Ideally I'd like an example, and why synchronous is better than standard ajax.
The only reasonable example I can think of (that can't be worked around another way) is making a call in window.onbeforeunload, where you need it to be synchronous, or the page will move on and the request will never complete.
In this specific case using standard/asynchronous behavior, you're all but assured the request will die too early to have any impact, or ever contact the server.
I'm not saying I'm in favor of doing this, quite the opposite (as it negatively impacts the user's browsing speed). But...there's not much option here.
In sum, please do not use synchronous requests as #Brandon says: they are a cheap/easy/quick way to avoid making a callback. In addition, modern browsers show warnings if synchronous requests are made and we do not like that. Make your world asynchronous.
synchronous ajax is often used to retrieve a valued from the server which is required to further continue processing of client side code. in such case, the ajax call will block until the call returns with the desired value. example:
a javascript function needs to compute salary for an employee:
step1 : get the employee id from the form
step2 : make a sync server call passing the emp.id to get his salary/hour
step3 : multiply salary rate by number of working hours
as you can see, total salary cannot be computed unless the server call is finished so this should be a sync function, although if using jquery, one could handle onSuccess to compute the salary asynchronously but processing will continue in this if you have a message box to display the salary, it will appear empty...
I would venture a guess that it'd be good in a scenario where you want to perform some ajax calls but you have one call that relies on the results of another call. If you perform them synchronously you can wait for the independent to finish before the dependent call fires.