I have a synchronous API to invoke a server (HTTP GET) currently implemented with XMLHttpRequest.
The API does caching and will, if cache isn't deemed too old, return from the cache and invoke the server asynchronous to refresh cache.
Sometimes cache isn't available or too old, and then the API will synchronous call the server to fetch an accurate value before returning result to caller.
Result will contain a boolean success flag along with payload and clients handles result accordingly by looking at this flag.
There are two problems I can see with doing like this;
When cache isn't available and server isn't reachable or answering slow I would like to bring up a spinner so that the user is aware we are waiting for server.
In addition I would like to set a timeout value where we abort server request and handle the error accordingly.
Seems like I should be able to use setTimout operations but I have not been successful.
Preferably I would like to keep clients intact (not change the API to asynchronous).
Is there a way to achieve this?
The synchronous API was made responsive by maintaining a cache that was pulled from server asynchronous.
The cache was protected by a grace period under which we do not pull new value from server to avoid hammering the server.
For the most cases this was enough to assert there was always a cached value that could be provided to the client.
For a few cases where we have to pull new data the best solution would be to go fully asynchronous, that is also update client code.
Currently that is not an option, so in addition to above a heartbeat mechanism was put in place that toggles online/offline status to prevent trying synchronous pulls when offline.
Related
I know that different browsers have different amounts of concurrent connections they can handle to the same hostname, but what exactly happens to a new request when that limit is hit?
Does it automatically wait and retry again later or is there something I need to do to help this process along?
Specifically, if this is a XMLHttpRequest executed via JavaScript and not just some assets being loaded by the browser from markup, could that automatically try again?
I have a client side library that makes multiple API requests and occasionally it tries to send too many too quickly. When this happens, I can see server side API errors, but this doesn't make sense. If the concurrency limit stops requests, then they would have never hit the server, would they?
Update: Thanks to #joshstrike and some more testing, I've discovered that my actual problem was not related to concurrent HTTP request limits in the browser. I am not sure these even apply to JavaScript API calls. I have a race condition in the specific API calls I'm making, which gave an error that I initially misunderstood.
The browser will not retry any request on its own if that request times out on the server (for whatever reason - including if you exceed the API's limits). It's necessary to check the status of each request and handle retrying them in some way that's graceful to the application and the user. For failed requests you can check the status code. However for requests which simply hang for a long time it may be necessary to attach a counter to your request, and "cancel" it after a delay... Then if a result comes back bearing the number of one that has already been canceled, ignore that result if a newer one has already returned. This is what typically happens in a long-polling application that is hitting a server constantly and not knowing whether some pings will return later or never return at all.
When the limit on the Chrome is reached it pauses anymore requests. Once one request has been responded to, the browser sends the next request. On Chrome that limit is six for me.
To allow an express endpoint to return more quickly to the caller, is it reasonable to invoke an async task, but not await it before returning if the caller does not need a confirmation?
This obviously limits the error handling and retry options if the async task fails, but assuming you don’t care about that, it does allow for express API calls to complete more quickly (assuming awaiting the async task is not semantically meaningful for the API call)
How would this approach compare to other web background job approaches that are invoked via an API request?
It is perfectly appropriate sometimes and not appropriate at other times. It really depends upon the specific API operation, the expectations of the client, the likely use by the client and how the API is documented.
A core database operation, for example, would never do that because obviously the client usually needs to know whether the database operation was successful or not.
But, an advertising-related API that is just collecting tracking data and storing it away for later use probably has a client that is just going to ignore any return from the API call anyway and the whole thing may scale better if the HTTP connection can get shut-down as soon as possible (without waiting for some storage commit to finish). In addition, if some of this kind of data is lost when some unusual circumstance happens, it's no big deal and the client wouldn't be doing anything differently anyway. So, if you've determined that the client doesn't need a final status on the operation and there are scale or performance benefits to ending the request before the async operation is done, then it can be perfectly appropriate.
In another example, imagine you have a write queue where you queue up N log items in memory to write to disk before you actually write them all to disk because that improves scalability and performance of your disk I/O significantly. In that case, the final write to disk doesn't happen until some future request by some other client when the queue hits a certain size that triggers the actual physical write. It would be a significant penalty to the host API server (and perhaps even to the client) to wait until the final write occurs in order to return a successful API response.
Its kind of ok, but as you mentioned it has some drawbacks with regards to retrying and monitoring and synchronizing calls.
What you probably should do is push a message into a queue and have a background worker do the processing. Save a record to a database which represents the state of the operation, then queue a message to the queue. You can monitor the progress of that work by looking at the state of the record you just saved, have the worker update it.
I have a Node.JS api using Restify which is basically an aggregater for several other API's for use inside a Google sheet.
When the Google sheet is initially opened, it makes several requests to my server to request a bunch of data from various API's, which my server then dutifully looks up and returns.
I have implemented rudimentary memory based caching - If a request for the same data comes in it will serve it from memory until an expiry time is reached (I'm open to moving to Redis for this soon).
My issue is that quite regularly a second request for the same data will come in while the first request is still being looked up/parsed/served, meaning I'm requesting the same (several gigabytes) of data in parallel.
How can I effectively pause the second request and have it wait until the data is available from the first request? I don't mind having a high timeout and waiting for the first request to end before the second starts, or alternatively some sort of "back off and try again in a minute" logic is doable.
I imagine some sort of promises or saving callbacks somewhere would be best for this, but I'm not sure what kind of best practices or suggested methods there are for this.
It is not possible for me to regularly request and cache the data server-side as the potential range of values that clients can request is fairly high.
Keep a cache of promises. If a request is in progress, the cache will indicate it was already requested and you can still await the unresolved promise, then respond to both requests when the data is ready.
The rest of your code will need to be refactored to await the values in the cache regardless of whether the promises are resolved or not, and to add entries to the cache when a request is initiated rather than completed.
If a promise in the cache is already resolved, then await will only stall the asynchronous flow of your user function by a single tick in the event loop, meaning you get the advantage of seeing pending requests basically for free.
I have made an application in which i make Ajax call to node server, but the problem is that, if user redirects to another page then also server continues to process the request made by Ajax call and after completing that request, it starts processing the new redirect request.
I am using express as a framework,i don't want to know about client side abort functions, all i need to do is stop server side processing of request made by Ajax call, and instantly start processing for the new request.
all i need to do is stop server side processing of request made by Ajax call, and instantly start processing for the new request.
If your server is not already processing new requests whether the old request is still processed or not, then you have a bigger problem that cannot be solved just by cancelling the requests that are not needed any more.
Your server should always process new requests no matter if the old ones are still processed or not. To make sure it does you need to make sure to never use any blocking call in your handlers. No functions like fs.readfileSync (or anything with Sync in the name), no long running for or while loops (unless they are in async function and don't block the event loop) etc.
When you don't block on old requests before you handle the new ones, you will not have the problem that you describe in your question in which you ask about a specific solution instead of the problem, and the solution you ask about has two issues: it wouldn't solve the problem anyway and it would be impossible to implement.
This question is purely based on assumptions. May or may not be valid problem. Anyway, here it goes
Let's say we have a heavy javascript client app with some numbers
of UI components / widgets, Each of these widgets has
an endpoint to query data from
On a page load, these components will make http request.
Multiple of them; to multiple different endpoints.
Obviously we see that the number of http requests will increase
with heavy client side architecture as compared to traditional web where
UI is generated from the server side.
Sample case:
widget A requests resource A
widget B requests resource B
Of course, we can minimize the http request by having:
parent widget requests an endpoint that return { resource A, resource B }
parent widget distributes data to widget A
parent widget distributes data to widget B
This can be done by, sort of, grouping related widgets based on business logic
Not all can be framed this way. Even if it can, how would maintain code modularity?
Is there any well known design pattern for large javascript apps wrt. performance?
Maybe I am overthinking as I certainly dont have the numbers here.
Any thought guys?
for starters I would consider creating a client JavaScript library that would handle fetching/sending data and make all the widgets use this API.
this way you can optimize/group the flow of data to/from all of your widgets in one place.
One idea that comes to mind (which wouldn't reduce the amount of data transferred, but would reduce the number of HTTP requests) is to route all your AJAX requests on the client side through some common Javascript interface that you control.
Then, instead of sending out one HTTP request per UI request, you can wait a few milliseconds and batch all the requests that occur within that interval, sending out just one HTTP request for the whole batch (you'd have to be careful to only do this for requests going to your server).
On the server, you could have a special generic "batched" endpoint that internally services all the batched requests (preferably in parallel) and returns the results in a batched response.
Then the client side distributes the batched results to the original requesters.
Note that this only works if the requests all take approximately the same length of time to service; you wouldn't want the batched response waiting 30s for one sub-request to finish when all the others are already done. You might be able to address this with a blacklist or something.
Also, try to identify which requests need to be serviced first and assign them priority!
It sounds like what you want is a central queue for all the data requests. Each widget on it's own can make requests by queuing up a specific request. Then, you would have a common piece of code that examines all the requests in the queue and figures out if they can be optimized a bit (requesting multiple pieces of data in one request from the same endpoint).
Using this type of design pattern still keeps all the widgets modular, yet has one small library of code that handles the optimization of the requests.
Technically, how this would work is that you'd create a library function for adding a request to the queue. That function would require an endpoint, a callback function when the data is ready and a description of the request. Each widget would call this common function for making it's data request.
This common function would put each request into a queue and then do a setTimeout() for 0ms (if one wasn't already set). That setTimeout() will be called when the current thread of execution is done which will be when all requests for this current thread of initialization are now in the queue. The queue can then be examined and any requests to the same endpoint that can be combined into one request and the request can be sent. When the data arrives, the separate pieces of data are then parceled out and the appropriate widget's callback is called.
If caching of data would be helpful (if multiple requests over time for the exact same data are happening), this layer could also implement a cache.