Get partially response from node js api - javascript

I want to get a response from node js API to client-side on every step the API completes till API completes its full step.
Suppose I call localhost/api/doStepCompleate on my backend(Node js) API will complete multiple steps but want to send every step response to the client-side till the API completes its full task.
Thank you

If this is an http request/response, then you can use res.write() on the server to send partial responses and finally call res.end() when you're done.
A challenge here will be that many clients are not set up to process partial responses. Libraries such as fetch() will wait for the entire response and not notify their caller of partial responses unless you configure the response as a stream and you have client-side code to parse arbitrary chunks of data as it arrives. See here for some idea how that can work.

Related

Get ajax call progress status in front end

I have a web page which allows users to upload and process specific files. After an user uploads some files, after clicking the 'Process' button an ajax call is being sent to a backend service. In the beforeSend function there is an overlay applied to the screen and a spinner is displayed. When the success function is triggered, then the overlay is removed and a toast notification is being shown like 'Files were processed!'
My goal is to somehow show a progress status for each file based on specific checkpoints in the backend service.
Let's say that the backend service when called does following tasks: parse file, map to specific format, send data to database A.... and in the end it sends back http status 200 and a JSON like
{
"status":"Success",
"message": "File X was processed"
}
Now what I want is that instead of just getting an overlay and disabling the whole page until the success event is triggered, to have a progress bar which is updated for each file based on the exact step where the backend has reached.
For instance, for file A, I would like to see below transitions: 5 % Parsing file, 10 % Mapping file...90% sending data to database, 100% processed.
Is this somehow achievable?
There are few points that you need to look into.
Usually in production code, we need to have timeouts. If you are making an ajax call to the backend API, there will be a timeout associated with that api call. Suppose if the timeout is more than 2 mins, then it will send you a 504 Gateway timeout error.
To overcome this and to implement the functionality which you want, you can have any DB(lets consider SQL server). In your SQL server, make a table:
Process_Table
With schema:
Process_id( Will store the process id/name )
Percentage( Will store the percentage )
At_step ( Parsing, Mapping, Sending to DB etc)
Using Javascript(Or any framework of your choice), use setInterval(), to make check_process() api calls. For check_proceess, you can pass in the process_id, and check against the db. For interval, you can set it to 5 seconds. So that every 5 seconds the call is made.
You can read the response of those API calls and do your processing.
An HTTP request consists of a request and a response. There's no direct way to get status updates beyond the onprogress event which would let you see how much data has been transferred. This is useful for determining how much of the data has been sent to the server, but not so much for how far the server has got with working with that data.
You could store progress in a database and poll a webservice to read the most recent status.
You could also have the server push updates to the client using Websockets for bi-directional communication.
A rough outline for such a system might look like:
Open a Websocket
Send files with Ajax
Get server generated ID back in HTTP response
Pay attention to messages coming over the Websocket that mention that ID
You could also look at doing the whole thing over Websockets (i.e. upload the files that way too). A quick Google search turns up this library for uploading files to a Websocket service hosted on Node.js.

Multi-thread catching response of child thread on javascript

I have a scenario and cant find reasonable answer. So posting the question directly !
Scenario -
I make a ajax request and send a file to be uploaded to the server.
On server(tomcat), I download the file and start processing it.
I run the download-file method on main thread and process-file method on new thread since it takes too long for processing and I dont want user to wait so long.
My Issue :
The Ajax response will catch the response in call back method for download-file method.
My process-file method also return the status response which I require to display on front-end when the processing is complete.
But I cannot because Runnable Interface has return type void for its run method.
Question -
How can I get result from process-file method and send in as response to front-end
If I am able to acheive above, how can I catch the response on front end javascript since main thread response will be caught in ajax success method already.
First: You can get result from process-file method by using Callable along with ExecutorService. Here is an example. And return to the front end is explaining below.
Second: As you do not want to wait user for process-file method so that you already return from main thread and you can not get second return by one single request. The following process make you to understand for better and easier solution for this situation.
Assign a id (e.g. UUID) for the process-file method. Pass it to the process-file method and return the id from main thread to front-end if successfully downloaded.
Then after processing your file, store in DB/wherever you want or put result in a Map<id, result> where id will be the key.
In front end you get the id for the process-file method. You can do call ajax request for certain interval to get the result of this id to a different endpoint (e.g. /file/status/{id}).
In that endpoint for retrieving result, you can get result from Map by id which is key of the Map or if you store in DB then you can easily get by the id and return it to front end.
In case you can not get result in Map/DB (in case of the file-processing not finished yet) you can send exception to frontend so that you can send request again after next interval.

jQuery event when JSON is received - one time URL

I'm trying to scrape a site that uses lots of ajax effects to show data in a table.
There is some data returned via JSON when you interact with the site.
I know the URL and how to construct it but the server returns a HTTP 410 status if I try and re-request this JSON (I guess the server is expiring the data).
I have one chance to capture the data and I'm looking for a jQuery function, something like onJSONResourceReceived would be nice so that I can catch the response and store it in a variable.
Either a callback or a way to cache the data in a variable would be great.
Or if there is already a variable that stores all JSON resource already received in memory, that is even better.
All the functions I've looked at are for situations where you know or can re-request the URL.
This question is similar but for CasperJS:
How to get the response after a POST request in CasperJS
Look at the $.ajaxSuccess
Attach a function to be executed whenever an Ajax request completes
successfully.
$(document).ajaxSuccess(function( event, request, settings ) {
});

How to regulate sending requests to server?

I need to send many requests to server 50-100 requests to load data, each response has at least 0.5KB and at most 7KB of data.
I send the requests using ajax as following: (code is simplified)
for (var i=0; i<elements.length; i++) {
var element = elements[i];
// make ajax call with element as parameter and update page to show data for element
}
This works for my needs, because I dont need data to come from server in order, and it works most of the time. But sometimes the last few elements dont get loaded and I get communication link failure error in my chrome javascript console.
I am assuming that the server got overloaded, how can I regulate sending requests to make sure I get a response for each request in the shortest time possible?
Notes:
I use Spring MVC in the backend
I use ExtJS Ajax to make the requests
Try using seperate loops for your data uploading process. Overloading is the only cause for that Communication failure.
I solved this by recursively calling the each request, this way only one request is sent to the server at the time, and overload is avoided.

Regarding multiple ajax request for same function

My Aim:
To get response from ajax and show response from ajax as "First come first serve basis".
Technology: ruby on rails + ajax (javascript)
Explanation:
In image below 5 request are shown. 1st and 2nd request is re-run with same 4th and 5th request.
Third request should Ideally take time all other request should take less than a second.
I wish to get response from server via ajax independent of request sent.
In sort, If 3rd request complete in 4.49 second and 5th request take 0.5 second. 5th Request should not wait for third request. Is it possible ? How?
Kindly help me !
From the Ruby 1.9.x Web Servers Booklet
WEBrick is implemented as a single process multi threaded server. Nothing prevents you from starting several WEBricks, each listening on its own port and load balancing between them via and external load balancer. But the server itself does not provide any multiprocessing features of its own.
If you want to process several requests in parallel, you may need to chose a different server or server setup.

Categories