I have a web page which allows users to upload and process specific files. After an user uploads some files, after clicking the 'Process' button an ajax call is being sent to a backend service. In the beforeSend function there is an overlay applied to the screen and a spinner is displayed. When the success function is triggered, then the overlay is removed and a toast notification is being shown like 'Files were processed!'
My goal is to somehow show a progress status for each file based on specific checkpoints in the backend service.
Let's say that the backend service when called does following tasks: parse file, map to specific format, send data to database A.... and in the end it sends back http status 200 and a JSON like
{
"status":"Success",
"message": "File X was processed"
}
Now what I want is that instead of just getting an overlay and disabling the whole page until the success event is triggered, to have a progress bar which is updated for each file based on the exact step where the backend has reached.
For instance, for file A, I would like to see below transitions: 5 % Parsing file, 10 % Mapping file...90% sending data to database, 100% processed.
Is this somehow achievable?
There are few points that you need to look into.
Usually in production code, we need to have timeouts. If you are making an ajax call to the backend API, there will be a timeout associated with that api call. Suppose if the timeout is more than 2 mins, then it will send you a 504 Gateway timeout error.
To overcome this and to implement the functionality which you want, you can have any DB(lets consider SQL server). In your SQL server, make a table:
Process_Table
With schema:
Process_id( Will store the process id/name )
Percentage( Will store the percentage )
At_step ( Parsing, Mapping, Sending to DB etc)
Using Javascript(Or any framework of your choice), use setInterval(), to make check_process() api calls. For check_proceess, you can pass in the process_id, and check against the db. For interval, you can set it to 5 seconds. So that every 5 seconds the call is made.
You can read the response of those API calls and do your processing.
An HTTP request consists of a request and a response. There's no direct way to get status updates beyond the onprogress event which would let you see how much data has been transferred. This is useful for determining how much of the data has been sent to the server, but not so much for how far the server has got with working with that data.
You could store progress in a database and poll a webservice to read the most recent status.
You could also have the server push updates to the client using Websockets for bi-directional communication.
A rough outline for such a system might look like:
Open a Websocket
Send files with Ajax
Get server generated ID back in HTTP response
Pay attention to messages coming over the Websocket that mention that ID
You could also look at doing the whole thing over Websockets (i.e. upload the files that way too). A quick Google search turns up this library for uploading files to a Websocket service hosted on Node.js.
Related
This is part of an experiment I am working on.
Let's say I upload a file eg: .psd (photoshop file) or .sketch (sketch) through the input type file tag, it displays the name of the file and can be downloaded as a .psd / .sketch on click of a button (without data corruption)
How would this be achieved?
Edit 1:
I'm going to add a bit more info as the above was not completely clear.
This is the flow:
User uploads any file
File gets encrypted in the client before sending to a sockets.io server
User on the other end receives this file and is able to decrypt and download.
Note: There is not database connected with the sockets.io. It just listens and responds to whoever connected to the server.
I got the enc/dec part covered. Only thing is uploading and store as ? in a variable so it can be encrypted and doing the opposite on the recepient end (dec and downlodable)
Thanks again in advance :)
I think these are your questions:
How to read a file that was opened/dropped into a <file> element
How to send a file to a server
How to receive a file from a server
When a user opens a file on your file element, you'll be able to use its files property:
for (const file of fileInputEl.files) {
// Do something with file here...
}
Each file implements the Blob interface, which means you can call await file.arrayBuffer() to get an ArrayBuffer, which you can likely use directly in your other library. At a minimum, you can create your byte array from it.
Now, to send data, I strongly recommend that you use HTTP rather than Socket.IO. If you're only sending data one way, there is no need for a Web Socket connection or Socket.IO. If you make a normal HTTP request, you offload all the handling of it to the browser. On the upload end, it can be as simple as:
fetch('https://files.example.com/some-id-here', {
method: 'PUT'
body: file
});
On the receive end, you can simply open a link <a href="https://files.example.com/some-id-here">.
Now, the server part... You say that you want to just pass this file through. You didn't specify at all what you're doing on the server. So, speaking abstractly, when you receive a request for a file, you can just wait and not reply with data until the sending end connects and start uploading. When the sending end sends data, send that data immediately to the receiving end. In fact, you can pipe the request from the sending end to the response on the receiving end.
You'll probably have some initial signalling to choose an ID, so that both ends know where to send/receive from. You can handle this via your usual methods in your chat protocol.
Some other things to consider... WebRTC. There are several off-the-shelf tools for doing this already, where the data can be sent peer-to-peer, saving you some bandwidth. There are some complexities with this, but it might be useful to you.
I am using DRF and AngularJS for my website. I make a simple request like-
$http({
method: "GET",
url: "http://127.0.0.1:8000/api/sendeop",
}).then(function success(response) {
//some processing on the data returned
}, function error(response) {});
Now if data changes in the database, it is not reflected on angularJS as this only makes the request once while it loads. How to watch the api for data change and process it in angularJS?
I'm not quite sure if $watch or $digest solve my problem, but if they do, how do I use them?
From your problem statement I understand the following:
You have a backend DB. The server-side/middleware queries the DB and sends the data to the front-end.
The client/front-end which is using Angular JS is accepting the data from the server side and utilizing it in the screens.
You want to check if the data in the backend DB is changed and if it is changed you want the client to request the data and refresh the screen content.
If the above understanding is correct, then I would suggest the following:
(For capturing changes of data in a single row)
Add a tsChanged(TimeStamp for changed data) column in the table which is storing the data in the DB.
Every Update query will update the tsChanged field of the row.
When the data will be first sent to the client then the tsChanged of the that state is also sent to the client.
Create a service (say updateCheckerService) in the server side that compares the tsChanged values (between the current tsChanged in the DB vs the tsChanged that is sent from the client side). A light weight Select query will do.
Using $interval hit the updateCheckerService by passing the tsChanged thats already present in the client side.
If the server responds true (means the data is changed), then call the data load service again.
(For capturing the changes of the data in multiple rows)
Add a tsChanged column in the table which is storing the data in the DB.
Every Update query will update the tsChanged field of the row.
When the data will be first sent to the client then the highest value of tsChanged is sent to the client.
Create a service (say updateCheckerService) in the server side that compares the tsChanged values (between the current highest tsChanged in the DB vs the tsChanged that is sent from the client side). A light weight Select query will do.
Using $interval hit the updateCheckerService by passing the tsChanged thats already present in the client side.
If the server responds true (means the data is changed), then call the data load service again.
Advantages:
Since the Client side data loading service is expensive and heavy, its best to call only when you are sure that the data is changed in the DB.
In order to be sure that the data is changed, we are using a light weight service, thus effectively making the process lighter.
Hope it helps!
wel you got it all wrong, angular js is client side scripting language only,
if you want to track the changes in database and want to show updated data.
you will need to take help any of middleware (i.e socket.io) or
you can use webworkers for ajax poling which will make continous http call to your server in every definite interval or
you can use any third party (i.e. firebase or PubNub) library.
You can implement a notification system (see Amazon SQS or SNS), or, as stated by #str, you can implement a real-time call (see ajax long polling, websockets).
I have an AJAX call that is running a long PHP script where it has 20 different results, I would like to show when each step in the script is done.
Like so 1/20 done, 2/20 done, 3/20 done.
Latest 29-12-2015 03:17.
I tried to create the JSON file as so (jsonFileName_uniqueTimeStampHere.json) by PHP, but the time taken to create the file with PHP, result in a 404 file not found error!
Because when the AJAX call is running it comes to the progress call before the file has been created, I know I can't create the file with JavaScript but is there anyway to create.
The file before the success callback from jQuery AJAX?
What would be the best way to show progress information while AJAX call is running.
The way I have it now, I have a JSON file saved on the server that gets updated with the latest state that has completed, but if multiple users is running the same script the JSON file with the state gets overwritten.
Should I save the state of each progress in DB and then receive it with multiple calls to a PHP method that get state that has been completed?
Should I keep using the current method I use and add a userID to the JSON file so it is unique on each call to the file?
How should I go about doing it the same way as Seositecheckup?
What is the best way to make a progress with AJAX and PHP?
Please tell me if you need any more information.
I have looked around and don't feel like the info or half of info, there is to find online has been enough to do this myself.
I would like to use jQuery AJAX and not XMLHttpRequest, I'm looking for something similar to seositecheckup.com, when you scan a page you can see the state update on each completed function in the console and is done with different AJAX calls. How is that possible?
Should I forget about using jQuery and keep focus on plain JavaScript instead?
Right now I have a setup with jQuery that works the problem is, that I use a JSON file to get the result from and it gets overwritten when multiple users request the same script, is it possible to store the state in db instead and receive it from there with some unique identifier?
In the future I would like to make it possible to put the script into a queue that could be run and when the script ends it should send an e-mail to the user.
The HTTP way of handling requests that may take a long time is for requests to return a 202 and the body of the response should contain the URL where the user can query for the result.
#Request
POST /some/entitities
...
#Response
HTTP/1.0 202 Accepted
/jobs/{jobId}
The user can then poll /jobs/{jobId} which can return a number to represent progress. Do you have to use this? No, but if you do, others developers can immediately recognize what is going on.
Even if you don't use the approach I recommend, you will also have to keep track of job progress in your database and have a separate AJAX call to find out the current progress.
I'm trying to make auto-reload counter (for ex.: Messages [num]).
So, I just in setTimeout(); getting JSON code from test_ajax.php. I think it's not correctly..
Can I send info by server (I think not, but suddenly I something don't know..)?
Why I think that's not correctly: because when I'm looking in my chrome network log (F12 -> network tab), I see a lot of requests (to test_ajax.php), but when, I'm visiting vk.com (great example for ajax) or facebook.com, I don't see any requests while something will not change.
So, what's incorrectly in my solution (or what's bad..)?
UPD: Sorry, vk.com sending requests to q%NUM%.queue.vk.com every 25s, but until 25s last request's status is "Pending". When someone, for example, sending me a message it immediately display it. And request has parameter "wait" which equals 25. This delay in requests doing on server side.. But how?
Ajax counter can be done in easy just include below files
index.html
counter.php (ajax file)
necessary images
JS file (for jquery paging call)
download link: https://docs.google.com/open?id=0B5dn0M5-kgfDcE0tOVBPMkg2bHc
What you are looking for is called COMET (also sometimes called Reverse AJAX) techniques.
Doing what you want to do, e.g. regular polls, is one way of doing it.
A lot is actually happening on the server side; to avoid recreating new connections on every poll, some servlet containers like Jetty started to implement techniques like Continuation which basically maintain a two-way connection open.
In the Java world, with Servlet 3, you have asynchronous calls as part of the specs.
I am trying to develop an Interactive chat application using AppWeb open source web Server.
I need to have some machanism that will enable Web server to send updated messages to client, so that when remote usre sends messages that will get updated automaticaly at the client end.
There are some methods to do this using HTML5 Web Sockets and Server sent events.
But we need to implement it in HTML and JavaScript only not HTML5.
So I need some pooling machanism that will keep pooling my Web Server for New events.
So how should I write pooling machanism in Javascript using Sockets.
How it should be implemented at server end?
Thanks!
there are already some examples out there... depending on the server-side, you could go for java-hello-world or php-hello-world or ...
if you can't use websocket, you have to can go the old way, create an interval by window.setInterval and pull data from the server with eg. $.ajax(). i don't know any other alternative to bidirectional connection (websocket)... see kayahrs answer
as you've asked for it:
$.ajax() is the jQuery way to do xhr. basically, it fires an asynchronous request to the server, which returns xml or json or text or ... (whatever). when this request comes back, the supported eventHandler gets fired and you can react to the response. you could also use the plain xhr, but it's a bit awkward to handle the original xhr.
jQuery supports some shorthand overloads for $.ajax(), eg. $.getJSON(), $.get(), ...
sample implementation:
$.get("test.cgi", function(data){
alert("Data Loaded: " + data);
});
There is another technique for sending messages from the server to the client. You have to use an iframe for this which connects to a PHP script (Or whatever technique you are using on the server side) which does not close the connection. The PHP script then sends JavaScript messages whenever the client must be informed about something. After each message the server flushes the output stream to enforce that the data really finds its way to the client and is not cached by some output buffer. Here is a small example code of the PHP script loaded in the iframe (not tested and not complete, just to show the basics):
<html>
<body>
<script type="text/javascript">
function receiveMsg(data)
{
// Do something with the data, for example send it to some function
// in the parent frame (Where your chat application lives)
}
<?php
while (true) // You may also implement some abort state which should
// be checked here
{
$data = waitForData(); // This is your magic function on the server
// which waits for data to be send to the client
echo "receiveMsg('" . $data . "');"; // Let's say data is just a string.
// You may want to use JSON instead
flush();
}
?>
</script>
</body>
</html>
Advantage of this method is that it doesn't rely on polling. So you don't have to send requests to the server every x seconds. And when you do things right on the server side then messages sent by one user are received as fast as possible by the other users and not x seconds later. Disadvantage is that you have a permanent HTTP connection for each chat user. But this may need less resources on the server then having dozens of complete HTTP requests per minute per chat user.