Dealing with incremental server response in AJAX (in JavaScript) - javascript

I have AJAX app which updates page based on server response. The command that AJAX server response is based on takes long time to generate full response, but it sends partial information as soon as it is calculated. This partial response / partial info is send in "burst", and time and size of each burst is unpredictable. CGI script (in Perl) that streams command output to web browser (to AJAX request) has autoflush turned on.
The server response is based on output of external command. While 'time cmd > /dev/null" gives around 10.0 seconds on average, 'time cmd | head > /dev/null' gives less than 0.1 seconds (for example data). All data is result of single call to this external command.
The situation looks like the following (ASCII-art diagram follows):
client | | server
--------- ---------
request -\
\
\
\
\->
/- response
/ .
/ .
/ /- .
<-/ / .
/ .
/ /- [end]
<-/ /
/
/
<-/
I have a few questions about this problem.
Note: server side is done as CGI script in Perl, and I would prefer to see (also) solution without using JavaScript library / framework like jQuery.
The output of command used by server side of AJAX app is line based. Each group of lines, beginning with one defined kind of line, and ending with other kind of line, consist of independend and unchangeable data. Should I just stream response from a command as 'text/plain' and do processing in JavaScript on client side, or should I pre-process data on server, and send whole chunks of data as JSON using 'application/json' mimetype?
It might happen that large chunk of data send at once by server is followed soon by another chunk of data. How to deal with situation when onreadystatechange handler is invoked while previous invocation didn't finished work? Should I use global variable as semaphore, or pass state variable as handler parameter (well, use xhr.onreadystatechange = function() { handleRequest(xhr, state) })?
Should I use 'text/plain' or 'application/json', or perhaps 'multipart/x0mixed-replace' for this? Note: this app should work in (alomst) any browser.
How to deal with web browser (JavaScript engines) which invoke onReadyStateChange only after receiving complete response (so I don't see xhr.readyState == 3 i.e. partial response more than once)? Well, beside using some JavaScript framework.
How to deal with incomplete responses (which in this situation mean incomplete lines).
Should I send end of response marker, or rely on counter to check if we received all data, or can I simply rely on detecting xhr.readyState == 4?
Even partial response would be helpful.

I think Comet is part of what you need for your solution. You can additionally (if I got that right) checkout the Bayeux Protocol which was implemented by the Dojo Foundation.
The whole thing is still very new (though some of it might be possible with the first HTML5 implementations).
Besides that you would probably have to implement the polling approach. The other question is, how much data the client side JavaScript interpreter can handle. Is there any possibility for you to somehow "page" your data so that you won't have the problem of request handling still processing while aother response comes in already?

I think the client side could be designed to process data in chunks, sending repeated AJAX requests until all the data had been supplied. This assumes that each chunk could be sent in a timely manner (without client-side timeout problems) even if the entire response was large; and this design is probably simpler than developing checks for specific partial-response statuses which might vary from browser to browser.
In terms of whether you do processing server-side or client-side, that would depend on e.g. how many concurrent clients you would need to service, and whether you could use caching for any of the responses; in some scenarios where there are lots of clients, it's better to parcel out some of the processing load to them (as long as they can handle it, of course).

Related

Jmeter logging for API URL+Request+ResponseCode+ResponseBody in csv

I like to save the following parameters in a CSV file in single line for HTTP REST call in Jmeter.
Endpoint URL, request body, request header, response code, response body . could someone help with JS code or groovy code. Thanks in advance!
I don't think you will be able to save request and response "in single line" because:
for sure it will have line breaks
most probably it will have delimiters (i.e. CSV stands for "comma-separated values" and if your response will contain a comma - it will create another "column")
If you're going to use JSR223 Test Elements for this it can be done, but the correct work is guaranteed only for a one thread (virtual user), if you have more than 1 user concurrently writing the data into the same file you'll face the race condition resulting in data corruption or loss
So I would recommend considering using Flexible File Writer which provides full freedom to choose what to store, where to store and it "flushes" the metrics periodically in a thread-safe manner.

AJAX calls - where does logic go?

This might not even be an AngularJS question and could just be an AJAX question. I'm new to the "developer" side of the frontend so bear with me.
When making an AJAX call to fetch JSON data, where does the logic behind what data is returned and viewed fall? In my mind, there would be a couple of possibilities and I want to understand which is the proper choice and why.
Let's use an example of searching and playing a Youtube video.
The logic could fall to the backend (controller), where the JSON is rendered based on some logic to give you a JSON file with exactly the right data. i.e. you search "cat videos" and when making an AJAX call, the JSON file you pull has been rendered to be only cat videos.
The opposite end would be that the Angular controller has the logic. This would imply that all data is called (cat videos along with everything else... music videos, funny videos, tutorials, and so on) and then sorted through on the client side. This, to me anyway, would be more inefficient / slow for the client, so doesn't seem to make sense. I suppose still might do some filtering of the data on the client side though. So, maybe a search for "cat videos" wouldn't return ALL videos, but definitely all cat videos and any filtering based on, say, # of views, video length, and so on would be done on the client side (vs. calling the database again for a "new" set of videos).
Not sure if this is accurate, but could you have logic in your factory to return only a portion of the data? However, I believe the entire JSON file would need to be rendered, but only portions would be returned. I guess depending on where the JSON file renders (i.e. backend or frontend) this could be similar to either option #1 or #2.
Or maybe I'm misunderstanding things entirely and the way this works is entirely different!
I'm basically looking to figure out how the scenarios of 1. user searches a term and results are shown, 2. user clicks a search result and now more detailed data of the result is on it's own page. And how this ends up working out. I'm looking for help with AngularJS, but I think this ultimately an AJAX question (single page app or not) more than anything.
There's a few critical concepts you may be confused about.
First. JSON is not a file, it's a format, more simply, a type of string. It's really good for collapsing arrays and storing address-value pairs, so a lot of data flies around in that format. Strictly speaking, they are JSON objects, but they're a lot like strings and arrays. It looks like this, if I remember correctly:
{ "name" : "john doe", "pet" : "dog", "hobby" : "parasailing" }
Second, AJAX is a request to the server, made from the client (the browser) after the original page has loaded. That is, you type in 'youtube.com' and the youtube server receives the request and sends a big pile of HTML back to your browser.
You watch your video, make a rating, and the browser doesn't reload the page but instead sends a separate request back to the youtube server with your rating. There's a parameter in the request that says "send it to ratingspage.php". This request is AJAX.
Now, the logic happens (server-side). ratingspage.php receives your request. It contacts the databases, updates or fails or whatever, and sends back a response to your browser. This response may be in JSON format.
Finally, your browser parses that response and updates the DOM (HTML document) as appropriate.
At this point, it's worth noting that if the logic happened on the client-side (browser), the user could see it - this is a security problem! So, sensitive operations should be carried out on the server side, where you can test and sanitize the request data.
In summary:
AJAX is separate from the initial load event.
Information sent is gathered from the client browser
Logic happens server-side
Logic can use whatever language the server understands (PHP, Java, Ruby, etc.)
Information is returned to the browser
Information sent and received may use JSON format
Everything client-side happens in Javascript
Here's a bare-bones ajax request (done in Javascript) with comments. This has no exception handling, state checking, or anything so don't use it! But it gives you the basic idea.
// Make a new request
var req = new XMLHttpRequest(); }
// Requests will have various states depending on whether they're processing,
// finished, error, etc. We'll assume everything went OK.
// We need to establish a handler before the request
// is sent so it knows what to do.
req.onreadystatechange = function() {
// Here's what the server sent back to the browser
alert(req.responseText);
}
// Using the GET method, set up some parameters
req.open("GET", "somelogicpage.php?blah=blee&bloo=bar", true);
// Send the request
req.send(null);
Server-side, somelogicpage.php may look like:
<?php
if ($_GET['blah'] != 'blee']) {
// This is the response text!
echo "Sorry, you need to blee when you blah.";
}
else {
// (or this)
echo "I'm ecstatic to report nothing is wrong!";
}
?>
Your alert(req.responseText) from the handler function in the previous Javascript will say whatever the PHP has dumped out.
So yes, you can use whatever portion of the request you like, and return whatever you like. Javascript kicks bleep.

How to validate remote data in a client?

I'm designing a client-server system, and i need to understand how to check if the client's data is correct when they send operations and requests. In this particular case, i've got a browser and a javascript client that gets data from longpolling and updates a series of objects wich get binded to html elements, pretty much MVVM.
The steps are something like this:
start polling
get full data
convert the json into a javascript object
update every html object tied to the data
The user can fire an event at any time and works with the latest updated local model.
user fires event
event + full data(all objects converted to json) is sent
Problems are: It's very rough and possibly slow, heavy on the client and the server.
My objectives are to reduce the data transfer to a minimum, and avoid client side corruption/attacks.
How should i go about this?
My objectives are to reduce the data transfer to a minimum
Send only the data that's changed, but the highest cost in AJAX is the request, so unless you are sending a lot of data, it may not make any noticeable difference.
and avoid client side corruption/attacks
Impossible. Your code is running in a browser, the user can do whatever they want.
My objectives are to reduce the data transfer to a minimum
Some things to try:
Reduce the number or frequency of client events that send an update
Send only what data has changed
Compress the data you send
bundle several events into a single request
and avoid client side corruption/attacks.
To avoid attacks, you need to validate all input on the server. You should write your validator without knowledge of the client. You can assume nothing about what combination of data you can get--instead you should assume that someone is hand-crafting requests with a text editor and sending them with CURL.
To avoid corruption (really a "lost update"), use conditional PUTs or POSTs with the if-none-match or if-unmodified-since headers.

How to make auto-updating (ajax) counter correctly? Or how to disable network log?

I'm trying to make auto-reload counter (for ex.: Messages [num]).
So, I just in setTimeout(); getting JSON code from test_ajax.php. I think it's not correctly..
Can I send info by server (I think not, but suddenly I something don't know..)?
Why I think that's not correctly: because when I'm looking in my chrome network log (F12 -> network tab), I see a lot of requests (to test_ajax.php), but when, I'm visiting vk.com (great example for ajax) or facebook.com, I don't see any requests while something will not change.
So, what's incorrectly in my solution (or what's bad..)?
UPD: Sorry, vk.com sending requests to q%NUM%.queue.vk.com every 25s, but until 25s last request's status is "Pending". When someone, for example, sending me a message it immediately display it. And request has parameter "wait" which equals 25. This delay in requests doing on server side.. But how?
Ajax counter can be done in easy just include below files
index.html
counter.php (ajax file)
necessary images
JS file (for jquery paging call)
download link: https://docs.google.com/open?id=0B5dn0M5-kgfDcE0tOVBPMkg2bHc
What you are looking for is called COMET (also sometimes called Reverse AJAX) techniques.
Doing what you want to do, e.g. regular polls, is one way of doing it.
A lot is actually happening on the server side; to avoid recreating new connections on every poll, some servlet containers like Jetty started to implement techniques like Continuation which basically maintain a two-way connection open.
In the Java world, with Servlet 3, you have asynchronous calls as part of the specs.

Asynchronous Servlet Client, Server push

Hello guys i want to process some server pushes. I have an asynchronous servlet processing something, pushing it to the client and then it processes something else and pushes it again to the client (same connection). The servet just returns data (Json in this case, but that does not really mather) nothing more.
So my problem is the client. How do i build a client for that? If i make an ajax request with JQuery for example how can i react on the data that comes after the first response?
To make it more clear what i want here is a comparison : With websockets i have the method onmesssage.
websocket ws = new WebSocket("ws://myserver.com");
ws.onmessage = function(event)
{
var x = event.data
.... // some other code here
}
So all i want is a onmessage Method :). I guess it is not that easy as it is with websockets but maybe someone has an idea.
Greetings Aleks
You can have your server generate a response which is loaded into an hidden iframe by the client. The generated response would contain occasional JavaScript statements which call to the "outside" (the containing document). You can get your hands on the containing document using parent.
But please not that this technique is pretty hackish (at least it seems to me). You might want to re-consider just using the XMLHttpRequest, especially because it gives you simple and robust error handling. You can just do more requests (instead of appending to an "old" response on the server side). This will probably introduce additional lag, but that iframe trick is really troublesome in practice.

Categories