What is the magic behind Lightstreamer? - javascript

I'm gonna develop a framework for comet programming, and I can't use Web Sockets, or Server-Sent Events (because browser support really sucks). So, I need to keep the HTTP connection alive, and send chunked data back to the client.
However, problems show themselves as you get into the work:
Using XMLHttpRequest is not possible, due to the fact that IE doesn't give you xhr.responseText while the xhr.readyState is 3.
A hidden iframe can't be useful, because browser shows the loader while I send data back to the client.
I tried to send a JavaScript file back to the client, sending function execution commands each time, but browsers won't execute JavaScript till it's completely loaded.
However, when I look at Lightstreamer demo page, I see that it sends a JavaScript file back to the client little by little and in each step, it sends a call to the function and that function simply gets executed (I can't do this part). It seems that Lightstreamer uses AJAX, since the request simply shows up in Firebug's console tab, but it works like a charm in IE too.
I tried to use every HTTP header field they've set on their request, and no result. I also tried to use HTTP Post instead of HTTP Get, but still got no result.
I've read almost over 20 articles on how to implement comet, but none of'em appear to solve problems I have:
How to make it cross-browser?
How to get notified when new data is arrived from server (what event should I hook into)?
How to make my page appear as completely loaded to the user (how to implement it, so that browser doesn't show loading activity)?
Can anyone please help? I think there should be a very little tip or trick that I don't know here to glue all the concepts together. Does anyone know what lightstreamer do to overcome these problems?

SockJS author here.
How to make it cross-browser?
This is hard, expect to spend a few months on getting streaming transports on opera and IE.
How to get notified when new data is arrived from server (what event should I hook into)?
There are various techniques, depending on a particular browser. For a good intro take a look at different fallback protocols supported by Socket.IO and SockJS.
How to make my page appear as completely loaded to the user (how to implement it, so that browser doesn't show loading activity)?
Again, there are browser-specific tricks. One is to delay loading AJAX after onload event. Other is to bind-and-unbind an iframe from DOM. ETC. If you still feel interested read SockJS or Socket.io code.
Can anyone please help? I think there should be a very little tip or trick that I don't know here to glue all the concepts together. Does anyone know what lightstreamer do to overcome these problems?
Basically, unless you have a very strong reason to, don't reinvent the wheel. Use SockJS, Socket.io, faye, or any other of dozens projects that do solve this problem already.

The methods you want is the streaming.
How to make it cross-browser?
Considering most browsers, there is no consistent way. You have to choose a proper transport according to the browser. Even worse, you have to rely on the browser sniffing to recognize which browser is being used, and the feature detection counts for nothing about this. You can use XDomainRequest for IE8+, XMLHttpRequest for non-IE and Iframe for IE 6+. Avoid iframe transport if possible.
How to get notified when new data is arrived from server (what event should I hook into)?
This varies according to the transport being used. For example, XDomainRequest fires progress event, XMLHttpRequest fires readystatechange event when chunk is arrived except Opera and IE.
How to make my page appear as completely loaded to the user (how to implement it, so that browser doesn't show loading activity)?
I don't know this issue with iframe, but still occurs in WebKit based browsers such as Chrome and Safari with XMLHttpRequest. The only way to avoid this is to connect after the onload event of window, but, in case of Safari, this does not work.
There are some issues you have to consider besides the above questions.
Event-driven server - The server should be able to process asynchronously.
Transport requirements - The server behaves differently for required transport.
Stream format - If the server is going to send big message or multiple messages in a single chunk, a single chunk does not mean a single data. It could be fragment of a single data or concatenation of multiple data. To recognize what is data, the response should be formatted.
Error handling - Iframe transport does not provide any evidence for disconnection.
...
Last but not least, to implement streaming is pretty tiresome than it looks unlike with long polling. I recommend you use solid framework for doing that such as socketio, sockjs and jquery socket which I've created and managed.
Good luck.

but browsers won't execute JavaScript till it's completely loaded.
Have you tried sending back code wrapped in <script> tags? For example, instead of:
<script type="text/javascript">
f(...data1...);
f(...data2...);
try
<script type="text/javascript">f(...data1...);</script>
<script type="text/javascript">f(...data2...);</script>

The best option in your case would be to use JSONP + Long Pulling on server side. You just have to remember to reconnect any time connection drops (times out) or you receive response.
Example code in jquery:
function myJSONP(){
$.getScript(url, {
success: function(){
myJSONP(); //re-connect
},
failure: function(){
myJSONP(); //re-connect
}
})
}
Obviously your response from server has to be javascript code that will call on of your global functions.
Alternatively you can use some jquery JSONP plugin.
Or take a look on this project http://www.meteor.com/ (really cool, but didn't try it)

Related

How to communicate both ways between embedded browser and webside

I have a C++ project for windows, using MiniBlink as embedded browser. (MiniBlink is a smaller Blink, which is close to chromium). I use this embedded browser to show responsive and nice looking dialogs with Quasar.js (wrapper for vue.js).
Problem:
Mostly a browser is just the passive backend. In my case, both the backend (project with embedded browser) and the frontend (dialog) are active and thus I need some communication. At the moment I use a local server to catch HTTP send from the frontend to the backend.
But is there a way to communicate from the backend to the frontend? At the moment I could only think about catching cookies or using a permanent loop in JS to send http queries to check for a possible response.
And is there no other way to send information to a backend? Everything is local, I dont need nor really want to send it into the network.
Thanks!
Idea 1: Use a local temp file to save on one side and read on other (can be also used both way)
Idea 2 (similar to question author solution): Local server with both side communication (GET/POST request into one side, text/json other way around)
Idea 3: Use launch parameter to pass though data directly into links for example: instead of using browserprocess.exe file.html, use browserprocess.exe file.html#showsomething
There are also other ways which like catching for example: checking window title of process with certain binary name from running tasks by other side; we didin't get good enough info about your background becouse you coud either use it in same process or other process, if thats same process you coud also just directly use variables both ways directly in code of miniblink and do action when they meet if statement
As CertainPerformance added as a comment, WebSockets might be the best way to go.
If one does not like to implement a websocket server, because a http server is already running, long polling requests might be the best workaround to simulate this behaviour.
Long polling: The client sends a request, which stays open as long as possible. If the server needs to communicate, it can use the open request to send its own "request" via response. It is a bit hacky, but essentially the idea behind websockets.
Mozilla has a nice article to help with websockets:
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers
If you (like me) use vuejs or quasar, you might want have a look at vue-native-websocket.
https://github.com/nathantsoi/vue-native-websocket
Good luck

Chrome Extension webRequest synchronous behavior for async calls

I've created a test web server that I'm using to act as a 'web filter' of sorts. I'm trying to create an extension that uses the webRequest API to make sure that my web server allows or blocks all incoming URL's.
To do this, I'm making an AJAX call from within webRequest to my web server and I'd like to use the response to determine whether to block or allow the specified URL. The problem is, the webRequest method is async, and AJAX calls are async, so I can't wait reliably wait for a response from my server.
I also can't store all blocked / allowed URL's in localStorage, because there could potentially be hundreds of thousands. I've tried using jQuery's async: false property in it's ajax implementation, but that makes the browser almost completely unusable when hundreds of requests are happening at the same time. Anyone have any ideas as to how I might be able to work around this?
EDIT: I know similar questions to this have been asked before, but there haven't been any viable solutions to this problem that I've seen.
I see only two good choices:
make that site a webproxy
use unlimitedStorage permission and store the urls in WebSQL database (it's also the fastest). Despite the general concern that it may be deprecated in Chrome after W3C stopped developing the specification in favor of IndexedDB I don't think it'll happen any time soon because all the other available storage options are either [much] slower or less functional.

AJAX listener rather than polling?

I created my own chat webserver in python and wanted to know instead of AJAX making a call to the server every second (JS below). I can modify my server so when ever it updates the chat.html file it pushes it out to all the clients. Is there a way using javascript to have it listen for any received data rather then polling?
<script>
// Request the AJAX update the chat window every second
setInterval(function(){loadChat()},1000);
function loadChat()
{
var xmlhttp;
if (window.XMLHttpRequest)
{
xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange=function()
{
if (xmlhttp.readyState==4 && xmlhttp.status==200)
{
document.getElementById("chatWindow").innerHTML=xmlhttp.responseText;
}
}
xmlhttp.open("GET","chat.html",true);
xmlhttp.send(null);
}
}
</script>
Yes, there certainly is. There are (at least) four techniques you can use:
WebSockets. This is the most obvious solution. It allows you to send and receive messages on-demand without need for polling. It may, however, be somewhat more difficult to implement on the server side, as it is not plain HTTP. Also, old browsers don't support WebSockets.
Server-Sent Events. This is less desirable, but will still work for you. With it, you can receive mes­sages from the server without polling. It is also easier to implement on the server as it's just plain HTTP except that the connection doesn't close. It, too, is not supported in some older web brow­sers, but it's more supported than WebSockets and is pretty easy to shim.
COMET. This is basically an improvement of the thing below. Basically, you have an iframe that's hidden offscreen. Whenever an event comes in, you send (and flush!) a script tag, but don't close the connection. After a timeout, close the connection. Then refresh the iframe. This is also pretty easy to implement, is plain HTTP, and needs no special browser support. However, browser time­outs vary and this is somewhat inelegant.
Keeping the connection open until an event comes. This is probably the least desirable. Simply don't send a response until an event comes in. When an event comes in or a timeout occurs, send the response. When the client receives the response, reconnect. This is also somewhat inelegant but it works.
What you are talking about is pushing from the server to the client which is comet you can use such way
But this can be done perfectly with node.js and WebSockets in HTML5 but it's not supported in all browsers check this out
And you can make another thing
Send a request from your JS and make it alive for 1 minute and make an infinite loop that sleeps for some seconds and if it found anything while that return it to the client else make a new request this technique is called HeartBeat
I hope this can help
Simply, in most cases you will not be able to.. However, you can use HTML5 Websockets, though they are commonly unsupported.
SockJS, uses a variety of methods to do callbacks in JavaScript, one of which is the blocking query, another WebSockets. I would highly recommend it if you are trying to do this sort of thing.
You can do this using websockets. Unfortunately not all browsers currently support them.
More info on websockets here.

Using a client's IP to get content

I'm a bit embarrassed here because I am trying to get content remotely, by using the client's browser and not the server. But I have specifications which make it look impossible to me, I literally spent all day on it with no success.
The data I need to fetch is on a distant server.
I don't own this server (I can't do any modification to it).
It's a string, and I need to get it and pass it to PHP.
It must be the client's (user browsing the website) browser that actually gets the data (it needs to be it's IP, and not the servers).
And, with the cross-domain policy I don't seem to be able to get around it. I already knew about it, still tried a simple Ajax query, which failed. Then I though 'why not use iFrames', but the same limitation seems to apply to them too. I then read about using YQL (http://developer.yahoo.com/yql/) but I noticed the server I was trying to reach blocked YQL's user-agent making it impossible to use this technique.
So, that's all I could think about or find. But I can't believe it's not possible to achieve such a thing, that doesn't even look hard...
Oh, and my Javascript knowledge is very basic, this mustn't help either.
This is one reason that the same-origin policy exists. You're trying to have your webpage access data on a different server, without the user knowing, and without having "permission" from the other server to do so.
Without establishing a two-way trust system (ie modifying the 'other' server), I believe this is not possible.
Even with new xhr and crossdomain support, two-way trust is still required for the communication to work.
You could consider a fat-client approach, or try #selbie suggestion and require manual user interaction.
The same origin policy prevents document or script loaded from one
origin from getting or setting properties of a document from a different
origin.
-- From http://www.mozilla.org/projects/security/components/same-origin.html
Now if you wish to do some hackery to get it... visit this site
Note: I have never tried any of the methods on the aforementioned site and cannot guarantee their success
I can only see a really ugly solution: iFrames. This article contains some good informations to start with.
You could do it with flash application:
flash with a crossdomain.xml file (won't help though since you don't control the other server)
On new browsers there is CORS - requires Access-Control-Allow-Origin header set on the server side.
You can also try to use JSONP (but I think that won't work since you don't own the other server).
I think you need to bite the bullet and find some other way to get the content (on the server side for example).

call jquery function from server

In asp.net, How can i trigger client side jquery event from server. I want to implement it in my chat section... current the chat seems to work fine... but it has one problem... i have to send a request every 5 seconds from client's browser to the his chat history.. which i feel is not a good idea...
can anyone provide any solution for my problem
By default you need to do it with polling from the clients browser. That's how ajax works out of the box. There's a technique called Comet which is a push model.
You can try to use a jQuery plugin like this: http://plugins.jquery.com/project/jQuery-Comet-Push-API.
In the (near) future you could make use of websockets as well in HTML5. Here's information taken from http://channel9.msdn.com/Events/MIX/MIX11/HTM10.
WebSockets, an emerging specification
being standardized by W3C and IETF,
will enable web browsers as well as
client applications to open a
bi-directional, full-duplex
communication channel with a remote
host.
Be sure to check out that MIX11 video!
Although, I am not that skilled in ASP.Net, this problem can be solved in another way.
Have the server return the name of function you need to execute, then call it in the callback function?
Kinda like this (It is just a typo but)
$.post("yourpage.aspx",
{
d1: "v1"
},
function(data) {
//now the data will hold the name of the function
window[data]();
}
);
Now you can wrap the above code, in the another function, and set up a timer, to check for the response regularly, and execute the function if a condition is matched.
P.S. I have skipped the part, where the scripts check if the condition is matched.
You are doing the best that you can do without getting into something like HTML5 WebSockets which are not really ready for regular usage yet.
Two options:
(conservative option) Accept that you will be polling the server and optimize the way you do the polling to keep the impact low.
(edgy option) Try out some of the new libraries that emulate Websockets.
Welcome to the world of HTTP.
Thats how the browsers work. Client sends a request to the Server and Server responds. There is no other way (for now) to go the other way. It's simple and that's one of the reasons HTTP protocol was so popular.
But now things are changing and HTML5 support web sockets. If you developing for a HTML5 supporting device (latest versions of all browsers + iPhone/iPad support sockets) then websockets is the way to go.
On the other side, you can go with Comet polling (as mentioned by XIII). Basically it client sends a request to the server and waits until server has anything to respond to. I am not too sure what is a good way of implementing it in ASP.NET (as I think there is a limit of concurrent connections) but it's a pretty useful technique in Nodejs.

Categories