Of course I am aware of Ajax, but the problem with Ajax is that the browser should poll the server frequently to find whether there is new data. This increases server load.
Is there any better method (even using Ajax) other than polling the server frequently?
Yes, what you're looking for is COMET http://en.wikipedia.org/wiki/Comet_(programming). Other good Google terms to search for are AJAX-push and reverse-ajax.
Yes, it's called Reverse Ajax or Comet. Comet is basically an umbrella term for different ways of opening long-lived HTTP requests in order to push data in real-time to a web browser. I'd recommend StreamHub Push Server, they have some cool demos and it's much easier to get started with than any of the other servers. Check out the Getting Started with Comet and StreamHub Tutorial for a quick intro. You can use the Community Edition which is available to download for free but is limited to 20 concurrent users. The commercial version is well worth it for the support alone plus you get SSL and Desktop .NET & Java client adapters. Help is available via the Google Group, there's a good bunch of tutorials on the net and there's a GWT Comet adapter too.
Nowadays you should use WebSockets.
This is 2011 standard that allows to initiate connections with HTTP and then upgrade them to two-directional client-server message-based communication.
You can easily initiate the connection from javascript:
var ws = new WebSocket("ws://your.domain.com/somePathIfYouNeed?args=any");
ws.onmessage = function (evt)
{
var message = evt.data;
//decode message (with JSON or something) and do the needed
};
The sever-side handling depend on your tenchnology stack.
Look into Comet (a spoof on the fact that Ajax is a cleaning agent and so is Comet) which is basically "reverse Ajax." Be aware that this requires a long-lived server connection for each user to receive notifications so be aware of the performance implications when writing your app.
http://en.wikipedia.org/wiki/Comet_(programming)
Comet is definitely what you want. Depending on your language/framework requirements, there are different server libraries available. For example, WebSync is an IIS-integrated comet server for ASP.NET/C#/IIS developers, and there are a bunch of other standalone servers as well if you need tighter integration with other languages.
I would strongly suggest to invest some time on Comet, but I dont know an actual implementation or library you could use.
For an sort of "callcenter control panel" of a web app that involved updating agent and call-queue status for a live Callcenter we developed an in-house solution that works, but is far away from a library you could use.
What we did was to implement a small service on the server that talks to the phone-system, waits for new events and maintains a photograph of the situation. This service provides a small webserver.
Our web-clients connects over HTTP to this webserver and ask for the last photo (coded in XML), displays it and then goes again, asking for the new photo. The webserver at this point can:
Return the new photo, if there is one
Block the client for some seconds (30 in our setup) waiting for some event to ocurr and change the photograph. If no event was generated at that point, it returns the same photo, only to allow the connection to stay alive and not timeout the client.
This way, when clients polls, it get a response in 0 to 30 seconds max. If a new event was already generated it gets it immediately), otherwise it blocks until new event is generated.
It's basically polling, but it somewhat smart polling to not overheat the webserver. If Comet is not your answer, I'm sure this could be implemented using the same idea but using more extensively AJAX or coding in JSON for better results. This was designed pre-AJAX era, so there are lots of room for improvement.
If someone can provide a actual lightweight implementation of this, great!
An interesting alternative to Comet is to use sockets in Flash.
Yet another, standard, way is SSE (Server-Sent Events, also known as EventSource, after the JavaScript object).
Comet was actually coined by Alex Russell from Dojo Toolkit ( http://www.dojotoolkit.org ). Here is a link to more infomration http://cometdproject.dojotoolkit.org/
There are other methods. Not sure if they are "better" in your situation. You could have a Java applet that connects to the server on page load and waits for stuff to be sent by the server. It would be a quite a bit slower on start-up, but would allow the browser to receive data from the server on an infrequent basis, without polling.
You can use a Flash/Flex application on the client with BlazeDS or LiveCycle on the server side. Data can be pushed to the client using an RTMP connection. Be aware that RTMP uses a non standard port. But you can easily fall back to polling if the port is blocked.
It's possible to achive what you're aiming at through the use of persistent http connections.
Check out the Comet article over at wikipedia, that's a good place to start.
You're not providing much info but if you're looking at building some kind of event-driven site (a'la digg spy) or something along the lines of that you'll probably be looking at implementing a hidden IFRAME that connects to a url where the connection never closes and then you'll push script-tags from the server to the client in order to perform the updates.
Might be worth checking out Meteor Server which is a web server designed for COMET. Nice demo and it also is used by twitterfall.
Once a connection is opened to the server it can be kept open and the server can Push content a long while ago I did with using multipart/x-mixed-replace but this didn't work in IE.
I think you can do clever stuff with polling that makes it work more like push by not sending content unchanged headers but leaving the connection open but I've never done this.
You could try out our Comet Component - though it's extremely experimental...!
please check this library https://github.com/SignalR/SignalR to know how to push data to clients dynamically as it becomes available
You can also look into Java Pushlets if you are using jsp pages.
Might want to look at ReverseHTTP also.
Related
What I am trying to build here looks something like this.
MyClients(Multiple) <------> Server(Me) <------> TheirClients(multiple, live)
MyClient1 (HTML)
Through Server (HTML -> Servlet -> Android)
HisClient1 (uses Android app)(monitored live by MyClient1)
HisClient2 (uses Android app)(monitored live by MyClient1)
...
...
The flow: (Can have multiple MyClients and HisClints online at the same time)
MyClient comes online.
Timely pinging HisClients get connected to the Server.
MyClient gets notified.
MyClient sends request to HisClient through server.
HisClient replies through Server.
MyClient is updated.
On the HisClient end i.e. Android, I'm running TCP Socket() to timely ping the server and stay alive. Cannot use URLConnection as the client have to stay alive all the time if the server requests something. Also I don't want to use any external library, so GCM is not an option.
On the MyClient side, I first tried using PHP but it is inefficient of handling multiple live clients. Then, after some searching I came to know that Servlets are good in such cases. So I started developing a Servlet.
Now the questions are:
Am I going in the right direction? I mean is there anything better for such a scenario? If yes, what and how? I mean what are the APIs and how will it maintain the live traffic?
If no, how do you handle multiple live HisClients with Servlet? I know, here multiple is not a problem but how to keep them live and keep updating MyClient?
Why are you going to spend so much time reinventing the wheel? Maintaining 100 connections with a web socket isn't hard. Maintaining 1,000,000 will be impossible. If you need an event system, use something like Amazon SNS or Pubnub to distribute events. Sure, you'll add a couple of Kbytes or MBytes to your app. But how will you know you can scale your solution?
Maintaining any kind of a connection will be just waiting for a scalability nightmare. Make things a bit less tightly coupled and you'll be much better off.
I am looking at facebook news feed/ticker right now and I am wondering what technology/architecture it uses to pull in data asynchronously when any of my connections make an update. One possibility that I can think of is a javascript setInterval on a function that aggressively polls the server for new data.
I wonder how efficient that is.
Another possible technology that I can think of is something like Comet/NodeJS architecture that pings the client when there is an update on the server. I am not too familiar with this technology.
If I wanted to create something similar to this. What should I be looking into? Is the first approach the preferred way to do this? What technologies are available out there that will allow me to do this?
There are several technologies to achieve this:
polling: the app makes a request every x milliseconds to check for updates
long polling: the app makes a request to the server, but the server only responds when it has new data available (usually if no new data is available in X seconds, an empty response is sent or the connection is killed)
forever frame: a hidden iframe is opened in the page and the request is made for a doc that relies on HTTP 1.1 chunked encoding
XHR streaming: allows successive messages to be sent from the server without requiring a new HTTP request after each response
WebSockets: this is the best option, it keeps the connection alive at all time
Flash WebSockets: if WS are not natively supported by the browser, then you can include a Flash script to enhance that functionality
Usually people use Flash WebSockets or long-polling when WebSockets (the most efficient transport) is not available in the browser.
A perfect example on how to combine many transport techniques and abstract them away is Socket.IO.
Additional resources:
http://en.wikipedia.org/wiki/Push_technology
http://en.wikipedia.org/wiki/Comet_(programming))
http://www.leggetter.co.uk/2011/08/25/what-came-before-websockets.html
Server polling with JavaScript
Is there a difference between long-polling and using Comet
http://techoctave.com/c7/posts/60-simple-long-polling-example-with-javascript-and-jquery
Video discussing different techniques: http://vimeo.com/27771528
The book Even Faster Websites has a full chapter (ch. 8) dedicated to 'Scaling with Comet'.
I could be wrong, but I think that Facebook relies on a "long polling" technique that keeps an http connection open to a server for a fixed amount of time. The data sent from the server triggers an event client side that is acted upon at that time. I would imagine that they use this technique to support the older browsers that do not have websocket support built in.
I, personally, have been working on an application with similar requirements and have opted to use a combination of node.js and socket.io. The socket.io module uses a variety of polling solutions and automatically chooses the best one based on what is available on the client.
Maybe you may have a look to Goliath (non-blocking IO server written in Ruby) : http://postrank-labs.github.com/goliath/
In asp.net, How can i trigger client side jquery event from server. I want to implement it in my chat section... current the chat seems to work fine... but it has one problem... i have to send a request every 5 seconds from client's browser to the his chat history.. which i feel is not a good idea...
can anyone provide any solution for my problem
By default you need to do it with polling from the clients browser. That's how ajax works out of the box. There's a technique called Comet which is a push model.
You can try to use a jQuery plugin like this: http://plugins.jquery.com/project/jQuery-Comet-Push-API.
In the (near) future you could make use of websockets as well in HTML5. Here's information taken from http://channel9.msdn.com/Events/MIX/MIX11/HTM10.
WebSockets, an emerging specification
being standardized by W3C and IETF,
will enable web browsers as well as
client applications to open a
bi-directional, full-duplex
communication channel with a remote
host.
Be sure to check out that MIX11 video!
Although, I am not that skilled in ASP.Net, this problem can be solved in another way.
Have the server return the name of function you need to execute, then call it in the callback function?
Kinda like this (It is just a typo but)
$.post("yourpage.aspx",
{
d1: "v1"
},
function(data) {
//now the data will hold the name of the function
window[data]();
}
);
Now you can wrap the above code, in the another function, and set up a timer, to check for the response regularly, and execute the function if a condition is matched.
P.S. I have skipped the part, where the scripts check if the condition is matched.
You are doing the best that you can do without getting into something like HTML5 WebSockets which are not really ready for regular usage yet.
Two options:
(conservative option) Accept that you will be polling the server and optimize the way you do the polling to keep the impact low.
(edgy option) Try out some of the new libraries that emulate Websockets.
Welcome to the world of HTTP.
Thats how the browsers work. Client sends a request to the Server and Server responds. There is no other way (for now) to go the other way. It's simple and that's one of the reasons HTTP protocol was so popular.
But now things are changing and HTML5 support web sockets. If you developing for a HTML5 supporting device (latest versions of all browsers + iPhone/iPad support sockets) then websockets is the way to go.
On the other side, you can go with Comet polling (as mentioned by XIII). Basically it client sends a request to the server and waits until server has anything to respond to. I am not too sure what is a good way of implementing it in ASP.NET (as I think there is a limit of concurrent connections) but it's a pretty useful technique in Nodejs.
I have a very simple question about ajax.
If I'd like to refresh a particular area of my site I supose ajax would be the best way.
But is there anyway instead of having a javascript periodically checking for changes on the server, the server would send the data when a given event would happen?
What I'd like was the client not needing to send requests periodically but instead the server would only send the info to the client which in turn would have some kind of event listener.
Thanks in advance
Yes, this can be done. It is referred to as "push" or "push streaming".
Here is one website that offers the ability to do this: InstantPush. And a brief quote from their home page:
"InstantPush is used to make web pages
and mobile phones go live. They will
instantly be updated in real time when
a change occurs at the server side.
Standard web communication makes
updates pass firewalls and proxies.
Without any modules at the client
side!
InstantPush has been used since 2001,
before "Ajax was invented". It is
probably the First Ajax Push
Framework.
InstantPush is leading the market in
northern Europe."
Here is another company offering this technology: LightStreamer. And a quote from their home page:
"Lightstreamer is a scalable and reliable Server for pushing live data to Rich Internet Applications
Based on the Comet and Real-Time Web
paradigms, it streams real-time data
to any Web browser and client
application. HTML, HTML5, AJAX, Flex,
Silverlight, Java, .NET, iOS, Android,
and BlackBerry applications, can
easily receive live data from
Lightstreamer Server.
Lightstreamer has been used in many
mission-critical production systems,
where scalability, low network impact,
bandwidth management, adaptive
streaming, and other advanced
features, have proven fundamental."
This cannot be done because the http protocol works by sending a request and receiving a response from the server, hence the server cannot a response without receiving a request.
No this cannot be done. A server's job is to serve up results from a request, one that it must have to begin with.
That is not possible using traditional HTTP. You can, however, use long polling or one of its siblings to simulate push behaviour.
I think that websockets is the way to go, but is not supported for all browsers yet.
I used them with ruby and chrome and was pretty easy.
this is indeed a difficult ask where server broadcasts/pushes data to clients without being requested. HTTP is stateless and even if browser is a registered client, it still needs to request either through code or through some tags like meta refresh. New but still not so stable options are Comet or websockets.
Answer is Comet rather than websockets. YES, it is possible.
Another way is using Browser plugin.
This is an except from wiki page at Push Technology
* Apple Push Notification Service
* BOSH
* Comet
* Client–server model
* File transfer
* Pull technology
* Push Access Protocol
* Push e-mail
* Reverse Ajax
* Streaming media
* WebSockets
I'm just wondering if there is a way to have a server push information to a JavaScript function. Essentially I have a Dashboard-type page that has a javaScript function to get updates from the server and update the dashboard.
I would like my server to be able to "ping" the JS.
I don't even know how that could be possible (I'm guessing Twitter and Facebook use polling?), but I'd thought I ask.
I heard of Comet, but I don't know if that works with a plain standard IIS 7 installation? (It's a SharePoint 2010 site if that matters in any way) If I understand it correctly, Comet is essentially a constantly open connection, so it seems like it's actually the opposite of what I want (reducing # of requests and therefore load)
If you're looking for a comet server for IIS, check out WebSync; it's exactly that :)
Truly initiating a connection from the server is not possible using HTTP. Comet isn't really a single technique, but a set of different workarounds (many of which are described at the article you linked).
For information on Comet techniques with IIS, see the prior question, Comet Programming in IIS. One of the programs discussed there is WebSync.
A Comet style workaround is the most common way to get this functionality. The connection is not constantly open, but rather throttled to make calls every x seconds, then try again upon timeout. The timeout essentially means that the server didn't have anything to give to the client in the duration of the poll. You'll see that the Etherpad code used this same approach, which has been integrated into other Google products now like Google Docs and Wave.
As Samuel Neff says, "You're going to need an open connect to "push" data from the server to the client."
You can use a service like pubnub to open persistent connections from the client and support fallbacks for older browsers.
I made a small demo to show you how the front-end of this application may work. The demo shows PubNub latency over time. The source is available here.
The browser subscribes to a channel and fires a callback when a message is received.
pubnub.subscribe({
channel: 'my_channel',
message: function(m){console.log(m)}
});
In the demo the client also publishes messages. In your case you would include the PubNub IIS library.
pubnub.Subscribe<string>(channel="mychannel", DisplaySubscribeReturnMessage, DisplaySubscribeConnectStatusMessage, DisplayErrorMessage);
// NOTE: DisplaySubscribeReturnMessage, DisplaySubscribeConnectStatusMessage and DisplayErrorMessage are callback methods
You're going to need an open connect to "push" data from the server to the client. So even if you went the route of using a plugin like Flash to open a socket connection which supports two-way communications, you have an open socket connection.
Your statement "reducing # of requests and therefore load" really is problematic. You're equating number of requests with load and that is not accurate. With Comet the majority of requests are waiting on data. Therefore you can have a very high number of requests, but really a very low load on the server--it's hardly using resources besides a waiting thread from the worker thread pool.
Use Comet. Works great, is simple to implement, and does exactly what you need.
You have to do it the other way around, by having the client "pinging" the server with JS.
You can do something like:
function pollServer()
{
// Get some parameter
var param = .......
AJAXCall("page.php?param="+param, onReturn);
}
function onReturn(response)
{
// do something with response
setTimeout("pollServer()", 5000);
}
pollServer();
AJAXCall being the function you use to do an AJAX call and that calls onReturn when it gets a response.
Once it gets a response it waits in this case 5 seconds and polls the server again