I'm creating a personal application which displays stock quotes realtime (updating every second), and I was wondering what was the best way to approach this project?
I'm going to query using Yahoo YQL: example query.
I've been researching WebSockets and sockets.io, but I don't believe you can use this unless you own the server with the data. Is this approach not possible?
Send an XMLHttpRequest every second? This seems really bad for some reason, just seeing all the requests in the developer tools makes me cringe and my laptop heat up.
Any thoughts? I've heard of people using an iframe or something to make the requests?
I cannot for the love of programming figure out how Google and Yahoo do it.
An IFRAME that updates each second would have similar effect than an AJAX request every second. Some pages uses an IFRAME that refresh each X time, but there is no magic there, an IFRAME is like another browser window inside the web page.
You are right about websockets, the server must expose a websocket endpoint, otherwise is not possible. If you have this option, go for it.
There is other push technology named "Server Sent Events" (aka SSE, Event Source): http://caniuse.com/eventsource Again the server must expose it, but it basically allows the client to keep a persistent connection to the server, and this push events to the client. Again if you have this option, and websocket is not available, go for it.
If you are not in control of the server, and the only provided option is regular HTTP calls, I guess you have no other option. Please mind that some trading providers limit the amount of requests you can do per minute, or limit the amount of times the information changes per minute, so maybe doing one per second your are not achieving anything relevant... or you can get yourself banned.
I'm pretty new to javascript and API's but I think a google or yahoo API (Application Programming Interface) would be appropriate to link the stock quotes to your app.
Related
I want to create a multiplayer game using JavaScript (no jQuery) and PHP, where most of the mechanics use AJAX calls. However, I need to determine when a user has left the game to update the player status on other players' screens (I assume by regular AJAX requests?). Also, once all players have left the game files (.txts) on the server need to be deleted.
I am using a free web hosting service which means I can't use WebSockets or cron jobs. I also don't want to use Node.js. Most of what I have read advise regularly timestamping with PHP sessions and this is fine, but I would like to know how to then check to see if the user/game has been inactive for a period of time.
Also, using window.onbeforeunload is too unreliable, in case browsers crash etc.
Don't use the wrong tool for the job.
Any kind of AJAX-based solution you attempt for this purpose is likely to very be inefficient, or unreliable, or probably both. If you have more than a tiny number of concurrent users, the sheer volume of AJAX requests would be likely to overwhelm the server and potentially bust your monthly quota. And as you've discovered, determining when someone has ended their session by closing the browser window is not straightforward or reliable. I would advise against any such architecture.
Websockets is really the correct solution for real-time or near-real-time updates between client and server (and vice versa). It'a also easy for the socket server to know when someone has disconnected (which would occur if they close the window/tab).
So you could either upgrade your hosting so you're able to run a websocket server successfully, or try to integrate a websocket based solution hosted elsewhere, e.g. Azure SignalR or some similar product (I am not making specific recommendations in an answer, as that's regarded as off-topic).
use an interval function and ping the php endpoint every second.
If it stops the User is out
I am working on a web application where users have to log in (which gives me a handle that I can track them by in the websocket server) and am wondering whether it is worth it to track them or not. I need to send out a notification to make changes to the home page only if they are on that page. Would it be better to maintain a map of everyone who is on the home page and only send the notification to them, or send the notification to everyone and then have the javascript see if they are on the home page?
Thanks for any input!
Generally speaking, Websocket (and TCP/IP) Unicasting is less resource intensive than broadcasting.
TCP/IP doesn't really allow broadcasting, so broadcasting is implemented by iterating over the connections and sending each one a message (unicasting).
Having said that, the server's code might be better optimized than your own, so you might get better performance while broadcasting under some circumstances than if you iterated over the connections yourself to run some code on each one. This would be the exception rather than the rule.
Some websocket libraries in the wild have a PUB/SUB feature, which let you subscribe connections to a channel that you can publish specific content to.
Knowing whether or not such a feature runs well with respect to performance (etc.), would requires some research into the specific library.
Of course I am aware of Ajax, but the problem with Ajax is that the browser should poll the server frequently to find whether there is new data. This increases server load.
Is there any better method (even using Ajax) other than polling the server frequently?
Yes, what you're looking for is COMET http://en.wikipedia.org/wiki/Comet_(programming). Other good Google terms to search for are AJAX-push and reverse-ajax.
Yes, it's called Reverse Ajax or Comet. Comet is basically an umbrella term for different ways of opening long-lived HTTP requests in order to push data in real-time to a web browser. I'd recommend StreamHub Push Server, they have some cool demos and it's much easier to get started with than any of the other servers. Check out the Getting Started with Comet and StreamHub Tutorial for a quick intro. You can use the Community Edition which is available to download for free but is limited to 20 concurrent users. The commercial version is well worth it for the support alone plus you get SSL and Desktop .NET & Java client adapters. Help is available via the Google Group, there's a good bunch of tutorials on the net and there's a GWT Comet adapter too.
Nowadays you should use WebSockets.
This is 2011 standard that allows to initiate connections with HTTP and then upgrade them to two-directional client-server message-based communication.
You can easily initiate the connection from javascript:
var ws = new WebSocket("ws://your.domain.com/somePathIfYouNeed?args=any");
ws.onmessage = function (evt)
{
var message = evt.data;
//decode message (with JSON or something) and do the needed
};
The sever-side handling depend on your tenchnology stack.
Look into Comet (a spoof on the fact that Ajax is a cleaning agent and so is Comet) which is basically "reverse Ajax." Be aware that this requires a long-lived server connection for each user to receive notifications so be aware of the performance implications when writing your app.
http://en.wikipedia.org/wiki/Comet_(programming)
Comet is definitely what you want. Depending on your language/framework requirements, there are different server libraries available. For example, WebSync is an IIS-integrated comet server for ASP.NET/C#/IIS developers, and there are a bunch of other standalone servers as well if you need tighter integration with other languages.
I would strongly suggest to invest some time on Comet, but I dont know an actual implementation or library you could use.
For an sort of "callcenter control panel" of a web app that involved updating agent and call-queue status for a live Callcenter we developed an in-house solution that works, but is far away from a library you could use.
What we did was to implement a small service on the server that talks to the phone-system, waits for new events and maintains a photograph of the situation. This service provides a small webserver.
Our web-clients connects over HTTP to this webserver and ask for the last photo (coded in XML), displays it and then goes again, asking for the new photo. The webserver at this point can:
Return the new photo, if there is one
Block the client for some seconds (30 in our setup) waiting for some event to ocurr and change the photograph. If no event was generated at that point, it returns the same photo, only to allow the connection to stay alive and not timeout the client.
This way, when clients polls, it get a response in 0 to 30 seconds max. If a new event was already generated it gets it immediately), otherwise it blocks until new event is generated.
It's basically polling, but it somewhat smart polling to not overheat the webserver. If Comet is not your answer, I'm sure this could be implemented using the same idea but using more extensively AJAX or coding in JSON for better results. This was designed pre-AJAX era, so there are lots of room for improvement.
If someone can provide a actual lightweight implementation of this, great!
An interesting alternative to Comet is to use sockets in Flash.
Yet another, standard, way is SSE (Server-Sent Events, also known as EventSource, after the JavaScript object).
Comet was actually coined by Alex Russell from Dojo Toolkit ( http://www.dojotoolkit.org ). Here is a link to more infomration http://cometdproject.dojotoolkit.org/
There are other methods. Not sure if they are "better" in your situation. You could have a Java applet that connects to the server on page load and waits for stuff to be sent by the server. It would be a quite a bit slower on start-up, but would allow the browser to receive data from the server on an infrequent basis, without polling.
You can use a Flash/Flex application on the client with BlazeDS or LiveCycle on the server side. Data can be pushed to the client using an RTMP connection. Be aware that RTMP uses a non standard port. But you can easily fall back to polling if the port is blocked.
It's possible to achive what you're aiming at through the use of persistent http connections.
Check out the Comet article over at wikipedia, that's a good place to start.
You're not providing much info but if you're looking at building some kind of event-driven site (a'la digg spy) or something along the lines of that you'll probably be looking at implementing a hidden IFRAME that connects to a url where the connection never closes and then you'll push script-tags from the server to the client in order to perform the updates.
Might be worth checking out Meteor Server which is a web server designed for COMET. Nice demo and it also is used by twitterfall.
Once a connection is opened to the server it can be kept open and the server can Push content a long while ago I did with using multipart/x-mixed-replace but this didn't work in IE.
I think you can do clever stuff with polling that makes it work more like push by not sending content unchanged headers but leaving the connection open but I've never done this.
You could try out our Comet Component - though it's extremely experimental...!
please check this library https://github.com/SignalR/SignalR to know how to push data to clients dynamically as it becomes available
You can also look into Java Pushlets if you are using jsp pages.
Might want to look at ReverseHTTP also.
I'm already tossing around a solution but as I haven't done something like this before I wanted to check what SO thought before implementation.
Basically I need to modify an existing web based application that has approximately 20 users to add push notifications. It is important that the users get the notifications at the same time (PC-A shouldn't get an alert 20 seconds before PC-B). Currently the system works off of AJAX requests, sending to the server every 20 seconds and requesting any updates and completely rebuilding the table of data each time (even if data hasn't changed). This seems really sloppy so there's two methods I've come up with.
Don't break the connection from server-client. This idea I'm tossing around involves keeping the connection between server and client active the entire time. Bandwidth isn't really an issue with any solution as this is in an internal network for only approximately 20 people. With this solution the server could push Javascript to the client whenever there's an update and modify the table of data accordingly. Again, it's very important that every connected PC receives the updates as close to the same time as possible. The main drawback to this is my experience, I've never done it before so I'm not sure how well it'd work or if it's just generally a bad idea.
Continue with the AJAX request, but only respond in intervals. A second solution I've thought of would be to allow the clients to make AJAX requests as per usual (currently every 20 seconds) but have the server only respond in 30 second intervals (eg 2:00:00 and 2:00:30 regardless of how many AJAX requests it recieves in that span of time). This would require adjusting the timeout for the AJAX request to prevent the request timing out, but it sounds okay in theory, at least to me.
This is for an internal network only, so bandwidth isn't the primary concern, more so that the notification is received as close to each other as possible. I'm open to other ideas, those are just the two that I have thought of so far.
Edit
Primarily looking for pros and cons of each approach. DashK has another interesting approach but I'm wondering if anyone has experience with any of these methods and can attest to the strengths and weaknesses of each approach, or possibly another method.
If I understand well your needs I think you should take a look to Comet
Comet is a web application model in which a long-held HTTP request allows a web server to push data to a browser, without the browser explicitly requesting it. Comet is an umbrella term, encompassing multiple techniques for achieving this interaction. All these methods rely on features included by default in browsers, such as JavaScript, rather than on non-default plugins.
The Comet approach differs from the original model of the web, in which a browser requests a complete web page at a time.
How about using an XMPP server to solve the problem?
Originally designed to be an Instant Messaging platform, XMPP is a messaging protocol that enables users in the system to exchange messages. (There's more to this - But let's keep it simple.)
Let's simplify the scenario a little bit. Imagine the following:
You're a system admin. When the system
has a problem, you need to let all the
employees, about 20 of them, know that
the system is down.
In the old days, every employee will
ask you, "Is the system up?" every
hour or so, and you'll response
passively. While this works, you are
overloaded - Not by fixing system
outage, but by 20 people asking for
system status every hour.
Now, AIM is invented! Since every
employee has access to AIM, you
thought, "Hey, how about having every
single one of them join a 'System
Status' chat room, and I'll just send
a message to the room when the system
is down (or is back)?" By doing so,
employees who are interested in
knowing system status will simply join
the 'System Status' room, and will be
notified of system status update.
Back to the problem we're trying to solve...
System admin = "System" who wants to notify the web app users.
Employees = Web app users who wants to receive notification.
System Status chat room = Still, system Status chat room
When web app user signs on to your web app, make the page automatically logs them onto the XMPP server, and join the system status chat room.
When system wants to notify the user, write code to logon to the XMPP server, join the chat room, and broadcast a message to the room.
By using XMPP, you don't have to worry about:
Setting up "Lasting connection" - Some open source XMPP server, eJabberd/OpenFire, has built-in support for BOSH, XMPP's implementation of the Comet model.
How the message is delivered
You however will need the following:
Find a Javascript library that can help you to logon to an XMPP server. (Just Google. There're a lot.)
Find a XMPP library for the server-side code. (XMPP library exists for both Java & C#. But I'm not sure what system you're using behind the scene.)
Manually provision each user on the XMPP server (Seems like you only have 20 people. That should be easy - However, if the group grows bigger, you may want to perform auto-provisioning - Which is achievable through client-side Javascript XMPP library.)
As far as long-lasting AJAX calls, this implementation is limited by the at-most-2-connection-to-the-same-domain issue. If you used up one connection for this XMPP call, you only have 1 more connection to perform other AJAX calls in the web-app. Depending on how complex your webapp is, this may or may not be desirable, since if 2 AJAX calls have already been made, any subsequent AJAX call will have to wait until one of the AJAX pipeline freed up, which may cause "slowness" on your app.
You can fix this by converting all AJAX calls into XMPP messages, and have a bot-like user on the server to listen to those messages, and response to it by, say, sending back HTML snippets/JSON objects with the data. This however might be too much for what you're trying to achieve.
Ahh. Hope this makes sense... or not. :p
See http://ajaxpatterns.org/HTTP_Streaming
It allows You to push data from the server when server wants it. Not just after the query.
You could use this technique without making large changes to the current application, and synchronize output by the time on the server.
In addition to the other two great options above, you could look at Web Workers if you know they have latest Chrome, Safari, FF, or Opera for a browser.
A Worker has the added benefit of not operating in the same thread as the rest of the page, so performance will be better. The downside is that, for security purposes, you can only send string data between the two scripts and the worker does not have window or document context. However, JSON can be represented as a string, so there's really no limit to the data.
Workers can receive data multiple times and asynchronously. You set the onmessage handler to act each time it receives something.
If you can ask every user to use a specific browser (Latest Safari or Chrome), you can try WebSockets too.
what they do on this demo is exactly what i wanna do.
http://www.lightstreamer.com/demo/RoundTripDemo/
i wonder what comet technique they are using.
it cant be iframe cause on Firefox i can open two tabs with same link. with iframe u cant do that. and it cant be long polling with ajax cause i didnt see it polled anything with firebug.
someone knows the answer? (would be great with some link to good tutorials that do exactly the same thing with same technique).
Whilst digging through the obfuscated scripts is not something I fancy right now, judging by the contents of the page DOM it is posting data from a <form> inside a hidden <iframe> to send data to the server, and having the server send back <script> tags with code to pass data back to the caller.
This is a rather heavyweight and obtrusive technique. It was the only way of doing in-page server communication in the days before XMLHttpRequest existed; I typically wouldn't use it today.
(I wish WebSocket would hurry up and get implemented, doing away with all the long-polling nastiness.)
Looks like several techniques developed by Lightstream which include "vanilla" comet. A brief excerpt from the Lightstreamer white paper:
Each Lightstreamer client typically opens a single permanent connection
with Lightstreamer Server, on which the push updates relating to an
arbitrary number of items, frames and windows travel by means of
multiplexing techniques.
The white paper and demos are very interesting...
Once I developed a module for the Lighttpd web server. The module implemented a Full Duplex Ajax technique, very similar to Comet. In my blog posts you'll find everything you need about FDAjax / Comet, JavaScript examples, problems with firewalls and anti-virus programs, etc.
Lighttpd project seems to be dead. As far I know there is a similar module for the popular nginx. However in future we'll use web sockets.
BTW I used few HTTP addresses (www1.example.com, www2.example.com, ...) to work around the browsers limit of max two IP concurrent connections to the same web server. www[n] were in fact resolved to the same IP address. In case of possible lockup, a browser was automatically redirected to the next www[n] address.