We Keep Coding

What is the best technology for a chat/shoutbox system? Plain JS, JQuery, Websockets, or other?

I have an old site running, which also has a chat system, which always used to work fine. But recently I picked up the project again and started improving and the user base has been increasing a lot. (running on a VPS)
Now this shoutbox I have (running at http://businessgame.be/shoutbox) has been getting issues lately, when there are over 30 people online at the same time, it starts to really slow down the entire site.
This shoutbox system was written years ago by the old me (which ironically was the young me) who was way too much into old school Plain Old JavaScript (POJS?) and refused to use frameworks like JQuery.
What I do is I poll every 3 seconds with AJAX if there are new messages, and if YES, load all those messages (which are handed as an XML file which is then parsed by the JS code into HTML blocks which are added to the shoutbox content.
Simplified the script is like this:
The AJAX functions
function createRequestObject() {
var xmlhttp;
if (window.XMLHttpRequest) { // code for IE7+, Firefox, Chrome, Opera, Safari
xmlhttp = new XMLHttpRequest();
} else { // code for IE6, IE5
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
// Create the object
return xmlhttp;
}
function getXMLObject(XMLUrl, onComplete, onFail) {
var XMLhttp = createRequestObject();
// Check to see if the latest shout time has been initialized
if(typeof getXMLObject.counter == "undefined") {
getXMLObject.counter = 0;
}
getXMLObject.counter++;
XMLhttp.onreadystatechange = function() {
if(XMLhttp.readyState == 4) {
if(XMLhttp.status == 200) {
if(onComplete) {
onComplete(XMLhttp.responseXML);
}
} else {
if(onFail) {
onFail();
}
}
}
};
XMLhttp.open("GET", XMLUrl, true);
XMLhttp.send();
setTimeout(function() {
if(typeof XMLhttp != "undefined" && XMLhttp.readyState != 4) {
XMLhttp.abort();
if(onFail) {
onFail();
}
}
}, 5000);
}
Chat functions
function initShoutBox() {
// Check for new shouts every 2 seconds
shoutBoxInterval = setInterval("shoutBoxUpdate()", 3000);
}
function shoutBoxUpdate() {
// Get the XML document
getXMLObject("/ajax/shoutbox/shoutbox.xml?time=" + shoutBoxAppend.lastShoutTime, shoutBoxAppend);
}
function shoutBoxAppend(xmlData) {
process all the XML and add it to the content, also remember the timestamp of the newest shout
}
The real script is far more convoluted, with slower loading times when the page is blurred and keeping track of AJAX calls to avoid double calls at the same time, ability to post a shout, load settings etc. All not very relevant here.
For those interested, full codes here:
http://businessgame.be/javascripts/xml.js
http://businessgame.be/javascripts/shout.js
Example of the XML file containing the shout data
http://businessgame.be/ajax/shoutbox/shoutbox.xml?time=0
I do the same for getting a list of the online users every 30 seconds and checking for new private messages every 2 minutes.
My main question is, since this old school JS is slowing down my site, will changing the code to JQuery increase the performance and fix this issue? Or should I choose to go for an other technology alltogether like nodeJS, websockets or something else? Or maybe I am overlooking a fundamental bug in this old code?
Rewriting an entire chat and private messages system (which use the same backend) requires a lot of effort so I'd like to do this right from the start, not rewriting the whole thing in JQuery, just to figure out it doesn't solve the issue at hand.
Having 30 people online in the chatbox at the same time is not really an exception anymore so it should be a rigid system.
Could perhaps changing from XML data files to JSON increase performance as well?
PS: Backend is PHP MySQL

I'm biased, as I love Ruby and I prefer using Plain JS over JQuery and other frameworks.
I believe you're wasting a lot of resources by using AJAX and should move to websockets for your use-case.
30 users is not much... When using websockets, I would assume a single server process should be able to serve thousands of simultaneous updates per second.
The main reason for this is that websockets are persistent (no authentication happening with every request) and broadcasting to a multitude of connections will use the same amount of database queries as a single AJAX update.
In your case, instead of everyone reading the whole XML every time, a POST event will only broadcast the latest (posted) shout (not the whole XML), and store it in the XML for persistent storage (used for new visitors).
Also, you don't need all the authentication and requests that end up being answered with a "No, there aren't any pending updates".
Minimizing the database requests (XML reads) should prove to be a huge benefit when moving from AJAX to websockets.
Another benifit relates to the fact that enough simultaneous users will make AJAX polling behave the same as a DoS attack.
Right now, 30 users == 10 requests per second. This isn't much, but it can be heavy if each request would take more than 100ms - meaning, that the server answers less requests than it receives.
The home page for the Plezi Ruby Websocket Framework has this short example for a shout box (I'm Plezi's author, I'm biased):
# finish with `exit` if running within `irb`
require 'plezi'
class ChatServer
def index
render :client
end
def on_open
return close unless params[:id] # authentication demo
broadcast :print,
"#{params[:id]} joind the chat."
print "Welcome, #{params[:id]}!"
end
def on_close
broadcast :print,
"#{params[:id]} left the chat."
end
def on_message data
self.class.broadcast :print,
"#{params[:id]}: #{data}"
end
protected
def print data
write ::ERB::Util.html_escape(data)
end
end
path_to_client = File.expand_path( File.dirname(__FILE__) )
host templates: path_to_client
route '/', ChatServer
The POJS client looks like so (the DOM update and from data access ($('#text')[0].value) use JQuery):
ws = NaN
handle = ''
function onsubmit(e) {
e.preventDefault();
if($('#text')[0].value == '') {return false}
if(ws && ws.readyState == 1) {
ws.send($('#text')[0].value);
$('#text')[0].value = '';
} else {
handle = $('#text')[0].value
var url = (window.location.protocol.match(/https/) ? 'wss' : 'ws') +
'://' + window.document.location.host +
'/' + $('#text')[0].value
ws = new WebSocket(url)
ws.onopen = function(e) {
output("<b>Connected :-)</b>");
$('#text')[0].value = '';
$('#text')[0].placeholder = 'your message';
}
ws.onclose = function(e) {
output("<b>Disonnected :-/</b>")
$('#text')[0].value = '';
$('#text')[0].placeholder = 'nickname';
$('#text')[0].value = handle
}
ws.onmessage = function(e) {
output(e.data);
}
}
return false;
}
function output(data) {
$('#output').append("<li>" + data + "</li>")
$('#output').animate({ scrollTop:
$('#output')[0].scrollHeight }, "slow");
}
If you want to add more events or data, you can consider using Plezi's auto-dispatch feature, that also provides you with an easy to use lightweight Javascript client with an AJAJ (AJAX + JSON) fallback.
...
But, depending on your server's architecture and whether you mind using heavier frameworks or not, you can use the more common socket.io (although it starts with AJAX and only moves to websockets after a warmup period).
EDIT
Changing from XML to JSON will still require parsing. The question is actually whether XML vs. JSON parsing speeds.
JSON will be faster on the client javascript, according to the following SO question and answer: Is parsing JSON faster than parsing XML
JSON seems to be also favored on the server-side for PHP (might be opinion based rather than tested): PHP: is JSON or XML parser faster?
BUT... I really think your bottleneck isn't the JSON or the XML. I believe the bottleneck relates to the multitude of times that the data is accessed, (parsed?) and reviewed by the server when using AJAX.
EDIT2 (due to comment about PHP vs. node.js)
You can add a PHP websocket layer using Ratchet... Although PHP wasn't designed for long running processes, so I would consider adding a websocket dedicated stack (using a local proxy to route websocket connections to a different application).
I love Ruby since it allows you to quickly and easily code a solution. Node.js is also commonly used as a dedicated websocket stack.
I would personally avoid socket.io, because it abstracts the connection method (AJAX vs Websockets) and always starts as AJAX before "warming up" to an "upgrade" (websockets)... Also, socket.io uses long-polling when not using websockets, which I this is terrible. I'd rather show a message telling the client to upgrade their browser.
Jonny Whatshisface pointed out that using a node.js solution he reached a limit of ~50K concurrent users (which could be related to the local proxy's connection limit). Using a C solution, he states to have no issues with more than 200K concurrent users.
This obviously depends also on the number of updates per second and on whether you're broadcasting the data or sending it to specific clients... If you're sending 2 updates per user per second for 200K users, that's 400K updates. However, updating all the users only once every 2 seconds, that's 100K updates per second. So trying to figure out the maximum load can be a headache.
Personally, I didn't get to reach these numbers on my apps, so I never got to discover Plezi's limits first hand... although, during testing, I had no issues with sending hundred of thousands of updates per second (but I did had a connection limit due to available ports and open file handle limits on my local machine).
This definitely shows how vast of an improvement you can reach by utilizing websockets (especially since you stated to notice slowdowns with 30 concurrent users).

Ajax Request in Progress suddenly started returning status = 0

I have a web project in PHP and it accesses a Java Project that uses the Restlet Framework. The web project is running on Apache and I am testing it using localhost. The Restlet Framework also uses localhost as the domain, but the url is slightly different: localhost:8888/
This is the Javascript that, using Ajax, makes a call to one of the Java classes (CollectionPublic) using the URL above.
var url = "<?php echo $config['restServer_url'] ?>collectionPublic";
var params= "pageList="+facebookPages+"&time="+time;
var client = new XMLHttpRequest();
client.open("POST", url,true);
client.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
client.onreadystatechange = function () {
if (client.readyState != 4) return;
if (client.status != 200 && client.status != 304) {
alert("error "+client.status);
} else {
alert("success");
}
callback(client);
}
if (client.readyState == 4) return;
client.send(params);
I have tested and the call is being made correctly, using the URL localhost:8888/collectionPublic, and it is reaching the CollectionPublic class (the class is working fine).
The PROBLEM is: When this call is made, the CollectionPublic class takes a long time to complete its task, and the user should be able to access other pages (on the same server) or reload the page. However, when either of these things happen, the alert("error "+client.status) pops up and the value of client.status is 0. The call is then aborted, but the CollectionPublic's task continue normally, and when it finishes, nothing happens in the web page (before, the alert("success") was being fired).
I spent hours trying to figure out what was causing the error, since this was working last week. Most of the posts I found said that it could be a Cross-Origin Resource problem, since localhost and localhost:8888 are not considered as the same domain. To see if that was really the problem, I started Chrome using the --disable-web-security argument (and it was really disabled) but the issue was still there.
The weirdest thing is that it has worked before, and I changed absolutely NOTHING in the code.
I have seen this post Reloading page while an Ajax request in progress gives empty response and status as zero and it seems quite similar to what I am facing.
Hopefully, I have made myself clear, but if you have any doubts regarding this issue, just ask.
Thanks a lot in advance.

I'm not convinced that the ajax request itself is quite right. if (client.readyState != 4) return; will always be true aside from when its actually 4. This may be better:
client.onreadystatechange = function () {
if(client.readyState < 4) {
//not complete yet
return;
}
if(client.status != 200 && client.status != 304) {
//an error
alert("error "+client.status);
return;
}
if(client.readyState === 4) {
//complete
callback(client);
}
}
As for the problem whereby the ajax call is aborted: This is correct behaviour. All XHR calls will be aborted by the browser as soon the page is reloaded or unloaded. Perhaps this was somehow not the case when viewing pages locally. I would not allow the user to navigate away (or reload) whilst the ajax in progress. As a work-around, your class could set a session variable that is read by your page.

decodeAudioData returning a null error

I come here hoping that you lovely folks here on SO can help me out with a bit of a problem that I'm having.
Specifically, every time I attempt to use the decodeAudioData method of a webkitAudioContext, it always triggers the error handler with a null error. This is the code that I'm currently using:
var soundArray;
var context = new webkitAudioContext();
function loadSound(soundName) {
var request = new XMLHttpRequest();
request.open('GET',soundName);
request.responseType = 'arraybuffer';
request.onload = function() {
context.decodeAudioData(this.response, function(buf) {
sounds[soundName] = buf;
},function(err) { console.log("err(decodeAudioData): "+err); });
}
request.send();
}
At this point, it constantly logs error messages to the console saying err(decodeAudioData) = null, mostly because that was just how I decided to log it.
In any case, any idea why this might be going on?
I'm using Chrome Canary, v20.0.1121.0, to try and get something working. But, obviously, it's not working! So, any idea what I might be able to do? If any new information is needed, let me know, and I'll update as necessary.

The real reason is that both createBuffer and decodeAudioData right now have a Bug and throw weird vague DOM exception 12 for files they should normally play.
But we should be aware that this is new and evolving technology and be thankful even for web audio api as it is now since its small miracle that happened to us.
They are missing stream syncing on header boundary that any reasonable decoder of streaming audio format should start with.
And mp3 or many aac/adts files are streaming fileformats. streaming means that you can cut them anywhere or insert append anything (various tags even image artwork) decoder shouldnt care about unknown data. decoder should just seek until he finds header he knows and can decode.
I thrown together this temporary solution that seeks to nearest frame header start and passes data from this offset only.
mp3 or mp2 all start header for every audio frame (every around 200bytes) with 0XFFE and aac(adts) on oxFFF syncword that is there just for this reason. therefore both will sync on 0xFFE.
Here is the code I currently use to play previously not played files.
What I hate is that arrayBuffer doesnt have subarray() like its typed childs to return just different view from different offset instead of whole new array copy that slice() returns. if only webaudio api accepted typedarrays as input but unfortunately the only way to create arraybuffer back seems huge slice() copy.
thankfully usually only one or two seeks are needed.
Forcing Web Audio Api to not being Picky about files
node={};
node.url='usual_mp3_with_tags_or_album_artwork.mp3';
function syncStream(node){ // should be done by api itself. and hopefully will.
var buf8 = new Uint8Array(node.buf);
buf8.indexOf = Array.prototype.indexOf;
var i=node.sync, b=buf8;
while(1) {
node.retry++;
i=b.indexOf(0xFF,i); if(i==-1 || (b[i+1] & 0xE0 == 0xE0 )) break;
i++;
}
if(i!=-1) {
var tmp=node.buf.slice(i); //carefull there it returns copy
delete(node.buf); node.buf=null;
node.buf=tmp;
node.sync=i;
return true;
}
return false;
}
function decode(node) {
try{
context.decodeAudioData(node.buf,
function(decoded){
node.source = context.createBufferSource();
node.source.connect(context.destination);
node.source.buffer=decoded;
node.source.noteOn(0);
},
function(){ // only on error attempt to sync on frame boundary
if(syncStream(node)) decode(node);
});
} catch(e) {
log('decode exception',e.message);
}
}
function playSound(node) {
node.xhr = new XMLHttpRequest();
node.xhr.onload=function(){
node.buf=node.xhr.response;
node.sync=0;
node.retry=0;
decode(node);
}
node.xhr.open("GET", node.url, true);
node.xhr.responseType = "arraybuffer";
node.xhr.send();
}

I was using webkitAudioContext with Chrome 19. Today I've upgraded to Chrome 20 and I have the same problem as you.
I have taken another MP3 file and it works again. The only difference between the two files is the cover embedded in the wrong MP3 file.
I have removed the cover and it works again.

How to know if the network is (dis)connected?

How can I know, in Xul, if the network is (dis)connected?
--update
Using:
function observe(aSubject, aTopic, aState) {
if (aTopic == "network:offline-status-changed") {
write("STATUS CHANGED!");
}
}
var os = Components.classes["#mozilla.org/observer-service;1"].getService(Components.interfaces.nsIObserverService);
os.addObserver(observe, "network:offline-status-changed", false);
and the preference:
pref("network.manage-offline-status", true);
it's not working.. There's a bug report here, but I don't think it has something to do with it.
--
Actually I think it's not possible to be notified, as even in Firefox we're never notified, and the user need to manually mark "work offline" if he wants the browser to know that it's offline..
--
Screenshot my of Firefox "about:config" filtering for "offline" string, unfortunately, there no "network.manage-offline-status":

You should be able to use navigator.onLine. Here is the help page
https://developer.mozilla.org/en/Online_and_offline_events
navigator.onLine is a property that
maintains a true/false value (true for
online, false for offline). This
property is updated whenever the user
switches into "Offline Mode" by
selecting the corresponding menu item
(File -> Work Offline in Firefox).
Another solution (as commented by #Neil):
Components.classes["#mozilla.org/observer-service;1"]
.getService(Components.interfaces.nsIObserverService)
.addObserver(myF­unction, "network:offline-status-changed", false);

The best way I found is to use the following javascript code, that behaves like a ping, and make the test with some big websites, and assume that if none of them answers, so the network must be disconnected.
var ping = {};
ping = {
img:null,
imgPreload:null,
timer:null,
init:function() {
var sess = new Date();
var nocache = sess.getTime();
var imguri = ping.img+"?time="+nocache;
var ping.imgPreload = new Image();
ping.imgPreload.onload = function() {
clearTimeout(ping.timer);
ping.timer = null;
alert("Domain is available");
};
ping.imgPreload.src = imguri;
ping.timer = setTimeout("ping.fail_to_ping()",60000);
},
fail_to_ping:function() {
clearTimeout(ping.timer);
ping.timer = null;
ping.imgPreload = null;
alert("Ping to domain failed!");
}
};
(from http://crynobone.com/ci/index.php/archive/view/852)
--update
But, as it's not a reliable solution (as you can't rely that the image will be in the website forever), the best solution might be to develop a new XPCom component.

Eh... as per HTML5 (read echmascript 5), the on-/offline events are available.
See it here at Mozilla Hacks
Edit 20/4/2011:
I just encountered an update for this answer, when i was watching a podcast from MS MIX11:
http://channel9.msdn.com/Events/MIX/MIX11/HTM14 around time 43:36, the lecturer is actually talking about the window.navigator.onLine property, where he uses it for detecting if the browser (and the computer) is online. Then he uses the online event to do something when he gets online again.
This method is only available in modern browsers, however. So IE 8 and below have to poll for the connection.

Query URL for Log In Status, javascript

This may seem like a no-brainer, but I can't find a way to do this that isn't considered a security issue (other than the obvious ways)...
So, I want to build an add-on for Firefox to use with my team. Basically it will be a status bar icon letting us know if the authentication cookie for our tools site has expired, so we can tell without losing any work currently in the browser.
At first I thought I could have the add-on check the cookie, but this seems to be a huge hassle for such a simple idea. Then it occurred to me...DUH...that I could just have the add on try to access the main page of our site. If it gets a "Access Denied" response, it can show the icon for "Not Logged In", but if it gets anything else, it can show "Signed In".
However, all attempts to do this with AJAX are proving to be almost as difficult as my cookie attempts.
Is there a simple way, with javascript preferably, but in XUL otherwise, to say
var url = "http://example.com";
var response = pingURL(url, "blah);
status = (response = "Welcome!") ? "Signed in" : "Not Signed In";
where "pingURL" would be the method of "going" to the url and getting the response?

function checkAccess(url, callback) {
var xhr = new XMLHttpRequest();
xhr.open("GET", url);
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
if (xhr.status == 200) {
callback(true);
} else {
callback(false);
}
}
};
}
This should work... Just call with "checkAccess('http://example.com', function(ready){});" as an example where ready is a boolean value.

Exactly why do you consider cookies a huge hassle? That would undoubtedly be faster and probably simpler to implement. Reading cookies from chrome is simple and well-documented. Ask for help if you can't figure out how to parse the cookie.