So we have a logic whereby we will check the status of a URL every 15 seconds like so:
const http = new XMLHttpRequest();
const alive = (url) => {
http.open("HEAD", url, false);
http.send();
return http.status !== 404;
};
This url is scheduled to be working at a certain time of the day for a period of time only and our app will have to detect when it has returned 200. Once it does, we'll display an iframe that has the url content (and also clear the interval). So whenever it's still returning a 404, we will display an "upcoming" image instead.
Is it okay to keep doing this check? Won't we encounter too many requests error on the url at some point? Realise that we may not be able to have control on the server for that url. Let's say about 1000 people will be accessing the page that does the checking. Some of them may visit the page earlier and stay there. The page will start doing this check every 15 seconds up until the url becomes alive (which can take hours after the moment they land on the page).
To determine whether or not something is alive, you'll probably want to return http.status === 200 instead. This means if say there's an internal server error for example, an error 500, then it's assumed it is not alive.
For a polling approach, any interval can be too frequent if you have lots of users.
Your best bet for scalability is to have a socket-based system for communication. As you mentioned, you are using node.js. You can look into using socket.io. When you go "live" then you can just emit an event to all connected clients.
Related
i have an object that use some resources like
class UseResources {
protected $proc_open;
public function __construct()
{
$this->proc_open = proc_open( ... );
}
public function cleanup()
{
proc_close($this->proc_open);
}
};
i tried to send an ajax request on .unload event, but its not what am looking for.
i want to be able to call the cleanup method once the user close the window or the browser, or once the connection is lost?
The unload event fires when the window closes, but also when user refreshes the page or navigates to a new page (even if new page is within your site). I recommend against using unload in this case. (unload is better used for things like prompting user to save or lose changes before leaving a form, for example.)
Also, if you implement a javascript solution you either want to (1) require javascript to use the site (and show <noscript></noscript> element to folks with JS disabled), or (2) code such that you are not relying solely on your window close / cleanup detection routine to release resources.
The strategy I typically implement to do stuff like this is to use a ping process. Basically: the client sends regular pings to the server which resets the connection's/session's idle counter to 0 when it receives a ping. The pings occur every minute or so depending on need. A long-running looping thread (a Session manager, if you will) then checks for any clients that have idled beyond the idle threshhold (say 3 minutes, but up to you), and if exceeded releases the resources tied to that connection/session. Then every page (where appropriate) in your site adds this ping code so that the client starts pinging for as long as the page is open in the browser.
Benefits
keeps track of how long user is viewing your site (useful for metrics, especially when coupled with a metric for tracking number of requests)
closes all connection-related resources when browser is closed, and also when user navigates away from your site
if user navigates to different page [that has ping code] on your site their connection resources will not be cleared because the idle counter will reset on the next page
Drawbacks
requires javascript (setTimeout or setInterval and XMLHttpRequest, for example)
unless you also have something like an "authentication timeout" on the server, the client could potentially keep your server (and resources) connected for a long time if user walks away from an open web browser for a while (you could get around this by using alternate ping triggers such as mouse movement, setting focus to form fields, clicking, scrolling, etc.)
Rough example code for client page:
function pingServer() {
var req;
if (window.XMLHttpRequest) {
req = new XMLHttpRequest();
} else {
req = new ActiveXObject("Microsoft.XMLHTTP");
}
req.onreadystatechange = function() {
if (req.readyState == 4 && req.status == 200) {
alert("Idle timer reset on server");
}
}
var url = 'ping.php'; // call php code that will reset idle timer for this client
req.open("GET", url, true);
req.send();
}
var pingFrequencyMs = 5 * 1000; // ping every 5 secs
setInterval(pingServer, pingFrequencyMs);
<noscript>Sorry, JavaScript is required for this website.</noscript>
And here is a really rough example of process you could use on server side, but until you provide more details I can only speculate about what kind of web app you are working with:
Server Ping routine (pseudocode, could be PHP, ASP, JSP, etc.)
get client connection unique ID (or Session ID, or whatever)
get current value for client idle timer, else default to 0 (could get this from memory cache, database, file on disk; your choice)
get system value for max idle (timeout)
compare client idle to max idle
if client idle exceeds max idle then end the session (whatever that means; close connection/session-specific resources, etc. -- in your case it means call the cleanup routine, but make sure it has the correct object context), else reset idle to 0
I want to gather some information using the visitors of my websites.
What I need is for each visitor to ping 3 different hostnames and then save the following info into a DB.
Visitor IP, latency 1,latency 2, latency 3
Of course everything has to be transparent for the visitor without interrupting him in any way.
Is this possible? Can you give me an example? Are there any plugins for jQuery or something to make it easier
EDIT
This is what I have so far jsfiddle.net/dLVG6 but the data is too random. It jumps from 50 to 190
This is going to be more of a pain that you might think.
Your first problem is that Javascript doesn't have ping. Mostly what Javascript is good at is HTTP and a few cousin protocols.
Second problem is that you can't just issue some ajax requests and time the results (that would be way too obvious). The same origin policy will prevent you from using ajax to talk to servers other than the one the page came from. You'll need to use JSONP, or change the src of an image tag, or something else more indirect.
Your third problem is that you don't want to do anything that will result in a lot of data being returned. You don't want data transfer time or extensive server processing to interfere with measuring latency.
Fourth, you can't ask for URLs that might be cached. If the object happened to be in the cache, you would get really low "latency" measurements but it wouldn't be meaningful.
My solution was to use an image tag with no src attribute. On document load, set the src to point to a valid server but use an invalid port. Generally, it is faster for a server to simply reject your connection than to generate a proper 404 error response. All you have to do then is measure how long it takes to get the error event from the image.
From The Filddle:
var start = new Date().getTime();
$('#junkOne').attr('src', 'http://fate.holmes-cj.com:8886/').error(function () {
var end = new Date().getTime();
$('#timer').html("" + (end - start) + "ms");
});
The technique could probably be improved. Here's some ideas:
Use IP address instead of DNS host name.
Do the "ping" multiple times, throw out the highest and lowest scores, then average the rest.
If your web page has a lot heavy processing going on, try to do the tests when you think the UI load is lightest.
With jQuery you could:
$.ajax(url,settings)(http://api.jquery.com/jQuery.ajax/) and take the time from beforeSend and on complete via Date.now(), subtract those times -> then you have the time for the request (not excactly the "Ping" though)
2021:
Tried this again for a React app I'm building. I don't think the accuracy is too great.
const ping = () => {
var start = new Date().getTime();
api.get('/ping').then((res) => {
console.log(res)
var end = new Date().getTime();
console.log(`${end-start} ms`)
}, (err) => {
console.log(err)
})
};
Wrote my own little API, but I suppose there's just way too much going on during the request.
In terminal, I get about 23ms ping to my server.. using this it shoots up to like 200-500ms.
I would like to keep track of how long visitors spend reading a page. If they tab away, or minimize the window, time should not count towards the time on page until they look at the tab again.
I assume some combination of javascript and server side work will be necessary.
A couple of issues I'm struggling with:
What's the best way to store this information in the database?
How do I, with Javascript, capture the time on page with a reasonable degree of accuracy? Do I store events like "page loaded", "user idle", "user returned", "page unloaded", and then separately process all the events in the DB to come up with a time on page?
I've put some work into a small JavaScript library that times how long a user is on a web page. It has the added benefit of more accurately (not perfectly, though) tracking how long a user is actually interacting with the page. It ignores time that a user switches to different tabs, goes idle, minimizes the browser, etc. The Google Analytics method suggested has the shortcoming (as I understand it) that it only checks when a new request is handled by your domain. It compares the previous request time against the new request time, and calls that the 'time spent on your web page'. It doesn't actually know if someone is viewing your page, has minimized the browser, has switched tabs to 3 different web pages since last loading your page, etc.
https://github.com/jasonzissman/TimeMe.js
An example of its usage:
On loading your page:
document.onload = function() {
TimeMe.setIdleDurationInSeconds(30);
TimeMe.setCurrentPageName("my-home-page");
TimeMe.initialize();
}
Retrieving time spent on the page, and sending it to your server when the user leaves your page:
window.onbeforeunload = function (event) {
xmlhttp=new XMLHttpRequest();
xmlhttp.open("POST","ENTER_URL_HERE",false);
xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
var timeSpentOnPage = TimeMe.getTimeOnCurrentPageInSeconds();
xmlhttp.send(timeSpentOnPage);
};
First, you need to detect when a user has moved away from a tab or is inactive. When this happens, start a timer, when they come back, stop the timer. Counting time with setTimeout/setInterval can be innacurate because of blocking, so I made myself an accurate javascript timer based on the actual difference in datetime: https://gist.github.com/4600726
So your code would look something like this:
timer = new Timer;
window.addEventListener('focus', function() {
timer.start();
}, false);
window.addEventListener('blur', function() {
timer.stop();
// send timer.msecs() to the server maybe??
// if so, also call timer.reset();
}, false);
window.addEventListener('beforeunload', function() {
timer.stop();
// send timer.msecs() to the server via jquery post, or better yet websocket
}, false);
Then you can get the elapsed time with timer.secs(). I guess it depends on your preference how often you want to send info to the server. You could do it on blur.
Another option could be to decree that no mouseover means inactivity. Start the timer and then do a setTimeout, then on the window's mousemove event cancel the setTimeout and start another setTimeout , after which you stop the timer.
As far as sending data to the server, I'd probably opt for sending it on blur and of course beforeunload. My preferred method would be with socket.io since it is fast and always connected, so you could use it to track lots of user events in real time, but you could just to an ajax call to your server. If you just send them as mini user sessions, { user: userId, page: pageId, elapsedTime: msecs } then you could then aggregate the data on the server end when you are doing analysis.
I am trying to use periodic refresh(ajax)/polling on my site by XMLHttp(XHR) to check if a user has a new message on the database every 10 seconds, then if there is inform him/her by creating a div dynamically like this:
function shownotice() {
var divnotice = document.createElement("div");
var closelink = document.createElement("a");
closelink.onclick = this.close;
closelink.href = "#";
closelink.className = "close";
closelink.appendChild(document.createTextNode("close"));
divnotice.appendChild(closelink);
divnotice.className = "notifier";
divnotice.setAttribute("align", "center");
document.body.appendChild(divnotice);
divnotice.style.top = document.body.scrollTop + "px";
divnotice.style.left = document.body.scrollLeft + "px";
divnotice.style.display = "block";
request(divnotice);
}
Is this a reliable or stable way to check message specifically since when I look under firebug, a lot of request is going on to my database? Can this method make my database down because of too much request? Is there another way to do this since when I login to facebook and check under firebug, no request is happening or going on but I know they are using periodic refresh too... how do they do that?
You can check for new data every 10 seconds, but instead of checking the db, you need to do a lower impact check.
What I would do is modify the db update process so that when it makes a change to some data, it also updates the timestamp on a file to show that there is a recent change.
If you want better granularity than "something changed somewhere in the db" you can break it down by username (or some other identifier). The file(s) to be updated would then be the username for each user who might be interested in the update.
So, when you script asks the server if there is any information for user X newer than time t, instead of making a DB query, the server side script can just compare the timestamp of a file with the time parameter and see if there is anything new in the database.
In the process that is updating the DB, add code that (roughly) does:
foreach username interested in this update
{
touch the file \updates\username
}
Then your function to see if there is new data looks something like:
function NewDataForUser (string username, time t)
{
timestamp ts = GetLastUpdateTime("\updates\username");
return (ts > t);
}
Once you find that there is new data, you can then do a full blown DB query and get whatever information you need.
I left facebook open with firebug running and I'm seeing requests about once a minute, which seems like plenty to me.
The other approach, used by Comet, is to make a request and leave it open, with the server dribbling out data to the client without completing the response. This is a hack, and violates every principle of what HTTP is all about :). But it does work.
This is quite unreliable and probably far too taxing on the server in most cases.
Perhaps you should have a look into a push interface: http://en.wikipedia.org/wiki/Push_technology
I've heard Comet is the most scalable solution.
I suspect Facebook uses a Flash movie (they always download one called SoundPlayerHater.swf) which they use to do some comms with their servers. This does not get caught by Firebug (might be by Fiddler though).
This is not a better approach. Because you ended up querying your server in every 10 seconds even there is no real updates.
Instead of this polling approach, you can simulate the server push (reverrse AJAX or COMET) approach. This will compeletly reduce the server workload and only the client is updated if there is an update in server side.
As per wikipedia
Reverse Ajax refers to an Ajax design
pattern that uses long-lived HTTP
connections to enable low-latency
communication between a web server and
a browser. Basically it is a way of
sending data from client to server and
a mechanism for pushing server data
back to the browser.
For more info, check out my other response to the similar question
I run a couple of game tunnelling servers and would like to have a page where the client can run a ping on all the servers and find out which is the most responsive. As far as I can see there seems to be no proper way to do this in JavaScript, but I was thinking, does anybody know of a way to do this in flash or some other client browser technology maybe?
Most applet technology, including Javascript, enforces a same-origin policy. It may be possible to dynamically add DOM elements, such as images, and collect timing information using the onload event handler.
Psuedo-code
for (server in servers) {
var img = document.createElement('IMG');
server.startTime = getCurrentTimeInMS();
img.onload=function() { server.endTime = getcurrentTimeInMS(); }
img.src = server.imgUrl;
}
Then wait an appropriate time and check the timing for each server object. Repeat as needed and compute averages if you want. I'm not sure what kind of accuracy you can expect.
Disadvantages:
You are probably using the wrong tool for the job. A browser is not equipped for this sort of application.
It's probably quite inaccurate.
If the resource you request is cached it won't give you the results you want, but you can work around that by changing the url each time.
This is bandwidth-intensive compared to a normal ping. Make the image tiny, such as a spacer.gif file.
The timing depends not only on the latency of the remote server but the bandwidth of that server. This may be a more or less useful measure but it's important to note that it is not simply the latency.
You need to be able to serve HTTP requests from the various servers and, crucially, each server should serve the exact same resource (or a resource of the same length). Conditions on the server can affect the response time, such as if one server is compressing the data and another isn't.
Before the call to the server, record the Javascript time:
var startTime = new Date();
Load an image from the server:
var img = new Image()
img.onload = function() {
// record end time
}
img.src = "http://server1.domain.com/ping.jpg";
As soon as the request is finished, record the time again. (Given of course that the request didn't time out.)
var endTime = new Date();
Your ping in milliseconds is:
var ping = endTime. getTime() - startTime.getTime();
All you really need is the time from the connection start, to the time of the first readystate change...
function getPing() {
var start;
var client = getClient(); // xmlhttprequest object
client.onreadystatechange = function() {
if (client.readyState > 0) {
pingDone(start); //handle ping
client.onreadystatechange = null; //remove handler
}
}
start = new Date();
client.open("HEAD", "/ping.txt"); //static file
client.send();
}
function pingDone(start) {
done = new Date();
ms = done.valueOf() - start.valueOf();
alert(ms + "ms ping time");
}
function getClient() {
if (window.XMLHttpRequest)
return new XMLHttpRequest();
if (window.ActiveXObject)
return new ActiveXObject('MSXML2.XMLHTTP.3.0');
throw("No XMLHttpRequest Object Available.");
}
Here's an <iframe> approach:
(source: magnetiq.com)
Create a table (not necessarily in the literal <table> sense) with two columns. The first column will hold the name of servers (and possibly links to them). The second column has iframes that load probe documents from the respective servers. Each probe document does this on the initial fetch request:
Get current system time
Do a redirect (302) to a second probe document while passing the system time as a query parameter
The second probe document reads the current system time, calculates the delta from the initial reading that was passed to it and just displays it in big fat letters. This delta will be the time it took for the server to respond to the client with a redirect response plus the time it took for the client to make the second request to the redirection target. It's not exactly a "ping" but it's a comparable measure of the client's relative latency with each server. In fact, it's a "reverse ping" from the server to the client.
You'd be using iframes without infringing the same-domain policy because there's no attempt at manipulating the iframe contents at all. The player will simply see the values with his/her own eyes and you'll rely on the user glancing at the numbers and clicking on the server link that makes the most sense.
Anything that makes an HTTP request (like most of the answers here) will generally measure a latency that's at least twice of what you'd see for a normal ping, because you'll need the three way handshake and the termination packet at minimum (two round trips rather than one). If you make HTTP requests, try to keep the headers to a minimum. A long enough header (due to a chatty server, or cookies etc on the client) can add additional round trips into the mix, throwing off your measurements.
As Cherona points out, if you already have an active HTTP 2 connection to the server, or if the server speaks HTTP 3, then this may not be the case.
The most accurate option would be to open a websocket connection to each server and measure the time it takes to send a tiny message and receive a tiny response (after the connection has been established).
If you are talking about running something client side, I am not sure this is possible due to security reasons.
Maybe your best bet would be a java applet - but again this needs to be checked against local security policy.
If I try to think about some hack in JS to do this, maybe you can try to send an async request with a callback function which measures the milliseconds it took - but this is just off the top of my head.
It's not that hard to measure server response time in Flash.
Flash must ask for a policy file before accessing remote servers.
The default location for such policy file is at the root folder of the server: /crossdomain.xml
(You can easily find information about the crossdomain file format)
Since such file is needed anyway, why not use it to measure server response time? Load the file itself instead of an image and measure the time it took using getTimer() .
This will give you a good estimate on HTTP connections.
But if you're dealing with game servers, you might want to directly check the speed of the TCP connection. To do that you'll need to use the flash.net.Socket
You'll also have to ask for a policy file first by running:
Security.loadPolicyFile("xmlsocket://server.domain.com:5342");
Where 5342 represents your server's port number where it should respond with the proper XML policy string.
After making the socket connection, any request/response will let you measure different server response times.
The problem with 'file pings' is that you would evaluate the http server response whereas your target resource for the games you serve may have a very different behavior and thereby a different latency.
Just an idea out of the blue, maybe even unrealistic depending on the actual context:
but, wouldn't it be interesting to make a server script based on a short sequence of tasks typically executed by the servers during the gameplay (e.g. opening a RTMP connection, retrieving an information, sending it back). Depending on the total number of servers, you could almost opening them simultaneously and define the first response as winner (subtracting the time your client requires independently to process each query).
Of course this is a quite expensive method server-side-speaking, but at least you would hopefully get a reliable result (server and network latencies summed up). Even if it takes a couple seconds to evaluate, this would be the matter of a fraction of the total enjoyable game-play.
Based on the responses of #Mr. Shiny and #Georg Schölly , a complete and commented example.
In order to test, just copy and paste the codes below in the same order, in a empty .html, .php or other compatible file.
Before start the get, record the current Javascript time.
Using new Date(), we create a new date object with the current date and time.
<script type="text/javascript">
var startTime = new Date();
Now let's create a html object image, still without source, and attribute it to the variable img.
var img = new Image();
The next spet is put a source in the image. The .src reflects the src html attribute.
Important! Point your img.src to a very small and lightweight image file, if possible anything less than 10KB.
To prevent cache a random parameter was added at the end of file, after the .png extension.
var random_string = Math.random().toString();
img.src = "http://static.bbci.co.uk/frameworks/barlesque/5.0.0/orb/4/img/bbc-blocks-dark.png" + "?" + random_string;
Now we may call our function which will run just when the image loads, because the .onload:
img.onload = function() {
var endTime = new Date();
var ping = endTime. getTime() - startTime.getTime();
alert(img.src + " loaded in " + ping + " ms");
}
</script>
Inside the function we have the variable endTime that receives a date time after the source image was loaded.
Lastly, the ping variable receives the initial time minus the final time.
The alert popup shows the result.