I'd like to measure the latency in opening (and closing) a tcp connection to a server. I don't want to send/receive any data since I want as little overhead as possible to eliminate anything on the server side increasing the request time.
Think of it like a TCP ping. Record current time, connect to host:port, disconnect, calculate delta time.
I believe Javascript doesn't allow opening direct sockets, but I'm hoping given what it can do (e.g. AJAX requests) that it can be used in some shape or form to fit my requirements.
Edit:
Some information about the server:
It's a remote server, so I'd need to be able to handle the error regarding Same Origin
It's not a standard webserver, it doesn't support HEAD (this is why I just wanted to open the connection and not send data)
If I try a GET request, it resets the connection
Even if I could just attempt a connection, and then get a refusal due to the above points, if I can catch the exception I could still use the fact it had connected to determine the latency.
Since we are running inside the application layer, we can only do a latency test over HTTP.
var xhr = new XMLHttpRequest()
xhr.open("HEAD", "/", false)
console.time("latency");
xhr.send();
console.timeEnd("latency");
Code description:
I create an synchronous AJAX request to the current host. I used "HEAD" as the method which is very lightweight and does not receive any contents. So, we can assume that the round-trip for a "HEAD" request is very close to actual ping over ICMP or TCP.
For URL, "/" (current host) is used. Because of Same Origin Policy, you cannot just use any domain like http://google.com unless you are allowed to do so.
I used console.time() and console.timeEnd() to measure the duration which is more accurate than using a regular Date. However, you can also use Date to measure the duration:
//...
var now = (new Date()).getTime()
xhr.send();
var duration = (new Date()).getTime() - now; //ms
UPDATE:
Try this code for measuring the latency even if an exception occurs:
try {
var xhr = new XMLHttpRequest()
xhr.open("HEAD", "http://google.com" + "/?" + Math.random(), false)
console.time("latency");
xhr.send();
}
catch (e){
//block exception
}
finally {
console.timeEnd("latency");
}
Please note that I have also added a random number at the end of the URL to prevent browser caching.
Related
I am trying to get the results from wikidata API using a POST XMLHttpRequest to get the results of the query. But only few of the requests pass and some return CORS issue error which is pretty confusing.
My request looks like this, I have set the origin parameter in the url itself as I understood from wikidata documentation. I have also tried setting the origin in the headers which also didn't work.
setTimeout(function () {
var xhr = new XMLHttpRequest();
xhr.open(
"POST",
"https://query.wikidata.org/sparql?origin=*&format=json&query=" +
encodeURIComponent(
queryService(settingsService()).getPropObject(
vm.selected.uri,
prop
)
),
true
);
xhr.onreadystatechange = function () {
if (xhr.readyState == 4 && xhr.status == 200) {
let data = JSON.parse(xhr.response);
setObjectInnerHtml(label, data, prop);
}
running -= 1;
};
xhr.send();
}, 300);
But it returns an error at the xhr.send() as shown below:
Access to XMLHttpRequest at 'https://query.wikidata.org/sparql?origin=*&format=json&query=PREFIX%20rdf%3A%20%3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23%3E%0APREFIX%20rdfs%3A%20%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E%0ASELECT%20DISTINCT%20%3Furi%20%3FuriLabel%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2FQ183%3E%20%3Chttp%3A%2F%2Fwww.wikidata.org%2Fprop%2Fdirect%2FP1889%3E%20%3Furi%20.%0A%20%20OPTIONAL%20%7B%20%3Furi%20rdfs%3Alabel%20%3FuriLabel%20.%20FILTER%20(lang(%3FuriLabel)%20%3D%20%22en%22)%7D%0A%7D' from origin 'http://localhost:8000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
What could be going wrong here?
Update
This is the series of requests from the same method, of which it can be seen that some requests pass through and some don't. The error that throws up is the CORS issue.
Okay, I have actually seen this happen to me (not with wikipedia) but with other API servers including some of my own.
After much head banging and trying to exclude other factors (I tried running the code on localhost and on multiple computers), this is what I found out.
The issue was with my ISP. my local ISP had an unreliable DNS server, and the connection would stop and start working, sometimes repeatedly every few seconds. Sometimes, within a second, there were any number of fast connections and disconnections. Every time this happened, the console would keep throwing a CORS error, even though it has nothing to do with CORS itself.
Also, when I deployed this app from localhost to the cloud, I never had any issues. Obviously, the cloud server has zero internet connectivity issues.
In summary, if you are intermittently getting a CORS error (as you have described), I would first start by ensuring that your internet connectivity is reliable.
And, deploy the code to an online cloud server and see if the issue replicates.
Please note : I am assuming that the calls you have mentioned in the screenshot are identical in every possible way. In my case, the issues were happening on identical calls with identical payload data.
First of all, I am quite new to Websockets(Socket.io) therefore I have a security-type question.
Is it there a way that someone can send a message towards my Websocket server through browser developer console ?
To me, this question matters a lot for the security of my website. If a client can write JavaScript code in his browser console, and send a successful websocket message, I shall change a lot of my current code that I have already done.
In AJAX, you can make XMLHttpRequest object and send it without any problem if you know where the url should be pointed at.
Example :
var request = new XMLHttpRequest(),
url = 'url_of_.php_file',
data = 'somedata=123';
request.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
console.log("The request and response was successful!");
}
};
request.open('POST', url, true);
request.setRequestHeader('Content-type', 'application/x-www-form-urlencoded; charset=UTF-8');
request.send(data);
If a client does something like this to on my website, will he be able to send message through my websocket without my permission?(If my idea is wrong, please write legit code for doing so)
.......socket implementation.........
mywebsocket.emit('send_money','230.000');
Is there a way to send Websocket messages through browser console?
Absolutely. You can use the developer tools in almost any web browser to find the socket object and call its functions or modify it from the console. And this goes for any javascript object that exists locally in the client's browser.
Obfuscation can only be used to deter a less determined user from tampering with your client. It doesn't actually prevent the user from accessing those functions and variables.
If a client does something like this to on my website, will he be able to send message through my websocket without my permission?
Simply opening a websocket to the client is implicitly giving permission for the client to send anything to your server. So,
Never trust the client.
If you are allowing the client to determine anything of importance locally, you should assume that a savvy opponent will abuse or circumvent that client-side decision. Such as in your example, they could send more money than they actually have, or maybe even negative money!
Your server needs to be capable of handling any data that comes in on that connection, from incoherent garbage to attempted exploits. So always validate that messages sent from the clients are legal using server-side logic. This is the only place where it makes sense to implement your rules or permissions, since everything in the client code is open to tampering.
I am working with cross-domain remote resources that require locking. CORs headers are set appropriately.
I am trying to solve the case where the resource is not released by the client (remains locked until the lock expires) when the browser window is closed.
I had hoped to send a synchronous DELETE request on window unload. I am using jquery (answer can be plain javascript if necessary... mentioning jquery for context) and noticed their docs say "Cross-domain requests ... do not support synchronous operation" and I became very sad.
Is it possible to make a synchronous cross-domain ajax request? Is the jquery limitation due to older browsers? Everything I've read indicates the unload event listener will not be around long enough for the ajax call to complete if it is async and suggests using a synchronous request for this type of cleanup. Unfortunately the call is cross-domain... what can I do?
EDIT
So I am curious if I am getting lucky during local development (i.e. client on 127.0.0.1:8080 and api on 127.0.0.1:8081) or the jquery docs are just misleading. Will the following end up causing me problems down the road?
This appears to be working in Chrome45:
var unload_event = "unload." + lock.id
function release_lock(sync) {
$.ajax({
method: "DELETE",
async: !sync,
data: lock,
url: lock.url,
error: function(){
console.log("failed to release lock " + JSON.stringify(lock));
},
success: function(){
console.log("lock " + lock.id + " released");
lock = null;
$(window).off(unload_event);
}
});
}
$(window).on(unload_event, function(){
release_lock(true);
});
It does generate the following warning in the console:
Synchronous XMLHttpRequest on the main thread is deprecated because of
its detrimental effects to the end user's experience.
For more help, check http://xhr.spec.whatwg.org/.
I would avoid doing this in the unload event due to the fact that synchronous ajax is the only way that will work, and synchronous ajax requests are deprecated in some modern browsers.
Alternatives include:
keepalive requests
This would involve periodically sending a request to the server indicating that the user is still editing the resource. The downside to this technique is that the resource will remain locked until the timeout happens, so if you're keepalive is set to an interval of 1 minute with a 3 minute lock timeout, it will remain locked for up to 3 minutes after the user has left the page. Additionally, if the user loses network connection for 3 minutes or longer, it will also become unlocked.
websockets
This would create an open connection between the client and the server, and while this connection is open, you can keep the resource locked. As soon as the client disconnects, you can assume that the client has closed the page and unlock it. The downside here is if the client loses network connection, it will also become unlocked.
In using Node.js to query some public APIs via HTTP requests. Therefore, I'm using the request module. I'm measuring the response time within my application, and see that my application return the results from API queries about 2-3 times slower than "direct" requests via curl or in the browser. Also, I noticed that connections to HTTPS enabled services usually take longer than plain HTTP ones, but this can be a coincidence.
I tried to optimize my request options, but to no avail. For example, I query
https://www.linkedin.com/countserv/count/share?url=http%3A%2F%2Fwww.google.com%2F&lang=en_US
I'm using request.defaults to set the overall defaults for all requests:
var baseRequest = request.defaults({
pool: {maxSockets: Infinity},
jar: true,
json: true,
timeout: 5000,
gzip: true,
headers: {
'Content-Type': 'application/json'
}
});
The actual request are done via
...
var start = new Date().getTime();
var options = {
url: 'https://www.linkedin.com/countserv/count/share?url=http%3A%2F%2Fwww.google.com%2F&lang=en_US',
method: 'GET'
};
baseRequest(options, function(error, response, body) {
if (error) {
console.log(error);
} else {
console.log((new Date().getTime()-start) + ": " + response.statusCode);
}
});
Does anybody see optimization potential? Am I doing something completely wrong? Thanks in advance for any advice!
There are several potential issues you'll need to address given what I understand from your architecture. In no particular order they are:
Using request will always be slower than using http directly since as the wise man once said: "abstraction costs". ;) In fact, to squeeze out every possible ounce of performance, I'd handle all HTTP requests using node's net module directly. For HTTPS, it's not worth rewriting the https module. And for the record, HTTPS will always be slower than HTTP by definition due to both the need to handshake cryptographic keys and do the crypt/decrypt work on the payload.
If your requirements include retrieving more than one resource from any single server, assure that those requests are made in order with the http KeepAlive set so you can benefit from the already open socket. The time it takes to handshake a new TCP socket is huge compared to making a request on an already open socket.
assure that http connection pooling is disabled (see Nodejs Max Socket Pooling Settings)
assure that your operating system and shell is not limiting the number of available sockets. See How many socket connections possible? for hints.
if you're using linux, check Increasing the maximum number of tcp/ip connections in linux and I'd also strongly recommend fine tuning the kernel socket buffers.
I'll add more suggestions as they occur to me.
Update
More on the topic of multiple requests to the same endpoint:
If you need to retrieve a number of resources from the same endpoint, it would be useful to segment your requests to specific workers that maintain open connections to that endpoint. In that way, you can be assured that you can get the requested resource as quickly as possible without the overhead of the initial TCP handshake.
TCP handshake is a three-stage process.
Step one: client sends a SYN packet to the remote server.
Step two: the remote server replies to the client with a SYN+ACK.
Step three: the client replies to the remote server with an ACK.
Depending on the client's latency to the remote server, this can add up to (as William Proxmire once said) "real money", or in this case, delay.
From my desktop, the current latency (round-trip time measure by ping) for a 2K octet packet to www.google.com is anywhere between 37 and 227ms.
So assuming that we can rely on a round-trip mean of 95ms (over a perfect connection), the time for the initial TCP handshake would be around 130ms or SYN(45ms) + SYN+ACK(45ms) + ACK(45ms) and this is a tenth of a second just to establish the initial connection.
If the connection requires retransmission, it could take much longer.
And this is assuming you retrieve a single resource over a new TCP connection.
To ameliorate this, I'd have your workers keep a pool of open connections to "known" destinations which they would then advertise back to the supervisor process so it could direct requests to the least loaded server with a "live" connection to the target server.
Actually, I have some new elements good enough to open a real answer. Having a look at the way request uses the HTTP agent can you please try the following :
var baseRequest = request.defaults({
pool: false,
agent: false,
jar: true,
json: true,
timeout: 5000,
gzip: true,
headers: {
'Content-Type': 'application/json'
}
});
This will disable connection pooling and should make it a lot faster.
I run a service where there is a javascript file that is called and self executed on a user's site.
This then calls an external server every 10 or so seconds with a bunch of variables.
I used to do this by using a createElement('script') and then setting the path to a file on the external server and passing the required variables across by means of GET variables. (works well for small URI's)
This worked really well and seemed to work cross browser as well with no undesired effects.
The problem I then ran into was when I needed to extend the amount or size of the variables that were being sent across. So obviously I decided to change from GET method to POST, but by doing that I could no longer use the createElement('script') trick and had to opt for the XMLHttpRequest() (ala Ajax - without jQuery) method which worked really well, except for the minor problem of having to also cater for Internet Explorer and Opera which didn't really play ball too well (big shock). So I used the following:
function createCORSRequest(method, url){
var xhr = new XMLHttpRequest();
if ("withCredentials" in xhr){
xhr.open(method, url, true);
} else if (typeof XDomainRequest != "undefined"){
xhr = new XDomainRequest();
xhr.open(method, url);
} else {
xhr = null;
}
return xhr;
}
var request = createCORSRequest("post", "http://xx.xxxx.com/");
if (request){
request.onload = function(){
//do something with request.responseText
};
request.send(myPostObjectDataVariableGoeshere);
}
..which I found over at this page
This is basically just a fallback to using the XDomainRequest() method which InternetExplorer wants you to use instead..
Fantastic, BUT -> Looking in the Console of Developer Tools in IE it says:
SEC7118: XMLHttpRequest for http://xx.xxxx.com/ required Cross Origin Resource Sharing (CORS).
SEC7120: Origin null not found in Access-Control-Allow-Origin header.
SCRIPT7002: XMLHttpRequest: Network Error 0x80070005, Access is denied.
But what's really odd about this is that I've already got the following as the first line in my backend PHP file that is being called (which works for other browsers...)
header('Access-Control-Allow-Origin: *');
Someone please tell me what's wrong here.. Also if there is a better way to be doing this instead of fighting the browser wars..
Note: I cannot use jQuery for this task!
You should try jQuery for this task. Its much easier and don't have that problem with IE.
http://api.jquery.com/jQuery.ajax/
IE unfortunately block Cross Origin requests, i believe there is no simple way to get around it by script only, but you can try tuning the options or via my proxy script.
Tuning the options
Internet Explorer ignores Access-Control-Allow headers and by default prohibits cross-origin access for Internet Zone. To enable CORS go to Tools->Internet Options->Security tab, click on “Custom Level” button. Find the Miscellaneous -> Access data sources across domains setting and select “Enable” option.
Proxy Script on local server as a Bridge
Previous post:
Remote POST request with jQuery and Ajax
This is for you to place a PHP script on a local server and do a local AJAX request and proxy to the remote server for good.