I'm writing my first JavaScript Chrome Extension using JQuery 2.2.0, which basically takes the current URL and polls a few different web services to see if they have a record of the URL. If it exists, I add a text link in the DOM. Here's a simplified working version:
// Does the provided URL exist?
function url_exists(url) {
var h = new XMLHttpRequest();
h.open('HEAD', url, false);
h.send();
return h.status!=404;
}
// Display a link to the database record URL in the DOM
function display_database_link(url) {
$('body').prepend('Link');
}
// Get the current URL
var url = window.location.href;
var database_number = 0;
// See if this URL exists in one of our databases via the API
// Does the URL exist in database 1?
if (url_exists("https://api.database1.com/urls/" + url)) {
database_number = 1;
}
// Does the URL exist in database 2?
else if (url_exists("https://api.database2.com/urls/" + url)) {
database_number = 2;
}
if (database_number > 0) {
display_database_link("https://api.database" + database_number + ".com/urls/" + url))
}
What I have works, but I'm wondering if:
there's a way to make multiple calls to url_exists at once, and
if there's a way to do this asynchronously.
If someone could respond with a link to relevant documentation or examples, I'd really appreciate it!
There are a couple of awesome es2015 features that will make this nice and easy: fetch and promises. You'll have to do some tweaking but something like this should work.
// Array of API endpoints to to make requests to
let url = window.location.href;
let database_urls = ["https://api.database1.com/urls/", "https://api.database2.com/urls/"];
// Promise.all will take an array of promises and perform all of them, and `then` process all results at the end
Promise.all(database_urls.map(database_url =>
// Make an HTTP request to the endpoint, and `then` get the status code from the response
fetch(database_url + url).then(response => response.status)
// Once `all` promises are resolved, `then` iterate over the resulting statuses
)).then(statuses => {
// Reduce `statuses` to a count of how many are not 404s
let existCount = statuses.reduce((prev, curr) => {return curr == 404 ? prev : prev + 1}, 0);
// Call method to display link if existsCount is > 0
});
Yes, you can make all of them at once, the browser will execute them at once but the results will take diferente amounts of time to reach you...
You have: h.open('HEAD', url, false); the third parameter (false) defines if the request is asynchronous or synchronous.... It should be set to true: http://www.w3schools.com/ajax/ajax_xmlhttprequest_send.asp
Since you have jQuery why not use the AJax function? http://api.jquery.com/jquery.ajax/
Be careful, if your extension is making 1000 requests to 1000 databases it will slow down the user browser/internet.
This requests take X amount of time until you get a response back, so don't immediately check if database_number is greater than 0.
Related
I have two websites, the first get web source code of the second (The two websites aren't one same host -> CORS). (The second website is not mine)
Example:
fetch("https://api.allorigins.win/get?url=" + url)
.then(response => {
if (response.ok) {
return response.json();
}
throw new Error('Network response was not ok.');
})
.then(data => {
var html = stringToHTML(data.contents);
});
It works, except that the concern is that the second page displays other elements several seconds after being displayed, so it does not display me because I retrieved the page too early.
How to make it wait a few seconds before recovering, while not forgetting that "api.allorigins.win"?
Do you have an idea? ( I use Vanillia JS )
Allorigins that must wait for rendering, but it does not.
Your alternative is to implement your own version of allorigins using an headless browser that waits the page render before returning it html. There's no ready solution for it
I don't know if you're using a framework or any library to handle the DOM, but with vanilla JS you can do something like this to check if your DOM element is ready
const DOMReadyCheck = setInterval(() => {
if (document.getElementsBy...) { //get your element here
//send your fetch request and set your element with data
clearInterval(DOMReadyCheck);
}
Link to a similar problem that has no answers, but written in C
I'm using NodeJS to parse output from ark-server-tools, which is a layer on top of SteamCMD. What I'd like to do is parse the progress of the update and assign it to a variable, which I'll return as a GET call that the client can use to check progress of the update.
I put the log results of an update into a file to run my code against, which I've put in a PasteBin for brevity.
update.js
app.get('/update', function(req, res) {
var toReturn;
var outputSoFar;
var total;
var startPos;
var endPos = 0;
//var proc = spawn('arkmanager', ['update', '--safe']);
var proc = spawn('./update-log.sh'); //for testing purposes
proc.stdout.on('data', function(data){
outputSoFar += data.toString();
//if server is already updated
if (outputSoFar.indexOf('Your server is already up to date!') !== -1) {
toReturn = 'Server is already up-to-date.';
}
//find update progress
if (outputSoFar.indexOf('progress:') !== -1) {
for(var line in outputSoFar.split('\n')){
console.log('found progress');
startPos = outputSoFar[line].indexOf('progress:', endPos) + 10; //get the value right after progress:_, which should be a number
endPos = outputSoFar[line].indexOf(' (', startPos); // find the end of this value, which is signified by space + (
console.log(outputSoFar[line].substring(startPos, endPos).trim());
updatePercent = outputSoFar[line].substring(startPos, endPos).trim(); //returned to the `checkUpdateProgress` endpoint
}
toReturn = 'Updating...';
}
});
proc.stderr.on('data', function(data){
console.log(data);
});
proc.on('close', function (code, signal) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(JSON.stringify(toReturn));
res.end();
});
}
/*
* Returns progress of an update
*/
app.get('/updateProgress', function(req, res){
console.log('updatePercent: ' + updatePercent);
res.send(JSON.stringify(updatePercent));
});
Couple questions:
1) Is this the best way to architect my RESTful API? One call for the action of updating and another for checking the progress of the update?
2) I'd love a better way to test the function, as echoing the console log returns the data in one piece, as opposed to a data stream. How do I do this?
3) I'm pretty sure the parsing function itself isn't quite right, but I'm having a hard time testing it because of #2.
If you want to take a look at the project in its entirety, here's the repo.
Thanks in advance for your help!
For one of your questions:
Is this the best way to architect my RESTful API? One call for the
action of updating and another for checking the progress of the
update?
As implemented now, I don't think your service can support concurrent requests correctly. updatePercent is a shared global variable. If i hit /update endpoint with a single client, it will start the ./update-log.sh command.
If I request /update again, it will start another update and overwrite the global updateProgress. There doesn't seem to be anything mapping an updatePercent with the correct process
Additionally, there could be serious performance issues to each request spawning a new process. Node might be able to handle hundreds or thousands of concurrent connections using a single thread, but each request is going to spawn a new process, just something to profile
I am trying to match a token (string token) in the RSS feed using casperjs waitFor() but it does not seem to work. There are other ways (not using polling) to get around but I need to poll for it. Here is the code snippet:
casper.then(function() {
this.waitFor(function matchToken() {
return this.evaluate(function() {
if(!this.resourceExists(token)) {
this.reload();
return false;
}
return true;
});
});
});
The updates to rss url are not dynamic and hence, a refresh would be needed to check for the token. But it seems (from the access log) that I am not getting any hits (reload not working) on the rss url. Ideally, I would want to refresh the page if it doesn't see the token and then check for the token again & it should keep doing that until the waitFor times out.
I also tried using assertTextExists() instead of resourceExists() but even that did not work.
I am using PhantomJS (1.9.7) & the url is: https://secure.hyper-reach.com:488/rss/323708
The token I am looking for is --> item/272935. If you look at the url I have mentioned above, you will find this in a each guid tag. The reason why I am including "item/" also as a part of my token is so that it doesn't match any other numbers incorrectly.
evaluate() is the sandboxed page context. Anything inside of it doesn't have access to variables defined outside and this refers to window of the page and not casper. You don't need the evaluate() function here, since you don't access the page context.
The other thing is that casper.resourceExists() works on the resource meta data such as URL and request headers. It seems that you want to check the content of the resource. If you used casper.thenOpen() or casper.open() to open the RSS feed, then you can check with casper.getPageContent(), if the text exists.
The actual problem with your code is that you mix synchronous and asynchronous code in a way that won't work. waitFor() is the wrong tool for the job, because you need to reload in the middle of its execution, but the check function is called so fast that there probably won't be a complete page load to actually test it.
You need to recursively check whether the document is changed to your liking.
var tokenTrials = 0,
tokenFound = false;
function matchToken(){
if (this.getPageContent().indexOf(token) === -1) {
// token was not found
tokenTrials++;
if (tokenTrials < 50) {
this.reload().wait(1000).then(matchToken);
}
} else {
tokenFound = true;
}
}
casper.then(matchToken).then(function(){
test.assertTrue(tokenFound, "Token was found after " + tokenTrials + " trials");
});
Disclaimer
Firstly, a disclaimer: I am working within specific boundaries, so whilst it may seem I'm going about something the long way round, I am limited as to what I can do. I know I should be doing this entirely differently, but I cannot. If it's not possible to do what I'm trying to do here, then that's fine, I just need to know.
Background
Basically, this boils down to a cross-domain javascript call. However, I need to wait for the response before returning the method.
Say I have a page - example1.com/host.html. This contains a javascript method of 'ProvideValue()' which returns an int. Edit: This method must be executed where it is found, since it may need to access other resources within that domain, and access global variables set for the current session.
https://example1.com/host.html
function ProvideValue(){
return 8; // In reality, this will be a process that returns a value
}
This host.html page contains an iframe pointing to example2.com/content.html (note the different domain). This content.html page contains a method that needs to display the value from host.html in an alert.
https://example2.com/content.html
function DisplayValue(){
var hostValue = //[get value from ProvideValue() in host.html]
alert(hostValue);
}
That's it.
Limitations
I can run any javascript I like on the host.html, but nothing server-side. On content.html I can run javascript and anything server-side. I have no control over the example1.com domain, but full control over example2.com.
Question
How can I retrieve the value from ProvideValue() on example1.com/host.html within the DisplayValue() method on example2.com/content.html?
Previous Attempts
Now, I've tried many of the cross-domain techniques, but all of them (that I've found) use an asynchronous callback. That won't work in this case, because I need to make the request to the host.html, and receive the value back, all within the scope of a single method on the content.html.
The only solution I got working involved relying on asynchronous cross-domain scripting (using easyXDM), and a server-side list of requests/responses in example2.com. The DisplayValue() method made the request to host.html, then immediately made a synchronous post to the server. The server would then wait until it got notified of the response from the cross-domain callback. Whilst waiting, the callback would make another call to the server to store the response. It worked fine in FireFox and IE, but Chrome wouldn't execute the callback until DisplayValue() completed. If there is no way to address my initial question, and this option has promise, then I will pose this as a new question, but I don't want to clutter this question with multiple topics.
Use XMLHttpRequest with CORS to make synchronous cross-domain requests.
If the server doesn't support cors, use a proxy which adds the appropriate CORS headers, e.g. https://cors-anywhere.herokuapp.com/ (source code at https://github.com/Rob--W/cors-anywhere).
Example 1: Using synchronous XHR with CORS
function getProvidedValue() {
var url = 'http://example.com/';
var xhr = new XMLHttpRequest();
// third param = false = synchronous request
xhr.open('GET', 'https://cors-anywhere.herokuapp.com/' + url, false);
xhr.send();
var result = xhr.responseText;
// do something with response (text manipulation, *whatever*)
return result;
}
Example 2: Use postMessage
If it's important to calculate the values on the fly with session data, use postMessage to continuously update the state:
Top-level document (host.html):
<script src="host.js"></script>
<iframe name="content" src="https://other.example.com/content.html"></iframe>
host.js
(function() {
var cache = {
providedValue: null,
otherValue: ''
};
function sendUpdate() {
if (frames.content) { // "content" is the name of the iframe
frames.content.postMessage(cache, 'https://other.example.com');
}
}
function recalc() {
// Update values
cache.providedValue = provideValue();
cache.otherValue = getOtherValue();
// Send (updated) values to frame
sendUpdate();
}
// Listen for changes using events, pollers, WHATEVER
yourAPI.on('change', recalc);
window.addEventListener('message', function(event) {
if (event.origin !== 'https://other.example.com') return;
if (event.data === 'requestUpdate') sendUpdate();
});
})();
A script in content.html: content.js
var data = {}; // Global
var parentOrigin = 'https://host.example.com';
window.addEventListener('message', function(event) {
if (event.origin !== parentOrigin) return;
data = event.data;
});
parent.postMessage('requestUpdate', parentOrigin);
// To get the value:
function displayValue() {
var hostName = data.providedValue;
}
This snippet is merely a demonstration of the concept. If you want to apply the method, you probably want to split the login in the recalc function, such that the value is only recalculated on the update of that particular value (instead of recalculating everything on every update).
I want to run a JavaScript code to ping 4 different IP addresses and then retrieve the packet loss and latency of these ping requests and display them on the page.
How do I do this?
You can't do this from JS. What you could do is this:
client --AJAX-- yourserver --ICMP ping-- targetservers
Make an AJAX request to your server, which will then ping the target servers for you, and return the result in the AJAX result.
Possible caveats:
this tells you whether the target servers are pingable from your server, not from the user's client
so the client won't be able to test hosts its LAN
but you shouldn't let the host check hosts on the server's internal network, if any exist
some hosts may block traffic from certain hosts and not others
you need to limit the ping count per machine:
to avoid the AJAX request from timing out
some site operators can get very upset when you keep pinging their sites all the time
resources
long-running HTTP requests could run into maximum connection limit of your server, check how high it is
many users trying to ping at once might generate suspicious-looking traffic (all ICMP and nothing else)
concurrency - you may wish to pool/cache the up/down status for a few seconds at least, so that multiple clients wishing to ping the same target won't launch a flood of pings
The only method I can think of is loading e.g. an image file from the external server. When that load fails, you "know" the server isn't responding (you actually don't know, because the server could just be blocking you).
Take a look at this example code to see what I mean:
/*note that this is not an ICMP ping - but a simple HTTP request
giving you an idea what you could do . In this simple implementation it has flaws
as Piskvor correctly points out below */
function ping(extServer){
var ImageObject = new Image();
ImageObject.src = "http://"+extServer+"/URL/to-a-known-image.jpg"; //e.g. logo -- mind the caching, maybe use a dynamic querystring
if(ImageObject.height>0){
alert("Ping worked!");
} else {
alert("Ping failed :(");
}
}
I was inspired by the latest comment, so I wrote this quick piece of code.
This is a kind of "HTTP ping" which I think can be quite useful to use along with XMLHttpRequest calls(), for instance to figure out which is the fastest server to use in some case or to collect some rough statistics from the user's internet connexion speed.
This small function is just connecting to an HTTP server on an non-existing URL (that is expected to return a 404), then is measuring the time until the server is answering to the HTTP request, and is doing an average on the cumulated time and the number of iterations.
The requested URL is modified randomely at each call since I've noticed that (probably) some transparent proxies or caching mechanisms where faking results in some cases, giving extra fast answers (faster than ICMP actually which somewhat weird).
Beware to use FQDNs that fit a real HTTP server!
Results will display to a body element with id "result", for instance:
<div id="result"></div>
Function code:
function http_ping(fqdn) {
var NB_ITERATIONS = 4; // number of loop iterations
var MAX_ITERATIONS = 5; // beware: the number of simultaneous XMLHttpRequest is limited by the browser!
var TIME_PERIOD = 1000; // 1000 ms between each ping
var i = 0;
var over_flag = 0;
var time_cumul = 0;
var REQUEST_TIMEOUT = 9000;
var TIMEOUT_ERROR = 0;
document.getElementById('result').innerHTML = "HTTP ping for " + fqdn + "</br>";
var ping_loop = setInterval(function() {
// let's change non-existent URL each time to avoid possible side effect with web proxy-cache software on the line
url = "http://" + fqdn + "/a30Fkezt_77" + Math.random().toString(36).substring(7);
if (i < MAX_ITERATIONS) {
var ping = new XMLHttpRequest();
i++;
ping.seq = i;
over_flag++;
ping.date1 = Date.now();
ping.timeout = REQUEST_TIMEOUT; // it could happen that the request takes a very long time
ping.onreadystatechange = function() { // the request has returned something, let's log it (starting after the first one)
if (ping.readyState == 4 && TIMEOUT_ERROR == 0) {
over_flag--;
if (ping.seq > 1) {
delta_time = Date.now() - ping.date1;
time_cumul += delta_time;
document.getElementById('result').innerHTML += "</br>http_seq=" + (ping.seq-1) + " time=" + delta_time + " ms</br>";
}
}
}
ping.ontimeout = function() {
TIMEOUT_ERROR = 1;
}
ping.open("GET", url, true);
ping.send();
}
if ((i > NB_ITERATIONS) && (over_flag < 1)) { // all requests are passed and have returned
clearInterval(ping_loop);
var avg_time = Math.round(time_cumul / (i - 1));
document.getElementById('result').innerHTML += "</br> Average ping latency on " + (i-1) + " iterations: " + avg_time + "ms </br>";
}
if (TIMEOUT_ERROR == 1) { // timeout: data cannot be accurate
clearInterval(ping_loop);
document.getElementById('result').innerHTML += "<br/> THERE WAS A TIMEOUT ERROR <br/>";
return;
}
}, TIME_PERIOD);
}
For instance, launch with:
fp = new http_ping("www.linux.com.au");
Note that I couldn't find a simple corelation between result figures from this script and the ICMP ping on the corresponding same servers, though HTTP response time seems to be roughly-exponential from ICMP response time. This may be explained by the amount of data that is transfered through the HTTP request which can vary depending on the web server flavour and configuration, obviously the speed of the server itself and probably other reasons.
This is not very good code but I thought it could help and possibly inspire others.
The closest you're going to get to a ping in JS is using AJAX, and retrieving the readystates, status, and headers. Something like this:
url = "<whatever you want to ping>"
ping = new XMLHttpRequest();
ping.onreadystatechange = function(){
document.body.innerHTML += "</br>" + ping.readyState;
if(ping.readyState == 4){
if(ping.status == 200){
result = ping.getAllResponseHeaders();
document.body.innerHTML += "</br>" + result + "</br>";
}
}
}
ping.open("GET", url, true);
ping.send();
Of course you can also put conditions in for different http statuses, and make the output display however you want with descriptions etc, to make it look nicer. More of an http url status checker than a ping, but same idea really. You can always loop it a few times to make it feel more like a ping for you too :)
I've come up with something cause I was bored of searching hours after hours for something that everyone is saying "impossible", only thing I've found was using jQuery.
I've came up with a new simple way using Vanilla JS (nothing else than base JavaScript).
Here's my JSFiddle: https://jsfiddle.net/TheNolle/5qjpmrxg/74/
Basically, I create a variable called "start" which I give the timestamp, then I try to set an invisible image's source to my website (which isn't an image) [can be changed to any website], because it's not an image it creates an error, which I use to execute the second part of the code, at this time i create a new variable called "end" which i give the timestamp from here (which is different from "start"). Afterward, I simply make a substraction (i substract "start" from "end") which gives me the latency that it took to ping this website.
After you have the choice you can store that in a value, paste it on your webpage, paste it in the console, etc.
let pingSpan = document.getElementById('pingSpan');
// Remove all the way to ...
let run;
function start() {
run = true;
pingTest();
}
function stop() {
run = false;
setTimeout(() => {
pingSpan.innerHTML = "Stopped !";
}, 500);
}
// ... here
function pingTest() {
if (run == true) { //Remove line
let pinger = document.getElementById('pingTester');
let start = new Date().getTime();
pinger.setAttribute('src', 'https://www.google.com/');
pinger.onerror = () => {
let end = new Date().getTime();
// Change to whatever you want it to be, I've made it so it displays on the page directly, do whatever you want but keep the "end - start + 'ms'"
pingSpan.innerHTML = end - start + "ms";
}
setTimeout(() => {
pingTest();
}, 1000);
} // Remove this line too
}
body {
background: #1A1A1A;
color: white
}
img {
display: none
}
Ping:
<el id="pingSpan">Waiting</el>
<img id="pingTester">
<br> <br>
<button onclick="start()">
Start Ping Test
</button>
<button onclick="stop()">
Stop
</button>
function ping(url){
new Image().src=url
}
Above pings the given Url.
Generally used for counters / analytics.
It won't encounter failed responses to client(javascript)
I suggest using "head" to request the header only.
xhr.open('head', 'asstes/imgPlain/pixel.txt' + cacheBuster(), true);
and than ask for readystate 2 - HEADERS_RECEIVED send() has been called, and headers and status are available.
xhr.onreadystatechange = function() {
if (xhr.readyState === 2) { ...
Is it possible to ping a server from Javascript?
Should check out the above solution. Pretty slick.
Not mine, obviously, but wanted to make that clear.
You can't PING with Javascript. I created Java servlet that returns a 10x10 pixel green image if alive and a red image if dead. https://github.com/pla1/Misc/blob/master/README.md