I have a function that do some HTTP requests, i need to do them often every certain amount of time (in this case 5 seconds) if a certain condition is triggered.
What happens is that sometimes the requests that are inside the setInterval loop takes more than the specified time and the loop triggers again, calling the request again, without waiting for the previous to resolve.
function doRequests() {
setInterval(async () => {
//Sometimes the following line takes more than 5 seconds to return its promise
const response = await myRequestFunction();
// do something with response...
if(/*certain condition is triggered*/)
{
//Call the function again
doRequests();
}
}, 5000);
}
doRequests();
I've already tried doing a recursive setTimeOut function like in this post, it worked but a few requests later it simply stopped working, not because a stack overflow happened but because IT SIMPLY STOPPED! The request was done, but it never brought the response back so the code stopped. And a few time before it stops it got slow.
I've also already tried using a setTimeOut inside a while loop but it seems that loops doesn't wait for intervals, not within this async context.
So the solution i need is: A way to every 5 seconds do the requests, but if it takes more than that to return the response, await for it.
Well, here is a solution in order to keep the cycle period as most regular as possible. As long the responses arrive within the 5s timeout, the cycle polling should be pretty regular. However, if a response comes in late, the new request is made immediately it is processed.
I also added a try-catch block, because it is likely a chance having some error on a HTTP request (and also because you're doing other stuffs in the while). This leads you to decide how to behave in such a cases.
async function doRequests() {
let answered = false;
let expired = false;
//start the timer
let tmr = setTimeout(function() {
if (answered) {
//response already received (and processed), so start a new request
doRequest();
}
else {
//mark the timer as expired
expired = true;
},
5000
);
try {
const response = await myRequestFunction();
// do something with response...
if(/*certain condition is triggered*/)
{
answered = true;
if (expired) {
//Call the function again
doRequests();
}
}
}
catch (err) {
//error trapped: clear timeout
clearTimeout(tmr);
if (/* decide to continue as it was fine */) {
answered = true;
if (expired) {
//Call the function again
doRequests();
}
}
}
}
doRequests();
Related
in my Javascript application the user can press a button, in order to continuously fetch data from a server.
Pressing the button triggers a function, which performs a HTTP request to the server. After the request is completed successfully, this function calls itself recursively again and again. The function is escaped if the user presses the button again or a specific limit is reached.
Now my problem is, that by each recursion the delay between each request gets bigger. This can be visualized in the screenshot I took from the developer tools of my browser:
The blue lines are my sets of requests in each function. The white empty slots between the sets of requests get longer.
This is how my code is structured:
async function startContinousFetch() {
addContinousData(true);
}
async function addContinousData() {
addNewData().then((resp) => {
if (LOGSLIMIT != null && Logs.length > LOGSLIMIT) {
stopContinousFetch();
return;
}
//Unexpected error, promise didn't resolve with true;
if (resp == false) {
//Error handling
}
if (continousFetch) {
addContinousData();
}
})
}
function addNewData() {
return new Promise(async resolve => {
try {
//Various http requests are here
await fetchData();
//Add, organize and manipulate data in the global "Logs" array
handleLogs();
//Resolve promise
resolve(true)
} catch (e) {
resolve(false, e)
}
})
}
Why do the delays get longer? What is my mistake here?
It is clear to me that my function waits till the response, so there is a delay. But why does this delay get bigger every time.
In the dev tools, the request time does not get any longer, just the time in between.
Thanks a lot.
I am reading data in realtime from a device through http requests, however the way I am currently doing it is, something like this:
setInterval(() => {
for (let request of requests) {
await request.GetData();
}
}, 1000);
However, sometimes there is lag in the network, and since there are 4-5 requests, sometimes they don't all finish within a second, so they start stacking up, until the device eventually starts to timeout, so I need to somehow get rid of the setInterval. Increasing the time is not an option.
Essentially, I want them to get stuck in an infinite loop and I can add an inner timer to let the requests run again when half a second or a second has passed since the last run, but how do I get them stuck in an infinite loop without blocking the rest of the application?
Or maybe a way to make setInterval wait for all requests to finish before starting to count the 1 second interval?
Try:
(async () => {
while (true) {
for (let request of requests) {
await request.GetData();
}
await new Promise((resolve) => setTimeout(resolve, 1000));
}
})();
This will only start waiting once all the requests are finished, preventing them from stacking up.
Alternatively, on the inside of the async function, use:
while (true) {
await Promise.all(requests.map(request => request.GetData()));
await new Promise((resolve) => setTimeout(resolve, 1000));
}
This is different because all the calls to request.GetData() will run concurrently, which may or may not be what you want.
My code is something like this:
var trueOrFalse = true;
while(trueOrFalse){
fetch('some/address').then(){
if(someCondition){
trueOrFalse = false;
}
}
}
But I can not issue the fetch request. It seems like while loop is schedule many fetches into next tick. But never skip to next tick.
How can I solve the problem?
while(true) creates an infinite loop, which will attempt to call fetch infinitely many times within a single "tick". Since it never finishes issuing new fetch calls, it never gets to the next tick.
This function is very CPU-intensive and will probably lock up the entire page.
What's the solution?
What you are probably trying to do is keep fetching until the result satisfies some condition. You can achieve that by checking the condition in the then callback, and re-issuing the fetch if it is false:
var resultFound = false;
var fetchNow = function() {
fetch('some/address').then(function() {
if(someCondition) {
resultFound = true;
}
else {
fetchNow();
}
});
}
fetchNow();
This way, instead of
fetch!
fetch!
fetch!
fetch!
...
...the behavior is going to be
fetch!
wait for response
check condition
if false, fetch!
wait for response
check condition
if true, stop.
...which is probably what you expected.
Now with async/await we can use awesome while loops to do cool stuff.
var getStuff = async () => {
var pages = 0;
while(true) {
var res = await fetch(`public/html/${pages ++}.html`);
if(!res.ok) break; //Were done let's stop this thing
var data = await res.text();
//Do something with data
};
console.log("Look ma! I waited!"); //Wont't run till the while is done
};
while loop is sync where fetch is async in nature, so while won't wait for fetch async operation to complete and going to next iteration immediately.
You can achieve this synchronously like following:
function syncWhile(trueOrFalse){
if(trueOrFalse) {
fetch('some/address').then(){
if(someCondition){
trueOrFalse = false;
}
syncWhile(trueOrFalse);
}
}
}
syncWhile(true);
The while loop fires off all the fetches before any one of them will reach the then(), so a while loop is incorrect here, even useless I would say.
You need to make the then() responsible for whether to continue fetching or not.
It also seems that your then()-syntax is wrong (maybe just an error editing the example). Furthermore you can omit the boolean helper variable (unless perhaps you need it in some other place).
function fetchUntilCondition(){
fetch('some/address').then(function(response){
if(!someCondition) {
fetchUntilCondition(); // fetch again
}
});
}
fetchUntilCondition();
I have a Node server that uses Connect to insert some middleware which attempt to transform a response stream from node-http-proxy. Occasionally this transformation can be quite slow and it would be preferable in such cases to simply return a response that doesn't include the transformations or alternatively includes their partial application.
In my application I've attempted to use setTimeout to call next after some number of milliseconds in the context of the transformation middleware. This generally works but exposes a race condition where if the middleware has already called next and then setTimeout fires and does the same an error occurs that looks like: Error: Can't set headers after they are sent.
Eventually I evolved the setTimeout to invoke next with an Error instance as its first argument and then later on in my middleware chain would catch that error and assuming res.headersSent was false would start sending the response via res.end.call(res). This worked and surprisingly I could set the timeout to nearly nothing and the response would happen significantly faster and be complete.
I feel like this last method is a bit of a hack and not immune from the same race condition, but perhaps appears to be a little more resilient. So I would like to know what sort of idiomatic approaches Node and Connect have for dealing with this kind of thing.
How can I go about timing out slow middleware and simply return the response stream?
Currently this seems to do what I want, more or less, but again feels a bit gross.
let resTimedout = false;
const timeout = setTimeout(() => {
if (!resTimedout) {
resTimedout = true;
next();
}
}, 100);
getSelectors(headers, uri, (selectors) => {
const resSelectors = Object.keys(selectors).map((selector) => {
...
};
const rewrite = resRewrite(resSelectors);
rewrite(req, res, () => {
if (!resTimedout) {
resTimedout = true;
clearTimeout(timeout);
next();
}
});
});
setTimeout returns the id of the timeout, so you can then run clearTimeout passing in the id. So when the transformation is complete just clear the timeout before you call next.
var a = setTimeout(()=>{}, 3000);
clearTimeout(a);
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout
Use async.timeout or Promise.timeout from Bluebird or Q library
You could eliminate the need for global variables and decide this per request:
const rewrite = resRewrite(resSelectors);
rewrite(req, res, () => {
// set a timer to fail the function early
let timer = setTimeout(() => {
timer = null;
next();
}, 100);
// do the slow response transformation
transformResponse((err, data) => { // eg. callback handler
clearTimeout(timer);
if (timer) next();
});
});
How it works
If the timer ends first, it sets itself to null and calls next(). When the transform function ends, it will see the timeout is null and not call next().
If the response transform is faster, it clears the timeout to prevent it running next later on.
Is it possible, in node.js, to make an asynchronous call that times out if it takes too long (or doesn't complete) and triggers a default callback?
The details:
I have a node.js server that receives a request and then makes multiple requests asynchronously behind the scenes, before responding. The basic issue is covered by an existing question, but some of these calls are considered 'nice to have'. What I mean is that if we get the response back, then it enhances the response to the client, but if they take too long to respond it is better to respond to the client in a timely manner than with those responses.
At the same time this approach would allow to protect against services that simply aren't completing or failing, while allowing the main thread of operation to respond.
You can think of this in the same way as a Google search that has one core set of results, but provides extra responses based on other behind the scenes queries.
If its simple just use setTimout
app.get('/', function (req, res) {
var result = {};
// populate object
http.get('http://www.google.com/index.html', (res) => {
result.property = response;
return res.send(result);
});
// if we havent returned within a second, return without data
setTimeout(function(){
return res.send(result);
}, 1000);
});
Edit: as mentioned by peteb i forgot to check to see if we already sent. This can be accomplished by using res.headerSent or by maintaining a 'sent' value yourself. I also noticed res variable was being reassigned
app.get('/', function (req, res) {
var result = {};
// populate object
http.get('http://www.google.com/index.html', (httpResponse) => {
result.property = httpResponse;
if(!res.headersSent){
res.send(result);
}
});
// if we havent returned within a second, return without data
setTimeout(function(){
if(!res.headersSent){
res.send(result);
}
}, 1000);
});
Check this example of timeout callback https://github.com/jakubknejzlik/node-timeout-callback/blob/master/index.js
You could modify it to do action if time's out or just simply catch error.
You can try using a timeout. For example using the setTimeout() method:
Setup a timeout handler: var timeOutX = setTimeout(function…
Set that variable to null: timeOutX = NULL (to indicate that the timeout has been fired)
Then execute your callback function with one argument (error handling): callback({error:'The async request timed out'});
You add the time for your timeout function, for example 3 seconds
Something like this:
var timeoutX = setTimeout(function() {
timeOutX = null;
yourCallbackFunction({error:'The async request timed out'});
}, 3000);
With that set, you can then call your async function and you put a timeout check to make sure that your timeout handler didn’t fire yet.
Finally, before you run your callback function, you must clear that scheduled timeout handler using the clearTimeout() method.
Something like this:
yourAsyncFunction(yourArguments, function() {
if (timeOutX) {
clearTimeout(timeOutX);
yourCallbackFunction();
}
});