Context:
I have a while loop that i want to run 10 times and within i have async/awaait code that runs every 3 seconds. The while loop works as sort of a timeout if it ever runs 10 times and the async/await check doesnt return the expected value then, break out of the while loop the process timed out.
Problem: The break out of loop portion of the code is running first with the value of i(loop variable) maxed out. As i figure the way i have it setup i have no access to the value of i while is looping and only when i is at its max value.
Question: How can i escape out of this loop early when condition is met or if i is exhausted?
var i = 0;
//Run 10 Times
while (i < 10) {
//Run every 3 seconds
((i) => {
setTimeout( async () => {
isAuthenticated = await eel.is_authenticated()();
sessionStorage.sessionStatus = JSON.stringify(isAuthenticated);
console.log('------NEW STATUS-------');
console.log(JSON.parse(sessionStorage.sessionStatus).authenticated);
console.log('Inside:' + i);
}, 3000 * i)
})(i++);
//Break out of loop early if condition is met or I is exhausted, but it only runs 1 time and i is always max
if (i === 9 || JSON.parse(sessionStorage.sessionStatus).authenticated) {
console.log('Outside:' + i);
checkStatus('retried');
break;
}
}
NOTE: In case anyone is wondering eel.is_authenticated()(); is not a typo, its a python library to create desktop applications the double ()() is normal.
Also if this approach is overly complicated for what it does, any other ways to approach it are welcome :)
Thanks
The issue here is that you are running through all your loop iterations immediately (10 times), setting up 10 timeouts in the process, 3s apart from each other:
Your loop runs 10 times, creating 10 timeouts
You reach the i === 9 case
3s later, 1st timeout runs
3s later, 2nd timeout runs
...
What you want is for the loop to actually wait 3s between iterations. For that, you'd use setTimeout in a different way - you'd create a promise that resolves when the timeout is hit and await that promise:
// As helper function for readability
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
// Your loop
let i;
for (i = 0; i < 10; i++) {
// Wait 3s
await delay(3000);
// Check authentication
const isAuthenticated = await eel.is_authenticated()();
sessionStorage.sessionStatus = JSON.stringify(isAuthenticated);
console.log('------NEW STATUS-------');
console.log(JSON.parse(sessionStorage.sessionStatus).authenticated);
console.log('Inside:' + i);
// Break loop if authenticated
if (isAuthenticated.authenticated) break;
}
// We were authenticated, or looped 10 times
// Note: Because I moved this outside, i will now actually be 10 if the loop
// ended by itself, not 9.
console.log('Outside:' + i);
checkStatus('retried');
One consequence here though would be that if the call to is_authenticated takes a significant amount of time, the checks will be more than 3s apart, because we are now waiting for 3s and for this call. If this is undesired, we can reduce the delay time based on how much time elapsed since the last call:
// As helper function for readability
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
// We will save here when the last delay completed, so that the checks are always
// 3s apart (unless the check takes longer than 3s)
// Initially we save the current time so that the first wait is always 3s, as before
let lastIteration = Date.now();
// Your loop
let i;
for (i = 0; i < 10; i++) {
// Wait until 3s after last iteration (limited to 0ms, negative waits won't work)
await delay(Math.max(lastIteration + 3000 - Date.now(), 0));
// Update "last iteration" time so the next delay will wait until 3s from now again
lastIteration = Date.now();
// Check authentication
const isAuthenticated = await eel.is_authenticated()();
sessionStorage.sessionStatus = JSON.stringify(isAuthenticated);
console.log('------NEW STATUS-------');
console.log(JSON.parse(sessionStorage.sessionStatus).authenticated);
console.log('Inside:' + i);
// Break loop if authenticated
if (isAuthenticated.authenticated) break;
}
// We were authenticated, or looped 10 times
// Note: Because I moved this outside, i will now actually be 10 if the loop
// ended by itself, not 9.
console.log('Outside:' + i);
checkStatus('retried');
All of this assumes that the function in which this code is located is async. If it isn't, you need to make it async but then you need to remember adding a .catch(e => handleTheErrorSomehow(e)) when you call it, to avoid unhandled promise rejections!
Related
When I run this code in Firefox and Chrome, the results are different:
function run() {
setTimeout(() => console.log("1"), 0);
setTimeout(() => console.log("2"), 100);
let start = Date.now();
while (Date.now() - start < 200) {
// do nothing
}
setTimeout(() => {
console.log("3");
}, 0);
start = Date.now();
while (Date.now() - start < 200) {
// do nothing
}
setTimeout(() => {
console.log("4");
}, 0);
}
run();
In Chrome (and Node.js), this is printed:
1
3
2
4
In Firefox, this is printed:
1
2
3
4
But if I remove the line 2 (setTimeout(() => console.log("1"), 0);), then the same thing is printed on every platform:
2
3
4
How to explain these different results?
Thanks!
The explanation: It doesn't matter.
The details of when deferred "messages" are added to the event loop message queue are implementation details, not documented guarantees. By the time your function yields control back to the event loop, all of your setTimeout call are eligible to execute (three of them were scheduled to run immediately, one of them was scheduled to run in 100 ms) and you've guaranteed it's been at least 400 ms since you scheduled it.
The difference between the two could be as simple as whether they choose to look for deferred tasks that have become ready (to move from the deferred queue to the main "ready to go" message queue) immediately before or immediately after new items are inserted in the main message queue. Chrome chooses to move immediately after 3 is scheduled (so 3 goes in, then the deferred 2), Firefox immediately before (moving in 2 before it puts 3 in).
Both of them could change in the next release without violating any documented guarantees. Don't rely on it, don't expect it to be stable. While immediately scheduled tasks are guaranteed to execute in FIFO order, there are no guarantees on when deferred tasks get moved onto the "ready-to-go" message queue. The spec seems to requires that 1, 3 and 4 execute in that order (since they were all immediately ready, not deferred), with only the ordering of 2 being flexible, but even that isn't a true guarantee; it can get weird with the various ways in which an "immediate" setTimeout task may not actually be scheduled immediately.
You may be interested in the MDN docs on why setTimeout can take longer than expected; it explains by side-effect a lot of how the event loop works, even as it carefully provides no guarantees on the details you're exploring.
I can't give you an full detailed explanation, but the second paramter of setTimeoput and setInterval doesn't mean, it will exactly execute it at that time. They will put it in a queue, so the background can execute it.
The browser has a lifecycle when to execute specific steps to update the data and the styles.
I can only send you this youtube link, that helped me to learn more about it:
https://www.youtube.com/watch?v=MCi6AZMkxcU
1, 2, 3, 4 is the behavior that is expected.
The specs ask to
Wait until any invocations of this algorithm that had the same global and orderingIdentifier, that started before this one, and whose milliseconds is equal to or less than this one's, have completed.
So any call to setTimeout that were both made before, and had their milliseconds set to a lower value should be called first.
Firefox, Safari, and the current stable channel of Chrome all do this.
So when the event loop gains control again, it sees that all the timers are ready to be called, and it queues tasks for each, in this scheduled called order:
"1": scheduled-time = t=0 + 0 = 0
"2": scheduled-time = t=0 + 100 = 100
"3": scheduled-time = t=200 + 0 = 300
"4": scheduled-time = t=400 + 0 = 400
But, what Chrome apparently used to do and still does in its other branches is that they only do look at the milliseconds param to do the ordering and ignore the first "that started before this one" condition.
So in there we've got,
"1": milliseconds = 0
"3": milliseconds = 0
"4": milliseconds = 0
"2": milliseconds = 100
Below is a rewrite of this logic:
// We use a MessageChannel to hook on each iteration of the event loop
function postTask(cb) {
const channel = postTask.channel ??= new MessageChannel();
const { port1, port2 } = channel;
port1.addEventListener("message", (evt) => { cb() }, { once: true });
port1.start();
port2.postMessage("");
}
const timers = new Set();
let ended = false; // So we can stop our loop after some time
function timeoutChecker() {
const now = performance.now();
const toCall = Array.from(timers)
.filter(({ startTime, millis }) => startTime + millis <= now)
.sort((a, b) => a.millis - b.millis);
while(toCall.length) {
const timer = toCall.shift();
timers.delete(timer);
timer.callback();
}
if (!ended) {
postTask(timeoutChecker);
}
}
function myTimeout(callback, millis) {
const startTime = performance.now();
timers.add({ startTime, millis, callback });
}
// Begin our loop
postTask(timeoutChecker);
// OP's code
function run() {
myTimeout(() => console.log("1"), 0);
myTimeout(() => console.log("2"), 100);
let start = Date.now();
while (Date.now() - start < 200) {
// do nothing
}
myTimeout(() => {
console.log("3");
}, 0);
start = Date.now();
while (Date.now() - start < 200) {
// do nothing
}
myTimeout(() => {
console.log("4");
}, 0);
}
run();
// all should be done after 1s
setTimeout(() => ended = true, 1000);
As for why you sometimes may see "2" before "4" in Chrome and node.js, it's because they do clamp 0ms timeout to 1ms (thought they're working on removing this in Chrome). So when the event loop gains control at t=400, this log("4") timeout may not have met the timer condition yet.
Finally about Chrome's branch thing, I must admit I'm not sure at all what happens there. Running a bisect (against Canary branch) I couldn't find a single revision where the current stable branch behavior happens, so this must be a branch settings thing.
As we may know, var keyword defines a variable globally, or locally to an entire function regardless of block scope. So the below code will log 5 times with the same value.
for(var i = 0; i < 5; i++){
setTimeout(() => console.log(i), 2000);
}
To visualize the above JS runtime like this
As you can see, 5 tasks in Callback Queue will wait until Call stack is empty. So after the synchronous loop is done - It means Call stack is empty in my case, then 5 scheduled tasks - console.log(i) with the value of i is equal to 5 will be executed. You can play around here
And what I want is to log right after i == 2. It works as I expected.
var sleep = (ms) => new Promise(resolve => setTimeout(() => resolve(1), ms));
async function runAsync(){
for(var i = 0; i < 5; i++){
if(i == 2) await sleep(2000);
setTimeout(() => console.log(i));
}
}
runAsync();
But I'm curious how it works while I'm not sure the call stack is empty? Whether when I'm using await, allowing the caller of the async function to resume execution, or some else?
Any explanation/visualization of image flow would be appreciated. Thanks.
await cedes control of the thread and allows other processes to run until the promise being awaited resolves. Even if the promise is already resolved, await will yield to any "microtasks" that have been waiting to execute, but that's a moot point in your case because your promise takes a full two seconds to resolve.
In your case, two setTimeouts are queued up before the await, so they are allowed to run when the await happens.
The timeline is basically like this:
i = 0
setTimeout 1 scheduled
i = 1
setTimeout 2 scheduled
i = 2
await
setTimeout 1 callback runs
setTimeout 2 callback runs
setTimeout 3 scheduled
i = 3
setTimeout 4 scheduled
i = 4
setTimeout 5 scheduled
i = 5
loop ends
setTimeout 3 callback runs
setTimeout 4 callback runs
setTimeout 5 callback runs
You can see that i is 2 when the first pair of setTimeouts are allowed to execute, and it is 5 when the remaining 3 execute.
Here is a snippet that hopefully demonstrates the process a little better:
var sleep = (ms) => new Promise(resolve => setTimeout(() => resolve(1), ms));
async function runAsync() {
for (var i = 0; i < 5; i++) {
console.log('i is now', i);
if (i == 2) {
console.log('about to sleep');
await sleep(5000);
console.log('now done sleeping');
}
console.log('about to setTimeout. i is', i, 'right now');
setTimeout(() => {
console.log('setTimeout task running:', i, '- scheduling a new timeout.');
setTimeout(() => console.log('inner timeout:', i), 1000);
});
}
console.log('done looping. i is', i);
}
runAsync();
I have 1 array with thousands link image like this
let imageList = ["http://img1.jpg", "http://img2.jpg", ...];
I want to loop over the imageList and delay after 20 times (n times) increase index like
for(let i = 0; i <= imageList.length; i+=20){
// with i from 0 -> 20
// do download image from server
downloadImages(0,20) // [start, end]
// delay 5s to prevent server timeout because request many times
// with i from 20 -> 40
// do download image from server
downloadImages(20,40)
// continue delay 5s
// .... try to finish
}
Try smth like this:
const imageList = ['***'];
downloadImages(imageList)
.then(/* done */)
.catch(/* error */);
async function downloadImages(images) {
for(let i = 0; i + 20 <= imageList.length; i += 20){
const n20images = imageList.slice(i, i + 20);
await fetchImages(n20images);
await delay(5);
}
}
function fetchImages(images) {
return Promise.all(
images.map(image => /* fetch(image) or smth else */)
)
}
function delay(seconds) {
return new Promise(resolve => setTimeout(resolve, seconds * 1000))
}
You can use modulus operator.
let imageList = ["http://img1.jpg", "http://img2.jpg", ...];
for(let i = 0; i <= imageList.length; i++){
if (i % n == 0) //n is number of images to start delay
START_YOUR_DELAY_HERE
downloadImage(20); //20 is number of images you want to download
}
I use async/await, for of and chunk from lodash for this kind of situation. It'll make the requests in groups of 20 for not flooding the server
let i = 0
const imageListChunks = _.chunk(imageList, 20)
for await (const chunk of imageListChunks){
const chunkPromises = downloadImage(0 + i*20, 20 + i*20)
const chunkResp = await Promise.all(chunkPromises)
i = i + 1
}
If you need more delay to let the server breath you can add a setTimeout with another await to slow it more
You can use async and setTimeout to achieve this:
let downloadImage = async url => {
console.log(`Downloading ${url}`);
// Simulate a download delay
return new Promise(r => setTimeout(r, 100 + Math.floor(Math.random() * 500)));
}
let downloadAllImages = async (imageUrls, chunkSize=20, delayMs=2000) => {
for (let i = 0; i < imageUrls.length; i += chunkSize) {
console.log(`Downloading chunk [${i}, ${i + 20 - 1}]`);
// This `chunk` is a subset of `imageUrls`: the ones to be downloaded next
let chunk = imageUrls.slice(i, i + 20);
// Call `downloadImage` on each item in the chunk; wait for all downloads to finish
await Promise.all(chunk.map(url => downloadImage(url)));
// Unless this is the last chunk, wait `delayMs` milliseconds before continuing
// (Note: this step may be poor practice! See explanation at bottom of this answer)
if ((i + chunkSize) < imageUrls.length) await new Promise(r => setTimeout(r, delayMs));
}
};
// Create an array of example urls
let imageUrls = [ ...new Array(100) ].map((v, n) => `http://example.com/image${n}.jpg`);
// Call `downloadAllImages`
downloadAllImages(imageUrls)
// Use `.then` to capture the moment when all images have finished downloading
.then(() => console.log(`Finished downloading ${imageUrls.length} images!`));
Note that if you implement downloadImage correctly, so that it returns a promise which resolves when the image is downloaded, it may be best practice to forego the timeout. The timeout is a heuristic way of ensuring not too many requests are running at once, but if you have a fine-grained sense of when a request finishes you can simply wait for a batch of requests to finish before beginning the next batch.
There is an even more efficient design to consider (for your further research). To understand, let's think about a problem with this current approach (which I'll call the "batch" approach). The batch approach is incapable of beginning a new batch until the current one completes. Imagine a batch of 20 images where 1 downloads in 1ms, 18 of them download within 5ms, but the final image takes 10+ seconds to download; even though this system ought to have the bandwidth to download 20 images at once, it winds up spending 10 entire seconds with only a single request in progress. A more efficient design (which we can call the "maximal bandwidth approach") would maintain a queue of 20 in-progress requests, and every time one of those requests completes a new request is immediately begun. Imagine that first image which downloads in 1ms; the moment it finishes, and only 19 requests are in progress, the "maximal bandwidth approach" could begin a new request right away without waiting for those other 19 to finish.
Set some offset
let offset = 0
for (let i = offset; i <= imageList.length; i += 20) {
downloadImage(offset, offset + 20)
offset += 20
}
I'm receiving data in browser through websockets (paho-mqtt) but problem is that the receiving callback gets fired only when another task ends (big for loop) and it gets fired with all the stacked data, I'm not losing data just getting delayed. Shouldn't the callback get fired even if there is a loop running? What is happening here?. Otherwise, how can I achieve this, keep receiving while inside a loop?
What I'm trying to say is equivalent to the following:
If I do this in chrome
setTimeout(() => {
console.log('hello!');
}, 10);
for (var i = 0; i < 50000; i++) {
console.log('for array');
}
I get
50000 VM15292:5 for array
VM15292:2 hello!
Shouldn't I get something like this?
1000 VM15292:5 for array
VM15292:2 hello!
49000 VM15292:5 for array
When you run JavaScript code in the browser (unless using Web Workers or other special technologies), it is executed on a single thread. That might not sound too important, but it is.
Your code consists of a for-loop (synchronous) and a call to setTimeout (asychronous). Since only one piece of JavaScript can be running at once, your for-loop will never be interrupted by setTimeout.
In fact, if your for-loop contained extremely intensive operations that required more than 10 ms to complete, your setTimeout callback might actually be delayed past that mark, because the browser always wait for the currently executing code to finish before continuing to run the event loop.
setTimeout(() => {
console.log('hello!');
}, 10);
for (var i = 0; i < /* 50000 */ 5; i++) {
console.log('for array');
}
The others have diagnosed the problem well, the single threaded nature of the browser. I will offer a possible solution: generators.
Here's a codepen which demonstrates the problem:
http://codepen.io/anon/pen/zZwXem?editors=1111
window.CP.PenTimer.MAX_TIME_IN_LOOP_WO_EXIT = 60000;
function log(message) {
const output = document.getElementById('output');
output.value = output.value + '\n' + message;
}
function asyncTask() {
log('Simulated websocket message')
}
function doWork() {
const timer = setInterval(1000, asyncTask);
let total = 0;
for (let i = 1; i < 100000000; i++) {
const foo = Math.log(i) * Math.sin(i);
total += foo;
}
log('The total is: '+ total);
clearInterval(timer);
}
When doWork() is called by clicking the 'Do Work' button, the asyncTask never runs, and the UI locks up. Horrible UX.
The following example uses a generator to run the long running task.
http://codepen.io/anon/pen/jBmoPZ?editors=1111
//Basically disable codepen infinite loop detection, which is faulty for generators
window.CP.PenTimer.MAX_TIME_IN_LOOP_WO_EXIT = 120000;
let workTimer;
function log(message) {
const output = document.getElementById('output');
output.value = output.value + '\n' + message;
}
function asyncTask() {
log('Simulated websocket message')
}
let workGenerator = null;
function runWork() {
if (workGenerator === null) {
workGenerator = doWork();
}
const work = workGenerator.next();
if (work.done) {
log('The total is: '+ work.value);
workerGenerator = null;
} else {
workTimer = setTimeout(runWork,0);
}
}
function* doWork() {
const timer = setInterval(asyncTask,1000);
let total = 0;
for (let i = 1; i < 100000000; i++) {
if (i % 100000 === 0) {
yield;
}
if (i % 1000000 == 0) {
log((i / 100000000 * 100).toFixed(1) + '% complete');
}
const foo = Math.log(i) * Math.sin(i);
total += foo;
}
clearInterval(timer);
return total;
}
Here we do work in a generator, and create a generator runner to call from the 'Do Work' button in the UI. This runs on the latest version of Chrome, I can't speak for other browsers. Typically you'd use something like babel to compile the generators down to ES5 syntax for a production build.
The generator yields every 10000 rows of calculation, and emits a status update every 100000 rows. The generator runner 'runWork' creates an instance of the generator and repeatedly calls next(). The generator then runs until it hits the next 'yield' or return statement. After the generator yields, the generator runner then gives up the UI thread by calling setTimeout with 0 milliseconds and using itself as the handler function. This typically means it will get called once every animation frame (ideally). This goes until the generator returns the done flag, at which point the generator runner can get the returned value and clean up.
Here the HTML for the example in case you need to recreate the codepen:
<input type='button' value='Do Work' onclick=doWork() />
<textarea id='output' style='width:200px;height:200px'></textarea>
Javascript engines tends to be single threaded.
So if you are in a long running tight loop that doesn't yield (e.g. to do some io) then the callback will never get a chance to run until the loop finishes
In my code, I'm trying to put a certain delay before continuing to the rest of the code. Pretty basic. I'm using a custom sleep function because javascript's native sleep function is not working at all. I'm actually working in app script in google spreadsheets so maybe that's why. But the following code is in the <script> tag of the html file in spreadsheet app script.
Anyway, when I use sleep(), it is being executed before the setTimeout
function get_ids(db){
window.alert("before window.ids is saved?");
google.script.run.withSuccessHandler(getIdsFromAppscript).getIdsFromModel(db);
//this returns a result to getIdsFromAppscript but the following code doesn't wait
//for the result to be returned so I want to use sleep before the ids
//variable is returned by get_ids
setTimeout(function(){
window.alert("checking if ids is saved after 10s?");
window.alert("In timeout ids="+window.ids);
var ids= window.ids; //is non empty
},10000);
sleep(10000);
var ids= window.ids;
window.alert("after wait");
window.alert("after sleep ids="+ids); //is empty but should be non empty
return ids; //=window.ids , Need to return a non-empty result
}
function getIdsFromAppscript(result){
window.ids=result;
}
and the sleep function:
function sleep(ms) {
var start = new Date().getTime(), expire = start + ms;
while (new Date().getTime() < expire) { }
return;
}
Current Order of output based on window.alert():
1) before window is saved?
2) after sleep
3) after sleep ids= //basically empty which shouldn't be the case
4) checking if ids is saved after 10s?
5) In timeout ids= [.....] //non empty list
However, my desired output order is:
1) before window is saved?
2) checking if ids is saved after 10s?
3) In timeout ids= [.....] //non empty list
4) after sleep
5) after sleep ids= [...] //should be a non empty list
The reason why I'm writing setTimeout is to show that after 10 seconds, the result is being stored in window.ids however even after I give a sleep for 10 seconds, or even 30 seconds, I can't get the result from window.ids.
What exactly am I doing wrong here? Thanks in advance~
Java Script, especially through the V8 engine does not sleep. Sleeping causes the entire thread that JavaScript runs on to stop, which breaks the whole point of asynchronocy. setTimeout() only waits to run the function you push into it for the time you also put into it. It doesn't stop the rest of the executions, and whatever happens first will happen first.
If it's important to you that something happens in order, always, then you need to use callbacks or promises.
An example in code could be:
function doTimeout(ms) {
setTimeout(function(){
window.alert("checking if ids is saved after 10s?");
window.alert("In timeout ids="+window.ids);
var ids= window.ids; //is non empty
},ms);
}
function sleep(ms, callback) {
var start = new Date().getTime(), expire = start + ms;
while (new Date().getTime() < expire) { }
callback(ms);
}
sleep(10000, doTimeout);
Javascript is single threaded. You must return from your code for scripts in other threads to execute. Script in other threads includes functions to handle a timeout event, functions called when promises are kept or fail, and call back functions provided for asynchronous requests made using an XMLHttpRequest object.
Writing a function and calling it sleep() does not change this. You could have called it waitingForGodot() for all the difference it would make. What the code you provided does is to spend a lot of time looping in the thread it was called in. It does not relinquish control and blocks all other scripts from executing. If it goes on for long enough my browser will ask me if I wish to abort the (as in your) script.
I have included two examples below showing that your sleep function blocks the entire Javascript engine. When I use your sleep function, the interval function does not get executed even though I have set an interval of 100 ms and the output is delayed by 10 seconds. However, in the second example the output does get printed immediately at the correct interval. This shows your sleep function is blocking the entire execution engine and that explains why your ids array is empty.
function sleep(ms) {
var start = new Date().getTime(),
expire = start + ms;
while (new Date().getTime() < expire) {}
return;
}
function get_ids() {
document.write("before window.ids is saved?" + "<br>");
var counter = 0;
setInterval(function() {
while (counter < 100) {
document.write("checking if ids is saved after 10s?" + "<br>");
counter = counter + 1
}
}, 100);
sleep(10000);
documen.write("after wait");
}
document.write("Start");
get_ids()
document.write("End");
In this example I have commented out your sleep function and as expected the output gets printed every 100 ms:
function sleep(ms) {
var start = new Date().getTime(),
expire = start + ms;
while (new Date().getTime() < expire) {}
return;
}
function get_ids() {
document.write("before window.ids is saved?" + "<br>");
var counter = 0;
setInterval(function() {
while (counter < 100) {
document.write("checking if ids is saved after 10s?" + "<br>");
counter = counter + 1
}
}, 100);
//sleep(10000);
documen.write("after wait");
}
document.write("Start");
get_ids()
document.write("End");