I am trying to improve my understanding of asynchronous JavaScript. To do this I have made a function that can take a long time to complete, which would completely block any other code from executing.
To test this, the function counts up. the first call counts from 1 to 10000000. The second call counts from 10 to 100.
CODE
async function LongTask(from, to) {
//Count up
let myNum = from
console.log(`counting from ${myNum} to ${to}`)
let start = new Date();
while (myNum != to) {
//console.log(`myNum: ${myNum}`);
await myNum++;
}
let end = new Date();
console.log(`counting from ${from} to ${to} done in ${end - start}ms!`);
}
//2 functions that take some time to finish
LongTask(1, 10000000);
LongTask(10, 100);
//Do unrelated stuff
console.log("Hello World!");
OUTPUT
counting from 1 to 10000000
counting from 10 to 100
Hello World!
counting from 10 to 100 done in 2ms!
counting from 1 to 10000000 done in 40389ms!
I managed to get it so that 10 to 100 will finish first due to it being faster. However, I get a warning on line 10 await myNum++; saying 'await' has no effect on the type of this expression. Removing the await keyword results in code blocking making it so that the 10 to 100 call of the function will have to wait for the much longer, unrelated 1 to 10000000 call to finish before being called.
Am I misunderstanding how asynchronous JavaScript works and is there a better way to ensure the 10 to 100 call finishes first?
You can only await a Promise which your function (and every async function) returns. You'll have to await both promises at some point in order to make sure the program does not exit before the promises resolve.
You can use Promise.all() to await multiple promises. I altered your example to await a predefined time span to illustrate how this works, as I then exactly know when I expect a function to finish.
(async() => {
async function LongTask(ms) {
console.log(`starting task that will take ${ms} milliseconds`)
await waitFor(ms);
console.log(`ended task that took ${ms} milliseconds`);
}
async function waitFor(ms) {
return new Promise((resolve, reject) => setTimeout(() => resolve(), ms))
};
//2 functions that take some time to finish
const start = new Date();
console.log(`Calling both functions at ${start}`)
const countToSmallNumberPromise = LongTask(4000);
const countToLargeNumberPromise = LongTask(1000);
//Do unrelated stuff
console.log("Hello World!");
// await both promises to make sure they finish before the program finishes
await Promise.all([countToLargeNumberPromise, countToSmallNumberPromise])
const end = new Date();
// program will end after approx. 4 seconds (at least 4 seconds), so both functions actually run in parallel
// (that's actually not correct, there is no real parallelism here, as Node or your browser are single-threaded)
// To be correct: they run concurrently, not in parallel.
console.log(`Ending program at ${end}. Program ran for ${end - start} milliseconds`);
})();
FYI: Difference between concurrent and in parallel
Related
I'm stuck in Async programming in Javascript.
I know that Promise.all() will run parallel.
Could you pls tell me what I'm wrong with this code below?
It should take 100ms. But actually, it takes 200ms :(
// 1. Define function here
var getFruit = async (name) => {
const fruits = {
pineapple: ":pineapple:",
peach: ":peach:",
strawberry: ":strawberry:"
};
await fetch('https://jsonplaceholder.typicode.com/photos'); // abount 100ms
return fruits[name];
};
var makeSmoothie = async () => {
const a = getFruit('pineapple');
const b = getFruit('strawberry');
const smoothie = await Promise.all([a, b]);
return smoothie;
//return [a, b];
};
/// 2. Execute code here
var tick = Date.now();
var log = (v) => console.log(`${v} \n Elapsed: ${Date.now() - tick}`);
makeSmoothie().then(log);
Your logic is fine, it is running as parallel as possible from client side.
You can test this by waiting on setTimeout instead of the fetch:
await new Promise(resolve => setTimeout(resolve, 100));
It's possible the placeholder site is queuing connections on its side or something. Either way you should measure with something predictable, not network.
Edit:
Just to explain a bit more that won't fit in a comment.
Let's put my waiting trick into an actual function:
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
OK, now you would call this as await wait(100). Now to compare:
await wait(100)
await fetch(...)
In terms of your code setup, they are the same. They are perform some async task that takes around 100ms and returns. If Promise.all() were not running this in parallel, the wait(100) version would certainly take 200ms or longer.
fetch is not as reliable of a test as setTimeout because it is running over the network. There's a lot of things you can't control with this call, like:
Some browsers limit how many parallel connections are made to the same domain
Lag in DNS or the server itself, typical network hiccups
The domain itself might force 1 connection at a time from your IP
It's not clear exactly what is causing the apparent synchronous behavior here. Maybe you will find some answers in the network panel. But in terms of code, there is nothing else for you to do. It is provably parallelized with the setTimeout test, which is much more reliable as a test since it is just a local timer.
This question already has answers here:
Correct way to write a non-blocking function in Node.js
(2 answers)
Closed 2 years ago.
I have a node js program. I want to run 2 countdown functions asynchronously by using Promise.all. It is expected that the two functions will count down together, and the program ends within 5 seconds. But they run sequentially and the program ends within 10 seconds. How to run them asynchronously? Thank you for your help.
const delayOneSecond = () => {
let start = new Date().getTime();
while (new Date().getTime() - start < 1000) { }
return;
}
const OK = true;
const func = (name, num) => {
return new Promise(
function (resolve, reject) {
if (OK) {
for(let i=1; i<=num; i++){
delayOneSecond();
console.log(`[${name}] - ${num - i}`);
}
resolve('OK');
} else {
reject(new Error('Not OK'));
}
});
}
Promise.all([func('xxx', 5), func('ooo', 5)])
.then((res) => { console.log(res); })
Your while loop is a loop that keeps JavaScript busy during one second. During that time no other JavaScript code can execute, no matter what else you had planned for execution. In your case this means that the second func('ooo', 5) does not get launched until the first call has returned.
In practice, the callback you provide to new Promise should not be a blocking piece of code: it should execute relatively quickly and return. It's job is mostly to call some (low-level) API that will trigger an asynchronous event, to which your code will listen. At that time resolve can be called.
You can use the Web API for this (or some other asynchronous library's API), which gives you setTimeout: that function will allow you to be notified when the delay (e.g. 1 second) has passed, but still executes the rest of your code to completion (which can possibly also call setTimeout).
If you would resolve a promise when setTimeout calls its callback, then you have a useful, non-blocking implementation of delayOneSecond. It can then be combined easily with await:
const delayOneSecond = () => new Promise(resolve => setTimeout(resolve, 1000));
const func = async (name, num) => {
for(let i=1; i<=num; i++){
await delayOneSecond();
console.log(`[${name}] - ${num - i}`);
}
return "OK";
}
Promise.all([func('xxx', 5), func('ooo', 5)])
.then((res) => { console.log(res); })
Although it may seem here that the execution of func still takes 5 seconds before it returns, this is actually not true. It returns when it arrives at the first await. It returns a promise at that time (so without any delay), and execution can continue with the second call of func.
The two function execution contexts get restored when their delayOneSecond() promises resolve, i.e. after (at least) one second. Only then their for loops continue... This happens one after the other: they don't run their JavaScript in parallel: one function context gets restored, does its thing until the next await, and then the same happens with the other function context.
I wrote two recursive functions that sum numbers from a array. They do the same thing, one asynchronously and the other synchronously. The async function took about 9x the time the sync one did.
Shouldn't the async function take advantage from the fact of running more tasks at the same time?
The functions
// Asynchronously sum the numbers in array
async function sumAsync(arr){
if(arr.length == 1) return arr[0];
const half = arr.length/2;
// Numbers on the left half
const left = arr.filter((e, i) => {
return i < half;
});
// Numbers on the right half
const right = arr.filter((e, i) => {
return i >= half;
});
// Recursive call
const leftR = sumAsync(left);
const rightR = sumAsync(right);
// Wait for resolves
await Promise.all([leftR, rightR]);
return await leftR + await rightR;
}
// Synchronously sum the numbers in array
function sumSync(arr){
if(arr.length == 1) return arr[0];
const half = arr.length/2;
// Numbers on the left half
const left = arr.filter((e, i) => {
return i < half;
});
// Numbers on the right half
const right = arr.filter((e, i) => {
return i >= half;
});
// Recursive call
const leftR = sumSync(left);
const rightR = sumSync(right);
return leftR + rightR;
}
Testing them
(async () => {
const LENGTH = 1048576; // 1024^2
const arr = Array.from(Array(LENGTH), num => Math.random()*10 | 0);
// arr[1048576] <- random (0 ~ 9)
// Async sum
console.log('ASYNC');
before = Date.now();
console.log(`SUM: ${await sumAsync(arr)}`);
after = Date.now();
console.log(`TIME: ${after - before} ms`);
// Sync sum
console.log('SYNC');
before = Date.now();
console.log(`SUM: ${sumSync(arr)}`);
after = Date.now();
console.log(`TIME: ${after - before} ms`);
})();
Results
// ASYNC
// SUM: 4720832
// TIME: 5554 ms
// SYNC
// SUM: 4720832
// TIME: 613 ms
The return value of an async function is always a Promise, even if the function only carries out synchronous operations, and the await (or .then) of a Promise will only run what follows during a microtask (after the current synchronous code is finished running). With a large array, that'll result in a lot of unnecessary microtasks wrapping synchronous code.
When nothing actual asynchronous is going on, this is just extra baggage, and results in additional processing time and power required.
Shouldn't the async function take advantage from the fact of running more tasks at the same time?
Javascript is single-threaded, even with async functions. If multiple async functions are called at once, only one path through the code may be "active" at any one time. If the total processing time required for all tasks is, say, 1000 ms, in standard Javascript, there's no way around spending at least 1000 ms.
You're not actually running more tasks at the same time - you're just wrapping the tasks in Promises (unnecessarily), while doing the same work.
For truly parallel actions, you'll have to use something provided by your current environment, such as child_process in Node, or a web worker.
Short version: async doesn't do more than one thing at a time. It switches between tasks (with overhead for each switch) in a queue, and when one task blocks, it hands control off to another (with overhead for the switch, and requeueing the blocked task when it unblocks).
Long version: Async doesn't mean parallel processing, it means interleaved (concurrent, cooperative) processing. JavaScript is still single-threaded even with async usage, and all of the actual work you do is purely CPU bound. In fact, your only real concurrency is that the async code will be scheduling, pausing and resuming your recursive calls repeatedly (but still only doing work for one at a time), while the sync code will just do them in order as fast as possible, with no event loop involvement.
The benefit of async code is that when blocking I/O (including stuff like waiting on user input) is being performed, that task can be suspended until it's unblocked by some out-of-band signal (I/O done, user clicked mouse, whatever), and other tasks can run. The benefit is in reaping the benefits of concurrent (but not parallel) processing in situations where most tasks, most of the time, are waiting for something, so the few that are ready to run can begin running immediately (and since they're usually not running, the overhead of task switching doesn't matter much; most of the time, there's nothing to switch to, so much of the overhead is paid when you've got nothing better to do). But it's definitely higher overhead than just doing number-crunching without a pause.
function sleep(ms) {
var start = new Date().getTime(),
expire = start + ms;
while (new Date().getTime() < expire) {}
return;
}
async function executeWithDelay(offers) {
return Promise.all(
offers.map((offer, i) =>
getDetailedInfo(offer).then(data => {
offer = data;
if (i % 5 === 0) {
console.log('executed but it delays now for 3 seconds');
sleep(3000);
}
})
)
).then(function(data) {
return offers;
});
}
Trying to achieve web-scraping with possible best solution available. I am combining cheerio and puppeteer together and i have some nice code to debug. The above code works fine if the offers data is less i.e 5-10 records. If it exceeds, the browser gets crashed, though i am running in headless version.
This is basically due to in-house bug from the package what i am using which is unable to handle the load. My experiment is to a little delay for every batch of records and complete the full records execution.
But with the below code, it's behaving as non-blocking code and first time it delays and rest is executed constantly.
Should i use for loop instead of map or is there any alternative to handle this situation?
You never want to "sleep" with a busy-wait loop like that. It means nothing else can be done on the main thread.
Once the operations the promises represent have been started, unless they provide some custom means of pausing them, you can't pause them. Remember, promises don't actually do anything; they provide a standard way to observe something being done.
Another problem with that code is that you're returning offers, not the results of calling getExtendedInfo.
If the problem is calling getExtendedInfo too often, you can insert a delay (using setTimeout, not a busy-loop). For instance, this does the first request to getExtendedInfo almost immediately, the next after one second, the next after two seconds, and so on:
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
function executeWithDelay(offers) {
return Promise.all(
offers.map((offer, i) =>
sleep(i * 1000).then(getDetailedInfo)
)
});
}
Obviously, you can adjust that delay as required.
(Note that executeWithDelay isn't declared async; no need, since it has to use Promise.all anyway.)
I'm trying to get a loop working with a longer delay, when the task takes longer as planned.
currently I'm using this code for the loop:
async function doSomeStuff() {
// do some stuff
// sometimes this action may take longer than 5 seconds
// after finishing wait 5 seconds
console.log('task completed, waiting 5 seconds for next run...');
}
setInterval(doSomeStuff, 5000);
works like a charm, but the delay is fixed at 5 seconds, even if the tasks takes longer as planned, so that sometimes the new loop starts only 1 second after finishing the last one instead of waiting 5 seconds.
unfortunately, I was not able to solve it myself by looking at the other questions.
I am grateful for any help.
Best regards
Paedda
async functions shouldn't be used with API that ignores a promise that it returns such as setInterval, in case it's expected that a promise should be chained.
This can be done with recursive async function:
(async function doSomeStuff() {
await new Promise(resolve => setTimeout(resolve, 5000));
// do some stuff
await doSomeStuff();
})();
Or infinite loop:
(async function doSomeStuff() {
while (true) {
await new Promise(resolve => setTimeout(resolve, 5000));
// do some stuff
}
})();
Function body can be wrapped with try..catch if needed to handle errors.
You were almost there
async function doSomeStuff() {
// do some stuff
// sometimes this action may take longer than 5 seconds
// after finishing wait 5 seconds
console.log('task completed, waiting 5 seconds for next run...');
setTimeout(doSomeStuff, 5000);
}