The module-scope variable "output" refuses to be overwritten by the async function "retrieveTextWrapper", and I cannot figure out why. My objective is to output the text on StackOverFlow's homepage. retrieveTextWrapper successfully scrapes this information, but I can't seem to assign this content to the output variable. What am I doing wrong? How can I print the scraped information from the main() function?
Note: I am using electron version 3.0.4 because bypassing CORS is less of a pain on that version.
const {BrowserWindow, app} = require('electron')
output = "this should be overwritten by the retrieveTextWrapper method"
async function main(){
navigate();
win.openDevTools();
await win.webContents.once('dom-ready',retrieveTextWrapper);
console.log(output);
//prints "this should be overwritten by the retrieveTextWrapper method"
}
function navigate() {
win = new BrowserWindow({width:900,height:900});
win.loadURL(`https://stackoverflow.com/`);
}
function retrieveText(){
return `document.querySelector("*").innerText`;
}
async function retrieveTextWrapper(){
output = await win.webContents.executeJavaScript(retrieveText().replace("*", "#content"));
}
app.on('ready',main)
win.webContents.once() does not return a promise (since interfaces generally don't accept both callbacks and return a promise at the same time).
Therefore await doesn't wait for the asynchronous operation to complete. Therefore, you're looking at output before its value has been reassigned. await only does something useful when you await a promise that is connected to the asynchronous operation you're trying to wait for.
To confirm this timing issue, add a unique console.log() statement before and after the await win.webContents.once('dom-ready',retrieveTextWrapper); and inside of retrieveTextWrapper and then you can see the sequencing of these log messages.
Yep, everything changes as it should within retrieveTextWrapper function. And your explanation makes a lot of sense. However, is it possible to wait for the callback to finish (using some other syntax aside from await)? That way, I can use the updated value for other operations in the main function?
You have a couple options.
You could "promisify" win.webContents.once() so you could then use await with it.
You could put the callback inline and put the rest of your code in main inside that callback (a classic way of dealing with asynchronous operations).
Here's an example of promisifying win.webContents.once():
function waitForDomReady() {
return new Promise((resolve, reject) => {
// may want to check if document already has dom-ready and resolve immediately
win.webContents.once('dom-ready', resolve);
});
}
And, you could then use it like this:
async function main(){
navigate();
win.openDevTools();
await waitForDomReady();
await retrieveTextWrapper();
console.log(output);
}
This assumes that the code in retrieveTextWrapper that calls win.webContents.executeJavaScript() does actually return a promise when it's done. If not, you have to promisify that too.
Related
I would like to run this code with babel:
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
inside an async function without await the first line. is this OK?
how else can I run something that I don't care?
Can I just fire the non-promisified function del('key',null) without a callback?
Yes, you can do that, and it will run the two asynchronous functions in parallel. You've just created a promise and thrown it away.
However, this means that when the promise is rejected you won't notice. You'll just get an unhandledRejection eventually which will crash your process if not handled.
Is this OK? How can I run something that I don't care?
Probably it's not OK. If you truly wouldn't care, you hadn't run it in the first place. So you should be clear and explicit what you care about (and what not):
do you want to wait? (for side effects)
do you need the result?
do you want to catch exceptions?
If you only want to wait and don't care for the result value, you can easily throw away the result:
void (await someAsyncFunction()); // or omit the void keyword,
// doesn't make a difference in an expression statement
If you don't care about exceptions, you can ignore them using
… someAsyncFunction().catch(function ignore() {}) …
You can throw that away, await it, do anything with it.
If you want the result, you have to await it. If you care about exceptions, but don't really want to wait, you may want to execute it in parallel with the following functions:
var [_, res] = await Promise.all([
someAsyncFunction(), // result is ignored, exceptions aren't
someOtherAsyncFunction()
]);
return res;
inside an async function without await the first line. is this OK?
Yes, there are cases where you'd want to do this which are perfectly reasonable. Especially where you don't care about the result - one example is an analytics tracking operation that should not interfere with business critical code.
how else can I run something that I don't care?
In many ways, however simply calling the promise function works. Your del without a callback would probably work in this case but some functions don't guard against not passing callbacks, so you can pass an empty function instead (.del('key', () => {})).
You do want to however make sure that you know about it failing, even if you don't want to disrupt the operation of code - so please consider adding a process.on("unhandledRejection', event handler to explicitly ignore these particular exceptions or suppress them via:
redisClient.delAsync('key').catch(()=>{});
Or preferably, something like:
redisClient.delAsync('key').catch(logErr);
From all the research I've made so far, I think it's fine to do it, as long as you guarantee that the function you are not awaiting for guarantees a way to handle its own errors in case that happens. For example, a try-catch wrapping the whole function body, like you see in the following snippet for the asyncFunction.
It doesn't matter if the function throws synchronously or asynchronously. It guarantees the your mainFunction will complete no matter what. That's the key point here.
If you don't guarantee that, you have to risks:
If it throws synchronously, your main function will not complete.
If it throws asynchronously, you'll get an unhandled excepction
// THIS IS SOME API CALL YOU DON'T WANT TO WAIT FOR
const mockAPI = () => {
console.log("From mockAPI");
return new Promise((resolve,reject) => {
setTimeout(() => reject("LATE THROW: API ERROR"), 500);
});
};
// THIS IS THE SOME ASYNC FUNCTION YOU CALL BUT NOT AWAIT FOR
const asyncFunction = async (syncThrow) => {
try {
console.log("Async function START");
if (syncThrow) throw new Error("EARLY THROW");
await mockAPI();
console.log("Async function DONE");
}
catch(err) {
console.log("From async function catch");
console.log(err.message || err);
return;
}
};
// THIS IS YOUR MAIN FUNCTION
const mainFunction = async (syncThrow) => {
try {
console.clear();
console.log("Main function START");
asyncFunction(syncThrow);
console.log("Main function DONE <<< THAT'S THE IMPORTANT PART");
}
catch(err) {
console.log("THIS WILL NEVER HAPPEN");
console.log(err);
}
};
<div>
<button onClick="mainFunction(true)">Sync throw</button>
<button onClick="mainFunction(false)">Async throw</button>
</div>
Not in Node.js.
Node does not wait for ever-pending Promises. If other tasks are already completed and there is nothing left in the event loop, the Node process will be terminated even though there exists pending promise.
For the following script, if someOtherAsyncFunction() get resolved in 5 seconds, but redisClientAsync.delAsync('key') takes 10 seconds to execute, the Node process will be terminated after 5 seconds in theory, before the first line is resolved.
async function doSomething() {
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
}
await doSomething();
I've been trying to learn the JS concurrency model given a background in Python's.
Running the following in Python:
async def myFunction():
print("abc")
myFunction()
will print nothing. Running the following in JavaScript:
async function myFunction() {
console.log("abc")
}
myFunction()
will print "abc". In both cases I did not await myFunction, yet the evaluations were different.
I understand why the Python program prints nothing. myFunction is a coroutine function that resolves to nothing, more specifically it returns an Awaitable[None]. But to actually have this awaitable's side effect executed, I must await it.
I have also read Are promises lazily evaluated? with an answer of no, talking about how the eager evaluation of promises is guaranteed.
Even though I've looked at both concurrency models separately, their difference is still confusing. While general clarity about the contrast here would be very helpful, I also do have a specific question: Is there ever a point to awaiting a Promise in JavaScript that resolves to nothing, and should only execute for its side effect? In other words, do await myFunction() and myFunction() possibly have a difference in behavior in a different case, even though they both gave the same output here?
async def myFunction():
print("abc")
awaitable = myFunction()
In Python, myFunction() returns an awaitable. The code hasn't run yet. It only runs when you execute a method (.send()) on the awaitable:
awaitable.send(None)
You typically don't use .send(), these mechanics are hidden. You will just use await myFunction() or asyncio.get_event_loop().run_until_complete(myFunction()), and this will execute the function's code behind the scenes.
In JavaScript, an async function returns a promise. The MDN JS reference says:
Promises in JavaScript represent processes that are already happening
So when you call the async function in JavaScript, the function's code starts running, and at some point a promise is returned. Here's what the MDN JS reference says about it:
The body of an async function can be thought of as being split by zero or more await expressions. Top-level code, up to and including the first await expression (if there is one), is run synchronously. In this way, an async function without an await expression will run synchronously. If there is an await expression inside the function body, however, the async function will always complete asynchronously.
Given all this, I don't see why you would await an async JS function if you only wanted its side effect (unless the caller needs the side effect to have occurred before proceeding).
Inside an async function all the asynchronous operations are done by using an await in front of it.
If there is an await in front of a promise, then the we wait until the async operation is completed and then continue the remaining code execution.
If we don't use await infront of an async operation, we don't resolve the promise and continue.
Await in front of a synchronous operation has no effect.
// Case I
const test = async() => {
let res = fetch("http://localhost:3000");
console.log(res);
console.log("It is working");
// We dont wait for the http request to complete,
// We print res as a <<promise>>
// and continue to the next line and print "It is working".
}
test();
// Case II
const test = async() => {
let res = await fetch("http://localhost:3000");
console.log(res);
console.log("It is working");
// We wait for the http request completion.
// After that we print the result.
// Then print ===> It is working
}
test();
()=> {} // stands for an anonymous function also know as Fat arrow function.
You use await in front of a function only if it returns a promise.
If it returns a promise, we wait for the promise to resolve and continue code execution.
If it doesn't returns a promise, default behavior is expected.
const test = () => {
return fetch("http://localhost:3000");
}
let testingAsynFunc = async() => {
let a = await test();
//Here we wait until the promise is resolved and then continue the code execution.
console.log(a);
console.log("working");
}
testingAsyncFunc()
I would like to run this code with babel:
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
inside an async function without await the first line. is this OK?
how else can I run something that I don't care?
Can I just fire the non-promisified function del('key',null) without a callback?
Yes, you can do that, and it will run the two asynchronous functions in parallel. You've just created a promise and thrown it away.
However, this means that when the promise is rejected you won't notice. You'll just get an unhandledRejection eventually which will crash your process if not handled.
Is this OK? How can I run something that I don't care?
Probably it's not OK. If you truly wouldn't care, you hadn't run it in the first place. So you should be clear and explicit what you care about (and what not):
do you want to wait? (for side effects)
do you need the result?
do you want to catch exceptions?
If you only want to wait and don't care for the result value, you can easily throw away the result:
void (await someAsyncFunction()); // or omit the void keyword,
// doesn't make a difference in an expression statement
If you don't care about exceptions, you can ignore them using
… someAsyncFunction().catch(function ignore() {}) …
You can throw that away, await it, do anything with it.
If you want the result, you have to await it. If you care about exceptions, but don't really want to wait, you may want to execute it in parallel with the following functions:
var [_, res] = await Promise.all([
someAsyncFunction(), // result is ignored, exceptions aren't
someOtherAsyncFunction()
]);
return res;
inside an async function without await the first line. is this OK?
Yes, there are cases where you'd want to do this which are perfectly reasonable. Especially where you don't care about the result - one example is an analytics tracking operation that should not interfere with business critical code.
how else can I run something that I don't care?
In many ways, however simply calling the promise function works. Your del without a callback would probably work in this case but some functions don't guard against not passing callbacks, so you can pass an empty function instead (.del('key', () => {})).
You do want to however make sure that you know about it failing, even if you don't want to disrupt the operation of code - so please consider adding a process.on("unhandledRejection', event handler to explicitly ignore these particular exceptions or suppress them via:
redisClient.delAsync('key').catch(()=>{});
Or preferably, something like:
redisClient.delAsync('key').catch(logErr);
From all the research I've made so far, I think it's fine to do it, as long as you guarantee that the function you are not awaiting for guarantees a way to handle its own errors in case that happens. For example, a try-catch wrapping the whole function body, like you see in the following snippet for the asyncFunction.
It doesn't matter if the function throws synchronously or asynchronously. It guarantees the your mainFunction will complete no matter what. That's the key point here.
If you don't guarantee that, you have to risks:
If it throws synchronously, your main function will not complete.
If it throws asynchronously, you'll get an unhandled excepction
// THIS IS SOME API CALL YOU DON'T WANT TO WAIT FOR
const mockAPI = () => {
console.log("From mockAPI");
return new Promise((resolve,reject) => {
setTimeout(() => reject("LATE THROW: API ERROR"), 500);
});
};
// THIS IS THE SOME ASYNC FUNCTION YOU CALL BUT NOT AWAIT FOR
const asyncFunction = async (syncThrow) => {
try {
console.log("Async function START");
if (syncThrow) throw new Error("EARLY THROW");
await mockAPI();
console.log("Async function DONE");
}
catch(err) {
console.log("From async function catch");
console.log(err.message || err);
return;
}
};
// THIS IS YOUR MAIN FUNCTION
const mainFunction = async (syncThrow) => {
try {
console.clear();
console.log("Main function START");
asyncFunction(syncThrow);
console.log("Main function DONE <<< THAT'S THE IMPORTANT PART");
}
catch(err) {
console.log("THIS WILL NEVER HAPPEN");
console.log(err);
}
};
<div>
<button onClick="mainFunction(true)">Sync throw</button>
<button onClick="mainFunction(false)">Async throw</button>
</div>
Not in Node.js.
Node does not wait for ever-pending Promises. If other tasks are already completed and there is nothing left in the event loop, the Node process will be terminated even though there exists pending promise.
For the following script, if someOtherAsyncFunction() get resolved in 5 seconds, but redisClientAsync.delAsync('key') takes 10 seconds to execute, the Node process will be terminated after 5 seconds in theory, before the first line is resolved.
async function doSomething() {
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
}
await doSomething();
This question is theoretical - I have no concrete problem to solve.
With that said, why does the async keyword wrap the return value of an async function in a promise? What's the point? Is it ONLY because the await expression expects a promise? Or is there some meaning / use behind this decision?
I thought i'd answer this primarily because async in Javascript used to confuse the hell out of me, and all of a sudden it snapped, so i hope this analogy may help this happen for you.
You have an async event. This could be anything, getting something from a server, doing something in the browser that takes time, training a machine learning model (!), executing a function or method that uses a setTimeout etc.
The beauty of Javascript and a key reason it works so well for the browser is that it uses the processor thread it runs on in a very clever way that stops the thread from getting blocked by processes that take time (like the ones mentioned above)
Many other languages, for example Ruby run on more than one thread. It is possible to use service workers to run processes on multiple threads in javascript but that is outside the scope of this answer!
The async nature of the JS event loop allows the thread to 'go off' and do something else while it is waiting for a process to finish.
The problem with this from a programming point of view is that it is possible for something in the code that relies on the result of a blocking event to get 'undefined' as a result of the event if it doesn't wait for the event to finish before it tries to use the result of it. Take this piece of code below
let scopedVariable
console.log('the code has started')
setTimeout(() => {
scopedVariable="I am the result of some async process"
}, 5000);
console.log(scopedVariable)
When the code reaches the console log, the setTimeout hasn't yet completed. As the setTimeout sets the scopedVariable only when it completes, the variable is undefined when we log it
if however
We wrap the timeout in a promise we can await it's resolve callback (first argument of promise) and the code will 'pause' until the promise reaches the resolve callback before continuing.
When we await the promise and the setTimeout completes, the resolve function sets the variable, so that when we console log it it holds the value from the promise
let scopedVariable
const asyncEvent = new Promise ((resolve,fail) => {
setTimeout(() => {
resolve(scopedVariable="I have resolved")
}, 5000);
})
const container = async () => {
const result = await asyncEvent
console.log(scopedVariable)
}
container()
You can use await and .then interchangably
For example we could go:
let scopedVariable
const asyncEvent = new Promise ((resolve,fail) => {
setTimeout(() => {
resolve(scopedVariable="I have resolved")
}, 5000);
})
const container = async () => {
asyncEvent.then(() => console.log(scopedVariable))
}
container()
once again the code will pause at .then and then continue when the asyncEvent promise has resolved.
In fact if we use .then we don't need to enclose it in an async function so we can rewrite it like this
let scopedVariable
const asyncEvent = new Promise ((resolve,fail) => {
setTimeout(() => {
resolve(scopedVariable="I have resolved")
}, 5000);
})
asyncEvent.then(() => console.log(scopedVariable))
The great thing about .then is that the accompanying .catch allows you to catch any errors thrown by the async event (for example if retrieving something from a server when there is an error). For async await you need to wrap potentially dangerous functions in a try catch.
In order to use await you need to be inside an async function (hence the async container function above). This is not necessary with .then, but .then and .catch chains can make your code messy.
I hope this helps!
The async and await operators are just syntactic sugar that hide the underlying use of promises to implement asynchronous code.
Using async before a function definition makes the function return a promise that resolves to the function's return value, rather than returning normally.
Using await before an asynchronous function call suspends the current function until the promise that it returns is resolved. It's basically equivalent to wrapping the remainder of the function in an anonymous function, and using that as the .then() callback of the promise.
For more information between the relationship, see How to translate Promise code to async await
I would like to run this code with babel:
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
inside an async function without await the first line. is this OK?
how else can I run something that I don't care?
Can I just fire the non-promisified function del('key',null) without a callback?
Yes, you can do that, and it will run the two asynchronous functions in parallel. You've just created a promise and thrown it away.
However, this means that when the promise is rejected you won't notice. You'll just get an unhandledRejection eventually which will crash your process if not handled.
Is this OK? How can I run something that I don't care?
Probably it's not OK. If you truly wouldn't care, you hadn't run it in the first place. So you should be clear and explicit what you care about (and what not):
do you want to wait? (for side effects)
do you need the result?
do you want to catch exceptions?
If you only want to wait and don't care for the result value, you can easily throw away the result:
void (await someAsyncFunction()); // or omit the void keyword,
// doesn't make a difference in an expression statement
If you don't care about exceptions, you can ignore them using
… someAsyncFunction().catch(function ignore() {}) …
You can throw that away, await it, do anything with it.
If you want the result, you have to await it. If you care about exceptions, but don't really want to wait, you may want to execute it in parallel with the following functions:
var [_, res] = await Promise.all([
someAsyncFunction(), // result is ignored, exceptions aren't
someOtherAsyncFunction()
]);
return res;
inside an async function without await the first line. is this OK?
Yes, there are cases where you'd want to do this which are perfectly reasonable. Especially where you don't care about the result - one example is an analytics tracking operation that should not interfere with business critical code.
how else can I run something that I don't care?
In many ways, however simply calling the promise function works. Your del without a callback would probably work in this case but some functions don't guard against not passing callbacks, so you can pass an empty function instead (.del('key', () => {})).
You do want to however make sure that you know about it failing, even if you don't want to disrupt the operation of code - so please consider adding a process.on("unhandledRejection', event handler to explicitly ignore these particular exceptions or suppress them via:
redisClient.delAsync('key').catch(()=>{});
Or preferably, something like:
redisClient.delAsync('key').catch(logErr);
From all the research I've made so far, I think it's fine to do it, as long as you guarantee that the function you are not awaiting for guarantees a way to handle its own errors in case that happens. For example, a try-catch wrapping the whole function body, like you see in the following snippet for the asyncFunction.
It doesn't matter if the function throws synchronously or asynchronously. It guarantees the your mainFunction will complete no matter what. That's the key point here.
If you don't guarantee that, you have to risks:
If it throws synchronously, your main function will not complete.
If it throws asynchronously, you'll get an unhandled excepction
// THIS IS SOME API CALL YOU DON'T WANT TO WAIT FOR
const mockAPI = () => {
console.log("From mockAPI");
return new Promise((resolve,reject) => {
setTimeout(() => reject("LATE THROW: API ERROR"), 500);
});
};
// THIS IS THE SOME ASYNC FUNCTION YOU CALL BUT NOT AWAIT FOR
const asyncFunction = async (syncThrow) => {
try {
console.log("Async function START");
if (syncThrow) throw new Error("EARLY THROW");
await mockAPI();
console.log("Async function DONE");
}
catch(err) {
console.log("From async function catch");
console.log(err.message || err);
return;
}
};
// THIS IS YOUR MAIN FUNCTION
const mainFunction = async (syncThrow) => {
try {
console.clear();
console.log("Main function START");
asyncFunction(syncThrow);
console.log("Main function DONE <<< THAT'S THE IMPORTANT PART");
}
catch(err) {
console.log("THIS WILL NEVER HAPPEN");
console.log(err);
}
};
<div>
<button onClick="mainFunction(true)">Sync throw</button>
<button onClick="mainFunction(false)">Async throw</button>
</div>
Not in Node.js.
Node does not wait for ever-pending Promises. If other tasks are already completed and there is nothing left in the event loop, the Node process will be terminated even though there exists pending promise.
For the following script, if someOtherAsyncFunction() get resolved in 5 seconds, but redisClientAsync.delAsync('key') takes 10 seconds to execute, the Node process will be terminated after 5 seconds in theory, before the first line is resolved.
async function doSomething() {
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
}
await doSomething();