I want to test my logic and ensure that I don't exceed a limit of X requests every X seconds. Using mocha#^9.1.3, chai#^4.3.4 and sinon#^12.0.1, latest versions as of this comment.
I've read answers to very similar problems to mine, tried applying the knowledge gained but to no avail:
https://stackoverflow.com/a/52196951 — great explanation as to the why, however I must've misunderstood something
https://stackoverflow.com/a/55448441 — similar to 1.
https://stackoverflow.com/a/43048888, https://stackoverflow.com/a/60629795 — adding clock.tickAsync inside the wait utility function would not let me step through the code
I have a wait utility function looking like this:
const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
and my code to reproduce my issue in my test file looks like (please do read comments in code):
import sinon from 'sinon';
import got from 'got';
describe('sinon usefaketimers promise-timeouts', () => {
let clock: sinon.SinonFakeTimers;
beforeEach(() => {
clock = sinon.useFakeTimers({ toFake: ['setTimeout'] });
// i've also tried this
// clock = sinon.useFakeTimers();
});
afterEach(() => {
clock.restore();
});
it('steps through all promise-timeouts manually', async () => {
const main = async () => {
console.log('starting');
await wait(1000);
console.log('waited 1s');
await wait(1000);
console.log('waited 2s');
// switching out `got()` below to `await Promise.resolve()`
// finishes the test successfully
await got('https://jsonplaceholder.typicode.com/todos/1').json(); // <-- switch to `await Promise.resolve()` makes it work
console.log('request done');
await wait(1000);
console.log('waited 3s');
};
const promise = main();
await clock.tickAsync(1000); // I have tried adding different combinations
await clock.tickAsync(1000); // of `await Promise.resolve()` after the tick(s)
await clock.tickAsync(1000);
return promise;
});
});
and the resulting output is, (it hangs after "request done", missing the last line "waited 3s"):
starting
waited 1s
waited 2s
request done
Note, I've tried switching out got to superagent and node-fetch without any success. I'm not entirely sure why they would be any different from Promise.resolve as they are all promises returned and awaited on.
Really hope to clear up whatever I didn't get from reading above questions (and so many other threads) from yesterday and todays debugging.
I wrote this code in lib/helper.js:
var myfunction = async function(x,y) {
....
return [variableA, variableB]
}
exports.myfunction = myfunction;
Then I tried to use it in another file :
var helper = require('./helper.js');
var start = function(a,b){
....
const result = await helper.myfunction('test','test');
}
exports.start = start;
I got an error:
await is only valid in async function
What is the issue?
The error is not refering to myfunction but to start.
async function start() {
....
const result = await helper.myfunction('test', 'test');
}
// My function
const myfunction = async function(x, y) {
return [
x,
y,
];
}
// Start function
const start = async function(a, b) {
const result = await myfunction('test', 'test');
console.log(result);
}
// Call start
start();
I use the opportunity of this question to advise you about an known anti pattern using await which is : return await.
WRONG
async function myfunction() {
console.log('Inside of myfunction');
}
// Here we wait for the myfunction to finish
// and then returns a promise that'll be waited for aswell
// It's useless to wait the myfunction to finish before to return
// we can simply returns a promise that will be resolved later
// useless async here
async function start() {
// useless await here
return await myfunction();
}
// Call start
(async() => {
console.log('before start');
await start();
console.log('after start');
})();
CORRECT
async function myfunction() {
console.log('Inside of myfunction');
}
// Here we wait for the myfunction to finish
// and then returns a promise that'll be waited for aswell
// It's useless to wait the myfunction to finish before to return
// we can simply returns a promise that will be resolved later
// Also point that we don't use async keyword on the function because
// we can simply returns the promise returned by myfunction
function start() {
return myfunction();
}
// Call start
(async() => {
console.log('before start');
await start();
console.log('after start');
})();
Also, know that there is a special case where return await is correct and important : (using try/catch)
Are there performance concerns with `return await`?
To use await, its executing context needs to be async in nature
As it said, you need to define the nature of your executing context where you are willing to await a task before anything.
Just put async before the fn declaration in which your async task will execute.
var start = async function(a, b) {
// Your async task will execute with await
await foo()
console.log('I will execute after foo get either resolved/rejected')
}
Explanation:
In your question, you are importing a method which is asynchronous in nature and will execute in parallel. But where you are trying to execute that async method is inside a different execution context which you need to define async to use await.
var helper = require('./helper.js');
var start = async function(a,b){
....
const result = await helper.myfunction('test','test');
}
exports.start = start;
Wondering what's going under the hood
await consumes promise/future / task-returning methods/functions and async marks a method/function as capable of using await.
Also if you are familiar with promises, await is actually doing the same process of promise/resolve. Creating a chain of promise and executes your next task in resolve callback.
For more info you can refer to MDN DOCS.
When I got this error, it turned out I had a call to the map function inside my "async" function, so this error message was actually referring to the map function not being marked as "async". I got around this issue by taking the "await" call out of the map function and coming up with some other way of getting the expected behavior.
var myfunction = async function(x,y) {
....
someArray.map(someVariable => { // <- This was the function giving the error
return await someFunction(someVariable);
});
}
I had the same problem and the following block of code was giving the same error message:
repositories.forEach( repo => {
const commits = await getCommits(repo);
displayCommit(commits);
});
The problem is that the method getCommits() was async but I was passing it the argument repo which was also produced by a Promise. So, I had to add the word async to it like this: async(repo) and it started working:
repositories.forEach( async(repo) => {
const commits = await getCommits(repo);
displayCommit(commits);
});
If you are writing a Chrome Extension and you get this error for your code at root, you can fix it using the following "workaround":
async function run() {
// Your async code here
const beers = await fetch("https://api.punkapi.com/v2/beers");
}
run();
Basically you have to wrap your async code in an async function and then call the function without awaiting it.
The current implementation of async / await only supports the await keyword inside of async functions Change your start function signature so you can use await inside start.
var start = async function(a, b) {
}
For those interested, the proposal for top-level await is currently in Stage 2: https://github.com/tc39/proposal-top-level-await
async/await is the mechanism of handling promise, two ways we can do it
functionWhichReturnsPromise()
.then(result => {
console.log(result);
})
.cathc(err => {
console.log(result);
});
or we can use await to wait for the promise to full-filed it first, which means either it is rejected or resolved.
Now if we want to use await (waiting for a promise to fulfil) inside a function, it's mandatory that the container function must be an async function because we are waiting for a promise to fulfiled asynchronously || make sense right?.
async function getRecipesAw(){
const IDs = await getIds; // returns promise
const recipe = await getRecipe(IDs[2]); // returns promise
return recipe; // returning a promise
}
getRecipesAw().then(result=>{
console.log(result);
}).catch(error=>{
console.log(error);
});
If you have called async function inside foreach update it to for loop
Found the code below in this nice article: HTTP requests in Node using Axios
const axios = require('axios')
const getBreeds = async () => {
try {
return await axios.get('https://dog.ceo/api/breeds/list/all')
} catch (error) {
console.error(error)
}
}
const countBreeds = async () => {
const breeds = await getBreeds()
if (breeds.data.message) {
console.log(`Got ${Object.entries(breeds.data.message).length} breeds`)
}
}
countBreeds()
Or using Promise:
const axios = require('axios')
const getBreeds = () => {
try {
return axios.get('https://dog.ceo/api/breeds/list/all')
} catch (error) {
console.error(error)
}
}
const countBreeds = async () => {
const breeds = getBreeds()
.then(response => {
if (response.data.message) {
console.log(
`Got ${Object.entries(response.data.message).length} breeds`
)
}
})
.catch(error => {
console.log(error)
})
}
countBreeds()
In later nodejs (>=14), top await is allowed with { "type": "module" } specified in package.json or with file extension .mjs.
https://www.stefanjudis.com/today-i-learned/top-level-await-is-available-in-node-js-modules/
This in one file works..
Looks like await only is applied to the local function which has to be async..
I also am struggling now with a more complex structure and in between different files. That's why I made this small test code.
edit: i forgot to say that I'm working with node.js.. sry. I don't have a clear question. Just thought it could be helpful with the discussion..
function helper(callback){
function doA(){
var array = ["a ","b ","c "];
var alphabet = "";
return new Promise(function (resolve, reject) {
array.forEach(function(key,index){
alphabet += key;
if (index == array.length - 1){
resolve(alphabet);
};
});
});
};
function doB(){
var a = "well done!";
return a;
};
async function make() {
var alphabet = await doA();
var appreciate = doB();
callback(alphabet+appreciate);
};
make();
};
helper(function(message){
console.log(message);
});
A common problem in Express:
The warning can refer to the function, or where you call it.
Express items tend to look like this:
app.post('/foo', ensureLoggedIn("/join"), (req, res) => {
const facts = await db.lookup(something)
res.redirect('/')
})
Notice the => arrow function syntax for the function.
The problem is NOT actually in the db.lookup call, but right here in the Express item.
Needs to be:
app.post('/foo', ensureLoggedIn("/join"), async function (req, res) {
const facts = await db.lookup(something)
res.redirect('/')
})
Basically, nix the => and add async function .
"await is only valid in async function"
But why? 'await' explicitly turns an async call into a synchronous call, and therefore the caller cannot be async (or asyncable) - at least, not because of the call being made at 'await'.
Yes, await / async was a great concept, but the implementation is completely broken.
For whatever reason, the await keyword has been implemented such that it can only be used within an async method. This is in fact a bug, though you will not see it referred to as such anywhere but right here. The fix for this bug would be to implement the await keyword such that it can only be used TO CALL an async function, regardless of whether the calling function is itself synchronous or asynchronous.
Due to this bug, if you use await to call a real asynchronous function somewhere in your code, then ALL of your functions must be marked as async and ALL of your function calls must use await.
This essentially means that you must add the overhead of promises to all of the functions in your entire application, most of which are not and never will be asynchronous.
If you actually think about it, using await in a function should require the function containing the await keyword TO NOT BE ASYNC - this is because the await keyword is going to pause processing in the function where the await keyword is found. If processing in that function is paused, then it is definitely NOT asynchronous.
So, to the developers of javascript and ECMAScript - please fix the await/async implementation as follows...
await can only be used to CALL async functions.
await can appear in any kind of function, synchronous or asynchronous.
Change the error message from "await is only valid in async function" to "await can only be used to call async functions".
I'm a bit confused about async functions. It says that they return a promise. A promise usually does not immediately run before the rest of the code, because javascript works with the "run to completion" approach.
But if async functions return a promise, how can you explain this?
const _prefixJS = async file => {
console.log('running...')
fs.copyFileSync(file, `${file}.bak`);
const content = fs.readFileSync(file, 'utf8');
// Some modifications...
fs.writeFileSync(file, newContent);
console.log('stopping!')
};
console.log('start')
_prefixJS(f);
console.log('end')
// Output: start > running... > stopping! > end
In my opinion, the output should be: start > end > starting... > stopping!
Because promises run concurrently and they are placed at the end of the event loop.
I know that those methods are still synchronous, but that's not the topic. I'm currently going from synchronized nodejs-methods to async ones. I am just asking that if async methods return a promise, how is it that it runs before the end console.log statement?
But if async functions return a promise, how can you explain this?
An async function runs synchronously up to the first await or return (including implicit return). (When I say "up to," the synchronous part includes the operand to the right of await or return.) At that point (await or return), it returns a promise. That's so it can start whatever asynchronous process it's supposed to start. This is covered in the specification here.
Your _prefixJS doesn't contain any await or return, so it synchronously runs through to the end, returning a promise that will be fulfilled with the value undefined.
To make your async function actually work asynchronously, you'd want to use the fs.promises version of those functions and await their results. Something like this (untested):
const fsp = require("fs").promises;
// ...
const _prefixJS = async file => {
console.log('running...');
await fsp.copyFile(file, `${file}.bak`);
const content = await fsp.readFile(file, 'utf8');
// Some modifications...
await fsp.writeFile(file, newContent);
console.log('stopping!');
};
With that function, the console.log('running'); call and the fsp.copyFile(...) call are done synchronously when _prefixJS is called, then the function returns its promise and waits for the result of fsp.copyFile(...) before continuing its logic asynchronously.
Live example using placeholders for the fsp functions:
const doWork = () => new Promise(resolve =>
setTimeout(resolve, Math.random() * 2000));
const fsp = {
async copyFile() {
console.log("copyFile started");
await doWork();
console.log("copyFile returning");
},
async readFile() {
console.log("readFile started");
await doWork();
console.log("readFile returning");
},
async writeFile() {
console.log("writeFile started");
await doWork();
console.log("writeFile returning");
}
};
const _prefixJS = async file => {
console.log('running...');
await fsp.copyFile(file, `${file}.bak`);
const content = await fsp.readFile(file, 'utf8');
// Some modifications...
await fsp.writeFile(file, "newContent");
console.log('stopping!');
};
console.log("calling _prefixJS");
_prefixJS()
.then(() => {
console.log("then handler on _prefixJS()");
})
.catch(error => {
console.log(`catch handler on _prefixJS(): ${error.message}`);
});
console.log("calling _prefixJS - done");
Here's my code which I use to delay process (for backoff)
export function promiseDelay(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
I would want to test it but I'm not able to. I tried working with fakeTimers but my test never ends.
test('promiseDelay delays for 1s', async (done) => {
jest.useFakeTimers();
Promise.resolve().then(() => jest.advanceTimersByTime(100));
await promiseDelay(100);
});
promiseDelay returns a Promise that resolves after ms so call a spy in then and test to see if the spy has been called after different intervals:
describe('promiseDelay', () => {
beforeEach(() => { jest.useFakeTimers(); });
afterEach(() => { jest.useRealTimers(); });
test('should not resolve until timeout has elapsed', async () => {
const spy = jest.fn();
promiseDelay(100).then(spy); // <= resolve after 100ms
jest.advanceTimersByTime(20); // <= advance less than 100ms
await Promise.resolve(); // let any pending callbacks in PromiseJobs run
expect(spy).not.toHaveBeenCalled(); // SUCCESS
jest.advanceTimersByTime(80); // <= advance the rest of the time
await Promise.resolve(); // let any pending callbacks in PromiseJobs run
expect(spy).toHaveBeenCalled(); // SUCCESS
});
});
Note that test code is synchronous and Timer Mocks make setTimeout synchronous but then queues a callback in PromiseJobs so any queued callbacks need to be allowed to run before testing if the spy has been called.
This can be done by using an async test function and calling await on a resolved Promise which effectively queues the rest of the test at the end of PromiseJobs allowing any pending callbacks to run before the test continues.
Additional information about how promises and fake timers interact is available in my answer here.
I think you just need to return the promise from the function like
test('promiseDelay delays for 1s',() => {
jest.useFakeTimers();
return Promise.resolve().then(() => jest.advanceTimersByTime(100));
});
and then spy the setTimeout to be called once.
I saw this good article to introduce async/await from Google.
However, I couldn't understand why these code run in parallel
async function parallel() {
const wait1 = wait(500);
const wait2 = wait(500);
await wait1;
await wait2;
return "done!";
}
And this run in series
async function series() {
await wait(500);
await wait(500);
return "done!";
}
Why is the key difference between these two methods ?
In my opinion, both of them are await promise and should work the same result.
Here is my test code. Can run in browser console which support async/await.
function wait(){
return new Promise((res)=>{setTimeout(()=>{res()}, 2000)})
}
async function parallel() {
const wait1 = wait();
const wait2 = wait();
await wait1;
await wait2;
return "done!";
}
async function series() {
await wait();
await wait();
return "done!";
}
parallel().then(res => console.log("parallel!"))
series().then(res => console.log("series!"))
======
Thanks for the answers.
But I still have some question. Why exact the async/await means?
In my knowledge, constructing Promise instance would execute directly.
Here is my test code
function wait(){
return new Promise((res)=>{setTimeout(()=>{console.log("wait!");res();}, 2000)})
}
wait()
//Promise {<pending>}
//wait!
let w = wait()
//undefined
//wait!
let w = await wait()
//wait!
//undefined
async function test(){await wait()}
// undefined
test()
//Promise {<pending>}
//wait!
So why const wait1 = wait(); inside parallel function execute directly?
By the way, should I open another question to ask these question?
await doesn't cause the Promise or its setTimeout() to start, which seems to be what you're expecting. The call to wait() alone starts them immediately, whether there's an active await on the promise or not.
await only helps you know when the already on-going operation, tracked through the promise, has completed.
So, the difference is just due to when wait() is being called and starting each timeout:
parallel() calls wait() back-to-back as quickly as the engine can get from one to the next, before either are awaited, so the 2 timeouts begin/end at nearly the same time.
series() forces the 2nd wait() to not be called until after the 1st has completed, by having an await act in between them.
Regarding your edit, async and await are syntactic sugar. Using them, the engine will modify your function at runtime, inserting additional code needed to interact with the promises.
A (possible, but not precise) equivalent of series() without await and async might be:
function series() {
return Promise.resolve()
.then(function () { return wait(500) })
.then(function () { return wait(500) })
.then(function () { return "done!"; });
}
And for parallel():
function parallel() {
const wait1 = wait(500);
const wait2 = wait(500);
return Promise.resolve()
.then(wait1)
.then(wait2)
.then(function () { return "done!"; });
}
In parallel(), you call both methods and then await their results while in series() you await the result of the first wait() call before calling the second wait().
Why exact the async/await means? In my knowledge, constructing Promise instance would execute directly.
The Promise instance is returned immediately, i.e. synchronously. But the value of the Promise is evaluated asynchronously by calling the first parameter given to its constructor, a function usually called resolve. This is what you are awaiting for.
The difference is that in parallel, both of the promises are scheduled right after each other.
In series, you wait for the first wait to resolve and then schedule the second wait to be called.
If we expand the methods we'd get something similar to:
function parallel() {
const wait1 = wait();
const wait2 = wait();
// Both wait1 and wait2 timeouts are scheduled
return wait1
.then(() => wait2)
.then(() => "done");
}
function series() {
// The second wait happens _after_ the first has been *resolved*
return wait()
.then(() => wait())
.then(() => "done");
}