Hello I need to call a REST function with an ID, that returns a promise in React.js. This function will at some point contain a certain value in its response when called . Until another service has processed an initial request this value will be null.
This is what I have done so far:
while(myVariable){
myfunction(myID).then( (response) => {
if(response['value'] != null
myVariable = false;
}
});
}
The problem with this code is that the while loop is called as fast as possible and thus completely utilises the computer. So I need a function that allows me to poll for a result by an ID until the response of one of the function calls contains a valid response.
I have tried the following method but without success, because I don't have a fixed number of runs:
Wait promise inside for loop
Thanks in regards.
As you state, the problem is that the while loop runs eagerly, not waiting for each promise to resolve.
One way to solve that is to use recursion. Recursion gives you more control over when exactly you want to 'loop' next:
let getValue = () => {
myFunction(myID).then(response => {
if (response['value'] === null) {
setTimeout(getValue);
} else {
// here you know the other service has processed the initial request
}
});
};
First I wrapped the whole thing in a function called getValue. Note that this function is only called again after the promise resolves. (The call to setTimeout is a trick to use recursion without consuming the stack.) If this still runs too quickly, pass an additional parameter of 100 or so to the setTimeout invocation.
Alternatively, you may be able to use async/await similarly to the link you shared. I'm no expert on async/await but it should work the same with while loops as with for loops, judging by this and this.
You can use the async function with await.
I also use a delay function to delay each call to the myfunction().
While you get a response, you can break the while loop.
const delay = ms => new Promise((resolve, reject) => setTimeout(resolve, ms));
async function main() {
const myID = 1;
let response;
while (true) {
response = await myfunction(myID);
if (response["value"] != null) {
break;
}
await delay(5000);
}
//do Something once you get the response here below:
}
main();
First, this is a very specific case of doing it the wrong way on-purpose to retrofit an asynchronous call into a very synchronous codebase that is many thousands of lines long and time doesn't currently afford the ability to make the changes to "do it right." It hurts every fiber of my being, but reality and ideals often do not mesh. I know this sucks.
OK, that out of the way, how do I make it so that I could:
function doSomething() {
var data;
function callBack(d) {
data = d;
}
myAsynchronousCall(param1, callBack);
// block here and return data when the callback is finished
return data;
}
The examples (or lack thereof) all use libraries and/or compilers, both of which are not viable for this solution. I need a concrete example of how to make it block (e.g. NOT leave the doSomething function until the callback is called) WITHOUT freezing the UI. If such a thing is possible in JS.
"don't tell me about how I should just do it "the right way" or whatever"
OK. but you should really do it the right way... or whatever
" I need a concrete example of how to make it block ... WITHOUT freezing the UI. If such a thing is possible in JS."
No, it is impossible to block the running JavaScript without blocking the UI.
Given the lack of information, it's tough to offer a solution, but one option may be to have the calling function do some polling to check a global variable, then have the callback set data to the global.
function doSomething() {
// callback sets the received data to a global var
function callBack(d) {
window.data = d;
}
// start the async
myAsynchronousCall(param1, callBack);
}
// start the function
doSomething();
// make sure the global is clear
window.data = null
// start polling at an interval until the data is found at the global
var intvl = setInterval(function() {
if (window.data) {
clearInterval(intvl);
console.log(data);
}
}, 100);
All of this assumes that you can modify doSomething(). I don't know if that's in the cards.
If it can be modified, then I don't know why you wouldn't just pass a callback to doSomething() to be called from the other callback, but I better stop before I get into trouble. ;)
Oh, what the heck. You gave an example that suggests it can be done correctly, so I'm going to show that solution...
function doSomething( func ) {
function callBack(d) {
func( d );
}
myAsynchronousCall(param1, callBack);
}
doSomething(function(data) {
console.log(data);
});
Because your example includes a callback that is passed to the async call, the right way would be to pass a function to doSomething() to be invoked from the callback.
Of course if that's the only thing the callback is doing, you'd just pass func directly...
myAsynchronousCall(param1, func);
Async functions, a feature in ES2017, make async code look sync by using promises (a particular form of async code) and the await keyword. Also notice in the code examples below the keyword async in front of the function keyword that signifies an async/await function. The await keyword won't work without being in a function pre-fixed with the async keyword. Since currently there is no exception to this that means no top level awaits will work (top level awaits meaning an await outside of any function). Though there is a proposal for top-level await.
ES2017 was ratified (i.e. finalized) as the standard for JavaScript on June 27th, 2017. Async await may already work in your browser, but if not you can still use the functionality using a javascript transpiler like babel or traceur. Chrome 55 has full support of async functions. So if you have a newer browser you may be able to try out the code below.
See kangax's es2017 compatibility table for browser compatibility.
Here's an example async await function called doAsync which takes three one second pauses and prints the time difference after each pause from the start time:
function timeoutPromise (time) {
return new Promise(function (resolve) {
setTimeout(function () {
resolve(Date.now());
}, time)
})
}
function doSomethingAsync () {
return timeoutPromise(1000);
}
async function doAsync () {
var start = Date.now(), time;
console.log(0);
time = await doSomethingAsync();
console.log(time - start);
time = await doSomethingAsync();
console.log(time - start);
time = await doSomethingAsync();
console.log(time - start);
}
doAsync();
When the await keyword is placed before a promise value (in this case the promise value is the value returned by the function doSomethingAsync) the await keyword will pause execution of the function call, but it won't pause any other functions and it will continue executing other code until the promise resolves. After the promise resolves it will unwrap the value of the promise and you can think of the await and promise expression as now being replaced by that unwrapped value.
So, since await just pauses waits for then unwraps a value before executing the rest of the line you can use it in for loops and inside function calls like in the below example which collects time differences awaited in an array and prints out the array.
function timeoutPromise (time) {
return new Promise(function (resolve) {
setTimeout(function () {
resolve(Date.now());
}, time)
})
}
function doSomethingAsync () {
return timeoutPromise(1000);
}
// this calls each promise returning function one after the other
async function doAsync () {
var response = [];
var start = Date.now();
// each index is a promise returning function
var promiseFuncs= [doSomethingAsync, doSomethingAsync, doSomethingAsync];
for(var i = 0; i < promiseFuncs.length; ++i) {
var promiseFunc = promiseFuncs[i];
response.push(await promiseFunc() - start);
console.log(response);
}
// do something with response which is an array of values that were from resolved promises.
return response
}
doAsync().then(function (response) {
console.log(response)
})
The async function itself returns a promise so you can use that as a promise with chaining like I do above or within another async await function.
The function above would wait for each response before sending another request if you would like to send the requests concurrently you can use Promise.all.
// no change
function timeoutPromise (time) {
return new Promise(function (resolve) {
setTimeout(function () {
resolve(Date.now());
}, time)
})
}
// no change
function doSomethingAsync () {
return timeoutPromise(1000);
}
// this function calls the async promise returning functions all at around the same time
async function doAsync () {
var start = Date.now();
// we are now using promise all to await all promises to settle
var responses = await Promise.all([doSomethingAsync(), doSomethingAsync(), doSomethingAsync()]);
return responses.map(x=>x-start);
}
// no change
doAsync().then(function (response) {
console.log(response)
})
If the promise possibly rejects you can wrap it in a try catch or skip the try catch and let the error propagate to the async/await functions catch call. You should be careful not to leave promise errors unhandled especially in Node.js. Below are some examples that show off how errors work.
function timeoutReject (time) {
return new Promise(function (resolve, reject) {
setTimeout(function () {
reject(new Error("OOPS well you got an error at TIMESTAMP: " + Date.now()));
}, time)
})
}
function doErrorAsync () {
return timeoutReject(1000);
}
var log = (...args)=>console.log(...args);
var logErr = (...args)=>console.error(...args);
async function unpropogatedError () {
// promise is not awaited or returned so it does not propogate the error
doErrorAsync();
return "finished unpropogatedError successfully";
}
unpropogatedError().then(log).catch(logErr)
async function handledError () {
var start = Date.now();
try {
console.log((await doErrorAsync()) - start);
console.log("past error");
} catch (e) {
console.log("in catch we handled the error");
}
return "finished handledError successfully";
}
handledError().then(log).catch(logErr)
// example of how error propogates to chained catch method
async function propogatedError () {
var start = Date.now();
var time = await doErrorAsync() - start;
console.log(time - start);
return "finished propogatedError successfully";
}
// this is what prints propogatedError's error.
propogatedError().then(log).catch(logErr)
If you go here you can see the finished proposals for upcoming ECMAScript versions.
An alternative to this that can be used with just ES2015 (ES6) is to use a special function which wraps a generator function. Generator functions have a yield keyword which may be used to replicate the await keyword with a surrounding function. The yield keyword and generator function are a lot more general purpose and can do many more things then just what the async await function does. If you want a generator function wrapper that can be used to replicate async await I would check out co.js. By the way co's function much like async await functions return a promise. Honestly though at this point browser compatibility is about the same for both generator functions and async functions so if you just want the async await functionality you should use Async functions without co.js.
(I recommend just using async/await it's pretty widely supported in most environments that the above strikethrough is supported in.)
Browser support is actually pretty good now for Async functions (as of 2017) in all major current browsers (Chrome, Safari, and Edge) except IE.
Take a look at JQuery Promises:
http://api.jquery.com/promise/
http://api.jquery.com/jQuery.when/
http://api.jquery.com/deferred.promise/
Refactor the code:
var dfd = new jQuery.Deferred();
function callBack(data) {
dfd.notify(data);
}
// do the async call.
myAsynchronousCall(param1, callBack);
function doSomething(data) {
// do stuff with data...
}
$.when(dfd).then(doSomething);
You can force asynchronous JavaScript in NodeJS to be synchronous with sync-rpc.
It will definitely freeze your UI though, so I'm still a naysayer when it comes to whether what it's possible to take the shortcut you need to take. It's not possible to suspend the One And Only Thread in JavaScript, even if NodeJS lets you block it sometimes. No callbacks, events, anything asynchronous at all will be able to process until your promise resolves. So unless you the reader have an unavoidable situation like the OP (or, in my case, are writing a glorified shell script with no callbacks, events, etc.), DO NOT DO THIS!
But here's how you can do this:
./calling-file.js
var createClient = require('sync-rpc');
var mySynchronousCall = createClient(require.resolve('./my-asynchronous-call'), 'init data');
var param1 = 'test data'
var data = mySynchronousCall(param1);
console.log(data); // prints: received "test data" after "init data"
./my-asynchronous-call.js
function init(initData) {
return function(param1) {
// Return a promise here and the resulting rpc client will be synchronous
return Promise.resolve('received "' + param1 + '" after "' + initData + '"');
};
}
module.exports = init;
LIMITATIONS:
These are both a consequence of how sync-rpc is implemented, which is by abusing require('child_process').spawnSync:
This will not work in the browser.
The arguments to your function must be serializable. Your arguments will pass in and out of JSON.stringify, so functions and non-enumerable properties like prototype chains will be lost.
There is one nice workaround at http://taskjs.org/
It uses generators which are new to javascript. So it's currently not implemented by most browsers. I tested it in firefox, and for me it is nice way to wrap asynchronous function.
Here is example code from project GitHub
var { Deferred } = task;
spawn(function() {
out.innerHTML = "reading...\n";
try {
var d = yield read("read.html");
alert(d.responseText.length);
} catch (e) {
e.stack.split(/\n/).forEach(function(line) { console.log(line) });
console.log("");
out.innerHTML = "error: " + e;
}
});
function read(url, method) {
method = method || "GET";
var xhr = new XMLHttpRequest();
var deferred = new Deferred();
xhr.onreadystatechange = function() {
if (xhr.readyState === 4) {
if (xhr.status >= 400) {
var e = new Error(xhr.statusText);
e.status = xhr.status;
deferred.reject(e);
} else {
deferred.resolve({
responseText: xhr.responseText
});
}
}
};
xhr.open(method, url, true);
xhr.send();
return deferred.promise;
}
What you want is actually possible now. If you can run the asynchronous code in a service worker, and the synchronous code in a web worker, then you can have the web worker send a synchronous XHR to the service worker, and while the service worker does the async things, the web worker's thread will wait. This is not a great approach, but it could work.
let result;
async_function().then(r => result = r);
while (result === undefined) // Wait result from async_function
require('deasync').sleep(100);
In Node.js it's possible to write synchronous code which actually invokes asynchronous operations.
node-fibers allows this. It's a 3rd party native extension provided as an npm module.
It implements fibers/coroutines, so when a specific fiber is blocked waiting for asynchronous operation, the whole program events loop doesn't block - another fiber (if exists) continues its job.
With fibers your code would look like this:
var Fiber = require('fibers');
function doSomething() {
var fiber = Fiber.current;
function callBack(data) {
fiber.run(data);
}
myAsynchronousCall(param1, callBack);
// execution blocks here
var data = Fiber.yield();
return data;
}
// The whole program must be wrapped with Fiber
Fiber(function main() {
var data = doSomething();
console.log(data);
}).run();
Note, that you should avoid it and use async/await instead. See below a note from the project readme https://github.com/laverdet/node-fibers:
NOTE OF OBSOLESCENCE -- The author of this project recommends you avoid its use if possible. The original version of this module targeted nodejs v0.1.x in early 2011 when JavaScript on the server looked a lot different. Since then async/await, Promises, and Generators were standardized and the ecosystem as a whole has moved in that direction.
I'll continue to support newer versions of nodejs as long as possible but v8 and nodejs are extraordinarily complex and dynamic platforms. It is inevitable that one day this library will abruptly stop working and no one will be able to do anything about it.
I'd like to say thank you to all the users of fibers, your support over the years has meant a lot to me.
Using Node 16's worker threads actually makes this possible, The following example the main thread is running the asynchronous code while the worker thread is waiting for it synchronously.
Not that is is very useful, but it at least does vaguely what the original question asked by waiting for asynchronous code synchronously.
const {
Worker, isMainThread, parentPort, receiveMessageOnPort
} = require('worker_threads');
if (isMainThread) {
const worker = new Worker(__filename);
worker.on('message', async () => {
worker.postMessage(await doAsyncStuff());
});
} else {
console.log(doStuffSync());
}
function doStuffSync(){
parentPort.postMessage({fn: 'doStuff'});
let message;
while (!message) {
message = receiveMessageOnPort(parentPort)
}
return message;
}
function doAsyncStuff(){
return new Promise((resolve) => setTimeout(() => resolve("A test"), 1000));
}
One thing people might not consider: If you control the async function (which other pieces of code depend on), AND the codepath it would take is not necessarily asynchronous, you can make it synchronous (without breaking those other pieces of code) by creating an optional parameter.
Currently:
async function myFunc(args_etcetc) {
// you wrote this
return 'stuff';
}
(async function main() {
var result = await myFunc('argsetcetc');
console.log('async result:' result);
})()
Consider:
function myFunc(args_etcetc, opts={}) {
/*
param opts :: {sync:Boolean} -- whether to return a Promise or not
*/
var {sync=false} = opts;
if (sync===true)
return 'stuff';
else
return new Promise((RETURN,REJECT)=> {
RETURN('stuff');
});
}
// async code still works just like before:
(async function main() {
var result = await myFunc('argsetcetc');
console.log('async result:', result);
})();
// prints: 'stuff'
// new sync code works, if you specify sync mode:
(function main() {
var result = myFunc('argsetcetc', {sync:true});
console.log('sync result:', result);
})();
// prints: 'stuff'
Of course this doesn't work if the async function relies on inherently async operations (network requests, etc.), in which case the endeavor is futile (without effectively waiting idle-spinning for no reason).
Also this is fairly ugly to return either a value or a Promise depending on the options passed in.
("Why would I have written an async function if it didn't use async constructs?" one might ask? Perhaps some modalities/parameters of the function require asynchronicity and others don't, and due to code duplication you wanted a monolithic block rather than separate modular chunks of code in different functions... For example perhaps the argument is either localDatabase (which doesn't require await) or remoteDatabase (which does). Then you could runtime error if you try to do {sync:true} on the remote database. Perhaps this scenario is indicative of another problem, but there you go.)
This ability of promises includes two key features of synchronous operations as follows (or then() accepts two callbacks).
When you get the result, call resolve() and pass the final result.
In case of error, call reject().
The idea is that the result is passed through the chain of .then() handlers.
const synchronize = (() => {
let chain = Promise.resolve()
return async (promise) => {
return chain = chain.then(promise)
}
})()
I wondered the same thing and noticed that the currently best answer contains the right idea in my mind for most use cases, but forgets to mention a couple of things. When using a global variable to lock execution, we're talking about Semaphores, and there are some packages which implement those (my recommendation: async-sema). I think this makes it a little simpler and cleaner.
import { Sema } from 'async-sema'
const sema = new Sema(1) // allow only one concurrent process
async function doSomething() {
var data;
await sema.acquire();
// only one process gets inside here
data = await myAsynchronousCall(param1);
sema.release();
return data;
}
The advantage is obviously that the rest of your program can still do other things asynchronously, only the single block is kind of forced to be synchronously. Disadvantage is that you have to be careful what and where to lock, try/catch/finally possible errors, etc.
You can also convert it into callbacks.
function thirdPartyFoo(callback) {
callback("Hello World");
}
function foo() {
var fooVariable;
thirdPartyFoo(function(data) {
fooVariable = data;
});
return fooVariable;
}
var temp = foo();
console.log(temp);
The idea that you hope to achieve can be made possible if you tweak the requirement a little bit
The below code is possible if your runtime supports the ES6 specification.
More about async functions
async function myAsynchronousCall(param1) {
// logic for myAsynchronous call
return d;
}
function doSomething() {
var data = await myAsynchronousCall(param1); //'blocks' here until the async call is finished
return data;
}
I was unsure how node.js was able to realize what functions where async and which were not and how to create a custom async function.
Say I wanted to create a custom asynchronous function. I would be surprised if just because I called my last argument to the async function callback or cb that it would just know its an async function:
function f(arg1, callback){
//do stuff with arg1
arg1.doStuff()
//call callback
callback(null, arg1.result());
}
I tried something like that and it did not work async. How do you tell node.js that f is actually async?
NOTE: this answer was written in 2014, before the existence of async function, and before Promises gaining popularity. While the same principles apply as well, I would recommend reading on Promises before trying to get your head around how they relate to "traditional" callback-driven async functions.
To create a function that calls its callback asynchronously, you have to use some platform-provided async primitive (typically IO-related) on it - timers, reading from the filesystem, making a request etc.
For example, this function takes a callback argument, and calls it 100ms after:
function asyncFn(callback) {
setTimeout(() => {
callback();
}, 100);
}
A possible reason for making a function async when it doesn't need to be, is for API consistency. For example, suppose you have a function that makes a network request, and caches the result for later calls:
var cache = null;
function makeRequest(callback) {
if (!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
callback(cache);
}
}
The problem is, this function is inconsistent: sometimes it is asynchronous, sometimes it isn't. Suppose you have a consumer like this:
makeRequest(result => doSomethingWithResult(result));
doSomethingElse();
The doSomethingElse function may run before or after the doSomethingWithResult function, depending on whether the result was cached or not. Now, if you use an async primitive on the makeRequest function, such as process.nextTick:
var cache = null;
function makeRequest(callback) {
if(!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
process.nextTick(() => callback(cache));
}
}
The call is always async, and doSomethingElse always runs before doSomethingWithResult.
Only native functions (with access to the event loop) are asynchronous. You would need to call one of them to get asynchronity for your callback. See What is a simple example of an asynchronous javascript function?.
If you aren't using any, there's hardly a reason to make your function asynchronous.
I would like to do a synchronous loop in a part of my code.
The function saveInDatabase checks if the item title (string) already exists in the database. That's why it can't be resolved in parallel, otherwise the condition will never apply (and would create duplicates).
Promise.all(arr.map(item => {
saveInDatabase(item).then((myResult) => ... );
}));
I tried to encapsulate this function into separate promises, also tried with npm packages (synchronous.js, sync), but it seems that it does not fit with my code.
Maybe this solution is completely silly.
Do you think it's a better idea to replace promise.all by a synchronous loop (foreach for example) ? The problem is that I need the results of each iteration...
I'm using Node 6.11.2. Could you give me some tips to handle that ?
Thank you in advance.
Without using await (which is not in node.js v6.11.2, but would make this simpler), a classic pattern for serializing a bunch of async operations that return a promise is to use a reduce() loop like this:
arr.reduce(function(p, item) {
return p.then(function() {
return saveInDatabase(item).then((myResult) => ... );
});
}, Promise.resolve()).then(function() {
// all done here
}).catch(function(err) {
// error here
});
If you want to save all the results, you can use your .then(myResult => ...) handler to .push() the result into an array which you can access when done.
This will serialize all the calls to saveInDatabase(item) to it waits for the first one to be done before calling the second one, waits for the second one to be done before calling the third one, etc...
The default implementation here will stop if saveInDatabase(item) rejects. If you want to keep going (you don't say in your question), even when it gives an error, then you can add a .catch() to it to turn the rejected promise into a fulfilled promise.
In node.js v7+, you can use await in a regular for loop:
async function myFunc() {
let results = [];
for (let item of arr) {
let r = await saveInDatabase(item).then((myResult) => ... );
results.push(r);
}
return results;
}
myFunc().then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
If you could run all the requests in parallel, then you could do that like this:
Promise.all(arr.map(item => {
return saveInDatabase(item).then((myResult) => ... );
})).then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
In any of these, if you don't want it to stop upon a rejection, then add a .catch() to your saveInDatabase() promise chain to turn the rejection into a resolved promise with some known value or error value you can detect.
For example, I have function which Promise.resolve() if I already have any cached entity id else it make ajax call to reserve entity id and then Promise.resolve() new entity id
function getReservedEntityId(collectionName) {
//if (!haveCachedIds) {
//make ajax call to reserve new ids
Promise.resolve(newId);
}
return Promise.resolve(cachedId);
};
How can we synchronously call it multiple times to reserve multiple entity ids?
P.S. I know that the correct approach is to make this function take parameter that will specify the count of entity ids and make request and return ids accordingly but I wanted to understand how can we call synchronously multiple times any function which is returning promise.
First, the implementation of getReservedEntityId() needs to make correct use of promises. I recommend a thorough reading of how promises work. In particular, it's important to understand that when your function performs an asynchronous task, you need to return a promise that will either resolve or reject based on the result of the asynchronous task.
function getReservedEntityId(collectionName) {
if (haveCachedIds) {
return Promise.resolve(cachedId);
} else {
return new Promise((resolve, reject) => {
// Make the AJAX call and call resolve(newId) in the success callback
// or call reject(errCode) in the failure callback.
// The arguments newId and errCode can be any values you like and are
// the values that get passed to the next link in the promise chain
// i.e. the values passed to then() or catch()
});
}
}
With that squared away, there are two recommended ways to make the calls synchronous:
1) Utilize a promise chain
getReservedEntityId(collectionName)
.then((id) => {
// Probably want to do something with `id` first...
return getReservedEntityId(collectionName);
})
.then( ... )
.then( ... );
Of course, if you're going to pass the same function to each .then() call, you could just as well declare it as a regular function so as to not repeat yourself.
2) Using async/await
This is a new ES2017 feature and is still not widely supported. As of the time of this writing, Node.js supports async/await with the --harmony flag, but most browsers do not. That said, async/await is intended for this exact purpose, treating functions returning promises as though they were synchronous. If you want to start using async/await in your code now, it is common practice to use JavaScript transpilers which which transpile your future-ready JavaScript to code that is supported by all major browsers.
This is how you would use async/await:
(async function someAsyncFunction {
const id1 = await getReservedEntityId(collectionName);
const id2 = await getReservedEntityId(collectionName);
const id3 = await getReservedEntityId(collectionName);
.
.
.
})();
The syntax is much nicer and more readable than the promise chain because it's designed for this exact purpose. Note that I have made the function self-invoking here so that it matches your behavior without having to make an extra function call. But you can use and call a function defined with async function just like any other function that returns a promise.
#fvgs your answer is also correct. But here's the complete solution and the challenge which I have faced is to maintain list of reserveIds which was in response of every getReservedEntityId call.
getEntityIds: function (entity, count) {
if (!count || count == 1)
return Utils.getReservedEntityId(entity);
var promise = new Promise(function(resolve, reject) {
var result = [];
var chain = Utils.getReservedEntityId(entity);
var callBack = function CB (result, newId) {
result.push(newId);
if (result.length == count)
resolve(result);
else
return Utils.getReservedEntityId(entity);
}.bind(null, result);
for (var i=1; i <= count; i++) {
chain.then(callBack);
}
chain.catch(reject);
});
return promise;
}