We Keep Coding

Javascript functions not called in correct order, need async

The code snippet below is not running in the proper order... the for loop is calling the downloadFile function call over every iteration before any other functions can resolve, which is affecting the number of links returned to the final callback.
What I need is an loop iteration to finish only once the appendLinksList function has resolved and returned a link. I'm assuming I need async to block the functions? How would I use async to get the proper flow instead of the downloadFiles function being called before anything else can resolve a value?
Note that the callback functions have been defined above this code snippet
const entries [ /* Array of objects containing a path_display property */];
let links = [];
for(let i = 0; i < entries.length; i++) {
const appendLinksList = link => {
if(i !== entries.length - 1) return links = [...links, link];
return callback(null, links);
};
const downloadFile = path => {
Dropbox.filesGetTemporaryLink({ path })
.then(file => {
return appendLinksList(file.link);
})
.catch(err => {
return res.status(500).json(err.message);
});
};
downloadFile(entries[i].path_display);
};

You can wrap your loop in an immediately invoked async function.
const linksResolved = await (async() => {
let links = [];
for(let i = 0; i < entries.length; i++) {
const appendLinksList = link => {
if(i !== entries.length - 1) return links = [...links, link];
return callback(null, links);
};
const path = entries[i].path_display;
await Dropbox.filesGetTemporaryLink({ path })
.then(file => {
return appendLinksList(file.link);
})
.catch(err => {
return res.status(500).json(err.message);
});
};
};
return links;
})()
However, the result you want (links) will forever be a promise.
Can only be resolved in an async function. Or you can call a callback with links from inside the immediatelyInvokedAsyncFunction

Using promise in loop results in Promise failure

I'd like to reuse the same code in a loop. This code contains promises. However, when iterating, this code results in an error.
I've tried using for and while loops. There seems to be no issue when I use the for loop for a single iteration.
Here is a minimal version of my code:
var search_url = /* Some initial URL */
var glued = "";
for(var i = 0; i < 2; i++)
{
const prom = request(search_url)
.then(function success(response /* An array from a XMLHTTPRequest*/) {
if (/* Some condition */)
{
search_url = /* Gets next URL */
glued += processQuery(response[0]);
} else {
console.log("Done.")
}
})
.catch(function failure(err) {
console.error(err.message); // TODO: do something w error
})
}
document.getElementById('api-content').textContent = glued;
I expect the results to append to the variable glued but instead, I get an error: failure Promise.catch (async) (anonymous) after the first iteration of the loop.

Answer:
You can use the Symbol.iterator in accordance with for await to perform asynchronous execution of your promises. This can be packaged up into a constructor, in the example case it's called Serial (because we're going through promises one by one, in order)
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
What is the above?
It's a constructor called Serial.
It takes as an argument an array of Functions that return Promises.
The functions are stored in Serial.promises
It has an empty array stored in Serial.resolved - this will store the resolved promise requests.
It has two methods:
addPromise: Takes a Function that returns a Promise and adds it to Serial.promises
resolve: Asynchronously calls a custom Symbol.iterator. This iterator goes through every single promise, waits for it to be completed, and adds it to Serial.resolved. Once this is completed, it returns a map function that acts on the populated Serial.resolved array. This allows you to simply call resolve and then provide a callback of what to do with the array of responses. A.e. .resolve()((resolved_requests) => //do something with resolved_requests)
Why does it work?
Although many people don't realize this Symbol.iterator is much more powerful than standard for loops. This is for two big reasons.
The first reason, and the one that is applicable in this situation, is because it allows for asynchronous calls that can affect the state of the applied object.
The second reason is that it can be used to provide two different types of data from the same object. A.e. You may have an array that you would like to read the contents of:
let arr = [1,2,3,4];
You can use a for loop or forEach to get the data:
arr.forEach(v => console.log(v));
// 1, 2, 3, 4
But if you adjust the iterator:
arr[Symbol.iterator] = function* () {
yield* this.map(v => v+1);
};
You get this:
arr.forEach(v => console.log(v));
// 1, 2, 3, 4
for(let v of arr) console.log(v);
// 2, 3, 4, 5
This is useful for many different reasons, including timestamping requests/mapping references, etc. If you'd like to know more please take a look at the ECMAScript Documentation: For in and For Of Statements
Use:
It can be used by calling the constructor with an Array of functions that return Promises. You can also add Function Promises to the Object by using
new Serial([])
.addPromise(() => fetch(url))
It doesn't run the Function Promises until you use the .resolve method.
This means that you can add promises ad hoc if you'd like before you do anything with the asynchronous calls. A.e. These two are the same:
With addPromise:
let promises = new Serial([() => fetch(url), () => fetch(url2), () => fetch(url3)]);
promises.addPromise(() => fetch(url4));
promises.resolve().then((responses) => responses)
Without addPromise:
let promises = new Serial([() => fetch(url), () => fetch(url2), () => fetch(url3), () => fetch(url4)])
.resolve().then((responses) => responses)
Data:
Since I can't really replicate your data calls, I opted for JSONPlaceholder (a fake online rest api) to show the promise requests in action.
The data looks like this:
let searchURLs = ["https://jsonplaceholder.typicode.com/todos/1",
"https://jsonplaceholder.typicode.com/todos/2",
"https://jsonplaceholder.typicode.com/todos/3"]
//since our constructor takes functions that return promises, I map over the URLS:
.map(url => () => fetch(url));
To get the responses we can call the above data using our constructor:
let promises = new Serial(searchURLS)
.resolve()
.then((resolved_array) => console.log(resolved_array));
Our resolved_array gives us an array of XHR Response Objects. You can see that here:
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
let searchURLs = ["https://jsonplaceholder.typicode.com/todos/1", "https://jsonplaceholder.typicode.com/todos/2", "https://jsonplaceholder.typicode.com/todos/3"].map(url => () => fetch(url));
let promises = new Serial(searchURLs).resolve().then((resolved_array) => console.log(resolved_array));
Getting Results to Screen:
I opted to use a closure function to simply add text to an output HTMLElement.
This is added like this:
HTML:
<output></output>
JS:
let output = ((selector) => (text) => document.querySelector(selector).textContent += text)("output");
Putting it together:
If we use the output snippet along with our Serial object the final functional code looks like this:
let promises = new Serial(searchURLs).resolve()
.then((resolved) => resolved.map(response =>
response.json()
.then(obj => output(obj.title))));
What's happening above is this:
we input all our functions that return promises. new Serial(searchURLS)
we tell it to resolve all the requests .resolve()
after it resolves all the requests, we tell it to take the requests and map the array .then(resolved => resolved.map
the responses we turn to objects by using .json method. This is necessary for JSON, but may not be necessary for you
after this is done, we use .then(obj => to tell it to do something with each computed response
we output the title to the screen using output(obj.title)
Result:
let output = ((selector) => (text) => document.querySelector(selector).textContent += text)("output");
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
let searchURLs = ["https://jsonplaceholder.typicode.com/todos/1", "https://jsonplaceholder.typicode.com/todos/2", "https://jsonplaceholder.typicode.com/todos/3"].map(url => () => fetch(url));
let promises = new Serial(searchURLs).resolve()
.then((resolved) => resolved.map(response =>
response.json()
.then(obj => output(obj.title))));
<output></output>
Why go this route?
It's reusable, functional, and if you import the Serial Constructor you can keep your code slim and comprehensible. If this is a cornerstone of your code, it'll be easy to maintain and use.
Using it with your code:
I will add how to specifically use this with your code to fully answer your question and so that you may understand further.
NOTE glued will be populated with the requested data, but it's unnecessary. I left it in because you may have wanted it stored for a reason outside the scope of your question and I don't want to make assumptions.
//setup urls:
var search_urls = ["https://jsonplaceholder.typicode.com/todos/1", "https://jsonplaceholder.typicode.com/todos/2"];
var request = (url) => () => fetch(url);
let my_requests = new Serial(search_urls.map(request));
//setup glued (you don't really need to, but if for some reason you want the info stored...
var glued = "";
//setup helper function to grab title(this is necessary for my specific data)
var addTitle = (req) => req.json().then(obj => (glued += obj.title, document.getElementById('api-content').textContent = glued));
// put it all together:
my_requests.resolve().then(requests => requests.map(addTitle));
Using it with your code - Working Example:
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
//setup urls:
var search_urls = ["https://jsonplaceholder.typicode.com/todos/1", "https://jsonplaceholder.typicode.com/todos/2"];
var request = (url) => () => fetch(url);
let my_requests = new Serial(search_urls.map(request));
//setup glued (you don't really need to, but if for some reason you want the info stored...
var glued = "";
//setup helper function to grab title(this is necessary for my specific data)
var addTitle = (req) => req.json().then(obj => (glued += obj.title, document.getElementById('api-content').textContent = glued));
// put it all together:
my_requests.resolve().then(requests => requests.map(addTitle));
<div id="api-content"></div>
Final Note
It's likely that we will be seeing a prototypal change to the Promise object in the future that allows for easy serialization of Promises. Currently (7/15/19) there is a TC39 Proposal that does add a lot of functionality to the Promise object but it hasn't been fully vetted yet, and as with many ideas trapped within the Proposal stage, it's almost impossible to tell when they will be implemented into Browsers, or even if the idea will stagnate and fall off the radar.
Until then workarounds like this are necessary and useful( the reason why I even went through the motions of constructing this Serializer object was for a transpiler I wrote in Node, but it's been very helpful beyond that! ) but do keep an eye out for any changes because you never know!
Hope this helps! Happy Coding!

Your best bet is probably going to be building up that glued variable with recursion.
Here's an example using recursion with a callback function:
var glued = "";
requestRecursively(/* Some initial URL string */, function() {
document.getElementById('api-content').textContent = glued;
});
function requestRecursively(url, cb) {
request(url).then(function (response) {
if (/* Some condition */) {
glued += processQuery(response[0]);
var next = /* Gets next URL string */;
if (next) {
// There's another URL. Make another request.
requestRecursively(next, cb);
} else {
// We're done. Invoke the callback;
cb();
}
} else {
console.log("Done.");
}
}).catch(function (err) {
console.error(err.message);
});
}

forEach function scoping - how to parse array to function without repeat function calls

I have a function that I call and within the function I use a promise, I also want to call this function ( it's actually the second function called in the code) first, retrieve some data, then parse this data through after my second function which calls out to also retrieve some data.
getAllProductGroups(): Promise<void> {
return new Promise((resolve, reject) => {
this.allEngagementTypes().subscribe(data => {
const products = {};
const potentialEvents = [];
data.forEach(product => {
// products[product.id] = product.description;
this.potentialForEvents(this.organization.id, product.id)
.subscribe(potentialEventData => {
potentialEvents.push(potentialEventData);
this.allEngagementAreas().subscribe(areas => {
this.organization.addAreas(areas, potentialEvents);
resolve();
}, error => reject(error));
});
});
})
});
}
I call the forEach on the first function call allEngagementTypes, as I need to use each element to then make my second function call on potentialForEvents, then I create an array with the responses with potentialEvents.push,
I want to then with this array, parse it after my third call 'allEngagementAreas' as when this calls on the 'addAreas' function, I want to parse the array potentialEvents, but because of the forEach It is sent 21 times, only two of which area actually sent before the 'addAreas' fully loads, I need just one array to be sent and for it to be parsed and be ready before the allEngagementAreas is actually called.
Any help is greatly appreciated.

if I understood your problem correctly:
getAllProductGroups(): Promise<void> {
return new Promise((resolve, reject) => {
this.allEngagementTypes().subscribe(data => {
const products = {};
const potentialEvents = [];
data.forEach(product => {
// products[product.id] = product.description;
this.potentialForEvents(this.organization.id, product.id)
.subscribe(potentialEventData => {
potentialEvents.push(potentialEventData);
if(potentialEvents.length === someConstValue) { //not sure how, but you need to find out the needed array length
resolve(potentialEvents);
}
});
});
})
}).then(potentialEvents => {
this.allEngagementAreas().subscribe(areas => {
this.organization.addAreas(areas, potentialEvents);
}, error => reject(error));
})
}
main idea is to split it to two promises, and resolve the first one when the array if filled. in forEach loops they usually compare the result length with some const value.

Iterating javascript object synchronously

I have an object like so:
let myObject = {
'db1': [db1_file1Id,db1_file2Id,db_1file3Id],
'db2': [db2_file1Id, db2_file2Id]
...
}
I iterate through through this object and on each iteration: I connect to the database, retrieve the file, do some stuff and save the file back. Basically asynchronous stuff.
for (let prop in myObject) {
if (myObject.hasOwnProperty(prop)) {
doStuff(prop, myObject[prop]);
}
}
Now the doStuff function makes sure I have a local scope so there is no inconsistencies. But still, the execution is not synchronous due to the asynchronous operations inside each loop. I basically need one db to be completely processed before moving on to the next. How do I fix this?
One approach that I thought of was to have recursive loop. But as per my understanding, this would require me to change my data structure extensively which is sub-optimal imo.
let arr; //containing my data
process(x) {
if (x < arr.length){
//process(x+1) is called inside doStuff after asynchronous operations are complete
doStuff(arr[x]);
}
}

You could use the solution you proposed at the end using Object.entries(obj). For example,
let arrProps = Object.entries(myObject);
process(index) {
if (index < arrProps.length){
// Call the callback once you complete execution of doStuff
doStuff(arrProps[index], () => process(index + 1));
}
}
Inside doStuff:
function doStuff(props, callback) {
// Process props
//finally in the promise of async call, on success call
.then(callback)
}
OR you could use a generator function, if you want to use for ... in loop.

The following will do what you ask, it returns an array of resolve values.
Do you want to stop processing if any one of them rejects? In case you need to make some changes, now it rejects if any of them reject and won't continue processing they keys in your object (object named myObject):
var myObject = {
'one': ["one"],
'two': ["two"]
};
var doStuff = arr =>
console.log("starting:", arr[0]) ||
Promise.resolve(arr[0]);
var [promise,result] =
Object.keys(myObject)
.reduce(
([promise,results], key) =>
[
promise
.then(
resolve =>
doStuff(myObject[key])
)
.then(
resolve => results.push(resolve)&&resolve
)
.then(
resolve => console.log("done:", resolve)
)
,results
]
, [Promise.resolve(), []]
)
promise.then(
_ => {
console.log("done all",result)
}
);
The answer ayushgp uses recursion, here is a working example that doesn't need changes to doSomething:
var myObject = {
'one': ["one"],
'two': ["two"]
};
var doStuff = arr =>
console.log("starting:",arr[0]) ||
Promise.resolve(arr[0])
var process = (arr,processFn) => {
const rec = (arr,processFn,promise,results) =>
arr.length === 0
? promise.then(_=>results)
: promise
.then(_ => processFn(arr[0][1]))
.then(result=>results.push(result)&&console.log("resolved:",result))
.then(_ => rec(arr.slice(1),processFn,promise,results));
return rec(arr,processFn,Promise.resolve(),[]);
};
process(
Object.keys(myObject).map(key=>[key,myObject[key]]),
doStuff
)
.then(
results => console.log("done all, results:",results)
);

One solution would be to make doStuff return a Promise which you can use to build a chain of promises using calls to then.
The Bluebird promise library provides this functionality with .each and .mapSeries.
You could implement it as:
Promise.forEachSeries = function(array, action) {
return array.reduce(function(prevPromise, value, index, array) {
return prevPromise.then(function() {
return action(value, index, array);
});
}, Promise.resolve());
}
You would use it like this:
Promise.forEachSeries(arr, doStuff);

The following code might be close to what you are asking. I use indices i and j to loop through databases and files respectively :
var dbs = {
db1: ["q", "w", "e", "r"],
db2: ["t", "y"]
};
var names = Object.keys(dbs);
var db, x, i = 0, j = 0;
if (names.length > 0) {
db = dbs[names[i]];
x = db[j];
console.log("start");
asyncProcessing(x)
.then(onSuccess)
.catch(onFailure);
}
function onFailure (e) {
console.log("[FAILURE]", e);
console.log("end");
}
function onSuccess (xx) {
console.log("[SUCCESS]", xx);
j = (j + 1) % db.length; // next j
if (j === 0) i = i + 1; // next i
if (i < names.length) {
db = dbs[names[i]];
x = db[j];
asyncProcessing(x)
.then(onSuccess)
.catch(onFailure);
} else {
console.log("end");
}
}
function asyncProcessing (x) {
return new Promise(function (resolve, reject) {
setTimeout(function () {
// force first two success then random
if (x === "q" || x === "w" || Math.random() * 3 > 1) {
resolve(x + x);
} else {
reject("Not lucky. Try again.");
}
}, 1000);
});
}

The Promise object represents the eventual completion (or failure) of an asynchronous operation, and its resulting value. you can try like .
$("#myPara").delay(4500).fadeOut().promise().done(function(){
$("#myHeading").attr("style","display:none;") ;
for(var i=10;i<15;i++){
console.log(i);
}
});
console.log("Hello promise !");
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<p id="myPara"> Hello </p>
<h1 id="myHeading">to be hide</h1>
for (let prop in myObject) {
if (myObject.hasOwnProperty(prop)) {
var stuff= doStuff(prop, myObject[prop]).promise().done(function(){
// Do whatever u want after completion of doStuff
});
}
}
Have a look at Mozila ref.

multiple promises running in parallel, $q.all needs chaining?

I have 3 parallel promises or api requests once all teh three are done, I need to call another api request based on the second promise and then finally call .then( of $q.all
Here is the code
getAllLocations() {
//make a promise call for all here .
var promise = [];
̶p̶r̶o̶m̶i̶s̶e̶.̶p̶u̶s̶h̶(̶t̶h̶i̶s̶.̶g̶e̶t̶A̶l̶l̶L̶o̶c̶a̶t̶i̶o̶n̶s̶(̶I̶d̶)̶.̶t̶h̶e̶n̶(̶
promise.push(this.getLocations(Id).then(
(locationsData) => {
this.locations = locationsData;
}));
promise.push(this.getAllStates(Id).then(
(resp) => {
this.states = resp.data;
}));
promise.push(this.getTerritories(Id).then(
(resp) => {
this.utilizations = resp.data;
}));
$q.all(promise).then(() => {
var nodePromise = [];
angular.forEach(this.states, function(node) {
var nodeId = node.Id;
nodePromise.push(this.getNodeHealthSummary(nodeId).then(
(resp) => {
node.healthStatus = resp.data.operationalStatus;
}));
this.$q.all(nodePromise).then(() => {
var index = this.states.indexOf(node);
this.states.splice(index, 1, angular.copy(node));
});
},this);
}).then(() => {
for (var i = 0; i < this.locations.length; i++) {
//do something here with this.states
}
this.gridData = this.locations;
});
}
I need this.states updated with healthStatus property when i am in the for loop of this.locations. ( the last.then )
However , i see that this.locations for loop is done ahead before the node.healthStatus property is set on each state.
How can this be done? Using Promises instead of $q is fine. Please let me know how can i achieve this , i have tried all in vain

The inner $q.all is called in each iteration of the forEach loop, and gets as argument the array that is being populated during that forEach loop. This is obviously not right; it should be called only once, and its result should be the return value of the then callback.
So instead of this block:
$q.all(promise).then(() => {
var nodePromise = [];
angular.forEach(this.states, function(node) {
var nodeId = node.Id;
nodePromise.push(this.getNodeHealthSummary(nodeId).then(
(resp) => {
node.healthStatus = resp.data.operationalStatus;
}));
this.$q.all(nodePromise).then(() => {
var index = this.states.indexOf(node);
this.states.splice(index, 1, angular.copy(node));
});
},this);
}).then( ......
Do this:
$q.all(promise).then(() => {
return $q.all(this.states.map((node, index) => {
return this.getNodeHealthSummary(node.Id).then(resp => {
node.healthStatus = resp.data.operationalStatus;
this.states[index] = angular.copy(node);
});
}));
}).then( ......