We Keep Coding

Issues with Async/Await during SOAP API call Javascript

Hopefully someone can point me to the right direction. I read up on waiting for functions to complete before continuing and I resolved myself to using await/async but I am just stuck now.
I tried to get the Async/Await process to work, tried to inject the await in various locations, with adjusting the functions to be async, but i can not get the PSA_Resultbody to return to the original request. Any pointers would be appreciated.
Thank you,
CE
PSA_Resultbody = ProcessPSAAPI(xmlpackage, PSA_Action);
console.log("3 - Returned data:" + PSA_Resultbody);
calls the below:
async function ProcessPSAAPI(xmlpackage, PSA_Action) { //psa action is part of the options
var options = {...};
var req = https.request(options, function (res) {
var chunks = [];
res.on("data", function (chunk) {
chunks.push(chunk);
});
res.on("end", function (chunk) {
var body = Buffer.concat(chunks);
console.log('0 - Start '+body.toString());
if(res.statusCode != 200) {
PSA_Resultcode = "Error: " +res.statusCode +" - "+ res.statusMessage;
} else {
PSA_Resultcode = "Success: " +res.statusCode +" - "+ res.statusMessage;
PSA_Resultbody = ParseResults(body.toString()); //parse the results for later use --SCRIPT NEEDS TO WAIT FOR RESULTBODY TO COMPLETE
console.log("1 -PSA_Resultbody as part of RES = "+PSA_Resultbody);
}
});
res.on("error", function (error) {
console.error(error);
PSA_Resultcode = res.statusCode +" - "+ res.statusMessage;
});
});
console.log('2 -RESULT BODY BEFORE SENDING BACK TO INITIATING FUNCTION: '+PSA_Resultbody);
req.write(xmlpackage);
req.end();
return PSA_Resultbody;
Based on the above, my console log order is: 3,2,0,1 in stead of 0,1,2,3.
0 and 1 will have the correct data, so the API Call does work, but 2 will be "undefined" and should have the same data that is in 1.

There's no way to await an event emitter, so using async in this case isn't going to be useful. You cannot "return" from inside an event either.
The solution here is to return a new custom promise and to use resolve() inside of the "end" event of your emitter.
It will look something like this:
function ProcessPSAAPI(xmlpackage, PSA_Action) {
return new Promise( (resolve, reject) => {
// other code
res.on("end", function (chunk) {
// other code
resolve(PSA_Resultbody);
});
res.on("error", function (error) {
// other code
reject(error);
});
});
}
Here's a quick tutorial on creating your own promises, which I've written to simplify comprehension of the subject (official docs are somewhat dry and complex imho).

I did not change your code. I just put the appropriate promise structure in to get you started. This should really be a lesson in promises. async await is a shorthand promise structure. A Promise is one way you wait on code. It can be thought of as an array of callbacks that will be executed when the Promise is resolved.
A simple promise works like this:
const myPromise = new Promise(function(resolve, reject) {
/* Your logic goes in here. It can be anything.
* But the important part to remember is that when you have success, resolve it.
* When you have a failure, reject it.
*/
someCallBackPattern(function(error, data) {
if(error) {
reject(error);
} else {
resolve(data);
}
});
});
// To get the data out you use 'then', and 'catch'. then has two arguments.
myPromise.then(function(data) {
// The first argument is the result from resolve.
}, function(err) {
// The second argument is the result from reject.
}).catch((err) => {
// you can also get to the error from the catch callback
});
This is kinda messy and complex. So there is async await.
async function() {
try {
const result = await myFunctionThatReturnsAPromise();
// result is the resolved data
} catch (err) {
// err is the rejected Error
}
}
function myFunctionThatReturnsAPromise() {
return new Promise((resolve, reject) => {
// your code
})
}
And thats how it works.
async function someFunction () { // You can not wait on results unless you are in an await function
PSA_Resultbody = await ProcessPSAAPI(xmlpackage, PSA_Action); // await on your results.
console.log("3 - Returned data:" + PSA_Resultbody);
}
function ProcessPSAAPI(xmlpackage, PSA_Action) { // This does not need to be async. Unless you are awaiting in it.
return new Promise((resolve, reject) => { // async await is a shorthand promise structure. Although you do not need to use promises. It really helps to get the structure correct.
var options = {...};
var req = https.request(options, function (res) {
var chunks = [];
res.on("data", function (chunk) {
chunks.push(chunk);
});
res.on("end", function (chunk) {
var body = Buffer.concat(chunks);
console.log('0 - Start '+body.toString());
if(res.statusCode != 200) {
PSA_Resultcode = "Error: " +res.statusCode +" - "+ res.statusMessage;
reject(new Error(PSA_Resultcode)); // Reject you errors
} else {
PSA_Resultcode = "Success: " +res.statusCode +" - "+ res.statusMessage;
PSA_Resultbody = ParseResults(body.toString()); //parse the results for later use --SCRIPT NEEDS TO WAIT FOR RESULTBODY TO COMPLETE
console.log("1 -PSA_Resultbody as part of RES = "+PSA_Resultbody);
resolve(PSA_Resultbody); // Resolve your result
}
});
res.on("error", function (error) {
console.error(error);
PSA_Resultcode = res.statusCode +" - "+ res.statusMessage;
reject(new Error(PSA_Resultcode)); // Reject you errors
});
});
console.log('2 -RESULT BODY BEFORE SENDING BACK TO INITIATING FUNCTION: '+PSA_Resultbody);
req.write(xmlpackage);
req.end();
})
}

Download files from links in array

I'm trying to download files from links in an array with a length of several thousand positions. The problem is that when I iterate over the array I hit a wall when trying to synchronize the file fetch and write ( Maximum call stack size exceeded). I've tried to make a recursive function and played with promises but I still haven't managed to find a solution.
Help please!
My code so far:
function download(url, dest, cb) {
return new Promise(function (resolve, reject) {
let request = https.get(url, function (response) {
let file = fs.createWriteStream(dest);
response.pipe(file);
file.on('finish', function () {
console.log('File downloaded')
resolve(file.close(cb));
});
}).on('error', function (err) {
reject(err)
});
})
};
function recursiveDownload(links, i) {
if (i < links.length) {
download(links[i], './data/' + i + '.csv')
.then(recursiveDownload(links, ++i))
} else {
console.log('ended recursion')
}
}
recursiveDownload(links, 0)

You can use a for loop to serialize all your promises and synchronize them. Try the following:
var promise = Promise.resolve();
for(let i = 0; i < links.length; i++){
promise = promise.then(()=> download(links[i], './data/' + i + '.csv'));
}
Or you can even chain your promises using Array.reduce() :
var promise = links.reduce((p, link, index) => p.then(()=>download(link, './data/' + index + '.csv')),Promise.resolve());

Because you call recursiveDownload immeadiately withou waiting for the download to succeed. You actually want to call it when .then calls back:
download(links[i], './data/' + i + '.csv')
.then(() => recursiveDownload(links, ++i))

How to wait for each iteration in for loop and return response as API response in nodeJS

I'm using for loop to iterate over an array of elements and to call the same function with different parameters inside the for loop. Here is my code:
exports.listTopSongs = function(query) {
return new Promise(function(resolve, reject) {
var str = query.split(","), category,
for(var i=0; i<str.length; i++) {
sampleFn(str[i], 'sample', resolve, reject);
}
});
};
function sampleFn(lang, cat, resolve, reject) {
client.on("error", function (err) {
console.log(err);
var err = new Error('Exception in redis client connection')
reject(err);
});
client.keys(lang, function (err, keys){
if (err) return console.log(err);
if(keys.length != 0) {
client.hgetall(keys, function (error, value) {
var objects = Object.keys(value);
result['title'] = lang;
result[cat] = [];
var x =0;
for(x; x<objects.length; x++) {
var val = objects[x];
User.findAll({attributes: ['X', 'Y', 'Z'],
where: {
A: val
}
}).then(data => {
if(data != null) {
//some actions with data and stored it seperately in a Json array
if(result[cat].length == objects.length) {
resolve(result);
}
} else {
console.log(""+cat+" is not avilable for this value "+data.dataValues['X']);
}
});
}
});
});
}
Here it won't wait for completion of first iteration. It just run asyncronously before completing first iteration function. I need to return the result as result:[{ 1, 2}, {3,4}]. but it runs seamlessly and returns empty or only one object before completing all. How to resolve it.
I used node-async-loop. But it uses next and i can't able to send my parameteres while using that package. Please help me

Async provides control flow methods allowing to do so.
Using async.each:
async.each(openFiles, function(file, callback) {
// Perform operation on file here.
console.log('Processing file ' + file);
if( file.length > 32 ) {
console.log('This file name is too long');
callback('File name too long');
} else {
// Do work to process file here
console.log('File processed');
callback();
}
}, function(err) {
// if any of the file processing produced an error, err would equal that error
if( err ) {
// One of the iterations produced an error.
// All processing will now stop.
console.log('A file failed to process');
} else {
console.log('All files have been processed successfully');
}
});

If you don't want to use a library, you can code it yourself. It would also be very instructive. I took your issue and coded a dummy async loop :
function listTopSongs(query) {
return new Promise(async(resolve, reject) => { //add async here in order to do asynchronous calls
const str = query.split(",") //str is const, and the other variable was not used anyway
for( let i = 0;i < str.length; i++) {
const planet = await sampleFn(str[i], 'sample', resolve, reject)
console.log(planet)
}
});
};
function sampleFn(a, b, c, d) {
return fetch(`https://swapi.co/api/planets/${a}/`)
.then(r => r.json())
.then(rjson => (a + " : " + rjson.name))
}
listTopSongs("1,2,3,4,5,6,7,8,9")
I used some dummy star wars API to fake a long promise but it should work with your sampleFn. Be careful, it is very, very slow if you have network call like the one in the example.
EDIT: I ran your code and I noticed there are a few mistakes: there is no resolve in your promise so it's not a thenable (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/resolve see thenable )
Here is a fully working code. The nice part : no library needed, no dependencies.
//for node.js, use node-fetch :
//const fetch = require("node-fetch")
function listTopSongs(query) {
return new Promise(async(resolve, reject) => { //add async here in order to do asynchronous calls
const str = query.split(",") //str is const, and the other variable was not used anyway
const planets = []
for (let i = 0; i < str.length; i++) {
const planet = await sampleFn(i + 1, str[i], resolve, reject)
planets[i] = planet
console.log(planet)
}
resolve(planets)
});
};
function sampleFn(a, b, c, d) {
return fetch(`https://swapi.co/api/planets/${a}/`)
.then(r => r.json())
.then(rjson => (a + b + " : " + rjson.name))
}
listTopSongs("a,b,c,d").then(planets => console.log(planets))

Since you are using promise, you can do something like this
exports.listTopSongs = function(query) {
return Promise.resolve(true).then(function(){
var str = query.split(",");
var promises = str.map(function(s){
return sampleFn(str[i], 'sample');
});
return Promise.all(promises);
}).then(function(results){
//whatever you want to do with the result
});
};
For this to work you have to change your sampleFn to not to depend on external resolve and reject functions. I don't see a reason using external resolve and reject. why just not use Promise.Resolve, Promise.Reject;

Pattern for dynamic Javascript promises

Inside a promise, I need to call and process an indeterminate number of asynch API responses after individually calling them either inside another promise, or after said promise, but before another so the order of execution is respected.
var promiseA = function() {
return new Promise(function(resolve, reject) {
// 1. Establish objects needed from one API endpoint
// 2. Call API endpoint for each object and parse
// 3. Only then continue to next promise
}
}
var finalPromise = function() {
return new Promise(function(resolve, reject) {
//
}
}
promiseA()
.then(finalPromise)
So inside promiseA, I find out how many objects I'll need to poll individually from an API. Each request is of course asynchronous. I need to make these calls and process the response before the final promise is called.
I am struggling to determine a pattern for this with promises, where I can dynamically create these promises and only allow the final promise to execute after the indeterminate and asynchronous have executed and processed. I've worked with other languages where this is possible, but I'm struggling to see it here with Promises.
Any help is appreciated.

I have changed the answer to incorporate the comments below. Since, you mentioned ES6 promises I shall stick to that. There are two basic types of callbacks that we might care about.
DOM load or other one time event callbacks (window.onload and so on)
Async method callback (AJAX call, setTimout and so on)
Since,
1.DOM load or other one time event
var p = new Promise(function(res, rej) {
window.onload = res();
};
2.Plain callback: these are callbacks that don't conform to a convention. e.g. setTimeout
var p = new Promise(function(res, rej){
setTimeout(function() {
//your business/view logic
success? res():rej(); //if successful resolve else reject
}, 2000);
});
In each of the above case the promise (var p) can be wrapped to be returned by a function.
var myAsyncMethod = function () {
var p = new ... // as mentioned in 1 or 2
return p;
}
Then the usage:
myAsyncMethod()
.then(function(){/* success-handler */})
.catch(function(/* failure-handler */));
Specific to your question you may have many such methods:
function baseAJAXCall (url) {
new Promise(functoin(rej, res) {
$.get(url, function(err, data){
if(err) {
rej();
}
else {
resolve(data);
}
});
}
};
function callAPIEndpoint(url) {
return baseAJAXCall(url);
}
function finalPromiseHandler () {
//your final business/view logic
}
//USAGE
callAPIEndpoint('/my-first-call')
.then(function(data){
var promiseArray = data.map(function(item){
return baseAJAXCall(item.url);
});
return Promise.all(promiseArray);
})
.then(finalPromiseHandler)
.catch(function(){
console.log('.error-message.');
});
Ref:
How do I convert an existing callback API to promises?.
http://www.datchley.name/es6-promises/
Links from comments below.
---OLD ANSWER: PLEASE OVERLOOK---
I am familiar with this library : https://github.com/kriskowal/q. And, you can do this using using the q.all and q.allSettled constructs. May be that is what you are looking for.
Normally, the pattern is to create a function that returns a promise.
function someAsyncFuncName1(url) {
var def = q.defer();
//async function
$.get(url, function(err, data){ //suppose
if(err){
def.reject();
}
else {
def.resolve(data); //pass the data to the .then() handler.
}
});
return def.promise;
}
function someAsyncFuncName2() {
var def = q.defer();
//async function
setTimeout(function(){ //suppose
//do something
if(good) {
def.resolve();
} else {
def.reject();
}
}, 1000); //arbitrary timeout of 1 second
return def.promise;
}
USAGE:
q.all([someAsyncFuncName1('/api-1'), someAsyncFuncName2()])
.then(function() {
//final handler
});
On a similar line of thought one can use q.allSettled() if you want to wait for all promises to return.
Hope this helps.
---EOF OLD ANSWER---

First of all, if async functions used in PromiseA don't return promises, you need to promisify them. You can do that with Promise constructor, but it's much better to use libraries, such as bluebird with their promisify methods.
Let's imagine, that we have two functions getUserIdsAsync and getUserAsync. The first on returns a list of user ids, getUserAsync returns an user data by userId. And you need to get a list of users by their ids. The code of PromiseA could look so:
var promiseA = function() {
return getUserIdsAsync()
.then(userIds => {
let ops = users.map(uid => getUserAsync(uid));
return Promise.all(ops);
});
}

The following snippet shows a solution without using any external library like bluebird. It follows the code snippet in your question (which seems to be more complicate than needed).
You have to collect all api promisses in an array. Then you can call Promise.all() to get a Promise for the end of all api promisses. Then you can do some final stuff, like parsing the result of each promise and continue afterwards.
function getRandomInt(min, max) {
return Math.floor(Math.random() * (max - min + 1)) + min;
}
var apiEndpoint = function (name) {
return new Promise( (resolve, reject) => {
setTimeout(() => resolve('API ' + name + ' job done'), 1000);
});
}
var promiseA = function() {
return new Promise( (resolve, reject) => {
const promisses = [];
for (var i=1; i < getRandomInt(3,6); i++) {
// 1. Establish objects needed from one API endpoint
promisses.push(apiEndpoint('This is number ' + i));
}
Promise.all(promisses).then( results => {
// do final stuff
for (const s of results) {
// 2. Call API endpoint for each object and parse
console.log(s);
}
// continue ...
// 3. Only then continue to next promise
resolve('now it is finished');
}).catch( err => reject(err) );
});
}
var finalPromise = function() {
return new Promise( (resolve, reject) => {
console.log('finalPromise');
resolve();
});
}
promiseA()
.then( () => finalPromise())
.catch(err => console.log(err) );
Please hold in mind that this solution is not easy to read. Using external libraries or reducing promisses can improve readability. Maybe you should take a look to the async/await pattern to get a much more better (readable) solution.
Here is a solution with async/await:
function getRandomInt(min, max) {
return Math.floor(Math.random() * (max - min + 1)) + min;
}
const apiEndpoint = function (name) {
return new Promise( (resolve, reject) => {
setTimeout(() => resolve('API ' + name + ' job done'), 1000);
});
}
async function promiseParallel () {
const promisses = [];
for (let i = 1; i < getRandomInt(3,6); i++) {
promisses.push(apiEndpoint('This is number ' + i));
}
for (const p of promisses) {
const x = await p;
console.log(x);
}
return ('everything is done');
}
promiseParallel().then( result => {
console.log(result);
}).catch( err => console.log(err) );
If you want call the promisses sequentially you can replace with:
async function promiseSequ () {
for (let i = 1; i < getRandomInt(3,6); i++) {
const x = await apiEndpoint('This is number ' + i);
console.log(x);
}
return ('everything is done');
}

Providing previous values of promises

My question involves promises and providing previous values of chain promises. The question is, is the item passed from first return promise to second promise "runItem --> testItem"?
Or do we have to pass the item through all promises?
Example:
db.items.find({_id: id}).then(function (res) {
runItem(item);
})
function runItem(item) {
removeFromQueue(item.id).then(function () {
testItem(item);
});
}
function testItem(item) {
...
}
function removeFromQueue(item) {
return db.queue.remove({_id: item._id});
}
EDIT:
Maybe this would be a better example:
Can we access original attribute item, or is it going to be overwritten when the next time function is called?
function start(id)
db.find({_id: id}).then(function (item) {
test(item).then(function (res) {
// can we access original attribute item, or is it going to be overwritten when the next time function is called
resolve({ res: res, item: item });
});
});
}
function test(item) {
return $test(item).then(function () {
resolve('success');
});
}

You can change the return value of a previous promise and simply return it. The next promise will be influenced. Check this example:
var promise = new Promise(function(res,rej){ res({data: 7}) }); // start with {data:7}
var promise1 = promise.then(function(res){ console.log('promise.then: ' + JSON.stringify(res)); return res.data; }); // log: {data:7}. return: 7
var promise2 = promise1.then(function(res) { console.log('promise1.then : ' + res); return res + 1;}); // log:7. return 8.
var promise3 = promise2.then(function(res) { console.log('promise2.then: ' + res); return res;}); // log:8. return 8.
var promise4 = promise3.then(function(res) { console.log('promise3.then: ' + res); return res;}); // log:8. return 8.

Try Promise.props function.
Something like
...
{
...
return Promise.props({res: res, item: item}).
.then(function(result){
return Promise.resolve({ res: res, item: item });
});
}

The example you have posted doesn't returns promise object .
Below is an example for the chaining things in promise, have a look.
functionA = function(text) {return new Promise(function(resolve) {
resolve(text);
console.log(text + " at function A");
});
}
functionB = function(text) {return new Promise(function(resolve) {
resolve(text);
console.log(text + " at function B");
});}
functionA("test").then(function (res)
{
console.log(res + " at then");
runItem(res);
})
function runItem(item) {
functionB(item).then(function (res) {
console.log(res + " at Function Run Item");
testItem(res);
});
}
function testItem(item) {
console.log(item + " at Function testItem");
}
in the above example you can see the runItem(item)'s item is passed to promiseobject of functionB, which wen resolved returns the same in then as res, which is further used by testItem(res).
if you want to do any operation on runItem(item)'s item you can modify it before the promise is resolved for FunctionB