We Keep Coding

Incorporating async actions, promise.then() and recursive setTimeout whilst avoiding "deferred antipattern"

I have been reading up on methods to implement a polling function and found a great article on https://davidwalsh.name/javascript-polling. Now using a setTimeout rather than setInterval to poll makes a log of sense, especially with an API that I have no control over and has shown to have varying response times.
So I tried to implement such a solution in my own code in order to challenge my understanding of callbacks, promises and the event loop. I have followed guidance outlined in the post to avoid any anti-patterns Is this a "Deferred Antipattern"? and to ensure promise resolution before a .then() promise resolve before inner promise resolved and this is where I am getting stuck. I have put some code together to simulate the scenario so I can highlight the issues.
My hypothetical scenario is this:
I have an API call to a server which responds with a userID. I then use that userID to make a request to another database server which returns a set of data that carries out some machine learning processing that can take several minutes.
Due to the latency, the task is put onto a task queue and once it is complete it updates a NoSql database entry with from isComplete: false to isComplete: true. This means that we then need to poll the database every n seconds until we get a response indicating isComplete: true and then we cease the polling. I understand there are a number of solutions to polling an api but I have yet to see one involving promises, conditional polling, and not following some of the anti-patterns mentioned in the previously linked post. If I have missed anything and this is a repeat I do apologize in advance.
So far the process is outlined by the code below:
let state = false;
const userIdApi = () => {
return new Promise((res, rej) => {
console.log("userIdApi");
const userId = "uid123";
setTimeout(()=> res(userId), 2000)
})
}
const startProcessingTaskApi = userIdApi().then(result => {
return new Promise((res, rej) => {
console.log("startProcessingTaskApi");
const msg = "Task submitted";
setTimeout(()=> res(msg), 2000)
})
})
const pollDatabase = (userId) => {
return new Promise((res, rej) => {
console.log("Polling databse with " + userId)
setTimeout(()=> res(true), 2000)
})
}
Promise.all([userIdApi(), startProcessingTaskApi])
.then(([resultsuserIdApi, resultsStartProcessingTaskApi]) => {
const id = setTimeout(function poll(resultsuserIdApi){
console.log(resultsuserIdApi)
return pollDatabase(resultsuserIdApi)
.then(res=> {
state = res
if (state === true){
clearTimeout(id);
return;
}
setTimeout(poll, 2000, resultsuserIdApi);
})
},2000)
})
I have a question that relates to this code as it is failing to carry out the polling as I need:
I saw in the accepted answer of the post How do I access previous promise results in a .then() chain? that one should "Break the chain" to avoid huge chains of .then() statements. I followed the guidance and it seemed to do the trick (before adding the polling), however, when I console logged out every line it seems that userIdApi is executed twice; once where it is used in the startProcessingTaskApi definition and then in the Promise.all line.
Is this a known occurrence? It makes sense why it happens I am just wondering why this is fine to send two requests to execute the same promise, or if there is a way to perhaps prevent the first request from happening and restrict the function execution to the Promise.all statement?
I am fairly new to Javascript having come from Python so any pointers on where I may be missing some knowledge to be able to get this seemingly simple task working would be greatly appreciated.

I think you're almost there, it seems you're just struggling with the asynchronous nature of javascript. Using promises is definitely the way to go here and understanding how to chain them together is key to implementing your use case.
I would start by implementing a single method that wraps setTimeout to simplify things down.
function delay(millis) {
return new Promise((resolve) => setTimeout(resolve, millis));
}
Then you can re-implement the "API" methods using the delay function.
const userIdApi = () => {
return delay(2000).then(() => "uid123");
};
// Make userId an argument to this method (like pollDatabase) so we don't need to get it twice.
const startProcessingTaskApi = (userId) => {
return delay(2000).then(() => "Task submitted");
};
const pollDatabase = (userId) => {
return delay(2000).then(() => true);
};
You can continue polling the database by simply chaining another promise in the chain when your condition is not met.
function pollUntilComplete(userId) {
return pollDatabase(userId).then((result) => {
if (!result) {
// Result is not ready yet, chain another database polling promise.
return pollUntilComplete(userId);
}
});
}
Then you can put everything together to implement your use case.
userIdApi().then((userId) => {
// Add task processing to the promise chain.
return startProcessingTaskApi(userId).then(() => {
// Add database polling to the promise chain.
return pollUntilComplete(userId);
});
}).then(() => {
// Everything is done at this point.
console.log('done');
}).catch((err) => {
// An error occurred at some point in the promise chain.
console.error(err);
});

This becomes a lot easier if you're able to actually use the async and await keywords.
Using the same delay function as in Jake's answer:
async function doItAll(userID) {
await startTaskProcessingApi(userID);
while (true) {
if (await pollDatabase(userID)) break;
}
}

Unexpected Promise in React Native

I am new to React Native and coding in general. I paid for some code on upwork and am having a hard time integrating it in my program.
async pullBatch(since){
let param = {
userScreenName: '?screen_name=google',
count: "&count=5",
retweets: "&include_rts=false",
replies: "&exclude_replies=false",
trim: "&trim_user=true",
since: "&max_id=" + since
};
let twitterRest = new TwitterRest(); //create a new instance of TwitterRest Class
let batch = await twitterRest.pullTweets(param); //pull the Google TimeLine
return batch;
}
pullTimeline(){
let timeLine = []
for(i = 0; i <= 2; i++){
let currentBatch = this.pullBatch("1098740934588751900")
console.log(currentBatch);
timeLine = timeLine.concat(currentBatch);
}
console.log(timeLine);
// timeLine = currentBatch
return(timeLine)
}
I believe that when running pullTimeLine() the program is returning an array of three promises. (I have also run the code with "await" before pullBatch(), but it is erroring out telling me await is a reserved word) This means I am making two mistakes:
I am not correctly understanding promises in JS or how they are resolved.
I am incorrectly concatenating the arrays.
I am constantly trying to learn, so while I greatly appreciate suggestions for code fixes, I also really would appreciate if you'd teach me about where my lapses in understanding lies.
Thank you

Let's break it down. You seem to understand that pullBatch is an async function, and so calling it will return a promise create by the twitterRest interaction.
The problem is that your call to pullBatch inside your for loop will not resolve these promise (which seems to be what you want to do). The easiest way is to use await for currentBatch, but as you tried, you got the reserved error. Basically you just need to also make pullTimeline async like this:
async pullTimeline(){
...
Just realise that once you do this, pullTimeline is now an async function that will also return a promise. So to use this function you need to either use .then(), for example:
pullTimeline().then(timeLine => {
// do something with your timeline here
})
Or if you are using it within another async function, you can use await.
const timeLine = await pullTimeline() // must be inside async function
Basically at some point in your calling chain, you will have to resolve a promise using .then(), or disregard the top level promise by making a top level async function. For example:
async useTimeline() {
const timeLine = await pullTimeline()
// do something with your timeline
}
// call the function above, and just disregard its promise
useTimeLine()
Just don't forget to handle errors somewhere. Either use a .catch() on your top level promise, or use try / catch around any of your await calls.

why my javascript its use to long time to run?

I'm working with Cloud Functions for Firebase, and I get a timeout with some of my functions. I'm pretty new with JavaScript. It looks like I need to put a for inside a promise, and I get some problems. The code actually get off from for too early, and I think he make this in a long time. Do you have some way to improve this code and make the code faster?
exports.firebaseFunctions = functions.database.ref("mess/{pushId}").onUpdate(event => {
//first i get event val and a object inside a firebase
const original = event.data.val();
const users = original.uids; // THIS ITS ALL USERS UIDS!!
// so fist i get all users uids and put inside a array
let usersUids = [];
for (let key in users) {
usersUids.push(users[key]);
}
// so now i gonna make a promise for use all this uids and get token's device
//and save them inside a another child in firebase!!
return new Promise((resolve) => {
let userTokens = [];
usersUids.forEach(element => {
admin.database().ref('users/' + element).child('token').once('value', snapShot => {
if (snapShot.val()) { // if token exist put him inside a array
userTokens.push(snapShot.val());
}
})
})
resolve({
userTokens
})
}) // now i make then here, from get userTokens and save in another child inside a firebase database
.then((res) => {
return admin.database().ref("USERS/TOKENS").push({
userTokens: res,
})
})
})

You are making network requests with firebase, so maybe that's why it's slow. You are making one request per user, so if you have 100 ids there, it might as well take a while.
But there's another problem that I notice, that is: you are just resolving to an empty list. To wait for several promises, create an array of promises, and then use Promise.all to create a promise that waits for all of them in parallel.
When you call resolve, you have already done the forEach, and you have started every promise, but they have not been added to the list yet. To make it better, chance it to a map and collect all the returned promises, and then return Promise.all.

Promises nodejs readability improvements

I have writting a few functions which returns a promise, to get data from google analytics api. I think what I've written is called a callback hell..
Can someone help me optimise this code (or give tips/best practices), so it's better readable.
var express = require('express');
var router = express.Router();
var googleAuth = require('google-oauth-jwt');
var google = require('googleapis');
var app = express();
module.exports.getGoogleData = function (jwtClient,analytics,VIEW_ID){
return new Promise(function(resolve, reject){
return getOrdersToday(jwtClient,analytics,VIEW_ID).then(function (orders) {
return getOnlineUsersToday(jwtClient,analytics,VIEW_ID).then(function (users) {
return getSearchedToday(jwtClient, analytics, VIEW_ID).then(function (searched){
return getPageviewsTodayAndUsersToday(jwtClient, analytics, VIEW_ID).then(function (pageviews){
var returndata =[
{
"Orders":orders,
"Onlineusers":users,
"searched":searched,
"pageviews":pageviews[0].pageviews,
"usersToday":pageviews[0].users
}
]
resolve(returndata);
});
});
});
});
});
}
Example getfunction
function getOrdersToday(jwtClient,analytics,view_id){
return new Promise(function(resolve,reject){
analytics.data.ga.get({
'auth':jwtClient,
'ids': view_id,
'metrics': 'ga:totalEvents',
'start-date': 'today',
'end-date': 'today',
filters: 'ga:eventAction==Bestelling geplaatst',
'max-results': '1'
}, function(err, response) {
// handle the errors (if any)
if(err){
console.log(err)
reject(err)
} else
console.log('Response:',response)
resolve(response.totalsForAllResults["ga:totalEvents"]);
});
});
}

There's no need for the new Promise in there at all, and in fact by using it, you leave it open to never settling if an error occurs in one of your calls. Remember that then returns a new promise. So you can just chain all those together if you want them run sequentially:
module.exports.getGoogleData = function (jwtClient,analytics,VIEW_ID){
var result = {};
return getOrdersToday(jwtClient,analytics,VIEW_ID)
.then(function (orders) {
result.Orders = orders;
return getOnlineUsersToday(jwtClient,analytics,VIEW_ID);
})
.then(function (users) {
result.Onlineusers = users;
return getSearchedToday(jwtClient, analytics, VIEW_ID);
}).then(function (searched){
result.searched = searched;
return getPageviewsTodayAndUsersToday(jwtClient, analytics, VIEW_ID);
}).then(function (pageviews){
result.pageviews = pageviews[0].pageviews;
result.usersToday = pageviews[0].users;
return [result]; // (Seems a bit odd that it's wrapped in an array, but
// that's what the original code did...)
});
}
But, those operations look independent of one another. If that's really true, run them in parallel with Promise.all:
module.exports.getGoogleData = function (jwtClient,analytics,VIEW_ID){
return Promise.all([
getOrdersToday(jwtClient,analytics,VIEW_ID),
getOnlineUsersToday(jwtClient,analytics,VIEW_ID),
getSearchedToday(jwtClient, analytics, VIEW_ID),
getPageviewsTodayAndUsersToday(jwtClient, analytics, VIEW_ID)
]).then(results => {
return [{
Orders: results[0],
Onlineusers: results[1],
searched: results[2],
pageviews: results[3][0].pageviews,
usersToday: results[3][0].users
}];
});
}

The only way to fix that ugly code is killing Promise hell, for doing that you must use:
ES2017 async/await syntax
So, take a look at this new code
var express = require('express');
var router = express.Router();
var googleAuth = require('google-oauth-jwt');
var google = require('googleapis');
var app = express();
module.exports.getGoogleData = foo
async function foo (jwtClient,analytics,VIEW_ID){
var orders = await getOrdersToday(jwtClient,analytics,VIEW_ID)
var users = await getOnlineUsersToday(jwtClient,analytics,VIEW_ID)
var searched = await getSearchedToday(jwtClient, analytics, VIEW_ID)
var pageviews = await getPageviewsTodayAndUsersToday(jwtClient, analytics, VIEW_ID)
return [{
"Orders":orders,
"Onlineusers":users,
"searched":searched,
"pageviews":pageviews[0].pageviews,
"usersToday":pageviews[0].users
}]
}

Since you have the promises created already. You can use Promise.all() and pass it the array of promises in the order you want them to be executed.
Refer this for more details - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
Alternatively, you can use async/await as well - https://javascript.info/async-await

Below is the correct and readable way to chain your promises. Correct chaining reduce nesting by expanding it vertically. Also, you don't need to instantiate a new Promise because all your get methods already returns promises.
function (jwtClient,analytics,VIEW_ID) {
var returndata =[{}];
return getOrdersToday(jwtClient,analytics,VIEW_ID).then(function (orders) {
returndata[0].Orders = orders;
return getOnlineUsersToday(jwtClient,analytics,VIEW_ID);
}).then(function(users) {
returndata[0].Onlineusers= users;
return getSearchedToday(jwtClient, analytics, VIEW_ID);
}).then(function(searched) {
returndata[0].searched = searched;
return getPageviewsTodayAndUsersToday(jwtClient, analytics, VIEW_ID);
}).then(function(pageviews) {
returndata[0].pageviews = pageviews[0].pageviews;
returndata[0].usersToday = pageviews[0].users;
return returndata;
});
}

As others have noted, these calls do not depend on each other, which means you don't need to sequence them and can instead execute them all simultaneously. Here is what that might look like:
module.exports.getGoogleData = function(jwtClient, analytics, VIEW_ID) {
return Promise.all([
getOrdersToday(jwtClient, analytics, VIEW_ID),
getOnlineUsersToday(jwtClient, analytics, VIEW_ID),
getSearchedToday(jwtClient, analytics, VIEW_ID),
getPageviewsTodayAndUsersToday(jwtClient, analytics, VIEW_ID)
])
.then(function(results) {
return {
"Orders": results[0],
"Onlineusers": results[1],
"searched": results[2],
"pageviews": results[3][0].pageviews,
"usersToday": results[3][0].users
};
});
}
But I think your question is a really good one, and I hope your interest in good patterns doesn't end just because this specific case calls for a different solution. As I have noted elsewhere, async patterns is a complicated problem domain.
So, let's pretend you need to make 4 calls, each depending on the results of the previous one, so they must be executed in sequence. We have access to this fictional API:
function fetchOne(null) // returns opaque ONE
function fetchTwo(ONE) // must feed it ONE, returns opaque TWO
function fetchThree(TWO) // must feed it TWO, returns opaque THREE
function fetchFour(THREE) // must feed it THREE, returns opaque FOUR
And furthermore, let's pretend that at the end of this process, you wish to have access to all four of these return values so you can craft a response with all of them. After all, what good is a pattern if it can't accommodate the kinds of complicated workflows we see in the real world?
Thus, our goal is to flesh out the implementation of this function:
function doTheWork() {
// DO STUFF somehow, finally...
return {
one: ONE,
two: TWO,
three: THREE,
four: FOUR,
checksum: ONE + TWO + THREE + FOUR
};
}
A naively straightforward attempt might look like this:
function doTheWork() {
return fetchOne()
.then((ONE) => {
return fetchTwo(ONE)
})
.then((TWO) => {
return fetchThree(TWO)
})
.then((THREE) => {
return fetchFour(THREE)
})
.then((FOUR) => {
// all work is done, let's package up the results
return {
one: ONE // uh-oh...
};
})
}
This will execute, but the problem is that we no longer have access to the earlier values in the final handler.
There are really only two ways around this: (1) declare variables with a scope that will be shared by all the callbacks, or (2) pass all data between handlers in some kind of structure. I think solution 1 is really inelegant, and I recommend against it, but let's see what that might look like:
function doTheWork() {
var A, B, C
return fetchOne()
.then((ONE) => {
A = ONE // store ONE in A
return fetchTwo(ONE)
})
// ...
.then((FOUR) => {
// all work is done, let's package up the results
// at this point, A & B & C contain the results of previous calls
// and are within scope here
return {
one: A,
two: B,
three: C,
four: FOUR,
checksum: A + B + C + FOUR
};
})
}
This will work, but I think it's bad practice and I dislike it. It's bad practice because these variables are now "public" in a limited kind of way, exposing them to everything else within doTheWork. They can be clobbered by statements anywhere in this function. And unless they are scalars, they will be passed by reference, which means any mutation bugs associated with them can manifest themselves in sometimes bizarre ways.
Also, each of these "temporary" variables now competes for good names in a shared namespace (the scope of doTheWork). Ideally, you should create as few variables as possible, and each should live only as long as necessary. This might save memory, but it definitely saves your "naming pool." Naming things is mentally exhausting. I am not kidding. And every name must be good -- never name variables in a slapdash way. The names of things are often the only clues within the code about what is happening. If some of those names are throwaway garbage, you are making your life and the lives of future maintainers harder.
So, let's look at solution 2. At this point, I want to point out that this approach works best when you can use ES6 destructuring syntax. You can do this without it, but it's a bit clunkier.
We're going to construct an array that will slowly accumulate all the data fetched by each of these async calls. At the end, the array will look like this:
[ ONE , TWO , THREE , FOUR ]
By chaining promises efficiently, and by passing this array from one handler to the next, we can both avoid the Pyramid of Doom and share async results among all of these methods easily. See below:
function doTheWork() {
return fetchOne()
.then((ONE) => {
return fetchTwo(ONE)
.then((TWO) => [ ONE , TWO ])
})
.then(([ ONE , TWO ]) => {
return fetchThree(TWO)
.then((THREE) => [ ONE , TWO , THREE ])
})
.then(([ ONE , TWO , THREE ]) => {
return fetchFour(THREE)
.then((FOUR) => [ ONE , TWO , THREE , FOUR ])
})
.then(([ ONE , TWO , THREE , FOUR ]) => {
return {
one: ONE,
two: TWO,
three: THREE,
four: FOUR,
checksum: ONE + TWO + THREE + FOUR
}
})
}
That's the whole thing, but let's step through just the first part:
return fetchOne()
.then((ONE) => {
return fetchTwo(ONE)
.then((TWO) => [ ONE , TWO ])
})
As usual, fetchOne returns ONE -- it's the third-party API we have no control over. And as you might expect, we use ONE to make the second call, making sure to return its promise. But that last line is the real magic:
.then((TWO) => [ ONE , TWO ])
The second API call still returns just TWO, but rather than us simply returning TWO alone, we instead return an array that contains both ONE and TWO. That value -- the array -- becomes the argument to the next .then handler.
This works because nested promises are automatically unwrapped. Here's a simple example showing that at work:
function getDogName() {
return fetchCatName()
.then((catName) => 'Fido')
}
getDogName()
.then((dogName) => {
console.log(dogName);
})
// logs 'Fido'
This illustrates that you can attach .then handlers to nested promises, and the outer promises will return the results of those handlers. This is a really common pattern when using fetch to get JSON:
function makeApiCall() {
fetch(api_url) // resolves with a Response object
.then((response) => response.json()) // extracts just the JSON and returns that instead!
}
Going back to our code, since we want to avoid the Pyramid, we leave this handler along (rather than nesting the next call inside, as you originally did). The next line looks like this:
.then(([ ONE , TWO ]) => {
This is the ES6 destructuring at work. By destructuring the array argument, we can name the elements as they enter the function. And if we're doing that, we might as well give them the names we like best, the ones the came into the world as: ONE and TWO.
We then repeat the pattern, using TWO to invoke fetchThree, making sure to return its promise, but not before tacking on a tiny .then handler that bundles the newly-available THREE into the package that we pass forward.
I hope this helps. It's a pattern I worked out to deal with some complicated branching workflows in AWS, making calls against S3 and Dynamo and other platforms with a lot of parallel conditionals, mixing blocking and non-blocking behavior.
Async patterns are a special problem domain. Even with the right tech, it's can be hard to find clear ways to express behavior, especially when the underlying workflow is convoluted. This is often the case when exploring the data graphs that are so common these days.
Happy coding.

How to wait for typescript / javascript promise before returning value

This is my first post so I hope I do it right. This is my first exposure to promises and 3 hours in I still can't get it. Our project does not have support for Async Wait, unfortunately. I'm using pouchdb over sqlite.
this.recipes is used to display a page of recipes. It returns as empty, however the first time, and then always lags one recipe behind after each recipe is added. I am pretty sure this is because I return this.recipeBook before all the promises are completed. I realize in the code I posted the final return is dangling outside of any promise, but when I put it in a promise I get a "type void is not assignable to type any[]" error in my RecipesPage.ts.
I appreciate any support you guys can offer and am excited to learn the right way to hand this, and thank you in advance.
This is my call in RecipesPage.ts:
this.recipes = this.database.getAllRecipes();
This is the method in database.service.ts:
public getAllRecipes() {
this.recipeBook = new Array<Recipe>();
this.recipeDatabase.allDocs
({
include_docs: true,
}).then(docs => {
this.objectArray = docs.rows.map(row => {
return row.doc;
})
}).then(function (){
for (let object of this.objectArray) {
this.myRecipe = this.convertObjectToRecipe(object);
this.recipeBook.push(this.myRecipe);
}
})
return this.recipeBook;
}

You don't. There's no real way to 'unpack' a promise like this. The closest you can realistically get is polling.
Instead, return the promise and call .then on the promise where you're waiting for it. This is ultimately what async and await are doing under the hood. Make decisions in the UI to choose how to render your content before its loaded (and as its being loaded).