We Keep Coding

JS Promise function call that returns object [duplicate]

This question already has answers here:
await setTimeout is not synchronously waiting
(2 answers)
Closed last month.
I have a async fetch function that waits 2 seconds and returns a object:
async function fetch() {
var object;
await setTimeout(() => { object = { name: 'User', data: 'API Data' } }, 2000);
return object;
}
I want to display the object when the initialization is completely done (after 2 seconds)
fetch().then((val) => {
console.log("DONE!");
console.log(val.name);
}).catch((err) => {
console.log("ERROR!");
console.log(err);
});
The code prints both DONE and ERROR Cannot read properties of undefined (reading 'name')
I have tried with Promise, no luck
let promise = new Promise((resolve, reject) => {
let request = fetch();
if (request !== undefined)
resolve(request);
else
reject(request);
}).then((val) => {
console.log(val);
});
How can I properly check that fetch() has returned a value before printing without changing the inside of the function. I can delete the async and await in it but I am unable to edit it (I.E. adding a Promise inside)

Based on requirement
I can delete the async and await in it (fetch function) but I am unable to edit it (I.E. adding a Promise inside)
The only way I see is to override window.setTimeout function to make it to return a promise. That way you will be able to await it and there will be no need to modify your fetch function.
const oldTimeout = window.setTimeout;
window.setTimeout = (fn, ms) => {
return new Promise((resolve, reject) => {
oldTimeout(() => {
fn();
resolve();
}, ms);
});
};
async function fetch() {
var object;
await setTimeout(() => {
object = { name: "User", data: "API Data" };
}, 2000);
return object;
}
fetch()
.then((val) => {
console.log("DONE!");
console.log(val.name);
})
.catch((err) => {
console.log("ERROR!");
console.log(err);
});
NOTE: For anyone without this requirement - please, use other answers to this question or check await setTimeout is not synchronously waiting for additional details/explanations. This kind of overridings are very confusing due to everyone expect common and well-known functions to behavior in a way described in the docs.

You cannot await the setTimeout function, this is because your function returns undefined. You have used the promise in the wrong way. Below code will fix your issue.
function fetch() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve({ name: "User", data: "API Data" });
}, 2000);
});
}
fetch()
.then((val) => {
console.log("DONE!");
console.log(val.name);
})
.catch((err) => {
console.log("ERROR!");
console.log(err);
});
And remember that there is no need to change the setTimeout function.

The problem is that setTimeout does not actually return a promise, which means you cannot use await with setTimeout, that's why the var object; is returned instantly as undefined.
To solve this issue, you simply need to wrap setTimeout around a promise.
Like so:
function setTImeoutAwait(time) {
return new Promise((resolve) => {
setTimeout(resolve, time);
});
}
You can then use it like this:
async function fetch() {
var object;
await setTImeoutAwait(1000).then(() => {
object = { name: "test" };
});
return object;
}

Run secondary action after promise regardless of outcome?

I found this previous thread (How to perform same action regardless of promise fulfilment?), but it's 5 years old and references winjs is a kludge.
What I would like to do is load a list of data elements. I've got local copies of the list and local copies of the elements -- but they may have changed on the server side.
That process should work like this: load the LIST from the database into the local storage (Comparing against the local) --> THEN load the (multiple) DATA ELEMENTS from the database that are listed in the LIST.
So if the "loadList" async function succeeds... I want to run the "loadElements" async function. If the loadList function rejects... I STILL want to run the "loadElements" function (Which fires off multiple fetch requests - one for each element).
"Use 'finally'" I hear you say... but I want to pass the results of the "loadList" resolve/reject and "loadElements" resolve/reject functions to the calling function. 'finally' doesn't receive or pass properties as far as I know.
The reason I want to pass the results to the calling function is to see if the rejection reasons are acceptable reasons and I can trust the local copy as the authoritative copy or not (for example, if the DB doesn't contain the LIST, I can trust that the local list is the authoritative version)... so I need a way to analyze the 'failures' within the calling function.
Here is what I have:
export function syncLinkTablesAndElementsWithDB(username) {
return (dispatch, getState) => {
return new Promise((resolve, reject) => {
dispatch(loadLinkTableAndElementsFromDB(STATIONS_LINK_TABLE_TYPE, username))
.then((msg) => {
console.log("loadLinkTableAndElementsFromDB RESOLVED: ", msg);
resolve(msg)
})
.then(() => {
dispatch(pushLinkTableToDB(STATIONS_LINK_TABLE_TYPE, username))
dispatch(pushAllUserStationsToDB(username))
})
.catch((allPromReasons) => {
console.log("loadLinkTableAndElementsFromDB REJECTED: ", allPromReasons);
allReasonsAcceptable = true;
allPromReasons.forEach(reason => {
if (!isAcceptableLoadFailureReasonToOverwrite(reason)) {
allReasonsAcceptable = false;
}
});
if (allReasonsAcceptable) {
//TODO: DO push of local to DB
// eventually return results of the push to DB...
} else {
reject(allPromReasons)
}
})
});
}
}
export function loadLinkTableAndElementsFromDB(tableType, username) {
return (dispatch, getState) => {
return new Promise((resolve, reject) => {
dispatch(loadLinkTableFromDB(tableType, username))
.then(successMsg => {
resolve(Promise.all([successMsg, dispatch(loadAllUsersStationsFromDB(username)).catch(err=>err)]))
})
.catch(err => {
reject(Promise.all([err, dispatch(loadAllUsersStationsFromDB(username)).catch(err=>err)]))
})
});
}
}
export function loadAllUsersStationsFromDB(username) {
return (dispatch, getState) => {
return new Promise((resolve, reject) => {
let linkTable = getStationsLinkTable(username); // get the local link table
if (linkTable && Array.isArray(linkTable.stations)) { // if there is a local station list
let loadPromises = linkTable.stations.map(stationID => dispatch(loadStationFromDB(stationID)).catch((err) => err));
Promise.all(loadPromises)
.then((allReasons) => {
let allSuccesses = true;
allReasons.forEach(reason => {
if (!reason.startsWith(SUCCESS_RESPONSE)) {
allSuccesses = false;
}
});
if (allSuccesses) {
resolve(SUCCESS_RESPONSE + ": " + username);
} else {
reject(allReasons);
}
})
} else {
return reject(NO_LINK_TABLE_AVAILABLE + ": " + username);
}
});
};
}
loadStationFromDB and loadLinkTableFromDB do what you'd expect... try to load those things from from the DB. I can include their code if you think it's worthwhile.
----------- EDIT -----------
To clarify what I'm trying to accomplish:
I'm trying to sync local storage with a database. I want to do this by pulling the data from the database, compare the time/datestamps. This will make the local storage version the authoritative copy of all the data. After the loads from the DB, I'd like to then push the local storage version up to the DB.
I need to care for the fact that the database will often simply not have the data at all, and thus might 'reject' on a pull... even though, in the instance of a sync, that rejection is acceptable and should not stop the sync process.
Per suggestions below, I've modified my code:
export function loadLinkTableAndElementsFromDB(tableType, username) {
console.log("loadLinkTableAndElementsFromDB(", tableType, username, ")");
return (dispatch, getState) => {
return new Promise((resolve, reject) => {
dispatch(loadLinkTableFromDB(tableType, username))
.then(successMsg => {
console.log("loadLinkTableFromDB RESOLVED: ", successMsg)
resolve(Promise.all([successMsg, dispatch(loadAllUsersStationsFromDB(username)).catch(err => err)]))
})
.catch(err => {
console.log("loadLinkTableFromDB REJECTED: ", err)
reject(Promise.all([err, dispatch(loadAllUsersStationsFromDB(username)).catch(err => err)]))
})
});
}
}
export function syncLinkTablesAndElementsWithDB(username) {
console.log("syncLinkTablesAndElementsWithDB(", username, ")");
return (dispatch, getState) => {
dispatch(loadLinkTableFromDB(STATIONS_LINK_TABLE_TYPE, username))
.then((successLoadLinkTableMsg) => {
console.log('Successfully loaded link table: ', successLoadLinkTableMsg)
return dispatch(pushLinkTableToDB(STATIONS_LINK_TABLE_TYPE, username))
})
.catch((rejectLoadLinkTableReason) => {
console.log("Failed to load link table from DB: " + rejectLoadLinkTableReason);
if (allReasonsAcceptableForOverwrite(rejectLoadLinkTableReason)) { // some rejection reasons are accectable... so if failed reason is okay....
console.log("Failure to load link table reasons were acceptable... pushing local link table anyway");
return dispatch(pushLinkTableToDB(STATIONS_LINK_TABLE_TYPE, username))
} else {
console.log("Throwing: ", rejectLoadLinkTableReason);
throw rejectLoadLinkTableReason;
}
})
.then((successPushLinkTaleMsg) => {
console.log("Successfully pushed link table: " + successPushLinkTaleMsg);
return dispatch(loadAllUsersStationsFromDB(username)); // I want this to occur regardless of if the link table stuff succeeds or fails... but it must occur AFTER the loadLinkTableFromDB at least tries...
})
.catch((rejectPushLinkTableReason) => {
console.log("Failed to push link table: " + rejectPushLinkTableReason);
return dispatch(loadAllUsersStationsFromDB(username)); // I want this to occur regardless of if the link table stuff succeeds or fails... but it must occur AFTER the loadLinkTableFromDB at least tries...
})
.then((successLoadAllUserStationsMsg) => {
console.log("Successfully loaded all user stations: " + successLoadAllUserStationsMsg);
return dispatch(pushAllUserStationsToDB(username))
})
.catch((rejectLoadAllUserStationsReason) => {
console.log("Failed to push all users stations: " + rejectLoadAllUserStationsReason);
if (allReasonsAcceptableForOverwrite(rejectLoadAllUserStationsReason)) { // some rejection reasons are accectable... so if failed reason is okay....
console.log("Load users stations reasons are acceptable...");
return dispatch(pushAllUserStationsToDB(username))
} else {
console.log("throwing: ", rejectLoadAllUserStationsReason);
throw rejectLoadAllUserStationsReason;
}
})
.then((successPushAllUserStationsMgs) => {
console.log("Successfully pushed all users stations: " + successPushAllUserStationsMgs);
return Promise.resolve();
})
.catch((rejectPushAllUserStationsReason) => {
console.log("Failed to push all users stations: " + rejectPushAllUserStationsReason);
throw rejectPushAllUserStationsReason;
})
};
}
export function syncAllWithDB(username) {
return (dispatch, getState) => {
// other stuff will occur here...
dispatch(syncLinkTablesAndElementsWithDB(username)) // *** Error here ***
.then((successMsg) => {
console.log("Successful sync for : " + successMsg);
})
.catch(allReasons => {
console.warn("Link tables and elements sync error: ", allReasons);
})
// });
}
}
Unfortunately, I'm now getting getting 'TypeError: dispatch(...) is undefined' on the dispatch in the syncAllWithDB function. This function hasn't changed...

I don't entirely follow what you're trying to accomplish (more on that below), but the first thing to do here is to clean up the flow and not wrap an extra new Promise() around existing promises. There is never a reason to do this:
function someFunc() {
return new Promise((resolve, reject) => {
callSomething.then(result => {
...
doSomethingElse(result).then(result2 => {
...
resolve(result2);
}).catch(err => {
...
reject(err);
});
}).catch(err => {
...
reject(err);
});
});
}
That is a well-known promise anti-pattern. You don't need the extra manually created promise wrapped around your function that already makes a promise. Instead, you can just return the promise you already have. This is called "promise chaining". From within the chain you can reject or resolve the chain from anywhere.
function someFunc() {
return callSomething.then(result => {
...
// return promise here, chaining this new async operation
// to the previous promise
return doSomethingElse(result).then(result2 => {
...
return result2;
}).catch(err => {
...
// after doing some processing on the error, rethrow
// to keep the promise chain rejected
throw err;
});
}).catch(err => {
...
reject err;
});
}
Or, you can even flatten the promise chain like this:
function someFunc() {
return callSomething.then(result => {
...
return doSomethingElse(result);
}).then(result2 => {
...
return result2;
}).catch(err => {
...
throw err;
});
}
As an example of that, you can simplify syncLinkTablesAndElementsWithDB() like this:
export function syncLinkTablesAndElementsWithDB(username) {
return (dispatch, getState) => {
return dispatch(loadLinkTableAndElementsFromDB(STATIONS_LINK_TABLE_TYPE, username)).then((msg) => {
console.log("loadLinkTableAndElementsFromDB RESOLVED: ", msg);
dispatch(pushLinkTableToDB(STATIONS_LINK_TABLE_TYPE, username))
dispatch(pushAllUserStationsToDB(username))
// have msg be the resolved value of the promise chain
return(msg);
}).catch((allPromReasons) => {
console.log("loadLinkTableAndElementsFromDB REJECTED: ", allPromReasons);
let allReasonsAcceptable = allPromReasons.every(reason => {
return isAcceptableLoadFailureReasonToOverwrite(reason);
});
if (allReasonsAcceptable) {
//TODO: DO push of local to DB
// eventually return results of the push to DB...
} else {
// have promise stay rejected
throw allPromReasons;
}
});
}
}
As for the rest of your question, you're asking this:
So if the "loadList" async function succeeds... I want to run the "loadElements" async function. If the loadList function rejects... I STILL want to run the "loadElements" function (Which fires off multiple fetch requests - one for each element).
But, there are not functions in your code called loadList() and loadElements() so you lost me there so I'm not sure how to make a specific code suggestion.
Inside a .then() handler in a promise chain, you can do three things:
Return a value. That value becomes the resolved value of the promise chain.
Return a promise. That promise is attached to the promise chain and the whole promise chain (the top-most promise that a caller would be watching) will eventually resolve/reject when this promise you are returning resolves/rejects (or anything that is also chained onto it resolves/rejects).
Throw an exception. All .then() handlers are automatically watched for exceptions and if any exception is throw, then the promise chain is automatically rejected with the exception value set as the reject reason.
So, that gives you the ultimate flexibility to finish the promise chain with a value or an error or to link it to another promise (more asynchronous operations).

Promise.all rollback successful promises' actions on failure

I am executing multiple promises with the following snippet:
await Promise.all([promise1, promise2, promise3]);
What I would like to achieve is to rollback the effects of the successful promises on the case of a failure from Promise.all().
In more specific terms, this means that the above will do some file encryptions, but if one fails, I would like to delete the other two (or one) files that were encrypted successfully so as to have consistent and clean file groups.
From what I've read this means that I would need two steps:
1. Catching the errors for each promise so that Promise.all() won't throw an error.
2. The puzzling part: Having another Promise.all() sort of:
await Promise.all([rollbackPromise1, rollbackPromise2, rollbackPromise3]);
This one seems to be the tricky part: Should I execute all the rollbacks independent of the promise that failed? This means that I should do another catch for every error such that the Promise.all() waits for every rollback to finish.
Is this the best way to do this, I find it pretty inefficient and ugly in terms of code.

You could create your own function implementing the asynchronous call of the functions and performing a rollback if required.
// Function that'll perform a promise.all and rollback if required
async function allWithRollback(promises) {
// using the map we are going to wrap the promise inside of a new one
return Promise.all(promises.map(([
func,
rollbackFunc,
], xi) => ((async() => {
try {
await func;
console.log('One Function succeed', xi);
} catch (err) {
console.log('One Function failed, require rollback', xi);
await rollbackFunc();
}
})())));
}
// Call the custom Promise.all
allWithRollback([
[
// First param is the promise
okPromise(),
// Second param is the rollback function to execute
() => {},
],
[okPromise(), () => {}],
[errPromise(), rollback1],
[errPromise(), rollback2],
[okPromise(), () => {}],
]);
// ---------
async function okPromise() {
return true;
}
async function errPromise() {
throw new Error('no one read this');
}
async function rollback1() {
console.log('Performed the rollback1');
}
async function rollback2() {
console.log('Performed the rollback2');
}

You can create a naive solution as follows:
const errorHandlers = []
function enc1 () {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve('str')
}, 1000)
errorHandlers.push(() => {
console.log('handler 1')
})
})
}
function enc2 () {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve('str')
}, 2000)
errorHandlers.push(() => {
console.log('handler 2')
})
})
}
function enc3 () {
return new Promise((resolve, reject) => {
setTimeout(() => {
reject('str')
}, 3000)
errorHandlers.push(() => {
console.log('handler 3')
})
})
}
Promise.all([enc1(), enc2(), enc3()]).then(() => {
console.log('all resovled')
}).catch((e) => {
errorHandlers.forEach(handler => handler(e))
})
It'd give you option to handle the 'global' error in each promise. Before creating promise all, you can reset the errorHandlers to prevent multiple errorHandler execution

Does promise resolved in n-th setTimeout cause memory leak?

I can see in Chrome task manager that the tab in which following code is running eats more and more memory, and it is not released until the promise is resolved
UPDATE
Main idea here is to use a single 'low level' method which would handle "busy" responses from the server. Other methods just pass url path with request data to it and awaiting for a valuable response.
Some anti-patterns was removed.
var counter = 1
// emulates post requests sent with ... axios
async function post (path, data) {
let response = (counter++ < 1000) ? { busy: true } : { balance: 3000 }
return Promise.resolve(response)
}
async function _call (path, data, resolve) {
let response = await post()
if (response.busy) {
setTimeout(() => {
_call(path, data, resolve)
}, 10)
throw new Error('busy')
}
resolve(response.balance)
}
async function makePayment (amount) {
return new Promise((resolve, reject) => {
_call('/payment/create', {amount}, resolve)
})
}
async function getBalance () {
return new Promise((resolve, reject) => {
_call('/balance', null, resolve)
})
}
makePayment(500)
.then(() => {
getBalance()
.then(balance => console.log('balance: ', balance))
.catch(e => console.error('some err: ', e))
})

The first time you call _call() in here:
async function getBalance () {
return new Promise((resolve, reject) => {
_call('/balance', null, resolve)
})
}
It will not call the resolve callback and it will return a rejected promise and thus the new Promise() you have in getBalance() will just do nothing initially. Remember, since _call is marked async, when you throw, that is caught and turned into a rejected promise.
When the timer fires, it will call resolve() and that will resolve the getBalance() promise, but it will not have a value and thus you don't get your balance. By the time you do eventually call resolve(response.balance), you've already called that resolve() function so the promise it belongs to is latched and won't change its value.
As others have said, there are all sorts of things wrong with this code (lots of anti-patterns). Here's a simplified version that works when I run it in node.js or in the snippet here in the answer:
function delay(t, val) {
return new Promise(resolve => {
setTimeout(resolve.bind(null, val), t);
});
}
var counter = 1;
function post() {
console.log(`counter = ${counter}`);
// modified counter value to 100 for demo purposes here
return (counter++ < 100) ? { busy: true } : { balance: 3000 };
}
function getBalance () {
async function _call() {
let response = post();
if (response.busy) {
// delay, then chain next call
await delay(10);
return _call();
} else {
return response.balance;
}
}
// start the whole process
return _call();
}
getBalance()
.then(balance => console.log('balance: ', balance))
.catch(e => console.error('some err: ', e))

Run callback function after forEach is done

In the project, I have a loop going through a list of urls. It downloads file from every url and do some post process over the downloaded file.
After the all the process done (both download process and post process), I want to execute a callback function. Because post process includes some streaming task, it has close event. If the last item can be identified, I can pass the callback function to the close event. However, since the loop is async, I can't track which item is done at last.
For now, I use a 5 second timeout to make sure the callback is executed after the whole process. Obviously, this is not sustainable. What's a good way to handle this?
loop code:
exports.processArray = (items, process, callback) => {
var todo = items.concat();
setTimeout(function() {
process(todo.shift());
if(todo.length > 0) {
// execute download and post process each second
// however it doesn't guarantee one start after previous one done
setTimeout(arguments.callee, 1000);
} else {
setTimeout(() => {callback();}, 5000);
}
}, 1000);
};
processArray(
// First param, the array
urlList,
// Second param, download and post process
(url) => {
if(url.startsWith('http')) {
getDataReg(url, uid);
}
else if(url.startsWith('ftp')) {
getDataFtp(url, uid);
}
else {
console.log('not a valid resource');
}
},
// Third param, callback to be executed after all done
() => {
Request.get(`${config.demouri}bound=${request.query.boundary};uid=${uid}`, {
method: 'GET',
auth: auth
})
.on('response', (response) => {
console.log('response event emmits');
zipFiles(uid)
.then((path) => {
reply.file(path, { confine: false, filename: uid + '.zip', mode: 'inline'}).header('Content-Disposition');
});
});
}
);
Download and post process:
exports.getDataFtp = (url, uid) => {
console.log('get into ftp');
var usefulUrl = url.split('//')[1];
var spliter = usefulUrl.indexOf('/');
var host = usefulUrl.substring(0, spliter);
var dir = usefulUrl.substring(spliter+1, usefulUrl.length);
var client = new ftp();
var connection = {
host: host
};
var fileNameStart = dir.lastIndexOf('/') + 1;
var fileNameEnd = dir.length;
var fileName = dir.substring(fileNameStart, fileNameEnd);
console.log('filename: ', fileName);
client.on('ready', () => {
console.log('get into ftp ready');
client.get(dir, (err, stream) => {
if (err) {
console.log('get file err:', err);
return;
} else{
console.log('get into ftp get');
stream.pipe(fs.createWriteStream(datadir + `download/${uid}/${fileName}`));
stream.on('end', () => {
console.log('get into ftp close');
unzipData(datadir + `download/${uid}/`, fileName, uid);
client.end();
});
}
});
});
client.connect(connection);
};
exports.getDataReg = (url, uid) => {
console.log('get into http');
var fileNameStart = url.lastIndexOf('/') + 1;
var fileNameEnd = url.length;
var fileName = url.substring(fileNameStart, fileNameEnd);
var file = fs.createWriteStream(datadir + `download/${uid}/${fileName}`);
if (url.startsWith('https')) {
https.get(url, (response) => {
console.log('start piping file');
response.pipe(file);
file.on('finish', () => {
console.log('get into http finish');
unzipData(datadir + `download/${uid}/`, fileName, uid);
});
}).on('error', (err) => { // Handle errors
fs.unlink(datadir + `download/${uid}/${fileName}`);
console.log('download file err: ', err);
});
} else {
http.get(url, (response) => {
console.log('start piping file');
response.pipe(file);
file.on('finish', () => {
unzipData(datadir + `download/${uid}/`, fileName, uid);
});
}).on('error', (err) => {
fs.unlink(datadir + `download/${uid}/${fileName}`);
console.log('download file err: ', err);
});
}
};
function unzipData(path, fileName, uid) {
console.log('get into unzip');
console.log('creating: ', path + fileName);
fs.createReadStream(path + fileName)
.pipe(unzip.Extract({path: path}))
.on('close', () => {
console.log('get into unzip close');
var filelist = listFile(path);
filelist.forEach((filePath) => {
if (!filePath.endsWith('.zip')) {
var components = filePath.split('/');
var component = components[components.length-1];
mv(filePath, datadir + `processing/${uid}/${component}`, (err) => {
if(err) {
console.log('move file err: ');
} else {
console.log('move file done');
}
});
}
});
fs.unlink(path + fileName, (err) => {});
});
}

After the all the process done (both download process and post process), I want to execute a callback function.
The interesting thing about a series of asynchronous processes is that you can never know when exactly all processes will complete. So setting a timeout for the callback is quick&dirty way to do it, but it's not reliable for sure.
You can instead use a counter to solve this problem.
Let's say you have 10 operations to perform. At the beginning you set your counter to ten counter = 10 And after each process is completed, regardless how (it can either succeed or fail), you can decrement the counter by 1 like counter -= 1 and right after it you can check if the counter is 0, if so that means all processes are completed and we reached the end. You can now safely run your callback function, like if(counter === 0) callback();
If I were you, I would do something like this:
*Notice that the called process should return a promise, so that I can know when it finishes (again regardless how)
*If you need help about promises, this useful article might help you: https://howtonode.org/promises
*Oh and one more thing, you should avoid using arguments.callee, because it's deprecated. Here is why Why was the arguments.callee.caller property deprecated in JavaScript?
exports.processArray = (items, process, callback) => {
var todo = [].concat(items);
var counter = todo.length;
runProcess();
function runProcess() {
// Check if the counter already reached 0
if(checkCounter() === false) {
// Nope. Counter is still > 0, which means we got work to do.
var processPromise = process(todo.shift());
processPromise
.then(function() {
// success
})
.catch(function() {
// failure
})
.finally(function() {
// The previous process is done.
// Now we can go with the next one.
--counter;
runProcess();
})
}
};
function checkCounter() {
if(counter === 0) {
callback();
return true;
} else {
return false;
}
}
};

What you want to do is to make all your asynchronous processes converge into a single promise that you can use to execute the callback at the correct moment.
Lets start at the point each process is complete, which I assume is in the callback passed to the mv() function in unzipData(). You want to wrap each of these asynchronous actions in a Promise that resolves in the callback and you also want to use these promises later and for that you use the .map() method to collect the promises in an array (instead of .forEach()).
Here's the code:
var promises = filelist.map((filePath) => {
if (!filePath.endsWith('.zip')) {
var components = filePath.split('/');
var component = components[components.length-1];
return new Promise((resolve, reject) =>
mv(filePath, datadir + `processing/${uid}/${component}`, (err) => {
if(err) {
console.log('move file err: ');
reject(); // Or resolve() if you want to ignore the error and not cause it to prevent the callback from executing later
} else {
console.log('move file done');
resolve();
}
}));
}
return Promise.resolve();
});
(if the asynchronous action is not to be executed, a Promise that resolves immediately is returned instead)
Now, we can turn this list of Promises into a single Promise that resolves when all of the promises in the list has resolved:
var allPromise = Promise.all(promises);
Next, we need to look further up in the code. We can see that the code we've just been looking at is itself part of an event handler of an asynchronous action, i.e. fs.createReadStream(). You need to wrap that in a promise that gets resolved when the inner promises resolve and this is the promise that the unzipData() function shall return:
function unzipData(path, fileName, uid) {
console.log('get into unzip');
console.log('creating: ', path + fileName);
return new Promise((outerResolve) =>
fs.createReadStream(path + fileName)
.pipe(unzip.Extract({path: path}))
.on('close', () => {
console.log('get into unzip close');
var filelist = listFile(path);
// Code from previous examples
allPromise.then(outerResolve);
}));
}
Next, we look at the functions that use unzipData(): getDataReg() and getDataFtp(). They only perform one asynchronous action so all you need to do is to make them return a promise that resolves when the promise returned by unzipData() resolves.
Simplified example:
exports.getDataReg = (url, uid) => {
return new Promise((resolve, reject) => {
// ...
https.get(url, (response) => {
response.pipe(file);
file.on('finish', () => {
unzipData(datadir + `download/${uid}/`, fileName, uid)
.then(resolve);
});
}).on('error', (err) => { // Handle errors
fs.unlink(datadir + `download/${uid}/${fileName}`);
reject(); // Or resolve() if you want to ignore the error and not cause it to prevent the callback from executing later
});
// ...
});
}
Finally, we get to the processArray() function and here you need to do the same thing we did to begin with: map the processes into a list of promises. First, the process function passed needs to return the promises returned by getDataReg() and getDataFtp():
// Second param, download and post process
(url) => {
if(url.startsWith('http')) {
return getDataReg(url, uid);
}
else if(url.startsWith('ftp')) {
return getDataFtp(url, uid);
}
else {
console.log('not a valid resource');
}
return Promise.reject(); // or Promise.resolve() if you want invalid resources to be ignored and not prevent the callback from executing later
}
Now, your processArray() function can look like this:
exports.processArray = (items, process, callback) =>
Promise.all(items.map(process))
.then(callback)
.catch(() => console.log('Something went wrong somewhere'));
Your callback will get invoked when all asynchronous actions have completed, regardless of in which order they do. If any one of the promises rejects, the callback will never be executed so manage your promise rejections accordingly.
Here's a JSFiddle with the complete code: https://jsfiddle.net/upn4yqsw/

In general, since nodejs does not appear to have implemented Streams Standard to be Promise based, at least from what can gather; but rather, uses an event based or callback mechanism, you can use Promise constructor within function call, to return a fulfilled Promise object when a specific event has been dispatched
const doStuff = (...args) => new Promise((resolve, reject)) => {
/* define and do stream stuff */
doStreamStuff.on(/* "close", "end" */, => {
// do stuff
resolve(/* value */)
})
});
doStuff(/* args */)
.then(data => {})
.catch(err => {})