I have the following function. It calls itself repeatedly and iterates through ftp servers checking for new files. I'm trying to make it a promise so that I can operate().then(function(newFilesObject), but I can't get the .then on operate to activate. It does attempt to resolve it but doesn't send through. By the way, newFiles is a global variable that gets the files per server appended to it. If more code is wanted I can post or github it.
function operate(){
return new Promise(function(resolve, reject){
if(servers[i]){
if(i!== 0) ftp = new JSFtp(servers[i].server)
local = servers[i].local
remote = servers[i].remote
localFiles = fs.readdirSync(local)
}else{
console.log('trying to resolve')
console.log(newFiles)
resolve(newFiles)
}
gatherFiles(remote).then(function(files){
if(files.length>0){
downloadNew(files).then(function(){
console.log('Done: ' + servers[i].server.host)
i++
operate()
})
}else{
console.log('No updates: ' + servers[i].server.host)
i++
operate()
}
})
})
}
operate().then(function(files){
console.log('files: ' + files)
})
The promises in the code sample do not return as their resolvers or rejectors are not always invoked. In fact, resolve is only invoked when i === 0. According to the Promises/A+ specification, promises may only be transitioned to a fulfilled state by invoking resolve. It also can only be transitioned to a rejected state by invoking reject or throwing an exception from within the executor. Therefore, reaching the end of the executor without invoking either or passing one as a callback ensures the promise remains in pending state indefinitely.
The goal you seek may be achieved with a little refactoring. Considering the following as your goal:
Sequentially through each FTP server...
Read a given directory for a list of files
Compare list of files to one stored locally to determine new ones
If there are new ones, download them sequentially
Return a list of all newly downloaded files
Data
var knownFTPServers = [{
'localDirectory': 'sub/',
'localFilepaths': ['docA.json', 'docB.json'],
'remoteDirectory': 'remsub/',
'remoteFilepaths': [],
'jsftpHandle': undefined,
'host': 'example.com'
},
{
'localDirectory': 'root/',
'localFilepaths': ['file1.txt', 'file2.txt'],
'remoteDirectory': 'remroot/',
'remoteFilepaths': [],
'jsftpHandle': undefined,
'host': 'geocities.com'
}];
Logic
function pullNewFilesFromFTPServer(ftpServer) {
return new Promise(function (resolve, reject) {
var handle = new JSFtp(ftpServer);
ftpServer.jsftpHandle = new JSFtp(ftpServer);
// Returns a promise for reading a directory from JSFtp server
// resolves with file list
// rejects with FTP error
function readdir(directory) {
return new Promise(function (resolve, reject) {
handle.ls(ftpServer.remoteDirectory, function (err, res) {
if (err) return reject(err);
resolve(res);
});
});
}
// Returns a promise for downloading a file from a remote JSFtp server
// resolves with the filepath of the downloaded filepath
// rejects with FTP error
function downloadFile(path) {
return new Promise(function (resolve, reject) {
handle.get(path, path, function (err) {
if (err) return reject(err);
resolve(path);
});
});
}
// get all remote filepaths on server
readdir(ftpServer.remoteDirectory)
// filter out filepaths already present locally
.then(function (remoteFilepaths) {
return remoteFilepaths.filter(function (path) {
return ftpServer.localFilepaths.indexOf(path) < 0;
});
})
// download new filepaths sequentially
// reduce turns the array of new filepaths into a promise chain
// return new filepaths after completing the promise chain
.then(function (newFilepaths) {
return newFilepaths.reduce(function (previousDownloadPromise, newPath) {
return previousDownloadPromise.then(function () {
return downloadFile(newPath);
});
}, Promise.resolve())
.then(function () { return newFilepaths; });
})
// resolve server promise with new filepaths or reject with errors
.then(resolve, reject);
});
}
var allFilesDownloaded = [];
knownFTPServers.reduce(function (previousServerPromise, server) {
return previousServerPromise.then(function (filesDownloaded) {
allFilesDownloaded = allFilesDownloaded.concat(filesDownloaded);
return pullNewFilesFromFTPServer(server);
});
}, Promise.resolve([]))
.then(function () {
console.log(allFilesDownloaded);
}, function (err) {
console.err(err);
});
Though it may seem a little more complicated in some places, the actions of each function are more modular. The idea that is somewhat unintuitive is using Array.prototype.reduce to turn an array of data into an array of promises executed sequentially.
Since creating a promise to download a file attempts to download the file immediately, one can't create all the promises at once if one intends to download them one at a time. Otherwise, the sequence might look a somewhat simpler.
Related
One of the issues I have faced when crafting Promises for certain app context is that of wanting to delay any execution of the code inside the promise until later. This happens frequently when I have manager objects that maintain a collection of Promises for execution at a later time. To remedy this, I end up creating builder functions that are called by the manager objects at the time the Promise needs to be executed. This is tedious and leads to a fair amount of "boilerlate" code.
For example, here's one of my Promise builder functions:
this._buildPollingPromise = function(ethTransWaiter) {
return new Promise(function(resolve, reject) {
// Execute the function that builds a polling method promise.
ethTransWaiter.confirmTransPromiseBuilder.buildPromise()
.then(function(result) {
...
})
.then(function(ignoreResult) {
resolve(ethTransWaiter.isConfirmed);
})
.catch(function(err)
{
// Reject the promise with the error received.
reject(err);
});
});
}
I have to delay execution of the ethTransWaiter.confirmTransPromiseBuilder.buildPromise() method because if it executes at the time the Promise is created, it will fail because the conditions aren't in place yet for it to execute successfully.
Therefore, I am wondering if there is a built-in method, or an NPM package, that creates or helps create Promises that can be built in a dormant state, so that the code in the function that lives inside the Promise constructor does not execute until some later time (i.e. - at the precise time when you want it to execute) That would save me a lot of boilerplate coding.
May be something like this?
function wait(ms) {
return new Promise(function (resolve) {
setTimeout(resolve, ms);
});
}
this._buildPollingPromise = function (ethTransWaiter) {
var waitUntilLater = wait(3000);
var buildPromise = new Promise(function (resolve, reject) {
// Execute the function that builds a polling method promise.
ethTransWaiter.confirmTransPromiseBuilder.buildPromise()
.then(function (result) {
//...
})
.then(function (ignoreResult) {
resolve(ethTransWaiter.isConfirmed);
})
.catch(function (err) {
// Reject the promise with the error received.
reject(err);
});
});
return Promise.all([waitUntilLater, buildPromise]).then(function (results) {
return results;
});
}
I have a sequence of function calls, connected with ES6 promises. Apparently, there is something wrong with this implementation, as API calls to the endpoint are not returning anything and the browser is stuck waiting for a response.
Please advise.
module.exports.insertTreatmentDetails = function (req, res) {
var doctorId = 10000
var departmentId = 10000
var procedureid = 10000
var hospitalSchema = new hospitalModel();
var p = new Promise(function (resolve, reject) {
counterSchema.getNext('Treatment.doctor.doctorId', collection, function (doctorId) {
doctorId = doctorId;
})
counterSchema.getNext('Treatment.departmentId', collection, function (departmentId) {
departmentId = departmentId
})
counterSchema.getNext('Treatment.procedureid', collection, function (procedureid) {
procedureid = procedureid
})
}).then(function () {
setData()
}).then(function (){
hospitalSchema.save(function (error, data) {
if (error) {
logger.error("Error while inserting record : - " + error)
return res.json({ "Message": error.message.split(":")[2].trim() });
}
else {
return res.json({ "Message": "Data got inserted successfully" });
}
});
});
};
The short answer is that you aren't calling resolve or reject inside the first promise in your chain. The promise remains in a pending state. Mozilla has a good basic explanation of promises.
How to Fix
It appears that you want to retrieve doctorId, departmentId, and procedureId before calling setData. You could try to wrap all three calls in one promise, checking whether all three have returned something in each callback, but the ideal is to have one promise per asynchronous task.
If it's feasible to alter counterSchema.getNext, you could have that function return a promise instead of accepting a callback. If not, I would recommend wrapping each call in its own promise. To keep most true to what your code currently looks like, that could look like this:
const doctorPromise = new Promise((resolve, reject) =>
counterSchema.getNext('Treatment.doctor.doctorId', collection, id => {
doctorId = id;
resolve();
}));
Then you could replace the first promise with a call to Promise.all:
var p = Promise.all([doctorPromise, departmentPromise, procedurePromise])
.then(setData)
.then(/* ... */);
Promises allow you to pass a value through to the next step, so if you wanted to get rid of your broadly-scoped variables (or set them in the same step where you call setData), you could just pass resolve as your callback to counterSchema.getNext and collect the values in the next step (also how you'd want to do it if you have counterSchema.getNext return a promise:
Promise.all([/* ... */])
.then(([doctorID, departmentID, procedureID]) => {
// If you aren't changing `setData`
doctorId = doctorID;
departmentId = departmentID;
procedureid = procedureID;
setData();
// If you are changing `setData`
setData(doctorID, departmentID, procedureID);
}).then(/* ... */).catch(/* I would recommend adding error handling */);
I'm kind of new with the whole promise thing so I might be doing something really wrong. If anyone can enlighten me, all advices/informations are welcome.
So here's the code of what I'm trying to accomplish (simplified and absolutly not optimal for understanding purpose):
// Get data from webservice
$scope.sendGet(id, option).then(function (response){
// Fill the model
$scope.model[option] = response.data;
}).then(function(){
if(option == $scope.PROFILES){
var p1 = new Promise((resolve, reject) => {
$scope.getX1($scope.model[option][0][0].id);
});
var p2 = new Promise((resolve, reject) => {
$scope.getX2($scope.model[option][0][0].id);
});
var p3 = new Promise((resolve, reject) => {
$scope.getX3($scope.model[option][0][0].id);
});
var p4 = new Promise((resolve, reject) => {
$scope.my_data = JSON.parse($scope.model[option][0][0].list);
});
// Execute all promises to get the data
Promise.all([p1,p2,p3,p4]).then(() => {
debugger;
// Do some validation and extra formatting on the data we just downloaded
$scope.update();
});
}
}).then(function(){
// Display the data to the user
$scope.move(option, 1, $scope.EDITING);
});
The intended behavior here is:
Get data -> With this data, use id to get data from 4 sources (the 4 promises) -> Once all the data is downloaded, update some references and do some cleaning -> move (which is a method that updates the view and do some other stuff UI related)
But for some reason, the debugger; and the $scope.update(); never get executed. I tried moving these in the same .then as the $scope.move() function but then it executes before the data from the Promise.all has been retrieved.
You never resolve promises 1-4, so the "success callback" to Promise.all(...).then never fires. In the callback given to the constructor of each promise, call resolve with the data each promise is getting.
// ...
var p1 = new Promise((resolve, reject) => {
resolve($scope.getX1($scope.model[option][0][0].id));
});
// ...
This is how you "return" data, so to speak, from a Promise. Please see this article for details.
EDIT: if $scope.getX1 itself returns a Promise, you can simply assign it to p1, ie:
var p1 = $scope.getX1($scope.model[option][0][0].id);
I have a relatively simple task; existing information is out of date, so I have to request information from an API, modify the existing info file and push it back to the server, updated. The information is in the form of a json file; simple enough. That file contains an object with an array of objects that have several properties that must be updated. This is where the problems occur; the array of objects generates an array of API requests, whose responses must match the original object that spawned the request (since the response contains info that must be updated in the object).
This is the gist of what I've managed to do so far with promises:
function main() {
// First get existing data.
getExistingData().then(function(result) {
console.log(result); // It worked, return it for next 'then' to use.
return result;
}, function(err) {
console.log(err); // This usually never happens.
}).then(function(result) { // Use the existing data to generate the requests for new data.
requestNewData(result).then(function(moddedJson) {
console.log(moddedJson); // This happens BEFORE I get responses back from the request, which is wrong.
});
});
}
function getExistingData() {
return new Promise(function(resolve, reject) {
fetch('dataURLHere')
.then(function(res) {
resolve( res.json()); // Turn result into JSON, and return it.
})
})
}
function requestNewData(rawJson) {
return new Promise(function(resolve) {
// Loop over the number of objects in the original data.
for (var i = 0; i < rawJson.length; i++) {
// Loop over the array of objects within each object.
for (var multiId = 0; multiId < rawJson.hits.length; multiId++) {
var requestUrl = "someURLConstructedFromJsonData";
var hit = rawJson.hits[multiId];
new Promise(function(resolve) {
request(requestUrl, function(error, response, body) {
if (!error && response.statusCode == 200) {
// Need to parse the XML response into a js object.
parseString(body, function (err, result) {
hit.propertyToChange = result.propertyToChange;
hit.propertyToChange2 = result.propertyToChange2;
});
}
else {
console.log("No data for this item.");
}
resolve(hit);
});
})
}
}
resolve(rawJson);
})
}
Basically, the things I want to happen are:
1) Get original data. This is easy and accomplished by my code already.
2) Use original data to generate requests for each document in the data, and for each set of properties within each document. This is also not a problem.
3) Ensure the returning data from requests gets matched to existing data. THIS is the problem part that I can't wrap my head around.
The problem is that you're resolving too early.
The red flag is when you create a promise but never do anything with it:
new Promise(function(resolve) {
request(requestUrl, function(error, response, body) {
...
That promise does get resolved correctly, but no one is waiting on it. The simple solution is Promise.all:
function requestNewData(rawJson) {
return new Promise(function(resolve, reject) {
var promises = [];
for (var i = 0; i < rawJson.length; i++) {
...
promises.push(new Promise(function(resolve) {
...
}));
}
resolve(Promise.all(promises));
});
}
Now, Promise.all(promises) will resolve with an array of results. This might not be ideal, but you can but a then on it if you just want to wait using it:
return Promise.all(promises).then(function() {
resolve(updatedJson);
}, reject);
In this way, you could have each of the individual promises modify the response data. The promise returned by requestNewData won't resolve until all of them are done, so at that point updatedJson would be updated.
Be warned: Promise.all has fast-fail behavior. In your case, I think this is exactly what you want. But if you need to know which ones failed, or if you need to wait until all requests complete (fail or otherwise), Promise.all may not be the right thing.
PS: You should maybe reject if the request() function provides an error. Otherwise you could have holes in your data if there's a network error, without actually getting a reject.
I know of this question, but my need is a little different.
I'm trying to download multiple images but cannot prevent the app from exiting before the process is complete. I've gone through quite a few iterations, this is where I'm at currently:
router
.post('/', koaBody(), function *() {
var images = [];
//console.log(this.request.body.fields);
if (this.request.body.fields) {
images = this.request.body.fields['images'];
imageCount = images.length;
} else {
this.status = 500;
return;
}
var url = 'https://www.google.ca/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png';
var filepath = '../images/image_1.png';
var pipeRequest = function (url, filepath) {
return new Promise(function (resolve, reject) {
request.get(url)
.pipe(fs.createWriteStream(filepath))
.on('finish', function () {
console.log('piped');
});
});
};
co(function*() {
yield pipeRequest(url, filepath);
}).then(function () {
console.log('done co-func');
}).catch(function (err) {
return console.error(err);
});
console.log('App exiting...');
});
It's not the most elegant code but it works, if I wrap the co-function in a loop I can download as many as I want (the end-goal is to have this API receive a JSON list of image URLs). What I cannot do however is return the results of the piping to the caller, the app will always exit before that process is complete. I thought wrapping the function call in a yield would be the answer...but no.
When you're inside a generator function (a Koa handler), you use yield to wait.
The general pattern is to extract your async logic into a function that returns a promise, and then you yield it inside your route.
The first problem with your code is that you never use reject and resolve in the promise to transition it to a finished/errored state. The promise never knows when it's finished. Yielding it will never return control back to the route, which is probably why you nested it in another co process.
Here, I fix it so that it resolve()s on completion and reject(err)s on error. Note: yielding a promise that hits a reject(err) path will throw an exception (err) that you can try/catch at the call site.
var pipeRequest = function(url, filepath) {
return new Promise(function(resolve, reject) {
request.get(url)
.pipe(fs.createWriteStream(filepath))
.on('finish', function() { resolve() })
.on('error', function(err) { reject(err) })
});
};
The second problem I see is that you're nesting a co process in your route which is going to ensure that the route never waits on the yield inside it.
A Koa handler is already run inside a co process. Just replace that whole co block with a top-level yield pipeRequest(url, filepath) once you've implemented my resolve/reject changes above.