I have a system that wants to create many folders in dropbox using the api, however i appear to attempt to create all the folders at once which causes errors to be generated, stating that i am performing too many write operations from dropbox.
My code is as follows and first uses reduce to create multiple promises which i thought were chained. In these promises the function add is called, which uploads the case to mongodb and then creates a dropbox folder for it, however this results in throttling errors..
bulkAdd: function (req, callback) {
issues = []
i = 1
req.reduce((promise, audit) => {
return promise.then(_ => this.add(audit, function(err,data){
if (err){
console.log('\n'+i+ ' ' + data.scanner_ui + '\n');
}
}));
}, Promise.resolve()).catch(error => {console.log(error)});
},
add: function (req, callback) {
delete req.status
var audit = new Audit(req);
if (req['status_value'] != undefined && req.status_value != ''){
console.log(req['status_value'])
audit.status = [{
status_value : req['status_value'],
status_notes : req['status_notes'],
status_date : req['status_date'],
}]
}
audit.save(function (err, data) {
if (err) {
callback(err, data)
}
else {
return dropbox_functions.createFolder(data.ui)
.then(response => {console.log(response)}, error=> {console.log('\n\n\n',error.error.error)})
.catch(error => console.log(error))
}
});
},
So the problem in your current question comes from the fact that your add function doesn't return a value, I only see it returning undefined.
If a promise returns something else than a promise in its then / catch block, it will use this input for the following function, and it will not wait for any internal processes to run through before finishing
If inside your then / catch blocks, you would return a promise, it would wait before continuing to the next then block, thus handling your requests sequential.
Now in your current code, I believe the easiest would be to handle the resolve inside your reduce, since you seem to be stuck with your callback handle already.
req.reduce((promise, audit) => {
return promise.then(_ => new Promise(
function( resolve, reject) {
this.add(audit, function(err,data){
if (err){
console.log('\n'+i+ ' ' + data.scanner_ui + '\n');
reject( err );
return;
}
resolve( data );
});
})
);
}, Promise.resolve()).catch(error => {console.log(error)});
in this case, the promise would either reject or resolve itself. I have chosen to submit err and data respectively, so this would be handled in the end in case an error occurs, and so that you can have the last data that got saved successfully
To properly chain you should make sure that you return a promise object.
Your reduce in the end creates a promise chain something like this.
Promise.resolve()
.then(() => {
console.log('setting up first timeout');
setTimeout(() => {
console.log("1 wait");
}, 2000);
})
.then(() => {
console.log('setting up second timeout');
setTimeout(() => {
console.log("2 wait");
}, 2000);
})
.catch(err => console.log("error", err));
If you run it, you'll see that it does not wait for one promise to end before moving down the chain.
Whereas, in the second example it waits for the first one to complete because a Promise object is returned by the first promise.
Promise.resolve()
.then(() => {
return new Promise(function(resolve, reject) {
console.log('setting up first timeout');
setTimeout(() => {
console.log("1 wait");
resolve();
}, 2000);
});
})
.then(() => {
return new Promise(function(resolve, reject) {
console.log('setting up second timeout');
setTimeout(() => {
console.log("2 wait");
resolve();
}, 2000);
});
})
.catch(err => console.log("error", err));
The difference is the first example is not returning Promise object. So, if you make sure each success handler returns a Promise object which is resolved only after your add function is done executing, you should be fine.
Unless there is a good reason for bulkAdd() to be written in nodeback style, you will find it more convenient to return a promise. The need for a nodeback will disappear and .reduce() will sit much more comfortably inside the function.
Using .reduce() and preserving nodeback style is possible but cumbersone, as it involves a rather ugly double-shuffle from nodeback to promise, and promise back to nodeback.
Assuming you are free to adapt the caller(s) to accept a returned promise, the code would be something like this :
'bulkAdd': function(req) {
return req.reduce((promise, audit) => {
return promise.then(_ => this.add(audit));
}, Promise.resolve());
},
'add': function(req) {
delete req.status;
var audit = new Audit(req);
if (req.status_value != undefined && req.status_value != '') {
audit.status = [{
'status_value': req.status_value,
'status_notes': req.status_notes,
'status_date': req.status_date
}];
}
return new Promise((resolve, reject) => { // in-line promisification of audit.save()
audit.save((err, data) => {
err ? reject(err) : resolve(data);
});
})
.then(data => dropbox_functions.createFolder(data.ui));
},
all catching and logging intentionally removed
The detail may differ, depending primarily on :
what data (if any) you want to be delivered to the caller
what you want to happen when errors occur.
Related
Desperately trying to write a sync version of https://www.npmjs.com/package/node-firebird#reading-blobs-aasynchronous
Basically I need to (a)wait twice:
for the callback function to execute so that the eventEmitter is available
for the "end" event to occur
and then return the Buffer.
my code (JS/TS mix for now) currently does 2, but not 1 : readBlob returns undefined, then Buffer.concat(buffers) is called later ... :
function readBLOB(callback: any): Buffer {
return callback(async (err, _, eventEmitter) => {
let buffers = []
if (err)
throw err
eventEmitter.on('data', chunk => {
buffers.push(chunk);
});
return await eventEmitter.once('end', function (e) {
return Buffer.concat(buffers)
})
})
}
Sorry to ask one more time (yes, I checked a lot of other questions and tried a lot of things...), but how to make this work (simply...) ?
(the function that calls the callback is fetch_blob_async in https://github.com/hgourvest/node-firebird/blob/master/lib/index.js#L4261 , just in case...)
There are few mistakes here like returning an callback function, witch returns, i guess, undefined or returning something IN an callback function that makes no sense.
Also async / await makes no sense here it has no effect. async / await is only useful if you want to await till some Promise resolves. But you have no Promise in your code at all.
What you need is new Promise
function readBLOB(callback) {
return new Promise((resolve, reject) => {
callback((err, _, eventEmitter) => {
let buffers = [];
if (err) reject(err);
eventEmitter.on("data", chunk => {
buffers.push(chunk);
});
eventEmitter.once("end", function(e) {
resolve(Buffer.concat(buffers));
});
});
});
}
Simple like that. You resolve your Buffer and reject if some error occurs
Now you can use it like:
readBLOB(cb).then(data => {
console.log(data);
})
Promises just baffle me.
I'm trying to make a mock data service to imitate axios.
My mock put call passes a targetUrl to _fetch which then sees if it's a valid url and either returns a new Promise with a delayed .resolve
const _returnResponse = (mockData, time = 0) => new Promise((resolve) => {
setTimeout(() => {
resolve(mockData);
}, time);
});
or a new Promise with a delayed .reject
const _returnError = (time = simulatedDelay) => {
const returnValue = new Promise(((resolve, reject) => {
setTimeout(() => {
reject(new Error('error'));
}, time);
}));
return returnValue;
};
but when I make my mock put call this returns a mock data that the calling method interprets as a success and console logs in its .then
put(target, putBody) {
const returnValue = _fetch(target, simulatedDelay)
returnValue.then(response => _console('PUT', target, response, putBody));
return returnValue;
},
But with an invalid target console logs an uncaught error
or this handles the error correctly, but console logs an undefined response
put(target, putBody) {
const returnValue = _fetch(target, simulatedDelay).then(response => _console('PUT', target, response, putBody));
return returnValue;
},
Here's the calling method:
saveStuff({ commit, state }, newStuff) {
//other code
return this.$mockAxios.put(url, putBody)
.then((response) => {
return response;
});
},
I feel like I'm completely missing something and I've researched this for hours and I'm still not getting it.
As a direct answer to the question: yes, you can add .then() to a promise after it's created.
Example:
const hi = new Promise((resolve, reject) => {
setTimeout(() => resolve('hello'), 2000);
});
hi.then(result => console.log(result));
As for promises baffling you, I would recommend (aside from more reading) just playing around a lot in your IDE with setTimeout. I know you're already using setTimeout in your mock, but strip it down further and just run the code in its own file, so that you control the whole environment. With lots of console.log('blah') to see the order and such. Also ensure you're just as familiar with callbacks, as promises are just syntactic sugar for callbacks. Try to read up on the JS event loop at the same time - it might provide some context if you know how and when a callback/promise executes.
Note that you can even add .then() after the callback has resolved or rejected. Hence the term "promise" - it's literally a promise that your .then() function will run. https://en.wikipedia.org/wiki/Promise_theory
const hi = new Promise((resolve, reject) => {
setTimeout(() => {
console.log('one second');
resolve();
}, 1000);
});
setTimeout(() => {
hi.then(() => console.log('two seconds. this executes approximately a full second after the first promise has resolved'));
}, 2000);
I've been trying to figure out what the proper way would be to write a promise for this function. I have an asynchronous function that makes an HTTP request to the server to retrieve a response, "documents_fileUploader." I am mapping through the "url" of each item within the response, and each url will go in to a function that makes another HTTP request and then sets the state. I want to fire the "upload()" function only after everything within the "documents_fileUploader()" function is complete. I tried doing this without a promise and it went straight to my "upload()" function because request was still pending. Any suggestions on how to go about this?
documents_fileUploader(formData).then(resp => {
resp.data.items.map(url => {
const key = url.split("/")[4];
this.setState({
urls: [...this.state.urls, url],
keys: [...this.state.keys, key]
});
this.getFileObject(key);
})
}).then(() => {
this.upload();
})
getFileObject = file => {
file_view(file).then(resp => {
this.setState({
mimeTypes: [...this.state.mimeTypes, resp.data.item.headers.contentType]
})
}).catch(err => {
console.log(err);
})
}
To your main question, you can wait for every promise that your .map call returns by using the Promise.all method.
Second, in order for that to work, your getFileObject function must return the promise it creates.
So incorporating those two changes, your snippet might look like:
documents_fileUploader(formData).then(resp => {
return Promise.all(resp.data.items.map(url => { // Wrap in Promise.all and return it
const key = url.split("/")[4];
this.setState({
urls: [...this.state.urls, url],
keys: [...this.state.keys, key]
});
return this.getFileObject(key); // Make sure to return this promise as well.
}));
}).then(() => {
// Now this won't happen until every `getFileObject` promise has resolved...
this.upload();
})
getFileObject = file => {
return file_view(file).then(resp => { // And return the promise here.
this.setState({
mimeTypes: [...this.state.mimeTypes, resp.data.item.headers.contentType]
})
}).catch(err => {
console.log(err);
})
}
I'm writing unit-tests for my function that fetches info from some REST API. I am using ramda Future type (source).
The following test works weird:
it('should return Maybe of Nothing', done => {
let response = {
status: 200,
json: () => {
return {
results: []
}
}
}
let fakeFetch = {
fetch: () => {
return new Promise((resolve, reject) => {
resolve(response)
})
}
}
// String -> Future Error Maybe
let result = Utils.fetchGiantBomb(faker.random.word(), fakeFetch.fetch);
result.fork(err => {
assert.fail(err, 'expected to return Maybe of Nothing');
done();
}, data => {
expect(Maybe.isJust(data)).to.be.true;
done();
})
})
data should be of type Maybe.Nothing. If I expect Maybe.isNothing the test passes, but I want to see what happens when the test fails, so I set it to Maybe.isJust, which return false. After looking at this for a while, I noticed that when the expect fail it jump up to the error handling (err callback), which then just stop executing any assertion (which result in a 2000ms timeout).
In the Future sources I saw that when the success callback fails, it executes the failure callback. How can I complete this test so it display that the data is not what I expect?
I think the problem is that, when your REST call fails, done() is never called.
Not sure if expect has a .catch method when it fails, but you can try to add
.catch(done);
at the end of your expect function.
Hope that helps.
Calling future.fork(errorHandler, successHandler) will currently ensure that any exceptions thrown in the successHandler will propagate through to the errorHandler.
One way around this (though perhaps not ideal as it is undocumented) is to call future._fork(errorHandler, successHandler) rather than future.fork where errors thrown in the successHandler will not be captured.
Alternatively, a number of test frameworks support passing an error to the done callback such as:
result.fork(err => {
done('Expected to return Maybe of Nothing: ' + err);
}, data => {
expect(Maybe.isJust(data)).to.be.true;
done();
})
I think Ramda shouldn't catch the exception there. But i don't know what's they are trying to do.
It look like you are using Mocha. It maybe good idea to convert your Future to Promise first, then observe the Promise. ie:
const futureToPromise = future => {
return new Promise((resolve, reject) => future.fork(reject, resolve))
}
it('should return Maybe of Nothing', () => {
let response = {
status: 200,
json: () => {
return {
results: []
}
}
}
let fakeFetch = {
fetch: () => {
return new Promise((resolve, reject) => {
resolve(response)
})
}
}
// String -> Future Error Maybe
let result = Utils.fetchGiantBomb(faker.random.word(), fakeFetch.fetch);
// return it because Mocha can handle this
return futureToPromise(result).then(data => {
expect(Maybe.isJust(data)).to.be.true;
}, () => {
// fail
assert.fail(err, 'expected to return Maybe of Nothing');
})
})
I want to execute these three Firebase requests in parallel and receive a callback when all requests are finished. I thought I can use await*, but it seems it is not supported in Babel. I am not worried about data consistency, as I am writing to/updating different firebase data nodes.
I am using babel.
await* Promise.all([
firebaseRef1.push(data1),
firebaseRef2.push(data2),
firebaseRef3.push(data3)
]);
// all three are finished
which initially was this:
const [first, second, third] = await* Promise.all([
firebaseRef1.push(data1),
firebaseRef2.push(data2),
firebaseRef3.push(data3)
]);
// all three are finished
push() does not return a promise, so you'd have to use it's callback to resolve or reject one of your own.
const allPushes = Promise.all([
new Promise((resolve, reject) => {
firebaseRef1.push(data1, error => {
error ? reject(error) : resolve();
});
}),
new Promise((resolve, reject) => {
firebaseRef2.push(data2, error => {
error ? reject(error) : resolve();
});
}),
new Promise((resolve, reject) => {
firebaseRef3.push(data3, error => {
error ? reject(error) : resolve();
});
})
]);
allPushes.then(() => {
// all three are finished
}, error => {
// A push failed
});
Anid Monsur's answer is 100% right.
But, it's important to note here that Firebase uses WebSockets. This means that there are no actual requests. In your case it's perfectly fine to just call .push() and not worry about the callback.
firebaseRef1.push(data1);
firebaseRef2.push(data2);
firebaseRef3.push(data3);
It's in the listeners that you'll know the callback was completed.
firebaseRef1.on('value', (snap) => console.log(snap.val()));
firebaseRef2.on('value', (snap) => console.log(snap.val()));
firebaseRef3.on('value', (snap) => console.log(snap.val()));