node.js asynchronous azure queue storage output binding - javascript

I am trying to create a JS function that reads a file from storage, writes its content to the database and pushes a result message to a queue. Reading and db processing are ok but nothing happens when writing to the queue.
The code is like:
process(file)
.then(() => {
context.bindings.outQueue = { file };
return context.log('File processing completed successfully');
})
.catch(err => context.log.error(err));
If I use context.done() instead, I get the following error:
Error: Choose either to return a promise or call 'done'. Do not use both in your script.
In either case no message is written to the queue.
What's wrong?

There are a couple of things that are most likely causing you issues. First, you should be returning within the main scope of the function. Not returning will cause your Function App to hang, which is probably why you tried adding context.done(). Second, you should not call context.done() when it could return prematurely. The error message you saw is trying to prevent you from doing so. Third, you should be returning your chained promise instead of calling context.done()`. This is what will let you write to the queue correctly.
To the first point: I think you're assuming that returning inside of .then(() => { ... }); is returning for the larger-scoped function. Remember that the lambda function passed into the .then() method is just returning a fulfillment value, which is used in order to pass the result of one promise to another promise (docs here). As a side-note, I'd suggest against returning context.log('File processing completed successfully'); (just call it, don't return it), as this is the same as returning void but leads to more confusing code.
To the second point: Calling context.done() inside of .then(() => { ... }); is calling a method to indicate that the main function from a fork'd asynchronous function. This is dangerous because theoretically, at any point after .then(() => { ... }); is defined, the main function code could be killed by that call to context.done().
To the third point: If you just call context.done() in the scope of the main function instead of returning the promise, you will have the opposite problem of point #2 (the main code will exit and stop execution of the fork'd asynchronous task before the asynchronous code has completed).
Here's an example of a simple JS function that waits on a promise and writes to a queue:
Example:
module.exports = function (context, req) {
context.log("Starting function");
// Mock Promise
let doWork = new Promise(
(resolve, reject) => {
setTimeout(resolve, 100, 'foo');
console.log("Work in promise");
}
);
doWork.then(() => {
context.log("Work AFTER promise");
context.bindings.queue1= "Queue message";
context.log("Message queued!");
}).catch(err => context.log.error(err));
// This is the correct scope to return promise
return doWork;
};

Related

How can I run watch changes in asynchronous way NodeJS Mongoose

For the past 2 days I have been struggling on this thing. I want to watch for changes on particular collection. All of the logic for these changes is in asynchronous way. I tried to make the EventEmitter callback function to be asynchronous but still works in synchronous manner.
Pseudo code example:
const someLogicFunction1 = async () => {
// some logic here
}
const someLogicFunction2 = async () => {
// some logic here
}
const collectionEventEmitter = Collection.watch();
collectionEventEmitter.on('change', async (change) => {
await someLogicFunction1();
await someLogicFunction2(); // has to wait for first function to finish
});
As I explained above I tried this approach but still runs synchronously. The second function usually has to wait for the first function to finish its task in order for the whole code to be properly working.
EDIT:
From what I have found it seems that the callback function firstly gets all this "change" parameter event objects and then executes the code. What I need actually is to execute the code for every "change" event parameter - not for all of them at once.
One solution is to make someLogicFunction1() returning a promise and calling someLogicFunction2(); in then of first function, like this:
await someLogicFunction1()
.then(function(){
someLogicFunction2();
});
You just need to modify someLogicFunction1(); like this:
someLogicFunction1( )
{
return new Promise( async function( resolve, reject ) {
// Do your stuff here
}); // eo promise
}
Don't forget to resolve() in someLogicFunction1 function.
resolve();
You can also reject() in someLogicFunction1 and catch the error when you call this function.
reject();
You can pass an argument to resolve and reject and get it in your call.

Resolve await when message arrives

I have some websocket code in JS. I have a message-handling loop like this:
socket.addEventListener('message', function (event) {
payload = JSON.parse(event.data)
method = payload.method
// Dispatch messages
if (method == 'cmd1') {
handle_cmd1(payload); // trigger event/semaphore here to wait up waiter
}
else if (method == 'cmd2') { ... }
});
And elsewhere, I have a button callback like this:
$('#my-button').change(function() {
handle_button();
});
async function handle_button() {
send_msg('msg1', 'hi');
// wait for server to reply with cmd1
cmd1_data = await something(); // what?
alert(`cmd1 data: $(cmd1_data)`);
}
The idea is that the button sends 'msg1' and the server is supposed to reply with 'cmd1' and some info. I want to wait for that reply and then do some more stuff.
So my question is how to interlock these? In C++ I'd use a semaphore. I'd rather not spin-loop; is there something in Javascript/JQuery I can use to trigger and then wait for a user-defined event like this? I'm sort of new to JS, and very new to JS async/await.
EDIT: I've made a simple jsfiddle to show what I'm after.
http://jsfiddle.net/xpvt214o/484700/
Now that I understand how promises in Javascript work, here's a working example of a promise that can get woken up from anywhere by calling a function:
wakeup = null;
// returns a promise that will be resolved by calling wakeup()
// (could be a list of these or whatever, this is just a simple demo)
function wakeable() {
return new Promise( (resolve) => {
wakeup = () => { resolve(true); }
});
}
// demo of waiting and getting woken up:
async function handle_event() {
while (true) {
console.log("waiting...")
await wakeable(); // returns to event loop here
console.log("handle event woke up!");
}
}
handle_event(); // start in "background"
wakeup(); // wake it up
setTimeout(_ => { wakeup(); }, 500); // wake it up again after a delay
What's happening here is that when you call wakeable(), it returns a promise. That promise is constructed with an anonymous function (the one taking resolve as arg); the promise constructor synchronously calls that function, passing it the promise's resolve method. In our case, the function sets wakeup to another anonymous function that calls the original resolve; it's a closure so it has access to that resolve function even when it's called later. Then it returns the new promise.
In this demo, we then await on that promise; that puts the pending promise on a queue, saves the current function state, and returns, like a generator calling yield.
A promise is resolved when its resolve function is called; in this case calling wakeup() calls the promise's internal resolve() method, which triggers any .then methods on the next tick of the Javascript event loop (using the promise queue mentioned above). Here we use await, but .then(...) would work the same way'
So there's no magic; I/O and timeout promises work the same way. They keep a private registry of functions to call when the I/O event or timeout happens, and those functions call the promise's resolve() which triggers the .then() or satisfies the await.
By the way, unlike async in python, leaving a pending promise "open" when the process exits is perfectly fine in Javascript, and in fact this demo does that. It exits when there's no more code to run; the fact that the while loop is still "awaiting" doesn't keep the process running, because it's really just some closures stored in a queue. The event loop is empty, so the process exits (assuming it's in node.js -- in a browser it just goes back to waiting for events).
something() should be a method that returns a promise() or should be another method that is also notated with async.
function something(){
return new Promise(resolve,reject) {
//... call your database
// then
resolve(theResult)
// or
reject(theError)
}
}
The async and await for the most part are really just wrappers around promises. The await returns when the promise calls resolve, and throws an exception when the promise calls reject.
Your async function can return another promise; If it returns another value, it gets turned into a resolved promise with that value.

Compose functions which return Promises

I would like to compose two functions which return Promises into a function which returns a Promise.
I really want to do this for asynchronous system calls, but to simplify things, for now I'm playing with Promises wrapped around setTimeout. Most of the system calls can be sent out and returned in any order, but in a few instances, data from one system call is needed to create a second system call.
In the enclosed example, I want to send out alerts with an interval of 5 seconds, then 3 seconds, then 10 seconds. When I run the code, things run in the wrong order.
I commented out code where I was able to run the two calls chained with ".then". The commented out code works as desired. But I haven't yet figured out how to put that into a function which returns a promise, and still have it work.
I'm interested in doing this using plain JavaScript only. After I understand how to do that, I'll look at answers which do the same using a library.
jsfiddle
// Returns a Promise.
// I purposely chose not to use the callback function inside setTimeout in order to demonstrate
// how to combine functions which return Promises, and normal functions which don't.
function sleep(delay, callbackFunction) {
return new Promise(function(resolve) {
setTimeout(resolve.bind(null, callbackFunction), delay)
});
}
// Normal function, does not return a promise.
const myPrint = (message, value) => {
alert(message + " " + value);
};
// A function combining a promise and then a normal function.
// Returns a Promise.
const sleepPrint = (delay) => {
return sleep(delay)
.then(() => {
myPrint("sleepPrint()", delay);
});
};
// A function combining two functions which return promises.
// Returns a Promise.
const sleepPrintSleepPrint = (delay1, delay2) => {
return sleepPrint(delay1)
.then(sleepPrint(delay2));
};
alert("Begin");
/*
// Run a Pomise function, then a normal function.
const delay = 4000;
sleep(delay)
.then(() => {
myPrint("init", delay);
});
*/
const firstDelay = 5000;
const secondDelay = 3000;
/*
// Call sleepPrint() which runs a Promise function then a normal function.
// After all that finishes, call another Promise function, than another normal function.
// This code works as desired, but the code at the bottom doesn't!
sleepPrint(firstDelay)
.then(() => {
return sleep(secondDelay)
})
.then(() => {
myPrint("After sleepPrint() and sleep()", secondDelay);
});
*/
// Comine two calls to sleepPrint() into a new functiion which returns a Promise, sleepPrintsleepPrint().
// After all that finishes, call another Promise function, than another normal function.
// I want this to have 5 second delay and then a 3 second delay, but they are not running in that order.
const thirdDelay = 10000;
sleepPrintSleepPrint(firstDelay, secondDelay)
.then(() => {
return sleep(thirdDelay)
})
.then(() => {
myPrint("After sleepPrintSleepPrint()", thirdDelay);
});
Remember how functions work (in any programming language). This:
const sleepPrintSleepPrint = (delay1, delay2) => {
return sleepPrint(delay1)
.then(sleepPrint(delay2));
};
Is the same as this:
const sleepPrintSleepPrint = (delay1, delay2) => {
var x = sleepPrint(delay2);
return sleepPrint(delay1)
.then(x);
};
What you are doing is calling both functions simultaneously. What I think you intend is to call the second function once then resolves. So you need to do this:
const sleepPrintSleepPrint = (delay1, delay2) => {
return sleepPrint(delay1)
.then(function(){return sleepPrint(delay2)});
};
Or if you prefer arrow function syntax:
const sleepPrintSleepPrint = (delay1, delay2) => {
return sleepPrint(delay1)
.then(() => sleepPrint(delay2));
};
#slebetman supplied the correct code, and he explained the issue and the fix correctly at a high level. This is my understanding of his answer in more detail. This should be a comment to his answer, but it's too big.
The JavaScript interpreter makes two passes through code which contains chained ".then"s.
The first pass happens right away without any waiting. The interpreter looks at the argument(s) to .then, and does something different if it sees a function or an expression. Functions are placed onto the call stack, and associated with a promise. Expressions are evaluated, and if the completion value is a function, the function is placed onto the call stack, and associated with a promise.
In the second pass, after a promise has been resolved, the associated function from the call stack is evaluated.
In production code, chained .then's should always follow a specific pattern. The pattern is that the argument to ".then" is one function which returns a second callback function. In the first pass, nothing very interesting happens, the first function is executed, and the return value is the second callback function. The second callback function is placed onto the call stack and associated with a promise. In the second pass, after the associated promise has been resolved, the second callback function is executed.
But in this example, I am breaking the pattern in order to learn and demonstrate how chained .then's work.
Below is an example which does not use the pattern, and exposes the two pass behavior.
This is talking about the two arguments to .then, the success callback function, and the failure callback function.
"If one or both arguments are omitted or are provided non-functions, then then will be missing the handler(s), but will not generate any errors."
If you give a function to ".then", then in the first pass, the function is placed on the call stack, but nothing else happens, and in the second pass, the function is executed.
But if you give an expression to ".then", then in the first pass, the expression is evaluated. If the completion value is a function, it is placed onto the call stack, and executed in the second pass. If the completion value is not a function, in the second pass, no further action is taken, and no error is generated.
First Pass (The JavaScript interpreter runs these things immediateley.)
Kick off sleepPrint(firstDelay). Let it run in the background.
Immediately go to the next ".then".
Print the first message. Put nothing on the call stack.
Immediately go to the next ".then".
Put the function () => sleepPrint(secondDelay) onto the call stack, and associate it with the preceding promise.
Immediately go to the next ".then".
Print the second message. Put nothing on the call stack.
Second Pass (The JavaScript interpreter runs each of these steps from the call stack much later, after the corresponding promise has been resolved.)
Wait until sleepPrint(firstDelay) prints the third message, and is resolved, then go to the next item on the call stack.
There's nothing on the call stack for this slot. There's no promise to resolve, so go to the next item on the call stack. No error is generated.
Execute function () => sleepPrint(secondDelay) which prints the fourth message. Wait for the promise to be resolved, then go to the next item on the call stack.
There's nothing on the call stack for this slot. There's no promise to resolve. No error is generated.
jsFiddle
// Returns a Promise.
// I purposely chose not to use the callback function inside setTimeout in order to demonstrate
// how to combine functions which return Promises, and normal functions which don't.
function sleep(delay, callbackFunction) {
return new Promise(function(resolve) {
setTimeout(resolve.bind(null, callbackFunction), delay)
});
}
// Normal function, does not return a promise.
const myPrint = (message, value) => {
alert(message + " " + value);
};
// A function combining a promise and then a normal function.
// Returns a Promise.
const sleepPrint = (delay) => {
return sleep(delay)
.then(() => {
myPrint("sleepPrint()", delay);
});
};
alert("Begin");
const firstDelay = 5000;
const secondDelay = 3000;
// This message is printed third.
sleepPrint(firstDelay)
.then(
myPrint("This message is printed first", 0)
)
.then(
// This message is printed fourth.
() => {sleepPrint(secondDelay)}
)
.then(
myPrint("This message is printed second", 0)
);

Unit testing NodeJS Promise inside a function

Im trying to unit test a function that calls a promise...
Using Mocha, Sinon. I have a functional block like this:
myfile.js:
let OuterDependecy = require('mydep');
function TestFunction(callback) {
OuterDependency.PromiseFunction().then(response => {
//some logic here
}).catch(err => {callback(err)});
inside my test i have used proxyquire to mock the outerdependecy
testfile.js
let proxyquire = require('proxyquire');
let OuterDepStub = {};
let testingFunc = proxyquire('myfile.js', {'mydep': OuterDepStub});
... then inside my testing block
let stubCallback = function() {
console.log('Stub dubadub dub'); //note...i can use sinon.spy here instead
};
beforeEach(()=>{
OuterDependency.PromiseFunction = function(arg) {
return new Promise((resolve, reject)=>{
reject('BAD');
});
};
spy = sinon.spy(stubCallback);
});
my actual test now calls the main "testfunction"
it('Catches Errors, and calls back using error', done => {
TestFunction(stubCallback);
expect(spy).to.have.been.called;
done();
});
I see the stub being called (the console log, hence why i didnt want to use sinon.spy) but the spy is saying its not called. and unit test fails.
I believe this is probably due to a race condition of sorts where the promise is resolving after my test is run... is there anyway to delay the test until my promise is resolve.
i know in in angularjs promise testing, there was a way to "tick" the promise so it resolves when you want to. possible in nodejs?
is there anyway to delay the test until my promise is resolve.
As far as I understand your issue, yes, you should only call done() after the promise is settled. In order to do that,you need two things:
1- Enforce TestFunction to return a Promise, so you can wait until it resolves:
function TestFunction(callback) {
return OuterDependency.PromiseFunction().then(response => {
//some logic here
}).catch(err => { callback(err) });
}
2- Wait to that promise to settle, then call done.
it('Catches Errors, and calls back using error', done => {
TestFunction(stubCallback).then(() => {
expect(spy).to.have.been.called;
done();
})
});
now, our then block won't run until the catch block within TestFunction, so if the test works as expected (i.e. the catch block fires and the callback gets fired), the expectation and the done calls will always fire after the callback gets called.
I see the stub being called (the console log, hence why i didnt want to use sinon.spy) but the spy is saying its not called. and unit test fails.
That's because your expectation runs right after the TestFunction calls, without waiting for it to settle. However, it will get called lately, thus the console.log appears in the next spec.

Bluebird: How to bind a promise so that I can customize behavior in .catch, and then allow error to continue cascading?

Note- see the update below
I want have a flow with a dependency between the first task to the execution between the other secondary tasks, but the other secondary tasks can all be run concurrently.
I want to write a clean flow which will make it easy to handle errors, and I've tried several variations but can't get it right.
Here's what I will be working with, regardless of the pattern I compose it:
var primaryAsyncTask = {...}; //single 'task' which has an execution function
var secondaryAsyncTasks = [{...}]; //array of tasks with same format as above
function promisifyTask(task){ .... }; //returns a promise which executes the task and appends the resolve/reject functions as the last arguments to the task execution function
Now here's the two options I currently tried:
In promisifyTask, I wrap the rejection handler and bind it to the task, so that I can customize the failure info and then check it upon final catch. In other words, the flow looks like this:
.
function taskRejecter(reject, reason) {
//do something to save info from reason onto this = task
reject();
}
function promisifyTask(task) {
return new Promise(function (resolve, reject) {
var rejecter = taskRejecter.bind(task, reject);
task.execute.apply(null, task.args.concat([resolve, rejecter]));
});
}
//and then when executing:
promisifyTask(primaryAsyncTask)
.then(function () {
return Promise.settle(secondaryAsyncTasks.map(function (task) {
return promisifyTask(task);
}));
})
.then(function onSuccess() {...
})
.catch(function onFail() {
//map all the info from the failed tasks (primary/secondary) to the info saved in it by the taskRejecter.
});
The advantage here is that if the primary task fails, it doesn't execute the secondary tasks and reaches the onFail in the catch... but if it succeeds, it executes them and will reach the onFail only if one of them fails (which is also desired behavior).
Alternatively, which looks much nicer internally, is to bind the promise and catch it, instead of wrapping the rejection handler, and then I can handle everything in a single 'taskFailCatcher' function, so it would look like this:
function onTaskFail(reason){
//do something to save info from reason onto this = task, since it was bound before the catch
}
function promisifyTask(task){
return new Promise(function(resolve, reject) {
task.execute.apply(null, task.args.concat([resolve, reject]));
});
}
function promisifyTaskAndCatch(task){
return promisifyTask(task).bind(task).catch(onTaskFail)
}
//and then when executing:
promisifyTask(primaryAsyncTask)
.then(function (){
return Promise.settle(secondaryAsyncTasks.map(function (task) {
return promisifyTaskAndCatch(task);
}));
})
.then(function onSuccess(){...})
.catch(function onFail(){...})
I like the .bind.catch, but the problem here is twofold:
The flow is not consistent. I don't want to execute the secondary tasks when the primary fails, so I use promisifyTask for the primary (so that it is not caught, and reaches the catch at the end), but I promisifyTaskAndCatch inside the .settle, so that I can easily bind and edit the failure info for the task directly after the rejection.
The .catch here is reached only after the primary fails. Now that I catch all the secondary tasks, the .settle will always receive fulfilled promises, so I reach onSuccess even if a secondary task fails.
How can I edit this flow so that I make use of the best of both worlds (.bind.catch for error handling, with a consistent and clear flow)?
-----------------UPDATE------------------
I almost figured this out. I changed it by removing promisifyTaskAndCatch and changing promisifyTask to be:
function promisifyTask(task) {
var promise = new Promise(function (resolve, reject) {
task.execute.apply(null, task.args.concat([resolve, reject]));
});
promise.bind(task).catch(onTaskFail);
return promise;
}
Now the syntax is consistent and the error cascades through, and I also get my error reporting.
The only problem is now that I don't have a guarantee that the internal catch for the task will happen before the external final catch (with the onFail at the end of the chain), since this is async.
Is there a way to return the caught promise, but still have it fail? can I do this without just rethrowing an error inside of .catch ?
The only problem is now that I don't have a guarantee that the internal catch for the task will happen before the external final catch (with the onFail at the end of the chain), since this is async
Actually you do have, since the .catch(onTaskFail) is invoked first the onTaskFail would be executed before the final one. But you are right, if the final catch depends on things that onTaskFail does then it would be much cleaner to actually return a promise for that result.
Is there a way to return the caught promise, but still have it fail?
I think re-throwing the reason would be the best.
Or you even don't have them fail, and use Promise.all and inspect the task objects about their results instead of using Promise.settle and its PromiseInspection values. As you say the reason is saved on the tasks, the most consistent flow would be to return the task objects:
function onTaskFail(reason){
this.error = reason; // or so
return this;
}
runTask(primaryAsyncTask).then(function(primResult) {
return Promise.all(secondaryAsyncTasks.map(function (task) {
return runTask(task).bind(task).catch(onTaskFail);
}));
}).then(function(taskResults) {
// test for errors in the secondary results
// and throw them if you want the global onFail
// else do onSuccess()
}).catch(onFail);
can I do this without just rethrowing an error inside of .catch ?
Yes, you can also return a rejected promise. That might be a Promise.reject(reason), but simpler might be to return promise that is already there (though not currently in scope of onTaskFail). The .return() helper method could be used here though:
…
return promise.bind(task).catch(onTaskFail).return(promise);

Categories