I want to fire and forget in a nodejs durable orchestrator
a call to an activity
yield context.df.callActivity("Activity1", "name")
but I noticed that the orchestrator waits for this to complete I want to fire the activity and forget about it and proceed how can I achieve that in nodejs?
You can just remove the yield from your statement to kick off the activity call and continue on without waiting for it to complete. An orchestrator only blocks when it encounters a yield. This is essentially what's being done in the fan-out/fan-in pattern, described here.
const df = require("durable-functions");
module.exports = df.orchestrator(function*(context) {
const parallelTasks = [];
// Get a list of N work items to process in parallel.
const workBatch = yield context.df.callActivity("F1");
for (let i = 0; i < workBatch.length; i++) {
parallelTasks.push(context.df.callActivity("F2", workBatch[i]));
}
yield context.df.Task.all(parallelTasks);
// Aggregate all N outputs and send the result to F3.
const sum = parallelTasks.reduce((prev, curr) => prev + curr, 0);
yield context.df.callActivity("F3", sum);
});
In the example above, the invocations of context.df.callActivity("F2", ...) are being done in a "fire-and-forget" way, where we're adding the tasks to a list instead of yielding them. The example only waits for them after the for loop in the call to context.df.Task.all(...).
You can apply this pattern to have the orchestrator continue to take more steps before waiting for activities to complete. Note, however, that if you never eventually yield the tasks, then the orchestrator may complete but remain in a "Running" state. It will only transition into a "Completed" state after all the fire-and-forget tasks have completed.
Related
I wrote a simple function that returns Promise so should be non-blocking (in my opinion). Unfortunately, the program looks like it stops waiting for the Promise to finish. I am not sure what can be wrong here.
function longRunningFunc(val, mod) {
return new Promise((resolve, reject) => {
sum = 0;
for (var i = 0; i < 100000; i++) {
for (var j = 0; j < val; j++) {
sum += i + j % mod
}
}
resolve(sum)
})
}
console.log("before")
longRunningFunc(1000, 3).then((res) => {
console.log("Result: " + res)
})
console.log("after")
The output looks like expected:
before // delay before printing below lines
after
Result: 5000049900000
But the program waits before printing second and third lines. Can you explain what should be the proper way to get "before" and "after" printed first and then (after some time) the result?
Wrapping code in a promise (like you've done) does not make it non-blocking. The Promise executor function (the callback you pass to new Promise(fn) is called synchronously and will block which is why you see the delay in getting output.
In fact, there is no way to create your own plain Javascript code (like what you have) that is non-blocking except putting it into a child process, using a WorkerThread, using some third party library that creates new threads of Javascript or using the new experimental node.js APIs for threads. Regular node.js runs your Javascript as blocking and single threaded, whether it's wrapped in a promise or not.
You can use things like setTimeout() to change "when" your code runs, but whenever it runs, it will still be blocking (once it starts executing nothing else can run until it's done). Asynchronous operations in the node.js library all use some form of underlying native code that allows them to be asynchronous (or they just use other node.js asynchronous APIs that themselves use native code implementations).
But the program waits before printing second and third lines. Can you explain what should be the proper way to get "before" and "after" printed first and then (after some time) the result?
As I said above, wrapping things in promise executor function doesn't make them asynchronous. If you want to "shift" the timing of when things run (thought they are still synchronous), you can use a setTimeout(), but that's not really making anything non-blocking, it just makes it run later (still blocking when it runs).
So, you could do this:
function longRunningFunc(val, mod) {
return new Promise((resolve, reject) => {
setTimeout(() => {
sum = 0;
for (var i = 0; i < 100000; i++) {
for (var j = 0; j < val; j++) {
sum += i + j % mod
}
}
resolve(sum)
}, 10);
})
}
That would reschedule the time consuming for loop to run later and might "appear" to be non-blocking, but it actually still blocks - it just runs later. To make it truly non-blocking, you'd have to use one of the techniques mentioned earlier to get it out of the main Javascript thread.
Ways to create actual non-blocking code in node.js:
Run it in a separate child process and get an asynchronous notification when it's done.
Use the new experimental Worker Threads in node.js v11
Write your own native code add-on to node.js and use libuv threads or OS level threads in your implementation (or other OS level asynchronous tools).
Build on top of previously existing asynchronous APIs and have none of your own code that takes very long in the main thread.
The executor function of a promise is run synchronously, and this is why your code blocks the main thread of execution.
In order to not block the main thread of execution, you need to periodically and cooperatively yield control while the long running task is performed. In effect, you need to split the task into subtasks, and then coordinate the running of subtasks on new ticks of the event loop. In this way you give other tasks (like rendering and responding to user input) the opportunity to run.
You can either write your own async loop using the promise API, or you can use an async function. Async functions enable the suspension and resumation of functions (reentrancy) and hide most of the complexity from you.
The following code uses setTimeout to move subtasks onto new event loop ticks. Of course, this could be generalised, and batching could be used to find a balance between progress through the task and UI responsiveness; the batch size in this solution is only 1, and so progress is slow.
Finally: the real solution to this kind of problem is probably a Worker.
const $ = document.querySelector.bind(document)
const BIG_NUMBER = 1000
let count = 0
// Note that this could also use requestIdleCallback or requestAnimationFrame
const tick = (fn) => new Promise((resolve) => setTimeout(() => resolve(fn), 5))
async function longRunningTask(){
while (count++ < BIG_NUMBER) await tick()
console.log(`A big number of loops done.`)
}
console.log(`*** STARTING ***`)
longRunningTask().then(() => console.log(`*** COMPLETED ***`))
$('button').onclick = () => $('#output').innerHTML += `Current count is: ${count}<br/>`
* {
font-size: 16pt;
color: gray;
padding: 15px;
}
<button>Click me to see that the UI is still responsive.</button>
<div id="output"></div>
I wrote a simple function that returns Promise so should be non-blocking (in my opinion). Unfortunately, the program looks like it stops waiting for the Promise to finish. I am not sure what can be wrong here.
function longRunningFunc(val, mod) {
return new Promise((resolve, reject) => {
sum = 0;
for (var i = 0; i < 100000; i++) {
for (var j = 0; j < val; j++) {
sum += i + j % mod
}
}
resolve(sum)
})
}
console.log("before")
longRunningFunc(1000, 3).then((res) => {
console.log("Result: " + res)
})
console.log("after")
The output looks like expected:
before // delay before printing below lines
after
Result: 5000049900000
But the program waits before printing second and third lines. Can you explain what should be the proper way to get "before" and "after" printed first and then (after some time) the result?
Wrapping code in a promise (like you've done) does not make it non-blocking. The Promise executor function (the callback you pass to new Promise(fn) is called synchronously and will block which is why you see the delay in getting output.
In fact, there is no way to create your own plain Javascript code (like what you have) that is non-blocking except putting it into a child process, using a WorkerThread, using some third party library that creates new threads of Javascript or using the new experimental node.js APIs for threads. Regular node.js runs your Javascript as blocking and single threaded, whether it's wrapped in a promise or not.
You can use things like setTimeout() to change "when" your code runs, but whenever it runs, it will still be blocking (once it starts executing nothing else can run until it's done). Asynchronous operations in the node.js library all use some form of underlying native code that allows them to be asynchronous (or they just use other node.js asynchronous APIs that themselves use native code implementations).
But the program waits before printing second and third lines. Can you explain what should be the proper way to get "before" and "after" printed first and then (after some time) the result?
As I said above, wrapping things in promise executor function doesn't make them asynchronous. If you want to "shift" the timing of when things run (thought they are still synchronous), you can use a setTimeout(), but that's not really making anything non-blocking, it just makes it run later (still blocking when it runs).
So, you could do this:
function longRunningFunc(val, mod) {
return new Promise((resolve, reject) => {
setTimeout(() => {
sum = 0;
for (var i = 0; i < 100000; i++) {
for (var j = 0; j < val; j++) {
sum += i + j % mod
}
}
resolve(sum)
}, 10);
})
}
That would reschedule the time consuming for loop to run later and might "appear" to be non-blocking, but it actually still blocks - it just runs later. To make it truly non-blocking, you'd have to use one of the techniques mentioned earlier to get it out of the main Javascript thread.
Ways to create actual non-blocking code in node.js:
Run it in a separate child process and get an asynchronous notification when it's done.
Use the new experimental Worker Threads in node.js v11
Write your own native code add-on to node.js and use libuv threads or OS level threads in your implementation (or other OS level asynchronous tools).
Build on top of previously existing asynchronous APIs and have none of your own code that takes very long in the main thread.
The executor function of a promise is run synchronously, and this is why your code blocks the main thread of execution.
In order to not block the main thread of execution, you need to periodically and cooperatively yield control while the long running task is performed. In effect, you need to split the task into subtasks, and then coordinate the running of subtasks on new ticks of the event loop. In this way you give other tasks (like rendering and responding to user input) the opportunity to run.
You can either write your own async loop using the promise API, or you can use an async function. Async functions enable the suspension and resumation of functions (reentrancy) and hide most of the complexity from you.
The following code uses setTimeout to move subtasks onto new event loop ticks. Of course, this could be generalised, and batching could be used to find a balance between progress through the task and UI responsiveness; the batch size in this solution is only 1, and so progress is slow.
Finally: the real solution to this kind of problem is probably a Worker.
const $ = document.querySelector.bind(document)
const BIG_NUMBER = 1000
let count = 0
// Note that this could also use requestIdleCallback or requestAnimationFrame
const tick = (fn) => new Promise((resolve) => setTimeout(() => resolve(fn), 5))
async function longRunningTask(){
while (count++ < BIG_NUMBER) await tick()
console.log(`A big number of loops done.`)
}
console.log(`*** STARTING ***`)
longRunningTask().then(() => console.log(`*** COMPLETED ***`))
$('button').onclick = () => $('#output').innerHTML += `Current count is: ${count}<br/>`
* {
font-size: 16pt;
color: gray;
padding: 15px;
}
<button>Click me to see that the UI is still responsive.</button>
<div id="output"></div>
I wrote a simple function that returns Promise so should be non-blocking (in my opinion). Unfortunately, the program looks like it stops waiting for the Promise to finish. I am not sure what can be wrong here.
function longRunningFunc(val, mod) {
return new Promise((resolve, reject) => {
sum = 0;
for (var i = 0; i < 100000; i++) {
for (var j = 0; j < val; j++) {
sum += i + j % mod
}
}
resolve(sum)
})
}
console.log("before")
longRunningFunc(1000, 3).then((res) => {
console.log("Result: " + res)
})
console.log("after")
The output looks like expected:
before // delay before printing below lines
after
Result: 5000049900000
But the program waits before printing second and third lines. Can you explain what should be the proper way to get "before" and "after" printed first and then (after some time) the result?
Wrapping code in a promise (like you've done) does not make it non-blocking. The Promise executor function (the callback you pass to new Promise(fn) is called synchronously and will block which is why you see the delay in getting output.
In fact, there is no way to create your own plain Javascript code (like what you have) that is non-blocking except putting it into a child process, using a WorkerThread, using some third party library that creates new threads of Javascript or using the new experimental node.js APIs for threads. Regular node.js runs your Javascript as blocking and single threaded, whether it's wrapped in a promise or not.
You can use things like setTimeout() to change "when" your code runs, but whenever it runs, it will still be blocking (once it starts executing nothing else can run until it's done). Asynchronous operations in the node.js library all use some form of underlying native code that allows them to be asynchronous (or they just use other node.js asynchronous APIs that themselves use native code implementations).
But the program waits before printing second and third lines. Can you explain what should be the proper way to get "before" and "after" printed first and then (after some time) the result?
As I said above, wrapping things in promise executor function doesn't make them asynchronous. If you want to "shift" the timing of when things run (thought they are still synchronous), you can use a setTimeout(), but that's not really making anything non-blocking, it just makes it run later (still blocking when it runs).
So, you could do this:
function longRunningFunc(val, mod) {
return new Promise((resolve, reject) => {
setTimeout(() => {
sum = 0;
for (var i = 0; i < 100000; i++) {
for (var j = 0; j < val; j++) {
sum += i + j % mod
}
}
resolve(sum)
}, 10);
})
}
That would reschedule the time consuming for loop to run later and might "appear" to be non-blocking, but it actually still blocks - it just runs later. To make it truly non-blocking, you'd have to use one of the techniques mentioned earlier to get it out of the main Javascript thread.
Ways to create actual non-blocking code in node.js:
Run it in a separate child process and get an asynchronous notification when it's done.
Use the new experimental Worker Threads in node.js v11
Write your own native code add-on to node.js and use libuv threads or OS level threads in your implementation (or other OS level asynchronous tools).
Build on top of previously existing asynchronous APIs and have none of your own code that takes very long in the main thread.
The executor function of a promise is run synchronously, and this is why your code blocks the main thread of execution.
In order to not block the main thread of execution, you need to periodically and cooperatively yield control while the long running task is performed. In effect, you need to split the task into subtasks, and then coordinate the running of subtasks on new ticks of the event loop. In this way you give other tasks (like rendering and responding to user input) the opportunity to run.
You can either write your own async loop using the promise API, or you can use an async function. Async functions enable the suspension and resumation of functions (reentrancy) and hide most of the complexity from you.
The following code uses setTimeout to move subtasks onto new event loop ticks. Of course, this could be generalised, and batching could be used to find a balance between progress through the task and UI responsiveness; the batch size in this solution is only 1, and so progress is slow.
Finally: the real solution to this kind of problem is probably a Worker.
const $ = document.querySelector.bind(document)
const BIG_NUMBER = 1000
let count = 0
// Note that this could also use requestIdleCallback or requestAnimationFrame
const tick = (fn) => new Promise((resolve) => setTimeout(() => resolve(fn), 5))
async function longRunningTask(){
while (count++ < BIG_NUMBER) await tick()
console.log(`A big number of loops done.`)
}
console.log(`*** STARTING ***`)
longRunningTask().then(() => console.log(`*** COMPLETED ***`))
$('button').onclick = () => $('#output').innerHTML += `Current count is: ${count}<br/>`
* {
font-size: 16pt;
color: gray;
padding: 15px;
}
<button>Click me to see that the UI is still responsive.</button>
<div id="output"></div>
I wrote two recursive functions that sum numbers from a array. They do the same thing, one asynchronously and the other synchronously. The async function took about 9x the time the sync one did.
Shouldn't the async function take advantage from the fact of running more tasks at the same time?
The functions
// Asynchronously sum the numbers in array
async function sumAsync(arr){
if(arr.length == 1) return arr[0];
const half = arr.length/2;
// Numbers on the left half
const left = arr.filter((e, i) => {
return i < half;
});
// Numbers on the right half
const right = arr.filter((e, i) => {
return i >= half;
});
// Recursive call
const leftR = sumAsync(left);
const rightR = sumAsync(right);
// Wait for resolves
await Promise.all([leftR, rightR]);
return await leftR + await rightR;
}
// Synchronously sum the numbers in array
function sumSync(arr){
if(arr.length == 1) return arr[0];
const half = arr.length/2;
// Numbers on the left half
const left = arr.filter((e, i) => {
return i < half;
});
// Numbers on the right half
const right = arr.filter((e, i) => {
return i >= half;
});
// Recursive call
const leftR = sumSync(left);
const rightR = sumSync(right);
return leftR + rightR;
}
Testing them
(async () => {
const LENGTH = 1048576; // 1024^2
const arr = Array.from(Array(LENGTH), num => Math.random()*10 | 0);
// arr[1048576] <- random (0 ~ 9)
// Async sum
console.log('ASYNC');
before = Date.now();
console.log(`SUM: ${await sumAsync(arr)}`);
after = Date.now();
console.log(`TIME: ${after - before} ms`);
// Sync sum
console.log('SYNC');
before = Date.now();
console.log(`SUM: ${sumSync(arr)}`);
after = Date.now();
console.log(`TIME: ${after - before} ms`);
})();
Results
// ASYNC
// SUM: 4720832
// TIME: 5554 ms
// SYNC
// SUM: 4720832
// TIME: 613 ms
The return value of an async function is always a Promise, even if the function only carries out synchronous operations, and the await (or .then) of a Promise will only run what follows during a microtask (after the current synchronous code is finished running). With a large array, that'll result in a lot of unnecessary microtasks wrapping synchronous code.
When nothing actual asynchronous is going on, this is just extra baggage, and results in additional processing time and power required.
Shouldn't the async function take advantage from the fact of running more tasks at the same time?
Javascript is single-threaded, even with async functions. If multiple async functions are called at once, only one path through the code may be "active" at any one time. If the total processing time required for all tasks is, say, 1000 ms, in standard Javascript, there's no way around spending at least 1000 ms.
You're not actually running more tasks at the same time - you're just wrapping the tasks in Promises (unnecessarily), while doing the same work.
For truly parallel actions, you'll have to use something provided by your current environment, such as child_process in Node, or a web worker.
Short version: async doesn't do more than one thing at a time. It switches between tasks (with overhead for each switch) in a queue, and when one task blocks, it hands control off to another (with overhead for the switch, and requeueing the blocked task when it unblocks).
Long version: Async doesn't mean parallel processing, it means interleaved (concurrent, cooperative) processing. JavaScript is still single-threaded even with async usage, and all of the actual work you do is purely CPU bound. In fact, your only real concurrency is that the async code will be scheduling, pausing and resuming your recursive calls repeatedly (but still only doing work for one at a time), while the sync code will just do them in order as fast as possible, with no event loop involvement.
The benefit of async code is that when blocking I/O (including stuff like waiting on user input) is being performed, that task can be suspended until it's unblocked by some out-of-band signal (I/O done, user clicked mouse, whatever), and other tasks can run. The benefit is in reaping the benefits of concurrent (but not parallel) processing in situations where most tasks, most of the time, are waiting for something, so the few that are ready to run can begin running immediately (and since they're usually not running, the overhead of task switching doesn't matter much; most of the time, there's nothing to switch to, so much of the overhead is paid when you've got nothing better to do). But it's definitely higher overhead than just doing number-crunching without a pause.
I've got a rate limiter for an API I am using which allows 20 requests per second. All requests are promise based and the promise will be resolved with the API data once there is a response.
The problem:
I setup a promiseArray which contains 58k promises all waiting for a response. So slowly the memory is increasing until I am running out of memory. In my specific situation I don't need to pass the resolved data to my then() and the data is eating up all my RAM.
The code:
}).then(() => {
// 2. Crawl for all clanprofiles from these leaderboards
const promiseArray = []
for (let i = 0; i < clanTags.length; i++) {
// Resolved data from getClanProfile() is eating up all my RAM
const p = backgroundScheduler.getClanProfile(clanTags[i], true)
promiseArray.push(p)
}
return Promise.all(promiseArray)
}).then(() => {
So is there a way to await until the promiseArray is resolved without needing the resolved data?
You will use a lesser amount of memory if you don't ever have 58k promises, their associated async operations and their result data active at once.
Instead you want to run X operations at once and then when one finishes, you start the next one with never more than X in flight at the same time and never more than X promises in use at once.
You can experiment with an appropriate value of X. A value of 1 is sequential operations but you can often improve overall end-to-end operation time by using some higher value of X. If all requests are hitting the same host, then X is probably no more than 5-10 (since a given host can't really do a lots of things at once and asking it to do more than it can do at once just slows it down).
If every request is to a different host, then you may be able to make X higher. Experimentation would give you an optimal value for both peak memory usage and overall throughput and somewhat depends upon your specific circumstances.
Bluebird's Promise.map() has a concurrency option that will do this for you, but there are also numerous ways to code for only X in flight at the same time.
Here are some other coding examples of managing how many are in flight at a time:
Make several requests to an API that can only handle 20 request a minute
How to execute promises in series?
unable to complete promises due to out of memory
Fire off 1,000,000 requests 100 at a time
How to make it so that I can execute say 10 promises at a time in javascript to prevent rate limits on api calls?
If you don't need the resolved data, you can allow it to be GCed sooner by replacing it like this:
const p = backgroundScheduler.getClanProfile(clanTags[i], true).then(data => {
return 0; // make resolved value just be a simple number
// so other data is now eligible for GC
});
promiseArray.push(p)
And, here's a simple implementation that iterates an array with no more than X requests in flight at the same time:
// takes an array of items and a function that returns a promise
// runs no more than maxConcurrent requests at once
function mapConcurrent(items, maxConcurrent, fn) {
let index = 0;
let inFlightCntr = 0;
let doneCntr = 0;
let results = new Array(items.length);
let stop = false;
return new Promise(function(resolve, reject) {
function runNext() {
let i = index;
++inFlightCntr;
fn(items[index], index++).then(function(val) {
++doneCntr;
--inFlightCntr;
results[i] = val;
run();
}, function(err) {
// set flag so we don't launch any more requests
stop = true;
reject(err);
});
}
function run() {
// launch as many as we're allowed to
while (!stop && inFlightCntr < maxConcurrent && index < items.length) {
runNext();
}
// if all are done, then resolve parent promise with results
if (doneCntr === items.length) {
resolve(results);
}
}
run();
});
}