The inconvenience I have is that there is no way, to my knowledge, using built-in Promise, to resolve a Promise within a synchronous block.
Specifically, the following code
var o = {};
function set(index, value) {
console.log(`setting ${index} to ${o.index = value}`);
}
Promise.resolve().then(()=>set('a', 1)).then(()=>set('b',2))
Promise.resolve().then(()=>set('c', 1)).then(()=>set('d',2))
console.log(`a is ${o.a}`);
Outputs :
a is undefined
setting a to 1
setting c to 1
setting b to 2
setting d to 2
Are there Promise like lib whose Promise would yield the following outputs:
setting a to 1
setting b to 2
setting c to 1
setting d to 2
a is 1
Obviously, no need to resort to Promises in the example above, but in my use case, setting 'a' leads to a pretreatment which only needs to be done once, whereas 'b' can be set multiple times, and is used within the same synchronous block it is set (no freedom on that).
I've looked up bluebird, and I have no clue how their promises work internally, but they seem to embrace using only promise chains.
Q Promise might be what I'm looking for, I'm not sure.
More generally, it feels daunting scouring the docs of the many Promise librairies out there querying some niche, depreciated need with likely the wrong keywords and concepts in mind.
This is definitely a case of ProblemXY where X could be solved in a variety of ways without Promises, for example by synchronous initializers, but I feel problem Y is interesting in and of itself (and likely a duplicate), so I'm still asking.
I'll get around giving problem X its own question, but long story short, I'd love to use, sometimes within a single js synchronous execution block, the expressive power of promises to pull values instead of pushing them and to flatten callbacks and try-catch pyramids.
Edit:
To clarify what I'm curious about, the way I would build the Promiselike with the desired properties is as follow :
let f = myPasync(function*(){ //equivalent to async function declaration
v = yield myPAwait(myPromiseInstance); //equivalent of v = await myPromiseInstance;
})
let pa = new myP( *(resolve, reject)=>{} ) //equivalent to new Promise((resolve, reject)=>{}) ; for simplicity, resolve and reject can be naked properties of pa
pa.myPthen((*()=>{})()) //equivalent to pa.then((v)=>{})
And then, instead of dynamically iterating over a task queue every tick, pushing thens and awaits of newly resolved promised to that queue in a FIFO manner,
I would push to the task queue in a LIFO manner and iterate over it upon any myP related function call.
So for example, when a promise is resolved, every routine awaiting it will resume before anything else, and in particular before the rest of the routine resolving the promise.
Other example when a routine awaits a promise, if that promise is already resolved, the routine is immediately resumed instead of letting other execute.
Related
Question:
new Promise((resolve) => {
console.log(1)
resolve()
}).then(async () => {
console.log(2)
}).then(async () => {
console.log(3)
})
new Promise((resolve) => {
console.log('a')
resolve()
}).then(() => {
console.log('b')
}).then(() => {
console.log('c')
}).then(() => {
console.log('d')
}).then(() => {
console.log('e')
})
Answer:
1
a
2
b
c
d
3
e
But I think answer is.... 1 a 2 b 3 c d e
Please help to share how to think to get answer detailed. Thanks.
First I would explain that nobody should ever rely on precise timing between two separate promise chains. They run independently of one another and as soon as you insert any real world asynchronous operations in either of those promise chains (which all real-world programming would contain), then the timing of each chain is entirely unpredictable vs. the other.
Instead the timing in real code depends upon the timing of the asynchronous operations, not on anything else. Imagine that each of the steps in this promise chain was reading a file or doing some random delay. Those are entirely unpredictable operations so which chain does what first depends upon the timing of the actual asynchronous operations, not on what is shown in this example.
So, trying to dissect the details of exactly when each item goes into the promise job queue and when it gets serviced is a complete waste of programming time and not useful in any real programming problem.
Further, if you really need a specific ordering of operations, then you don't program with two independent promise chains. Instead, you use promise flow-of-control tools (like chaining everything into one chain or using Promise.all(), Promise.race(), etc...) to guide the execution/completion order into exactly what you want it to be regardless of the detailed inner workings of the promise implementation.
Then, I would explain the basics of how the promise queue works by walking through the first two links of one of the promise chains just to show that I understand how a promise gets resolved, gets added to the promise queue and then, when control is about to return to the event loop, the oldest item in the promise queue gets to run and call its .then() or .catch() handlers. This is just to illustrated that you understand the basics of how promises are scheduled and that they get serviced in LIFO order from their own job queue and before most other things in the event loop.
Then, I would explain that a few years ago, the spec was changed for some promise steps in the interest of improving performance and a JS engine before or after that spec change would likely generate different results for something like then. Yet another reason why you shouldn't rely on that level of implementation detail.
If the interviewer insisted on me trying to explain the precise order in this problem and wouldn't listen to my reasoning why that's a pointless exercise, even after hearing my explanation for how I'd code a real world situation where execution order does matter, then unless this was just some really junior interviewer trying to be clever (and outsmarting themselves), I'd have to conclude this is not a good place to work. No senior developer should insist that this is a super valuable or practical exercise beyond showing that you have a basic understanding of how the promise job queue works.
In this answer, a promise chain is built recursively.
Simplified slightly, we have :
function foo() {
function doo() {
// always return a promise
if (/* more to do */) {
return doSomethingAsync().then(doo);
} else {
return Promise.resolve();
}
}
return doo(); // returns a promise
}
Presumably this would give rise to a call stack and a promise chain - ie "deep" and "wide".
I would anticipate a memory spike larger than either performing a recursion or building a promise chain alone.
Is this so?
Has anyone considered the memory issues of building a chain in this way?
Would memory consumption differ between promise libs?
a call stack and a promise chain - ie "deep" and "wide".
Actually, no. There is no promise chain here as we know it from doSomeThingAsynchronous.then(doSomethingAsynchronous).then(doSomethingAsynchronous).… (which is what Promise.each or Promise.reduce might do to sequentially execute handlers if it was written this way).
What we are facing here is a resolve chain1 - what happens in the end, when the base case of the recursion is met, is something like Promise.resolve(Promise.resolve(Promise.resolve(…))). This is only "deep", not "wide", if you want to call it that.
I would anticipate a memory spike larger than either performing a recursion or building a promise chain alone.
Not a spike actually. You'd slowly, over time, build a bulk of promises that are resolved with the innermost one, all representing the same result. When, at the end of your task, the condition is fulfilled and the innermost promise resolved with an actual value, all of these promises should be resolved with the same value. That would end up with O(n) cost for walking up the resolve chain (if implemented naively, this might even be done recursively and cause a stack overflow). After that, all the promises except for the outermost can become garbage collected.
In contrast, a promise chain that is built by something like
[…].reduce(function(prev, val) {
// successive execution of fn for all vals in array
return prev.then(() => fn(val));
}, Promise.resolve())
would show a spike, allocating n promise objects at the same time, and then slowly resolve them one by one, garbage-collecting the previous ones until only the settled end promise is alive.
memory
^ resolve promise "then" (tail)
| chain chain recursion
| /| |\
| / | | \
| / | | \
| ___/ |___ ___| \___ ___________
|
+----------------------------------------------> time
Is this so?
Not necessarily. As said above, all the promises in that bulk are in the end resolved with the same value2, so all we would need is to store the outermost and the innermost promise at one time. All intermediate promises may become garbage-collected as soon as possible, and we want to run this recursion in constant space and time.
In fact, this recursive construct is totally necessary for asynchronous loops with a dynamic condition (no fixed number of steps), you cannot really avoid it. In Haskell, where this is used all the time for the IO monad, an optimisation for it is implemented just because of this case. It is very similar to tail call recursion, which is routinely eliminated by compilers.
Has anyone considered the memory issues of building a chain in this way?
Yes. This was discussed at promises/aplus for example, though with no outcome yet.
Many promise libraries do support iteration helpers to avoid the spike of promise then chains, like Bluebird's each and map methods.
My own promise library3,4 does feature resolve chains without introducing memory or runtime overhead. When one promise adopts another (even if still pending), they become indistinguishable, and intermediate promises are no longer referenced anywhere.
Would memory consumption differ between promise libs?
Yes. While this case can be optimised, it seldom is. Specifically, the ES6 spec does require Promises to inspect the value at every resolve call, so collapsing the chain is not possible. The promises in the chain might even be resolved with different values (by constructing an example object that abuses getters, not in real life). The issue was raised on esdiscuss but remains unresolved.
So if you use a leaking implementation, but need asynchronous recursion, then you better switch back to callbacks and use the deferred antipattern to propagate the innermost promise result to a single result promise.
[1]: no official terminology
[2]: well, they are resolved with each other. But we want to resolve them with the same value, we expect that
[3]: undocumented playground, passes aplus. Read the code at your own peril: https://github.com/bergus/F-Promise
[4]: also implemented for Creed in this pull request
Disclaimer: premature optimization is bad, the real way to find out about performance differences is to benchmark your code, and you shouldn't worry about this (I've only had to once and I've used promises for at least 100 projects).
Is this so?
Yes, the promises would have to "remember" what they're following, if you do this for 10000 promises you'd have a 10000 long promise chain, if you don't then you won't (for example, with recursion) - this is true for any queueing flow control.
If you have to keep track of 10000 extra things (the operations) then you need to keep memory for it and that takes time, if that number is a million it might not be viable. This varies among libraries.
Has anyone considered the memory issues of building a chain in this way?
Of course, this is a big issue, and a use case for using something like Promise.each in libraries like bluebird over thenable chaining.
I've personally had in my code to avoid this style for a quick app that traverses all the files in a VM once - but in the vast majority of cases it's a non issue.
Would memory consumption differ between promise libs?
Yes, greatly. For example bluebird 3.0 will not allocate an extra queue if it detects a promise operation is already asynchronous (for example if it starts with a Promise.delay) and will just execute things synchronously (because the async guarantees are already preserved).
This means that what I claimed in my answer to the first question isn't always true (but is true in the regular use case) :) Native promises will never be able to do this unless internal support is provided.
Then again - it's no surprise since promise libraries differ by orders of magnitude from one another.
I just came out a hack that may help solving the problem: don't do recursion in the last then, rather, do it in the last catch, since catch is out of the resolve chain. Using your example, it would be like this:
function foo() {
function doo() {
// always return a promise
if (/* more to do */) {
return doSomethingAsync().then(function(){
throw "next";
}).catch(function(err) {
if (err == "next") doo();
})
} else {
return Promise.resolve();
}
}
return doo(); // returns a promise
}
To complement the awesome existing answers I'd like to illustrate the expression, which is the result of such an asynchronous recursion. For the sake of simplicity I use a simple function that computes the power of a given base and exponent. The recursive and base case are equivalent to those of the OP's example:
const powerp = (base, exp) => exp === 0
? Promise.resolve(1)
: new Promise(res => setTimeout(res, 0, exp)).then(
exp => power(base, exp - 1).then(x => x * base)
);
powerp(2, 8); // Promise {...[[PromiseValue]]: 256}
With the help of some substitution steps the recursive portion can be replaced. Please note that this expression can be evaluated in your browser:
// apply powerp with 2 and 8 and substitute the recursive case:
8 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 8)).then(
res => 7 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 7)).then(
res => 6 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 6)).then(
res => 5 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 5)).then(
res => 4 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 4)).then(
res => 3 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 3)).then(
res => 2 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 2)).then(
res => 1 === 0 ? Promise.resolve(1) : new Promise(res => setTimeout(res, 0, 1)).then(
res => Promise.resolve(1)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2)
).then(x => x * 2); // Promise {...[[PromiseValue]]: 256}
Interpretation:
With new Promise(res => setTimeout(res, 0, 8)) the executor is invoked immediately and executes a non-bllocking computation (mimicked with setTimeout). Then an unsettled Promise is returned. This is equivalent with doSomethingAsync() of the OP's example.
A resolve callback is associated with this Promise via .then(.... Note: The body of this callback was substituted with the body of powerp.
Point 2) is repeated and a nested then handler structure is build up until the base case of the recursion is reached. The base case returns a Promise resolved with 1.
The nested then handler structure is "unwound" by calling the associated callback correspondingly.
Why is the generated structure nested and not chained? Because the recursive case within the then handlers prevents them from returning a value until the base case is reached.
How can this work without a stack? The associated callbacks form a "chain", which bridges the successive microtasks of the main event loop.
This promise pattern will generate a recursive chain. So, each resolve() will create a new stack frame (with its own data), utilizing some memory. This means that large number of chained functions using this promise pattern can produce stack overflow errors.
To illustrate this, I'll use a tiny promise library called Sequence, which I've written. It relies on recursion to achieve sequential execution for chained functions:
var funcA = function() {
setTimeout(function() {console.log("funcA")}, 2000);
};
var funcB = function() {
setTimeout(function() {console.log("funcB")}, 1000);
};
sequence().chain(funcA).chain(funcB).execute();
Sequence works great for small/medium sized chains, in the range of 0-500 functions. However, at about 600 chains Sequence starts degradating and generating often stack overflow errors.
The bottom line is: currently, recursion-based promise libraries are more suitable for smaller/medium sized function chains, while reduce-based promise implementations are ok for all cases, including larger chains.
This of course doesn't mean that recursion-based promises are bad. We just need to use them with their limitations in mind. Also, it's rare that you'll need to chain that many calls (>=500) via promises. I typically find myself using them for async configurations which utilize heavily ajax. But even if the most complex cases I haven't seen a situation with more than 15 chains.
On a side note...
These statistics were retrieved from tests performed with another of my libraries - provisnr - which captures the achieved number of function calls within a given interval of time.
I'm a little confused about how the sequencing works in various documents I've come across. For instance, I have seen this sort of thing
let p = Promise.resolve();
for (let x in something)
{
/* do some hairy time consuming code */
p = p.then(dosomething(x));
}
return p.then(finalthing()).catch(pretendnobadthing());
What I really don't understand is that if you do something that takes a large amount of time in the main code, wouldn't that mean that one of the promises could complete before the main code actually got round to to setting up the .then.
Can you always use .then/.catch on a promise, and if its already reached completion, it'll carry on from the point it got to, so it conceptually looks like a big chain that will run to completion at some indeterminate point? And if so, doesn't that mean every time you create a promise chain it'll hang about for ever?
Can you always use .then/.catch on a promise, and if its already reached completion, it'll carry on from the point it got to, so it conceptually looks like a big chain that will run to completion at some indeterminate point?
Yes, you can always use .then/.catch on a promise. When you call p.then(), there are three possibilities.
The promise p is still pending (not fulfilled or rejected yet). If that's the case, then the function reference(s) you passed to .then() are registered as listeners for that promise. So, when a future state transition happens on the promise (either going from pending => fulfilled or from pending => rejected), then the appropriate registered listeners will be called.
The promise p is already fulfilled. If that's the case, then calling .then(f1, f2) will schedule f1 to be called on the next tick (after the current piece of Javascript finishes executing) and it will be passed the saved resolved value.
The promise p is already rejected. If that's the case, then calling .then(f1, f2) will schedule f2 to be called on the next tick (after the current piece of Javascript finishes executing) and it will be passed the saved reject reason.
So, it's perfectly safe to call .then() on a promise that is already fulfilled or rejected. The appropriate listener will just be scheduled to run on the next tick.
The same logic applies to .catch() except it is only interested in cases 1 and 3 above.
Here are
And if so, doesn't that mean every time you create a promise chain it'll hang about for ever?
Promises are just objects like any other objects in Javascript. They will hang around only while other code still has some live reference to them. As soon as there is no longer any way to reach that promise object, they will be eligible for garbage collection just like any other objects in Javascript.
So, if you do:
var p = somePromiseReturningFunction();
p.then(f1).then(f2).then(f3);
Then, p will remain around until somePromiseReturningFunction() is done with any references to the promise that it returned (usually, though not always, this occurs when the promise is finally fulfilled or rejected) and when the variable p goes out of scope. If p never goes out of scope (like when it's global or in some other lasting scope), then it will remain forever (just like any other Javascript object).
There are some misconceptions in your question so let me attempt to square those up.
You're using the construct p = p.then(dosomething(x)); which is likely not correct. You need to pass .then() a function reference. So, unless you want doSomething(x) to execute immediately and it also returns another function that is what you want called as then .then() handler (which seems unlikely here), then this is not the right construct. You probably meant to have:
p = p.then(result => dosomething(x));
or in ES5 syntax:
p = p.then(function(result) {
return dosomething(x)
});
You show the same issue in this too:
return p.then(finalthing()).catch(pretendnobadthing());
which should probably be:
return p.then(finalthing).catch(pretendnobadthing);
Remember, when you use f(), that means to execute f immediately. When you just pass f, that passes a function reference which the underlying function/method you are passing it to can then call later at the time of its choosing which is what you want for .then() and .catch() handlers.
What I really don't understand is that if you do something that takes a large amount of time in the main code, wouldn't that mean that one of the promises could complete before the main code actually got round to to setting up the .then.
First off, my original explanation at the beginning of my answer should explain that calling .then() on an already resolved promise is perfectly fine so this isn't an issue at all. It will just schedule action on the next tick of the event loop.
But, that isn't even the case here because Javascript in the browser and node.js is single-threaded so while your long-running code is running, that promise (who's async action was previously started) can't yet get resolved. Though the underlying async operation may be done and an event may be sitting in the Javascript event queue that will trigger a callback that will resolve the promise, that event in the event queue won't get processed until the current piece of Javascript that is executing is done and returns control to the system.
What I really don't understand is that if you do something that takes a large amount of time in the main code, wouldn't that mean that one of the promises could complete before the main code actually got round to to setting up the .then.
Since common implementations of JavaScript never run it in parallel, no. Nevertheless,
Can you always use .then/.catch on a promise, and if its already reached completion, it'll carry on from the point it got to, so it conceptually looks like a big chain that will run to completion at some indeterminate point?
yes! This is a big advantage of promises. Take this for example:
var promise = Promise.resolve(5); // already resolved here
setTimeout(function () {
promise.then(function (x) {
console.log(x); // still logs 5
});
}, 1000);
And if so, doesn't that mean every time you create a promise chain it'll hang about for ever?
Until it’s resolved, yes, but if the promise has already been resolved and there are no ways to reference it anymore, it can be disposed of like any other object.
Promise = require 'bluebird'
cb = ->
console.log 'callback!'
p = Promise.resolve(5)
.cancellable()
.tap -> p.cancel()
setInterval(cb, 100)
The cb function only is only called once. Commenting out .tap -> p.cancel() allows it to run repeatedly. Adding a try block doesn't help. Perhaps this is something obvious, but I did some research and can't find an explanation.
It would seem that the act of returning the value of p.cancel() from the tap handler is causing bluebird to go into some sort of infinite loop. You never see the second 'callback!' because the execution context is stuck in this loop before 100 ms have elapsed.
I'm still far from understanding all the factors at play here (see below), but it looks like this can be fixed by not returning p.cancel():
Promise = require 'bluebird'
cb = ->
console.log 'callback!'
p = Promise.resolve(5)
.cancellable()
.tap ->
p.cancel()
null
setInterval(cb, 100)
Edit: Ok, after looking at the source and unknotting my brain a few times, I think it boils down to this:
Execution is getting stuck in an infinite loop here, where .cancel() tries to climb up the promise chain:
while ((parent = promiseToReject._cancellationParent) !== undefined &&
parent.isCancellable()) {
promiseToReject = parent;
}
The salient points are the following:
p.cancel() returns p.
.tap() returns a promise that resolves whenever the promise returned from its handler resolves (if it returns a promise)
p is the promise that .tap() returns
In other words, p is a promise that will resolve after p resolves. It is its own ancestor in the promise chain (at least, I think so).
When .cancel() tries to climb up the promise chain to find a cancellable promise, it happens upon this incestuous relationship and starts going in circles forever.
In the end, it's an unfortunate consequence of CoffeeScript's eagerness to turn almost everything into a return statement. But I would imagine that there's some way Bluebird could detect loops in promise chains and prevent an infinite loop from happening here.
I have filed an issue for this on the bluebird GitHub repository, but as the ensuing discussion reveals, this use of .cancel() doesn't really make any sense at all anyway.
If I create a single Deferred object, then it reports progress like I would expect.
var d1 = function() {
var d = $.Deferred();
d.notify('d1');
d.resolve('d1');
return d.promise();
};
d1().progress(function(a) {
log('Progress: ' + a);
});
However, if I chain two Deferred objects using then(), the progress callbacks are not called.
d1().then(d1).progress(function(a) {
log('Progress: ' + a);
});
It seems to me that then() should propagate the progress notification, but it doesn't appear to be the case. Am I missing something?
I have tested this with jQuery 1.8.0 and 1.8.1. A full working example is here: http://jsfiddle.net/eWQuG/13/
I can see what's going on but it's slightly tricky to explain. You may have to read this through a couple of times ...
In Test 1, the promise passed to test() is the "original promise" ie. the object returned by d1(). Simple.
In Test 2, the object passed to test() is a new promise resulting from d1().then(d2).then(d3). Less simple. This new promise will have inherited only some of the original promise's properties - not including its progressCallbacks (as put in place in the original promise with notify()). This is reasonable for a promise returned by .then(), because .then() is designed to put in place its own doneCallbacks, failCallbacks and progressCallbacks, not to propagate those of the deferred/promise to its immediate left. In short, .then() does exactly what its name implies - it specifies what is to be done after its preceding deferred/promise has been processed.
You can better see what's going on by keeping a reference to the original promise (ie. the output of d1()), thus allowing the simple Test 1 to be run both before and after Test 2.
Here is a composite Test 3 :
log('-- Test 3 --');
var p = d1();//p is a promise
test(p);//equivalent to Test 1
test(p.then(d2).then(d3));//equivalent to Test 2
test(p);//equivalent to Test 1 again
DEMO
In the log, you will see that the behaviour of test(p) is identical in both cases. However, test(p.then(d2).then(d3)) behaves differently because the object passed is no longer the original promise but a new promise with no progressCallbacks.
EDIT
Ryan, something maybe worth considering in achieving what you want with :
d1().then(d2).progress(function(a) {
log('Progress: ' + a);
});
is that a the chain can be rearranged as follows :
d1().progress(function(a) {
log('Progress: ' + a);
}).then(d2);
The progress callback is thus applied to the original promise and because .progress() propagates the original promise down the chain, .then() will still act as before.
Both versions are valid but would be applicable in different circumstances.
If I'm right then maybe this is the short answer to your question.
You are doing a few weird things in your tests that are not really logical.
One, you are resolving the promise before you even return it, which defeat the purpose of promises. Promise callbacks are not supposed to be called on previously resolved promises, simple as that.
var p = $.Deferred();
p.then(function(p){log("ok")}).progress(function(p){log("progress " + p)});
p.notify('n1'); // will call handler and be logged
p.resolve();
p.notify('n2'); // will not call handler and will not be logged
Two, you are calling "notify" before you have set the progress callback, but that is somehow still working properly to a degree - it seems jQuery's particular implementation of notify is somewhat async... Except it fails with chaining. How or why, I don't know, but I don't think it is important to debug as your usage is improper anyway.
See this update to your fiddle:
http://jsfiddle.net/eWQuG/15/