When passing parameters to next() of ES6 generators, why is the first value ignored? More concretely, why does the output of this say x = 44 instead of x = 43:
function* foo() {
let i = 0;
var x = 1 + (yield "foo" + (++i));
console.log(`x = ${x}`);
}
fooer = foo();
console.log(fooer.next(42));
console.log(fooer.next(43));
// output:
// { value: 'foo1', done: false }
// x = 44
// { value: undefined, done: true }
My mental model for the behavior of such a generator was something like:
return foo1 and pause at yield (and the next call which returns foo1 takes as argument 42)
pause until next call to next
on next yield proceed to the line with var x = 1 + 42 because this was the argument previously received
print x = 43
just return a {done: true} from the last next, ignoring its argument (43) and stop.
Now, obviously, this is not what's happening. So... what am I getting wrong here?
I ended up writing this kind of code to investigate the behavior more thoroughly (after re-re-...-reading the MDN docs on generators):
function* bar() {
pp('in bar');
console.log(`1. ${yield 100}`);
console.log(`after 1`);
console.log(`2. ${yield 200}`);
console.log(`after 2`);
}
let barer = bar();
pp(`1. next:`, barer.next(1));
pp(`--- done with 1 next(1)\n`);
pp(`2. next:`, barer.next(2));
pp(`--- done with 2 next(2)\n`);
pp(`3. next:`, barer.next(3));
pp(`--- done with 3 next(3)\n`);
which outputs this:
in bar
1. next: { value: 100, done: false }
--- done with 1 next(1)
1. 2
after 1
2. next: { value: 200, done: false }
--- done with 2 next(2)
2. 3
after 2
3. next: { value: undefined, done: true }
--- done with 3 next(3)
So apparently the correct mental model would be like this:
on first call to next, the generator function body is run up to the yield expression, the "argument" of yield (100 the first time) is returned as the value returned by next, and the generator body is paused before evaluating the value of the yield expression -- the "before" part is crucial
only on the second call to next is the value of the first yield expression computed/replaced with the value of the argument given to next on this call (not with the one given in the previous one as I expected), and execution runs until the second yield, and next returns the value of the argument of this second yield -- here was my mistake: I assumed the value of the first yield expression is the argument of the first call to next, but it's actually the argument of the second call to next, or, another way to put it, it's the argument of the call to next during whose execution the value is actually computed
This probably made more sense to who invented this because the # of calls to next is one more times the number of yield statements (there's also the last one returning { value: undefined, done: true } to signal termination), so if the argument of the first call would not have been ignored, then the one of the last call would have had to be ignored. Also, while evaluating the body of next, the substitution would have started with the argument of its previous invocation. This would have been much more intuitive imho, but I assume it's about following the convention for generators in other languages too and consistency is the best thing in the end...
Off-topic but enlightening: Just tried to do the same exploration in Python, which apparently implements generators similar to Javascript, I immediately got a TypeError: can't send non-None value to a just-started generator when trying to pass an argument to the first call to next() (clear signal that my mental model was wrong!), and the iterator API also ends by throwing a StopIteration exception, so no "extra" next() needed just to check if the done is true (I imagine using this extra call for side effects that utilize the last next argument would only result in very hard to understand and debug code...). Much easier to "grok it" than in JS...
I got this from Axel Rauschmayer's Exploring ES6, especially 22.4.1.1.
On receiving a .next(arg), the first action of a generator is to feed arg to yield. But on the first call to .next(), there is no yield to receive this, since it is only at the end of the execution.
Only on the second invocation x = 1 + 43 is executed and subsequently logged, and the generator ends.
Also had a hard time wrapping my head around generators, especially when throwing in if-statements depended on yielded values. Nevertheless, the if-statement was actually what helped me getting it at last:
function* foo() {
const firstYield = yield 1
console.log('foo', firstYield)
const secondYield = yield 3
console.log('foo', secondYield)
if (firstYield === 2) {
yield 5
}
}
const generator = foo()
console.log('next', generator.next( /* Input skipped */ ).value)
console.log('next', generator.next(2).value)
console.log('next', generator.next(4).value)
/*
Outputs:
next 1
foo 2
next 3
foo 4
next 5
*/
Everything immediately became clear once I made this realization.
Here's your typical generator:
function* f() {
let a = yield 1;
// a === 200
let b = yield 2;
// b === 300
}
let gen = f();
gen.next(100) // === { value: 1, done: false }
gen.next(200) // === { value: 2, done: false }
gen.next(300) // === { value: undefined, done: true }
But here's what actually happens. The only way to make generator execute anything is to call next() on it. Therefore there needs to be a way for a generator to execute code that comes before the first yield.
function* f() {
// All generators implicitly start with that line
// v--------<---< 100
= yield
// ^-------- your first next call jumps right here
let a = yield 1;
// a === 200
let b = yield 2;
// b === 300
}
Related
Why is it that I get different results when calling next() directly on a generator, versus on a variable with the same generator assigned as its value?
All code/output below.
Below is the generator, plus variable declaration/assignment:
function* gen() {
yield 1;
yield 2;
yield 3;
};
let genVar = gen();
First code snippet:
let first = genVar.next();
console.log(first);
second = genVar.next();
console.log(second);
Output of first code snippet:
{ value: 1, done: false }
{ value: 2, done: false }
Second code snippet:
let one = gen().next();
console.log(one);
two = gen().next();
console.log(two);
Output of second code snippet:
{ value: 1, done: false }
{ value: 1, done: false }
My best guess at the moment is this has something to do with assignment by value/reference?
Anytime you execute the generator function, you create a brand new generator object that allows to iterate over the steps of the generator function from the start.
So in your second example, you actually create two different iterators.
I'm trying to learn about generator functions in javascript and am experimenting with using a for..of loop in order to iterate over my yield statements. One thing I noticed is that the function terminates after the final yield statement and does not attempt to execute the return statement. If I were to create a generator object and call generator.next() 4 times, the return statement would trigger and we would see the associated generator object. Would anyone have any idea what's going on under the hood to cause this behavior?
Thanks!
function* generator(){
yield 1
yield 2
yield 3
return "hello there"
}
for (const value of generator()){
console.log('value', value)
}
the function terminates after the final yield statement and does not attempt to execute the return statement
This is wrong, the return statement is executed, which is easy to see with
function* generator() {
yield 1
yield 2
yield 3
console.log('returning')
return "hello there"
}
for (const value of generator()) {
console.log('value', value)
}
And indeed the next method is called 4 times by the loop:
function* generator() {
yield 1
yield 2
yield 3
return "hello there"
}
const iterator = {
state: generator(),
[Symbol.iterator]() { return this; },
next(...args) {
const res = this.state.next(...args)
console.log(res)
return res
},
}
for (const value of iterator) {
console.log('value', value)
}
However, the return value, not being a yielded value, marks the end of the iteration and does not get to become a value in the for loop body, which only runs 3 times. If you wanted it to run 4 times, you'd need to yield 4 values.
This question already has answers here:
In ES6, what happens to the arguments in the first call to an iterator's `next` method?
(3 answers)
Closed 6 months ago.
Consider this generator function.
Why is the argument of the first call to .next() essentially lost? It will yield and log each of the letter strings but skips "A". Can someone offer an explanation. Please advise on a way that I can access each argument each argument of the .next() method and store it in an array within the generator function?
function* gen(arg) {
let argumentsPassedIn = [];
while (true) {
console.log(argumentsPassedIn);
arg = yield arg;
argumentsPassedIn.push(arg);
}
}
const g = gen();
g.next("A"); // ??
g.next("B");
g.next("C");
g.next("D");
g.next("E");
This is a limitation of generator functions. If you really want to be able to use the first argument, you can construct your own iterator manually.
const gen = () => {
const argumentsPassedIn = [];
const makeObj = () => ({
done: false,
value: undefined,
next: (arg) => {
argumentsPassedIn.push(arg);
console.log(argumentsPassedIn);
return makeObj();
},
});
return makeObj();
}
const g = gen();
g.next("A");
g.next("B");
g.next("C");
g.next("D");
g.next("E");
As per the docs
The first call of next executes from the start of the function until the first yield statement
So when you call the first next, it just calls the generator function from start to till the first yield and then returns and from next call it works normal.
To make you code work, you should try like this.
function* gen(arg) {
let argumentsPassedIn = [];
while (true) {
console.log(argumentsPassedIn);
arg = yield arg;
argumentsPassedIn.push(arg);
}
}
const g = gen();
g.next()
g.next("A"); // ??
g.next("B");
g.next("C");
g.next("D");
g.next("E");
The parameter passed to the first call to .next() is ignored as of ES2022. This is because the first call to .next() runs the function until the first yield or return is encountered and the following calls will make yield operator return the value passed as a parameter. This is usually solved by calling .next() unconditionally after calling the generator function.
There is, however, a stage 2 proposal that aims to solve this problem by introducing new syntax to get the value passed to the .next() method that most recently resumed execution of the generator.
I am learning this feature from ES6(Function generators) and I am having difficulties to understand the following code :
function* HelloGen2() {
var a = yield 100;
var b = yield a + 100;
console.log(b);
}
var gen2 = HelloGen2();
console.log(gen2.next()); // {value: 100, done: false}
console.log(gen2.next(500)); // {value: 600, done: false}
console.log(gen2.next(1000)); // {value: undefined, done: true}
Questions :
1 : In the first gen2.next() the line of code var a = yield 100; is called , is it set some value on the var a variable ?
2: In each gen2.next() only the line until the semicolon is executed ?
So for instance , in the second call gen2.next(500) the line console.log(b); is not executed
3: i do not understand the last line gen2.next(1000), how the b variable get the 1000 value ?
When working with coroutines, it's important to understand, that although yield and next are different keywords, what they do is essentially the same thing. Imagine two coroutines as being connected by a two-way communication pipe. Both
Y = yield X
and
Y = next(X)
perform the same set of operations:
write X to the pipe
wait for the answer
once the answer is there, read it from the pipe and assign it to Y
proceed with the execution
Initially, the main program is in the active state, and the generator (gen2 in your example) is waiting listening to the pipe. Now, when you call next, the main program writes a dummy value (null) to the pipe, thus waking up the generator. The generator executes yield 100, writes 100 to the pipe and waits. The main wakes up, reads 100 from the pipe, logs it and writes 500. The generator wakes up, reads 500 into a, etc. Here's the complete flow:
gen wait
main next() null -> pipe
main wait
gen pipe -> null
gen yield 100 100 -> pipe
gen wait
main pipe -> arg (100)
console.log(arg) 100
main next(500) 500 -> pipe
main wait
gen a= pipe -> a (500)
gen yield a + 100 600 -> pipe
gen wait
main pipe -> arg (600)
console.log(arg) 600
main next(1000) pipe -> 1000
main wait
gen b= pipe -> b
console.log(b) 1000
gen (ended) done -> pipe
main pipe -> arg (done)
console.log(arg)
main (ended)
Essentially, to understand generators, you have to remember that when you assign or somehow else use the result of yield/next, there's a pause between the right and the left (or "generate" and "use") parts. When you do
var a = 5
this is executed immediately, while
var a = yield 5
involves a pause between = and yield. This requires some mental gymnastics, very similar to async workflows, but with time you'll get used to that.
The relationship between yield x and generator.next(y) is as follows:
generator.next(y) returns {value: x, done: false}
yield x evaluates to y
Additionally, a generator function doesn't begin execution when initially invoked. You can think of it as a constructor. Only after generator.next() is called does the actual function code execute until the first yield statement (or the function terminates).
In your example:
var gen2 = HelloGen2() creates the generator, but nothing happnes.
gen2.next() executes the function up to yield 100 and then returns {value: 100...}.
gen2.next(500) resumes function execution, firstly evaluating the yield statement to 500 (and immediately setting this value into a), and continue execution up to yield a+100 and then returns {value: 600...}.
gen2.next(1000) resumes function execution, firstly evaluating the yield statement to 1000 (and immediately setting this value into b), and continue execution up to termination and then returns {value: undefined, done: true}.
That last undefined is the result of the function not returning any value explicitly.
Questions :
In the first gen2.next() the line of code var a = yield 100; is called, is it set some value on the var a variable?
Not at this time, with the next call of gen2.next(500) it gets the value of 500. At the first call the value is still undefined.
In each gen2.next() only the line until the semicolon is executed?
So for instance, in the second call gen2.next(500) the line console.log(b); is not executed.
Right. The function stops directly after the last yield statement.
I do not understand the last line gen2.next(1000), how the b variable get the 1000 value?
With the call of next(), the function continues at the last stop with the yield statement, the value is handed over and assigned to b.
The console.log(b)is performed and the function exits at the end and the result of the generator is set to { value: undefined, done: true }.
function* HelloGen2() {
var a, b;
console.log(a, b); // undefined undefined
a = yield 100;
console.log(a, b); // 500 undefined
b = yield a + 100;
console.log(b); // 1000
}
var gen2 = HelloGen2();
console.log('#1');
console.log(gen2.next()); // { value: 100, done: false }
console.log('#2');
console.log(gen2.next(500)); // { value: 600, done: false }
console.log('#3');
console.log(gen2.next(1000)); // { value: undefined, done: true }
console.log('#4');
.as-console-wrapper { max-height: 100% !important; top: 0; }
Learn Generators - 4 » CATCH ERROR!
The solution uses a for loop but I just couldn't find anything in MDN - Iteration Protocols that refers to yield within callbacks.
I'm going to guess the answer is just don't do that but thanks in advance if anyone has the time or inclination to provide an explanation!
Code:
function *upper (items) {
items.map(function (item) {
try {
yield item.toUpperCase()
} catch (e) {
yield 'null'
}
}
}
var badItems = ['a', 'B', 1, 'c']
for (var item of upper(badItems)) {
console.log(item)
}
// want to log: A, B, null, C
Error:
⇒ learn-generators run catch-error-map.js
/Users/gyaresu/programming/projects/nodeschool/learn-generators/catch-error-map.js:4
yield item.toUpperCase() // error below
^^^^
SyntaxError: Unexpected identifier
at exports.runInThisContext (vm.js:73:16)
at Module._compile (module.js:443:25)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3
Even my editor knows this is a terrible idea...
Disclaimer: I'm the author of Learn generators workshopper.
Answer by #slebetman is kinda correct and I also can add more:
Yes, MDN - Iteration Protocol doesn't refer directly about yield within callbacks.
But, it tell us about importance from where you yield item, because you can only use yield inside generators. See MDN - Iterables docs to find out more.
#marocchino suggest just fine solution iterate over Array that was changed after map:
function *upper (items) {
yield* items.map(function (item) {
try {
return item.toUpperCase();
} catch (e) {
return null;
}
});
}
We can do it, because Array has iteration mechanism, see Array.prototype[##iterator]().
var bad_items = ['a', 'B', 1, 'c'];
for (let item of bad_items) {
console.log(item); // a B 1 c
}
Array.prototype.map doesn't have default iteration behavior, so we couldn't iterate over it.
But generators is not just iterators. Every generator is an iterator, but not vice versa. Generators allows you to customize iteration (and not only) process by calling yield keyword. You can play and see the difference between generators/iterators here:
Demo: babel/repl.
One problem is yield yields just one level to the function's caller. So when you yield in a callback it may not do what you think it does:
// The following yield:
function *upper (items) { // <---- does not yield here
items.map(function (item) { // <----- instead it yields here
try {
yield item.toUpperCase()
} catch (e) {
yield 'null'
}
}
}
So in the code above, you have absolutely no access to the yielded value. Array.prototype.map does have access to the yielded value. And if you were the person who wrote the code for .map() you can get that value. But since you're not the person who wrote Array.prototype.map, and since the person who wrote Array.prototype.map doesn't re-yield the yielded value, you don't get access to the yielded value(s) at all (and hopefully they will be all garbage collected).
Can we make it work?
Let's see if we can make yield work in callbacks. We can probably write a function that behaves like .map() for generators:
// WARNING: UNTESTED!
function *mapGen (arr,callback) {
for (var i=0; i<arr.length; i++) {
yield callback(arr[i])
}
}
Then you can use it like this:
mapGen(items,function (item) {
yield item.toUpperCase();
});
Or if you're brave you can extend Array.prototype:
// WARNING: UNTESTED!
Array.prototype.mapGen = function *mapGen (callback) {
for (var i=0; i<this.length; i++) {
yield callback(this[i])
}
};
We can probably call it like this:
function *upper (items) {
yield* items.mapGen(function * (item) {
try {
yield item.toUpperCase()
} catch (e) {
yield 'null'
}
})
}
Notice that you need to yield twice. That's because the inner yield returns to mapGen then mapGen will yield that value then you need to yield it in order to return that value from upper.
OK. This sort of works but not quite:
var u = upper(['aaa','bbb','ccc']);
console.log(u.next().value); // returns generator object
Not exactly what we want. But it sort of makes sense since the first yield returns a yield. So we process each yield as a generator object? Lets see:
var u = upper(['aaa','bbb','ccc']);
console.log(u.next().value.next().value.next().value); // works
console.log(u.next().value.next().value.next().value); // doesn't work
OK. Let's figure out why the second call doesn't work.
The upper function:
function *upper (items) {
yield* items.mapGen(/*...*/);
}
yields the return value of mapGen(). For now, let's ignore what mapGen does and just think about what yield actually means.
So the first time we call .next() the function is paused here:
function *upper (items) {
yield* items.mapGen(/*...*/); // <----- yields value and paused
}
which is the first console.log(). The second time we call .next() the function call continue at the line after the yield:
function *upper (items) {
yield* items.mapGen(/*...*/);
// <----- function call resumes here
}
which returns (not yield since there's no yield keyword on that line) nothing (undefined).
This is why the second console.log() fails: the *upper() function has run out of objects to yield. Indeed, it only ever yields once so it has only one object to yield - it is a generator that generates only one value.
OK. So we can do it like this:
var u = upper(['aaa','bbb','ccc']);
var uu = u.next().value; // the only value that upper will ever return
console.log(uu.next().value.next().value); // works
console.log(uu.next().value.next().value); // works
console.log(uu.next().value.next().value); // works
Yay! But, if this is the case, how can the innermost yield in the callback work?
Well, if you think carefully you'll realize that the innermost yield in the callback also behaves like the yield in *upper() - it will only ever return one value. But we never use it more than once. That's because the second time we call uu.next() we're not returning the same callback but another callback which in turn will also ever return only one value.
So it works. Or it can be made to work. But it's kind of stupid.
Conclusion:
After all this, the key point to realize about why yield doesn't work the way we expected is that yield pauses code execution and resumes execution on the next line. If there are no more yields then the generator terminates (is .done).
Second point to realize is that callbacks and all those Array methods (.map, .forEach etc.) aren't magical. They're just javascript functions. As such it's a bit of a mistake to think of them as control structures like for or while.
Epilogue
There is a way to make mapGen work cleanly:
function upper (items) {
return items.mapGen(function (item) {
try {
return item.toUpperCase()
} catch (e) {
return 'null'
}
})
}
var u = upper(['aaa','bbb','ccc']);
console.log(u.next().value);
console.log(u.next().value);
console.log(u.next().value);
But you'll notice that in this case we return form the callback (not yield) and we also return form upper. So this case devolves back into a yield inside a for loop which isn't what we're discussing.
You can use another method by "co - npm": co.wrap(fn*)
function doSomething(){
return new promise()
}
var fn = co.wrap(function* (arr) {
var data = yield arr.map((val) => {
return doSomething();
});
return data;
});
fn(arr).then(function (val) {
consloe.log(val)
});