I am trying to build a way to create a generator which can yield DOM events. More generally, I want to create a way to convert an event system to an async system yielding events.
My initial code example works, but I can see an issue with lifting the resolve function from the Promise so that I can call that function once the event comes in.
class EventPropagation {
constructor(id) {
const button = document.getElementById(id);
let _resolve;
button.addEventListener("click", event => {
if (_resolve) {
_resolve(event);
}
});
let _listen = () => {
return new Promise(resolve => {
_resolve = resolve;
});
}
this.subscribe = async function*() {
const result = await _listen();
yield result;
yield * this.subscribe();
}
}
}
async function example() {
const eventPropagation = new EventPropagation("btn");
for await (const event of eventPropagation.subscribe()) {
console.log(event);
}
}
// call the example function
example();
My question is: Is there a better way of building something like this? There are a lot of things to think about, like multiple events coming in at the same time or cleaning up the listener and the subscriptions. My goal is not to end up with a reactive library but I do want to create small transparent functions which yield events asynchronously.
fiddle
Edited 14 dec 2017 (Edited in response to Bergi's comment)
Async Generators
Babel and a few plugins later; async generators aren't a problem:
const throttle = ms => new Promise(resolve => setTimeout(resolve, ms));
const getData = async() => {
const randomValue = Math.floor(Math.random() * 5000 + 1);
await throttle(randomValue);
return `The random value was: ${randomValue}`;
}
async function* asyncRandomMessage() {
const message = await getData();
yield message;
// recursive call
yield *asyncRandomMessage();
}
async function example() {
for await (const message of asyncRandomMessage()) {
console.log(message);
}
}
// call it at your own risk, it does not stop
// example();
What I want to know is how I transform a series of individual callback calls into an async stream. I can't imagine this problem isn't tackled. When I look at the library Bergi showed in the comments I see the same implementation as I did, namely: "Store the resolve and reject functions somewhere the event handler can call them." I can't imagine that would be a correct way of solving this problem.
You need a event bucket, here is an example:
function evtBucket() {
const stack = [],
iterate = bucket();
var next;
async function * bucket() {
while (true) {
yield new Promise((res) => {
if (stack.length > 0) {
return res(stack.shift());
}
next = res;
});
}
}
iterate.push = (itm) => {
if (next) {
next(itm);
next = false;
return;
}
stack.push(itm);
}
return iterate;
}
;(async function() {
let evts = evtBucket();
setInterval(()=>{
evts.push(Date.now());
evts.push(Date.now() + '++');
}, 1000);
for await (let evt of evts) {
console.log(evt);
}
})();
My best solution thus far has been to have an internal EventTarget that dispatches events when new events are added onto a queue array. This is what I've been working on for a JS modules library (including used modules here). I don't like it... But it works.
Note: This also handles the new AbortSignal option for event listeners in multiple places.
export function isAborted(signal) {
if (signal instanceof AbortController) {
return signal.signal.aborted;
} else if (signal instanceof AbortSignal) {
return signal.aborted;
} else {
return false;
}
}
export async function when(target, event, { signal } = {}) {
await new Promise(resolve => {
target.addEventListener(event, resolve, { once: true, signal });
});
}
export async function *yieldEvents(what, event, { capture, passive, signal } = {}) {
const queue = [];
const target = new EventTarget();
what.addEventListener(event, event => {
queue.push(event);
target.dispatchEvent(new Event('enqueued'));
}, { capture, passive, signal });
while (! isAborted(signal)) {
if (queue.length === 0) {
await when(target, 'enqueued', { signal }).catch(e => {});
}
/**
* May have aborted between beginning of loop and now
*/
if (isAborted(signal)) {
break;
} else {
yield queue.shift();
}
}
}
The example provided by NSD, but now in Typescript
class AsyncQueue<T> {
private queue: T[] = [];
private maxQueueLength = Infinity;
private nextResolve = (value: T) => {};
private hasNext = false;
constructor(maxQueueLength?: number) {
if (maxQueueLength) {
this.maxQueueLength = maxQueueLength;
}
}
async *[Symbol.asyncIterator]() {
while (true) {
yield new Promise((resolve) => {
if (this.queue.length > 0) {
return resolve(this.queue.shift());
}
this.nextResolve = resolve;
this.hasNext = true;
});
}
}
push(item: T) {
if (this.hasNext) {
this.nextResolve(item);
this.hasNext = false;
return;
}
if (this.queue.length > this.maxQueueLength) {
this.queue.shift();
}
this.queue.push(item);
}
}
(async function () {
const queueu = new AsyncQueue<string>();
setInterval(() => {
queueu.push(Date.now().toString());
queueu.push(Date.now().toString() + "++");
}, 1000);
for await (const evt of queueu) {
console.log(evt);
}
})();
Related
context: Two javascript classes in separate files, each integrating a different external service and being called in a express.js router.
See "problematic code" below:
route
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
const data = await ( await ( await awsTranscribe.Upload(req.file, bucket)).CreateJob(transcribeParams)).GetJob()
res.send(data)
})
S3 class
class AmazonS3 {
constructor() {
this.Upload = this.Upload
}
async Upload(file, bucket) {
const uploadParams = {
Bucket: bucket,
Body: fs.createReadStream(file.path),
Key: file.filename,
}
this.data = await s3.upload(uploadParams).promise()
return this
}
}
Transcribe class
class Transcribe extends AwsS3 {
constructor() {
super()
this.CreateJob = this.CreateJob
this.GetJob = this.GetJob
}
async CreateJob(params) {
if(this.data?.Location) {
params.Media = { ...params.Media, MediaFileUri: this.data.Location }
}
this.data = await transcribeService.startTranscriptionJob(params).promise()
return this
}
async GetJob(jobName) {
if(this.data?.TranscriptionJob?.TranscriptionJobName) {
jobName = this.data.TranscriptionJob.TranscriptionJobName
}
this.data = await transcribeService.getTranscriptionJob({TranscriptionJobName: jobName}).promise()
return this
}
}
problem: the problem is with the chained awaits in the router file:
await ( await ( await awsTranscribe.Upload...
Yes, it does work, but it would be horrible for another person to maintain this code in the future.
How can i make so it would be just
awsTranscribe.Upload(req.file, bucket).CreateJob(transcribeParams).GetJob() without the .then?
The problem is with the chained awaits in the router file: await ( await ( await awsTranscribe.Upload...
No, that's fine. In particular it would be trivial to refactor it to separate lines:
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
const a = await awsTranscribe.Upload(req.file, bucket);
const b = await b.CreateJob(transcribeParams);
const c = await b.GetJob();
res.send(c);
});
Your actual problem is that a, b, and c all refer to the same object awsTranscribe. Your code would also "work" if it was written
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
await awsTranscribe.Upload(req.file, bucket);
await awsTranscribe.CreateJob(transcribeParams);
await awsTranscribe.GetJob();
res.send(awsTranscribe);
});
The horrible thing is that you are passing your data between these methods through the mutable awsTranscribe.data property - even storing different kinds of data in it at different times! One could change the order of method calls and it would completely break in non-obvious and hard-to-debug ways.
Also it seems that multiple requests share the same awsTranscribe instance. This will not work with concurrent requests. Anything is possible from just "not working" to responding with the job data from a different user (request)! You absolutely need to fix that, then look at ugly syntax later.
What you really should do is get rid of the classes. There's no reason to use stateful objects here, this is plain procedural code. Write simple functions, taking parameters and returning values:
export async function uploadFile(file, bucket) {
const uploadParams = {
Bucket: bucket,
Body: fs.createReadStream(file.path),
Key: file.filename,
};
const data = s3.upload(uploadParams).promise();
return data.Location;
}
export async function createTranscriptionJob(location, params) {
params = {
...params,
Media: {
...params.Media,
MediaFileUri: location,
},
};
const data = await transcribeService.startTranscriptionJob(params).promise();
return data.TranscriptionJob;
}
async function getTranscriptionJob(job) {
const jobName = job.TranscriptionJobName;
return transcribeService.getTranscriptionJob({TranscriptionJobName: jobName}).promise();
}
Then you can import and call them as
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
const location = await uploadFile(req.file, bucket);
const job = await createTranscriptionJob(location, transcribeParams);
const data = await getTranscriptionJob(job);
res.send(c);
});
I got interested in whether it was possible to take an object with several async methods and somehow make them automatically chainable. Well, you can:
function chain(obj, methodsArray) {
if (!methodsArray || !methodsArray.length) {
throw new Error("methodsArray argument must be array of chainable method names");
}
const methods = new Set(methodsArray);
let lastPromise = Promise.resolve();
const proxy = new Proxy(obj, {
get(target, prop, receiver) {
if (prop === "_promise") {
return function() {
return lastPromise;
}
}
const val = Reflect.get(target, prop, receiver);
if (typeof val !== "function" || !methods.has(prop)) {
// no chaining if it's not a function
// or it's not listed as a chainable method
return val;
} else {
// return a stub function
return function(...args) {
// chain a function call
lastPromise = lastPromise.then(() => {
return val.apply(obj, args);
//return Reflect.apply(val, obj, ...args);
});
return proxy;
}
}
}
});
return proxy;
}
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
function log(...args) {
if (!log.start) {
log.start = Date.now();
}
const delta = Date.now() - log.start;
const deltaPad = (delta + "").padStart(6, "0");
console.log(`${deltaPad}: `, ...args)
}
class Transcribe {
constructor() {
this.greeting = "Hello";
}
async createJob(params) {
log(`createJob: ${this.greeting}`);
return delay(200);
}
async getJob(jobName) {
log(`getJob: ${this.greeting}`);
return delay(100);
}
}
const t = new Transcribe();
const obj = chain(t, ["getJob", "createJob"]);
log("begin");
obj.createJob().getJob()._promise().then(() => {
log("end");
});
There's a placeholder for your Transcribe class that has two asynchronous methods that return a promise.
Then, there's a chain() function that returns a proxy to an object that makes a set of passed in method names be chainable which allows you to then do something like this:
const t = new Transcribe();
// make chainable proxy
const obj = chain(t, ["getJob", "createJob"]);
obj.createJob().getJob()
or
await obj.createJob().getJob()._promise()
I wouldn't necessarily say this is production-ready code, but it is an interesting feasibility demonstration and (for me) a chance to learn more about a Javascript proxy object.
Here's a different approach that (instead of the proxy object) adds method stubs to a promise to make things chainable:
function chain(orig, methodsArray) {
let masterP = Promise.resolve();
function addMethods(dest) {
for (const m of methodsArray) {
dest[m] = function(...args) {
// chain onto master promise to force sequencing
masterP = masterP.then(result => {
return orig[m].apply(orig, ...args);
});
// add methods to the latest promise befor returning it
addMethods(masterP);
return masterP;
}
}
}
// add method to our returned promise
addMethods(masterP);
return masterP;
}
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
function log(...args) {
if (!log.start) {
log.start = Date.now();
}
const delta = Date.now() - log.start;
const deltaPad = (delta + "").padStart(6, "0");
console.log(`${deltaPad}: `, ...args)
}
class Transcribe {
constructor() {
this.greeting = "Hello";
this.cntr = 0;
}
async createJob(params) {
log(`createJob: ${this.greeting}`);
++this.cntr;
return delay(200);
}
async getJob(jobName) {
log(`getJob: ${this.greeting}`);
++this.cntr;
return delay(100);
}
}
const t = new Transcribe();
log("begin");
chain(t, ["getJob", "createJob"]).createJob().getJob().then(() => {
log(`cntr = ${t.cntr}`);
log("end");
});
Since this returns an actual promise (with additional methods attached), you can directly use .then() or await with it without the separate ._promise() that the first implementation required.
So, you can now do something like this:
const t = new Transcribe();
chain(t, ["getJob", "createJob"]).createJob().getJob().then(() => {
log(`cntr = ${t.cntr}`);
});
or:
const t = new Transcribe();
await chain(t, ["getJob", "createJob"]).createJob().getJob();
log(`cntr = ${t.cntr}`);
And, here's a third version where it creates a thenable object (a pseudo-promise) with the added methods on it (if it bothers you to add methods to an existing promise):
function chain(orig, methodsArray) {
if (!methodsArray || !methodsArray.length) {
throw new Error("methodsArray argument must be array of chainable method names");
}
let masterP = Promise.resolve();
function makeThenable() {
let obj = {};
for (const m of methodsArray) {
obj[m] = function(...args) {
// chain onto master promise to force sequencing
masterP = masterP.then(result => {
return orig[m].apply(orig, ...args);
});
return makeThenable();
}
}
obj.then = function(onFulfill, onReject) {
return masterP.then(onFulfill, onReject);
}
obj.catch = function(onReject) {
return masterP.catch(onReject);
}
obj.finally = function(onFinally) {
return masterP.finally(onFinally);
}
return obj;
}
return makeThenable();
}
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
function log(...args) {
if (!log.start) {
log.start = Date.now();
}
const delta = Date.now() - log.start;
const deltaPad = (delta + "").padStart(6, "0");
console.log(`${deltaPad}: `, ...args)
}
class Transcribe {
constructor() {
this.greeting = "Hello";
this.cntr = 0;
}
async createJob(params) {
log(`createJob: ${this.greeting}`);
++this.cntr;
return delay(200);
}
async getJob(jobName) {
log(`getJob: ${this.greeting}`);
++this.cntr;
return delay(100);
}
}
const t = new Transcribe();
log("begin");
chain(t, ["getJob", "createJob"]).createJob().getJob().then(() => {
log(`cntr = ${t.cntr}`);
log("end");
});
What I want to do is to create an iterator, which is only triggered when an external function is called, say an external event.
An iterator that simply waits for custom events.
function createIteratorWithFunction() {
var thingThatResolves;
var asyncIterable = {
thingThatResolves,
[Symbol.asyncIterator]() {
return {
next() {
return (new Promise((resolve, reject) => asyncIterable.thingThatResolves = (resolve))).then(_ => ({
value: _,
done: false
}));
},
return () {
return {
done: true
}
}
};
}
};
return asyncIterable;
}
iter = createIteratorWithFunction();
(async function() {
for await (let val of iter) {
console.log(val);
}
})()
<button onclick="iter.thingThatResolves('execute');iter.thingThatResolves(3)">execute next!</button>
As you can see, it only resolves 'execute', but not 3, of course because promises can't be resolved more than once, and it only is updated asynchronously, I understand this, but since the iterator is async, how would I create a queue, so that any values that could've synchronously been triggered are retrieved by next(), as well?
I have this feeling that there's a more elegant solution involving promise chains, but it's escaping me at the moment. :-) See inline comments:
function createIteratorWithFunction() {
// Our pending promise, if any
let promise = null;
// The `resolve` function for our `pending` promise
let resolve = null;
// The values in the queue
const values = [];
// The async iterable
const asyncIterable = {
add(value) {
// Add a value to the queue; if there's a pending promise, fulfill it
values.push(value);
const r = resolve;
resolve = pending = null;
r?.();
},
[Symbol.asyncIterator]() {
return {
async next() {
// If we don't have a value...
while (!values.length) {
// ...we need to wait for one; make sure we have something
// to wait for
if (!resolve) {
pending = new Promise(r => { resolve = r; });
}
await pending;
}
// Get the value we waited for and return it
const value = values.shift();
return {
value,
done: false,
};
},
return() {
return {
done: true,
};
}
};
}
};
return asyncIterable;
}
const iter = createIteratorWithFunction();
(async function() {
for await (let val of iter) {
console.log(val);
}
})();
document.getElementById("execute").addEventListener("click", () => {
iter.add("execute");
iter.add(3);
});
<button id="execute">execute next!</button>
One of the key things here is that an async iterable can have overlapping iterations, and it has to not get confused by that. This implementation avoids that by creating the promise it'll wait on synchronously if it needs one.
function createIteratorWithFunction() {
// Our pending promise, if any
let promise = null;
// The `resolve` function for our `pending` promise
let resolve = null;
// The values in the queue
const values = [];
// The async iterable
const asyncIterable = {
add(value) {
// Add a value to the queue; if there's a pending promise, fulfill it
values.push(value);
const r = resolve;
resolve = pending = null;
r?.();
},
[Symbol.asyncIterator]() {
return {
async next() {
// If we don't have a value...
while (!values.length) {
// ...we need to wait for one; make sure we have something
// to wait for
if (!resolve) {
pending = new Promise(r => { resolve = r; });
}
await pending;
}
// Get the value we waited for and return it
const value = values.shift();
return {
value,
done: false,
};
},
return() {
return {
done: true,
};
}
};
}
};
return asyncIterable;
}
const iter = createIteratorWithFunction();
(async function() {
for await (let val of iter) {
console.log("first:", val);
}
})();
(async function() {
for await (let val of iter) {
console.log("second:", val);
}
})();
document.getElementById("execute").addEventListener("click", () => {
iter.add("execute");
iter.add(3);
});
<button id="execute">execute next!</button>
I'm never happy when I have to make the promise's resolve function accessible outside the promise executor function (the function you pass new Promise), but as I say, the elegant solution with promise chains is escaping me. I sense strongly that it's there...somewhere... :-)
Another idea & way of doing this, you could use custom events, one advantage is that the code is much easier to reason with.
Below I've knocked up a simple example, it also allows you to cancel the iterator & handle errors. makeIter simple gives you 4 functions,
add = use this to add an item to the itterator.
iter = this is the iterator you can for await on.
done = if were done, you can call this and let the GC do it's thing.
error = allows you to put an error into the iterator, you can test this by un-commenting the last line.
To prevent any race conditions I've simply used a stack..
function makeIter() {
const obj = new EventTarget();
const evName = 'my-iter';
const stack = [];
obj.addEventListener(evName, e => {
stack.push(e.detail);
resolve();
});
async function *iter() {
while (true) {
await new Promise(r => resolve = r);
while (stack.length) {
const s = stack.shift();
if (s.resolve) yield(s.resolve);
if (s.reject) throw s.reject;
if (s.cancel) return;
}
}
}
function ev(p) {
obj.dispatchEvent(new CustomEvent(evName, {detail:p}));
}
return {
error: (e) => ev({reject: e}),
done: () => ev({cancel: true}),
add: item => ev({resolve: item}),
iter: iter()
}
}
///testing...
const test = makeIter();
(async function () {
try {
for await (const item of test.iter) {
console.log(item);
}
} finally {
console.log('iter done');
}
}());
test.add('hello');
setTimeout(() => test.add('there'), 100);
setTimeout(() => {test.add('1'); test.add('2'); test.add('3'); }, 200);
setTimeout(() => test.add('4'), 400);
setTimeout(() => test.done(), 1000);
//setTimeout(() => test.error(new Error('oops')), 1000);
=================== Progress Update ===================
I've made my first Async function according to Mozilla and another tutorial. However, it couldn't prevent the event as I expected. It goes stacking when I click multiple times before the whole code is done.
My expectation is using async and promise to disable the event until the entire code is done which is the way that I use callback like this CodePen example.
Additionally, I can't fully understand the concept of Async and Promise. I think the Async + Promise function separates the code as itself such as the bookmark of the book in real life? It's really hard to understand what's going on inside of the code.
Can somebody explain to me that how async and promise work in the code and prevent the event?
This is a result that I've done so far:
class Async {
constructor(elem) {
this.elem = document.querySelectorAll(elem)
this.flag = true;
this.selector(this.elem, 'click');
}
selector(node, eventName) {
node.forEach(item => {
item.addEventListener(eventName, (e) => this.group(e))
})
}
waiting() {
if (this.flag) {
this.flag = false;
return new Promise(resolve => {
setTimeout(() => {
resolve(console.log('waiting . . .'))
}, 2000)
})
}
}
result() {
console.log('test');
this.flag = true;
}
async group(e) {
const a = await this.waiting();
const b = await this.result();
}
}
const async = new Async('.button');
=================== Original Question ===================
I'm separating the code blocks with import/export in Node js for looking clearer when I refactoring it.
A problem is boolean flag called this.flag doesn't prevent the event overriding when I pass it to init.js as a parameter like this:
See the code first:
// =================== init ===================
'use strict';
import Terminal from './terminal.js';
import Touch from './touch.js';
export const slider = (function() {
class Slider {
constructor(elem) {
this.elem = document.querySelector(elem);
this.flag = false;
this.terminal = new Terminal(this.elem);
this.touch = new Touch(this.elem);
this.terminal.insert(this.elem, 'click', this.touch.clicked.bind(this.touch));
}
wait(flag, callback) {
flag = false; // the boolean can't prevent the event overriding right now.
let bound = callback.bind(this);
console.log('waiting . . .');
setTimeout(bound, 1000, flag);
}
done(flag) {
console.log('done');
flag = true;
}
}
return {
init: new Slider('.contents')
}
}())
// =================== terminal.js ===================
export default class Terminal {
constructor(elem) {
this.elem = elem;
}
insert(node, eventName, callback) {
node.addEventListener(eventName, callback);
}
}
// =================== touch.js ===================
import {slider} from './a.js';
export default class Touch {
constructor(elem) {
this.elem = elem;
this.flag = true;
}
clicked(e) {
if (this.flag) {
console.log(`clicked`);
let slide = slider;
slide.init.wait(this.flag, slide.init.done);
}
}
}
But the weird thing is, when I replace the both functions wait() and result() to touch.js, it prevents the event until the countdown is done.
// touch.js
wait(callback) {
this.flag = false;
let bound = callback.bind(this);
console.log('waiting . . .');
setTimeout(bound, 1000);
}
done(flag) {
console.log('done');
this.flag = true;
}
I would like to know why the flag can't prevent the event when it passed to the another js file and how to make it disables the event temporarily.
Explanation
Generally speaking
The async/await pattern got introduced with ES2017 of JavaScript and is syntactic sugar for the older Promise's
Some older browsers don't support async/await
async is a newer way of saying return Promise((resolve) => { ... }).
await is the counterpart to async and is the newer way of saying .then(result => { ... }).
await can only be used in functions marked as async
try/catch is the counterpart to .catch(error => { ... }). It is not actually new, but you can use it in this context.
You can read more about async/await here
Code
I have made some minor changes to your code, so that it makes more sense and written some comments, so that you understand everything that is happening here.
class Async {
constructor(elem) {
this.elem = document.querySelectorAll(elem)
this.isRunning = false; // <-- Rename the flag variable to something more meaningful
this.selector(this.elem, 'click');
}
selector(node, eventName) {
node.forEach(item => {
item.addEventListener(eventName, (e) => this.group(e))
})
}
waiting() {
return new Promise((resolve, reject) => { // <-- Move the Promise to here so that every codepath returns something
if (!this.isRunning) {
this.isRunning = true;
console.log('Waiting ... '); // <-- Move the waiting before the timeout, because inside it is not actually waiting, its rather done
setTimeout(() => { // <-- setTimeout runs the provided function after the provided time in milliseconds elapsed
this.isRunning = false; // <-- Switch the isRunning after the timeout, because that makes more sense (because now it is not running anymore)
resolve('Done'); // <-- Change the text to done and actually resolve it (eg. remove the console.log)
}, 2000)
} else {
reject('There is already a button function running'); // <-- reject is like throwing an error
}
})
}
result() {
console.log('test');
}
async group(e) {
try {
const a = await this.waiting(); // <-- Assigns 'Done' to the variable a
console.log(a); // <-- prints the message
this.result(); // <-- prints 'test' immidiatly after the above console.log
} catch (error) {
console.log(error); // <-- Prints the reject message in case the button is already running
}
/* group could also be written with the old syntax like this:
this.waiting().then(result => {
console.log(result); // Will print "Done" after 2000 milliseconds
this.result(); // Will print test instantly after the above console.log(). You dont need to await it, because it is not an async function
}).catch(error => {
console.log(error); // Will print the reject message in case the button is already running
});
*/
}
}
const asyncButton = new Async('.button'); // <-- Don't use async as a variable name. It's a reserved keyword
Running Example
It is the same code again, but without comments, so that you can test it directly here on StackOverflow.
class Async {
constructor(elem) {
this.elem = document.querySelectorAll(elem)
this.isRunning = false;
this.selector(this.elem, 'click');
}
selector(node, eventName) {
node.forEach(item => {
item.addEventListener(eventName, (e) => this.group(e))
})
}
waiting() {
return new Promise((resolve, reject) => {
if (!this.isRunning) {
this.isRunning = true;
console.log('Waiting ... ');
setTimeout(() => {
this.isRunning = false;
resolve('Done');
}, 2000)
} else {
reject('There is already a button function running');
}
})
}
result() {
console.log('test');
}
async group(e) {
try {
const a = await this.waiting();
console.log(a);
this.result();
} catch(error) {
console.log(error);
}
}
}
const asyncButton = new Async('.button');
<button class="button">Test</button>
Got a solution. Adding await functions into the boolean flag is working.
class Async {
constructor(elem) {
this.elem = document.querySelectorAll(elem)
this.flag = true;
this.selector(this.elem, 'click');
}
selector(node, eventName) {
node.forEach(item => {
item.addEventListener(eventName, (e) => this.group(e))
})
}
waiting() {
return new Promise(resolve => {
setTimeout(() => {
resolve(console.log('waiting . . .'))
}, 1000)
})
}
result() {
console.log('test');
this.flag = true;
}
async group(e) {
// console.log(this.flag);
if (this.flag) {
this.flag = false;
console.log('test');
const a = await this.waiting();
const b = await this.result();
}
}
}
const async = new Async('.button');
I'm working on converting a legacy callback-based API into an async library. But I just can't wrap my head around getting a "resultset" to work as a generator (Node 10.x).
The original API works like this:
api.prepare((err, rs) => {
rs.fetchRows(
(err, row) => {
// this callback is called as many times as rows exist
console.log("here's a row:", row);
},
() => {
console.log("we're done, data exausted");
}
);
});
But here is how I want to use it:
const wrapped = new ApiWrapper(api);
const rs = await wrapped.prepare({});
for (let row of rs.rows()) {
console.log("here's a row:", row);
}
let row;
while(row = await rs.next()) {
console.log("here's a row:", row);
}
I thought I had it under control with generators, but it looks like you cannot use yield inside a callback. It actually seems logical if you think about.
class ApiWrapper {
constructor(api) {
this.api = api;
}
prepare() {
return new Promise((resolve, reject) => {
this.api.prepare((err, rs) => {
if (err) {
reject(err);
} else {
resolve(rs);
}
});
});
}
*rows() {
this.api.fetchRows((err, row) => {
if (err) {
throw err;
} else {
yield row; // nope, not allowed here
}
});
}
next() { ... }
}
So what alternatives do I have?
Important: I don't want to store anything in an array then iterate that, we're talking giga-loads of row data here.
Edit
I'm able to simulate the behavior I want using stream.Readable but it warns me that it's an experimental feature. Here's a simplified array-based version of the issue I'm trying to solve using stream:
const stream = require('stream');
function gen(){
const s = new stream.Readable({
objectMode: true,
read(){
[11, 22, 33].forEach(row => {
this.push({ value: row });
});
this.push(null)
}
});
return s;
}
for await (let row of gen()) {
console.log(row);
}
// { value: 11 }
// { value: 22 }
// { value: 33 }
(node:97157) ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time
I finally realized I needed something similar to Go's channels that were async/await compatible. Basically the answer is to synchronize an async iterator and a callback, making them wait for each other as next() iterations are consumed.
The best (Node) native solution I found was to use a stream as an iterator, which is supported in Node 10.x but tagged experimental. I also tried to implement it with the p-defer NPM module, but that turned out to be more involved than I expected. Finally ran across the https://www.npmjs.com/package/#nodeguy/channel module, which was exactly what I needed:
const Channel = require('#nodeguy/channel');
class ApiWrapper {
// ...
rows() {
const channel = new Channel();
const iter = {
[Symbol.asyncIterator]() {
return this;
},
async next() {
const val = await channel.shift();
if (val === undefined) {
return { done: true };
} else {
return { done: false, value: val };
}
}
};
this.api.fetchRows(async (err, row) => {
await channel.push(row);
}).then(() => channel.close());
return iter;
}
}
// then later
for await (let row of rs.rows()) {
console.log(row)
}
Note how each iterating function core, next() and rows(), have a await that will throttle how much data can be pushed across the channel, otherwise the producing callback could end up pushing data uncontrollably into the channel queue. The idea is that the callback should wait for data to be consumed by the iterator next() before pushing more.
Here's a more self-contained example:
const Channel = require('#nodeguy/channel');
function iterating() {
const channel = Channel();
const iter = {
[Symbol.asyncIterator]() {
return this;
},
async next() {
console.log('next');
const val = await channel.shift();
if (val === undefined) {
return { done: true };
} else {
return { done: false, value: val };
}
}
};
[11, 22, 33].forEach(async it => {
await channel.push(it);
console.log('pushed', it);
});
console.log('returned');
return iter;
}
(async function main() {
for await (let it of iterating()) {
console.log('got', it);
}
})();
/*
returned
next
pushed 11
got 11
next
pushed 22
got 22
next
pushed 33
got 33
next
*/
Like I said, Streams and/or Promises can be used to implement this, but the Channel module solves some of the complexity that make it more intuitive.
The original question has two nested callback taking async functions
api.prepare((err,res) => ...)
rs.fetchRows((err,res) => ...)
The first one runs the callback only once so just promisifying it as follows is sufficient.
function promisfied(f){
return new Promise((v,x) => f(x,v));
}
However the second function will invoke it's callback multiple times and we wish to generate an async iterable from this function such that we can consume it in a for await of loop.
This is also possible by employing async generators as follows;
async function* rowEmitterGenerator(rs){
let _v, // previous resolve
_x, // previous reject
_row = new Promise((v,x) => (_v = v, _x = x));
rs.fetchRows((err, row) => ( err ? _x(err) : _v(row)
, _row = new Promise((v,x) => (_v = v, _x = x))
));
while(true){
try {
yield _row;
}
catch(e){
console.log(e);
}
}
}
Then putting all together in a top level await context;
const rows = await promisified(api.prepare),
rowEmitter = rowEmitterGenerator(rows);
for await (let row of rowEmitter){
console.log(`Received row: ${row}`);
// do something with the row
}
How do I clean up after a generating function has been exited. The problem I am having is I made a little utility to read-lines from a file using Async Iterators a stage 3 EcmaScript proposal and I want it to close the file it's reading after I exit a for-of loop. Currently since this feature is only stage 3, in order to get this to run you'll have to use babel to transpile it.
With the below code you can see the problem. If you pipe in an input file then it will read one line and print that the line reader is open still.
I would like to explicitly close the file inside the LineReader class when it's iterator is returned.
I know I can do this by not using a generating function but instead return an iterator object as outlined here, but is there any way I can keep the generating function and define a return method for it.
src/line-reader.js
function deferred() {
const def = {}
def.promise = new Promise((resolve, reject) => {
def.resolve = resolve
def.reject = reject
})
return def
}
/**
* PromiseQueue from GTOR adapted for ES2015
* https://github.com/kriskowal/gtor
*/
class PromiseQueue {
constructor (values) {
this.ends = deferred();
if (values) {
values.forEach(this.put, this);
}
}
put(value) {
const next = deferred();
this.ends.resolve({
head: value,
tail: next.promise
});
this.ends.resolve = next.resolve;
}
get () {
var result = this.ends.promise.then(node=>node.head);
this.ends.promise = this.ends.promise.then(node=>node.tail)
return result;
};
}
class LineReader {
constructor (input, output) {
this.lineReader = require('readline').createInterface({ input, output });
this.lineQueue = new PromiseQueue();
this.isClosed = false;
this.lineReader.on('line', (line) => {
this.lineQueue.put(line);
this.lineReader.pause();
});
this.lineReader.on('close', (line) => {
this.isClosed = true;
this.lineQueue.put(Promise.resolve({done: true}));
});
this.lineReader.on('SIGCONT', () => {
// `prompt` will automatically resume the stream
this.lineReader.prompt();
});
}
readLine(){
var result = this.lineQueue.get().then(function (data) {
if(data && data.done) {
throw new Error("EOF");
}
return data
});
this.lineReader.resume();
return result;
}
close () {
this.lineReader.close();
}
async * [Symbol.asyncIterator] () {
try {
while(!this.isClosed) {
yield await this.readLine();
}
} catch (e) {
this.close();
}
}
}
module.exports = LineReader;
test/test.js
var LineReader = require("../src/line-reader");
var lineReader = new LineReader(process.stdin);
(async ()=> {
var i = 1;
for await (var line of lineReader) {
console.log(`${i++} ${line}`);
break;
}
console.log(`The line-reader is ${lineReader.isClosed ? "closed" : "open" }.`);
lineReader.close();
})().catch(e=> {
console.error(e)
})
You can always Object.assign over a called generating function and extend its returned iterator with your return method.
Like so:
class LineReader {
// ... elided code ...
async * iterate() {
try {
while(!this.isClosed) {
yield await this.readLine();
}
} catch (e) {
this.close();
}
}
[Symbol.asyncIterator] () {
return Object.assign(this.iterate(), {
return: () => {
this.close();
}
});
}
}
Or optionally you can do it this way, but I prefer the first as it doesn't create a new function each time it's called and the first looks much better to me.
class LineReader {
// ... elided code ...
[Symbol.asyncIterator] () {
var that = this;
return Object.assign(async function * iterate() {
try {
while(!that.isClosed) {
yield await that.readLine();
}
} catch (e) {
that.close();
}
}(), {
return: () => {
this.close();
}
});
}
}
Just add a finally to your try block. finally will execute even if the function has returned (which it returns when someone breaks out of a for of loop). This guarantees that you function will clean up and you don't have to modify your function much. I just found this out thanks to this article by Jake Archibald.
class LineReader {
// ... elided code ...
async * [Symbol.asyncIterator] () {
try {
while(!this.isClosed) {
yield await this.readLine();
}
} finally {
this.close();
}
}
}