We Keep Coding

Question: Concurrent queue with max concurrency (per cacheKey)

this is more of a opinionated question. I do have a working solution, but I'm not 100% comfortable with it, as it has it's flaws.
Maybe someone can help me to improve this.
Goal:
I have an external api that only allows 4 calls to be made concurrently against it (for each user). Our app can impersonate multiple users at once.
So the issue comes, if more than 4 calls are made against the api simultaneously. (sometimes more than 20 calls are made)
Using a batched approach with Promise.all and chunking would be very inefficient, as the calls have a different runtime each.
Ideally, the queue would work FIFO and as soon as one call finishes, the next call is started. At the current standing, I created 4 own FIFO queues with somewhat of a Promise chain and I fill these evenly (if all are running).
The problem that I have is, that I do not know how long a request is running.
So choosing one of the queues can lead to an overall longer runtime as necessary.
The calls are automatically rejected after 30s from the external api, so no dead lock here.
Another problem that I have is, that I have to return the data provided from the api to a dependency. The queue is called/filled from withan a callback function...
This is wrapped in a Promise itself, so we can wait as long as we want.
But storing the values in a cache for later retrieval is no option either.
So long story short, here is the code
class QueuingService {
/** ~FIFO queue */
private static queue: Record<string, { promise: Promise<Function>, uuid: string }[]> = {};
private static MAX_CONCURRENCY = 4
/** cacheKey is used as cachekey for grouping in the queue */
public static async addToQueueAndExecute(func: Function, cacheKey: string) {
let resolver: Function | null = null;
let promise = new Promise<Function>(resolve => {
resolver = resolve;
});
//in the real world, this is a real key created by nanoId
let uuid = `${Math.random()}`;
Array.isArray(this.queue[cacheKey])
? this.queue[cacheKey].push({ promise: promise, uuid: uuid })
: this.queue[cacheKey] = [{ promise: promise, uuid: uuid }];
//queue in all calls, until MAX_CONCURRENCY is reached. After that, slice the first entry and await the promise
if (this.queue[cacheKey].length > this.MAX_CONCURRENCY) {
let queuePromise = this.queue[cacheKey].shift();
if (queuePromise){
await queuePromise.promise;
}
}
//console.log("elements in queue:", this.queue[cacheKey].length, cacheKey)
//technically this wrapping is not necessary, but it makes to code more readable imho
let result = async () => {
let res = await func();
if (resolver) {
//resolve and clean up
resolver();
this.queue[cacheKey] = this.queue[cacheKey].filter(elem => elem.uuid !== uuid);
}
//console.log(res, cacheKey, "finshed after", new Date().getTime() - enteredAt.getTime(),"ms", "entered at:", enteredAt)
return res;
}
return await result();
}
}
async function sleep(ms:number){
return await new Promise(resolve=>window.setTimeout(resolve,ms))
}
async function caller(){
/* //just for testing with fewer entries
for(let i=0;i<3;i++){
let groupKey = Math.random() <0.5? "foo":"foo"
QueuingService.addToQueueAndExecute(async ()=>{
await sleep(Math.floor(4000-Math.random()*2000));
console.log(i, new Date().getSeconds(), new Date().getMilliseconds(),groupKey)
return Math.random()
},groupKey)
}
*/
for(let i=0;i<20;i++){
let groupKey = Math.random() <0.5? "foo":"foo";
let startedAt = new Date();
QueuingService.addToQueueAndExecute(async ()=>{
await sleep(Math.floor(4000-Math.random()*2000));
console.log(i, new Date().getTime()-startedAt.getTime(),groupKey)
return Math.random()
},groupKey)
}
}
caller()
Also, here is a playground with the code to play around with:
https://www.typescriptlang.org/play?#code/MYGwhgzhAECKCuBTeBLAdgcwMqIE4DcVhFoBvAWACgBIAegCp7oA-AMQElWB5aARySTR6tKtQAOuFPjAAXEhBmyifAYgBc0AEqJgAe1wATADwLJmADRloE3QFsUEddAAKuOw8RHW8NMBkpdNAA+S3hUAw1TdAxoAF8AbQBdIOgAXjJYgG5RCSlZeUV-YGgAWQBBAA0AfQBhLgA5GoBVTU0AUUaATTToABYqUQYmYDBgAAtEAGlEAE9oB2h4RwNoSGgR8cQAa1noADN9aAw3eDFo+bRoGQmVZBJhHPgAIxBlBSViyBnfVYMDABVdAg7mU0AY2gAPHTwOQACj2PmAGm8vn8gUsGwm0xmkRkZgwAEoyKJqCBEDJoLhEBBdCB8HhkYi0ZcAD7QNDwEAgHocrnZGik8nWNz2Rw8xAAdxcIo8XiZAWCsKpNLpJFSKQoAuoytp9NwPR1qv51GosQJ-OglqtltotHQVxuVLA3Il+hABks1wWCzAlMQzugOzmwCdchWTzmaDAaF07AMJLJFLCKBW6QABgASUglWRjAB0uGjBjssIJsTT-IGArKuELMzzDhrddhXogef4d3imKms0SBJJ1AA-A6HO3VF3Rlje3mxEsxrDSML3I4NDZRYhQuENMmVmaBxpW2PO93sYkevFF2uPKuZY5Nynt+E4olKwLbR3BPbndyRlyIKE0H8blymqOpGhadounmGAnU2Aw82gMo9jkfVrlkSwIFeYgHRIPYUFwBRoEQQDcDmItVglMAUApa4SCvRwSRQPZoBbMZRw-RAJ02U88zJTBrmgFJDxA2oGmaVoOhqToiU1E1BQpDjXGXNURzbDiuKnGZEjzCA2OQ0tjRNJiWMU29EAJWS5LASjqNuJAlPXGczIta1XMtWISQ8yg3JtWg9DQFVEF43QMFhAAiRAyVsYiZBge0OLUMLPTYtTxxPac+Iwa4MUnHsZn7SgSVtORxjQIhvzmVtoAlQsxDOTBoPZXQKTQHRqQgMBSMsJ4YXmClbDAHYYBkXR1l0AwSFsfQSCdAwwBeEgUFsMZdATIVlU5Cl0i+H5SzSDUB0TP0YG2myKQRXwDIHYylWpXU8Bkgc6FoQ16VWMF1jJaNFjEJ7XrwK6tWoQ91PSrSehBtLcp4vCQBQ2FIsQWx9qIqK8x3aAAEJUnSHdzQHLyfNc21-MC4LQuVHLuNmSwwrwgKJhWMBkLwJL2UlaAABF8lLPMMHJf4lsQPaAFoiMAvBEAMMoZD5gWhdLcwwtsCA2YiiWqSZmREssGLJelmQCoHKkZHgXBLmVQyvJJE2zcuayqIpDa4cB00qGtygduKC6-AVaBMMQRAxFhFW1A5Wwnge2TbfNijHfZqUHI8W6VXpdUJXQYsJR0+Xot0GEU-u8wVYJAqPa9-Z5UCdZvwBiyqGtBhoFtAArJZzsOOQFHODOBL2SU8HFvEUGpBurQOXBYSOlBUgABkyFAjAAZgXgBqVf6+8nyjuOfOxGxHoc2uAsixLIkjFnvMAFZhzp3RdDCxKDgfse3OBVBMBwAgiCCsA-kBd+iBQTgihMAGEwsK6lnVJqIm1oHa2QDkHWER98x7BAPfSevRZ7YJFigk+YIz70AAEzYNnqXFysCxoBVpEFdBoUUCWFalKbmcICRyxkDgfyBgICKwTlzHmbD+YyBKCgLkHguE8IJOYXepxsQFUoZaGOlw8GFgIbYUsr9XKxGkScfesx5FWkJlaB4W9LSaInlPIUM956LxIWvDeMDt5ChkXouY6QVGn3UefS+N9oB3wfk-e+YUKGuSOu8XAYYZbimYQIkJ1p37RC-oQYgeY-4AiBKoYBkJoRwkgQSaBmiibwIpIg4OeC0EYNhFgnBHi1GlmIaQ8hhSfKkxoeTWEDC+EsOFoI3OPSRbhMibLIRgtoqKxcXI5pbklGlFzPg4sXiplxB0XvSZpi4iaPdlQX8ZJJ4EiAA

Can I build a WebWorker that executes arbitrary Javascript code?

I'd like to build a layer of abstraction over the WebWorker API that would allow (1) executing an arbitrary function over a webworker, and (2) wrapping the interaction in a Promise. At a high level, this would look something like this:
function bake() {
... // expensive calculation
return 'mmmm, pizza'
}
async function handlePizzaButtonClick() {
const pizza = await workIt(bake)
eat(pizza)
}
(Obviously, methods with arguments could be added without much difficulty.)
My first cut at workIt looks like this:
async function workIt<T>(f: () => T): Promise<T> {
const worker: Worker = new Worker('./unicorn.js') // no such worker, yet
worker.postMessage(f)
return new Promise<T>((resolve, reject) => {
worker.onmessage = ({data}: MessageEvent) => resolve(data)
worker.onerror = ({error}: ErrorEvent) => reject(error)
})
}
This fails because functions are not structured-cloneable and thus can't be passed in worker messages. (The Promise wrapper part works fine.)
There are various options for serializing Javascript functions, some scarier than others. But before I go that route, am I missing something here? Is there another way to leverage a WebWorker (or anything that executes in a separate thread) to run arbitrary Javascript?

I thought an example would be useful in addition to my comment, so here's a basic (no error handling, etc.), self-contained example which loads the worker from an object URL:
Meta: I'm not posting it in a runnable code snippet view because the rendered iframe runs at a different origin (https://stacksnippets.net at the time I write this answer — see snippet output), which prevents success: in Chrome, I receive the error message Refused to cross-origin redirects of the top-level worker script..
Anyway, you can just copy the text contents, paste it into your dev tools JS console right on this page, and execute it to see that it works. And, of course, it will work in a normal module in a same-origin context.
console.log(new URL(window.location.href).origin);
// Example candidate function:
// - pure
// - uses only syntax which is legal in worker module scope
async function get100LesserRandoms () {
// If `getRandomAsync` were defined outside the function,
// then this function would no longer be pure (it would be a closure)
// and `getRandomAsync` would need to be a function accessible from
// the scope of the `message` event handler within the worker
// else a `ReferenceError` would be thrown upon invocation
const getRandomAsync = () => Promise.resolve(Math.random());
const result = [];
while (result.length < 100) {
const n = await getRandomAsync();
if (n < 0.5) result.push(n);
}
return result;
}
const workerModuleText =
`self.addEventListener('message', async ({data: {id, fn}}) => self.postMessage({id, value: await eval(\`(\${fn})\`)()}));`;
const workerModuleSpecifier = URL.createObjectURL(
new Blob([workerModuleText], {type: 'text/javascript'}),
);
const worker = new Worker(workerModuleSpecifier, {type: 'module'});
worker.addEventListener('message', ({data: {id, value}}) => {
worker.dispatchEvent(new CustomEvent(id, {detail: value}));
});
function notOnMyThread (fn) {
return new Promise(resolve => {
const id = window.crypto.randomUUID();
worker.addEventListener(id, ({detail}) => resolve(detail), {once: true});
worker.postMessage({id, fn: fn.toString()});
});
}
async function main () {
const lesserRandoms = await notOnMyThread(get100LesserRandoms);
console.log(lesserRandoms);
}
main();

How can I execute some async tasks in parallel with limit in generator function?

I'm trying to execute some async tasks in parallel with a limitation on the maximum number of simultaneously running tasks.
There's an example of what I want to achieve:
Currently this tasks are running one after another. It's implemented this way:
export function signData(dataItem) {
cadesplugin.async_spawn(async function* (args) {
//... nestedArgs assignment logic ...
for (const id of dataItem.identifiers) {
yield* idHandler(dataItem, id, args, nestedArgs);
}
// some extra logic after all tasks were finished
}, firstArg, secondArg);
}
async function* idHandler(edsItem, researchId, args, nestedArgs) {
...
let oDocumentNameAttr = yield cadesplugin.CreateObjectAsync("CADESCOM.CPAttribute");
yield oDocumentNameAttr.propset_Value("Document Name");
...
// this function mutates some external data, making API calls and returns void
}
Unfortunately, I can't make any changes in cadesplugin.* functions, but I can use any external libraries (or built-in Promise) in my code.
I found some methods (eachLimit and parallelLimit) in async library that might work for me and an answer that shows how to deal with it.
But there are still two problems I can't solve:
How can I pass main params into nested function?
Main function is a generator function, so I still need to work with yield expressions in main and nested functions
There's a link to cadesplugin.* source code, where you can find async_spawn (and another cadesplugin.*) function that used in my code.
That's the code I tried with no luck:
await forEachLimit(dataItem.identifiers, 5, yield* async function* (researchId, callback) {
//... nested function code
});
It leads to Object is not async iterable error.
Another attempt:
let functionArray = [];
dataItem.identifiers.forEach(researchId => {
functionArray.push(researchIdHandler(dataItem, id, args, nestedArgs))
});
await parallelLimit(functionArray, 5);
It just does nothing.
Сan I somehow solve this problem, or the generator functions won't allow me to do this?

square peg, round hole
You cannot use async iterables for this problem. It is the nature of for await .. of to run in series. await blocks and the loop will not continue until the awaited promise has resovled. You need a more precise level of control where you can enforce these specific requirements.
To start, we have a mock myJob that simulates a long computation. More than likely this will be a network request to some API in your app -
// any asynchronous task
const myJob = x =>
sleep(rand(5000)).then(_ => x * 10)
Using Pool defined in this Q&A, we instantiate Pool(size=4) where size is the number of concurrent threads to run -
const pool = new Pool(4)
For ergonomics, I added a run method to the Pool class, making it easier to wrap and run jobs -
class Pool {
constructor (size) ...
open () ...
deferNow () ...
deferStacked () ...
// added method
async run (t) {
const close = await this.open()
return t().then(close)
}
}
Now we need to write an effect that uses our pool to run myJob. Here you will also decide what to do with the result. Note the promise must be wrapped in a thunk otherwise pool cannot control when it begins -
async function myEffect(x) {
// run the job with the pool
const r = await pool.run(_ => myJob(x))
// do something with the result
const s = document.createTextNode(`${r}\n`)
document.body.appendChild(s)
// return a value, if you want
return r
}
Now run everything by mapping myEffect over your list of inputs. In our example myEffect we return r which means the result is also available after all results are fetched. This optional but demonstrates how program knows when everything is done -
Promise.all([1,2,3,4,5,6,7,8,9,10,11,12].map(myEffect))
.then(JSON.stringify)
.then(console.log, console.error)
full program demo
In the functioning demo below, I condensed the definitions so we can see them all at once. Run the program to verify the result in your own browser -
class Pool {
constructor (size = 4) { Object.assign(this, { pool: new Set, stack: [], size }) }
open () { return this.pool.size < this.size ? this.deferNow() : this.deferStacked() }
async run (t) { const close = await this.open(); return t().then(close) }
deferNow () { const [t, close] = thread(); const p = t.then(_ => this.pool.delete(p)).then(_ => this.stack.length && this.stack.pop().close()); this.pool.add(p); return close }
deferStacked () { const [t, close] = thread(); this.stack.push({ close }); return t.then(_ => this.deferNow()) }
}
const rand = x => Math.random() * x
const effect = f => x => (f(x), x)
const thread = close => [new Promise(r => { close = effect(r) }), close]
const sleep = ms => new Promise(r => setTimeout(r, ms))
const myJob = x =>
sleep(rand(5000)).then(_ => x * 10)
async function myEffect(x) {
const r = await pool.run(_ => myJob(x))
const s = document.createTextNode(`${r}\n`)
document.body.appendChild(s)
return r
}
const pool = new Pool(4)
Promise.all([1,2,3,4,5,6,7,8,9,10,11,12].map(myEffect))
.then(JSON.stringify)
.then(console.log, console.error)
slow it down
Pool above runs concurrent jobs as quickly as possible. You may also be interested in throttle which is also introduced in the original post. Instead of making Pool more complex, we can wrap our jobs using throttle to give the caller control over the minimum time a job should take -
const throttle = (p, ms) =>
Promise.all([ p, sleep(ms) ]).then(([ value, _ ]) => value)
We can add a throttle in myEffect. Now if myJob runs very quickly, at least 5 seconds will pass before the next job is run -
async function myEffect(x) {
const r = await pool.run(_ => throttle(myJob(x), 5000))
const s = document.createTextNode(`${r}\n`)
document.body.appendChild(s)
return r
}

In general, it should be better to apply #Mulan answer.
But if you also stuck into cadesplugin.* generator functions and don't really care about heavyweight external libraries, this answer may also be helpful.
(If you are worried about heavyweight external libraries, you may still mix this answer with #Mulan's one)
Async task running can simply be solved using Promise.map function from bluebird library and double-usage of cadesplugin.async_spawn function.
The code will look like the following:
export function signData(dataItem) {
cadesplugin.async_spawn(async function* (args) {
// some extra logic before all of the tasks
await Promise.map(dataItem.identifiers,
(id) => cadesplugin.async_spawn(async function* (args) {
// ...
let oDocumentNameAttr = yield cadesplugin.CreateObjectAsync("CADESCOM.CPAttribute");
yield oDocumentNameAttr.propset_Value("Document Name");
// ...
// this function mutates some external data and making API calls
}),
{
concurrency: 5 //Parallel tasks count
});
// some extra logic after all tasks were finished
}, firstArg, secondArg);
}
The magic comes from async_spawn function which is defined as:
function async_spawn(generatorFunction) {
async function continuer(verb, arg) {
let result;
try {
result = await generator[verb](arg);
} catch (err) {
return Promise.reject(err);
}
if (result.done) {
return result.value;
} else {
return Promise.resolve(result.value).then(onFulfilled, onRejected);
}
}
let generator = generatorFunction(Array.prototype.slice.call(arguments, 1));
let onFulfilled = continuer.bind(continuer, "next");
let onRejected = continuer.bind(continuer, "throw");
return onFulfilled();
}
It can suspend the execution of internal generator functions on yield expressions without suspending the whole generator function.

Comparing mutex implementations for async JavaScript

I've found somewhere implementation of the mutex pattern in JavaScript that works well but it looks really complex. Even, though I understand mutex pattern I don't know why promises in some places are being used in a given way. But about that in a moment. I was playing around with it and I've created my own implementation that is much easier to understand and has less code. Here is project: https://codesandbox.io/s/mutex-2e014
You have there 3 implementations of the mutex pattern. Mutex 0 and 1 is the same version with this difference, that in the version 1, I've simplified code by taking advantage of arrow functions and being able to access this from the parent context. So overall they are the same solutions. My solution is Mutex 2.
You can test code by opening console and clicking buttons from 0 to 2. You can see that each asynchronous action takes 1 second and it will always execute operations in proper order and wait for the previous task to finish. It works the same way in all versions. Also when you hold the SHIFT key, when pressing buttons it will throw an exception and that part is also working the same in all versions. Next thing, I had to compare is return value and that's not different. So my questions is why is it so complex in the first 2 solutions? Am I missing something? Does it really work the same way?
Now the question about the promise usage that I don't understand. In the Mutex1.ts file in line 7, we have return new Promise(resolve);. We’re passing resolve from the parent promise as the first argument of the Promise constructor which is executor. And from what I read executors are being invoked just before promise creation. In this case, thanks to that we have access to the unlock function in the run method but I don’t know how it’s possible that this.lock returns function. Probably, I don't know promises well enough. Any ideas?
Mutex 0
export default class MutexA {
private mutex = Promise.resolve();
lock() {
let begin = unlock => {};
this.mutex = this.mutex.then(() => {
return new Promise(begin);
});
return new Promise(res => {
begin = res;
});
}
async run(fn) {
const unlock = await this.lock();
try {
return await Promise.resolve(fn());
} finally {
unlock();
}
}
}
Mutex 1
export default class MutexB {
private mutex = Promise.resolve();
lock() {
return new Promise(resolve => {
this.mutex = this.mutex.then(() => {
return new Promise(resolve);
});
});
}
async run(fn) {
const unlock = await this.lock();
try {
return await fn();
} finally {
unlock();
}
}
}
Mutex 2
export default class MutexC {
private mutex = Promise.resolve();
async run(fn) {
return new Promise((resolve, reject) => {
this.mutex = this.mutex.then(async () => {
try {
resolve(await fn());
} catch (err) {
reject(err);
}
});
});
}
}

Handling subscription/unsubscription to a ton of events with redux-saga

In my current project, i'm dealing with firebase websocket subscriptions. Different components can subscribe to different data, e.g. in a list of items every ListItem component subscribes to a websocket "event" for that specific item by dispatching a SUBSCRIBE action in componentDidMount and unsubscribes by dispatching an UNSUBSCRIBE action in componentWillUnmount.
My sagas look like this:
const subscriptions = {}
export function * subscribeLoop () {
while (true) {
const { path } = yield take(SUBSCRIBE)
subscriptions[path] = yield fork(subscription, path)
}
}
export function * unsubscribeLoop () {
while (true) {
const { path } = yield take(UNSUBSCRIBE)
yield cancel(subscriptions[path])
}
}
export function * subscription (path) {
let ref
try {
const updateChannel = channel()
ref = api.child(path)
ref.on('value', snapshot => {
updateChannel.put(snapshot.val())
})
while (true) {
const data = yield take(updateChannel)
yield put(handleUpdate(path, data))
}
} finally {
if (yield cancelled()) {
ref.off()
ref = null
}
}
}
I assume this is not the right way to deal with this - it is indeed rather slow on a list of 500 items.
How can i optimize the performance?
Do i even need to fork?
Should i introduce some kind of delay to give the thread some space to handle other things?
Any hints are appreciated.

Should i introduce some kind of delay to give the thread some space > to handle other things?
First of all it is necessary to remember that use of redux saga and effects like fork actually doesn't create any threads which would be twisted in the infinite loops. It is only syntactic sugar for the organization of a chain of callbacks as the yield operator provides object passing in both sides. From this point of view a question of forced delays doesn't make a sense - as thread and doesn't exist.
Do i even need to fork?
In case of due skill it is possible to do generally without set of fork of calls, and to do everything in one root saga. The idea is in making a subscription with a callback function in the current lexical area on websocket, and to expect obtaining messages in the pseudo-infinite loop on the basis of delayed promise.
Conceptually the code can look approximately so:
const subscribers = new Map()
function * webSocketLoop() {
let resolver = null
let promise = new Promise(resolve => (resolver = resolve))
let message = null;
websocket.on('message', (payload) => {
message = Object.assign({}, payload)
resolver()
promise = promise.then(() => new Promise(resolve => (resolver = resolve)))
})
while(true) {
yield call(() => promise)
const type = message.type
const handlers = subscribers.get(type) || []
handlers.forEach(func => func(message))
}
}
export function * mainSaga () {
yield takeEvery(SUBSCRIBE, subscribe)
yield takeEvery(UNSUBSCRIBE, unsubscribe)
yield fork(webSocketLoop)
}