Consider the following:
A web application that can have up to 100 concurrent requests per second
Each incoming request currently makes a http request to an endpoint to get some data (which could take up to 5 seconds)
I want to only make the http request once, i.e. I don't want to make concurrent calls to the same endpoint as it will return the same data
The idea is only the first request will make the http call to get the data
While this call is 'inflight', and subsequent requests will not make the same call and instead 'wait' for the first inflight request to complete.
When the initial http request for data has responded, it must respond to all calls with the data.
I am using Bluebird promises to for the async function that performs the http request.
I would like to create/use some sort of generic method/class that wraps the business logic promise method. This generic method/call will know when to invoke the actual business logic function, when to wait for inflight to finish and then resolve all waiting calls when it has a response.
I'm hoping there is already a node module that can do this, but can't think of what this type of utility would be called.
Something similar to lodash throttle/debounce, but not quite the same thing.
I could write it myself if it doesn't exists, but struggling to come up with a sensible name for this.
Any help would be appreciated.
You can implement a PromiseCaching, like:
module.exports = function request(url) {
if (caches[url]) return caches[url];
var promise = req(url);
return (caches[url] = promise);
};
var req = require('');
var caches = {};
EDIT:
Let me be more explanatory:
Here is not about caching of the responses, but about caching of promises. Nodejs is single threaded, that means, there no concurrent function calls, even when everything is async, at one point of time, runs only one peace of code. That means, there will be somebody first calling the function with the url y.com/foo, there will be no promise in the cache, so it will fire the GET request und will cache and return that promise. When somebody immediately calls the function with the same url, no more requests are fired, but instead the very first promise for this url will be returned, and the consumer can subscribe on done/fail callbacks.
When the response is ready and the promise is fulfilled, and somebody makes the request with the same url, then again, it will get the cached promise back, which is already ready.
Promise caching is a good technique to prevent duplicate async tasks.
A web application can only have 6 concurrent requests because that's the hard browser limitation. Older IE can only do 2. So no matter what you do - this is a hard limit.
In general, you should solve the multiplexing on the server side.
Now - to your actual question - the sort of caching you're asking for is incredibly simple to do with promises.
function once(fn) {
var val = null; // cache starts as empty
return () => val || (val = fn()); // return value or set it.
}
var get = once(getData);
get();
get(); // same call, even though the value didn't change.
Now, you might want to add an expiry policy:
function once(fn, timeout) {
var val = null, timer = null; // cache starts as empty
return () => val || (val = fn().tap(invalidate)); // return value and invalidate
function invalidate() {
clearTimeout(timer); // remove timer.
timer = setTimeout(() => val = null, timeout);
}
}
var get = once(getData, 10000);
You might also want to uncache the result if it fails:
function once(fn, timeout) {
var val = null, timer = null; // cache starts as empty
return () => val ||
(val = fn().catch(e => value = null, Promise.reject(e)).tap(invalidate));
function invalidate() {
clearTimeout(timer); // remove timer.
timer = setTimeout(() => val = null, timeout);
}
}
Since the original functionality is one line of code, there isn't a helper for it.
You can used promise for prevent duplicate request same time
Example write in nodejs, you can using this pattern in browser as well
const rp = require('request-promise'),
var wait = null;
function getUser(req, rep, next){
function userSuccess(){
wait = null;
};
function userErr(){
wait = null;
};
if (wait){
console.log('a wait');
}
else{
wait = rp.get({ url: config.API_FLIX + "/menu"});
}
wait.then(userSuccess).catch(userErr);
}
Related
I'm currently working on a function that takes a pretty long time to finish and since I won't be able to make it finish faster and im going to call it from other Scripts I was wondering if there is a method to use something like a promise in that function.
Basically
function longrunning(){
var def = new $.Deferred();
var result = {};
[DO STUFF THAT TAKES A WHILE]
def.resolve();
return def.promise(result);
}
My basic problem is, that since all the stuff thats going on isn't async my promise won't be returned until everything is done, so the function that will later be calling longrunning won't know it's async. But of course if I return the promise before executing all of the code, it won't resolve at all. I hope you're getting what I'm trying to do. Hope someone has an idea. Thanks in advance and
Greetings Chris
Wrapping the code in a $.Deferred (or native promise) won't help even if you do manage to get the promise back to the calling code before doing the long-running work (for instance, via setTimeout). All it would accomplish is making the main UI thread seize up later, soon after longrunning returned the promise, instead of when calling longrunning itself. So, not useful. :-)
If the function in question doesn't manipulate the DOM, or if the manipulations it does can be separated from the long-running logic, this is a good candidate to be moved to a web worker (specification, MDN), so it doesn't get run on the main UI thread at all, but instead gets run in a parallel worker thread, leaving the UI free to continue to be responsive.
longrunning wouldn't do the actual work, it would just postMessage the worker to ask it to do the work, and then resolve your promise when it gets back a message that the work is done. Something along these lines (this is just a code sketch, not a fully-implemented solution):
var pendingWork = {};
var workId = 0;
var worker = new Worker("path/to/web/worker.js");
worker.addEventListener("message", function(e) {
// Worker has finished some work, resolve the Deferred
var d = pendingWork[e.data.id];
if (!d) {
console.error("Got result for work ID " + e.data.id + " but no pending entry for it was found.");
} else {
if (e.data.success) {
d.resolve(e.data.result);
} else {
d.reject(e.data.error);
}
delete pendingWork[e.data.id];
}
});
function longrunning(info) {
// Get an identifier for this work
var id = ++workid;
var d = pendingWork[id] = $.Deferred();
worker.postMessage({id: id, info: info});
return d.promise();
}
(That assumes what the worker sends back is an object with the properties id [the work ID], success [flag], and either result [the result] or error [the error].)
As you can see, there we have longrunning send the work to the worker and return a promise for it; when the worker sends the work back, a listener resolves the Deferred.
If the long-running task does need to do DOM manipulation as part of its work, it could post the necessary information back to the main script to have it do those manipulations on its behalf as necessary. The viability of that naturally depends on what the code is doing.
Naturally, you could use native promises rather than jQuery's $.Deferred, if you only have to run on up-to-date browsers (or include a polyfill):
var pendingWork = {};
var workId = 0;
var worker = new Worker("path/to/web/worker.js");
worker.addEventListener("message", function(e) {
// Worker has finished some work, resolve the Deferred
var work = pendingWork[e.data.id];
if (!work) {
console.error("Got result for work ID " + e.data.id + " but no pending entry for it was found.");
} else {
if (e.data.success) {
work.resolve(e.data.result);
} else {
work.reject(e.data.error);
}
delete pendingWork[e.data.id];
}
});
function longrunning(info) {
return new Promise(function(resolve, reject) {
// Get an identifier for this work
var id = ++workid;
pendingWork[id] = {resolve: resolve, reject: reject};
worker.postMessage({id: id, info: info});
});
}
I have a Node server that uses Connect to insert some middleware which attempt to transform a response stream from node-http-proxy. Occasionally this transformation can be quite slow and it would be preferable in such cases to simply return a response that doesn't include the transformations or alternatively includes their partial application.
In my application I've attempted to use setTimeout to call next after some number of milliseconds in the context of the transformation middleware. This generally works but exposes a race condition where if the middleware has already called next and then setTimeout fires and does the same an error occurs that looks like: Error: Can't set headers after they are sent.
Eventually I evolved the setTimeout to invoke next with an Error instance as its first argument and then later on in my middleware chain would catch that error and assuming res.headersSent was false would start sending the response via res.end.call(res). This worked and surprisingly I could set the timeout to nearly nothing and the response would happen significantly faster and be complete.
I feel like this last method is a bit of a hack and not immune from the same race condition, but perhaps appears to be a little more resilient. So I would like to know what sort of idiomatic approaches Node and Connect have for dealing with this kind of thing.
How can I go about timing out slow middleware and simply return the response stream?
Currently this seems to do what I want, more or less, but again feels a bit gross.
let resTimedout = false;
const timeout = setTimeout(() => {
if (!resTimedout) {
resTimedout = true;
next();
}
}, 100);
getSelectors(headers, uri, (selectors) => {
const resSelectors = Object.keys(selectors).map((selector) => {
...
};
const rewrite = resRewrite(resSelectors);
rewrite(req, res, () => {
if (!resTimedout) {
resTimedout = true;
clearTimeout(timeout);
next();
}
});
});
setTimeout returns the id of the timeout, so you can then run clearTimeout passing in the id. So when the transformation is complete just clear the timeout before you call next.
var a = setTimeout(()=>{}, 3000);
clearTimeout(a);
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout
Use async.timeout or Promise.timeout from Bluebird or Q library
You could eliminate the need for global variables and decide this per request:
const rewrite = resRewrite(resSelectors);
rewrite(req, res, () => {
// set a timer to fail the function early
let timer = setTimeout(() => {
timer = null;
next();
}, 100);
// do the slow response transformation
transformResponse((err, data) => { // eg. callback handler
clearTimeout(timer);
if (timer) next();
});
});
How it works
If the timer ends first, it sets itself to null and calls next(). When the transform function ends, it will see the timeout is null and not call next().
If the response transform is faster, it clears the timeout to prevent it running next later on.
I've implemented a cacheService that has method get(url) which gets the value from HTTP, puts it to localStorage and returns $q promise. On the next .get(url) run it gets value from localStorage and returns it wrapped with $q.
function get(url){
var saved = JSON.parse(localStorage.getItem(url));
if (angular.isObject(saved)){
return $q.when(saved);
}
return $http.get(url).then(function(xhr){
localStorage.setItem(url, JSON.stringify(xhr.data));
return xhr.data;
});
}
Here's a case when my method doesn't work - it makes more than 1 HTTP request.
If I call cacheService.get(url) twice (e.g. from different modules):
when it runs first time, it doesn't find cache value in localStorage and makes the HTTP request
when it runs second time just after first one, first HTTP request isn't done yet and it's result value isn't cached yet. So my method makes HTTP request second time.
And I don't like it.
So the question is:
how to syncronise requests by url to have only 1 request per url?
UPD:
I've got an idea:
define var currentRequests = {};
if localStorage has cached value, return the one
if currentRequests[url] is empty, do the request and set currentRequests[url] = $http.get(url) - store the promise in
if currentRequsts[url] isn't empty, then this request is running now, then just return currentRequests[url];
As JS is run in a single thread in browsers, currentRequests is thread-safe. But it isn't in nodejs. Correct?
What do you think about it?
It might be not that good solution. but you can give it a try.
I'll suggest you to maintain the array of promises.
var promiseArr = [];
function get(url){
var saved = JSON.parse(localStorage.getItem(url));
if (angular.isObject(saved)){
return $q.when(saved);
}
//check if $http request for the url has a pending promise.
if(!promiseArr.hasOwnProperty(url)){
promiseArr[url] = $http.get(url).then(function(xhr){
localStorage.setItem(url, JSON.stringify(xhr.data));
delete promiseArr[url]; //Delete the promise once resolved.. please check.
return xhr.data;
});
}
return promiseArr[url]; //return the promise from the array.
}
I want to implement a dynamic loading of a static resource in AngularJS using Promises. The problem: I have couple components on page which might (or not, depends which are displayed, thus dynamic) need to get a static resource from the server. Once loaded, it can be cached for the whole application life.
I have implemented this mechanism, but I'm new to Angular and Promises, and I want to make sure if this is a right solution \ approach.
var data = null;
var deferredLoadData = null;
function loadDataPromise() {
if (deferredLoadData !== null)
return deferredLoadData.promise;
deferredLoadData = $q.defer();
$http.get("data.json").then(function (res) {
data = res.data;
return deferredLoadData.resolve();
}, function (res) {
return deferredLoadData.reject();
});
return deferredLoadData.promise;
}
So, only one request is made, and all next calls to loadDataPromise() get back the first made promise. It seems to work for request that in the progress or one that already finished some time ago.
But is it a good solution to cache Promises?
Is this the right approach?
Yes. The use of memoisation on functions that return promises a common technique to avoid the repeated execution of asynchronous (and usually expensive) tasks. The promise makes the caching easy because one does not need to distinguish between ongoing and finished operations, they're both represented as (the same) promise for the result value.
Is this the right solution?
No. That global data variable and the resolution with undefined is not how promises are intended to work. Instead, fulfill the promise with the result data! It also makes coding a lot easier:
var dataPromise = null;
function getData() {
if (dataPromise == null)
dataPromise = $http.get("data.json").then(function (res) {
return res.data;
});
return dataPromise;
}
Then, instead of loadDataPromise().then(function() { /* use global */ data }) it is simply getData().then(function(data) { … }).
To further improve the pattern, you might want to hide dataPromise in a closure scope, and notice that you will need a lookup for different promises when getData takes a parameter (like the url).
For this task I created service called defer-cache-service which removes all this boiler plate code. It writted in Typescript, but you can grab compiled js file. Github source code.
Example:
function loadCached() {
return deferCacheService.getDeferred('cacke.key1', function () {
return $http.get("data.json");
});
}
and consume
loadCached().then(function(data) {
//...
});
One important thing to notice that if let's say two or more parts calling the the same loadDataPromise and at the same time, you must add this check
if (defer && defer.promise.$$state.status === 0) {
return defer.promise;
}
otherwise you will be doing duplicate calls to backend.
This design design pattern will cache whatever is returned the first time it runs , and return the cached thing every time it's called again.
const asyncTask = (cache => {
return function(){
// when called first time, put the promise in the "cache" variable
if( !cache ){
cache = new Promise(function(resolve, reject){
setTimeout(() => {
resolve('foo');
}, 2000);
});
}
return cache;
}
})();
asyncTask().then(console.log);
asyncTask().then(console.log);
Explanation:
Simply wrap your function with another self-invoking function which returns a function (your original async function), and the purpose of wrapper function is to provide encapsulating scope for a local variable cache, so that local variable is only accessible within the returned function of the wrapper function and has the exact same value every time asyncTask is called (other than the very first time)
I want to implement a dynamic loading of a static resource in AngularJS using Promises. The problem: I have couple components on page which might (or not, depends which are displayed, thus dynamic) need to get a static resource from the server. Once loaded, it can be cached for the whole application life.
I have implemented this mechanism, but I'm new to Angular and Promises, and I want to make sure if this is a right solution \ approach.
var data = null;
var deferredLoadData = null;
function loadDataPromise() {
if (deferredLoadData !== null)
return deferredLoadData.promise;
deferredLoadData = $q.defer();
$http.get("data.json").then(function (res) {
data = res.data;
return deferredLoadData.resolve();
}, function (res) {
return deferredLoadData.reject();
});
return deferredLoadData.promise;
}
So, only one request is made, and all next calls to loadDataPromise() get back the first made promise. It seems to work for request that in the progress or one that already finished some time ago.
But is it a good solution to cache Promises?
Is this the right approach?
Yes. The use of memoisation on functions that return promises a common technique to avoid the repeated execution of asynchronous (and usually expensive) tasks. The promise makes the caching easy because one does not need to distinguish between ongoing and finished operations, they're both represented as (the same) promise for the result value.
Is this the right solution?
No. That global data variable and the resolution with undefined is not how promises are intended to work. Instead, fulfill the promise with the result data! It also makes coding a lot easier:
var dataPromise = null;
function getData() {
if (dataPromise == null)
dataPromise = $http.get("data.json").then(function (res) {
return res.data;
});
return dataPromise;
}
Then, instead of loadDataPromise().then(function() { /* use global */ data }) it is simply getData().then(function(data) { … }).
To further improve the pattern, you might want to hide dataPromise in a closure scope, and notice that you will need a lookup for different promises when getData takes a parameter (like the url).
For this task I created service called defer-cache-service which removes all this boiler plate code. It writted in Typescript, but you can grab compiled js file. Github source code.
Example:
function loadCached() {
return deferCacheService.getDeferred('cacke.key1', function () {
return $http.get("data.json");
});
}
and consume
loadCached().then(function(data) {
//...
});
One important thing to notice that if let's say two or more parts calling the the same loadDataPromise and at the same time, you must add this check
if (defer && defer.promise.$$state.status === 0) {
return defer.promise;
}
otherwise you will be doing duplicate calls to backend.
This design design pattern will cache whatever is returned the first time it runs , and return the cached thing every time it's called again.
const asyncTask = (cache => {
return function(){
// when called first time, put the promise in the "cache" variable
if( !cache ){
cache = new Promise(function(resolve, reject){
setTimeout(() => {
resolve('foo');
}, 2000);
});
}
return cache;
}
})();
asyncTask().then(console.log);
asyncTask().then(console.log);
Explanation:
Simply wrap your function with another self-invoking function which returns a function (your original async function), and the purpose of wrapper function is to provide encapsulating scope for a local variable cache, so that local variable is only accessible within the returned function of the wrapper function and has the exact same value every time asyncTask is called (other than the very first time)