We Keep Coding

Abortable promises and using paged data from an API for an autocomplete text field

Disclaimer: I'm new to ES6 and promises in general so its possible my approach is fundamentally wrong.
The Problem
I have an api that returns paged data. That is it returns a certain number of objects for a collection and if there's more data it returns a Next property in the response body with the url to get the next set of data. The data will eventually feed the autocomplete on a text input.
To be specific, if my collection endpoint is called /Tickets the response would look like:
{
Next: "/Tickets?skip=100&take=100"
Items: [ ... an array of the first 100 items ... ]
}
My current solution to get all the ticket data would be to
Make a new promise for the returning the whole combined set of data
"Loop" the ajax calls by chaining dones until there is no more Next value
Resolve the promise
getTickets(filterValue) {
let fullSetPromise = new Promise();
let fullSet = [];
// This gets called recursively
let append = function(previousResult) {
fullSet.concat(previousResult.Items);
// Loop!
if(previousResult.Next) {
$.ajax({
url: previousResult.Next
})
.done(append)
.catch(reason => fullSetPromise.reject(reason));
}
else {
fullSetPromise.resolve(fullSet);
}
}
// We set things off by making the request for the first 100
$.ajax({
url: `/Tickets?skip=0&take=100&filter=${filterValue}`
})
.done(append)
.catch(reason => fullSetPromise.reject(reason));
return fullSetPromise;
}
Eventually the promise is used by the frontend for autocomplete on a text input. Ideally I'd like to be able to abort the previous call when new input comes in.
inputChanged(e) {
this.oldTicketPromise.abort();
this.oldTicketPromise =
Api.GetTickets(e.target.value).then(updateListWithResults);
}
I am aware of debouncing. But that just means the problem happens every n seconds instead of on every key press.
I know the jqxhr object has an abort() property on it and I'd like that to be available to the caller somehow. But because there are multiple jqXHR objects used in GetTickets I'm not sure how to handle this.
So my main 2 questions are:
What is the appropriate way to consume paged data from an api while returning a promise.
How can the returned promise be made abortable?
Side question:
I feel like if I don't catch the errors then my "wrapper" promise will swallow any thrown errors. Is that a correct assumption?
Note the javascript code might have errors. It's mostly demonstrative for the logic.
Edit: Solved
I have solved this by combining this answer https://stackoverflow.com/a/30235261/730326 with an array of xhrs as suggested in the comments. I will add a proper answer with code when I find the time.

Does .then(Promise.mapSeries(...)) behave differently than .then(function() { return Promise.mapSeries(...); })?

I initially assumed that passing a bare Promise.mapSeries(...) call as an argument to .then() would be the same as wrapping it in a function, like .then(function() { return Promise.mapSeries(...); }). Having written out this question, I'm no longer entirely sure why it works at all.
In the simplified code below, I open a couple of databases asynchronously (openDBAsync()) and then read a file containing a JS object. I use _.map() to iterate over all of the key/value pairs in the object and asynchronously update their values in a database, while keeping track of which ones meet certain criteria (whether the value is odd, in this toy example). Promise.all() waits for all of the async database calls to settle, and then Promise.mapSeries() is used to process each of the subset of keys, which makes another async database call for each one. Finally, I close all the databases.
function processData(path)
{
var oddKeys = [];
return openDBAsync()
.then(function() { return readFileAsync(path); })
.then(function(dataObject) {
return Promise.all(_.map(dataObject, function(value, key) {
if (value % 2) {
oddKeys.push(key);
}
return updateDBAsync(key, ++value);
}))
.then(Promise.mapSeries(
oddKeys,
function(key) {
return updateOddDBAsync(key);
}
))
})
.then(closeDBAsync);
}
The problem is that the database throws errors complaining that I'm trying to update the database after it's been closed. That means that some of the promises generated in the .mapSeries() call are being called after the final closeDBAsync(). I expected all of them to settle before the final .then() call.
If I wrap the call to Promise.mapSeries() in a function:
.then(function() {
return Promise.mapSeries(
oddKeys,
function(key) {
return updateOddDBAsync(key);
}
);
})
Then I don't get any errors. It also works if I put a .delay(2000) before the close database call, which indicates that Promise.mapSeries() isn't settling all of the promises before it finishes.
This seems like either a bug in Bluebird, or, much more likely, I'm not understanding something fundamental about how Promise.mapSeries() works. Any guidance would be much appreciated.

much more likely, I'm not understanding something fundamental about how Promise.mapSeries() works
Nope, this seems to be more a misunderstanding about how .then(…) works.
The then method of promises does always take a callback function (if you're passing anything else [but null], Bluebird should spit out a warning!). By calling .then(Promise.mapSeries(…)), you were passing a promise instead, which is simply ignored. Being ignored, it also is not awaited by anything, which leads to that error of the database being closed too early.
But the direct call to Promise.mapSeries() doesn't get applied to the array immediately. If it was, the array would be empty and this code wouldn't work at all.
Yes it does. Your array is filled from by the _.map callback, which is executed synchronously, before then and mapSeries are invoked.
So the solution is indeed to wrap the call in a function expression, which will only be executed when the Promise.all(…) fulfills, and whose result will then not be ignored but rather awaited. There might be more, different solutions, depending on what degree of parallel execution you want to allow.
Btw, given you are doing a database transaction, your current code is quite fragile. Look into the promise disposer pattern.

Should I emulate '.$promise' objects?

I am working on an admin app with AngularJS. The app fetches its data from the server using $resource. I end up with data objects containing '$promise' property to determine when the data has been fetched.
Everything's fine.
Now, this admin app can also create new objects. Those new objects are managed by the same controllers than the one that usually come from '$resource'.
So now I have 2 kind of objects:
Objects with $promise property. I should use $promise.then() before manipulating them with full data
Plain objects. They don't have a $promise property, their value is accessible instantly
I would like to reduce code and to have a single use-case, not having to check if object data is resolved, or if it is not a promise.
Is there any problem in building my 'plain' objects by adding them a '$promise' property which is already resolved to themselves? That way, I would always use 'myObject.$promise.then()'.
Is there any common pattern to handle this situation? I could not find any 'standard' method to create this kind of objects with Angular.

You could use $q.when if unsure whether the object has a promise or not.
(obj.$promise ? obj.$promise || $q.when(objResult)).then(function(result){
//handle success case.
//Incase of object not having the $promise property result will be object itself
})
if the resultant property does not have a promise this will resolve with promise.
Wraps an object that might be a value or a (3rd party) then-able promise into a $q promise. This is useful when you are dealing with an object that might or might not be a promise, or if the promise comes from a source that can't be trusted.
You do not need to always create a promise and attach it on the data that is being transferred, instead you could make your methods returns promise thus help you implement promise pattern and abstract out the promise logic on your service itself. Example:-
function getCurrentUserData(userId){
var defered = $q.defer();
...
//Check in my cache if this is already there, then get it from cache and resolve it
defered.resolve(objectFromCache);
//my else condition
//It is not in cache so let me make the call.
$http.get('myurl').then(function(result){
//Do validation with data and if it doesnot match then reject it
defered.reject(reason);
//Else Do something with the data put it into the cache, mapping logic etc..
defered.resolve(dto);
}).catch(function(error){
//do something with error and then reject
defered.reject(reasonDerived);
});
return defered.promise;
}
Here is a simplified and less explicit version (Credit: Benjamin Gruenbaum):
var cached = null;
function getCurrentUserData(userId){
return cached = cached || $http.get('myurl').then(function(result){
if(isNotValid(result)) return $q.reject(reason); // alternatively `throw`
return transformToDto(result);
}, function(response){
if(checkforsomethingonstatusandreject)
return $q.reject('Errored Out')
//do some actions to notify the error scenarios and return default actions
return someDefaults; });
}
You can of course return $q.reject(derivedReason) here rather than returning the reason and transform it based on further checks, the idea is caching the promise rather than the value. This also has the advantage of not making multiple http requests if the method is called before it returns once.
Now you could always do:-
getCurrentUserData(userid).then(function(user){
}).catch(function(error){
});
Promises can be chained through as well. So you could also do this:-
return $resource('myresource').$promise.then(function(result){
//Do some mapping and return mapped data
return mappedResult;
});

A+ promises should offer a static method:
Promise.resolve(val);
...which generates a pre-resolved promise. You should return one of these if your promises library offers this. Great. I do this frequently to avoid interface duplication.

$q.all(p1, p2).then() appears to fire before both promises are resolved

Are $resource promises supposed to be compatible with $q? Perhaps I am using it wrong. I have two $resource objects that I might load like:
$rootScope.foos = Res1.query(); // [ foo1, foo2...]
$rootScope.bars = Res2.query(); // [ bar1, bar2...]
I need to broadcast an event when both (and only both) queries have arrived. So I'm using $q.all like this:
$q.all([$rootScope.foos.$promise, $rootScope.bars.$promise])
.then(function(){
// sometimes $rootScope.bars == []
$rootScope.$broadcast('foos_and_bars!');
});
Event listeners find that $rootScope.bars is empty / [] (when a second later it has data)
Update in response to #Daiwei and #ExpertSystem
Ok here's the JSFiddle for this: Reproducing JSFiddle
The console shows $q.all() being returned before the promises are resolved. I suppose either it's being used wrong or it is an angular bug having to do with any of [$resource, $scope, $http, etc.]...

UPDATE:
It turned out to be a version-conflict issue.
The solution provided below is working with later versions (tested with 1.2.8), but is totally redundant.
See Purrell's answer for more info.
Quoting the docs on $resource:
It is important to realize that invoking a $resource object method immediately returns an empty reference (object or array depending on isArray). Once the data is returned from the server the existing reference is populated with the actual data. This is a useful trick since usually the resource is assigned to a model which is then rendered by the view. Having an empty object results in no rendering, once the data arrives from the server then the object is populated with the data and the view automatically re-renders itself showing the new data. This means that in most cases one never has to write a callback function for the action methods.
[...]
The Resource instances and collection have these additional properties:
$promise: ...
$resolved: ...
Your Res1 and Res2 are $resource objects (not instances) and (as per the docs) invoking the query() method "immediately returns an empty reference" (in your case an array). (UPD: The returned arrays still have the additional properties though.)
As the docs suggest, most of the time you just assign those empty references to the Scope and forget about them (once the data is fetched the model (and subsequently the view) will get automagically updated).
Yet, if you do have reasons for wanting to get explicitly informed as soon as the data is there, you could use two "real" promises:
// Create two deferred objects
var deferred1 = $q.defer();
var deferred2 = $q.defer();
// As soon as the data arrives, resolve the deferred objects
$rootScope.foos = Res1.query(function () { deferred1.resolve(); });
$rootScope.bars = Res2.query(function () { deferred2.resolve(); });
// Combine their promises using `$q.all()`
$q.all([deferred1.promise, deferred2.promise]).then(...);
See, also, this working example.
UPDATE
Since the returned arrays also have the $promise property, the more concise version is:
$rootScope.foos = Res1.query();
$rootScope.bars = Res2.query();
$q.all([$rootScope.foos.$promise, $rootScope.bars.$promise]).then(...);
Working demo

This turned out to be an accidental version conflict between ngResource and angular. Angular was on 1.2.6 but ngResource was on 1.2.3.
The originally posted fiddle was not working for a similar but different ngResource version problem, namely that it was using an old ngResource version which didn't use to expose the $promise objects (although they were still used under the hood). In later versions of ngResource the promises are exposed so they can be used for things like this.
#ExpertSystem's suggestion is good but would not have fixed the issue. I did try with $q.defer() promises prior and it has the same issue when the version conflict is present. It does work in the absence of the version conflict, but then it is unnecessary, no need to create your own promises when they are already given to you for convenience.
Here's the working JSFiddle (at angular 1.2.1 but that is good enough for ngResource)

After your $q.all()...you can use .spread(function(prom1, prom2){} instead to run once all promises are returned. With the way you are currently doing it the .then will fire every time one item in the array returns, hence why sometimes one of the items from your array are empty.

jQuery deferred: "then" ... "when" ... where's the "join" or "blockUntilDone"?

I'm using the JayData.js library. It works quite well. However, I have a few situations where I've got a toArray() call deep in the function tree. Rather than trying to access my "busy" signal from there, I'd just as soon have the method block. Is that possible? I'm picturing something like "context.Groups.toArray(myObservableVar).block()".
Update 1: It appears that the JayData library returns a jQuery deferred object judging from the use of "then" and "when" operators on the return value. Is there a corresponding method to "join" -- meaning wait for the finish?

Indeed JayData toArray() (and all relevant data returning or saving/updating method) implements jQuery deferred. As from 1.0.5 you have to include the JayDataModules/deferred.js in order to this functionality to work.
For your use case $.when might be an answer:
var customers = context.Customers.toArray();
var products = context.Products.toArray();
var suppliers = context.Suppliers.toArray();
$.when(customers, products, suppliers).then(function(customers, products, suppliers) {
//we have everything here
//notice the the parameter names are shadowed as the var customers variable only
//holds a promise not the value
customers.forEach( ... );
products[12].ProductName = "X";
});

A blockUntilDone() method would go against the principles of deferred execution and continuations. JayData's toArray() is asynchronous because it is designed not to block the caller.
If you want this kind of code:
// Initialize context and groups...
var arrayOfGroups = context.Groups.toArray(); // Want synchronous behavior.
// Do something with 'arrayOfGroups'...
Trying to block until the deferred is resolved is not the solution. Move the last part of your code into a callback passed to toArray() instead:
// Initialize context and groups...
context.Groups.toArray(function(arrayOfGroups) {
// Do something with 'arrayOfGroups'...
});
Alternatively, bind to the returned promise with done() or then():
context.Groups.toArray().done(function(arrayOfGroups) {
// Do something with 'arrayOfGroups'...
});