We Keep Coding

IndexedDB's callbacks not being executed inside the 'fetch' event of a Service Worker

I'm trying to do a couple of things in the IndexedDB database inside the 'fetch' event of a service worker, when the aplication asks the server for a new page. Here's what I'm going for:
Create a new object store (they need to be created dynamically, according to the data that 'fetch' picks up);
Store an element on the store.
Or, if the store already exists:
Get an element from the store;
Update the element and store it back on the store.
The problem is that the callbacks (onupgradeneeded, onsuccess, etc) never get executed.
I've been trying with the callbacks inside of each other, though I know that may not be the best approach. I've also tried placing an event.waitUntil() on 'fetch' but it didn't help.
The 'fetch' event, where the function registerPageAccess is called:
self.addEventListener('fetch', function (event) {
event.respondWith(
caches.match(event.request)
.then(function (response) {
event.waitUntil(function () {
const nextPageURL = new URL(event.request.url);
if (event.request.destination == 'document') {
if (currentURL) {
registerPageAccess(currentURL, nextPageURL);
}
currentURL = nextPageURL;
}
}());
/*
* some other operations
*/
return response || fetch(event.request);
})
);
});
registerPageAccess, the function with the callbacks.
I know it's plenty of code, but just look at secondRequest.onupgradeneeded in the 5th line. It is never executed, let alone the following ones.
function registerPageAccess(currentPageURL, nextPageURL) {
var newVersion = parseInt(db.version) + 1;
var secondRequest = indexedDB.open(DB_NAME, newVersion);
secondRequest.onupgradeneeded = function (e) {
db = e.target.result;
db.createObjectStore(currentPageURL, { keyPath: "pageURL" });
var transaction = request.result.transaction([currentPageURL], 'readwrite');
var store = transaction.objectStore(currentPageURL);
var getRequest = store.get(nextPageURL);
getRequest.onsuccess = function (event) {
var obj = getRequest.result;
if (!obj) {
// Insert element into the database
console.debug('ServiceWorker: No matching object in the database');
const addRes = putInObjectStore(nextPageURL, 1, store);
addRes.onsuccess = function (event) {
console.debug('ServiceWorker: Element was successfully added in the Object Store');
}
addRes.onerror = function (event) {
console.error('ServiceWorker error adding element to the Object Store: ' + addRes.error);
}
}
else {
// Updating database element
const updRes = putInObjectStore(obj.pageURL, obj.nVisits + 1, store);
updRes.onsuccess = function (event) {
console.debug('ServiceWorker: Element was successfully updated in the Object Store');
}
updRes.onerror = function (event) {
console.error('ServiceWorker error updating element of the Object Store: ' + putRes.error);
}
}
};
};
secondRequest.onsuccess = function (e) {
console.log('ServiceWorker: secondRequest onsuccess');
};
secondRequest.onerror = function (e) {
console.error('ServiceWorker: error on the secondRequest.open: ' + secondRequest.error);
};
}
I need a way to perform the operations in registerPageAccess, which involve executing a couple of callbacks, but the browser seems to kill the Service Worker before they get to occur.

All asynchronous logic inside of a service worker needs to be promise-based. Because IndexedDB is callback-based, you're going to find yourself needing to wrap the relevant callbacks in a promise.
I'd strongly recommend not attempting to do this on your own, and instead using one of the following libraries, which are well-tested, efficient, and lightweight:
idb-keyval, if you're okay with a simple key-value store.
idb if you're need the full IndexedDB API.
I'd also recommend that you consider using the async/await syntax inside of your service worker's fetch handler, as it tends to make promise-based code more readable.
Put together, this would look roughly like:
self.addEventListener('fetch', (event) => {
event.waitUntil((async () => {
// Your IDB cleanup logic here.
// Basically, anything that can execute separately
// from response generation.
})());
event.respondWith((async () => {
// Your response generation logic here.
// Return a Response object at the end of the function.
})());
});

Meteor - how do I make this "reactive" using Deps?

On my client side, I display a list of users and a small chart for each user's points stored in the DB (using jQuery plugin called sparklines).
Drawing the chart is done on Template.rendered method
// client/main.js
Template.listItem.rendered = function() {
var arr = this.data.userPoints // user points is an array of integers
$(this.find(".chart")).sparkline(arr);
}
Now I have a Meteor method on the server side, that is called on a regular basis to update the the user points.
Meteor.methods({
"getUserPoints" : function getUserPoints(id) {
// access some API and fetch the latest user points
}
});
Now I would like the chart to be automatically updated whenever Meteor method is called. I have a method on the template that goes and calls this Meteor method.
Template.listItem.events({
"click a.fetchData": function(e) {
e.preventDefault();
Meteor.call("getUserPoints", this._id);
}
});
How do I turn this code into a "reactive" one?

You need to use reactive data source ( Session, ReactiveVar ) together with Tracker.
Using ReactiveVar:
if (Meteor.isClient) {
Template.listItem.events({
"click a.fetchData": function(e) {
e.preventDefault();
var instance = Template.instance();
Meteor.call("getUserPoints", this._id, function(error, result) {
instance.userPoints.set(result)
});
}
});
Template.listItem.created = function() {
this.userPoints = new ReactiveVar([]);
};
Template.listItem.rendered = function() {
var self = this;
Tracker.autorun(function() {
var arr = self.userPoints.get();
$(self.find(".chart")).sparkline(arr);
})
}
}
Using Session:
if (Meteor.isClient) {
Template.listItem.events({
"click a.fetchData": function(e) {
e.preventDefault();
Meteor.call("getUserPoints", this._id, function(error, result) {
Session.set("userPoints", result);
});
}
});
Template.listItem.rendered = function() {
var self = this;
Tracker.autorun(function() {
var arr = Session.get("userPoints");
$(self.find(".chart")).sparkline(arr);
})
}
}
Difference between those implementation :
A ReactiveVar is similar to a Session variable, with a few
differences:
ReactiveVars don't have global names, like the "foo" in
Session.get("foo"). Instead, they may be created and used locally, for
example attached to a template instance, as in: this.foo.get().
ReactiveVars are not automatically migrated across hot code pushes,
whereas Session state is.
ReactiveVars can hold any value, while Session variables are limited
to JSON or EJSON.
Source
Deps is deprecated, but still can be used.

The most easily scalable solution is to store the data in a local collection - by passing a null name, the collection will be both local and sessional and so you can put what you want in it and still achieve all the benefits of reactivity. If you upsert the results of getUserPoints into this collection, you can just write a helper to get the appropriate value for each user and it will update automatically.
userData = new Meteor.Collection(null);
// whenever you need to call "getUserPoints" use:
Meteor.call("getUserPoints", this._id, function(err, res) {
userData.upsert({userId: this._id}, {$set: {userId: this._id, points: res}});
});
Template.listItem.helpers({
userPoints: function() {
var pointsDoc = userData.findOne({userId: this._id});
return pointsDoc && pointsDoc.points;
}
});
There is an alternative way using the Tracker package (formerly Deps), which would be quick to implement here, but fiddly to scale. Essentially, you could set up a new Tracker.Dependency to track changes in user points:
var pointsDep = new Tracker.Dependency();
// whenever you call "getUserPoints":
Meteor.call("getUserPoints", this._id, function(err, res) {
...
pointsDep.changed();
});
Then just add a dummy helper to your listItem template (i.e. a helper that doesn't return anything by design):
<template name="listItem">
...
{{pointsCheck}}
</template>
Template.listItem.helpers({
pointsCheck: function() {
pointsDep.depend();
}
});
Whilst that won't return anything, it will force the template to rerender when pointsDep.changed() is called (which will be when new user points data is received).

Struggling with Q.js promises

I am sure I am missing something obvious but I can't seem to make heads or tails of this problem. I have a web page that is being driven by javascript. The bindings are being provided by Knockout.js, the data is coming down from the server using Breeze.js, I am using modules tied together with Require.js. My goal is to load the html, load the info from Breeze.js, and then apply the bindings to show the data to the user. All of these things appear to be happening correctly, just not in the correct order which is leading to weird binding errors. Now on to the code.
I have a function that gets called after the page loads
function applyViewModel() {
var vm = viewModel();
vm.activate()
.then(
applyBindings(vm)
);
}
This should call activate, wait for activate to finish, then apply bindings....but it appears to be calling activate, not waiting for it to finish and then runs applybindings.
activate -
function activate() {
logger.log('Frames Admin View Activated', null, 'frames', false);
return datacontext.getAllManufacturers(manufacturers)
.then(function () {
manufacturer(manufacturers()[0]);
}).then(function () {
datacontext.getModelsWithSizes(modelsWithSizes, manufacturers()[0].manufacturerID())
.then(datacontext.getTypes(types));
});
}
datacontext.getAllManufacturers -
var getAllManufacturers = function (manufacturerObservable) {
var query = entityQuery.from('Manufacturers')
.orderBy('name');
return manager.executeQuery(query)
.then(querySucceeded)
.fail(queryFailed);
function querySucceeded(data) {
if (manufacturerObservable) {
manufacturerObservable(data.results);
}
log('Retrieved [All Manufacturer] from remote data source',
data, true);
}
};
datacontext.getModelsWithSizes -
var getModelsWithSizes = function (modelsObservable, manufacturerId) {
var query = entityQuery.from('Models').where('manufactuerID', '==', manufacturerId)
.orderBy('name');
return manager.executeQuery(query)
.then(querySucceeded)
.fail(queryFailed);
function querySucceeded(data) {
if (modelsObservable) {
for (var i = 0; i < data.results.length; i++) {
datacontext.getSizes(data.results[i].sizes, data.results[i].modelID());
// add new size function
data.results[i].addNewSize = function () {
var newValue = createNewSize(this.modelID());
this.sizes.valueHasMutated();
return newValue;
};
}
modelsObservable(data.results);
}
log('Retrieved [Models With Sizes] from remote data source',
data, false);
}
};
Any help on why this promise isn't working would be appreciated, as would any process to figure it out so I can help myself the next time I run into this.

A common mistake when working with promises is instead of specifying a callback, you specify the value returned from a callback:
function applyViewModel() {
var vm = viewModel();
vm.activate()
.then( applyBindings(vm) );
}
Note that when the callback returns a regular truthy value (number, object, string), this should cause an exception. However, if the callback doesn't return anything or it returns a function, this can be tricky to locate.
To correct code should look like this:
function applyViewModel() {
var vm = viewModel();
vm.activate()
.then(function() {
applyBindings(vm);
});
}

New to asynchronous programming in JavaScript. Any advice?

Asynchronous callbacks are great but when one callback depends on the result of another I have callbacks with api calls that have callbacks, and so on.
apiCall(function () { apiCall(function () { apiCall(function () ...
I can name the callback functions instead of including them inline. That looks prettier and has less nesting but I do not find it any easier to read.
Here is an example. I need to query the local sqlite database, use the result to query a server, then use the response to update the local database.
function sync() {
db.transaction(
function (transaction) {
execute(transaction, 'SELECT max(server_time) AS server_time FROM syncs;', [],
function (transaction, results) { // Query results callback
var t = results.rows.item(0).server_time;
$.post('sync.json', { last_sync_time: (t || '1980-01-01') },
function (data) { // Ajax callback
db.transaction(
function(transaction) {
$(data.thing).each(function () {
var thing = new Thing(this.thing);
thing.insert(transaction);
});
});
});
});
});
}
Is there a way to untagle this (other than naming the callbacks)?

I think you're too quick to discard un-nesting things by naming your functions rather than writing them inline. This is pretty much the only way to clean up that mess.
Instead of:
do_a(
function () {
// more nesting...
}
);
Use names to provide a bit of clarity and purpose to each function:
function on_a_complete() {
}
do_a(on_a_complete);

I think you answered your own question. Naming your callbacks is really the only way to clean this up anymore. Something like:
execute(transaction, 'SELECT max(server_time) AS server_time FROM syncs;', [],handleLocalResults, errorHandler);
handleLocalResults = function (transaction, results)...
handleServerResults = func...

Naming the callbacks is one thing you can do, but the other thing that you need to do is to have non-overlapping SQL transactions. (1)
Your first transaction should be like this:
// The whole thing starts here
db.transaction(selectTimeCB, null, ajaxPost);
The transaction starts with the callback to select the time, and when the transaction is complete, make a call to the ajaxPost operation.
// Initial transaction to get server_time
var selectTimeCB = function(t) {
var query = 'SELECT max(server_time) AS server_time FROM syncs';
t.executeSql(query, [], postLastSyncCB);
};
// This saves the results from the above select, and nothing else.
var server_time;
var postLastSyncCB = function(t, results) {
server_time = results.rows.item(0).server_time;
};
var ajaxPost = function() {
$.post('sync.json', { last_sync_time: (server_time || '1980-01-01') }, nextDbTransaction);
};
If you have overlapped SQL transactions, that can really kill the performance of the database. I recently did a test of 200 mixed transactions on a database with a bit over 500 rows, and found that keeping the transactions separate reduced a run time of over 90 seconds to 3-5 seconds.

Dojo fetch, how to wait on two simultaneous async fetches?

Hey guys, im not well versed in dealing with asynchronous design patterns, and im having a problem writing a script that does two async data fetches.
Im using Dojo.data.api.Read.Fetch() to make two fetch() calls from seperate databases. The reulsts come back asynchronously. However, I have to cross reference the results, so i want my script to continue once BOTH async fetches are complete. I dont know how to do this, and therein lies the problem.
I am aware of the fetch's onComplete field and how to use it, BUT the best case solution i see there is to call the second fetch in the onComplete of the first fetch. I would like to do these fetches at the same time. Is there a way to do this?
Here's the current structure of my program for illustration purposes:
this.dict1.fetch({query:"blahblahblah", onComplete: function(items) { something here? }});
this.dict2.fetch({query:"blahblahbleh", onComplete: function(items) { or maybe something here? }});
this.orMaybeDoSomethingAfterBothFetches()
Any help would be greatly appreciated!

You could create dojo.Deferreds for each of the fetches and then use dojo.DeferredList and add the deferreds to it - see here. This solution allows you to take advantage of adding 'n' functions to the list of functions you want to call. It also takes advantage of all the dojo.Deferred's callback and errBack functionality.
var fetch1 = new dojo.Deferred();
fetch1.addCallback(this.dict1.fetch...);
var fetch2 = new dojo.Deferred();
fetch2.addCallback(this.dict2.fetch...);
var allFuncs = new dojo.DeferredList([fetch1, fetch2]);
var doStuffWhenAllFuncsReturn = function() {...};
allFuncs.addCallback(doStuffWhenAllFuncsReturn);

// this is a variation of a function I have answered quite a few similar questions on SO with
function collected(count, fn){
var loaded = 0;
var collectedItems = [];
return function(items){
collectedItems = collectedItems.concat(items);
if (++loaded === count){
fn(collectedItems);
}
}
}
var collectedFn = collected(2, function(items){
//do stuff
});
this.dict1.fetch({query:"blahblahblah", onComplete: collectedFn);
this.dict2.fetch({query:"blahblahbleh", onComplete: collectedFn);
An alternative solution is
var store = {
exec: function(){
if (this.items1 && this.items2) {
// do stuff with this.items1 and this.items2
}
}
};
this.dict1.fetch({query:"blahblahblah", onComplete: function(items) {
store.items1 = items;
store.exec();
});
this.dict2.fetch({query:"blahblahbleh", onComplete: function(items) {
store.items2 = items;
store.exec();
});