We Keep Coding

Sequelize.js afterUpdate hook pass changed values

I'm building a node.js app and I'm evaluating Sequelize.js for persistent objects. One thing I need to do is publish new values when objects are modified. The most sensible place to do this would seem to be using the afterUpdate hook.
It almost works perfectly, but when I save an object the hook is passed ALL the values of the saved object. Normally this is desirable, but to keep the publish/subscribe chatter down, I would rather not republish fields that weren't saved.
So for instance, running the following
tasks[0].updateAttributes({assignee: 10}, ['assignee']);
Would automagically publish the new value for the assignee for that task on the appropriate channel, but not republish any of the other fields, which didn't change.
The closest I've come is with an afterUpdate hook:
Task.hook('afterUpdate', function(task, fn) {
Object.keys(task).forEach(function publishValue(key) {
pubSub.publish('Task:'+task.id+'#'+key, task[key]);
});
return fn();
});
which is pretty straightforward, but since the 'task' object has all the fields, I'm being unnecessarily noisy. (The pubSub system is ignorant of previous values and I'd like to keep it that way.)
I could override the setters in the task object (and all my other objects), but I would prefer not to publish until the object is saved. The object to be saved doesn't seem to have the old values (that I can find), so I can't base my publish on that.
So far the best answer I've come up with from a design standpoint is to tweak one line of dao.js to add the saved values to the returned object, and use that in the hook:
self.__factory.runHooks('after' + hook, _.extend({}, result.values, {savedVals: args[2]} ), function(err, newValues) {
Task.hook('afterUpdate', function(task, fn) {
Object.keys(task.savedVals).forEach(function publishValue(key) {
pubSub.publish('Task:'+task.id+'#'+key, task[key]);
});
return fn();
});
Obviously changing the Sequelize library is not ideal from a maintenance standpoint.
So my question is twofold: is there a better way to get the needed information to my hook without modifying dao.js, or is there a better way to attack my fundamental requirement?
Thanks in advance!

There is not currently. In the implementation for exactly what you describe we simply had to implement logic to compare old and new values, and if they differed, assume that they have changed.

Breeze JS: Is there a way to query entities from data.results?

I have a Breeze web api controller, with methods that accept parameters and do some work, filtering, sorting, etc, on the server.
On the querySucceeded, I'd like to do further querying to data.results. Is there a way to accomplish this? I got this working by exporting/importing data.results to a local manager, and do the projection from there. The projection is needed in order to use the observable collection in a vendor grid control.
var query = datacontext.EntityQuery.from("GetActiveCustomers")
.withParameters({ organizationID: "1" })
.toType("Customer")
.expand("Organization")
.orderBy('name');
var queryProjection = query
.select("customerID, organizationID, name, isActive, organization.name");
return manager.executeQuery(query)
.then(querySucceeded)
.fail(queryFailed);
function querySucceeded(data) {
var exportData = manager.exportEntities(data.results);
var localManager = breeze.EntityManager.importEntities(exportData);
var resultProjection = localManager.executeQueryLocally(queryProjection);
//This is the way I came up with to query data.results (exporting/importing the result entities to a local manager)
//Is there a better way to do this? Querying directly data.results. Example: data.results.where(...).select("customerID, organizationID...)
if (collectionObservable) {
collectionObservable(resultProjection);
}
log('Retrieved Data from remote data source',
data, true);
}

You've taken an interesting approach. Normally a projection returns uncacheable objects, not entities. But you casted this to Customer (with the toType clause) which means you've created PARTIAL Customer entities with missing data.
I must hope you know what you are doing and have no intention of saving changes to these customer entities while they remain partial else calamity may ensue.
Note that when you imported the selected Customers to the "localManager" you did not bring along their related Organization entities. That means an expression such as resultProjection[0].organization will return null. That doesn't seem correct.
I understand that you want to hold on to a subset of the Customer partial entities and that there is no local query that could select that subset from cache because the selection criteria are only fully known on the server.
I think I would handle this need differently.
First, I would bury all of this logic inside the DataContext itself; the purpose of a DataContext is to encapsulate the details of data access so that callers (such as ViewModels) don't have to know internals. The DataContext is an example of the UnitOfWork (UoW) pattern, an abstraction that helps isolate the data access/manipulation concerns from ViewModel concerns.
Then I would store it either in a named array of the DataContext (DC) or of the ViewModel (VM), depending upon whether this subset was of narrow or broad interest in the application.
If only the VM instance cares about this subset, then the DC should return the data.results and let the VM hold them.
I do not understand why you are re-querying a local EntityManager for this set nor for why your local query is ALSO appling a projection ... which would return non-entity data objects to the caller. What is wrong with returning the (partial) Customer entities.
It seems you intend to further filter the subset on the client. Hey ... it's a JavaScript array. You can call stuffArray.filter(filterFunction).
Sure that doesn't give you the Breeze LINQ-like query syntax ... but do you really need that? Why do you need ".select" over that set?
If that REALLY is your need, then I guess I understand why you're dumping the results into a separate EntityManager for local use. In that case, I believe you'll need more code in your query callback method to import the related Organization entities into that local EM so that someCustomer.organization returns a value. The ever-increasing trickiness of this approach makes me uncomfortable but it is your application.
If you continue down this road, I strongly encourage you to encapsulate it either in the DC or in some kind of service class. I wouldn't want my VMs to know about any of these shenanigans.
Best of luck.
Update 3 Oct 2013: Local cache query filtering on unmapped property
After sleeping on it, I have another idea for you that eliminates your need for a second EM in this use case.
You can add an unmapped property to the client-side Customer entity and set that property with a subset marker after querying the "GetActiveCustomers" endpoint on the server; you'd set the marker in the query callback.
Then you can compose a local query that filters on the marker value to ensure you only consider Customer objects from that subset.
Reference the marker value only in local queries. I don't know if a remote query filtering on the marker value will fail or simply ignore that criterion.
You won't need a separate local EntityManager; the Customer entities in your main manager carry the evidence of the server-side filtering. Of course the server will never have to deal with your unmapped property value.
Yes, a breeze local query can target unmapped properties as well as mapped properties.
Here's a small demonstration. Register a custom constructor like this:
function Customer() { /* Custom constructor ... which you register with the metadataStore*/
// Add unmapped 'subset' property to be queried locally.
this.subset = Math.ceil(Math.random() * 3); // simulate values {1..3}
}
Later you query it locally. Here are examples of queries that do and do not reference that property:
// All customers in cache
var x = breeze.EntityQuery.from("Customers").using(manager).executeLocally();
// All customers in cache whose unmapped 'subset' property === 1.
var y = breeze.EntityQuery.from("Customers")
.where("subset", 'eq', 1) // more criteria; knock yourself out
.using(manager).executeLocally();
I trust you'll know how to set the subset property appropriately in your callback to our "GetActiveCustomers" query.
HTH

Once you queried for some data breeze stores those entities on the local memory.
All you have to do is query locally when you need to filter the data some more.
You do this by specifying for the manager to query locally :
manager.executeQueryLocally(query);
Because querying from the database is done asynchronously you have to make sure that you retrieve from the local memory only if there is something there. Follow the promises.

Where does data returned by ember-data 'live'?

ya'll I have a bit of a structural/procedural question for ya.
So I have a pretty simple ember app, trying to use ember-data and I'm just not sure if I'm 'doing it right'. So the user hits my index template, I grab their location coordinates and encode a hash of it (that part works). Then on my server I have a db that stores 'tiles' named after there hash'd coords (if i hit my #/tiles/H1A2S3H4E5D route I get back properly formatted JSON).
What I would like to happen next, if to display each of the returned tiles to the user on the bottom of the first page (like in a partial maybe? if handlebars does that).
I have a DS.Model for the tiles, if I hard code the Hash'd cords into a App.find(H1A2S3H4E5D); I can see my server properly responding to the query. However, I cannot seem to be able to figure out how to access the returned JSON object, or how to display it to the user.
I did watch a few tutorial videos but they all seem to be outdated with the old router.
Mainly I would like to know:
1. Where does the information returned by App.find(); live & how to access it?
2. what is the 'correct' way to structure my templates/views to handle this?
3. how should I pass that id (the hash'd coords) to App.find? as a global variable? or is there a better way?
the biggest problem(to me) seems to be that the id I search by doesn't exist until the user hit the page tho first time. (since its dynamically generated) so I can't just grab it when the page loads.
I can post a fiddle if required, but I'm looking for more of a conceptual/instructional answer rather then some one to just write my code for me

I'm still learning a lot with Ember as well, but this is my understanding. When you follow the guides and the tutorials out there, you'll have something like this:
App.TileController = Ember.ObjectController.extend();
App.TileRoute = Ember.Route.extend({
setupController: function(controller) {
controller.set('content', App.Tile.find(MYHASH));
}
});
What it does is set the special content object to the result. So since we're declaring an object controller, and calling find with a parameter, it knows that a single result is expected. So a view & template that follow the naming convention of Tile will be loaded. And in there you can access properties on the Tile object:
<p>{{lat}}</p><p>{{lng}}</p>
I have to admit that this feels a bit mystical at times. The core to it is all in the naming convention. You need to be pretty specific in how you name all your various controllers, routes, etc. Once that's nailed down, it's a matter of binding what data you want to the controller's content.
1) Aside from the generic answer of "in memory", the .find() calls live where ever you return it to. Generally speaking, this is meant to be set on a 'content' property of a controller.
2) I more or less answered this, but generally speaking you take the name of your route, and base it off that. So for a route TileRoute, you have:
TileController = Ember.ObjectController.extend
Tile = DS.Model.extend
TileView = Ember.View.extend
tile.handlebars
I generally store all my handlebars files in a templates/ folder. If you nest them deeper, just specify the path in your view object:
App.TileView = Ember.View.extend({
templateName: "tiles/show"
});
3) This really depends on your app. Generally speaking its better for the id to be either obtained from the URL, or constructed locally in a function. Since you are encoding a hash, i imagine you're doing this in a function, and then calling find. I do something a bit similar for an Array controller.
I don't know at what point you are generating a hash, so let's say it's onload. You should be able to generate the hash just in the setupController function.
App.TileRoute = Ember.Route.extend({
generateHashBasedOnCoords: function() {
// ...
},
setupController: function(controller) {
var MYHASH = this.generateHashBasedOnCoords();
controller.set('content', App.Tile.find(MYHASH));
}
});
I hope that helps.

I believe that you can make use of the data binding in ember and basically have an array controller for tiles and set the content initially to an empty array. Then we you get back your response do a App.find() and set the content of the tiles controller with the data that is returned. This should update the view through the data binding. (Very high level response)
The data itself is stored in a store that is setup with ember data. You access it with the same method you are using the model methods App.Tile.find() ect. It checks to see if the data that is needed is in the store if so it returns the data otherwise it makes a call to the api to get the data.

Using Visualsearch.js in a backbone.js framework

First of all, thanks to the guys of DocumentCloud for releasing those two super-useful tools.
Here goes my question(s):
I'm trying to use visulasearch.js in a backbone.js app.
In my app I have a basic index.html and a myapp.js javascript file wich contains the main application done with backbone.js
I use CouchDB as data storage, and I successfully can retrieve in a restful way all the data to be put in the collection.
I must retrieve the search query given by visualsearch.js and use it to filter a collection.
I surely need a view for the searchbox, to trigger an event when enter is hit, but..
Should I initialze the searchbox externally to myapp.js, within an additional js file or my index.html page (as suggested in the visualsearch mini.tutorial)?
Or I should initialize it within the searchbox view (myapp.js)? This latter solution seems to be too tricky (it was what I was trying to do, but even when I succeed, it's too complicated and I lost the simplicity of bacbone mvc).
Let's say I succeed in retrieving the search string as a JSON object like {name:'Fat DAvid', address:'24, slim st', phone:'0098876534287'}. Once done that, which function can I use to retrieve, in the collection, only the models whose fields match with the given string. I understand that I should do a map or a filter, but those function seems to serve natively for slightly different tasks.
a. is it really the best way to filter results? It charges the client (which must filter the results), while making a new query (a view or a filter) to CouchDB would be quite simple and, considered the small amount of data and the low access rate to the site, not so expensive. However, making all the filtering action client-side, it's much simpler than making new view(or list or filters) in CouchDB and linking it the the backbone.js view

You can initialize your VisualSearch.js search box right in your myapp.js. Just make sure you keep a reference to it so you can then extract out the facets and values later.
For example:
var visualSearch = VS.init({...})
// Returns the unstructured search query
visualSearch.searchBox.value()
// "country: "South Africa" account: 5-samuel title: "Pentagon Papers""
// Returns an array of Facet model instances
visualSearch.searchQuery.facets()
// [FacetModel<country:"South Africa">,
// FacetModel<account:5-samuel>,
// FacetModel<title:"Pentagon Papers">]
If you have these models in a Backbone collection, you can easily perform a filter:
var facets = visualSearch.searchQuery.models;
_.each(facets, function(facet) {
switch (facet.get('category')) {
case 'country':
CountriesCollection.select(function(country) {
return country.get('name') == facet.get('value');
});
break;
// etc...
}
});

Persisting & loading metadata in a backbone.js collection

I have a situation using backbone.js where I have a collection of models, and some additional information about the models. For example, imagine that I'm returning a list of amounts: they have a quantity associated with each model. Assume now that the unit for each of the amounts is always the same: say quarts. Then the json object I get back from my service might be something like:
{
dataPoints: [
{quantity: 5 },
{quantity: 10 },
...
],
unit : quarts
}
Now backbone collections have no real mechanism for natively associating this meta-data with the collection, but it was suggested to me in this question: Setting attributes on a collection - backbone js that I can extend the collection with a .meta(property, [value]) style function - which is a great solution. However, naturally it follows that we'd like to be able to cleanly retrieve this data from a json response like the one we have above.
Backbone.js gives us the parse(response) function, which allows us to specify where to extract the collection's list of models from in combination with the url attribute. There is no way that I'm aware of, however, to make a more intelligent function without overloading fetch() which would remove the partial functionality that is already available.
My question is this: is there a better option than overloading fetch() (and trying it to call it's superclass implementation) to achieve what I want to achieve?
Thanks

Personally, I would wrap the Collection inside another Model, and then override parse, like so:
var DataPointsCollection = Backbone.Collection.extend({ /* etc etc */ });
var CollectionContainer = Backbone.Model.extend({
defaults: {
dataPoints: new DataPointsCollection(),
unit: "quarts"
},
parse: function(obj) {
// update the inner collection
this.get("dataPoints").refresh(obj.dataPoints);
// this mightn't be necessary
delete obj.dataPoints;
return obj;
}
});
The Collection.refresh() call updates the model with new values. Passing in a custom meta value to the Collection as previously suggested might stop you from being able to bind to those meta values.

This meta data does not belong on the collection. It belongs in the name or some other descriptor of the code. Your code should declaratively know that the collection it has is only full of quartz elements. It already does since the url points to quartz elements.
var quartzCollection = new FooCollection();
quartzCollection.url = quartzurl;
quartzCollection.fetch();
If you really need to get this data why don't you just call
_.uniq(quartzCollecion.pluck("unit"))[0];