We Keep Coding

Sync collection changes with the backend

I use Backbone.js and I have a collections of models. This collection is retrieved and displayed on the front-end. On the front-end, I want the user to remove and add new models to the collection.
When the user is finished and he clicked "save", I want the entire collection to be updated. Meaning that when clicking 'save', the collection is synced (somehow). Added models are saved and removed models are deleted.
If I manipulate the collection by removing and adding models, and then use ex:
this.collection.sync()
Will it remove and add models?

There are at least 2 ways to achieve this.
Make an API endpoint to manage the models
When adding/updating a model, save the model directly with .save and when a model is removed, call .destroy on it.
The collection also has a .create function which adds the new model to it and saves it at the same time.
Best thing to do everything in one request.
Not always. The collection could be big and the changes rather small, so exchanging 100 objects with the server each time instead of X small requests to add, delete or update a model within the list.
Pros
Reusable endpoints to manage individual models
Light data transfer, faster than sending the whole collection (only changes are sent through partial updates)
Possible custom behaviour for each action
Easy real-time update implementation
Cons
More requests when doing a lot of changes
Needs models to have ids
Additional endpoints to manage
Not meant for bulk operations (save all models at once)
Put the collection's models array into a model
Collections are not meant to be saved. Instead, put the models of the collection into a model which communicates with an API endpoint. This endpoint should expect an object with an array field, which can serve to replace the collection server-side.
var CollectionModel = Backbone.Model.extend({
urlRoot: "collection/endpoint/"
});
var myModel = new CollectionModel();
// ...sometime later...
myModel.save({
arrayAttribute: yourCollection.toJSON()
}, { patch: true });
Pros
One endpoint; always the same call to the API
Easy to implement; just in one place where the save occurs
Cons
All models are transferred on every request regardless of changes
Could be slow if the collection is big
Collection's .sync function is only a proxy to Backbone.sync and do nothing without the correct parameters. It is only used internally within .fetch (line 1055) and isn't meant to be used directly, unless adding a custom behavior.

Backbone.js - decoupling a data source?

I have this Backbone model:
graphModel
attributes
country
indicator
year
With a corresponding view:
graphView
render()
this.model.get(...)
The application also has a general datasource to which csv data is loaded:
dataSource
indicators
indicatorA
country
year
indicatorB
countries
years
Every time the model attributes are changed, I'd like to check if data for that combination of attributes is loaded, before firing change events.
My question is: What is a good way of decoupling the data source from the Backbone model and view so that I can easily try with injected mock data?

I'm a little unclear when you're talking about the application having a data source populated with CSV data whether you're speaking about the server side of an app that Backbone then talks to or something client side.
But either way. I can speak to our experiences on some of this decoupling stuff. We often build models and use Backbone's ability to have "default" values like this example from the Backbone docs:
var Meal = Backbone.Model.extend({
defaults: {
"appetizer": "caesar salad",
"entree": "ravioli",
"dessert": "cheesecake"
}
});
With a model pre-populated like that we can then tie a view to the model and render that data to the page. Similarly, if you have any other source you can think of for easily getting the data you want (a canned file can be loaded with jQuery's .load() function or the contents of a text area could be dumped in via the model's .set() function).

As far as I understood, you have single URL for different models and you want to somehow check it before you set or change model.
You can use parse() method on model to change it's content, see http://documentcloud.github.com/backbone/#Model-parse

ExtJS - How to use Proxy, Model? How are they related?

I've been trying to learn to work with Models and Stores. But the proxy bit is confusing me a lot. So I'm going to list out my understanding here - please point out the gaps in my understanding.
My understanding
Models are used to represent domain objects.
Models can be created by ModelManager, or by simply using the constructor
Models are saved in Stores
Stores may be in memory stores, or can be server stores. This is configured using Proxy.
Proxy tells the store how to talk to a backing store - be that a JSON array, or a REST resource, or a simply configured URL via ajax.
Stores are responsible for storing models, and Proxies are responsible for controlling/helping with that task.
When a model's values are changed, its dirty flag gets set. It gets automatically cleared when the model is saved. (more in this later)
The part that confuses me
Why is there a proxy config and save method on Model? I understand that models can only be stored on to stores.
Why is the dirty flag not cleared simply when I add a model object to a store?
When I add a model object to a store, why does the model not acquire the proxy configured with that store?
proxy is a static configuration for a Model. Does that mean that we cannot use objects of a particular model with multiple data sources? By extension, does this mean having multiple stores for a single model essentially useless?
When we define a Store, are we defining a class (store-type, if we may call it that), or is it an instance of a store? Reason I ask is when we declare a grid, we simply pass it a store config as store: 'MyApp.store.MyStore' - does the grid instantiate a grid of that type, or is it simply using the store we've already instantiated?
Thanks!
PS: +50 bounty to the person who explains all this :) - will offer bounty after those 48 hours are over..

The docs say:
A Model represents some object that your application manages.
A Store is just a collection of Model instances - usually loaded from a server somewhere.
Models are saved in Stores
Not only. The Models can be used separately (f.i. for populating forms with data. Check out Ext.form.Panel.loadRecord for more info).
Why is there a proxy config and save method on Model? I understand that models can only be stored on to stores.
As I've said, not only.
Why is the dirty flag not cleared simply when I add a model object to a store?
Why should it? The Record becomes clean only when it gets synchronized with the corresponding record on the server side.
When I add a model object to a store, why does the model not acquire the proxy configured with that store?
This is, again, because model can be used without store.
proxy is a static configuration for a Model. Does that mean that we cannot use objects of a particular model with multiple data sources?
We cannot use objects of a particular model but we can use one model definition for multiple store. For example:
Ext.define('MyModel', {
// ... config
});
var store1 = Ext.create('Ext.data.Store', {
model: 'MyModel',
// ... config
proxy: {
// ...
// this proxy will be used when store1.sync() and store1.load() are called
}
// ...
});
// ...
var storeN = Ext.create('Ext.data.Store', {
model: 'MyModel',
// ... config
proxy: {
// ...
// this proxy will be used when storeN.sync() and storeN.load() are called
}
// ...
});
Since store1 and storeN use different proxies, the records, contained by these stores, may be completely different.
When we define a Store, are we defining a class (store-type, if we may call it that), or is it an instance of a store?
Yes, when we define a Store, we are defining a class.
Reason I ask is when we declare a grid, we simply pass it a store config as store: 'MyApp.store.MyStore' - does the grid instantiate a grid of that type, or is it simply using the store we've already instantiated?
There are couple of ways to set store config for grid:
store: existingStore,
store: 'someStoresId',
store: 'MyApp.store.MyStore',
In the first and the second cases existing instances of stores would be used. In the third case newly created instance of 'MyApp.store.MyStore' would be used. So, store: 'MyApp.store.MyStore', is equal to
var myStore = Ext.create('MyApp.store.MyStore', {});
// ...
// the following - is in a grid's config:
store: myStore,
UPDATE
When a model is added to store and then the store's sync() is called, why is the model's dirty flag not cleared?
It should be cleared, unless reader, writer and server response are not set up properly. Check out writer example. There is dirty flag being cleared automaticaly on store's sync().
if a model class is not given any proxy at all, why does it track its dirty status?
To let you know if the record was changed since the creation moment.
What is achieved by introducing the concept of Models syncing themselves to the server?
Let's assume you are creating a widget which contains plain set of inputs. The values for this inputs can be loaded from db (this set of inputs represents one row in db table). And when user changes the inputs' values data should be sent to the server. This data can be stored in one record.
So what would you use for this interface: one record or a store with only one record?
Standalone model - is for widgets that represent one row in db table.
Store - is for widgets that represent set of rows in db table.

I'm coming from ExtJS 2.2 [sic] to 4, and the Model behavior and terminology threw me for a loop too.
The best quick explanation I can find is from this post in the "Countdown to ExtJS 4" series on Sencha's blog. Turns out a Model acts like it does because it's "really" a Record.
The centerpiece of the data package is Ext.data.Model. A Model
represents some type of data in an application - for example an
e-commerce app might have models for Users, Products and Orders. At
its simplest a Model is just a set of fields and their data. Anyone
familiar with Ext JS 3 will have used Ext.data.Record, which was the
precursor to Ext.data.Model.
Here's the confusing part: A Model is both a model for the data you're using and a single instance of an object following that model. Let's call its two uses "Model qua model" and "Model qua Record".
This is why its load method requires a unique id (full stop). Model qua Record uses that id to create RESTful URLs for retrieving (and saving, etc) a single Record worth of data. The RESTful URL convention is described here and is linked to in this post on Sencha's blog that talks specifically about Model usage.
Here are a few RESTful URLs formed to that standard to get you familiar with the format, which it appears ExtJS' Model does use:
Operate on a Record with id of 1
GET /people/1 <<< That's what's used to retrieve a single record into Model
return the first record with id of 2
DELETE /people/2
destroy the first record with id of 7
POST /people/7?_method=DELETE
Etc etc.
This is also why Models have their own proxies, so that they can run RESTful operations via that URL modified to follow the conventions described in that link. You might pull 100s of records from one URL, which you'd use as your Store's proxy source, but when you want to save what's in the single Model (again, think "Model qua Record" here), you might perform those Record-specific operations through another URL whose backend messes with one record at a time.
So When Do I use Stores?
To store more than one instance of that Model, you'd slap them into a Store. You add lots of Models qua Records into Stores and then access those "Models" to pull data out. So if you have a grid you'll naturally want to have all that data locally, without having to re-query the server for each row's display.
From the first post:
Models are typically used with a Store, which is basically a
collection of Model instances.
And, of course, the Stores apparently pull info from Model qua Model here to know what they're carrying.

I found it in the documentation of sencha App Architecture Part 2
Use proxies for models:
It is generally good practice to do this as it allows you to load and
save instances of this model without needing a store. Also, when
multiple stores use this same model, you don’t have to redefine your
proxy on each one of them.
Use proxies for stores:
In Ext JS 4, multiple stores can use the same data model, even if the
stores will load their data from different sources. In our example,
the Station model will be used by the SearchResults and the Stations
store, both loading the data from a different location. One returns
search results, the other returns the user’s favorite stations. To
achieve this, one of our stores will need to override the proxy
defined on the model.

Code organization in JavaScript: MVC? Self-rendering components?

I'm building an extremely JS-heavy web application. I say it's JS-heavy, because the vast majority of the work going on is done on the client (although there is some syncing back and forth to the server using AJAX and XMPP).
This is my first time building something of this scale in pure JS (using jQuery), so I began by organizing my code using MVC in a manner modeled after Rails. For example, if the user clicks a button, a method is called in the Controller object, which pulls some data from the models, and which then passed the data a view function. I do this for pretty much everything, even trivial actions like showing a small popup.
After a few months of this, I feel like I'm not taking full advantage of the language. Should I really be rendering my views as if they are webpages?
It seems like it'd make more sense to break down the views into components, which would be instances of Javascript objects/functions. For example, instead of...
var itemListHTML = '<ul id="item-list"><li>item1</li><li>item2</li></ul>';
jQuery('#container').html(itemListHTML);
...I could instead have...
components.itemList.render();
So here, there's just a component called itemList. Since all the data is stored on the client, these components could instantly access all of the data necessary to create and manage themselves. I imagine I would still use MVC, but there'd be no need for controller actions that are responsible for the entire view. If I want to refresh one part of the UI, I just call whateverComponentControlsThatArea.redraw() and it re-renders itself.
I'm sure someone had done this before. Is there a name for this style of code organization? Any guides or best-practices for how to implement it?

There are a number of javascript MVC frameworks available now
JavaScriptMVC, PureMVC, Sammy.js for example.
Rendering of views or sub-views in normally handled through some kind of templating engine.
JavaScriptMVC has a module EJS, which is modelled on ERB, which I have found quite useful. Templates can be compiled into functions to speed things up for production.
There are other templating solutions too for example John Resig's Micro-Templating and many more
I recommend you use one of these frameworks if it's not too late. I've used JavaScriptMVC for a few projects now and can recommend it (The docs take some getting used to though)

You should really look into jquery.tmpl (http://api.jquery.com/jquery.tmpl/) for creating fast javascript templating ideal for building your view rendering.

A little late to the party, but I've got $0.02 here that I'm not sure what to do with...
Forgetting about (typical) Web-MVC (ie: RailsMVC, et cetera) for a second:
Consider that JavaScript can all be resident in the same place, and that you don't have to worry about figuring out routing and class-instantiating (unless you really want to).
Ideally, from a software point of view, what you want MVC to do is to separate what the user does, from what makes up the content (based on the action), from what makes up the view (based on the action).
There's nothing saying that you can't have multiple views, or even that you can't have a view which contains multiple views (which might use templates, or be functionally generated, and attached to a DOM-node - either is valid).
In a simple pseudo-example, rather than having a flow that looks like:
Action Loads Controller -> Controller Loads Model -> Controller Queries Model -> Model Answers Controller -> Controller Loads View -> Controller Feeds Data to View -> View Sends Page to Controller -> Controller Outputs Page
Why not something like:
Controller Listens for Action (addEventListener) -> Controller Turns Action into State-Logic -> Controller Notifies Model (observer) OR Model Polls Controller -> Model Changes State based on Logic, and Collects / Sorts all Data -> Model Notifies View (observer) OR View Polls Model -> View Changes State based on Logic -> View Renders all Components, based on Data + ViewState (innerHTML OR documentFragment).
It looks a little longer, but in reality, everything is nice and separate.
With the View (or View Manager or however you want to think of it) dictating what windows are on the page, and how each window is made up, and where the data goes, inside of each article, in each window...
...now you have an MVC pattern, but you ALSO have the ability to say:
View["mainpage"] = {
data : { tweets : [{id:...}...]/* et cetera - pushed by Model, not Controller */ },
layout : [ "Header", "Carousel", "Articles", "TwitterFeed", "RSSFeed", "Footer" ],
// if your system is this clean, you could even prototype the content-builders
// rather than defining them in each ViewState - you'd just need layout, then
buildPage : function () {
var page = document.createDocumentFragment();
for (/* everything in this.layout */) {
View.build[this.layout[i]](this.data, page);
}
document.body.appendChild(page);
},
cleanUp : function () { /* fancy or simple DOM cleaning for state-change */ },
// grabs SubView by expected handle (id="tweetfeed" or whatever) and replaces it
// observer functionality, for views to automatically update as data changes
updateView : function (view, newData) { ... },
addData : function (data) { this.data = data; }, // for Observer
/* Observer - if you want to run the WHOLE site with AJAX from index.html
* clean up old (ex:"main") page and build and/or transition new (ex:"media") page
* could be unique for each page, for custom transitions, or just prototype it */
changeState : function (newState, newData) {
View["mainpage"].cleanUp();
View[newState].addData( newData );
View[newState].buildPage();
}
}
And now you've defined your whole main-page.
Of course, it's going to be extra work maintaining not only definitions for your main page, but for every page/section.
...and then, you need to make the functions which will create each sub-view -- but component-based design is EXACTLY what you were asking about.
In essence, for each widget, it will have its own building-logic.
That's what views do - they either have an individual template that gets reused, or they have a function that works the same way every time.
And an atomic view is one where you're dealing with a function that builds 1 item (1 Tweet, 1 Post, 1 Store item) and builds the view for that.
Having a Single_Tweet view post a single-tweet to the page, 20 times in a row would be a bad idea for JS performance.
But having a Single_Tweet view push tweets, in order, into a Tweets array that your Twitter_Feed view puts on the site is A-OK.
Even better is when your Pages are Views, made up of Widgets of Views, made up of Atomic-Units of Views.
And when all of that happens off-DOM (so innerHTML or documentFragment).
And the best would be a Framework where you define pages in this way, but you can also update an individual Widget, based on data-pushes from the Model, whenever you want.
And that's the beauty of AJAX -- old-fashioned MVC can happen, without a Controller that knows everything.
The Controller just holds a list of pressed-keys, the mouse coordinates, a clicked button's intent...
And it tells the Model that something happened.
The Model does everything state/data/data-manipulation (sorting) related, and feeds the new state or data to the view, which takes it all and puts it into HTML, in the order you specify.
That's about it.

Does it make sense to initialize models from the DOM in backbone.js?

backbone.js relies on restful applications to initialize models, but what about progressive enhancement? The data is already in the dom (or some of it), so should my models still make calls the the restful interface even though the html elements exist? Is there another library design that might be better suited for this case?

Backbone can handle that pretty well. The way I handle this case is to have a factory model that can receive a DOM node and parse it in order to extract data (id, fields and so on).
If you supply a 'el' option to a View constructor, backbone won't fetch nor render the model, so you can keep your node as is.
Upon data change, the controller will then sync to the server. You must be careful though to include whatever data your application needs to function whether it's displayed or not.

You should not use DOM element to initialize your model with backend data. You have a really nice infrastructure with backbone to not do this. When you rely on the DOM you need to change your javascript whenever the DOM structure change due to design for exemple.
Also do not rely on backbone view to create the model. It must go the other way around, the model dictate the views on the page.
Just add a script element and create your JS objects directly in there. You can initialize collections, single models, etc.
You can do the same with templates or DOM UI building blocks:
<script type="text/js-template">
<!-- Your template as realy do elements or using a js templating engine like _.template-->
</script>
Load up your page and have your app play locally.