I am more into front end development and have recently started exploring Backbone.js into my app. I want to persist the model data to the server.
Could you please explain me the various way to save the Model data (using json format). I am using Java on server side. Also I have mainly seen REST being used to save data. As i am more into front end dev, i am not aware of REST and other similar stuff.
It would be great if someone could please explain me the process with some simple example.
Basically Models have a property called attributes which are the various values a certain model may have. Backbone uses JSON objects as a simple way to populate these values using various methods that take JSON objects. Example:
Donuts = Backbone.Model.extend({
defaults: {
flavor: 'Boston Cream', // Some string
price: '0.50' // Dollars
}
});
To populate the model there are a few ways to do so. For example, you can set up your model instance by passing in a JSON OR use method called set() which takes a JSON object of attributes.
myDonut = new Donut({'flavor':'lemon', 'price':'0.75'});
mySecondHelping = new Donut();
mySecondHelping.set({'flavor':'plain', 'price':'0.25'});
console.log(myDonut.toJSON());
// {'flavor':'lemon', 'price':'0.75'}
console.log(mySecondHelping.toJSON());
// {'flavor':'plain', 'price':'0.25'}
So this brings us up to saving models and persisting them either to a server. There is a whole slew of details regarding "What is REST/RESTful?" And it is kind of difficult to explain all this in a short blurb here. Specifically with regard to REST and Backbone saving, the thing to wrap your head around is the semantics of HTTP requests and what you are doing with your data.
You're probably used to two kinds of HTTP requests. GET and POST. In a RESTful environment, these verbs have special meaning for specific uses that Backbone assumes. When you want to get a certain resource from the server, (e.g. donut model I saved last time, a blog entry, an computer specification) and that resource exists, you do a GET request. Conversely, when you want to create a new resource you use POST.
Before I got into Backbone, I've never even touched the following two HTTP request methods. PUT and DELETE. These two verbs also have specific meaning to Backbone. When you want to update a resource, (e.g. Change the flavor of lemon donut to limon donut, etc.) you use a PUT request. When you want to delete that model from the server all together, you use a DELETE request.
These basics are very important because with your RESTful app, you probably will have a URI designation that will do the appropriate task based on the kind of request verb you use. For example:
// The URI pattern
http://localhost:8888/donut/:id
// My URI call
http://localhost:8888/donut/17
If I make a GET to that URI, it would get donut model with an ID of 17. The :id depends on how you are saving it server side. This could just be the ID of your donut resource in your database table.
If I make a PUT to that URI with new data, I'd be updating it, saving over it. And if I DELETE to that URI, then it would purge it from my system.
With POST, since you haven't created a resource yet it won't have an established resource ID. Maybe the URI target I want to create resources is simply this:
http://localhost:8888/donut
No ID fragment in the URI. All of these URI designs are up to you and how you think about your resources. But with regard to RESTful design, my understanding is that you want to keep the verbs of your actions to your HTTP request and the resources as nouns which make URIs easy to read and human friendly.
Are you still with me? :-)
So let's get back to thinking about Backbone. Backbone is wonderful because it does a lot of work for you. To save our donut and secondHelping, we simply do this:
myDonut.save();
mySecondHelping.save();
Backbone is smart. If you just created a donut resource, it won't have an ID from the server. It has something called a cID which is what Backbone uses internally but since it doesn't have an official ID it knows that it should create a new resource and it sends a POST request. If you got your model from the server, it will probably have an ID if all was right. In this case, when you save() Backbone assumes you want to update the server and it will send a PUT. To get a specific resource, you'd use the Backbone method .fetch() and it sends a GET request. When you call .destroy() on a model it will send the DELETE.
In the previous examples, I never explicitly told Backbone where the URI is. Let's do that in the next example.
thirdHelping = Backbone.Model.extend({
url: 'donut'
});
thirdHelping.set({id:15}); // Set the id attribute of model to 15
thirdHelping.fetch(); // Backbone assumes this model exists on server as ID 15
Backbone will GET the thirdHelping at http://localhost:8888/donut/15 It will simply add /donut stem to your site root.
If you're STILL with me, good. I think. Unless you're confused. But we'll trudge on anyway. The second part of this is the SERVER side. We've talked about different verbs of HTTP and the semantic meanings behind those verbs. Meanings that you, Backbone, AND your server must share.
Your server needs to understand the difference between a GET, POST, PUT, and DELETE request. As you saw in the examples above, GET, PUT, and DELETE could all point to the same URI http://localhost:8888/donut/07 Unless your server can differentiate between these HTTP requests, it will be very confused as to what to do with that resource.
This is when you start thinking about your RESTful server end code. Some people like Ruby, some people like .net, I like PHP. Particularly I like SLIM PHP micro-framework. SLIM PHP is a micro-framework that has a very elegant and simple tool set for dealing with RESTful activities. You can define routes (URIs) like in the examples above and depending on whether the call is GET, POST, PUT, or DELETE it will execute the right code. There are other solutions similar to SLIM like Recess, Tonic. I believe bigger frameworks like Cake and CodeIgniter also do similar things although I like minimal. Did I say I like Slim? ;-)
This is what excerpt code on the server might look (i.e. specifically regarding the routes.)
$app->get('/donut/:id', function($id) use ($app) {
// get donut model with id of $id from database.
$donut = ...
// Looks something like this maybe:
// $donut = array('id'=>7, 'flavor'=>'chocolate', 'price'=>'1.00')
$response = $app->response();
$response['Content-Type'] = 'application/json';
$response->body(json_encode($donut));
});
Here it's important to note that Backbone expects a JSON object. Always have your server designate the content-type as 'application/json' and encode it in json format if you can. Then when Backbone receives the JSON object it knows how to populate the model that requested it.
With SLIM PHP, the routes operate pretty similarly to the above.
$app->post('/donut', function() use ($app) {
// Code to create new donut
// Returns a full donut resource with ID
});
$app->put('/donut/:id', function($id) use ($app) {
// Code to update donut with id, $id
$response = $app->response();
$response->status(200); // OK!
// But you can send back other status like 400 which can trigger an error callback.
});
$app->delete('/donut/:id', function($id) use ($app) {
// Code to delete donut with id, $id
// Bye bye resource
});
So you've almost made the full round trip! Go get a soda. I like Diet Mountain Dew. Get one for me too.
Once your server processes a request, does something with the database and resource, prepares a response (whether it be a simple http status number or full JSON resource), then the data comes back to Backbone for final processing.
With your save(), fetch(), etc. methods - you can add optional callbacks on success and error. Here is an example of how I set up this particular cake:
Cake = Backbone.Model.extend({
defaults: {
type: 'plain',
nuts: false
},
url: 'cake'
});
myCake = new Cake();
myCake.toJSON() // Shows us that it is a plain cake without nuts
myCake.save({type:'coconut', nuts:true}, {
wait:true,
success:function(model, response) {
console.log('Successfully saved!');
},
error: function(model, error) {
console.log(model.toJSON());
console.log('error.responseText');
}
});
// ASSUME my server is set up to respond with a status(403)
// ASSUME my server responds with string payload saying 'we don't like nuts'
There are a couple different things about this example that. You'll see that for my cake, instead of set() ing the attributes before save, I simply passed in the new attributes to my save call. Backbone is pretty ninja at taking JSON data all over the place and handling it like a champ. So I want to save my cake with coconuts and nuts. (Is that 2 nuts?) Anyway, I passed in two objects to my save. The attributes JSON object AND some options. The first, {wait:true} means don't update my client side model until the server side trip is successful. The success call back will occur when the server successfully returns a response. However, since this example results in an error (a status other than 200 will indicate to Backbone to use the error callback) we get a representation of the model without the changes. It should still be plain and without nuts. We also have access to the error object that the server sent back. We sent back a string but it could be JSON error object with more properties. This is located in the error.responseText attribute. Yeah, 'we don't like nuts.'
Congratulations. You've made your first pretty full round trip from setting up a model, saving it server side, and back. I hope that this answer epic gives you an IDEA of how this all comes together. There are of course, lots of details that I'm cruising past but the basic ideas of Backbone save, RESTful verbs, Server-side actions, Response are here. Keep going through the Backbone documentation (which is super easy to read compared to other docs) but just keep in mind that this takes time to wrap your head around. The more you keep at it the more fluent you'll be. I learn something new with Backbone every day and it gets really fun as you start making leaps and see your fluency in this framework growing. :-)
EDIT: Resources that may be useful:
Other Similar Answers on SO:
How to generate model IDs with Backbone
On REST:
http://rest.elkstein.org/
http://www.infoq.com/articles/rest-introduction
http://www.recessframework.org/page/towards-restful-php-5-basic-tips
Related
I'm trying to make a simple todo app in order to understand how frontend and backend are connected. I read some of the websites showing a tutorial for using and connecting rest API, express server, and database, but still, I was not able to get the fake data from a database. Anyway, I wanted to check if my understanding of how they are connected and talk to each other is correct or not. So could you give some advice please?
First of all, I'm planning to use either Javascript & HTML or React for frontend, Express for server, and Postgres for the database. My plan is a user can add & delete his or her task. I have already created a server in my index.js file and created a database using psql command. Now if I type "" it takes me to the page saying "Hello" (I made this endpoint), and I'm failing to seed my data to the database. Here are my questions↓
After I was able to seed my fake data into the database, how should I get the data from the database and send to the frontend? I think in my index.js file, create a new endpoint something like "app.get("/api/todo", (res, req) => ..." and inside of the callback function, I should write something like "select * from [table name]". Also, form the front end, I should probably access certain endpoints using fetch. Is this correct?
Also, how can I store data which is sent from the frontend? For example, if I type my new todo to <input> field and click the add <button>, what is the sequence of events looks like? Adding event listener to button and connect to the server, then create post method in the server and insert data, kind of (?) <= sorry this part it's super unclear for me.
Displaying task on the frontend is also unclear for me. If I use an object like {task: clean up my room, finished: false (or 0 ?)} in the front end, it makes sense but, when I start using the database, I'm confused about how to display items that are not completed yet. In order to display each task, I won't use GET method to get the data from the database, right?
Also, do I need to use knex to solve this type of problem? (or better to have knex and why?)
I think my problem is I kind of know what frontend, server, database for, but not clear how they are connected with each other...
I also drew some diagrams as well, so I hope it helps you to understand my vague questions...
how should I get the data from the database and send to the frontend?
I think in my index.js file, create a new endpoint something like
"app.get("/api/todo", (res, req) => ..." and inside of the callback
function, I should write something like "select * from [table name]".
Typically you use a controller -> service -> repository pattern:
The controller is a thin layer, it's basically the callback method you refer to. It just takes parameters from the request, and forwards the request to the service in the form of a method call (i.e. expose some methods on the service and call those methods). It takes the response from the service layer and returns it to the client. If the service layer throws custom exceptions, you also handle them here, and send an appropriate response to the client (error status code, custom message).
The service takes the request and forwards it to the repository. In this layer, you can perform any custom business logic (by delegating to other isolated services). Also, this layers will take care of throwing custom exceptions, e.g. when an item was not found in the database (throw new NotFoundException)
The repository layer connects to the database. This is where you put the custom db logic (queries like you mention), eg when using a library like https://node-postgres.com/. You don't put any other logic here, the repo is just a connector to the db.
Also, form the front end, I should probably access certain endpoints
using fetch. Is this correct?
Yes.
Also, how can I store data which is sent from the frontend? For
example, if I type my new todo to field and click the add , what is
the sequence of events looks like? Adding event listener to button and
connect to the server, then create post method in the server and
insert data, kind of (?) <= sorry this part it's super unclear for me.
You have a few options:
Form submit
Ajax request, serialize the data in the form manually and send a POST request through ajax. Since you're considering a client library like React, I suggest using this approach.
Displaying task on the frontend is also unclear for me. If I use an
object like {task: clean up my room, finished: false (or 0 ?)} in the
front end, it makes sense but, when I start using the database, I'm
confused about how to display items that are not completed yet. In
order to display each task, I won't use GET method to get the data
from the database, right?
If you want to use REST, it typically implies that you're not using backend MVC / server rendering. As you mentioned React, you're opting for keeping client state and syncing with the server over REST.
What it means is that you keep all state in the frontend (in memory / localstorage) and just sync with the server. Typically what is applied is what is referred to as optimistic rendering; i.e. you just manage state in the frontend as if the server didn't exist; yet when the server fails (you see this in the ajax response), you can show an error in the UI, and rollback state.
Alternatively you can use spinners that wait until the server sync is complete. It makes for less interesting user perceived performance, but is just as valid technical wise.
Also, do I need to use knex to solve this type of problem? (or better
to have knex and why?) I think my problem is I kind of know what
frontend, server, database for, but not clear how they are connected
with each other...
Doesn't really matter what you use. Personally I would go with the stack:
Node Express (REST), but could be Koa, Restify...
React / Redux client side
For the backend repo layer you can use Knex if you want to, I have used node-postgres which worked well for me.
Additional info:
I would encourage you to take a look at the following, if you're doubtful how to write the REST endpoints: https://www.youtube.com/watch?v=PgrP6r-cFUQ
After I was able to seed my fake data into the database, how should I get the data from the database and send to the frontend? I think in my index.js file, create a new endpoint something like "app.get("/api/todo", (res, req) => ..." and inside of the callback function, I should write something like "select * from [table name]". Also, form the front end, I should probably access certain endpoints using fetch. Is this correct?
You are right here, you need to create an endpoint in your server, which will be responsible for getting data from Database. This same endpoint has to be consumed by your Frontend application, in case you are planning to use ReactJS. As soon as your app loads, you need to get the current userID and make a fetch call to the above-created endpoint and fetch the list of todos/any data for that matter pertaining to the concerned user.
Also, how can I store data which is sent from the frontend? For example, if I type my new todo to field and click the add , what is the sequence of events looks like? Adding event listener to button and connect to the server, then create post method in the server and insert data, kind of (?) <= sorry this part it's super unclear for me.
Okay, so far, you have connected your frontend to your backend, started the application, user is present and you have fetched the list of todos, if any available for that particular user.
Now coming to adding new todo the most minimal flow would look something like this,
User types the data in a form and submits the form
There is a form submit handler which will take the form data
Check for validation for the form data
Call the POST endpoint with payload as the form data
This Post endpoint will be responsible for saving the form data to DB
If an existing todo is being modified, then this should be handled using a PATCH request (Updating the state, if task is completed or not)
The next and possibly the last thing would be to delete the task, you can have a DELETE endpoint to remove the todo item from the list of todos
Displaying task on the frontend is also unclear for me. If I use an object like {task: clean up my room, finished: false (or 0 ?)} in the front end, it makes sense but, when I start using the database, I'm confused about how to display items that are not completed yet. In order to display each task, I won't use GET method to get the data from the database, right?
Okay, so as soon as you load the frontend for the first time, you will make a GET call to the server and fetch the list of TODOS. Store this somewhere in the application, probably redux store or just the application local state.
Going by what you have suggested already,
{task: 'some task name', finished: false, id: '123'}
Now anytime there has to be any kind of interaction with any of the TODO item, either PATCH or DELETE, you would use the id for each TODO and call the respective endpoint.
Also, do I need to use knex to solve this type of problem? (or better to have knex and why?) I think my problem is I kind of know what frontend, server, database for, but not clear how they are connected with each other...
In a nutshell or in the most minimal sense, think of Frontend as the presentation layer and backend and DB as the application layer.
the overall game is of sending some kind of request and receiving some response for those sent requests. Frontend is what enables any end-user to create these so-called requests, the backend (server & database) is where these requests are processed and response is sent back to the presentational layer for the end user to be notified.
These explanations are very minimal to make sure you get the gist of it. Since this question almost revolves around the entire scope of web development. I would suggest you read a few articles about both these layers and how they connect with each other.
You should also spend some time understanding what is RESTful API. That should be a great help.
Good day all. I'm totally new to Extjs and I've been inserted into a big project already started and left alone.
I'm studing the framework of course, but I'm getting struck every bit of code.
Now, I'd like to understand the proper way to make a rest call, using JSON parameters in POST.
this is my actual situation:
I have defined a combobox in the view file (called setup.js) and I've put a listener on select like the following
emptyText: 'Select a country',
listConfig: {
itemTpl: [
'<div><span class="" style="margin-right: 10px;"><img src="resources/flags/{id}.png"></span>{name}</div>'
]
},
listeners: {
/*created the after render listener to wait until the store is loaded. then I set the default value*/
select: 'onCountrySelect',
afterrender: 'onAfterRenderCombo'
}
on another file, called setupController.js I've defined all my functions:
onCountrySelect: function(a){
console.log("selected ",a.value);
var controller = this,
userData = controller.getUserData();
console.log("user.id",userData.id);
},
Now, on every country select action, I've to send a JSON to a server, to "set" the country used by the user. The call depends also on the enviroment as the dev one is different from the production one, so I guess that I have to make some sort of proxy, but maybe this is a later issue, now I'd like to setup a call for this selection, so I'll use this logic for all the future calls I have to do.
questions are, is there a proper way to define it? do I need a model? or is good to have al the calls in the store? do I need a... proxy(?) or a store or something "external" so I can use the very same call in some other places? is there an example for this?
Guys/Girls, thanks in advance.
Like written here http://docs.sencha.com/extjs/6.0.2/classic/Ext.data.proxy.Ajax.html
A proxy is made to work with a server on the same domain:
AjaxProxy cannot be used to retrieve data from other domains. If your application is running on http://domainA.com it cannot load data from http://domainB.com because browsers have a built-in security policy that prohibits domains talking to each other via AJAX.
Personally Having a console connecting with more than one server, I use an ajax request like that http://docs.sencha.com/extjs/6.0.2/modern/Ext.Ajax.html#method-request
Ext.Ajax.request({
url:'',
method:'POST',
params:{
json:YOURJSON
},
success:function(response){
},
failure:function(response){
}
});
I'm working on a personal proxy, extended by ajax proxy, to do the ajax call without different domain problems. If you want to do an home made proxy you can read all you want about proxy here: http://docs.sencha.com/extjs/6.0.2/classic/Ext.data.proxy.Proxy.html
or simply you can use an external JS class that do the ajax request for you.
I'm working on a materials exchange site for instructors and students. The material is confidential in the sense that it's ours and we don't want other instructors to steal it. On the other hand, it has no personal or sensitive information so security is not a huge concern. If you steal my grammar exercise, I'm not going to sue you. I'm using Ion Auth in a Codeigniter framework and am satisfied with that. I use YUI 2.8 to display editable datatables and editable trees. Building on the Codeigniter MVC framework I have a controller to handle Ajax requests. When I post a request via the YUI Connection class I pass a serialised object called data as the only parameter, which comes from a JSON object assembled on the client side by passing a config object to YAHOO.si.factory.Method like this:
addTreeNode: new YAHOO.si.factory.Method({
className: "instructors_model",
methodName: "add_exercise_tree_node",
key: "id",
mode: "child",
fields: ['n0', 'n1','n2', 'n3'],
params: [{
key: "nodeData",
name: "nodeData",
type: "text",
direction: "IN",
value: "34"
},{
key: "name",
name: "name",
type: "text",
direction: "IN",
value: "Grammar"
}],
success: "You have successfully added a node to the tree."
}),
The controller parses the JSON string assembled by YAHOO.si.factory.Method and figures out that it has to call add_exercise_tree_node in the instructors_model and pass 2 parameters. The add_exercise_tree_node method does the dirty work on the database and, hopefully, returns a success message. If so, the success message is displayed on the client side and the tree is updated to include the new node.
My question is, am I giving away too much information about the server in this plain-text querystring? It's a post so it doesn't appear in the URL, but of course it's completely visible in something like the Firebug console.
The question behind that question of course is what do you security experts out there advise about parcelling out responsibility for Ajax in an MVC framework? I want the manageability of having just one controller responsible for handling Ajax requests, so I've ended up with a slightly complicated (and insecure?) overloading on the client-side. What patterns do you advise for this?
My claim in the comments was that you're defeating the purpose of RESTful architecture in the MVC paradigm, primarily with focus on "all AJAX calls go to one controller". So let's tear this down into something usable, and then rebuild our understanding of why/how.
REST just means REpresentational State Transfer but that's not what we mean when we say REST or RESTful. We mean that we intend each URL to represent a type of resource, and we use verbs to indicate what we're doing with each request. So "GET", "POST", "PUT", etc all have a purpose, and we use that purpose to dictate how we interact with the individual resources.
So let's put that into perspective, let's do something with that; consider an HR application (this is one of those easy ones that people adopt). We have People and Departments. Departments have an ID, a Name, and a Manager (who is a Person). People have an ID, a Name, a Manager (may be -1 in our system to indicate no manager, or an ID of a person) and a Phone Number.
When we HTTP GET http://server.domain/app/Person we get a directory of all people, that gives us their name and ID.
[ {Name: "Cole Brand", ID: 1}, {Name: "user341180", ID: 2} ]
When we HTTP GET http://server.domain/app/Person/1 we get
{ Name: "Cole Brand", ID: 1, Manager: -1, "Phone Number": "222-555-1234" }
When we HTTP GET http://server.domain/app/Person/2 we get
{ Name: "user341180", ID: 2, Manager: 1, "Phone Number": "222-555-6789" }
And just for the sake of completeness, a list of departments:
HTTP GET http://server.domain/app/Department/
{ Name: "HR", Manager: 1 }
Now obviously, my example is a little dry, because for instance, the employees certainly work in a department, but I haven't associated them to it by their profile. Surely the database would track that, but we don't have it here. This is a simple example.
So you can see from this that there is no HTML being returned. We will come back to that.
For the next part, notice that I've kept using HTTP GET. What if we used HTTP POST? That would mean that whatever we passed, would overwrite that data. So let me give a brief example:
HTTP POST http://server.domain/app/Person/2
send this data in the post body { Manager: -1, "Phone Number": "222-555-0089" }
get nothing in response (we could return the object, but my API doesn't for whatever reason, call it a specification deficiency)
When we HTTP GET http://server.domain/app/Person/2 we now get
{ Name: "user341180", ID: 2, Manager: -1, "Phone Number": "222-555-0089" }
See how we used HTTP POST to update the record? That's how we use our RESTful methods, we use the verb to indicate the action.
So what else could we use to communicate with the server? What if we wanted to have the RESTful methods return both a webpage and the data, but for GET? We could use an HTTP HEADER (maybe an X-Header?) to specify that we're making a data call, so by default we make a request to HTTP GET http://server.domain/app/Person/1 and we get a webpage. If you would like an example of that (a rest response providing a webpage) check the page we're on now. https://stackoverflow.com/questions/11061912/ <-- is actually served by MVC restful principles (but I don't think they supply the JSON by this method, because that's not how their system is setup).
So I'm probably scrambling you all up, I just wanted to review REST, and how we use it today for MVC purposes. Let's look at part of that again, the part where we do http://server.domain/app/Person
In MVC terms, that would mean that we've got a Person controller in our app. Any requests made against the Person controller would be routed based on their verb type first, then any additional parameters next. For instance, submitting this answer makes a post to https://stackoverflow.com/questions/11061912/answer/submit. So the Controller knows that it expects to receive "Post ID" (so it knows the parent that the answer belongs to), "answer" telling it that I'm adding an answer, and "submit" so it knows what I'm doing. If I were editing it would be an edit-submit instead of answer/submit.
Ok, so I feel like I'm making things confusing, and I don't know if you read my answer on MVC in node and general: How are models tied to views? but let me clarify: The Controller is a routing device to take input, send it to the appropriate model, and then render some view (HTML, JSON, XML, plain text, etc). Note that rendering a view is not required (remember my crappy API that didn't return anything on a HTTP POST?).
So now we get to the point of what I was trying to get at in my response earlier, which I'm still not being clear enough on. When you move methods from the "appropriate resource" (in the case of this post /question/), you're breaking the idea that one URL root is one resource. Oy, I've messed this all up haven't I?
In REST principles, each URL-root is one resource. You can do things from that resource, like rendering a certain view based on some bit of data, or you can add/modify data using a certain verb, or you can make flapjacks if that's your style. But all Person will always be at http://server.domain/app/Person, and not "some person commands are on http://server.domain/app/Person and some person commands are on http://server.domain/app/Department" because then you're just going to throw your arms up in the air and go "well how do I know where all to find people actions?!?!"
That, in a nutshell, is why you don't want to put some methods for accessing data on one controller (rendering the view) and some methods for accessing data on another controller (JSON) for the same resource-type.
Now that I've put all that braindump into a post, which parts am I fuzzy on? Feel free to edit the post and insert some **what do you mean here?** or **you seem to be jumping around on this point a lot** or whatever you feel is appropriate.
I'm currently researching how to add persistence to a realtime twitter json feed in node.
I've got my stream setup, it's broadcasting to the client, but how do i go about storing this data in a json database such as couchdb, so i can access the stores json when the client first visits the page?
I can't seem to get my head around couchdb.
var array = {
"tweet_id": tweet.id,
"screen_name": tweet.user.screen_name,
"text" : tweet.text,
"profile_image_url" : tweet.user.profile_image_url
};
db.saveDoc('tweet', strencode(array), function(er, ok) {
if (er) throw new Error(JSON.stringify(er));
util.puts('Saved my first doc to the couch!');
});
db.allDocs(function(er, doc) {
if (er) throw new Error(JSON.stringify(er));
//client.send(JSON.stringify(doc));
console.log(JSON.stringify(doc));
util.puts('Fetched my new doc from couch:');
});
These are the two snippets i'm using to try and save / retrieve tweet data. The array is one individual tweet, and needs to be saved to couch each time a new tweet is received.
I don't understand the id part of saveDoc - when i make it unique, db.allDocs only lists ID's and not the content of each doc in the database - and when it's not unique, it fails after the first db entry.
Can someone kindly explain the correct way to save and retrieve this type of json data to couchdb?
I basically want to to load the entire database when the client first views the page. (The database will have less than 100 entries)
Cheers.
You need to insert the documents in the database. You can do this by inserting the JSON that comes from the twitter API or you can insert one status at a time (for loop)
You should create a view that exposes that information. If you saved the JSON directly from Twitter you are going to need to emit several times in your map function
There operations (ingestion and querying) are not the same thing, so you should really do them at the different times in your program.
You should consider running a bg process (maybe in something as simple as a setInterval) that updates your database. Or you can use something like clarinet (http://github.com/dscape/clarinet) to parse the Twitter streaming API directly.
I'm the author of nano, and here is one of the tests that does most of what you need:
https://github.com/dscape/nano/blob/master/tests/view/query.js
For the actual query semantics and for you learn a bit more of how CouchDB works I would suggest you read:
http://guide.couchdb.org/editions/1/en/index.html
I you find it useful I would suggest you buy the book :)
If you want to use a module to interact with CouchDB I would suggest cradle or nano.
You can also use the default http module you find in Node.js to make requests to CouchDB. The down-side is that the default http module tends to be a little verbose. There are alternatives that give you an better API to deal with http requests. The request is really popular.
To get data you need to make a GET request to a view you can find more information here. If you want to create a document you have to use PUT request to your database.
I have a calendar application and it loads all of the event data using ajax and json results. the issue is that i have different view and right now i have to re call the server when i change views.
Is there any recommendation for ways i can cache this data on the client side and check if i have loaded these events already before firing off more ajax calls.
What is the best practice for this ?
Like hvgotcodes said, an MVC framework would help; try backbone.js (http://documentcloud.github.com/backbone/), for instance.
Alternatively, you might want to consider using jStorage (http://www.jstorage.info/). Every time you need to make an AJAX call, check first if it's in your storage object, then run the AJAX call if it isn't. On the other end, whenever you finish an AJAX call, store the results in the storage object. Make sure you have some kind of index (a CalendarEvent id) to reference when looking it up in the data store. Might want to add some kind of "expire time" to the data in your storage, too ... a timestamp after the AJAX call, and re-request up front if it's out of date.
It's called MVC.
You need to construct a data model for you application, write some sort of Record objects, and then you can determine their status. So your application would have some sort of CalendarEvent model, and when you load data from the server, you would instantiate instances.
So when changing views, you would first check to see if you had the model object for that view, and if you did, you wouldn't need to load it from the server (unless you want to check for changes).
Your scheme doesn't need to be that complicated. If you load events by Id, you can do something like
window.App = {};
window.App.Models = {};
when you load a record you could put
window.App.Models[id] = InstanceOfYourRecord
and that way its pretty fast to look for records. Or just use a framework (like Sproutcore) that has a robust data layer.
I had similar issues on a recent project.
Conceptually, I have the "real" data model (DM) kept on the server, persisted to a database.
To make life sane, the client keeps its own local data model. Outside of the client DM, all the client code thinks it's pulling results locally.
When reading data (GET) from the client DM it:
checks the cache for existing results
invokes appropriate AJAX queries when cached data is not available, then caches the results.
When changing data (POST) via the client DM it:
invalidates the cache as appropriate
invokes appropriate AJAX queries
emits custom jQuery event indicating client DM changed
Note that this client DM also:
centralizes AJAX error handling
tracks AJAX calls still in-flight. (Lets us warn users when leaving pages with unsaved changes).
allows a drop-in, dummy replacement for unit testing, where all the calls hit local data and are completely synchronous.
Implementation notes:
I coded this as a JavaScript class called DataModel. As the design becomes more complex, it makes sense to further break-down the responsibilities in to separate objects.
jQuery's custom events let you easily implement the observer pattern. Client components update themselves from the client DM whenever it indicates data has changed.
JSON in your remote API helps simplify the code. My client DM stores the JSON results directly in its cache.
The client dm function arguments include call-backs so everything can naturally be passed along via AJAX when needed: function listAll( contactId, cb ) { ... }
My project only allowed single user logins. If outside parties can change the server datamodel, some sort of has-data-changed probe should be fired regularly to ensure the client cache is still valid.
For my app, multiple client components would request the same data when receiving a client DM changed event. This resulted in multiple AJAX calls with the same info. I fixed this problem with a getJsonOnce() helper, which manages a queue of client component call-backs awaiting the same result.
Example function in my implementation:
listAll:
function( contactId, cb ) {
// pull from cache
if ( contactId in this.notesCache ) {
cb( this.notesCache[contactId] );
return;
}
// init queue if needed
this.listAllQueue[contactId] = this.listAllQueue[contactId] || [];
// pull from server
var self = this;
dataModelHelpers.getJsonOnce(
'/teafile/api/notes.php',
{'req': 'listAll', 'contact': contactId},
function(resp) { self.notesCache[contactId] = resp; },
this.listAllQueue[contactId],
cb
);
}
The getJsonOnce() helper makes sure that if multiple client components request the exact same (uncached) data, that we only send out a single AJAX request and inform everyone once it comes in.
The notesCache is just a simple javascript object:
this.notesCache = {};