Need guidance on efficiently fetching from Django server with Backbone.js - javascript

I have an app built on Backbone.js and Django that is just something I'm building for practice.
It's a webchat app that let's users post a message and then saves it to a database, all while constantly fetching new data into a collection I have, so it "updates live" with the server.
On a basic level, the front-end application works with a message model to model a user's message, a messages collection to store all of the posts, and a chat view to render the collection in the chat window.
The code I use to constantly fetch new data from the server is:
// This code is inside the `chat` view's `initialize` function,
// so `that` is set to `this` (the `chat` view) higher up in the code.
window.setInterval(function() {
theColl.fetch({
success: function() {
that.render();
}
});
}, 2000);
I have don't have reset: true because I thought that it would be more efficient to just append new messages to the collection as they were retrieved from the server, instead of reloading every model already in it.
Is this an efficient way to run a simple webchat? As the application has to fetch more models, I've noticed that it gets extremely sluggish, and adds a delay to the user input. Here is a photo to show what I think is one problem (p.s. ignore the stupid messages I wrote):
As the server sends back new models and the collection refreshes, I think that my app is initializing old chat models again, because at this point in the application the collection itself only had 25 models inside of it.
Is there something I can do to stop the reinitialization of every model in the database? As you can see, even though there are only 25 unique models in the chat, and in the database, after letting the app run for 2 minutes or so I get models with CIDs up to 460.
Regardless of that, I've noticed that if I flood the server with new messages things go awry, like new messages appear out of place, and the server gets backed up. The latter may be because I'm using the cheapest option on Digital Ocean to host the app, but I'm not sure what the first problem comes from. It may be from my Django app's views.py or the rest of my Backbone code, I'm not really sure how to debug it.

Related

Making really simple app including front & backend skills (js, node.js, psql, react...)

I'm trying to make a simple todo app in order to understand how frontend and backend are connected. I read some of the websites showing a tutorial for using and connecting rest API, express server, and database, but still, I was not able to get the fake data from a database. Anyway, I wanted to check if my understanding of how they are connected and talk to each other is correct or not. So could you give some advice please?
First of all, I'm planning to use either Javascript & HTML or React for frontend, Express for server, and Postgres for the database. My plan is a user can add & delete his or her task. I have already created a server in my index.js file and created a database using psql command. Now if I type "" it takes me to the page saying "Hello" (I made this endpoint), and I'm failing to seed my data to the database. Here are my questions↓
After I was able to seed my fake data into the database, how should I get the data from the database and send to the frontend? I think in my index.js file, create a new endpoint something like "app.get("/api/todo", (res, req) => ..." and inside of the callback function, I should write something like "select * from [table name]". Also, form the front end, I should probably access certain endpoints using fetch. Is this correct?
Also, how can I store data which is sent from the frontend? For example, if I type my new todo to <input> field and click the add <button>, what is the sequence of events looks like? Adding event listener to button and connect to the server, then create post method in the server and insert data, kind of (?) <= sorry this part it's super unclear for me.
Displaying task on the frontend is also unclear for me. If I use an object like {task: clean up my room, finished: false (or 0 ?)} in the front end, it makes sense but, when I start using the database, I'm confused about how to display items that are not completed yet. In order to display each task, I won't use GET method to get the data from the database, right?
Also, do I need to use knex to solve this type of problem? (or better to have knex and why?)
I think my problem is I kind of know what frontend, server, database for, but not clear how they are connected with each other...
I also drew some diagrams as well, so I hope it helps you to understand my vague questions...
how should I get the data from the database and send to the frontend?
I think in my index.js file, create a new endpoint something like
"app.get("/api/todo", (res, req) => ..." and inside of the callback
function, I should write something like "select * from [table name]".
Typically you use a controller -> service -> repository pattern:
The controller is a thin layer, it's basically the callback method you refer to. It just takes parameters from the request, and forwards the request to the service in the form of a method call (i.e. expose some methods on the service and call those methods). It takes the response from the service layer and returns it to the client. If the service layer throws custom exceptions, you also handle them here, and send an appropriate response to the client (error status code, custom message).
The service takes the request and forwards it to the repository. In this layer, you can perform any custom business logic (by delegating to other isolated services). Also, this layers will take care of throwing custom exceptions, e.g. when an item was not found in the database (throw new NotFoundException)
The repository layer connects to the database. This is where you put the custom db logic (queries like you mention), eg when using a library like https://node-postgres.com/. You don't put any other logic here, the repo is just a connector to the db.
Also, form the front end, I should probably access certain endpoints
using fetch. Is this correct?
Yes.
Also, how can I store data which is sent from the frontend? For
example, if I type my new todo to field and click the add , what is
the sequence of events looks like? Adding event listener to button and
connect to the server, then create post method in the server and
insert data, kind of (?) <= sorry this part it's super unclear for me.
You have a few options:
Form submit
Ajax request, serialize the data in the form manually and send a POST request through ajax. Since you're considering a client library like React, I suggest using this approach.
Displaying task on the frontend is also unclear for me. If I use an
object like {task: clean up my room, finished: false (or 0 ?)} in the
front end, it makes sense but, when I start using the database, I'm
confused about how to display items that are not completed yet. In
order to display each task, I won't use GET method to get the data
from the database, right?
If you want to use REST, it typically implies that you're not using backend MVC / server rendering. As you mentioned React, you're opting for keeping client state and syncing with the server over REST.
What it means is that you keep all state in the frontend (in memory / localstorage) and just sync with the server. Typically what is applied is what is referred to as optimistic rendering; i.e. you just manage state in the frontend as if the server didn't exist; yet when the server fails (you see this in the ajax response), you can show an error in the UI, and rollback state.
Alternatively you can use spinners that wait until the server sync is complete. It makes for less interesting user perceived performance, but is just as valid technical wise.
Also, do I need to use knex to solve this type of problem? (or better
to have knex and why?) I think my problem is I kind of know what
frontend, server, database for, but not clear how they are connected
with each other...
Doesn't really matter what you use. Personally I would go with the stack:
Node Express (REST), but could be Koa, Restify...
React / Redux client side
For the backend repo layer you can use Knex if you want to, I have used node-postgres which worked well for me.
Additional info:
I would encourage you to take a look at the following, if you're doubtful how to write the REST endpoints: https://www.youtube.com/watch?v=PgrP6r-cFUQ
After I was able to seed my fake data into the database, how should I get the data from the database and send to the frontend? I think in my index.js file, create a new endpoint something like "app.get("/api/todo", (res, req) => ..." and inside of the callback function, I should write something like "select * from [table name]". Also, form the front end, I should probably access certain endpoints using fetch. Is this correct?
You are right here, you need to create an endpoint in your server, which will be responsible for getting data from Database. This same endpoint has to be consumed by your Frontend application, in case you are planning to use ReactJS. As soon as your app loads, you need to get the current userID and make a fetch call to the above-created endpoint and fetch the list of todos/any data for that matter pertaining to the concerned user.
Also, how can I store data which is sent from the frontend? For example, if I type my new todo to field and click the add , what is the sequence of events looks like? Adding event listener to button and connect to the server, then create post method in the server and insert data, kind of (?) <= sorry this part it's super unclear for me.
Okay, so far, you have connected your frontend to your backend, started the application, user is present and you have fetched the list of todos, if any available for that particular user.
Now coming to adding new todo the most minimal flow would look something like this,
User types the data in a form and submits the form
There is a form submit handler which will take the form data
Check for validation for the form data
Call the POST endpoint with payload as the form data
This Post endpoint will be responsible for saving the form data to DB
If an existing todo is being modified, then this should be handled using a PATCH request (Updating the state, if task is completed or not)
The next and possibly the last thing would be to delete the task, you can have a DELETE endpoint to remove the todo item from the list of todos
Displaying task on the frontend is also unclear for me. If I use an object like {task: clean up my room, finished: false (or 0 ?)} in the front end, it makes sense but, when I start using the database, I'm confused about how to display items that are not completed yet. In order to display each task, I won't use GET method to get the data from the database, right?
Okay, so as soon as you load the frontend for the first time, you will make a GET call to the server and fetch the list of TODOS. Store this somewhere in the application, probably redux store or just the application local state.
Going by what you have suggested already,
{task: 'some task name', finished: false, id: '123'}
Now anytime there has to be any kind of interaction with any of the TODO item, either PATCH or DELETE, you would use the id for each TODO and call the respective endpoint.
Also, do I need to use knex to solve this type of problem? (or better to have knex and why?) I think my problem is I kind of know what frontend, server, database for, but not clear how they are connected with each other...
In a nutshell or in the most minimal sense, think of Frontend as the presentation layer and backend and DB as the application layer.
the overall game is of sending some kind of request and receiving some response for those sent requests. Frontend is what enables any end-user to create these so-called requests, the backend (server & database) is where these requests are processed and response is sent back to the presentational layer for the end user to be notified.
These explanations are very minimal to make sure you get the gist of it. Since this question almost revolves around the entire scope of web development. I would suggest you read a few articles about both these layers and how they connect with each other.
You should also spend some time understanding what is RESTful API. That should be a great help.

Rails - Call ajax request when database is updated

So I'm creating a basic jackpot betting site for fun and it allows users to put money into a pot and then based on the percentage of money each user puts in it randomly picks a winner. It's working fine at the moment but I feel like the way I'm updating the jackpot page with ajax is really bad. At the moment I have javascript that makes an ajax request each second to get the pot info (size of the pot, players, etc..). I feel like there is a much better way to do this. Is it possible to only make an ajax call when the database is updated? Thanks!
My javascript at the moment:
setInterval(update, 1000);
function update() {
getPotID();
$.ajax({
type: "POST",
url: "/jackpot/update/" + potID,
complete: function(response) {
$('.live-jackpot').html(response.responseText);
getPotInfo();
},
error: function(xhr, status,error) {
console.log("Error");
}
});
}
as said from 7urkm3n, ActionCable has great advantage for this functionality.
Right now you are writing Javascript code that is executed on the client side. Every user that will start a GET http request to your site, will load that javascript files. They will start performing POST request every second to your backend server.
ActionCable is a websocket preinstalled in Rails 5. This means that to configure the notifications with ActionCable and Rails 5, you already have installed everything in your app (if you are using rails 5), you just need to install Redis on your local machine for testing the app in development.
I am not an expert, my understanding is that you use a specific database called redis to store the information about the subscription. I quote a useful article
So PubSub is this concept that Redis and Postgres and a few other things supports, and basically you have this ability to have Channels and the ability to subscribe (and receive those messages) and publish to those channels and anyone listening will then receive those messages.
An example of this is a ChatroomChannel, you could have people publishing messages to the ChatroomChannel and anyone who was subscribed to the ChatroomChannel would receive the broadcast from the channel with the websocket.
This is something you are missing, this way you could only find which users are actually on the playing page and which users are just browsing around, based on this ActionCable creates a channel to communicate between server and client and then creates a subscription to distinguish users that are actually on the playing page and those that left and should not be anymore notified
I quote another useful article
Before we dive into some code, let's take a closer look at how Action Cable opens and maintains the WebSocket connection inside our Rails 5 application.
Action Cable uses the Rack socket hijacking API to take over control of connections from the application server.
For every instance of your application that spins up, an instance of Action Cable is created, using Rack to open and maintain a persistent connection, and using a channel mounted on a sub-URI of your main application to stream from certain areas of your application and broadcast to other areas.
so every user that connects, ActionCable creates a channel that uses a specific url localhost:3000/cable to communicate between server and client (browser)
Action Cable offers server-side code to broadcast certain content (think new messages or notifications) over the channel, to a subscriber. The subscriber is instantiated on the client-side with a handy JavaScript function that uses jQuery to append new content to the DOM.
This means that the server can call the client with parameters and change the page with jquery functions. For ex. appending a div, with a new message.
In my app https://sprachspiel.xyz I do the following in the MessagesController#create
ActionCable.server.broadcast 'messages',
message: message.content,
user: message.user.name,
chatroom_id: message.chatroom_id,
lastuser: chatroom.messages.last(2)[0].user.name
head :ok
so basically, I have my message in my controller and I can update the client by using the ActionCable.server.broadcast function
then in my asset pipeline file /app/assets/javascripts/channels/messages.js I have the following code that trigger the change in the browser adding the message
App.messages = App.cable.subscriptions.create('MessagesChannel', {
received: function(data) {
$('#messages').append(this.renderMessage(data));
},
renderMessage: function(data) {
return "<br><p> <strong>" + data.user + ": </strong>" + data.message + "</p>";
}
});
I build an app called https://sprachspiel.xyz that is an actioncable app, this is the github page for the project, on my portfolio you can read more info about my app, so please ask me anything, I'll be happy to look into it!
Good luck
Fabrizio

How to Share session between different clients accessing a Meteor website? Collaborative ToDos?

I am learning web development and lately Meteor has caught my fancy.
I went through the starter tutorial of creating to-dos and use save button to commit the list to database. It allows everybody who opens the website to see the same to-dos list.
I added user log in system in to-dos so that people can login and see only their own to-do list.
Now, I'm trying to extend above example, for Collaborative to-dos.
Here is a sample use case:
My boss logs-in at do.com and starts creating his to-do list. While the Boss is logged in, I also happen to open do.com from my laptop and I see a message flashing - A session is already open. Do you want to collaborate with Boss? If I say 'Yes', Boss will be notified at his screen to allow me access to his list, and on granting access, I will be able to collaborate with Boss's to-do with both of our changes in the list reflecting on each other's screen but the final save/commit button remains frozen for me (because I came later) and remains active only for Boss. So, when Boss hits the save button, the list is committed to database with his and my changes.
If Boss chooses to not allow me to contribute, I get to see my own to-do.
On the other hand, if I choose NO, I get a fresh start at my to-do list with no bearing on already open sessions elsewhere.
The scenario should work other way round too. If I am the one who has an active session at do.com and Boss happens to open his own later, he should get the message whether he wants to collaborate with me and so on.
What would be the best way to implement this in Meteor? I came across this Persistent Session package which could be the solution but I am not able to adapt it to my use-case of allowing/denying another user via message/notification. Appreciate, any help on this. I'm a complete newbie here, pls excuse of any un-necessary verbiage, I wanted to explain my question well.
Thanks in advance.
Session is not the right tool for this, you want to use the server db (Collections) to mediate this collaboration.
Given that you created todo lists specific to users, I'm going to assume you have a publication somewhat like this:
Tasks = new Mongo.Collection("tasks");
if (Meteor.isServer) {
Meteor.publish("tasks", function () {
return Tasks.find({owner: this.userId});
});
}
So the next step is to change this so you can see your own tasks, and also those belonging to any user who shares their tasks with you. This could be created like this:
Tasks = new Mongo.Collection('tasks');
CanView = new Mongo.Collection('canView');
// CanView holds docs with this schema:
// {
// user: 'DzxiSdNxEhiHMaoi6',
// taskLists: ['DzxiSdNxEhiHMaoi6', '7X97ZhPxjX6J4eNWx']
// }
if (Meteor.isServer) {
Meteor.publish('tasks', function () {
var canView = CanView.findOne({user: this.userId}).taskLists;
return Tasks.find({owner: {$in: canView}});
});
}
On the client tasks could be displayed as one single list, or segregated by the owner property.
How you add and remove ids into the CanViews tasklist list will depend on the workflow for requesting access/offering to share, etc.
The other part of the workflow you mentioned is only the Boss being able to save the changes, but still have them reactively update on both screens. This would take more work as you would need to implement a 2 step process, with two collections on the server. i.e. Boss's (task owner's) saves are committed directly to the canonical Tasks collection, and other users saves to a second TaskUpdates Collection. Both published to the clients, which then have to overlay the data from TaskUpdates over the actual Tasks in a way that is clear and meaningful.

Fetch current user through store.find('users', 'me') stores invalid User instance

I am trying this solution to fetch the current user before my application is rendered to the user:
Em.Route.extend({
model: function() {
var currentUserPromise = this.store.find('user', 'me');
return Em.RSVP.all([
currentUserPromise
]);
}
});
So, I query the user with the ID me, which is a constant that my API recognizes as the currently authenticated user.
It works really well most of the way. The issue is that Ember-Data immediately creates a User model in the store with this ID constant me and no other attributes. This instance is not removed after the response has arrived and Ember-Data correctly stores the real User model.
I can't figure out where Ember-Data creates this temporary instance and if there is a way to prevent this behavior.
Either a solution to prevent Ember-Data from creating the temp. instance or remove it when the real data arrives will be fine.
Anyone with a solution?
After consulting with #emberjs I found that a much cleaner solution to this challenge is to request the active user by requesting a session object from my API.
By exposing a /sessions endpoint in my API I could expose the current session—which can also include auth token, etc.—with a relationship to the currently logged in profile. That resulted in a JSON payload like this:
{
"sessions": [
{
"id": 1,
"user": { ... },
"accessToken": "abcdef1234567890"
}
]
}
This means that I will no longer have the issue that Ember-Data caches a bogus User object with the id 'me', instead I will now have a session object where the active user is correctly loaded into the store and have it available through the session.
I think you are going about things the wrong way. You're in a single page application, which means the user can log out or log in at any time, and any rendering is essentially already happening before and after that. You can only control what you allow to render, but not the when.
Also, you are explicitly returning a set of promises which resolves when the single promise it contains is resolved. This achieves the same:
Em.Route.extend({
model: function() {
return this.store.find('user', 'me');
}
});
You don't know when that will resolve, but you're asking to "fetch the user before the app is rendered to the user". That's not how it works in Ember, as you can't really delay rendering. (Technically you can, but that offers terrible user experience giving the feeling the browser has locked up).
What you would want to do instead is to modify what you are rendering based on the state of the logged in user, and on the backend side you only ever send what that user is allowed to see. Anything else is insecure, as you can't ever trust the browser. It's pretty trivial to make an Ember app think you are logged in.
The issue is that Ember-Data immediately creates a User model in the store with this ID constant me and no other attributes.
I'm not sure about that one; I'm guessing ember-data creates a placeholder while it waits for the population of the real data (if ever). It is as if you're creating a new empty object in code.
I can't figure out where Ember-Data creates this temporary instance
Ember-data is in its core basically a cache (why it is called the store) with various adapters to read from and write to that cache and an API to use it. In your example me is created in the cache only, at least until you decide to save it.
Ember data modifies the same record instead of creating a new record after the materialization. There might be some other problem while finding the records for table. For example In ember data 1.0.0-beta.8 the the following code logs correct id rather than printing 'me'.
this.store.find('user', 'me').then(function(user) {
console.log(user.get('id'));
});

Don't render backbone marionette collection view on change?

Working on an app that has real time messages with socket.io, I also need to save the messages. I choose to use mongodb and on the client side I am using backbone.
My problem is when I emit a new message all the sockets/users get the message in real time, but the socket/user that sent the message gets it twice, once from the socket and once from the collection re-rendering.
Here is what my collection view looks like.
module.exports = CollectionView = Marionette.CollectionView.extend({
className: 'collection',
initialize: function() {
//this.listenTo(this.collection, 'change', this.render);
},
itemView: MessageView
});
I commented out this.listenTo(this.collection, 'change', this.render);. So I am asking this question because maybe marionette by default renders the CollectionView? Maybe someone has an explanation for how I can prevent the socket/user that sends a message from being appended twice?
Edit: Just got an idea, instead of appending the message I could just fetch the collection when a new message is made? I tried it and it works, maybe there is a better way? I'm still thinking, and am open to new ideas!
Edit: This worked pretty well for me. Instead of appending the html, I just make a GET request to all the sockets connected, that way they get the fresh data.
createMessage: function(data) {
window.App.data.messages.fetch({
success: function() {
console.log('success')
}
});
//this.$el.find('.message-content').append('<div class="message"><b>'+data.username+':</b>'+data.message+'</div>');
window.App.core.vent.trigger('app:log', 'Chat View: Received a new message!');
}
Only issue with the above is the sequence is POST, then GET within milliseconds of each other, so the GET request may return data before the POST fully finishes.
So I am trying to figure out how I can set a callback so that when the POST successfully adds to the collection make the GET. I'll share the solution if I come to it.
You're not showing your socket.io client code, but I think your second approach is better. Socket should only send/receive data. I assume you have at least 1 event in your socket for "receiveMessage" and 1 emit for "sendMessage".
I think you could, in your client-side socket:
When receiving a message (or I'm notified that I have a new message), add the message to your collection using Collection.add(message). Marionette will render that message for you.
When sending a message, just add it to your collection, or wait for a callback from the server (see the docs) to be sure it was received correctly before adding to the collection.
Never, never, append something to the view's HTML with jQuery if you're using Marionette view! ;)
For your initial message load, use Collection.fetch() the first time (for example, as soon as your socket is connected) to get all messages that were already on the server. From that point on, add individual messages as they come instead of fetching them all (you'll be saving bandwith).
What I do in a similar application, is send from the socket a "Hello" message upon first connection, that includes the data I need. Then I just do Collection.reset(data) with what was sent from the socket, and you're good to go. Your socket will initialize your collection and will be updating it one message at a time.
Hope it helps!

Categories