I am using the inbuilt socket capabilities of SailsJS which have been working great. Now I've come across a hurdle that I can't find any info on.
My model is set-up to populate some of the attributes using waterline model associations for example:
getAll: function() {
return Issue.find()
.sort('createdAt DESC')
.populate('author')
.populate('group')
.populate('tags')
.then(function (models) {
return [models];
});
},
This is working fine when calling this method through the API. However in the case where an update is made via a put and Issue.publishUpdate(id, update); is called, the attributes are then sent un-populated to subscribed clients. This is not the behaviour I had expected as publishCreate, on the otherhand, sends populated results.
To workaround the hurdle I could manually populate the attributes before sending the publishUpdate, however this doesn't seem like the right way to do it with Sails? So before I go that route I would be interested to hear anyone else's thoughts or experience.
Sails v0.10.1
The blueprint API calls publishUpdate, but it is actually a standalone method that can be used anywhere:
http://sailsjs.org/#/documentation/reference/websockets/resourceful-pubsub/publishUpdate.html
publishUpdate broadcasts the data that you pass in, so to broadcast populated data, you'll need to pass it in.
Related
I'm trying to make a simple todo app in order to understand how frontend and backend are connected. I read some of the websites showing a tutorial for using and connecting rest API, express server, and database, but still, I was not able to get the fake data from a database. Anyway, I wanted to check if my understanding of how they are connected and talk to each other is correct or not. So could you give some advice please?
First of all, I'm planning to use either Javascript & HTML or React for frontend, Express for server, and Postgres for the database. My plan is a user can add & delete his or her task. I have already created a server in my index.js file and created a database using psql command. Now if I type "" it takes me to the page saying "Hello" (I made this endpoint), and I'm failing to seed my data to the database. Here are my questions↓
After I was able to seed my fake data into the database, how should I get the data from the database and send to the frontend? I think in my index.js file, create a new endpoint something like "app.get("/api/todo", (res, req) => ..." and inside of the callback function, I should write something like "select * from [table name]". Also, form the front end, I should probably access certain endpoints using fetch. Is this correct?
Also, how can I store data which is sent from the frontend? For example, if I type my new todo to <input> field and click the add <button>, what is the sequence of events looks like? Adding event listener to button and connect to the server, then create post method in the server and insert data, kind of (?) <= sorry this part it's super unclear for me.
Displaying task on the frontend is also unclear for me. If I use an object like {task: clean up my room, finished: false (or 0 ?)} in the front end, it makes sense but, when I start using the database, I'm confused about how to display items that are not completed yet. In order to display each task, I won't use GET method to get the data from the database, right?
Also, do I need to use knex to solve this type of problem? (or better to have knex and why?)
I think my problem is I kind of know what frontend, server, database for, but not clear how they are connected with each other...
I also drew some diagrams as well, so I hope it helps you to understand my vague questions...
how should I get the data from the database and send to the frontend?
I think in my index.js file, create a new endpoint something like
"app.get("/api/todo", (res, req) => ..." and inside of the callback
function, I should write something like "select * from [table name]".
Typically you use a controller -> service -> repository pattern:
The controller is a thin layer, it's basically the callback method you refer to. It just takes parameters from the request, and forwards the request to the service in the form of a method call (i.e. expose some methods on the service and call those methods). It takes the response from the service layer and returns it to the client. If the service layer throws custom exceptions, you also handle them here, and send an appropriate response to the client (error status code, custom message).
The service takes the request and forwards it to the repository. In this layer, you can perform any custom business logic (by delegating to other isolated services). Also, this layers will take care of throwing custom exceptions, e.g. when an item was not found in the database (throw new NotFoundException)
The repository layer connects to the database. This is where you put the custom db logic (queries like you mention), eg when using a library like https://node-postgres.com/. You don't put any other logic here, the repo is just a connector to the db.
Also, form the front end, I should probably access certain endpoints
using fetch. Is this correct?
Yes.
Also, how can I store data which is sent from the frontend? For
example, if I type my new todo to field and click the add , what is
the sequence of events looks like? Adding event listener to button and
connect to the server, then create post method in the server and
insert data, kind of (?) <= sorry this part it's super unclear for me.
You have a few options:
Form submit
Ajax request, serialize the data in the form manually and send a POST request through ajax. Since you're considering a client library like React, I suggest using this approach.
Displaying task on the frontend is also unclear for me. If I use an
object like {task: clean up my room, finished: false (or 0 ?)} in the
front end, it makes sense but, when I start using the database, I'm
confused about how to display items that are not completed yet. In
order to display each task, I won't use GET method to get the data
from the database, right?
If you want to use REST, it typically implies that you're not using backend MVC / server rendering. As you mentioned React, you're opting for keeping client state and syncing with the server over REST.
What it means is that you keep all state in the frontend (in memory / localstorage) and just sync with the server. Typically what is applied is what is referred to as optimistic rendering; i.e. you just manage state in the frontend as if the server didn't exist; yet when the server fails (you see this in the ajax response), you can show an error in the UI, and rollback state.
Alternatively you can use spinners that wait until the server sync is complete. It makes for less interesting user perceived performance, but is just as valid technical wise.
Also, do I need to use knex to solve this type of problem? (or better
to have knex and why?) I think my problem is I kind of know what
frontend, server, database for, but not clear how they are connected
with each other...
Doesn't really matter what you use. Personally I would go with the stack:
Node Express (REST), but could be Koa, Restify...
React / Redux client side
For the backend repo layer you can use Knex if you want to, I have used node-postgres which worked well for me.
Additional info:
I would encourage you to take a look at the following, if you're doubtful how to write the REST endpoints: https://www.youtube.com/watch?v=PgrP6r-cFUQ
After I was able to seed my fake data into the database, how should I get the data from the database and send to the frontend? I think in my index.js file, create a new endpoint something like "app.get("/api/todo", (res, req) => ..." and inside of the callback function, I should write something like "select * from [table name]". Also, form the front end, I should probably access certain endpoints using fetch. Is this correct?
You are right here, you need to create an endpoint in your server, which will be responsible for getting data from Database. This same endpoint has to be consumed by your Frontend application, in case you are planning to use ReactJS. As soon as your app loads, you need to get the current userID and make a fetch call to the above-created endpoint and fetch the list of todos/any data for that matter pertaining to the concerned user.
Also, how can I store data which is sent from the frontend? For example, if I type my new todo to field and click the add , what is the sequence of events looks like? Adding event listener to button and connect to the server, then create post method in the server and insert data, kind of (?) <= sorry this part it's super unclear for me.
Okay, so far, you have connected your frontend to your backend, started the application, user is present and you have fetched the list of todos, if any available for that particular user.
Now coming to adding new todo the most minimal flow would look something like this,
User types the data in a form and submits the form
There is a form submit handler which will take the form data
Check for validation for the form data
Call the POST endpoint with payload as the form data
This Post endpoint will be responsible for saving the form data to DB
If an existing todo is being modified, then this should be handled using a PATCH request (Updating the state, if task is completed or not)
The next and possibly the last thing would be to delete the task, you can have a DELETE endpoint to remove the todo item from the list of todos
Displaying task on the frontend is also unclear for me. If I use an object like {task: clean up my room, finished: false (or 0 ?)} in the front end, it makes sense but, when I start using the database, I'm confused about how to display items that are not completed yet. In order to display each task, I won't use GET method to get the data from the database, right?
Okay, so as soon as you load the frontend for the first time, you will make a GET call to the server and fetch the list of TODOS. Store this somewhere in the application, probably redux store or just the application local state.
Going by what you have suggested already,
{task: 'some task name', finished: false, id: '123'}
Now anytime there has to be any kind of interaction with any of the TODO item, either PATCH or DELETE, you would use the id for each TODO and call the respective endpoint.
Also, do I need to use knex to solve this type of problem? (or better to have knex and why?) I think my problem is I kind of know what frontend, server, database for, but not clear how they are connected with each other...
In a nutshell or in the most minimal sense, think of Frontend as the presentation layer and backend and DB as the application layer.
the overall game is of sending some kind of request and receiving some response for those sent requests. Frontend is what enables any end-user to create these so-called requests, the backend (server & database) is where these requests are processed and response is sent back to the presentational layer for the end user to be notified.
These explanations are very minimal to make sure you get the gist of it. Since this question almost revolves around the entire scope of web development. I would suggest you read a few articles about both these layers and how they connect with each other.
You should also spend some time understanding what is RESTful API. That should be a great help.
I just read this post, and I do understand what the difference is. But still in my head I have the question. Can/Should I use it in the same App/Website? Say I want the AngularJs to fetch content and update my page, connecting to a REST api and all of that top stuff. But on top of that I also want a realtime chat, or to trigger events on other clients when there is an update or a message received.
Does Angular support that? Or I need to use something like Socket.io to trigger those events? Does it make sense to use both?
If someone could help me or point me to some good reading about that if there is a purpose for using both of them together.
Hope I'm clear enough. thank you for any help.
Javascript supports WebSocket, so you don't need an additional client side framework to use it. Please take a look at this $connection service declared in this WebSocket based AngularJS application.
Basically you can listen for messages:
$connection.listen(function (msg) { return msg.type == "CreatedTerminalEvent"; },
function (msg) {
addTerminal(msg);
$scope.$$phase || $scope.$apply();
});
Listen once (great for request/response):
$connection.listenOnce(function (data) {
return data.correlationId && data.correlationId == crrId;
}).then(function (data) {
$rootScope.addAlert({ msg: "Console " + data.terminalType + " created", type: "success" });
});
And send messages:
$connection.send({
type: "TerminalInputRequest",
input: cmd,
terminalId: $scope.terminalId,
correlationId: $connection.nextCorrelationId()
});
Usually, since a WebSocket connection is bidirectional and has a good support, you can also use it for getting data from the server in request/response model. You can have the two models:
Publisher/Subscriber: Where the client declares its interest in some topics and set handlers for messages with that topic, and then the server publish (or push) messages whenever it sees fit.
Request/response: Where the client sends a message with a requestID (or correlationId), and listen for a single response for that requestId.
Still, you can have both if you want, and use REST for getting data, and WebSocket for getting updates.
In server side, you may need to use Socket.io or whatever server side framework in order to have a backend with WebSocket support.
As noted in the answer in your linked post, Angular does not currently have built-in support for Websockets. So, you would need to directly use the Websockets API, or use an additional library like Socket.io.
However, to answer your question of if you should use both a REST api and Websockets in a single Angular application, there is no reason you can't have both standard XmlHttpRequest requests for interacting with a REST api, using $http or another data layer library such as BreezeJS, for certain functionality included in various parts of the application and also use Wesockets for another part (e.g. real time chat).
Angular is designed to assist with handling this type of scenario. A typical solution to would be to create one or more controllers to handle the application functionality and update your page and then creating separate Services or Factories that encapsulate the data management of each of your data end points (i.e. the REST api and the realtime chat server), which are then injected into the Controllers.
There is a great deal of information available on using angular services/factories for managing data connections. If you're looking for a resource to help guide you on how to build an Angular application and where data services would fit in, I would recommend checking out John Papa's AngularJS Styleguide, which includes a section on Data Services.
For more information about factories and services, you can check out AngularJS : When to use service instead of factory
I am trying this solution to fetch the current user before my application is rendered to the user:
Em.Route.extend({
model: function() {
var currentUserPromise = this.store.find('user', 'me');
return Em.RSVP.all([
currentUserPromise
]);
}
});
So, I query the user with the ID me, which is a constant that my API recognizes as the currently authenticated user.
It works really well most of the way. The issue is that Ember-Data immediately creates a User model in the store with this ID constant me and no other attributes. This instance is not removed after the response has arrived and Ember-Data correctly stores the real User model.
I can't figure out where Ember-Data creates this temporary instance and if there is a way to prevent this behavior.
Either a solution to prevent Ember-Data from creating the temp. instance or remove it when the real data arrives will be fine.
Anyone with a solution?
After consulting with #emberjs I found that a much cleaner solution to this challenge is to request the active user by requesting a session object from my API.
By exposing a /sessions endpoint in my API I could expose the current session—which can also include auth token, etc.—with a relationship to the currently logged in profile. That resulted in a JSON payload like this:
{
"sessions": [
{
"id": 1,
"user": { ... },
"accessToken": "abcdef1234567890"
}
]
}
This means that I will no longer have the issue that Ember-Data caches a bogus User object with the id 'me', instead I will now have a session object where the active user is correctly loaded into the store and have it available through the session.
I think you are going about things the wrong way. You're in a single page application, which means the user can log out or log in at any time, and any rendering is essentially already happening before and after that. You can only control what you allow to render, but not the when.
Also, you are explicitly returning a set of promises which resolves when the single promise it contains is resolved. This achieves the same:
Em.Route.extend({
model: function() {
return this.store.find('user', 'me');
}
});
You don't know when that will resolve, but you're asking to "fetch the user before the app is rendered to the user". That's not how it works in Ember, as you can't really delay rendering. (Technically you can, but that offers terrible user experience giving the feeling the browser has locked up).
What you would want to do instead is to modify what you are rendering based on the state of the logged in user, and on the backend side you only ever send what that user is allowed to see. Anything else is insecure, as you can't ever trust the browser. It's pretty trivial to make an Ember app think you are logged in.
The issue is that Ember-Data immediately creates a User model in the store with this ID constant me and no other attributes.
I'm not sure about that one; I'm guessing ember-data creates a placeholder while it waits for the population of the real data (if ever). It is as if you're creating a new empty object in code.
I can't figure out where Ember-Data creates this temporary instance
Ember-data is in its core basically a cache (why it is called the store) with various adapters to read from and write to that cache and an API to use it. In your example me is created in the cache only, at least until you decide to save it.
Ember data modifies the same record instead of creating a new record after the materialization. There might be some other problem while finding the records for table. For example In ember data 1.0.0-beta.8 the the following code logs correct id rather than printing 'me'.
this.store.find('user', 'me').then(function(user) {
console.log(user.get('id'));
});
I'm using v0.10.
Simple blueprint request for a messaging app (my model is named message)
var socket = io.connect('http://localhost:1337');
//initiate the request
socket.request('/message', {}, function(users) {});
socket.on('message', function(m){
console.log(m)
});
Using postman to to delete a message sends the delete to the client, however create does not send anything. Thank you.
UPDATE:
created this repo to reproduce the issues: https://github.com/jamescharlesworth/testProject
Take a look to http://beta.sailsjs.org/#/documentation/reference/Upgrading search that page for "socket" and there you'll find the differences on sails 0.10.
The most important one now is that instead of the "type of message", the first parameter of the "on" is the model. As it was previously used to call "message" to one of the types of messages, perhaps there is a remaining bug or something that filters your "message" model notifications when creating.
Have you tried naming your model different? Just to validate that the issue is with your model's name.
Additionally: if you want some transparent binding of models into an angular app, you can do it seamlessly with angular-sails-bind:
https://github.com/diegopamio/angular-sails-bind
I made it for my own project and then decided to put it as a separated library so everybody could benefit and I could have my first experience developing a bower package.
I hope it could help you.
In your example, you are using autosubscribe: ['destroy', 'create', 'update'],
while in sails 0.10 they have with "ed":
The events that were formerly create, update, and destroy are now created, updated, and destroyed.
That may be your issue.
In the ember guides on models it says (1) :
Ember Data is also designed to work with streaming APIs like socket.io, Firebase, or WebSockets. You can open a socket to your server and push changes to records into the store whenever they occur.
I tried writing a custom adapter that uses a websocket but i'm not getting very far. I couldn't find any working examples anywhere.
This is my totally unfinished prototype:
DS.WSAdapter = DS.Adapter.extend(Ember.Evented, {
websocket: undefined,
init: function () {
if(this.websocket === undefined)
{
this.websocket = new WebSocket('ws://localhost:8887');
this.websocket.onopen = function(e) {
console.log("Connection established!");
};
this.websocket.onmessage = function(e) {
// What to do here?
};
}
this._loadData();
},
//....
Can somone please help me with the websocket adapter?
My main problem is that I have no clue what to do when the websocket.onmessage() gets executed. I can't even access the store (using DS.get('defaultStore')) or anything
I don't have experience working directly with sockets in Ember, however I have recently completed an Ember Data + Firebase adapter which should follow very similar methodologies.
You should, at the least, be able to use it as inspiration:
https://github.com/sandersonet/ember-data-firebase
Firebase does provide an additional layer of abstraction from the sockets underneath, but the methodologies are very similar.
Have a look at http://emberjs.com/guides/models/frequently-asked-questions/#toc_how-do-i-inform-ember-data-about-new-records-created-on-the-backend
Some applications may want to add or update records in the store
without requesting the record via store.find. To accomplish this you
can use the DS.Store's push, pushPayload, or update methods. This is
useful for web applications that have a channel (such as SSE or Web
Sockets) to notify it of new or updated records on the backend.
Basically, you need to deserialize data you receive in your onmessage hook and push new objects to the data store using store.push('model', record) or alternative methods.