I'm using Meteor to create dynamic client-side interfaces to a server which is tightly coupled to an external hardware system (i.e., a robot). The server can be in one of three different states (A/B/C), but takes a while to transition between the three states. Depending on which state it is in, I want the client UIs to display a different screen.
I have approached this problem by storing the server state in a Meteor collection and publishing this state variable to all connected clients. A client can call a Meteor method to ask the server to change state (e.g., A->B). As a result, once the server has changed state, the new screen is displayed on the client. I'm using this state variable in client-side templates like this:
<template name="root">
{{#if equals server.state "A"}}
{{> UI_for_state_A}}
{{/if}}
{{#if equals server.state "B"}}
{{> UI_for_state_B}}
{{/if}}
etc.
</template>
However, the state transition on the server may take upwards of 10 seconds to complete. During this time, the client is still displaying the old screen. The client UI feels extremely unresponsive. I have also tried having the server update the state variable in the collection at the beginning of the state transition, but this causes the client to switch to the new screen before the server is ready, meaning none of the data is yet populated or available for the new screen.
What I would like to have is for the client to display a "loading" page while the server is in transition, so that a user knows something is going on, but isn't allowed to click any buttons during the transition.
What's the best practice in Meteor for doing this kind of UI synchronization?
Update: there can be multiple connected clients, which all must be synchronized with the server's state. In addition the server can change its state due to external factors, not necessarily on a client event. All these state updates need to be propagated to all clients.
It sounds like you just need to introduce a third, loading state. Set the value of the state in the server collection to this loading state before starting the state transition, and then set it to the final, changed state upon completion.
Note that your app might still feel a little sluggish on noticing the loading state - so you should define method stubs on the client that simply set the state:
if (Meteor.isClient) {
Meteor.methods({
yourMethodName: function() {
States.insert({state: "loading", timestamp: Date.now()}); // or however you would set the state on the server
}
});
}
This will allow your methods to use latency compensation, so the client will immediately enter the loading state rather than waiting for the server to send a publication update.
This is a typical example of a two-phase-commit strategy. It is often used in database transactions, but can be applied to any state machine.
Basically, you actually have the intermediate states and need to store and reference to those as well.
Please have a look at the mongodb two phase commit tutorial for an example.
There is also a third party meteor package on atmosphere that wraps a good state-machine library.
Related
Introduction
Note: in this project, we are not using Redux.
I have a stack profile screen with a simple button "Follow user" and a counter "Followers".
A user can navigate to this stack profile screen, and close it whenever he wants.
If the user press the "Follow" button:
3.1 The button will perform a connection to my server, in order to run the logic for "following user"
3.2 Then, when this action is done, to re-render the screen (the other user's followers counter), I am updating the other user's data (state of the profile screen), increasing his followers counter.
3.3 When this state is updated, a useEffect will run and add the new other user data to a map that I have in a context (in which I 'store' users in a visitedUsers map, in order to avoid requesting the database every time I visit the profile of a user I have recently visited, and preserve the correct data between multiple profile screens of the same user in different tabs of the app (some kind of non-persistant local memory and state manager)).
Problem
Steps 3.1, 3.2 and 3.3 always have to run, in order to ensure the app data consistency. But... the user can close the screen during the asynchronous job... so, steps 3.2 and 3.3 might not run.
Here is a diagram which shows the issue:
As you can see, if the profile screen is unmounted, the setState and useEffect (which updates the context) will not run and might causes memory leaks.
How can I solve this issue??
What I have thought to do:
The "isMounted" anti-pattern is not a good option because steps 3.2 and 3.3 always have to run.
It seems that I need to move this logic up into the components hierarchy (what I have thought to do, but seems really strange)... is this the only way?
Any ideas?
I had a similar problem once. I was trying to start an upload but allow the user navigate away from the component while the upload was running. My solution was a bit roundabout but, well, it worked.
So what I did was move the upload function to the root component (App.js, not index.js!) and instead of using Redux, I used ContextAPI and set a useEffect hook that would listen to changes in the data stored and, each time the data changed, it would trigger the upload function using the new data it finds. That way, the actual upload is not handled by the component it was initiated from and the user is free to navigate elsewhere without interrupting the upload. The only way to interrupt it would be to close the app.
I'm looking for web framework for interactive, real-time web apps without writing the Client (Browser), everything will be done by the Server.
There's such framework - LiveView in Phoenix (Elixir/Erlang), see demos below. I'm looking for something similar in JavaScript/TypeScript or Ruby.
How it works, it's better to demonstrate by example. Let's imagine we already have such framework in JavaScript and building an Autocomplete component. It would look almost like React.JS, but with the huge difference - it will be run on the Server:
class Autocomplete extends MagicServerSideReactComponent {
state = {
query: '',
suggestions: []
}
async search(e) {
const query = e.target.value
// This whole component runs on the Server, not in the Browser.
// So it has full access to the Server infrastructure and Database.
const suggestions = await db.find(`select like '${query}'`)
this.setState({ query, suggestions })
}
render() {
return <div>
<input value={this.state.query} onKeyPress={this.search}/>
{this.state.suggestions.map((name) => <div>{name}</div>)}
</div>
}
}
How it works:
When rendered first time:
- Autocomplete component get rendered on the Server and final HTML sent
to the Browser.
The Server remembers the `state` and the Virtual DOM - i.e. it's a
statefull Server, not usual Stateless node.js Server.
- Browser gets the HTML and renders it into DOM.
When user type something in the input:
- Browser serializes the Event and sends it to the Server,
something like `{ method: 'Autocomplete.search', event: '...' }`
- Server get the serialized Event and the previously stored Virtual DOM
for the given Web Page.
Then Server process the Event, and as a result the Virtual DOM
on the Server gets updated.
Then Server finds out the difference between the new and old Virtual DOM
and generates the DIFF.
Then Server sends the DOM DIFF to the Browser
- Browser gets the DOM DIFF and modify its DOM.
And the user see its Web Page updated with the search suggestions.
Do you know any such web frameworks in JavaScript or Ruby?
Please don't suggest frameworks that do something like that - but where you have to mutate DOM manually. Virtual DOM on the Server is important because it allows to update DOM automatically. It doesn't have to be exactly like React.JS but it should update DOM automatically, like React does.
P.S.
Why? Because of the first law of distributed systems - "Don't build distributed systems". It's simpler to build web app as one instead of distributing it into client and server.
Latency - yes, nothing is free, you have to pay for the simplicity and the latency will be the price. The interactions would be delayed - to travel to the server and back.
Performance - yes, Server is not stateless anymore, it's stateful, runs Virtual DOM and consume more resources.
You can take a look at marko by ebay (https://markojs.com/docs/server-side-rendering/) but I don't think you can find a framework/library with all your requirements.
Because of the first law of distributed systems - "Don't build distributed systems". It's simpler to build web app as one instead of distributing it into client and server.
The code you send to user with react or any other single page application framework is defining view. So you shouldn't consider it as a system. It's just html, css, js + user's data
Virtual DOM on the Server is important because it allows to update DOM
automatically.
Why the goal is to update DOM? DOM is just a view of your state/data. And your frontend application is just a mapper/hash function from your state to DOM. So if you only have your state in your server, you kind of have your DOM as well.
If you don't want to write both server and client, but still you want to have fancy frontend features with thousands of open source repos, you can try react + firebase.
This a snippet from the todo list tutorial. Variable checked is represented both on the client and the server side? How the client and the server communicate to make checked consistent?
Template.task.events({
'click .toggle-checked'() {
// Set the checked property to the opposite of its current value
Tasks.update(this._id, {
$set: { checked: ! this.checked },
});
},
'click .delete'() {
Tasks.remove(this._id);
},
});
checked is an attrubite defined on a Tasks object, as defined in this app.
In Meteor, the definitive record of this object is stored on the server (in MongoDB), however there is a client side cache that is also being manipulated here, known as MiniMongo. The Meteor framework does a lot of work in the background (via the DDP protocol) to keep the server and client side objects in sync.
In this case the following is happening when a user clicks on a checkbox (firing the 'click .toggle-checked' event code) in the Tasks.update method:
First update client side MiniMongo Cache - this is known as Optimistic UI, and enables the client UI to respond fast (without waiting for the server)
Send a message to the server (Meteor Method) that the client wants to update the Tasks object, by setting the clicked variable to a new value.
Message requesting update received by server, which checks this is a valid operation, and either processes it (updating MongoDB version of the Tasks object, or refuses to process the update as appropriate.
Server will send out a DDP update of the resulting status of the Tasks object to all clients that have subscribed to a publication that includes it.
Clients that have previously subscribed will receive this DDP update, and will replace their MiniMongo version with the Server's version of the Tasks object, ensuring that all Clients are in sync with the Server.
Now in the ideal case, when the server accepts the clients changes, the new version of Tasks received (in step 5) by the initiating client will match the object it optimistically updated (in step 1).
However by implementing all these steps the Meteor framework also synchronizes other clients, and handles the case when the server rejects the update, or possibly modifies additional fields, as appropriate for the application.
Luckily though, this is all handled by the Meteor framework, and all you need to do is call Tasks.update for all this magic to happen!
Meteor likes the blur the lines between client and server. There are things you can do to abstract code -- for instance, javascript files (among all files) inside the /server directory to restrict access to it. This means that client users can't see this code.
/client obviously is the opposite. You can check a file with isClient and isServer.
Now, what does this mean to your code?
Depending on where your code is, there are different access levels. However, inside the script, there basically is no difference. checked is known on server/client inside that script because that's how Meteor runs, the blurred line between client and server makes this possible.
Meteor employs something called "database everywhere" which means it doesn't matter where the code is called, because it will run.
I have an app built on Backbone.js and Django that is just something I'm building for practice.
It's a webchat app that let's users post a message and then saves it to a database, all while constantly fetching new data into a collection I have, so it "updates live" with the server.
On a basic level, the front-end application works with a message model to model a user's message, a messages collection to store all of the posts, and a chat view to render the collection in the chat window.
The code I use to constantly fetch new data from the server is:
// This code is inside the `chat` view's `initialize` function,
// so `that` is set to `this` (the `chat` view) higher up in the code.
window.setInterval(function() {
theColl.fetch({
success: function() {
that.render();
}
});
}, 2000);
I have don't have reset: true because I thought that it would be more efficient to just append new messages to the collection as they were retrieved from the server, instead of reloading every model already in it.
Is this an efficient way to run a simple webchat? As the application has to fetch more models, I've noticed that it gets extremely sluggish, and adds a delay to the user input. Here is a photo to show what I think is one problem (p.s. ignore the stupid messages I wrote):
As the server sends back new models and the collection refreshes, I think that my app is initializing old chat models again, because at this point in the application the collection itself only had 25 models inside of it.
Is there something I can do to stop the reinitialization of every model in the database? As you can see, even though there are only 25 unique models in the chat, and in the database, after letting the app run for 2 minutes or so I get models with CIDs up to 460.
Regardless of that, I've noticed that if I flood the server with new messages things go awry, like new messages appear out of place, and the server gets backed up. The latter may be because I'm using the cheapest option on Digital Ocean to host the app, but I'm not sure what the first problem comes from. It may be from my Django app's views.py or the rest of my Backbone code, I'm not really sure how to debug it.
I am using React with react-router and Reflux as my datastore, but I am unsure on how to best deal with persistence to allow page refresh.
My components connect to the store with Reflux.connect, but since the store fetches the data from a backend, it is not available yet when the Components first initialize and render.
When the user enters my app from the start, then all this data is loaded in order and available when it needs to be, but if further down a route you trigger a page refresh, react tries to render components that rely on data that is not there yet.
I solved this by constantly keeping a copy of data in LocalStorage and serving that from the Reflux store getInitialState(), so that all components get the data before they render.
I wonder if this is the proper way to do it. When for some reason the local storage data gets cleared or corrupted, the interface goes blank, because the components cannot access the correct data. Substructures and properties don't exist and trigger javascript errors. It seems like a messy and unreliable solution.
I am curious to know what patterns are used to solve this.
------ edit -----
To answer to the comment of WiredPrairie:
1) Why are you initializing components with data in getInitialState?
When my components use Reflux.connect, they don't have the data in their state yet on the first render as the store still needs to fetch its data. My views currently don't work gracefully with undefined data. By returning the locally stored cache from the Reflux store in getInitialState(), all connected components will get that data before their first render call.
2) What's causing a page refresh and why can't the data be loaded in the same manner as it was the first time?
It's mainly a workaround I had to build around livereload refreshing the page when I make edits (will look into using react-hotloader later but is not an options yet), but users can also just hit refresh when they are somewhere in my nested views and that would have the same effect. When they manually refresh, they are not entering the app at the start.
3) When components are wired to the change events of a store, why don't they update then?
They do update, but like I said they don't deal with empty data right now and on first render they will miss it waiting for the store to fetch things. I can make all my views work gracefully with empty data, but that would add a lot of boilerplate code.
From the replies so far, I get the feeling that what I'm doing with localStorage is the common way to do it. Cache stuff locally in localStorage or sessionStorage or something similar, and serve that data immediately after a refresh.
I should make my views a bit more robust by gracefully handing empty data on the odd occasion that localStorage doesn't work properly.
I've been caching each Store in sessionStorage when its emitChange() fires, and initializing the store from sessionStorage if cached data exists, or null values otherwise. This seems to work provided that the views can handle null values, which is probably a good idea anyway (it sounds like this is your main problem).
I'd suggest making your views handle the case of no data gracefully, initialize everything with null values if the cache isn't available, and then call the backend to update the null values with an Action whenever the data returns.
I haven't tried Reflux, but in regular Flux it would look like this (maybe you can apply the same principle):
var _data;
if (sessionStorage.PostStore)
_data = JSON.parse(sessionStorage.PostStore);
else {
_data = {
posts: null
};
BackendAPI.getPosts(function(err, posts) {
if (posts) {
PostActions.setPosts(posts);
}
});
}
...
AppDispatcher.register(function(payload) {
var action = payload.action;
switch (action.actionType) {
...
case Constants.SET_POSTS:
_data.posts= action.data.posts;
break;
default:
return true
}
// Update cache because state changed
sessionStorage.PostStore = JSON.stringify(_data);
PostStore.emitChange();
return true;
});