I'm implementing a JavaScript-based Vaadin component that will need to show and update a relatively large data set. I'm doing this by extending AbstractJavaScriptComponent.
I'm trying to keep the JS side as "dumb" as possible, delegating user interactions to the server using RPC, and which updates the shared state. Then the JS connector wrapper's onStateChange function is called with the new state, which causes the DOM to be updated accordingly.
I have 2 problems:
I don't want to transfer the whole data set each time a small part gets updated.
I don't want to entirely rebuild the UI each time either.
I can solve the second problem by keeping the previous state and comparing parts of it to find out what changed and only make the necessary DOM changes.
But that still leaves the first problem.
Do I have to stop using Vaadin's shared state mechanism and instead only use RPC for communicating the changes to the state?
Update:
I've been doing some testing, and it certainly appears that Vaadin's shared state mechanism is horrible in terms of efficiency:
Whenever the component calls getState() in order to update some property in the state object (or even without updating anything), the whole state object is transferred. The only way to avoid this, as far as I can see, is to not use the shared state mechanism and instead use RPC calls to communicate specific state changes to client.
There are some issues with the RPC approach that will need to be resolved, for example: if you change a value multiple times within a single request/response cycle, you don't want to make the RPC call multiple times. Instead, you want only the last value to be sent just like the shared state mechanism only sends the final state in the response. You can keep dirty flags for each part of the state that you want to send separately (or just keep a copy of the previous state and compare), but then you need to somehow trigger the RPC call at the end of the request handling. How can this be done?
Any ideas on this are welcome!
Update 2:
Vaadin 8 fixes the root issue: it sends only the changed state properties. Also, it doesn't call onStateChange() on the JS connector anymore when only doing an RPC call (and not changing any state).
OP is correct in stating that shared state synchronisation is inefficient for AbstractJavaScriptComponent-based components. The entire state object is serialised and made available to the Javascript connector's onStateChange method whenever the connector is marked as dirty. Other non-javascript components handle state updates more intelligently by only sending changes. The exact place in the code where this happens is line 97 in com.vaadin.server.LegacyCommunicationManager.java
boolean supportsDiffState = !JavaScriptConnectorState.class
.isAssignableFrom(stateType);
I'm not sure why state update is handled differently for AbstractJavaScriptComponent-based components. Maybe it's to simplify the javascript connector and remove the need to assemble a complete state object from deltas. It would be great if this could be addressed in a future version.
As you suggest, you could dispense with JavaScriptComponentState completely and rely on server->client RPC for updates. Keep dirty flags in you server-side component or compare old state and new state by any mechanism you want.
To coalesce the changes and send only one RPC call for each change, you could override beforeClientResponse(boolean initial) in your server-side component. This is called just before sending a response to the client and is your chance to add a set of RPC calls to update the client-side component.
Alternatively, you could override encodeState where you have free-reign to send exactly whatever JSON you like to the client. You could choose to add a list of changes to the base JSON object returned by super.encodeSate. Your javascript connector could interpret as appropriate in its onStateChange method.
Edited to add: calling getState() in your server-side component will mark the connector as dirty. If you want to get state without marking it as dirty then use getState(false) instead.
Following our discussion about this, I've created a drop-in replacement for AbstractJavaScriptComponent that transmits state deltas and includes some extra enhancements. It's in the very early stages but should be useful.
https://github.com/emuanalytics/vaadin-enhancedjavascript
The solution is deceptively simple: basically re-enabling state difference calculation by bypassing this line of code in com.vaadin.server.LegacyCommunicationManager.java:
boolean supportsDiffState = !JavaScriptConnectorState.class
.isAssignableFrom(stateType);
The implementation of the solution is complicated by the fact that the Vaadin classes are not easily extended so I've had to copy and re-implement 6 classes.
Vaadin's shared state works exactly like you want out of the box: when a component is added to the DOM first time, the whole shared state is transferred from server to client, so that it's possible to display the component. After that, only changes are transferred. For example, one changes the caption of a visible component by calling component.setCaption("new caption"), Vaadin only transfers that new caption text to client and "merges" that to the client-side shared state instance of the component.
Related
Following the flux concepts we can get the next assertions for which I couldn't find explanations.
Every store will receive every action.
Why? My suggestion: since a store contents some business-logic we have to provide it with all possible changes and data so the store can decide what to do with them on its own.
The data in a store must only be mutated by responding to an action.
Why? My suggestion: the reason is violation of unidirectional data-flow in case of not responding to an action.
Every time a store's data changes it must emit a "change" event.
Why? I can't get this point.
Flux is just a way of managing the data flow of your application, so it is up to the developer to make sure this actually happens. But I'll try to paint a picture of why these concepts are a part of Flux.
Every store will receive every action.
If you have only one dispatcher in your application, every store will listen to actions dispatched through that dispatcher. It is up to you whether or not the store should act on the action dispatched, but to be able to react on it the store has to know of it.
Not all actions should lead to changes in a store, though. But the dispatcher simply doesn't care, because it won't know anything about the store implementation. It's just telling all stores that this action happened, do what you want with it or go on with your life without caring.
The data in a store must only be mutated by responding to an action.
You're right that doing it with a different approach could be violation of unidirectional data-flow. Doing things this way makes sure all parts of your application has the correct state based on the actions that happens.
By not doing it this way you would let go of one of the flux strengths. Update your store based on dispatched actions, and other stores will also be aware that the action happened, and thereby react to it if they want to. If you update the store directly you will end up having no clear picture of what parts of your application that are altering the state of your store.
Every time a store's data changes it must emit a "change" event.
People often describe the stores in a flux application as the source of truth. When a store's data changes, the basis for the visualization of your data changes. You want to be confident that if my store holds a certain value, this is what my application uses as it's data.
It's related to the first quote here. The store doesn't know if a listener depends on it's data. By emitting a change, it will let all listeners know that hey, I changed. Make sure you have all my latest changes. If you don't emit change, the listener could end up displaying something based on old data.
All of these statements are related to the same thing: If an action happens in your application, don't make any assumptions about which part of your application that wants to know the details of it. Make sure everyone can act on it, if they want to.
It is my understanding that you are not supposed to change states in the render function cause that would cause and infinite re render or the component.
This makes perfect sense, but I find myself in a particular situation. I am building an offline application and I am using an offline storage system to retrieve data. Now, whenever a method is called to get certain data, cache is checked, if it is not expired the component will be able to access the data and therefore nothing happens, but if it is expired there is a call to the API, the data is updated and the interested components re-rendered.
These methods will change the state of the component the first time they are called because they are going to the API grabbing the new data and re-rendering, and then they will not change the state anymore because the data will already be in cache.
Now, I could call these methods in component will mount, and that is what I am doing now, but if I am forced to re call them, I need to unmount and remount the components. Is this the only possible way of doing this?
Thanks!
Well the first step is understanding that state management and rendering needs to be decoupled which you got already.
Now what you can do is consider your external state/cache element as an observable object (ie. I want to do something like observableObject.listen('change', onChangeHandler); you can use EventsEmitter from the events library). You do that listening on componentDidMount and clean up in componentWillUnmout. That onChangeHandler is very simple: this.setState({ value: observableObject.value }) which will trigger a component re-render which should be a pure function that outputs DOM nodes depending on props being passed and it's own state.
This means that your logic of checking if the cache is invalid is not on a per request of the value (inside rendering) but rather think of the object as self contained. It regularly checks if itself needs to notify its listeners that it changed. Since JS does not do parallel execution you don't have to deal with threads and synchronization. You know that at the point in time your render function executes it will have the latest value. If after rendering the logic that checks for cache executes and it sees that it needs to be updated it simply notifies as said earlier its listeners and that makes your component re-render because your onChangeHandler changed the state.
Hope I helped.
In the last week, I’ve been trying to make sense how two-way data binding (Angular) and one-way data flow (React/Flux) are different. They say that one-way data flow is more powerful and easier to understand and follow: it is deterministic and helps avoiding side-effects. In my newbie eyes though, they both look pretty much the same: the view listens to the model, and the model reacts on actions done to the view. Both claim that the model is the single source of truth.
Could anybody comprehensively explain in understandable way how they are really different and how one-way data flow is more beneficial and easier to reason about?
Angular's two-way data binding
It's made possible by a mechanism that synchronizes the view and the model whenever either change. In Angular, you update a variable and its change detection mechanism will take care of updating the view, and viceversa. What's the problem? You don't control the change detection mechanism. I found myself having to resort to ChangeDetectorRef.detectChanges or NgZone.run to force the view to update.
To not dive too deep into change detection in Angular, you trust it will update what you need when you change a variable, or when it gets changed after an observable resolves, but you'll find you have no idea how and when it runs, and sometimes it will not update your view after a variable changes. Needless to say, it can sometimes
be a pain to find where and when a problem occured.
React's one-way data flow
It means that the view always gets its state from the model. To update the view, you need to update the model first, and then redraw the view. React makes the view redrawing process extremely efficient because it compares not the actual DOM but a virtual DOM it keeps on memory. But how does change detection work in this dynamic? Well, you trigger it manually.
In React, you set the state's new value, which then causes a ReactDOM.render, which causes the DOM comparing/updating process. In React/Redux you dispatch actions which update the store (single source of truth) and then the rest. Point is, you always know when the stuff changes, and what caused the change. This makes problem solving quite straight forward. If your app depends on the state, you look at it before and after the action that triggered the change, and you make sure variables have the value they're supposed to.
Implementations aside
From a platform independent point of view, they're not so different. What separates one-way flow from two-way binding is a variable update on change. So your impression that that they're conceptually not too far from each other is not too divorced from their practical uses.
In Angular you have many controllers. One example would be a user triggering an action on View 1 that is managed by Controller 1. Controller 1 does something but also fires an event that is caught by another Controller 2. Controller 2 updates some property on the $scope and View 2 is suddenly changed.
Suddenly an operation on View 1, updated View 2. If we now throw in some Async callbacks and a bit more event chains, you might no longer know exactly when/how your views are being updated.
With Flux/Redux, you have a one way data flow. The view never updates the model, the views can only dispatch an action (intention to update), but lets the store/reducer deciding how to handle the update. You can more easily reason about the data flow because you can easily see which actions can be fired by each view. Then follow up to see how that action is being handled by the store and you can know exactly what can be updated.
Data flow here is a flow of write events - i.e. state updates
These events are flowing between views and controllers (and services, such as HTTP backends)
One-way flow is basically the giant cycle:
app view uses (reads, not writes) app state to render
when application gets some stimuli from outside (user typed some text in input field, or result of HTTP request has arrived), it emits write event - or, in Redux/Flux slang, dispatches an action
all events, from all controllers and views, are flowing into the single sink - dispatch function (reducer); although the nature of dispatch function allows it to be composed from simpler dispatch functions, conceptually, there's only one dispatcher for the whole app
dispatcher uses an event to figure out which part of the state is to be updated
go to start
Two-way flow aka data binding binds two pieces of state: in most cases, one inside the controller (e. g. some variable), and one inside the view (e. g. contents of textbox). Binding means that, when one piece changes, the other piece changes as well and gets the same value, so you can pretend that there's only one piece of state involved (while there's two actually). Write events are going back and forth between controllers and views - thus two-way.
Data-binding is cool when you need to figure out what variable holds the contents of this particular textbox - it shows immediately. But it requires complex framework to maintain the illusion of one piece of state where there's two pieces really. Usually you'll be forced to use framework-specific syntax to write your views' code - i. e. to learn yet another language.
One-way data flow is cool when you can leverage that extra entity - events flow. And, usually, you can - it's useful for Undo/Redo, user actions replay (e. g. for debug), replication, etc, etc. And the code to support this is much, much simpler, and usually can be written in plain JavaScript instead of framework-specific syntax. On the other hand, since you no longer have data-binding, it no longer saves you some boilerplate.
Also, see great visual explanation in this answer: https://stackoverflow.com/a/37566693/1643115. Single-headed and two-headed arrows visually represents one-way and two-way data flow respectively.
Let's say your app is just a wizard flow, but it has some complex interactions i.e. one step might change a following step behavior.
Your app is running great, but one day an user reports a bug on one of the tricky steps.
How does debugging would work on two-way binding and one-way binding?
Two-way binding
I'd start checking what behavior is different and with some luck, get to the same point as the user and pinpoint the bug. But at the same time there might be some weird interaction between different parts of the app. I might have some data-binding that is incorrect (e.g. replicating the model state but not binding) or other weird intricacy between components that is hard to debug. It might be hard to isolate the bug.
One-way binding
You just grab the state object. It has all the information of the app currently in a big javascript object. You load the same state in your development environment, there is a big chance your app will behave exactly the same. You can even write a test with the given state for regression and pinpoint the exact problem that is happening.
Conclusion
In a few words, one-way binding makes it very easy to debug complex apps. You don't have to do much then copy over the current state of the user.
Even that doesn't work, you can log the actions as well. There isn't AFAIR an easy way to track all the state modifying actions on Angular, for instance. With Redux it's pretty, pretty easy.
I have an app that uses a sync API to get its data, and requires to store all the data locally.
The data set itself is very large, and I am reluctant to store it in memory, since it can contains thousands of records. Since I don't think the actual data structure is relevant, let's assume I am building an email client that needs to be accessible offline, and that I want my storage mechanism to be IndexedDB (which is async).
I know that a simple solution would be to not have the data structure as part of my state object and only populate the state with the required data (eg - store email content on state when EMAIL_OPEN action is triggered). This is quite simple, especially with redux-thunk.
However, this would mean I need to compromise 2 things:
The user data is no longer part of the "application state", although in truth it is. Since the sync behavior is complex, and removing it from the app state machine will hurt the elegance of the redux concepts (the way I understand them)
I really like the redux architecture and would like all of my logic to go through it, not just the view state.
Are there any best-practices on how to use redux with a not-in-memory state properties? The thing I find hardest to wrap my head around is that redux relies on synchronous APIs, and so I cannot replace my state object with an async state object (unless I remove redux completely and replace it with my own, async implementation and connector).
I couldn't find an answer using Google, but if there are already good resources on the subject I would love to be pointed out as well.
UPDATE:
Question was answered but wanted to give a better explantation into how I implemented it, in case someone runs into it:
The main idea is to maintain change lists of both client and server using simply redux reducers, and use a connector to listen to these change lists to update IDB, and also to update the server with client changes:
When client makes changes, use reducers to update client change list.
When server sends updates, use reducers to update server change list.
A connector listens to store, and on state change updates IDB. Also maintain internal list of items that were modified.
When updating the server, use list of modified items to pull delta from IDB and send to server.
When accessing the data, use normal actions to pull from IDB (eg using redux-thunk)
The only caveat with this approach is that since the real state is stored in IDB, so we do lose some of the value of having one state object (and also harder to rewind/fast-forward state)
I think your first hunch is correct. If(!) you can't store everything in the store, you have to store less in the store. But I believe I can make that solution sound much better:
IndexedDB just becomes another endpoint, much like any server API you consume. When you fetch data from the server, you forward it to IndexedDB, from where your store is then populated. The store gets just what it needs and caches it as long as it doesn't get too big or stale.
It's really not different than, say, Facebook consuming their API. There's never all the data for a user in the store. References are implemented with IDs and these are loaded when required.
You can keep all your logic in redux. Just create actions as usual for user actions and data changes, get the data you need and process it. The interface is still completely defined by the user data because you always have the information in the store that is needed to GET TO the rest of it when needed. It's just somewhat condensed, i. e. you only save the total number of messages or the IDs of a mailbox until the user navigates to it.
Struggling to find or come up with an elegant answer to this one:
If I have multiple dynamic react components that are listening to one flux store to update their child components is it possible to emit changes to specific components rather than emitting changes to all the components that are registered to listen to changes on that store?
E.G: A dynamic component has a button and when clicked its tells the flux store to send some data to API. The dynamic component will it update its child view depending on the response and change emitted by the flux store. But since all the dynamic components are listening to the store they will all update their child views which is the undesired behaviour. Ideally the flux store could identify which component to emit the change to, or the components can identify that change is not for them.
Is this possible? Or does it go against flux principles?
I don't know if it violate flux architecture, but it seems not leveraging some beauties of it.
The beauty of a simple emit change (without change detail) is that a store wouldn't need to have explicit knowledge on views, also, with the React Virtual Dom framework, it shouldn't cost too much performance hit.
To further optimize the performance, you can implement shouldComponentUpdate on your React view (base on the differences in it's own properties), to avoid triggering the tree-diff algorithm.
See this: https://facebook.github.io/react/docs/component-specs.html
== Add more info ==
In more traditional MVC, the model will emit changes to a particular source and with particular details, e.g.
this.emit({
details: { x: 'x', y: 'y' },
source: objectA
)};
The view (or controller) that receive this needs such detail to update it's Dom, you will call the update(changes.details) instead of the initial render() method because Dom manipulation is expensive.
ReactJS 'solved' this by having another virtual Dom layer, which use pure Javascript to compute the 'optimal' differences in Dom manipulation, so in React, you never have a method call update(), you will always call render() base on current state of the view, and React does the optimization for you.
So using Flux with React, your store can just emit change without any details and the views that listen to it can just render with 'optimal' Dom manipulation (so if it's state hasn't been changed, there will be no Dom manipulation).
But of course, you will say in this case React will still trigger the virtual Dom diff computation, which still cost something. So to further optimize it, you can implement shouldComponentUpdate on a view that contains big sub-tree (base on it's own state), to avoid React to run the diff computation.
The beauty of emit a simple change, besides easier code, is that Store can be pretty much decoupled from view.
For example if you trigger specific change details for particular views, then you will need to remove or change code in store(s) when the view is not listening the that store anymore.
It does not go against flux principle but beware not having only one big store, sometime it's better to split in several tiny store.
But I think I understand your use case, one store containing a collection of similar objects (like a backbone collection).
So lets say your store receive a new object or an array of new object (or things to update in your store), you have a register function which will add this object (or update) to your store.
For sure this object has an id field (or something similar). Then for each new object of your array you just received you'll emit the id.
And your view are binded to their id as change event. Basically you use your store like an array, when the array is change you emit the key as event. Your view listen to this key/id and then get the specific data from your store still using this id/key.
Hope it's clear, let me know.