We Keep Coding

Update data without reloading [duplicate]

I'm very very new to node.js, but there's actually only one simple thing that I am trying to achieve by learning the language.
I'd like to create a webpage, where by the code in a specific "div" can be hotswapped on the fly to users currently looking at that page. (ie. the div contains some text, but then an image replaces it.)
Ideally, the swap would be executed manually by the the webpage's admin through the click of a button, or some code fired off on the server or something. Regular viewers to the webpage would not be able to do this - they only see the live changes on the page.
real-life example:
live internet broadcast is off-air, therefore the "div" contains "off-air" text.
live hotswap of code happens when broadcast goes on-air, and the viewers of the webpage now see the html5 broadcast player in the "div" instead.
What's the simplest way to go about doing this for a node.js newbie?
Many thanks :)

Take a look at Socket.IO http://socket.io/#how-to-use
when the server decides to broadcast a change use:
io.sockets.emit('update-msg', { data: 'this is the data'});
on the client first connect socket.io and then wait for the "update-msg" event and update your dom:
var socket = io.connect('http://localhost');
socket.on('update-msg', function (msg) {
console.log(msg);
$('#mydiv').html(msg.data)
});

I created a system/methodology to live update (hot reload) front-end code using RequireJS and Node.js. I made sure it worked with both React and Backbone. You can read about it here:
https://medium.com/#the1mills/hot-reloading-with-react-requirejs-7b2aa6cb06e1
the basic steps involved in doing this yourself:
gulp.js watchers listen for filesystem changes
socket.io server in gulpfile sends a message to all browser clients
with the path of the file that changed
client deletes cache representing that file/module, and re-requires
it (using AJAX to pull it from the server filesystem)
front-end app is configured / designed to re-evaluate all references
to the modules that it wishes to hot-reload, in this case, only JS
views, templates and CSS are available to hot reload - the router,
controllers, datastores (Backbone Collections and Models) are not
configured yet. I do suspect all files could be hot reloaded with
the only exception being data stores.

Gatsby: Re-execute page queries manually

Do you know if it's possible to re-execute Gatsby page queries (normal queries) manually?
Note, This should happen in dev mode while gatsby develop runs.
Background info: I'm trying to set up a draft environment with Gatsby and a Headless CMS (Craft CMS in my case). I want gatsby develop to run on, say, heroku. The CMS requests a Gatsby page, passing a specific draft-token as an URL param, and then the page queries should be re-executed, using the token to re-fetch the draft content from the CMS rather than the published content.
I'm hooking into the token-request via a middleware defined in gatsby-config.js. This is all based on https://gist.github.com/monachilada/af7e92a86e0d27ba47a8597ac4e4b105
I tried
createSchemaCustomization({ refresh: true }).then(() => {
sourceNodes()
})
but this completely re-creates all pages. I really only want the page queries to be extracted/executed.

Probably you are looking for this. Basically, you need to set an environment variable (ENABLE_GATSBY_REFRESH_ENDPOINT) which opens and exposes a /__refresh webhook that is able to receive POST requests to refresh the sourced content. This exposed webhook can be triggered whenever remote data changes, which means you can update your data without re-launching the development server.
You can also trigger it manually using: curl -X POST http://localhost:8000/__refresh
If you need a detailed explanation of how to set .env variables in Gatsby just tell me and I will provide a detailed explanation. But you just need to create a .env file with your variables (ENABLE_GATSBY_REFRESH_ENDPOINT=true) and place this snippet in your gatsby-config.js:
require("dotenv").config({
path: `.env.${activeEnv}`,
})
Of course, it will only work under the development environment but in this case, it fits your requirements.

Rebuild for all is needed f.e. when you have indexing pages.
It looks like you need some logic to conditionally call createPage (with all data refetched) or even conditionally fetch data for selected pages only.
If amount (of pages) is relatively not so big I would fetch for all data to get page update times. Then in loop conditionally (time within a few minutes - no needs to pass parameter) call createPage.
If develop doesn't call 'createPage' on /__refresh ... dive deeper into gatsby code and find logic and way to modify redux touched nodes.
... or search for other optimization techniques you can use for this scenario (queried data cached into json files?).

Interactive web apps with react.js on the server

I'm looking for web framework for interactive, real-time web apps without writing the Client (Browser), everything will be done by the Server.
There's such framework - LiveView in Phoenix (Elixir/Erlang), see demos below. I'm looking for something similar in JavaScript/TypeScript or Ruby.
How it works, it's better to demonstrate by example. Let's imagine we already have such framework in JavaScript and building an Autocomplete component. It would look almost like React.JS, but with the huge difference - it will be run on the Server:
class Autocomplete extends MagicServerSideReactComponent {
state = {
query: '',
suggestions: []
}
async search(e) {
const query = e.target.value
// This whole component runs on the Server, not in the Browser.
// So it has full access to the Server infrastructure and Database.
const suggestions = await db.find(`select like '${query}'`)
this.setState({ query, suggestions })
}
render() {
return <div>
<input value={this.state.query} onKeyPress={this.search}/>
{this.state.suggestions.map((name) => <div>{name}</div>)}
</div>
}
}
How it works:
When rendered first time:
- Autocomplete component get rendered on the Server and final HTML sent
to the Browser.
The Server remembers the `state` and the Virtual DOM - i.e. it's a
statefull Server, not usual Stateless node.js Server.
- Browser gets the HTML and renders it into DOM.
When user type something in the input:
- Browser serializes the Event and sends it to the Server,
something like `{ method: 'Autocomplete.search', event: '...' }`
- Server get the serialized Event and the previously stored Virtual DOM
for the given Web Page.
Then Server process the Event, and as a result the Virtual DOM
on the Server gets updated.
Then Server finds out the difference between the new and old Virtual DOM
and generates the DIFF.
Then Server sends the DOM DIFF to the Browser
- Browser gets the DOM DIFF and modify its DOM.
And the user see its Web Page updated with the search suggestions.
Do you know any such web frameworks in JavaScript or Ruby?
Please don't suggest frameworks that do something like that - but where you have to mutate DOM manually. Virtual DOM on the Server is important because it allows to update DOM automatically. It doesn't have to be exactly like React.JS but it should update DOM automatically, like React does.
P.S.
Why? Because of the first law of distributed systems - "Don't build distributed systems". It's simpler to build web app as one instead of distributing it into client and server.
Latency - yes, nothing is free, you have to pay for the simplicity and the latency will be the price. The interactions would be delayed - to travel to the server and back.
Performance - yes, Server is not stateless anymore, it's stateful, runs Virtual DOM and consume more resources.

You can take a look at marko by ebay (https://markojs.com/docs/server-side-rendering/) but I don't think you can find a framework/library with all your requirements.
Because of the first law of distributed systems - "Don't build distributed systems". It's simpler to build web app as one instead of distributing it into client and server.
The code you send to user with react or any other single page application framework is defining view. So you shouldn't consider it as a system. It's just html, css, js + user's data
Virtual DOM on the Server is important because it allows to update DOM
automatically.
Why the goal is to update DOM? DOM is just a view of your state/data. And your frontend application is just a mapper/hash function from your state to DOM. So if you only have your state in your server, you kind of have your DOM as well.
If you don't want to write both server and client, but still you want to have fancy frontend features with thousands of open source repos, you can try react + firebase.

How to update service worker cache in PWA?

I use service worker with sw-toolbox library. My PWA caches everything except API queries (images, css, js, html). But what if some files will be changed someday. Or what if service-worker.js will be changed.
How application should know about changes in files?
My service-worker.js:
'use strict';
importScripts('./build/sw-toolbox.js');
self.toolbox.options.cache = {
name: 'ionic-cache'
};
// pre-cache our key assets
self.toolbox.precache(
[
'./build/main.js',
'./build/main.css',
'./build/polyfills.js',
'index.html',
'manifest.json'
]
);
// dynamically cache any other local assets
self.toolbox.router.any('/*', self.toolbox.cacheFirst);
// for any other requests go to the network, cache,
// and then only use that cached resource if your user goes offline
self.toolbox.router.default = self.toolbox.networkFirst;
I don't know what is the usual method to update cache in PWA. Maybe PWA should send AJAX request in background and check UI version?

AFAIK the sw_toolbox does not have a strategy for cache with network update. This is really what you want I think.
You want to modify the cache-network race strategy - > https://jakearchibald.com/2014/offline-cookbook/#cache-network-race
Instead of just letting the loser fade away, once the network responds you will want to update the client. This is a little more advanced that I have time or time to explain here.
I would post a message to the client to let it know there is an update. You may want to alert the user to the update or just force the update.
I don't consider this to be an edge case, but a very common, but advanced scenario. I hope to publish a more detailed solution soon.

There is nice solution written here where he states (in a nutshell) to either not use cache-first strategy or update a UX pattern of displaying a "Reload for the latest updates."

I dealt with services workers without using any library and the solution I ended up coming up with involved a bit of server side code and some client side. The strategy in a nutshell
Firstly the variables you will need and where:
On the server side have a "service worker version" variable (Put this in a database or config file if you are using something like php that will update immediately on the server side without requiring a redeploy. Let's call it serverSWVersion
On one of the javascript files you cache (I have a javascript file dedicated to this) have a global variable that will also be the "service worker version". Let's call it clientSWVersion
Now how to use the two:
Whenever a person lands on the page make an ajax call to your server to get the serverSWVersion value. Compare this with the clientSWVersion value.
If the values are different that means your web app version is not
the latest.
If this is the case then unregister the service worker and refresh the page so that the service worker will be re registered and the new files will be cached.
What to actually do when new file is available
Update the serviceSWVersion and clientSWVersion variables and upload to server where applicable.
When a person visits again then the service worker should be re registered and all the cached files will be retrieved.
I have provided a php server side based code that I used while I was implementing this strategy. It should show you the principles. Just drop the "Exercise" folder in a htdocs of a php server and it should work without you having to do anything else. I hope you find it useful... And remember you could just use a database instead of a config file to store the server side service worker variable if you are using some other server instead of php:
Zip file with code:
ServiceWorkerExercise.zip

When a service worker is altered, the browser will install it, but the new version will not be activated until the browser tab or PWA app window is closed and re-opened.
So, if you change the cache name, the new cache will not serve any files until the browser re-opens, nor will the old cache be deleted until that time.
You can detect service worker changes in your page javascript using registration.onupdatefound and ask the user to close and re-open the window - something like this:
// register the service worker
navigator.serviceWorker.register('sw.js').then(function(registration)
{
registration.onupdatefound = function()
{
console.log("ServiceWorker update found.");
alert("A new version is available - please close this browser tab or app window and re-open to update ... ");
}
}, function(err)
{
console.log('ServiceWorker registration failed: ', err);
});

change self.toolbox.router.any('/', self.toolbox.cacheFirst); to self.toolbox.router.any('/', self.toolbox.fastest);

Update data in (really) real-time with Javascript

hi
I want to build a control panel for a web art application that needs to run in fullscreen, so all this panel, that controls stuff like colors and speed values, have to be located at a different window.
My idea is to have a database storing all these values and when I make a change in the control panel window the corresponding variable in the application window gets updated too. So, it's basically a real-time update that I could do with AJAX setting a interval to keep checking for changes BUT my problem is: I can't wait 30 seconds or so for the update to happen and I guess a every-1-second AJAX request would be impossible.
Final question: is there a way to create a sort of a listener to changes in the database and fire the update event in the main application only immediately after I change some value in the control panel? Does Angular or another framework have this capability?
(Sorry for the long explanation, but I hope my question is clearer by offering the context [: )

A web socket powered application would have this benefit. This carries a bit more complexity on the back end, but has the benefit of making your application as close to real-time as can be reasonably expected.
The Mozilla Development Network has some good documentation on websockets.
On the front end, the WebSocket object should work for you on most modern browsers.
I'm not sure what your back end is written in, but Socket.IO for Node.js and Tornado for Python will make your applications web-socket capable

If one window is opening the other windows via JavaScript, you can keep the reference to the opened window and use otherWindow.postMessage to pass messages across
"Parent" window looks like
// set up to receive messages
window.addEventListener('message', function (e) {
if (e.origin !== 'http://my.url')
return; // ignore unknown source
console.log(e.message);
});
// set up to send messages
var otherWindow = window.open('/foo', '_blank');
otherWindow.postMessage('hello world', 'http://my.url');
"Child" windows look similar
// same setup to recieve
// ...
// set up to send
var otherWindow = window.opener;
// ... same as before

For the realtime I would recommend using a library like socket.io or using a database like firebase.
For the fullscreen I would recommend using a library like angular-screenfull

i use https://pushjs.io/, had exactly the same problem and this is a really simple solution for your problem. It is capable of sending and listening to events without any database interference.