We Keep Coding

Better alternative to pinging the database over and over?

I want to create a dashboard that automatically updates when new data is posted.
My first thought was to just make a javascript function and put a fetch statement in it and then loop the function every second or every couple of seconds...
Obviously, this is not a great solution. But I don't know what the better way is...
Some notes:
-PHP Server-Side Language
-Ran on Localhost so traffic is not going over the internet
Can anyone advise what I should be doing or if this is an acceptable approach?
Thanks in advance!

Server Side:
You can look for any onUpdate events if your database supports any such events
Or else just run a query in a timed interval to fetch new updates form the database (Connection to database is made just once and all subsequent requests go through the same connection. Hence this isn't a bad approach)
But when it comes to client side and receiving those updates, you can make it efficient in either of the two ways:
[Simple] Use Socket IO - Push an event with your new data and listen to them on the client side. (This way socket connection is made just once and all subsequent responses are received in the same connection)
Docs: https://socket.io/docs/v4/index.html
[Complex] Use HTTP stream
Example: https://gist.github.com/igrigorik/5736866

Redis ETIMEDOUT error after some time of connection

Overview
On two separate Azure instances, on first one node.js servers are running and they connect to single node redis server running on second instance and node.js is trying to keep active connection with redis. Node Redis module is being used to store and retrieve data in redis and socket.io-emitter module is used to allow different servers to send messages based on collective connectivity of clients.
Problem
After the initial connection is done after sometime (sporadic)the connection freezes and finally crashes with ETIMEDOUT error being thrown from all servers.
What have I tried.
i. Added socket_keepalive first and then together with socket_initialdelay
const redis = require('redis');
let options = {socket_keepalive : true, socket_initialdelay : 200000};
global.client = redis.createClient(port, host, options);
ii. With socket.io-emitter module tried initialising it with new redis object using node redis module itself but the notifications stopped working after that so retracted back to the same thing.
This stopped the notification to devices individually
let options = {socket_keepalive : true, socket_initialdelay : 200000};
let redis_socket = require('redis');
let pub = redis_socket.createClient(port, host, options);
let ioz = require('socket.io-emitter')(pub);
*Obviously the timed out issue exists with the working method.
iii. On redis's server the timeout config is set at 0 and tcpdelay was 300 seconds but we tried changing it to 0 (tcpdelay) but still the problem persists.
It definitely breaks my head because same piece of code works in another environment but what is causing this is still a mystery, after investigating a bit more I realised that the clients connected (available with monitor command) drops after some time and hence etimedout error is thrown.
Same redis machine is also used for caching and it is working without any issue.

Looks like you might be hitting the TCP idle timeout of 4 minutes.
According the self-documented config for Redis 3.2, the value for tcp-keepalive has to be non-zero for it to work. So, you might want to set a value like 120 (240 / 2) instead and try again.

Paho Websocket Connection Closes Upon Second Connection

I have built an MQTT server to test of M2M messages and I built a small Javascript application using Paho and I am able to connect, subscribe, and publish messages on a single connection. However, once I start up a new tab or browser, the first connection closes. I am not sure why and I happen when I have a new connection, even from another computer.

You can not use hard coded client ids, the best option is to use a random number or millisecond timestamp based id.
e.g.
var clientID = "web" + new Date().getTime();
var client = new Paho.MQTT.Client('localhost',1884,clientID);

In NodeJS, how do I re-establish a socket connection with another server that may have gone down?

So, I have a Express NodeJS server that is making a connection with another app via an upagraded WebSocket uri for a data feed. If this app goes down, then obviously the WebSocket connection gets closed. I need to reconnect with this uri once the app comes back online.
My first approach was to use a while loop in the socket.onclose function to keep attempting to make the re-connection once the app comes back online, but this didn't seem to work as planned. My code looks like this:
socket.onclose = function(){
while(socket.readyState != 1){
try{
socket = new WebSocket("URI");
console.log("connection status: " + socket.readyState);
}
catch(err) {
//send message to console
}
}
};
This approach keeps giving me a socket.readyState of 0, even after the app the URI is accessing is back online.
Another approach I took was to use the JavaScript setTimout function to attempt to make the connection by using an exponential backoff algorithm. Using this approach, my code in the socket.onclose function looks like this:
socket.onclose = function(){
var time = generateInterval(reconnAttempts); //generateInterval generates the random time based on the exponential backoff algorithm
setTimeout(function(){
reconnAttempts++; //another attempt so increment reconnAttempts
socket = new WebSocket("URI");
}, time);
};
The problem with this attempt is that if the app is still offline when the socket connection is attempted, I get the following error, for obvious reasons, and the node script terminates:
events.js:85
throw er; // Unhandled 'error' event
Error: connect ECONNREFUSED
at exports._errnoException (util.js:746:11)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1010:19)
I also began using the forever node module to ensure that my node script is always running and to make sure it gets restarted after an unexpected exit. Even though I'm using forever, after a few restarts, forever just stops the script anyway.
I am basically just looking for a way to make my NodeJS server more robust and automatically re-connect with another server that may have gone down for some reason, instead of having to manually restart the node script.
Am I completely off base with my attempts? I am a noob when it comes to NodeJS so it may even be something stupid that I'm overlooking, but I have been researching this for a day or so now and all of my attempts don't seem to work as planned.
Any suggestions would be greatly appreciated! Thanks!

Few suggestions
1) Start using domain which prevents your app from an unexpected termination. Ie your app will run under the domain(run method of domain). You can implement some alert mechanism such as email or sms to which will notify when any error occurs.
2) Start using socket.io for websocket communication, it automatically handles the reconnection. Socket.io uses keep-alive heartbeat and continuously polls from the server.
3) Start using pm2 instead of forever. Pm2 allows clustering for your app which improves the performance.
I think this may improve your app's performance, stability and robustness.

Spring Framework to Optimize SQL connection?

So I am on a webproject that succesfully connects to and reads from an SQL database. The code that connects to it looks like this.
//From Here
var connection = new ActiveXObject("ADODB.Connection") ;
var connectionstring="Data Source=<server>;Initial Catalog=<catalog>;User ID=<user>;
Password=<password>;Provider=SQLOLEDB";
connection.Open(connectionstring);
var rs = new ActiveXObject("ADODB.Recordset");
//To Here
rs.Open("SELECT * FROM table", connection);
rs.MoveFirst
while(!rs.eof)
{
document.write(rs.fields(1));
rs.movenext;
}
rs.close;
connection.close;
Simple and effective and I have it working fine. But those first 4 (marked from here to here) run horribly slow and I have to reconnect every time I need to read from or write to the sql database... which is a lot for my project. So every time I run this code (which is on every other webpage I am creating in this project) I have to sit and wait for this code to run.
I have been told/ required for the project, to configure the design using javascript and spring framework. Apparently there is either a) a way to hold the connection so I don't have to run this code a every time I hit a go to new page or b) a different method of connection to the SQL database (something to replace those 4 lines of code. Both of which have to do with my using the Spring Framework.
I have never used Spring Framework before and need to learn fast. Been watching their website tutorials but still have no idea what to do or how to do it. Let me know if have any ideas.
Please and Thanks.

I think you got something wrong here: You aren't supposed to access your database from JavaScript because any user can access any data that way, or delete everything.
You should access the database server-side only.

You need connection pooling, which means, instead of opening and closing a new connection every time, keep the one that you were about to close, and see if you can reuse it later.
Unfortunately, I've never seen anything in JavaScript that allows you to reuse objects across pages, so there's no way afaik to do this in JavaScript. It's considered a very bad practice to connect from a browser directly to a database, anyway.
Usually, what you do is put a java or C# application server in between, and let these to the database access, using a connection pool. From your javascript, you then do an AJAX call to the application server, which will use the connection pool, and return e.g. JSON objects.
See for example http://tomcat.apache.org/tomcat-6.0-doc/jndi-datasource-examples-howto.html#MySQL_DBCP_Example