Better alternative to pinging the database over and over? - javascript

I want to create a dashboard that automatically updates when new data is posted.
My first thought was to just make a javascript function and put a fetch statement in it and then loop the function every second or every couple of seconds...
Obviously, this is not a great solution. But I don't know what the better way is...
Some notes:
-PHP Server-Side Language
-Ran on Localhost so traffic is not going over the internet
Can anyone advise what I should be doing or if this is an acceptable approach?
Thanks in advance!

Server Side:
You can look for any onUpdate events if your database supports any such events
Or else just run a query in a timed interval to fetch new updates form the database (Connection to database is made just once and all subsequent requests go through the same connection. Hence this isn't a bad approach)
But when it comes to client side and receiving those updates, you can make it efficient in either of the two ways:
[Simple] Use Socket IO - Push an event with your new data and listen to them on the client side. (This way socket connection is made just once and all subsequent responses are received in the same connection)
Docs: https://socket.io/docs/v4/index.html
[Complex] Use HTTP stream
Example: https://gist.github.com/igrigorik/5736866

Related

Get an http call from express to angular when event happens

i'm building an angular app that will make about a thousand people to connect simultaneously to book a ticket. I want only "XYZ" of them to access simultaneously at the registration Angular component. The other ones will see a "waiting room" component until it's their turn.
I set up the whole thing like this:
User enters the page.
I make an http call to expressjs server
The server checks if the "connections" collection constains less than XYZ docs
If true, it unlocks the user registation component and with an http post req, it creates a new doc in the db. if false it leaves it hidden and shows up the waitingroom component
When user leaves the page, his doc in "connections" collection gets destroyed with an http delete call.
Fully working.
The problem is now that i want to create a kind of "priority" system, because, going like that, if you just refresh you may be lucky and get access, even if you are soon arrived and there is who is waiting since 1990's. So i introduced a "priority" system. When the user makes the first http call, if user is not allowed, the server creates a timestamp and pushes it into an array.
const timestamps = []
.
.
.
// this below is in http get req
Connessione.countDocuments({},(err,count)=>{
if(count<=nmax){
console.log("Ok")
res.status(200).json({allowed: true})
}
else{
const timestamp = req.params.timestamp;
timestamps.push(timestamp);
console.log("Semo troppi")
res.status(401).json({allowed: false})
}
});
The idea is to listen to db changes, and when there is just XYZ-1 in the db. Make a call to the first timestamp's angular frontend to say him: "Hey there, if you want we're done. You can go" and unlock him the access to registration component.
The problem is that i can't make continuous http requests every second from angular until there's a free place...
Is there any method to send a request at the server, and when server says OK, calls angular and says "Hey dude. You can go!"?
Hope you understood my question. If not ask me in the comments.
Thanks in advance
Even i had trouble with sockets in the beginning so i'll try to explain the concept in a simple way, Whenever you write an API or Endpoint you have a one way connection i.e. you send request to server and it return back some response as shown below.
Event 1:
(Client) -> Request -> (Server)
Event 2:
(Client) <- Response <- (Server)
For API's, without request you cannot get response.
To overcome this issue as of now i can think of two possible ways.
Using Sockets, With sockets you can create a two way connection. Something like this
(Server) <-> data <-> (Client)
It means you can pass data both ways, Client to server and Server to client. So whenever an event occurs(some data is added or updated in database) one can emit or broadcast it to the client and the client can listen to the socket and receive it.
In your case as it's a two connection you can emit the data from angular and
I've attached few links at the bottom. please have a look.
Using XML/AJAX Request, This is not a preferable method, using setInterval you can call the server in every 5 seconds or so and do the operation needed.
setInterval(ajaxCall, 5000); //5000 MS == 5 seconds
function ajaxCall() {
//do your AJAX stuff here
}
Links:
https://socket.io/docs/
https://alligator.io/angular/socket-io/

Node JS live text update with CloudMQTT

I have a node server which is connecting to CloudMQTT and receiving messages in app.js. I have my client web app running on the same node server and want to display my messages received in app.js elsewhere in a .ejs file, I'm struggling as to how best to do this.
app.js
// Create a MQTT Client
var mqtt = require('mqtt');
// Create a client connection to CloudMQTT for live data
var client = mqtt.connect('xxxxxxxxxxx', {
username: 'xxxxx',
password: 'xxxxxxx'
});
client.on('connect', function() { // When connected
console.log("Connected to CloudMQTT");
// Subscribe to the temperature
client.subscribe('Motion', function() {
// When a message arrives, do something with it
client.on('message', function(topic, message, packet) {
// ** Need to pass message out **
});
});
});
Basically you need a way for the client (browser code with EJS - HTML, CSS and JS) to receive live updates. There are basically two ways to do this from the client to the node service:
A websocket session instantiated by the client.
A polling approach.
What's the difference?
Under the hood, a websocket is full-duplex communication mechanism. That means that you can open a socket from the client (browser) to the node server and they can talk to each other both ways over a long-lived session. The pro is that updates are often times instantaneous without having to incur the cost of making another HTTP request as in the polling case. The con is that it uses a socket connection that may be long-lived, and there is typically a socket pool on any server that has limited ability to deal with many sockets. There are ways to scale around this issue, but if it's a big concern for you, you may want to go with polling.
Polling is where you set up an endpoint on your server that the client JS code hits every now and then. That endpoint will return you the updated information. The con is that you are now making a new request in order to get updates, which may not be desirable if a lot of updates are expected to come through and the app is expected to be updated in the timeliest manner possible (most of the time polling is sufficient though). The pro is that you do not have a live connection open on the server indefinitely.
Again, there are many more pros and cons, these are just the obvious ones. You decide how to implement it. When the client receives the data from either of these mechanisms, you may update the UI in any suitable manner.
From the server end, you will need a way to persist the information coming from CloudMQTT. There are multiple ways to do this. If you do not care about memory consumption and are ok with potentially throwing away old data if a client does not ask for it for a while, then it may be ok to just store this in memory in a regular javascript object {}. If you do care about persisting the data between server restarts/crashes (probably best), then you can persist to something like Redis, Mongo, any of the SQL stores if your data is relational in nature, or even a regular JSON file on disk (see fs.writeFile).
Hope this helped give you a step in the right direction!

Return JS function as string from server and execute?

I'm trying to time limit a socket.io connection time on a node.js server. I asked a previous question as to whether this was possible without causing a huge overhead on the server and or blocking the main thread if we had say 1000 concurrent socket connections in various rooms, through something like:
socket.on('connection', function(params){
var maxTime = params.maxTime;
socket.join(params.roomId);
setTimeout(function{
socket.leave(params.roomId);
}, 180000)
});
The best case scenario would be handle this on the client side from a resources perspective but it isn't exactly secure to send the timeout/disconnection value as any lines of client side code that dealt with it could be easily manipulated and a knowing user could in effect prevent the disconnect event/functionality from being called.
Could I execute a function client-side sent as a string? Say:
setTimeout(function(){//disconnect},18000);
socket.emit('timeout_set', function(params){foo:bar});
Then handle appropriately on the server with a response knowing that the timeout has indeed been set:
socket.on('timeout_set', function(params){
socket.emit('proceed_with_stuff', {foo:bar});//includes critical info for proceeding
});
I'm thinking this depends on a few things:
Can you take a string from a server response and execute said string as JS?
Can a client still disrupt the setTimeout function without also triggering the socket.disconnect event?
Is this logic or anything similar possible?
Would the first scenario work on a node.js server given a number of concurrent connections?
Use Function constructor, see https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Function

Poco C++ websockets - how to use in non-blocking manner?

I am using the Poco C++ libraries to setup a websocket server, which clients can connect to and stream some data to their webinterface. So I have a loop which continuously sends data and I also want to listen if the clients closes the connection by using the receiveFrame() function, for the rest, the client is totally passive and doesn't send any data or whatsoever. The problem is that receiveFrame() blocks the connection, which is not what I want. I basically want to check if the client has not yet called the close() javascript function and stop streaming data if it has. I tried using
ws.setBlocking(false);
But now receiveFrame throws an exception every time it is called. I also tried removing receiveFrame entirely, which works if the connection is terminated by closing the browser but if the client calls the function close(), the server still tries to send data to the client. So how can I pull this off? Is there somehow a way to check if there are client frames to be received and if not to just continue?
You can repeatedly call Socket::select() (with timeout) in a separate thread; when you detect a readable socket, call receiveFrame(). In spite of the misleading name, Socket::select() call wraps epoll() or poll() on platforms where those are available.
You can also implement this in somewhat more complicated but perhaps a more elegant fashion with Poco::NotificationQueues, posting a notification every time when a socket is readable and reading data in the handler.
setBlocking() does not do what you would expect it to. Here's a little info on it:
http://www.scottklement.com/rpg/socktut/nonblocking.html
What you probably want to do is use setReceiveTimeout() on your socket to control how long it will wait for before giving you back control. Then test your response and loop everything if needed. The Poco docs have more info on how to use that part of the API. Just look up WebSockets.

how to update chat window with new messages

setInterval(function{
//send ajax request and update chat window
}, 1000)
is there any better way to update the chat with new messages? is this the right way to update the chat using setInterval?
There are two major options (or more said popular ways)
Pulling
First is pulling, this is what you are doing. Every x (milli)seconds you check if the server config has changed.
This is the html4 way (excluding flash etc, so html/js only). For php not the best way because you make for a sinle user a lot of connections per minute (in your example code at least 60 connections per second).
It is also recommended to wait before the response and then wait. If for example you request every 1 second for an update, but your response takes 2 seconds, you are hammering your server. See tymeJV answer for more info
Pushing
Next is pushing. This is more the HTML5 way. This is implemented by websockets. What is happining is the client is "listing" to a connection and waiting to be updated. When it is updated it will triger an event.
This is not great to implement in PHP because well you need a constanct connection, and your server will be overrun in no time because PHP can't push connections to the background (like Java can, if I am correct).
I made personally a small chat app and used pusher. It works perfectly. I only used the free version so don't know how expensive it is.
Pretty much yes, one minor tweak, rather than encapsulate an AJAX call inside an interval (this could result in pooling of unreturned requests if something goes bad on the server), you should throw a setTimeout into the AJAX callback to create a recursive call. Consider:
function callAjax() {
$.ajax(options).done(function() {
//do your response
setTimeout(callAjax, 2000);
});
}
callAjax();

Categories