We Keep Coding

Webscokets vs Server Side Events - NodeJS backend and VueJs client

I have a front end client, which is written in VueJs and a Backend API which is written in Node Js. The Node API communicates with other third party APIs and in turn sent responses back to the client. Now for some of the APIs, it is taking a long time, more than a minute to complete the request and send the response back to the client. As the Node App is proxied over Akamai, it sends a 503 error after a certain time and thus and error will be thrown to the enduser. But the actual process that the third party API do is still in progress and it will send a success response back to the Node App once it is completed. As the client already received the error, it will not receive the success message.
I have this issue with the account creation flow. The client form data is posted to NodeJS backend, which eventually post to another third party API. While waiting for the call to finish, the Akamai proxy will send 503 HTTPS status with Zero Size object response. Client receives this error message and a custom error will be shown. But the account is being created in the backend and eventually it will send success response to the node app, but this never reaches the client and so the user. There is a chance that user will create another account.
The front end call is as follows:
return new Promise((resolve, reject) => {
const config = {}
config.method = 'POST'
config.url = APIaddress
config.data = data
config.params = params
config.withCredentials = true
config.httpsAgent = new https.Agent({ keepAlive: true })
console.log('Config: ', config)
axios(config).then(response => {
console.log('RESPONSE: ', response)
resolve(response)
}).catch(error => {
console.log('ERROR: ', error.response)
reject(error.response.data)
})
})
Here I added the KeepAlive option, but it has no effect and I still get the error.
Now, in the backend also, I use agentkeepalive, and the call is as follows:
const HttpsAgent = agentkeepalive.HttpsAgent
const keepaliveAgent = new HttpsAgent({
timeout:120000,
freeSocketTimeout:60000
});
const options = {
method: 'POST',
url: config.endpoint.url,
headers:
{
'Content-Type': 'application/json;charset=utf-8',
'Accept': 'application/json',
authorization: 'Bearer ' + token
},
data: data,
json: true,
httpsAgent:keepaliveAgent
};
axios(options)
.then(response => response.data)
.then(response => {
resolve(response)
})
.catch(function (error) {
logger.error({
message: `Error while creating account: ${error}`
});
reject(error);
});
Now in order to account for the delays, I am planning to use Server Side Events or WebSockets. I am very new to this and not sure which one to use. I think by using one of these, I can send response back to the client once the account is created. Currently the client is waiting for the account to be created, but I want to make it in such a way that client will send the initial requests and then the server will send notification to the client, once the account is created. This will avoid the unnecessary timeouts and other related issues.
I not sure which solution has to be used here. It will be helpful if someone can shed some light. Thanks for reading.

I switched from SSE and RestAPI to WebSocket on my Node and React app. My setup is as follows:
Create WebSocket server in Node
Create connection from client to server
Then I use "publish-subscribe" pattern.
When client needs something from server, it sends WebSocket message to server with specific sign (In my case it is called "route".) Server filters the message and sends it to proper router (not the Express one, these are routes in my server handling the WebSocket requests.)
As it is processed, server sends WebSocket message back to client, which filters it and processes.
This allows me to have always opened connection to server, what is very swift, and - that's what you are looking for - wait for some message from server without blocking the connection or risking timeout.
Very simple code example:
server:
ws.on('message', m => {
if (m.route === DO_SOMETHING) {
...do something...
ws.send(JSON.stringify({route: DO_SOMETHING_RESPONSE}, message: 'Something was
done'})
}
)
client:
// I want something to be done from server:
ws.send(JSON.stringify({route: DO_SOMETHING, message: 'something should be done'}))
// this is send and you can wait a year for a response, which is catched with:
ws.on('message', m => {
if (m.route === DO_SOMETHING_RESPONSE) {
console.log('Yupeee, something was done!')
}
)
This way you can handle unlimited number of requests. You can make them independent as in this example. Or you can force client to wait for the answger from server.

Process multiple unique Express JS requests

I've got a small Express JS api that I'm building to handle and process multiple incoming requests from the browser and am having some trouble figuring out the best approach to handle them.
The use case is that there's a form, with potentially up-to 30 or so people submitting form data to the Express JS api at any given time, the API then POSTS this data of to some place using axios, and each one needs to return a response back to the browser of the person that submitted the data, my endpoint so far is:
app.post('/api/process', (req, res) => {
if (!req.body) {
res.status(400).send({ code: 400, success: false, message: "No data was submitted" })
return
}
const application = req.body.Application
axios.post('https://example.com/api/endpoint', application)
.then(response => {
res.status(200).send({ code: 200, success: true, message: response })
})
.catch(error => {
res.status(200).send({ code: 200, success: false, message: error })
});
})
If John and James submit form data from different browsers to my Express JS api, which is forwarded to another api, I need the respective responses to go back to the respective browsers...

Let's make clear for you, A response of a request will only send to the requester, But if you need to send a process request and send a response like, hey i received your request and you can use another get route to get the result sometimes later, then you need to determine which job you mean. So You can generate a UUID when server receives a process request and send it back to the sender as response, Hey i received your process request, you can check the result of process sometimes later and this UUID is your reference code. Then you need to pass the UUID code as GETparam or query param and server send you the correct result.
This is the usual way when you are usinf WebSockettoo. send a process req to server and server sends back a reference UUID code, sometime later server sends the process result to websocket of requester and says Hey this is the result of that process with that UUID reference code.
I hope i said clear enough.

XMPP chat architecture

I am currently connecting to an external XMPP server (not mine). Since I couldn't find any XMPP PHP client which suits my needs, I have developed my own simple client (XMPP PHP). The client opens a socket connection to the server and that is where the XML exchange happens, and that part works well.
The main reason for developing this package was to integrate it within a corporate app. I have done so and can successfully send messages back and forth.
Even though my end goal is to have an open websocket connection, currently the program works by polling the server in 2s interval (I am using a simple 1on1 communication, no chat rooms or similar):
receiveMessages: function () {
this.poll = setInterval(function () {
Ext.Ajax.request({
url : 'index.php',
method : 'post',
params : {
method: 'receiveMessages',
},
scope : this,
callback: function (options, success, response) {
...
}
});
}, this.pollTimer);
}
And on the PHP side:
public function receiveMessages()
{
$messages = $this->client->getMessages();
if ($messages) {
foreach ($messages as $message) {
$message = $this->xml2array($message);
$conversation = $this->conversationExists($message['#attributes']['from']);
if ($conversation == null) {
$preparedConversation = array(
...
);
$conversation = $this->_save($preparedConversation ...);
}
$message = array(
...
);
$response = $this->_save($message ...);
return array(
'success' => true,
'response' => $response,
);
}
}
}
Upon success, this method updates frontend with the received message, as well as saves the message to the DB. DB is organized in a way that one User can have many Conversations and one Conversation can have many Messages.
What I fail to understand though is how should everything be structured in order to function like some of the real chat clients (Facebook or other messengers), because this way I can't get the "concurrency" I want. If I log in with my account on 2 different places, each of them will poll the server every 2s, and it is basically a race condition, first one to get the message will display it.
If I think about it, one way to do it properly would be to implement websockets to server, and have frontend wait for DB changes, however I think this may create much read overhead on DB. Can someone give me some advice on how to do this?

Update chat with MySQL on JavaScript event

I'm trying to make a chat system where anyone can go into the chat and send a message. The messages are stored in a MySQL database and my code looks like this at the moment ...
<script>
$('input[type=text]').on('keydown', function(e) {
if(e.which == 13) {
e.preventDefault();
$.ajax({
url: "php/chat.class.php",
data: { steamid: "<?php echo $steamprofile['steamid']?>", message: document.getElementById("chatMessage").value },
type: "GET",
context: document.body
}).done(function() {
alert("Message sent");
// This is when the chat should update for everyone
}).error(function() {
document.getElementById('chat-box').innerHTML += '<span class="text-muted"><i>Could not send chat message at this time.</i></span>';
});
}
});
Basically, it inserts a new row into the MySQL table once you press enter.
As it is a chat for everyone, I need the chat to update for every user when anyone sends a message.
I know how to display all the messages and who sent them, but I need it to update whenever someone sends a message, for everyone.
I've done research but I can't find anything useful unfortunately and now I'm clueless.
I've thought of updating the chat every x seconds, but I want to make the chat as smooth and fast as possible and wondering what the best solution would be.
Have a good day

So that is a read operation, and hence what you are trying is a write operation. Either you can do a continuous lookup to the server with a interval/timeout/ or initiate a rest call when last call was finished (success/error/timeout whatever). but better approach for this will be initiating a WebSocket Client and create and set up WebSocket Server in your backend and design it properly, so if it get any message from any client it will send that to all other client or something optimizer/ or something in schedule

Consider using a realtime database like Firebase https://firebase.google.com/docs/database/web/start and on your client side you can listen to messages being added using something like:
commentsRef.on('child_added', function(data) {
addCommentElement(postElement, data.key, data.val().text, data.val().author);
});
With firebase you can create a serverless chat application.

send http request to angular js

I know how to send an http request to a server using angular js. With the promise returned, I know how to listen for a response and manipulate the ui thereafter. But this approach cannot be used for what I have in mind.
However, what I cannot figure out, is how to send a request to a website.
I have a server localhost:800/receiveData which receives a POST request and then manipulate the UI and DoM on the angularjs site
app.get('/', function(req,res){
res.sendFile(__dirname+'/index.html')
})
app.post('/receiveData', function(req,res){
var data = req.body.data
// assume data is a boolean
if(data){
//show a view in index.html using angular js or anything else
}else {
//show a different view in index.html
}
});
Any help will be greatly appreciated. I have a need for angular js. Having a SPA is imperative. I am completely open to adding additional stacks if neccessary.
EDIT:
As pointed out by MarcoS, manipulation of dom should ideally not happen from the server side. I am combining IPFS with node js and angular js to develop a single page application. The swarm of nodes set up using IPFS has an open line of communication with my server (by design). Based on packets of data sent via the comm line to my server, I need to convey messages to the user via the index.html.

I think your approach is wrong: on server-side, you should NOT manipulate the UI and DOM...
You should just do server activity (update a database, send an email, ..., return a static page).
Then you can output a result (JSON/XML/... format) for your client-side calling script to read.
Following OP edit, what I do understand is he wants server push to the client.
To get serve side pushes, you should poll on the client.
In a controller:
function getServerState(changeState) {
return $http.get("/receiveData").then(function(res) {
changeState(res.data); // notify the watcher
}).catch(function(e) {
/* handle errors here */
}).then(function() {
return getServerState(changeState); // poll again when done call
});
}
Consuming it this way:
getServerState(function(status) {
$scope.foo = status; // changes to `foo` $scope variable will reflect instantly on the client
});
And, server side:
app.post('/receiveData', function(req, res) {
var data = req.body.data; // assume data is a boolean
res.end(JSON.stringify(data);
});