We Keep Coding

Webscokets vs Server Side Events - NodeJS backend and VueJs client

I have a front end client, which is written in VueJs and a Backend API which is written in Node Js. The Node API communicates with other third party APIs and in turn sent responses back to the client. Now for some of the APIs, it is taking a long time, more than a minute to complete the request and send the response back to the client. As the Node App is proxied over Akamai, it sends a 503 error after a certain time and thus and error will be thrown to the enduser. But the actual process that the third party API do is still in progress and it will send a success response back to the Node App once it is completed. As the client already received the error, it will not receive the success message.
I have this issue with the account creation flow. The client form data is posted to NodeJS backend, which eventually post to another third party API. While waiting for the call to finish, the Akamai proxy will send 503 HTTPS status with Zero Size object response. Client receives this error message and a custom error will be shown. But the account is being created in the backend and eventually it will send success response to the node app, but this never reaches the client and so the user. There is a chance that user will create another account.
The front end call is as follows:
return new Promise((resolve, reject) => {
const config = {}
config.method = 'POST'
config.url = APIaddress
config.data = data
config.params = params
config.withCredentials = true
config.httpsAgent = new https.Agent({ keepAlive: true })
console.log('Config: ', config)
axios(config).then(response => {
console.log('RESPONSE: ', response)
resolve(response)
}).catch(error => {
console.log('ERROR: ', error.response)
reject(error.response.data)
})
})
Here I added the KeepAlive option, but it has no effect and I still get the error.
Now, in the backend also, I use agentkeepalive, and the call is as follows:
const HttpsAgent = agentkeepalive.HttpsAgent
const keepaliveAgent = new HttpsAgent({
timeout:120000,
freeSocketTimeout:60000
});
const options = {
method: 'POST',
url: config.endpoint.url,
headers:
{
'Content-Type': 'application/json;charset=utf-8',
'Accept': 'application/json',
authorization: 'Bearer ' + token
},
data: data,
json: true,
httpsAgent:keepaliveAgent
};
axios(options)
.then(response => response.data)
.then(response => {
resolve(response)
})
.catch(function (error) {
logger.error({
message: `Error while creating account: ${error}`
});
reject(error);
});
Now in order to account for the delays, I am planning to use Server Side Events or WebSockets. I am very new to this and not sure which one to use. I think by using one of these, I can send response back to the client once the account is created. Currently the client is waiting for the account to be created, but I want to make it in such a way that client will send the initial requests and then the server will send notification to the client, once the account is created. This will avoid the unnecessary timeouts and other related issues.
I not sure which solution has to be used here. It will be helpful if someone can shed some light. Thanks for reading.

I switched from SSE and RestAPI to WebSocket on my Node and React app. My setup is as follows:
Create WebSocket server in Node
Create connection from client to server
Then I use "publish-subscribe" pattern.
When client needs something from server, it sends WebSocket message to server with specific sign (In my case it is called "route".) Server filters the message and sends it to proper router (not the Express one, these are routes in my server handling the WebSocket requests.)
As it is processed, server sends WebSocket message back to client, which filters it and processes.
This allows me to have always opened connection to server, what is very swift, and - that's what you are looking for - wait for some message from server without blocking the connection or risking timeout.
Very simple code example:
server:
ws.on('message', m => {
if (m.route === DO_SOMETHING) {
...do something...
ws.send(JSON.stringify({route: DO_SOMETHING_RESPONSE}, message: 'Something was
done'})
}
)
client:
// I want something to be done from server:
ws.send(JSON.stringify({route: DO_SOMETHING, message: 'something should be done'}))
// this is send and you can wait a year for a response, which is catched with:
ws.on('message', m => {
if (m.route === DO_SOMETHING_RESPONSE) {
console.log('Yupeee, something was done!')
}
)
This way you can handle unlimited number of requests. You can make them independent as in this example. Or you can force client to wait for the answger from server.

Nodejs server with Express not sending responses to HTTP requests from javascript file, but is receiving the requests and processing them

So I have a node-js server and an apache server on the same machine, and one of the javascript files is sending an HTTP request to the node-js server. The node-js server receives the file, reads the data, puts it in the database, as it should, but it isn't sending back any status codes or data.
Here is the XHTMLRequest send code snippet,
// creates a new http request to be sent to the nodejs server
function createNewUser(username, password, email) {
// The url is the URL of our local nodejs server
var userCreateRequest = new XMLHttpRequest();
userCreateRequest.open( "POST", "http://<machine's IP>:8080/api/users" );
// Create json object for user data
var user = "name="+username+"&password="+password+"&email="+email;
alert(user);
// set content type for http request
userCreateRequest.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
// Event listern for server response
// userCreateRequest.addEventListener("readystatechange", processRequest, false);
// Call process request whenever state changes
userCreateRequest.onreadystatechange = function() {
alert(this.readyState + ", " + this.status);
if (this.readyState == 4 && this.status == 200) {
var response = this.response;
alert(response.name);
}
}
// Send user data to server
userCreateRequest.send(user);
}
And here is the code for the node-js server (with express)
router.route('/users')
.post(function(req, res) { //create a new user
var user = new User();
user.name = req.body.name;
user.password = req.body.password;
user.email = req.body.email;
user.save(function(err) { //add user object to database
if(err)
res.send(err);
res.status(200).json(user);
});
});
As I said above, the code works fine in terms of putting the body of the request in the database and what-not, but the server is not sending back the 200 OK response (or I'm failing to receive it for some reason). The only times I get an alert from onreadystatechange is when it's state 2, status 0, and state 4, status 0.

Try below code snippet.
user.save(function(err, user) {
if(err)
res.send(err);
res.status(200).json(user);
});

It did end up being a CORS issue. I'm still a little iffy on exactly why, but after configuring the express/CORS package to allow requests from the IP and port of my apache server, it started working.
My understanding is that cross origin implies a different domain, where-as both of my servers are (as I understand it) on different ports on the same domain.
Either way, enabling CORS fixed the issue. Thank you to Jaromanda X for pointing it out and getting me on the right track.

How to build local node.js mail server

I have spent couple of days implementing my own mail server using node.js. I used modules like "smtp-server" for creating smtp server and also "smtp-connection" to connect and send mail to it. But I'm getting confused because I don't know how to send mails from my smtp server to providers smtp servers like google or yahoo.
Can anyone help me?
Here is my code for more information:
My index.js file:
var SMTPServer = require('smtp-server').SMTPServer;
var port = 9025;
var serverOptions = {
name: "smtp-interceptor",
onConnect: onConnect,
onAuth: onAuth,
onData: onData
};
var server = new SMTPServer(serverOptions);
server.listen(port, 'localhost', function () {
console.log('SMTP server is listening on port ' + port);
});
function onConnect(session, callback) {
console.log('Connected');
return callback(); // Accept the connection
}
function onData(stream, session, callback) {
stream.pipe(process.stdout); // print message to console
console.log('Session \n', session.envelope);
stream.on('end', callback);
}
function onAuth(auth, session, callback){
if(auth.username !== 'Mahan' || auth.password !== 'Tafreshi') {
return callback(new Error('Invalid username or password'));
}
callback(null, {user: 123}); // where 123 is the user id or similar property
}
And my connection.js file:
var SMTPConnection = require('smtp-connection');
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
var connection = new SMTPConnection({
host: 'localhost',
port: 9025,
secure: false
});
connection.connect(function (){
console.log('Connceted to SMTP server');
var auth = {
user: 'Mahan',
pass: 'Tafreshi'
};
connection.login(auth, function (err) {
if(err)
return console.log('Login Failed \n', err);
console.log('Login Successful');
var envelope = {
from: "testapp#testapplocal.com",
to: "mahantp19#gmail.com"
};
var message = 'Test message1';
connection.send(envelope, message, function (err, info) {
if(err)
return console.log('Error : ' + err);
console.log('Message sent');
console.log('Accepted : ' + info.accepted);
console.log('Rejected : ' + info.rejected);
console.log(info.response);
connection.quit();
console.log('Quit connection');
connection.close();
});
});
});

There are many checks an email must pass before it's accepted by most mail providers. These checks attempt to validate the server sending the message is authorized to send on behalf of the sender.
IE: My server can send an email saying it's from "someone-special#somewhere-important.com"... That doesn't mean I'm "anywhere important" by any means.
While you may have had success sending mail from an SMTP server in the past using another technology such as PHP or an Exchange Server, the rules have changed significantly. Gmail has just began full enforcement this year.
I would assume your current issue has nothing to do with node as much as recent changes by the big providers.
Some of the checks that are needed include:
DKIM Keys (DNS Record)
SPF Record (DNS Record)
DMARK has been setup.
Dedicated IP Address for the server is required.
Your servers IP not being blacklisted.
The content of your email passes their filters.
Attempt to have an email sent from your server appear to be from a visitor or customer.
Among many others.
Any domain you want to "Act as Sender" must have these in place for most of the major providers to accept you message.
Google has a great set of tools and walkthroughs on getting an IP/Domain setup.
Visit the Google MX Record Checker and enter in the domain/subdomain you want to use as sender and it will tell you everything that is failing as well as passing.
Alternative Solutions
I use sendgrid.com. They have A node library that makes sending mail very easy. They also provide me the ability to proxy messages via SMTP. This means you can utilize the standard methods to deliver messages. You will just change out "localhost" with an hostname they provide. However, if this is for a new setup, go for the API.
Whomever is hosting your email should offer the ability for you send messages via SMTP or an API
An endless supply of other providers are out their, most of which allow low volume senders to send for FREE.
Word of warning
I tried for a few years keeping up with all the changes and inevitably, I continued to hit barriers of blocked messages with no ability to know until someone did not get an email. If your sending low volume, you should be able to use third parties without paying paying for it. If you are sending high volume, the cost of their service is cheap compared to the endless issues you will encounter even once you get it initially rolling.
PS. I have no affiliation with any email provider or sender. I pay them too.

How can I authorize and identify users with a websocket connection request?

When I set up a websocket connection from JavaScript, how can I authorize that it is a legit user on the serverside? I am using JSON Web Tokens and when doing regular calls to REST backend I automatically add an Authorization: Bearer (JWT..) header on AngularJS and then check that on the server side to see if a user is logged in. How can I do that when upgrading the connection to a websocket connection? I am afraid that some people with connect to the server requesting a websocket connection and spoof some of the users id's and receive their messages without being logged in to the service.
I request a websocket connection like this:
var conn = new WebSocket("ws://localhost:8080/api/ws");
conn.onclose = function (e) {
console.log("disconnected");
};
conn.onopen = function (e) {
console.log("connected");
};
conn.onmessage = function (e) {
console.log(e.data);
};
On the first part, is that a GET request or a POST request? Can I add parameters to the url and check them on the serverside? For example:
var conn = new WebSocket("ws://localhost:8080/api/ws/token/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ");
Or is this not a good idea? I also thought sending a JWT would be a good idea because I would be able to extract the user_id from the JWT and associate a websocket connection to a specific user.
How can I solve this problem?

nodejs + WebSockets - reject the connection with a message

I want to provide a meaningful error to the client when too many users are connected or when they're connecting from an unsupported domain, so...
I wrote some WebSocket server code:
var http = require('http');
var httpServer = http.createServer(function (request, response)
{
// i see this if i hit http://localhost:8001/
response.end('go away');
});
httpServer.listen(8001);
// https://github.com/Worlize/WebSocket-Node/wiki/Documentation
var webSocket = require('websocket');
var webSocketServer = new webSocket.server({ 'httpServer': httpServer });
webSocketServer.on('request', function (request)
{
var connection = request.reject(102, 'gtfo');
});
And some WebSocket client code:
var connection = new WebSocket('ws://127.0.0.1:8001');
connection.onopen = function (openEvent)
{
alert('onopen');
console.log(openEvent);
};
connection.onclose = function (closeEvent)
{
alert('onclose');
console.log(closeEvent);
}
connection.onerror = function (errorEvent)
{
alert('onerror');
console.log(errorEvent);
};
connection.onmessage = function (messageEvent)
{
alert('onmessage');
console.log(messageEvent);
};
All I get is alert('onclose'); with a CloseEvent object logged to the console without any status code or message that I can find. When I connect via ws://localhost:8001 the httpServer callback doesn't come into play, so I can't catch it there. The RFC suggests I should be able to send any status code other than 101 when there's a problem, but Chrome throws an error in the console Unexpected response code: 102. If I call request.reject(101, 'gtfo'), implying it was successful I get a handshake error, as I'd expect.
Not really sure what else I can do. Is it just not possible right now to get the server response in Chrome's WebSocket implementation?
ETA: Here's a really nasty hack in the mean time, I hope that's not what I have to end up doing.
var connection = request.accept(null, request.origin);
connection.sendUTF('gtfo');
connection.close();

I'm the author of WebSocket-Node and I've also posted this response to the corresponding issue on GitHub: https://github.com/Worlize/WebSocket-Node/issues/46
Unfortunately, the WebSocket protocol does not provide any specific mechanism for providing a close code or reason at this stage when rejecting a client connection. The rejection is in the form of an HTTP response with an HTTP status of something like 40x or 50x. The spec allows for this but does not define a specific way that the client should attempt to divine any specific error messaging from such a response.
In reality, connections should be rejected at this stage only when you are rejecting a user from a disallowed origin (i.e. someone from another website is trying to connect users to your websocket server without permission) or when a user otherwise does not have permission to connect (i.e. they are not logged in). The latter case should be handled by other code on your site: a user should not be able to attempt to connect the websocket connection if they are not logged in.
The code and reason that WebSocket-Node allow you to specify here are an HTTP Status code (e.g. 404, 500, etc.) and a reason to include as a non-standard "X-WebSocket-Reject-Reason" HTTP header in the response. It is mostly useful when analyzing the connection with a packet sniffer, such as WireShark. No browser has any facility for providing rejection codes or reasons to the client-side JavaScript code when a connection is rejected in this way, because it's not provided for in the WebSocket specification.