We Keep Coding

MariaDB NodeJS backend : too many connections

I have a NodeJS express backend which uses a MariaDB database.
My file dbconnect.js creates a mariadb pool and has a function to make queries.
const mariadb = require('mariadb');
const pool = mariadb.createPool({
host: process.env.DBHost,
user: process.env.DBUser,
database: process.env.DB,
password: process.env.DBSecret
});
const dbQuery = async(query) => {
let conn;
let res = '';
try {
conn = await pool.getConnection();
res = await conn.query(query);
} catch (err) {
console.log("Error sending Query: ", query, err.text);
} finally {
if (conn) {
conn.end();
}
return res;
}
}
Everything seems to work perfectly, but after a few months with the server running these messages begin to appear on the console:
These messages keep appearing every 10-14 seconds, but no queries are being performed.
Thanks for any help

I am not sure but there is one way,
configure your connection pool to ping at particular time. so it will close any inactive connections before the server closes them. mariadb has pingInterval for this
Replace this code with your code
const pool = mariadb.createPool({
host: process.env.DBHost,
user: process.env.DBUser,
database: process.env.DB,
password: process.env.DBSecret,
pingInterval: 60000
});
This will send a ping to the server every 60 seconds, which will prevent the server from closing inactive connections.

Node.Js MSSQL Query Timeout Expired

I am using Node Express API to run SQL queries to populate a dashboard of data. I am using the mssql-node package to do so. Sometimes it runs flawlessly, other times I get the following error:
[Error: [Microsoft][SQL Server Native Client 11.0]Query timeout expired]
I am creating a poolPromise with a connectionPool to the db, then I pass that object to my other controllers which run the specific queries to populate data. I run the server which initiates the db.js script and connects to MSSQL with a pool connection.
db.js:
// for connecting to sql server
const sql = require('mssql/msnodesqlv8');
// db config to connect via windows auth
const dbConfig = {
driver: 'msnodesqlv8',
connectionString: 'Driver={SQL Server Native Client 11.0};Server={my_server};Database={my_db};Trusted_Connection={yes};',
pool: {
idleTimeoutMillis: 60000
}
};
// create a connectionpool object to pass to controllers
// this should keep a sql connection open indefinitely that we can query when the server is running
const poolPromise = new sql.ConnectionPool(dbConfig)
.connect()
.then(pool => {
console.log('Connected to MSSQL');
return pool;
})
.catch(err => console.log('Database Connection Failed! Bad Config: ', err))
module.exports = { sql, poolPromise };
An example of one of my controllers and how I use the poolPromise object is below. I currently have about 7 of these controllers that run their own specific query to populate a specific element on the dashboard. The performance of the queries each run within 1-10 seconds (depending on current server load, as I am querying an enterprise production server/db, this can vary). As I mentioned earlier, the queries run flawlessly sometimes and I have no issues, but at other times I do have issues. Is this a symptom of me querying from a shared production server? Is it preferred to query from a server that has less load? Or am I doing something in my code that could be improved?
const { sql, poolPromise } = require('../db');
// function to get data
const getData = async (req, res) => {
try {
// create query parameters from user request
let id= req.query.id;
// create query from connectionPool
let pool = await poolPromise;
let qry = `
select * from tbl where id = #Id
`
let data = await pool.request()
.input('Id', sql.VarChar(sql.MAX), id)
.query(qry);
// send 200 status and return records
res.status(200);
res.send(data.recordset);
} catch(err) {
console.log('Error:');
console.log(err);
res.sendStatus(500);
}
};
module.exports = { getData };

Trouble opening localhost with Node and mssql

I'm using Node to connect to a Microsoft SQL Developer database. I've finally gotten my code to run without errors:
var sql = require('mssql/msnodesqlv8');
const express = require('express');
const app = express();
// Get request
app.get('/', function (req, res) {
// Config your database credential
const config = {
server: "xxxx",
driver:"xxxx",
database: "xxxx",
user: "xxxx",
password: "xxxx",
options:{
trustServerCertificate: true,
}
};
// Connect to your database
new sql.ConnectionError(config,function(err){
// Create Request object to perform
// query operation
var request = new sql.Request();
// Query to the database and get the records
request.query('select * from mydb',
function (err, records) {
if (err) console.log(err)
// Send records as a response
// to browser
res.send(records);
});
});
});
var server = app.listen(5000, function () {
console.log('Server is listening at port 5000...');
});
But, when I go to :
http://localhost:5000/
It doesn't load, it says the page cannot be reached. What can I try to resolve this?

You're using the wrong thing to try and connect to SQL Server. You don't use new sql.ConnectionError(), you use sql.connect(). This error is causing your app to crash so nothing is listening on port 5000.
var sql = require('mssql/msnodesqlv8');
const express = require('express');
const app = express();
// Get request
app.get('/', function (req, res) {
// Config your database credential
const config = {
server: "xxxx",
driver:"xxxx",
database: "xxxx",
user: "xxxx",
password: "xxxx",
options:{
trustServerCertificate: true,
}
};
// Connect to your database
sql.connect(config,function(err){
// Create Request object to perform
// query operation
var request = new sql.Request();
// Query to the database and get the records
request.query('select * from mydb',
function (err, records) {
if (err) console.log(err)
// Send records as a response
// to browser
res.send(records);
});
});
});
var server = app.listen(5000, function () {
console.log('Server is listening at port 5000...');
});
Run that (after having applied proper database connection configuration values) and then you should be able to open your browser and connect to http://localhost:5000

NextJS, Express, Error during WebSocket handshake: Unexpected response code: 200

The basic problem can be summarized as follows: When creating a Websocket server in Node using ws with the server option populated by an express server(as in this example), while using that same express server to handle the routing for NextJS (as in this example), the upgrade header seems to not be properly parsed.
Instead of the request being routed to the Websocket server, express sends back an HTTP 200 OK response.
I've searched high and low for an answer to this, it may be that I simply do not understand the problem. A possibly related question was brought up in an issue on NextJS's github. They recommend setting WebsocketPort and WebsocketProxyPort options in the local next.config.js, however I have tried this to no avail.
A minimal example of the relevant server code can be found below. You may find the full example here.
const express = require('express')
const next = require('next')
const SocketServer = require('ws').Server;
const port = parseInt(process.env.PORT, 10) || 3000
const dev = process.env.NODE_ENV !== 'production'
const app = next({ dev })
const handle = app.getRequestHandler()
app.prepare().then(() => {
const server = express()
server.all('*', (req, res) => {
return handle(req, res)
})
server.listen(port, err => {
if (err) throw err
console.log(`> Ready on http://localhost:${port}`)
})
const wss = new SocketServer({ server });
wss.on('connection', function connection(ws, request) {
console.log('Client connected');
ws.on('close', () => console.log('Client disconnected'));
});
wss.on('error', function (error) {
console.log(error);
});
setInterval(() => {
wss.clients.forEach((client) => {
client.send(new Date().toTimeString());
});
}, 1000);
}).catch(ex => {
console.error(ex.stack);
process.exit(1);
});
The expected result, of course, is a connection to the websocket server. Instead I receive the following error:
WebSocket connection to 'ws://localhost:3000/' failed: Error during WebSocket handshake: Unexpected response code: 200
Can anyone elucidate anything for me here?

Ok, after more digging I have solved the problem. Quite simply, the ws.Server object to which I was trying to feed the server = express() object is not strictly speaking an http server object. However, server.listen() returns such an http server object. On such an object we can listen for an 'upgrade' call, which we can pass to our ws.Server object's handleUpgrade() event listener, through which we can connect. I will be updating the examples that I linked in my question, but the relevant code is below:
app.prepare().then(() => {
const server = express()
server.all('*', (req, res) => {
return handle(req, res)
})
const wss = new SocketServer({ server });
wss.on('connection', function connection(ws, request) {
console.log('Client connected');
ws.on('close', () => console.log('Client disconnected'));
});
wss.on('error', function (error) {
console.log(error);
});
let srv = server.listen(port, err => {
if (err) throw err
console.log(`> Ready on http://localhost:${port}`)
})
srv.on('upgrade', function(req, socket, head) {
wss.handleUpgrade(req, socket, head, function connected(ws) {
wss.emit('connection', ws, req);
})
});

Failing to connect to MongoDB hosted on mlab

Background
Making a small web app that connects to a Mongo DB hosted with Mlab. I've created the DB on mlab, and created users with read/write permission. I've also created a users collection with several records.
The Problem
When I try and connect to the database using the code on mongo.github.io, I hit the error:
/home/ed/dev/mongo-demo/node_modules/mongodb/lib/operations/mongo_client_ops.js:466
throw err;
^
TypeError: Cannot read property 'db' of null
The Code
var MongoClient = require('mongodb').MongoClient;
const url = 'mongodb://<user>:<pass>#ds115434.mlab.com:15434';
// Database Name
const dbName = 'princee3-music';
// Use connect method to connect to the server
MongoClient.connect(url, function(err, client) {
console.log("Connected successfully to server");
const db = client.db(dbName);
client.close();
});
What I Have Tried
Oddly, if I connect through the shell using:
mongo ds115434.mlab.com:15434/princee3-music -u <dbuser> -p <dbpassword>
That works fine, or if I wrap the connection in an anonymous self-calling async function, it also connects.
Async Wrapper
const MongoClient = require('mongodb').MongoClient;
const mongoUrl = 'mongodb://<user>:<pass>#ds115434.mlab.com:15434/';
const dbName = 'princee3-music';
(async() => {
const client = await MongoClient.connect(mongoUrl, { useNewUrlParser: true});
const db = client.db(dbName);
db.collection('users').insertOne({
email: user.email,
pass: hashedPassword,
admin: true
}, (err, result) => {
if (err) {
reject({error: err});
} else {
resolve({message: 'okay'});
}
});
client.close();
})();
Any pointers on where I may be going wrong would be great.

The official mLab docs advise to connect like below. It has to be asynchronous , in order to wait for the connection to occur, or the client will be null, thus throwing an error saying that it can’t read property db of null.
On the other hand, you async has useNewUrlParser which might be the key to have a successful connection, see this issue
MongoClient.connect(url, { useNewUrlParser: true }).then(client => client.db())