I have a NodeJS express backend which uses a MariaDB database.
My file dbconnect.js creates a mariadb pool and has a function to make queries.
const mariadb = require('mariadb');
const pool = mariadb.createPool({
host: process.env.DBHost,
user: process.env.DBUser,
database: process.env.DB,
password: process.env.DBSecret
});
const dbQuery = async(query) => {
let conn;
let res = '';
try {
conn = await pool.getConnection();
res = await conn.query(query);
} catch (err) {
console.log("Error sending Query: ", query, err.text);
} finally {
if (conn) {
conn.end();
}
return res;
}
}
Everything seems to work perfectly, but after a few months with the server running these messages begin to appear on the console:
These messages keep appearing every 10-14 seconds, but no queries are being performed.
Thanks for any help
I am not sure but there is one way,
configure your connection pool to ping at particular time. so it will close any inactive connections before the server closes them. mariadb has pingInterval for this
Replace this code with your code
const pool = mariadb.createPool({
host: process.env.DBHost,
user: process.env.DBUser,
database: process.env.DB,
password: process.env.DBSecret,
pingInterval: 60000
});
This will send a ping to the server every 60 seconds, which will prevent the server from closing inactive connections.
Related
I am using Node Express API to run SQL queries to populate a dashboard of data. I am using the mssql-node package to do so. Sometimes it runs flawlessly, other times I get the following error:
[Error: [Microsoft][SQL Server Native Client 11.0]Query timeout expired]
I am creating a poolPromise with a connectionPool to the db, then I pass that object to my other controllers which run the specific queries to populate data. I run the server which initiates the db.js script and connects to MSSQL with a pool connection.
db.js:
// for connecting to sql server
const sql = require('mssql/msnodesqlv8');
// db config to connect via windows auth
const dbConfig = {
driver: 'msnodesqlv8',
connectionString: 'Driver={SQL Server Native Client 11.0};Server={my_server};Database={my_db};Trusted_Connection={yes};',
pool: {
idleTimeoutMillis: 60000
}
};
// create a connectionpool object to pass to controllers
// this should keep a sql connection open indefinitely that we can query when the server is running
const poolPromise = new sql.ConnectionPool(dbConfig)
.connect()
.then(pool => {
console.log('Connected to MSSQL');
return pool;
})
.catch(err => console.log('Database Connection Failed! Bad Config: ', err))
module.exports = { sql, poolPromise };
An example of one of my controllers and how I use the poolPromise object is below. I currently have about 7 of these controllers that run their own specific query to populate a specific element on the dashboard. The performance of the queries each run within 1-10 seconds (depending on current server load, as I am querying an enterprise production server/db, this can vary). As I mentioned earlier, the queries run flawlessly sometimes and I have no issues, but at other times I do have issues. Is this a symptom of me querying from a shared production server? Is it preferred to query from a server that has less load? Or am I doing something in my code that could be improved?
const { sql, poolPromise } = require('../db');
// function to get data
const getData = async (req, res) => {
try {
// create query parameters from user request
let id= req.query.id;
// create query from connectionPool
let pool = await poolPromise;
let qry = `
select * from tbl where id = #Id
`
let data = await pool.request()
.input('Id', sql.VarChar(sql.MAX), id)
.query(qry);
// send 200 status and return records
res.status(200);
res.send(data.recordset);
} catch(err) {
console.log('Error:');
console.log(err);
res.sendStatus(500);
}
};
module.exports = { getData };
I have 2 servers, one running the frontend code (Node.js, express) and the other running MySQL with PHPMyAdmin, both are working standalone and I can confirm the MySQL database is running on 3306 but whenever I try to connect via Node.js (code below) I get a connection refused error.
const conn = mysql.createConnection({
host: '192.168.1.250',
user: 'mcd',
password: '**********',
database: 'mcd'
})
conn.connect(function(err) {
if (err) throw err;
})
The IP address I have used for the host is the IP address of the MySQL server. So unsure why it cannot connect since it is running on the default port.
Here is my connection to Node.js:
pool = mysql.createPool({host:"localhost"
,port:"3306"
,database:"db_name"
,user:"user_name"
,password:"password_for_user"
,timezone:"utc"
,multipleStatements:true
,max:1000
,min:1
,idleTimeoutMillis:defs.QUERY_TIMEOUT});
if ( pool && pool.getConnection ) {
pool.getConnection(function(errsts, conn) {
var resp = {};
if ( errsts ) {
resp['error'] = errsts;
return;
}
resp['state'] = "connected";
if ( cbRoutine ) {
cbRoutine(conn, resp, objParams);
if ( conn != undefined ) {
conn.release();
}
}
});
}
localhost is correct for my usage, you should replace its your name or IP address.
defs.QUERY_TIMEOUT is defined in my source as:
var QUERY_TIMEOUT = 10 * 1000;
In my code cbRoutine is a call back function passed in as a parameter to call on successful connection.
The ECONNREFUSED error indicates that it can't connect to where the function is hosted (in this case localhost)
dbConnection.js
const mysql = require('mysql');
const connection = mysql.createPool({
connectionLimit: 10, // default = 10
host: '127.0.0.1',
user: 'root',
password: '',
database: 'dname'
});
module.exports = connection;
Hiii,
Recently my elastic beanstalk server have started to become unresponsive and return's 504 gateway timeout. I suspect that my pool.connect is becoming unresponsive and there are no logs reporting error while connecting to pool but it stucks at connecting. My other controllers with no database query works fine.
This goes away when I restart the server but after some time same thing happens.
I making requests this way-
1) database.js
const pg = require("pg")
// setting timestamp for the postgres.
pg.types.setTypeParser(1184, function(stringValue)
{
console.log(stringValue)
return new Date(Date.parse(Date.parse(stringValue + "+0000")))
})
// configuration for postres for connecting.
const pgConfig = {
user: "USER",
database: "DATABASE",
password: "1234",
port: 5432
}
const pool = new pg.Pool(pgConfig)
module.exports = pool
2) somecontroller.js
const db = require("../database/database")
const constant = require("../utility/constant")
module.exports = function(req, res)
{
db.connect(function(err, client, done)
{
if(err)
{
done()
console.log(constant.error.db.CONNECT_CONSOLE, err)
return res.send({status: constant.status.ERROR, code: constant.error.db.CONNECT})
}
client.query("QUERY", [UID])
.then(result =>
{
// Processing this queries and Some other query.....
})
.catch(err =>
{
console.log(constant.error.db.QUERY_CONSOLE, err)
res.send({status: constant.status.ERROR, code: constant.error.db.QUERY})
})
done()
})
}
My every controller works in similar fashion.
Thanks,
I'm currently developing Discord Bot, for data storing I'm using MySQL, but after some hours de connection dies. I was wondering if someone has a clue on how to do this. This the way I currently connect:
// Initalise Variables
var config;
var mysql, conn;
var fs;
try {
// External Packages
fs = require('fs');
config = require('./config.json');
mysql = require('mysql');
// Connection Setup
conn = mysql.createConnection({
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.password,
database: config.mysql.database
});
conn.connect();
} catch (e) {
console.error(e);
}
You can try changing the timeout setting else if would default to 10000ms or 10 seconds.
try {
// External Packages
fs = require('fs');
config = require('./config.json');
mysql = require('mysql');
// Connection Setup
conn = mysql.createConnection({
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.password,
database: config.mysql.database,
connectTimeout: config.mysql.timeout //1000000 some large number
});
conn.connect();
} catch (e) {
console.error(e);
}
You should probably use a connection pool instead of a single DB connection (this doesn't seem to have always keep alive type of setting, and you would have to give a large timeout like above)
try {
// External Packages
fs = require('fs');
config = require('./config.json');
mysql = require('mysql');
// Connection Setup
conn = mysql.createConnection({
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.password,
database: config.mysql.database,
connectionLimit: 10,
queueLimit: 20
});
var pool = mysql.createPool(config);
var queryDB = function(qry, cb) {
pool.getConnection(function(error, connection) {
if(error) {
cb(error, null);
}
else {
connection.query(qry, function (e, rows) {
connection.destroy();
cb(e, rows);
});
}
});
I think you need to install forever and start the service again. Forever is a simple CLI tool for ensuring that a given script runs continuously. Once you install forever and run the node js file then it will keeps the file alive continuously. Using npm you can install forever
npm install forever -g
Then just restart the js file
Ex:
forever start app.js
Hope now the js serves continously without breaking the connection.
Using the following tedious code, I can successfully connect to an Azure SQL Server.
const Connection = require('tedious').Connection;
const connection = new Connection({
userName: '[USER]',
password: '[PASSWORD]',
server: '[HOSTNAME]',
options: {encrypt: true}
});
connection.on('connect', (err) => {
if (err) {
console.log('error connecting', err);
} else {
console.log('connection successful');
}
});
However, using what should be the equivalent Sequelize code, I get a connection timeout error.
const Sequelize = require('sequelize');
const sequelize = new Sequelize('[DBNAME]', '[USER]', '[PASSWORD]', {
dialect: 'mssql',
host: '[HOSTNAME]',
dialectOptions: {
encrypt: true
}
});
sequelize.authenticate().then((err) => {
console.log('Connection successful', err);
})
.catch((err) => {
console.log('Unable to connect to database', err);
});
Any thoughts?
Using: sequelize 3.29.0, tedious 1.14.0, SQL Server v12
I was getting below error
SequelizeConnectionError: Server requires encryption, set 'encrypt' config option to true.
I tried it out with Azure SQL Database and below way is working for me.
const sequelize = new Sequelize('DB Name', 'Username', 'Password', {
host: 'Host',
dialect: 'mssql',
dialectOptions: {
options: {
encrypt: true,
}
}
});
If you're trying it out with Azure SQL Database, you might also want to specify a longer request timeout value:
[...]
dialectOptions: {
requestTimeout: 30000 // timeout = 30 seconds
}
[...]
I tried your Sequelize code and it works fine. So you might need to add Client IP address to allow access to Azure SQL Server. To do this, go to the Azure portal, click on All Resources, select your SQL server, click on Firewall in the SETTINGS menu.
Your client address is conveniently included in the list, so you can just click on Add client IP followed by Save. When you run your code now, it should connect.
if you are using sql server management studio then simply replace dialect:'mysql' with dialect:'mssql':
const sequelize = new Sequelize('DB Name', 'Username', 'Password', {
host: 'Host',
dialect: 'mssql',
dialectOptions: {
options: {
encrypt: true,
}
}
});