In my express application I use Sequelize to manage the database. Here's my bootstrap code.
db.sequelize
.sync({ force: true})
.complete(function(err) {
if (err) {
throw err[0];
} else {
//seed
require('../db/seed')(db);
app.listen(app.get('port'), function() {
console.log('express listening on ' + app.get('port'));
});
}
});
As you can see, everytime I start the server first I seed the database with hard coded data.
Here's the code to initialize sequelize:
if (process.env.NODE_ENV === 'test') {
console.log('[test] using in memory database');
sequelize = new Sequelize('marbles-site-db', null, null, {
dialect: 'sqlite',
storage: ':memory:'
});
} else if (process.env.HEROKU_POSTGRESQL_WHITE_URL) {
var match = process.env.HEROKU_POSTGRESQL_WHITE_URL
.match(/postgres:\/\/([^:]+):([^#]+)#([^:]+):(\d+)\/(.+)/);
sequelize = new Sequelize(match[5], match[1], match[2], {
dialect: 'postgres',
protocol: 'postgres',
port: match[4],
host: match[3],
logging: true // false
});
} else {
sequelize = new Sequelize('marbles-site-db', null, null, {
dialect: 'sqlite',
storage: './db/development.sqlite'
});
}
As you can see, I use SQlite with flat file as storage for development and Postgres for production.
The problem is each time I restart the server, data in the database are gone. This is the case with SQlite, I suppose it's the same with Postgres too.
So what do I have to do to keep the data in the database even though I restart the server, and is this related to database migrations?
Note that the seed should not be needed after the initial boot.
sync({ force: true }) is mostly meant for testing, both user testing and testing internally in sequelize.
sync() does CREATE TABLE IF NOT EXISTS, so it can in theory be used in a production setting, since it will never destroy any existing data. However, if you change your schema in models, that will not be reflected in the DB if you only do sync
So, in a production setting you will need migrations. This means you both need to change your sequelize models, and write a migration each time you chnage something. There is talks about being able to automagically create migrations when the sequelize model file changes #120 but no work has been done on that yet
Be aware of sequelize.sync method resets all the data at the start of the code. Maybe you can produce another solution without using sync() method
Related
I'm working on a MERN webapp (using MySQL instead of MongoDB) and we are having an issue where at some point after querying to add an entry, the backend is somehow re-querying the last query. In the meantime, between the original entry and the duplicate one, the backend runs continuously to keep the frontend displaying up-to-date data.
I've read on old threads that MySQL for Node may have some issues with memory leaking and thus running sleep queries. I don't believe this is a frontend issue, and the backend isn't running the queries as if they were called from the function that handles the frontend button click (to submit).
I'm a noob to full-stack JS, so I don't really know where to look for an answer to this issue. I'm not sure if we should band-aid the solution and just make sure a query isn't sent without the submit button being clicked within a 1 second time span.
EDIT: Here's code for the query to add the entry (data for a music album).
db.query(insertAlbumQuery, insertAlbumValues, function(err, result) {
if (err) throw err;
console.log("1 Album added.");
});
We don't close the connection to the db after querying, one connection runs for the duration of the app's execution.
Here's the code for the db connection:
let mysql = require("mysql");
var db;
function connectDB() {
if (!db) {
db = connection = mysql.createConnection({
host: "******",
user: "******",
password: "******",
database: "******"
});
connection.connect(function(err) {
if (err) {
return console.error("error: " + err.message);
} else {
console.log("Connected to the MySQL server.");
}
});
}
return db;
}
module.exports = connectDB();
From the main, backend app.js file we have the router post
router.post("/addAlbum", (req, res) => {
const {
Album_title,
Artist,
Release_date,
Category,
Description,
Rotation
} = req.body;
album.add(Album_title, Artist, Release_date, Category, Description, Rotation, null);
});
From the front end we post with
addAlbum = currentState => {
this.setState({ Rotation: +this.state.Rotation });
axios.post("http://localhost:3001/api/addAlbum", this.state);
};
Not sure if this is enough background on the app, but I can add more snippets if needed.
I can connect to the DB through terminal, but getting this error using mongoose and gulp.
mongoose/node_modules/mongodb/lib/mongodb/connection/base.js:246
MongoError: auth failed
My connection string is:
mongodb://usr:psw#localhost:27017/dbname
Any idea what it can be?
I installed MEAN from MEAN packaged by Bitnami for windows 7 using the following password: 123456
Syntax for connection string to connect to mongodb with mongoose module
mongoose.connect("mongodb://[usr]:[pwd]#localhost:[port]/[db]",{auth:{authdb:"admin"}});
If you don't have {auth:{authdb:"admin"}} in the connection string, you will get the following error: MongoError: Authentication failed.
JS Example: mongo-test/app.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://root:123456#localhost/test',{auth:{authdb:"admin"}});
mongoose.set('debug', true); // turn on debug
just add ?authSource=yourDB&w=1 to end of db url
mongoose.connect('mongodb://user:password#host/yourDB?authSource=yourDB&w=1')
this work for me . &w=1 is important
There is many ways to make it work. This is what worked for me [mongoose v5.9.15] :
mongoose.connect('mongodb://localhost:27017/', {
auth: {
user:'root',
password:'example'
},
authSource:"admin",
useUnifiedTopology: true,
useNewUrlParser: true
})
You might want to do something like this...
var opt = {
user: config.username,
pass: config.password,
auth: {
authdb: 'admin'
}
};
var connection = mongoose.createConnection(config.database.host, 'mydatabase', config.database.port, opt);
'authdb' option is the database you created the user under.
mongoose.connect("mongodb://[host]/[db]", { auth:{
authdb: "admin",
user: [username],
password: [pw]
}}).then(function(db){
// do whatever you want
mongoose.connection.close() // close db
})
Do you have a user set up for dbname? By default, no user is required to connect to the database unless you explicitly set one. If you haven't, you should just try to connect to mongodb://localhost:27017/dbname and see if you still get an error.
I have found the solution hier, looks like when you create an user from the mongo shell, it makes SCRAM-SHA-1 instead of MongoDB-CR. So the solution to create a new user with MongoDB-CR authentication.
MongoDB-CR Authentication failed
just make sure that your database is created.
and also if your user is not added in the admin database, then make sure to add it by putting
db.createUser(
... {user:'admin',pwd:'admin',roles:['root']}
... )
This worked for me for mongod --version = db version v3.6.13
mongoose.connect('mongodb://localhost/expressapi', {
auth: {
authdb: "admin",
user: "root",
password: "root",
}
});
mongo mongodb://usr:psw#localhost:27017/dbname
Password should be alphanumeric only
User should be also available in db 'dbname' (Note : Even if user is super admin)
With above changes it connected successfully.
mongoose.connect("mongodb://[usr]:[pwd]#localhost:[port]/[db]",{ authSource: 'admin', useNewUrlParser: true, useUnifiedTopology: true });
I was getting same error. Resolved by adding authSource option to connect function solved the issue. see above code.
The connection string will be like
mongodb://username:password#localhost:27017/yourdbname?authSource=admin
When I try to use Node.js and Tedioius to connect to a local SQL Server instance I get this error:
{ [ConnectionError: Failed to connect to XXXXX:1433 - connect ECONNREFUSED]
name: 'ConnectionError',
message: 'Failed to connect to XXXXX:1433 - connect ECONNREFUSED',
code: 'ESOCKET' }
Here is my connection object:
var config = {
userName: 'username',
password: 'password',
server: 'XXXXX',
options: {
database: 'databasename',
instancename: 'SQLEXPRESS'
}
};
I have checked and TCP/IP is enabled and broadcasting on port 1443 according to Configuration Manager. The SQL Server Browser service is also running, which I read may be causing this type of issue if not. I have disabled my antivirus and firewall and that hasn't helped either.
Any insight?
So what I am guessing happens is that even though Tedious lets you include instance name in 'options' it either doesn't use it or can't use it as it needs to be used. After doing some research, what should be happening is when you give SQL Server the instance name, it redirects you from port 1433 to the dynamic port it is using for that instance. I didn't know it was using a dynamic port, but if your instance is named the port will always be dynamic. I don't know where I saw it broadcasting on 1433, that was my mistake.
To check the dynamic port, look here:
From this information, I changed my code to this:
var config = {
userName: 'username',
password: 'password',
server: 'XXXXX',
options: {
port: 49175,
database: 'databasename',
instancename: 'SQLEXPRESS'
}
};
All is good now, hope this helps someone.
If anyone else is new to SQL Server like I am, and is dealing with this issue, once you enable TCP/IP in SQL Server Config Manager by following these steps:
> SQL Server Network Config
> Protocols for YOURSQLSERVERINSTANCE
> TCP/IP
> Enable
you get a warning message that looks like this:
Any changes made will be saved; however, they will not take effect until the service is stopped and restarted.
I took this to mean, disconnect from the database service in SQL Server Management Studio and reconnect, but this needs to happen in SQL Server Config Manager under the SQL Server Services tab. Find you SQL Server instance, stop and restart it, and hopefully you will be golden! This worked like a charm for me. Oddly, enabling the Named Pipes protocol seemed to work without a restart (I could see the difference in the error message), so I thought for sure it had stopped and restarted as needed.
Also, be sure to enable SQL Server Browser services as well. This and enabling TCP/IP and restarting the service were the keys for me.
If you still have problems after enabling TCP/IP protocol, I would suggest you check that SQL Server Browser Service is running. In my case I spent a lot of time till I realised it wasn't running.
This configuration run fine for me:
var config = {
user: 'user',
password: 'userPwd',
server: 'localhost',
database: 'myDatabase',
options: {
truestedConnection: true,
instanceName: 'SQLEXPRESS'
}
If you still got this error,
"...'Failed to connect to Server:1433 - connect ECONNREFUSED Server IP:1433',
code: 'ESOCKET' }"
and you've checked all the following:
Enable TCP/IP
Open Port 1433
Config setup correctly (database, server, username and password}
No Dynamic ports configured
Check your SQL server version. In my case, I discovered that I could connect to SQL 2012, but not SQL server 2016 with the same code. It appears SQL Server 2016 is not supported by the tedious driver yet.
...
You have to enabled tcp/ip on protocol for MSSQLSERVER
and activate both authentication
here is the complete code
const {
Request
} = require('tedious');
var Connection = require('tedious').Connection;
var config = {
server: 'DESKTOP-RU9C12L', //update me
authentication: {
type: 'default',
options: {
userName: 'begaak', //update me
password: 'begaak#123', //update me
}
},
options: {
encrypt: true,
enableArithAbort: true,
integratedSecurity: true,
trustServerCertificate: true,
rowCollectionOnDone: true,
database: 'selvapoc' //update me
}
};
var connection = new Connection(config);
connection.connect(function(err) {
console.log('testing')
// var request = new Request("Select * from products", function(err, rowCount, rows) {
// console.log(rowCount);
// console.log(JSON.stringify(rows))
// });
// connection.execSql(request);
connection.execSql(new Request('SELECT * FROM Products', function(err, rowCount, rows) {
if (err) {
throw err;
}
})
.on('doneInProc', function(rowCount, more, rows) {
console.log(more, rows[0], rowCount); // not empty
}));
});
connection.on('connect', function(err) {
// If no error, then good to proceed.
if (err) console.log(err)
console.log("Connected");
});
before starting the code configure these with SQL SERVER CONFIGURATION MANAGER
Is it currently possible to connect breeze-sequelize with a MS SQL server?
According to the doc of Sequelize, Sequelize does support MSSQL Server.
Though in the breeze doc there is no MS SQL server listed.
I am a bit confused now. And if it is not possible, is the breeze dev team planning to impl that? Or are there alternatives to use breeze in nodejs with an MSSQL server?
Yes it is actually possible. It took quite some time since the breeze-sequelize documentation is not very "newb friendly".
Here is my configuration for the MS SQL server. The tempHire example from the breeze samples on github was helping out a lot.
var dbConfig = {
user: 'username',
password: 'secret',
dbName: 'myDatabase'
};
var sequelizeOptions = {
host: 'hostname',
dialect: 'mssql',
port: 1433
};
function createSequelizeManager() {
var metadata = readMetadata();
var sm = new SequelizeManager(dbConfig, sequelizeOptions);
sm.importMetadata(metadata);
return sm;
}
The only thing i could not figure out until now is how to communicate with a specific MS SQL instance on a host e.g. localhost\MY_MSSQL_INSTANCE.
UPDATE
I actually did find out how to connect to a specific named instance.
So if you want to connect to an mssql instance like localhost\MY_MSSQL_INSTANCE, the sequelizeOptions should look like the following:
var sequelizeOptions = {
host: 'localhost',
dialect: 'mssql',
dialectOptions: {
instanceName: 'MY_MSSQL_INSTANCE'
}
};
Let's suppose I have these two models below:
var user = Bookshelf.Model.extend({ tableName: 'users' });
var configuration = Bookshelf.Model.extend({ tableName: 'configurations' });
user.fetchAll().then(function(users) {
configuration.fetchAll().then(function(configurations) {
// Do something...
});
});
Do anyone knows if the fetchAll() calls shares the opened connection? Or maybe every database call uses a specific connection? If they don't share, is there a way to specify to use the same connection until I close the operation or something? How BookshelfJS behaves in this matter?
Thank you all!
Bookshelf.js uses Knex.js under the hoods so the answer to your question lies in how Knex handles connections.
Knex can use connection pooling from the generic-pool-redux library. It has a default of 2 open connection and a maximum of 10 for postgres and mysql databases and a single connection for sqlite.
This is an example of configuration file to change the default values.
development: {
client: 'postgresql',
connection: {
database: 'db',
user: 'username',
password: 'mypassword',
charset : 'utf8'
},
pool: {
min: 5,
max: 20
},
migrations: {
tableName: 'migrations'
}
}
And his is a portion of the bookshelf documentation for fetchAll
Fetches a collection of models from the database, using any query parameters currently set on the model to form a select query
So it will form a single Knex query and execute it consuming only one connection.
Depending on your configuration file it will use a single connection for everything or any available from the pool.
There is an option that allow you to change the connection used to execute the query but for that you must invoke the bookshelf query() method to tap directly into the Knex query builder.