Github repo. I am trying to use MongoDB Atlas database with my node JS Login & Signup app for storing data. The problem is that the data is not saving to the database or in other words the request isn't going through even if my app is connected to Atlas. Full code available on www.github.com/tahseen09/login
// Connection to mongodb atlas
const uri = "mongodb+srv://tahseen09:<PASSWORD>#cluster0-pirty.mongodb.net/userdb"
MongoClient.connect(uri, function(err, client) {
if(err) {
console.log('Error occurred while connecting to MongoDB Atlas...\n',err);
}
console.log('Connected to Atlas');
const collection = client.db("userdb").collection("credentials");
client.close();
});
//New User Registration
app.post('/register', function(req,res){
var cred= new credential();
cred.uname=req.body.uname;
const hash = bcrypt.hashSync(req.body.password, 10);
cred.password=hash;
collection.save(function(err,newuser){
if(err){
res.status(500).send("Username exists");
}
else{
res.status(200).send("New User Created");
}
})
})
The code that is important is attached as a snippet and the rest of the code is available on www.github.com/tahseen09/login
Note: I am running this app on localhost.
Let me describe your flow so you can understand wrong point there :)
Connect to MongoDB
Create reference to the collection
Close connection
When someone tries to access /register route, you already have closed connection by that time. Thus, any operation attempt to the database will end up with connection error.
From the documentation it's recommended calling MongoClient.connect once and reusing the database variable returned by the callback, i.e. do not close connection manually, driver will just create and use pool of connections, so don't worry about closing connection. Check out example code in the documentation.
Lets step through the code to see what happens:
MongoClient.connect(uri, function(err, client) {
A connection to mongodb is created, then somewhen the connection is established or it fails, then the callback gets called back. Now you create a local variable holding the database reference:
const collection = client.db("userdb").collection("credentials");
And then you close the connection:
client.close();
Then the callback ends:
});
which means that a variables inside (connection) can't be accessed anymore and get therefore recycled.
Now somewhen (that might even happen before the db connection gets established), someone requests the webpage and you try to do:
collection.save(/*...*/);
That won't work for various reasons:
1) The db might not even be opened
2) If it was opened already, it was also closed already.
3) Even if it is open at the moment, you still cannot access connection as it is not in scope.
Now to resolve that we have to:
1) only start the webserver when the db connection is establishee
2) don't close the connection
3) expose the connection so that it can be used elsewhere
For that it makes sense to create a function that establishes the connection and calls back with the db:
function withCredentials(callback) {
const uri = "mongodb+srv://tahseen09:<PASSWORD>#cluster0-pirty.mongodb.net/userdb"
MongoClient.connect(uri, function(err, client) {
if(err) {
console.log('Error occurred while connecting to MongoDB Atlas...\n',err);
} else {
console.log('Connected to Atlas');
const collection = client.db("userdb").collection("credentials");
callback(collection);
}
});
}
So now you can use that:
withCredentials(function(credentials) {
app.post('/register', function(req,res){
const cred = { };
cred.uname = req.body.uname;
cred.password = bcrypt.hashSync(req.body.password, 10);
credentials.insertOne(cred, function(err,newuser){
if(err){
res.status(500).send("Username exists");
} else {
res.status(200).send("New User Created");
}
})
});
});
Related
I started working on a MERN App today and am trying to write a restful api. First I am using mlab to store my mongodb database. I have succesfully connected to this database after creating a user. I can manually create a collection and inject some data into this collection. From my server.js file I can then get the data stored in here.
MongoClient.connect(db_url, (err, database) => {
if (err) return console.log(err);
var collection = database.collection('memories'); // Collection called memories
app.listen(3000, () => {
console.log("Listening on 3000");
});
});
Thats all fine and dandy but I want to take it to the next level. I want to write a CRUD api for the collection Memory. Coming from django, I would like to create my model first. Therefore, in my models/memory.js:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var MemorySchema = new Schema({
name: String,
description: String
});
module.exports = mongoose.model('Memory', MemorySchema);
Then I went ahead and started working on my routes/api/api.js:
let router = require('express').Router();
let Memory = require('../../../models/memories');
router.use(function (req, res, next) {
console.log("Something is happening");
next(); // Request stops at middleware without next()
});
router.route('/memory')
.post(function (req, res) {
let memory = new Memory();
memory.name = req.body.name;
memory.description = req.body.description;
memory.save(function (err) {
if (err) {
res.send(err);
}
res.json({message: 'Memory Created'});
});
})
.get(function (req, res) {
res.json({message: 'First memory'});
});
module.exports = router;
And in my server.js I call this module:
const apiRoutes = require('./routes/api/api');
app.use('/api/', apiRoutes);
However, after testing the post api with postman, it the POST request just takes forever before showing up as Could not get any response. However, the GET request works. What am I missing?
EDIT: So the post function is having trouble saving the model instance...
Try adding results as the first parameter in the callback of the save function, then res.json(results, { message: "Memory Created" }) to see if you are returned anything.
The main difference between the post and the get method is that the post method uses Mongoose, while the get doesn't. If you fail to connect to the database then the response can time out due to memory.save(...) not working as it should. And there are no responses sent outside the callback to save, so if your program never enter it, you will never send a response. The request will time out eventually.
In your model file you register a model on the following line:
module.exports = mongoose.model('Memory', MemorySchema);
Mongoose will then look for data in the memorys collection. If you change it to
module.exports = mongoose.model('Memory', MemorySchema, 'memories');
it will use the memories collection instead. This will make it consistent with the connection-to-db snippet you posted. I don't know if that will fix your issue though. I would suggest changing the connection code to
mongoose.connect(dburl, {
useMongoClient: true
});
instead of the native mongo client. You can add these lines too
mongoose.connection.on('connected', function () {
console.log('Mongoose connected');
});
mongoose.connection.on('error', function (err) {
console.log('Mongoose connection error: ' + err);
});
mongoose.connection.on('disconnected', function () {
console.log('Mongoose disconnected');
});
right after the connection code to help with debugging. Make sure you get connected when starting the app.
If you see an error similar to this Error: Can't set headers after they are sent. in the node terminal window, it might be because you are sending two responses in the post function. If an error occurs while saving it will enter the if(err) block, send a response async then go to the res.json(...) response and send that too.
So you have to return after sending the response to exit the function. Either like this
res.send(err);
return;
or like this
return res.send(err);
Same for the json response.
If that doesn't fix the problem you should either fire up the debugger (node --inspect or nodemon --inspect), or insert a console.log('inside post'); inside the post function to see that you're actually entering it.
I'm creating a web server that stores a user's data in a MongoDB database. The code behind the web requests uses asynchronous functions to insert a document into the database, but because these functions are asynchronous it means that for every request a new connection is made with the server.
exports.create_user = function(username, password, callback) {
mongo.connect(url, function(err, db) {
db.collection('users').insertOne({username: username, password: password}, function(err, result) {
callback(result)
db.close()
})
})
}
I'm under the impression that doing it this way is not the best practise, but I can't think a way to do it using the module model that I'm using above. Any suggestions or advice would be appreciated.
I stumbled upon this on my own research whether to use a new connection for mongodb on each query is the best practice or to use connection pooling. Turns out, that mongodb suggests connection pooling for most use-cases.
Citing from the docs:
A Connection Pool is a cache of database connections maintained by the driver so that connections can be re-used when new connections to the database are required. To reduce the number of connection pools created by your application, we recommend calling MongoClient.connect once and reusing the database variable returned by the callback
I am usually using the following form to establish and reuse a connection while firing queries:
// db.js
import { MongoClient } from 'mongodb';
// this will hold our cached database connection, which will itself hold multiple connections in a pool to be used
let connection,
database;
export {
connect: (next) => {
// already established? => return connection
if (database) return next(undefined, database);
// establish connection
MongoClient.connect('http://localhost:27017/admin', (err, db) => {
if (err) return next(err);
// save connection
connection = db;
// connect to database
database = db.db('myDatabase');
// call callback
next(undefined, database);
});
},
disconnect: (next) => {
if (!connection) return next();
// close connection
connection.close();
next();
}
};
Firing queries:
import db from './db';
db.connect((err, db) => {
if (err) return next(err);
db.collection('myUsers').insertOne({name: 'test'}, (err) => {
if (err) throw err;
db.disconnect((err) => {
if (err) throw err;
console.log('Everything finished, database connection closed');
});
});
});
Note: It is possible to determine the maximum amount of pooled connections manually (afaik the default is 5?). Refer to the docs about how to set the amount of opened connections via the mongodb url.
By doing db.close() you can close the connection, If you don't close your connection, event loop will keep the connection open and your process will not exit. If you are building a web server where your process will not be terminated, it's not necessary for you to close the connection.
For a reference node-mongodb-native
I'm going crazy with node pg module, getting 'too many clients already' error.
My app.js file for example, manages some routes in which I query some data to postgres. app.js looks like bellow:
//First I create a client
var client = new pg.Client(connectionString);
// Then I use that client to every routes, for example:
ContPg.prototype.someController = function(req, res){
client.connect(function(error){
if(error) return console.error('error conectando', error);
// Need to close client if there's an error connecting??
client.query(someQuery, function(e,r){
client.end();
// Here sometimes I dont end client if i need to query more data
if(e) return console.error('error consultando', e);
// Do anything with result...
})
});
}
As I said I use that client for all routes in file pg.js, but in other files with other routes I do the same to connect to postgres (create client and use for all routes that manage that file)
Questions
Is something wrong with my code? I ended wrong client connection?
If there's nothing wrong, what could be causing 'too many clients already' error?
Thanks in advance!!
The recommended pattern is to use client pooling. From the node-postgres documentation:
Generally you will access the PostgreSQL server through a pool of
clients. A client takes a non-trivial amount of time to establish a
new connection. A client also consumes a non-trivial amount of
resources on the PostgreSQL server - not something you want to do on
every http request. Good news: node-postgres ships with built in
client pooling.
var pg = require('pg');
var conString = "postgres://username:password#localhost/database";
//this initializes a connection pool
//it will keep idle connections open for a (configurable) 30 seconds
//and set a limit of 20 (also configurable)
pg.connect(conString, function(err, client, done) {
if(err) {
return console.error('error fetching client from pool', err);
}
client.query('SELECT $1::int AS number', ['1'], function(err, result) {
//call `done()` to release the client back to the pool
done();
if(err) {
return console.error('error running query', err);
}
console.log(result.rows[0].number);
//output: 1
});
});
Don't forget to call done() or you'll be in trouble!
I'm making a web application using the MEAN framework and MVC design pattern. I am trying to perform a POST request from the Angular front-end for finding a document in my server-side MongoDB (version 2.4.9). The console logs show that the query is successful, but when I try to send the response back to the client, the query result is undefined.
I understand that NodeJS is asynchronous and uses callbacks, but I am having trouble understanding what is wrong with my code. I tried using returns and callbacks but I can't get it working. I'm confused how to use the controller to access the model and have the controller ultimately send the response.
Here is my code to connect to the database (model):
module.exports = {
readDocument : function(callback, coll, owner) {
// Connect to database
MongoClient.connect("mongodb://localhost:27017/tradingpost", function(err, db) {
if (err) {
console.log("Cannot connect to db (db.js)");
callback(err);
}
else {
console.log("Connected to DB from db.js: ", db.databaseName);
//Read document by owner
// Get the documents collection
var collection = db.collection(coll);
// Find document
collection.find({owner: owner}).toArray(function (err, result) {
if (err) {
console.log(err);
} else if (result.length) {
console.log('Found:', result);
} else {
console.log('No document(s) found with defined "find" criteria!');
}
// Close connection
db.close();
return callback(result);
});
}
})
}}
And here is my controller that sends the response:
var model = require('../models/db');
exports.sendRecentPosts = function (req,res) {
// Connect to the DB
// Run query for recent posts
// Close the connection
// Send the data to the client
var result = model.readDocument(dbCallback, "gs", "Mana");
res.end( result );
};
Client's post request:
// Use post for secure queries
// Need recent posts for display
$http.post('/recent').
success(function(responseData) {
$scope.testValue = responseData;
}).
error(function(responseData) {
console.log('Recent posts POST error. Received: ', responseData);
});
Snippet for my express route:
var goodsServices = require('../controllers/gs-server-controller.js');
app.post('/recent', goodsServices.sendRecentPosts);
I have been struggling with this for a long time and searched the forum for solutions but could not find any. Thanks for any feedback.
I do not know why this question has not been answered yet. When I faced the same problem, I learnt that the response to all DB queries are returned after the DB transaction is complete. Try placing db.close() within the success callback response of the find() call.
I'm using socket.io and Redis together with node.js - code below.
I'm definitely being stupid here, but I can't understand why the Redis event only fires with the first user who connects.
I see the NEW CONNECTION console message again for each user who opens the page, but I only see the REDIS console message with the first user.
io = io.listen(app);
client.flushall();
io.sockets.on('connection', function (socket) {
// Update Redis with information about this connection.
console.log('*************** NEW CONNECTION');
client.multi()
.hmset("user:" + user_id, "nickname", nick)
.sadd("chatroom:" + room_id, user_id)
.exec();
// Check that the Redis record was set.
client.smembers("chatroom:" + room_id, function (err, data) {
if (err) { console.log(err); return; }
console.log('REDIS CALL FOR chatroom:' + room_id, data);
});
});
Do I need to create a separate Redis instance inside io.sockets.on for each client - and if so, will this scale?
Or otherwise, what am I doing wrong?
Got it! I was being stupid.
I was calling client.quit() inside socket.on('disconnect'...) - which was obviously deleting the Redis client altogether. DUH.
Hope this helps someone else in the future, anyway.