Async: Connecting to MongoDB after the server started - javascript

I would love to know a few things about Node.js async and MongoDB.
If the server starts before my connection to, and my templates or Ajax depend on data from the database will serving (I precompile my handlebars) the HTML file fail or will my templates/Ajax wait for the connection and then continue?
If it works, I would love to understand better how exactly it worked!
If it fails, how can I "fix" it elegantly?
This is an example of a solution, using hoisting (seems bad to me tbh):
//connect to db
mongodb.connect("someDBplace",
(err, db)=>{
if(err){
return generalLogger.error(`The database has failed to connect: ${err}.`);
}else{ //start the server now:
generalLogger.info(`Connected to database: ${stringify(db.databaseName)}.`);
server.listen(`${port}`, (err)=>{
if(err){
generalLogger.error(err);
}
//for demo: console th eMIMEtype
generalLogger.info(`The Server started on port: ${port}.`);
});
}
});

Yes, this is a correct way to do it. If you want to make sure that the database is available before the server starts serving requests, then you need to start the server in the callback of the database connection method.
Some higher level frameworks like Hapi may provide you some mechanisms to simplify the case when you need to wait for several things before starting. For example see the Hapi plugins:
https://hapijs.com/tutorials/plugins

Related

Node.js app that runs without a user and stores data in a database

I'm trying to build a backend infrastructure that hits specific API endpoints repetitively on a given time duration, once every second for example. The system would then post (or whatever the appropriate equivalent to a post request would be) the response to my database so that I can work with the data at a later time.
I was thinking Node would work since I am already working with Javascript, as I understand it, it essentially allows JS to run on a server. I want to use MongoDB because it's easier to change the schema vs something like postgreSQL.
With Express you could do something like this when getting all the users (this code is from an api I made so it might not be exact):
router.get('/users', function(req, res, next) {
const userData = {
_id: false,
password: false,
about: false,
__v: false
}
User.find({}, userData, function(err, user) {
if(err) {
res.send('Unable to find user');
console.log(err);
} else {
res.json(user);
}
});
});
Obviously the main problem is that this relies on a user going to a route the executes this code, How can I make it so a Node app (or express if possible) can just sit on the server, I can run the file node app.js and then it would constantly run and add the response data to the database (which would be on the same server) without having to worry about going to any pages or otherwise making the code execute?
Please let me know if you have any questions, or would like more clarification.
Store the code you'd like to have run in a reusable regular function instead coupling it as an anonymous function with the route handler. Then, use a scheduler like node-cron or something similar which calls the function at intervals.

Not connecting to database with node.js

I'm trying to connect and request queries from my database, but I wanna do it with JavaScript (or jQuery).
I'm trying this:
var mysql = require('mysql');
var con = mysql.createConnection({
host: "localhost",
user: "myUser",
password: "myPassword",
database: "mydb"
});
function GetSqlResult (sql_query) {
con.connect(function(err) {
if (err) throw err;
//Select all customers and return the result object:
con.query(sql_query, function (err, result, fields) {
if (err) throw err;
return result;
});
});
};
But I get this error:
Uncaught ReferenceError: require is not defined
I've tried using using the module "require.js" from this website http://requirejs.org/docs/release/2.2.0/minified/require.js
I still got the same error.
I want a solution for this error (if it exists), not an alternative.
I know that connecting to a database from JavaScript isn't very safe, but I really wanna do it, so I would love for someone to help me, please.
Thank you
What you are trying to do without knowing it, is build a RESTful API. In web architecture, you have your front end code (browser javascript) which makes HTTP requests to a backend. That backend will connect to the database and run queries. Even if you wanted to do this from browser javascript, your browser wouldn't let you. I would recommend you write a REST Api using NodeJS, and have your browser javascript make HTTP requests to that backend. Hope this helps.
NodeJS is not jquery, nodeJs it runs on a backend web server.
you can use Meteor for connecting to database from the client side.

How to run simultaneous Node child processes

TL;DR: I have an endpoint on an Express server that runs some cpu-bound logic in a child_process. The problem is that if the server gets more than one request for that endpoint it won't run both requests simultaneously- it queues them up and runs them one-at-a-time. Is there a way to use Node child_process so that my server will perform multiple child processes simultaneously?
Long-Version: The major downfall of Node is that it is single-threaded and a logic-heavy (cpu-bound) request can make the server stop dead in its tracks so that it can't take anymore requests until that logic is finished running. I thought that I could work around this using child_process, which is working great in freeing up my server to take other requests. BUT- it will only execute child_processes one at a time, creating a queue that can get pretty backed-up. I also have a Node cluster setup so that my server is split into 8 separate "virtual servers" (8-core machine), so I guess I can technically run 8 of these child processes at once, but I want to be able to handle more traffic than that. Looking for a solution that will still allow me to use Node and Express, please only suggest using different technologies if you are absolutely sure this can't be efficiently done in my current environment. Thanks in advance for the help!
Endpoint:
app.get('/cpu-exec-file', function(req, res) {
child_process.execFile('node', ['./blocking_tasks/mathCruncher.js'], {timeout:30000}, function(err, stdout, stderr) {
var data = JSON.parse(stdout);
res.send(data);
})
});
mathCruncher.js:
var obj = {}
function myLoop (i) {
setTimeout(function () {
obj[i] = Math.random() * 100;
if (--i) {
myLoop(i);
} else {
string = JSON.stringify(obj);
console.log(string); // goes to stdout.
}
}, 1000)
};
myLoop(10);
Is there a way to use Node child_process so that my server will perform multiple child processes simultaneously?
message queue and back-end process.
i do exactly what you're wanting, using RabbitMQ. there are several other great messaging systems out there, like ZeroMQ and even Redis w/ some pub-sub libraries on top of it.
the gist of it is to send a request to your queueing system and have another process pick up the message, then run the process to do the work.
if you need a response from the worker, you can use bi-directional messaging with either a Request/Reply setup, or use status messages for really-long-running things.
if you're interested in the RabbitMQ side of things, I have a free email course on various patterns with RabbitMQ, including Request/Reply and status emails: http://derickbailey.com/email-courses/rabbitmq-patterns-for-applications/
and if you're interested in ground-up training on RMQ w/ Node, check out my training course at http://rabbitmq4devs.com

Node.js - passing sql data to a module on client request

I am connecting to a sql db and returning data on the view using res.json. The client sends a request - my server uses a mssql driver and a connection string to connect to that database and retrieve some data. So I've got a connection using GET, POST, ect.
However I am encountering a logical problem as I want to pass some data from the sql db to a module which will then use that data to prepare a json response. When I hard code an array with couple of parameters it works, but it don't know how to send a request from node.js to a db and propagate that array for a module to consume when a client sends a request. (When a client sends a request the module send a request to that db and the db returns some parameters which then the module can use to prepare a response.)
Any ideas? Could you point me in the right direction? How from a logical point of view such solution can work?
I am using node.js, express, mssql module to connect to db. I am not looking for specific code just to point me in the right direction, and if you've got any examples of course I'm happy to see those.
You will probably need to have a chain of callbacks and pass data through them, something like this:
app.get('/users', function (req, res) {
database.find('find criteria', function (err, data) {
mymodule.formatData(data, function(err, json) {
res.json(json);
})
});
});
So you just nest the callbacks until you have everything you need to send the response.
You need to get used to this style of programming in node.js.
There are also some solutions to avoid too deep callbacks nesting - split your callbacks into individual functions or use async, promises, es6 generators.

Node/Express pending request

I'm bit new on the Node.js front and for now its awesome. I've encountered a small problem while running node (/w express) locally - Every request after the 10th one hangs and is marked as Pending in the Chrome Inspect Network.
As for modules i'm using less-middleware, express, jade and MySQL and i only do one SQL query (using mysql.createPool). Why is this request still Pending and how can i fix this?
Since i'm new at Node i'm not sure if i've tried everything so any help would be appreciated!
It sounds like you're not releasing the MySQL connection you're getting back from the pool. If you don't, the pool will run out of free connections and will start waiting for any to become available (and until then, stall the request).
So your code should look similar to this:
var pool = mysql.createPool(...);
...
// in your request handler:
pool.getConnection(function(err, connection) {
if (err) ...handle error...;
connection.query(function(err, results) {
// release connection
connection.release();
// handle results
...
// send back a response
res.send(...);
});
});

Categories