We Keep Coding

Node JS: How do I request multiple source codes from different websites?

I get an issue where If I try to use request more than once, it doesn't work. I can give an example:
request('https://www.google.com', function (error, response, vBody1) {
request('https://www.purple.com', function (error, response, vBody2) {
request('https://www.stackoverflow.com', function (error, response, vBody3) {
console.log(vBody3);
});
console.log(vBody2);
});
console.log(vBody1);
});
It reads and outputs the body of the first site, but not the next two.
Here is another example:
vSites = ['https://www.google.com','https://www.purple.com','https://www.stackoverflow.com'];
for (i = 0; i < vSites.length; i++){
request(vSites[i], function (error, response, vBody[i]) {
console.log(vBody[i]);
});
}
This one looks cleaner and I would like this version, but it doesn't work at all. What am I doing wrong?

In your loop version you don't need the index on the vBody variable.
var request = require('request');
vSites = ['https://www.google.com','https://www.purple.com','https://www.stackoverflow.com'];
for (i = 0; i < vSites.length; i++){
request(vSites[i], function (error, response, vBody) {
console.log(vBody);
});
}
This code works fine for me on node v4.3.2

In addition to the correct solution from Michael's reply, please be aware the for-loop sends the requests in parallel. Data and error handling could be challenging if you want to do more than console.log.
To better handle the requests, I would suggest using an asynchronous control library such as async or bluebird. In comparison, async is easier to understand while bluebird provides better code readability.
Below is an example async implementation:
var async = require('async');
var request = require('request');
var vSites = ['https://www.google.com', 'http://www.purple.com','https://www.stackoverflow.com'];
async.map(
vSites,
// `async` will call this function for each `vSite`
function(vSite, next) {
// Reuse the code inside your for-loop, but call `next` to pass the error and result to the final callback
request(vSite, function(err, response, vBody) {
next(err, vBody);
});
},
// The final callback will be executed when all requests are completed or either of them fails
function(err, results) {
console.log(err);
console.log(results);
}
);
Hope this helps :)

How to store multiple query values with MongoDB and express?

app.get('/ratings', function (req, res){
db.database.find({rating:5}).count(function(err, doc) {
review.FiveStarCount=doc;
});
db.database.find({rating:4}).count(function(err, doc) {
review.FourStarCount=doc;
});
I am new to using the MEAN stack and trying to return the values of the number of 5 and 4 star reviews. I can retrieve the number of 5 stars just fine, but when I do multiple , I seem to be running into trouble and not sure how to set it up.

You kind of need to get the gist of how to handle control flow while doing async operations.
The function execution would not stop, unlike php, when you are running an async operation.
To accomplish what you are trying to do, the second db call needs to be inside the callback of the first. Try this out.
app.get('/ratings', function (req, res, next) {
db.database.find({rating:5}).count(function(err, doc) {
// always handle error first
if (err) {
return next(err);
}
if (doc) {
review.FiveStarCount = doc;
}
db.database.find({rating:4}).count(function(err, doc) {
if (err) {
return next(err);
}
if (doc) {
review.FourStarCount = doc;
}
console.log(review.FiveStarCount, review.FourStarCount);
});
});
});

So it seems like you are running against the async nature of JavaScript. in the case of the functions above, they will not execute in the order you have written (and you wouldn't want them too!) because they are async. There are few simple solutions to get you going while you learn more about the problem.
Solution 1: Nested callbacks. Hard to read. also, it waits for the first to complete before firing the second, possibly slowing everything down.
app.get('/ratings', function(req, res) {
var review = {};
db.database.find({rating:5}).count(function(err, doc) {
review.FiveStarCount = doc;
db.database.find({rating:4}).count(function(err, doc) {
review.FourStarCount = doc;
// DO something with data
res.json(review);
});
});
Solution 2: Latch. This has the advantage of making the requests in parallel and not being as awful to read.
app.get('/ratings', function(req, res) {
var review = {};
db.database.find({rating:5}).count(function(err, doc) {
review.FiveStarCount=doc;
if (review.FiveStarCount && review.FourStarCount) {
// DO something here
// Will only execute if the other has finished
}
});
db.database.find({rating:4}).count(function(err, doc) {
review.FourStarCount=doc;
if (review.FiveStarCount && review.FourStarCount) {
// DO something here
// Will only execute if the other has finished
}
});
});
There are course more solutions, such as maybe a better query or promises. Maybe read a bit more about async JavaScript here to get a better understanding.

Database call in a function without callback with Node.js and Postgresql

I'm trying out the framework node.js on one of my projects.
I'm really seeing some good advantages on what they called "event-driven, non-blocking I/O model" however if my project there are some moments where I don't necessarily want to have some asynchronous calls and to be able to several operation before launching some asynchronous call.
Especially when I want to do some factorization and create some functions.
Typically I have the following case:
I know that in several part of my program I have to check if a media is existing in my database for a given string or id.
So as a guy who tried to stay organize I want to create a function that I will call each time I need to check this.
However, I did not find the way to do that with node.js and pg (the npm PostgreSQL library (https://github.com/brianc/node-postgres/) . Indeed, there is always a callback in the function and the return is null because of the callback. Here is an example below
/*
Function which is supposed to check if a media existing
*/
function is_media_existing (url_or_id){
log.debug("is_media_existing : begin of the function", {"Parameter" : url_or_id});
pg.connect(connectionstring, function (err, client, done) {
if (err) {
log.warning("is_media_existing : Problem with Database connection", {
"Parameter": url_or_id,
"Error": err
});
}
if (isNaN(url_or_id)) {
// Case is parameter is not a number (string)
var query = client.query('SELECT COUNT(*) as count FROM media WHERE url = $1::string ', url_or_id);
query.on('error', function (error) {
log.warning("is_media_existing : Problem with Database query (connection to db passed but not query " +
"", {"Parameter": url_or_id, "Error": error});
});
return query;
} else {
// Case is parameter is a int
log.debug("is_media_existing : Type of Parameter is a string");
// Case is parameter is not a number (string)
var query = client.query('SELECT COUNT(*) as count FROM media WHERE id = $1::id ', url_or_id);
query.on('error', function (error) {
log.warning("is_media_existing : Problem with Database query (connection to db passed but not query " +
"", {"Parameter": url_or_id, "Error": error});
});
return query;
}
});
}
// Executing the function
var test = is_media_existing("http://random_url_existing_in_db");
// test is always null as the return is in a callback and the callback is asynchronous
i have the feeling my question is touching the core concepts of node.js, and perhaps my approach is wrong and I apologize in advance.
I know it's not good to wait for a response before doing something.
But what's the alternative? How can I factorize my code into functions when I need some functionalities in several part of my code?
So if there would be anyone who could explain how to do that with a best practice of programming it would be great.
Thanks
Anselme

As Cody says, you probably dont want to do synchronous function.
The way you should handle the situation in your example is to pass in your own callback like this
function is_media_existing (url_or_id, callback){
and then instead of return query; use your callback like this-
callback(query);
or probably better to follow the node convention for callback functions to have two parameters (err, result) so your callback would look like this
callback(null, query);
Here is a rework of your sample
function is_media_existing (url_or_id, callback){ /* callback(err, result) */
log.debug("is_media_existing : begin of the function", {"Parameter" : url_or_id});
pg.connect(connectionstring, function (err, client, done) {
if (err) {
done(err);
log.warning("is_media_existing : Problem with Database connection", {
"Parameter": url_or_id,
"Error": err
});
return callback(err, null);
/* note that this return is simply used to exit the connect's callback and the return value is typically
* not used it is the call to callback() that returns the error value */
}
var qrystr;
if (isNaN(url_or_id)) {
log.debug("is_media_existing : Type of Parameter is a string");
qrystr = 'SELECT COUNT(*) as count FROM media WHERE url = $1::string;';
} else {
qrystr = 'SELECT COUNT(*) as count FROM media WHERE id = $1::id;';
}
client.query(qrystr, [url_or_id], function(err, result){
done();
if(err){
/* .. */
}
callback(err, result);
});
});
}
// Executing the function
var test = is_media_existing("http://random_url_existing_in_db", function(err, result){
if(err){
}else {
}
});
If you end up with a hard nest of callbacks, promises are really worth looking into.

I don't think you really do want a synchronous call. The problem with synchronous calls in node is that it stops the entire process from doing anything while a synchronous function is running as it will stop the event loop. As an example lets say your sync function takes 2 seconds to complete. Your server will then do nothing for 2 full seconds. That 2 seconds includes everything (accepting new connections, everything else, etc). The reason blocking functions don't exist is because they are (very) bad. Here is an example how your function will react in an async manor.
is_media_existing("http://random_url_existing_in_db", function(exists){
if (exists){
//do stuff
} else {
//do this other stuff
}
});
Then within is_media_existing you will need to call that callback function when your query completes.
//psuedo
function is_media_existing(url, callback){
query('select COUNT(*) as count FROM media WHERE id = $1::id '. [url], function(err, result){
if (err)
callback(false)
else
callback(result.count > 0)
})
}

With the new ES6 plus async stuff and babel its simpler. You can npm i -g babel npm i babel-runtime then compile and run the following with babel test.js --optional runtime --stage 2 | node. Please read the following example carefully to see how to adapt it to your use case:
let testData = [
{ id: 0, childIds: [1,2]},
{ id: 1, childIds:[] }
];
function dbGet(ids) {
return new Promise( r=> {
// this an example; you could do any db
// query here and call r with the results
r(ids.map((id) => { return testData[id];}));
});
}
async function dbExists(ids) {
let found = await dbGet(ids);
return (found && found.length>0);
}
async function test() {
var exists = await dbExists([0]);
console.log(exists);
}
test().then(f=>{}).catch( e=> {console.log('e',e)});

node.js asynchronous passing variable between functions

i am taking my first steps with node.js and i came across this issue with passing variable in asynchronous way. I have this piece of code im using to create Facebook user:
req.tmpPassport = {};
var fb = new fbgraph.Facebook(accessToken, 'v2.2');
function initUser() {
fb.me(function (err, me) {
req.tmpPassport.me = me;
console.log(req.tmpPassport.me) // works
});
}
console.log(req.tmpPassport.me) // not working -> undefined var
i tried to figure out why the second log isn't working and i ended up reading about synchronous and asynchronous functions, so in attempt to implement what i read i tried coming up with a solution using callbacks, but no success.
my last attempt was this:
req.tmpPassport = {};
var fb = new fbgraph.Facebook(accessToken, 'v2.2');
function initUser() {
fb.me(function (err, me) {
req.tmpPassport.me = me;
});
fb.my.events(function (err, events) {
//console.log(events);
req.tmpPassport.events = events;
});
fb.my.friends(function (err, result) {
req.tmpPassport.results = result;
});
}
function passUser(){
console.log(req.tmpPassport);
return req.tmpPassport;
}
cp.exec(initUser, passUser);
but its not working...
what i am actually trying to achieve its to render this object with my express router var which looks like this:
router.get('/welcome', securePages, function(req, res, next){
res.render('welcome', {title:'Welcome to aDating', user:req.tmpPassport});
})
but i cant figure out how to pass this object only after created...any help please?

A method of chaining function calls when certain async tasks are done is one way to deal with this.
Looking at the first snippet of code, it would be rewritten as follows:
req.tmpPassport = {};
var fb = new fbgraph.Facebook(accessToken, 'v2.2');
function initUser() {
fb.me(function (err, me) {
console.log(req.tmpPassport.me) // works
req.tmpPassport.me = me;
// triggers execution of the next step
post_populating_passport();
});
}
function post_populating_passport() {
// this function is only executed after the callback from the async call
console.log(req.tmpPassport.me);
}

Node.js is an asynchronous programming language, a fundamental concept in its core. You either need to use function callbacks (not a good practice) or available npms utilities for async flow.

Get node.js neDB data into a variable

I am able to insert and retrieve data from an neDB database in nodejs. But I cannot pass the data outside of the function that retrieves it.
I have read through the neDB documentation and I searched for and tried different combinations of callbacks and returns (see code below) without finding a solution.
I'm new to javascript so I do not know if I am misunderstanding how to use variables in general or if this issue is related to using neDB specifically or both.
Could someone please explain why "x" in my code does not ever contain the docs JSON results from the database? How can I make it work?
var fs = require('fs'),
Datastore = require('nedb')
, db = new Datastore({ filename: 'datastore', autoload: true });
//generate data to add to datafile
var document = { Shift: "Late"
, StartTime: "4:00PM"
, EndTime: "12:00AM"
};
// add the generated data to datafile
db.insert(document, function (err, newDoc) {
});
//test to ensure that this search returns data
db.find({ }, function (err, docs) {
console.log(JSON.stringify(docs)); // logs all of the data in docs
});
//attempt to get a variable "x" that has all
//of the data from the datafile
var x = function(err, callback){
db.find({ }, function (err, docs) {
callback(docs);
});
};
console.log(x); //logs "[Function]"
var x = db.find({ }, function (err, docs) {
return docs;
});
console.log(x); //logs "undefined"
var x = db.find({ }, function (err, docs) {
});
console.log(x); //logs "undefined"*

Callbacks are generally asynchronous in JavaScript meaning you can not use assignment operator, consequently you do not return anything from the callback function.
When you call an async function execution of you programme carries on, passing the 'var x = whatever' statement. The assignment to a variable, the result of whatever callback is received, you need to perform from within the callback itself... what you need is something in the lines of ...
var x = null;
db.find({ }, function (err, docs) {
x = docs;
do_something_when_you_get_your_result();
});
function do_something_when_you_get_your_result() {
console.log(x); // x have docs now
}
EDIT
Here is a nice blog post about asynchronous programming. And there is a lot more resources on this topic that you can pick up.
This is a popular library to help with async flow-control for node.
P.S.
Hope this helps. Please, by all means ask if you need something clarified :)

I ran into the same problem. In the end I used a combination between async-await and a promise with resolve to solve it.
In your example the following would work:
var x = new Promise((resolve,reject) {
db.find({ }, function (err, docs) {
resolve(docs);
});
});
console.log(x);

I had to learn a bit about async functions to get it right. For those who are looking for specific help about getting a return value from nedb, here's a snippet of what worked for me. I was using it in electron.
function findUser(searchParams,callBackFn)
{
db.find({}, function (err, docs))
{
//executes the callback
callBackFn(docs)
};
}
usage
findUser('John Doe',/*this is the callback -> */function(users){
for(i = 0; i < users.length; i++){
//the data will be here now
//eg users.phone will display the user's phone number
}})