We Keep Coding

Why does promise chaining not work here in Node.js?

I have a script that scrapes a website and is supposed to send data back to client... JUST SO YOU KNOW IT'S ALL WITHIN app.get
Here is the code... The second .then does not work. It's supposed to send the arrays to the client after they have been populated once cheerio iterated through them. Yet it does not work... Maybe something wrong with the way I set up the second promise? Please help.
axios.get("https://www.sciencenews.org/").then(function(response){
var $ = cheerio.load(response.data);
$("div.field-item-node-ref").each(function(i,element){
if($(element).children('.ad').length == 0 && $(element).children('article').length > 0) {
array1.push($(element).find('h2.node-title').text());
array2.push($(element).find('div.content').text());
array3.push(`https://www.sciencenews.org${$(element).find('a').attr('href')}`);
array4.push(`${$(element).find('img').attr('src')}`);
}
})
}).then(()=>{
res.send({titles:array1,summaries:array2,links:array3,base64img:array4});
})

In the provided code snippet, none of the arrays are declared and there is no res object to call 'send' on since none is passed into the function.
Once the arrays are declared and res is temporarily replaced with a console.log, it appears to work on my end.
Working Repl.it
I'm assuming on your end that this function is called within a route, so there should be a 'res' object available within the scope of this function. If that's the case, it looks like it was just a matter of declaring your arrays before pushing data to them.

Nodejs & Request & Cheerio: Can't add to a global array from within the request

I am a relatively new javascript developer and I am stuck with an issue that I cannot seem to track down, and I think that I may be looking in the wrong spot. It looks like it may have something to do with scope/closures but I just cannot seem to get it working. Here it goes:
I have a globally scoped Array named arr. I am using node.js, request, and cheerio to hit a website and grab information -- in this illustration, the href attribute of every link on http://www.google.com. I am then inserting them into my globally scoped Array. However, outside of the 'request', the Array still seems to be empty.
var request = require('request');
var cheerio = require('cheerio');
var arr = []
var url = 'http://www.google.com';
request(url, function(err, resp, body) {
var $ = cheerio.load(body);
$('a').each(function(i, link){
arr.push(link['attribs']['href']);
});
});
console.log(arr); // prints nothing
I know that this is most likely something very simple that I am overlooking but if someone could help me shed some light as to exactly why, I would greatly appreciate it. Thank you.

Actually arr is being populated. But the statement console.log(arr) is excuated before the request call is completed. This is because of non-blocking property of node.js.
You can log link['attribs']['href'] inside the loop and you will see that arr will be printed first as an empty array [], and then the values inside the loop will be printed afterwards.

How to understand asynchronous Meteor.call on the client side

I'm working on a Meteor project and want to get the return value of Meteor.call in template helpers on client side. At very first, I just set a variable in the call back function and get the variable's value outside the Meteor.call. I found out the code after Meteor.call doesn't execute at all. Then I searched a bit and use Session, it works. But I don't really understand the reason. Here's my original code and modified code. Can anyone explain a bit for me? Thanks!!
Original wrong code: html
<div id="text-result-main">
<h2>{{title}}</h2>
</div>
js
Template.texts.helpers({
title: function(){
var index = Router.current().params.index;
Meteor.call('getTitle', index,function(error, result){
titles = result;
});
console.log(titles);
return titles;
}});
Collection text.js
Text = new Mongo.Collection("text");
Meteor.methods({
'getTitle': function(myindex){
return Text.findOne({index: myindex}).title;
}});
The working code: js
Template.texts.helpers({
title: function(){
var index = Router.current().params.index;
Meteor.call('getTitle', index,function(error, result){
Session.set("titles",result);
});
console.log(Session.get("titles"));
return Session.get("titles");
}});
Notice that I didn't publish Collection Text to the client at all because it's just so huge. Every time when I refresh the page when running the wrong code, I can't see the content of "title" or see it on the console. But when I set the session, it works. I don't really understand how it works here. Thanks

There is two issues Asynchronicity and Reactivity
This affectation
Meteor.call('getTitle', index,function(error, result){
titles = result;
});
inside the meteor call is executed but in a asynch way. So the return of your helper is immediately called, and return a empty value.
Try it out in the console of your browser.
But then, why your template render correctly with {{title}} when you use a Session Variable ?
It's because the Session is a reactive data source, witch means that every change to it trigger a re-computation of all templates involving this piece of data.
Here is a timeline:
Methods is called
Return empty value
Method is executed, setting variable value
If the Variable is a reactive data source, template is re-computed. ( in your case, the session is a reactive data source. )
To go further
I would use a reactive var in that case, it's very close from a session variable, but the scope is limited to a template.
A good read on Reactive data source: http://richsilv.github.io/meteor/meteor-reactive-data-types/

The problem is the fact that Meteor.call() is asynchronous when paired with a callback.
So when title() starts executing, it does not wait for your Meteor.call() invocation to return a result (or possibly an error). It continues execution. This is called asynchronous execution.
In short, you are trying to log the value for the key titles which doesn't exist in Session (since the state of your asynchronous Meteor call is unknown, at this point of time).
Try moving the console log statement into the callback paired with your Meteor.call() and you can see the result once it has successfully been set in Session.
A workaround to your problem is to make your Meteor.call() synchronous like this:
Template.texts.helpers({
title: function(){
var index = Router.current().params.index;
var result = Meteor.call('getTitle', index); // <--- this is synchronous code now
Session.set("titles",result);
console.log(Session.get("titles"));
return Session.get("titles");
}});
Removing the callback makes Meteor.call() behave synchronously.
If you do not pass a callback on the server, the method invocation
will block until the method is complete. It will eventually return the
return value of the method, or it will throw an exception if the
method threw an exception.
(from http://docs.meteor.com/api/methods.html#Meteor-call)

Why not use something like this:
title: function(){
var index = Router.current().params.index;
var a = Text.findOne({index: myindex}).title;
console.log(a);
return a;
without methods

Called a stored javascript function from Mongoose?

I have created a stored javascript function in my MongoDB instance that counts the number of records in each collection. If I go to my mongo shell and type:
> db.eval("getTotals()");
it works as expected. if I try to call it through mongo like so:
totals = mongoose.connection.db.eval("getTotals()");
console.log(totals);
undefined gets logged. Does anyone see what I am doing wrong here?

Most mongoose calls do not return in-line like this, but rather expect a callback to be passed in to process the results.
Completely untested, but you probably want something like:
mongoose.connection.db.eval("getTotals()", function(err, retVal) {
console.log(retVal)
});
And in the real world, assign your result to a var outside of that scope or whatever you want to do.

Initializing a variable declared outside of jQuery $.getJSON callback function

My script makes an Ajax request through jQuery. The data received is json_encoded via a server side script (php). This is further, stringified and then parseJSON'ed using jQuery to yield a recordCollection. I loop through recordCollection and push it onto records array, the variable that was declared outside the scope of the callback function.
The function looks like this:
var records = [] ;
$.getJSON(url, {id : recordid}, function(data) {
var recordCollection = jQuery.parseJSON(JSON.stringify(data));
$.each(recordCollection, function(){
records.push(this);
});
console.log(records) // displays all the objects, thus, the above code is right
});
console.log(records); // displays []
As you can notice above, the second time I logged records onto the console, it yielded an empty array, which is leading me to two conclusions:
Javascript arrays are passed by value, hence records outside the scope of the callback function does not retain it's value.
Ajax is asynchronous, hence records is being logged before the ajax call was complete and hence it still retains the value of the empty uninitialized records array and is logging that onto the console.
Now if 1 is true, how can I initialize the records array ?
And if 2 is true, should I make this Ajax call synchronous? This will make javascript block until the value is returned and thus the second time records array is logged onto the console, it will display the updated value of the array ?
The third is that I am completely missing a trick and doing something really dumb here.
I do need the records array to be set with the data returned from the ajax call as it needs to be passed around to different javascript functions in the script and the DOM basically waits for this result to load the data.
Thanks guys!

You are correct in that, ajax calls are async, hence Asynchronous Javascript and XML. You can make it sync, but you shouldn't because your array will then be available for anyone to play with, which can cause some big headaches.
Instead, have an init call that is run once your ajax call is completed.
function callback(records) {
// do stuff with records.
}
$.getJSON(url, {id : recordid}, function(data) {
var recordCollection = jQuery.parseJSON(JSON.stringify(data));
callback(recordCollection);
});

Ajax stands for Asynchronous JavaScript and XML (forget the XML part) so i think you can guess what is right ;-)
I added the execution order with comments:
var records = [] ;
// 1. send request and return immediately
$.getJSON(url, {id : recordid}, function(data) {
// 3. response from server is received
var recordCollection = jQuery.parseJSON(JSON.stringify(data));
$.each(recordCollection, function(){
records.push(this);
});
console.log(records)
});
// 2. print records
console.log(records);
You should not try to make the request synchronous in any way. If you need to do something with records after the data is available you should create a new function and call it after you pushed the values into records.