I have a script that scrapes a website and is supposed to send data back to client... JUST SO YOU KNOW IT'S ALL WITHIN app.get
Here is the code... The second .then does not work. It's supposed to send the arrays to the client after they have been populated once cheerio iterated through them. Yet it does not work... Maybe something wrong with the way I set up the second promise? Please help.
axios.get("https://www.sciencenews.org/").then(function(response){
var $ = cheerio.load(response.data);
$("div.field-item-node-ref").each(function(i,element){
if($(element).children('.ad').length == 0 && $(element).children('article').length > 0) {
array1.push($(element).find('h2.node-title').text());
array2.push($(element).find('div.content').text());
array3.push(`https://www.sciencenews.org${$(element).find('a').attr('href')}`);
array4.push(`${$(element).find('img').attr('src')}`);
}
})
}).then(()=>{
res.send({titles:array1,summaries:array2,links:array3,base64img:array4});
})
In the provided code snippet, none of the arrays are declared and there is no res object to call 'send' on since none is passed into the function.
Once the arrays are declared and res is temporarily replaced with a console.log, it appears to work on my end.
Working Repl.it
I'm assuming on your end that this function is called within a route, so there should be a 'res' object available within the scope of this function. If that's the case, it looks like it was just a matter of declaring your arrays before pushing data to them.
Related
I am trying to understand how pm.sendRequest works.
I have a query and some Tests code that loops through a response and makes a second request using pm.sendRequest with variables from the initial response.
This works, but within the pm.sendRequest I have a loop that creates an array that I need to push into a global array (defined before pm.sendRequest is called).
The problem is the pushing doesn’t work. I end up with an empty array.
So my question is: are variables that are supposed to be global (to the code, not Postman global variables) not available inside pm.sendRequest?
Code sample:
let myArray = [];
pm.sendRequest( url, function (err, response){
let foo = [];
//a for loop here that push populates foo[] just fine
for {
foo.push(name);
} //end loop
console.info(foo); //all good
myArray.push(foo);
console.info(myArray) //all good
});
console.info(myArray); //empty
foo array is properly populated at the end, but my Array is empty.
Any ideas?
Thanks.
After much searching on this, I found that the pm.sendRequest method is an asynchronous request: https://learning.postman.com/docs/writing-scripts/script-references/postman-sandbox-api-reference/#sending-requests-from-scripts:~:text=You%20can%20use%20the%20pm.sendRequest%20method%20to%20send%20a%20request%20asynchronously%20from%20a%20Pre%2Drequest%20or%20Test%20script.
That means it's not a blocking request and it runs in the background, independently of the code that follows it.
As such, the last console.log. output does not wait for the pm.sendRequest function to complete and populate the array.
Some workarounds for this are possible, using global variables and promises:
https://community.postman.com/t/can-i-use-pm-sendrequest-to-send-requests-synchronously/225/14
https://james.jacobson.app/javascript-promises-and-postmans-sendrequest/
https://stackoverflow.com/a/53934401/10792097
However, overall, it looks like this is not the way to go about it in the Postman universe.
I think we're supposed to split out each request by itself and use request workflows: https://learning.postman.com/docs/running-collections/building-workflows/
This is what I'll be trying to do next.
I'm working on a Meteor project and want to get the return value of Meteor.call in template helpers on client side. At very first, I just set a variable in the call back function and get the variable's value outside the Meteor.call. I found out the code after Meteor.call doesn't execute at all. Then I searched a bit and use Session, it works. But I don't really understand the reason. Here's my original code and modified code. Can anyone explain a bit for me? Thanks!!
Original wrong code: html
<div id="text-result-main">
<h2>{{title}}</h2>
</div>
js
Template.texts.helpers({
title: function(){
var index = Router.current().params.index;
Meteor.call('getTitle', index,function(error, result){
titles = result;
});
console.log(titles);
return titles;
}});
Collection text.js
Text = new Mongo.Collection("text");
Meteor.methods({
'getTitle': function(myindex){
return Text.findOne({index: myindex}).title;
}});
The working code: js
Template.texts.helpers({
title: function(){
var index = Router.current().params.index;
Meteor.call('getTitle', index,function(error, result){
Session.set("titles",result);
});
console.log(Session.get("titles"));
return Session.get("titles");
}});
Notice that I didn't publish Collection Text to the client at all because it's just so huge. Every time when I refresh the page when running the wrong code, I can't see the content of "title" or see it on the console. But when I set the session, it works. I don't really understand how it works here. Thanks
There is two issues Asynchronicity and Reactivity
This affectation
Meteor.call('getTitle', index,function(error, result){
titles = result;
});
inside the meteor call is executed but in a asynch way. So the return of your helper is immediately called, and return a empty value.
Try it out in the console of your browser.
But then, why your template render correctly with {{title}} when you use a Session Variable ?
It's because the Session is a reactive data source, witch means that every change to it trigger a re-computation of all templates involving this piece of data.
Here is a timeline:
Methods is called
Return empty value
Method is executed, setting variable value
If the Variable is a reactive data source, template is re-computed. ( in your case, the session is a reactive data source. )
To go further
I would use a reactive var in that case, it's very close from a session variable, but the scope is limited to a template.
A good read on Reactive data source: http://richsilv.github.io/meteor/meteor-reactive-data-types/
The problem is the fact that Meteor.call() is asynchronous when paired with a callback.
So when title() starts executing, it does not wait for your Meteor.call() invocation to return a result (or possibly an error). It continues execution. This is called asynchronous execution.
In short, you are trying to log the value for the key titles which doesn't exist in Session (since the state of your asynchronous Meteor call is unknown, at this point of time).
Try moving the console log statement into the callback paired with your Meteor.call() and you can see the result once it has successfully been set in Session.
A workaround to your problem is to make your Meteor.call() synchronous like this:
Template.texts.helpers({
title: function(){
var index = Router.current().params.index;
var result = Meteor.call('getTitle', index); // <--- this is synchronous code now
Session.set("titles",result);
console.log(Session.get("titles"));
return Session.get("titles");
}});
Removing the callback makes Meteor.call() behave synchronously.
If you do not pass a callback on the server, the method invocation
will block until the method is complete. It will eventually return the
return value of the method, or it will throw an exception if the
method threw an exception.
(from http://docs.meteor.com/api/methods.html#Meteor-call)
Why not use something like this:
title: function(){
var index = Router.current().params.index;
var a = Text.findOne({index: myindex}).title;
console.log(a);
return a;
without methods
Before anyone marks it as duplicate, this post does not actually answer the question but suggests a different way altogether to solve that particular issue.
Mine is a different issue. Please let me explain.
In my case, there are various .js files (plugins) which are being loaded with jquery getscript and stored in variables. Then whenever required they will be executed (more than once)
The code for loading script (this code will only run once at the init of the system for each plugin js file)
var storedFunc;
$.getScript(pathToPluginJSFile, function( data, textStatus, jqxhr ) {
storedFunc = data;
});
All the plugins are in this format
(function(){
//lots of code here
})()
But when I checked the storedFunc variable in console, I found out that it has been stored as String variable. Like this,
"(function(){
//lots of code here
})()"
Now to execute this, I used eval, like this (this code can be executed multiple times based on the need)
eval(storedFunc)
Everything is working fine and i am happy with it, but here comes the problem, I read in somewhere that the usage of eval is kind of like a bad thing to do. So now I am afraid that thought everything is working fine, all these negativity of using eval spread on the internet might scare my client away. :(
So, please tell me how I can run that stored function (which has become a string) without using eval.
Or should I use anything else than $.getScript which does not convert a function into a string ?
Or if there is any other way altogether rewriting this plugin functionality?
Please show me the way. I am in need of this solution badly.
Any help will be appreciated.
Thanks in advance.
Understanding how $.getScript works
Seems there is some confusion on how $.getScript works. If you notice jQuery's documentation on the method, and as #Pointy made mention of in the comments, this is stated:
Load a JavaScript file from the server using a GET HTTP request, then execute it.
Here's an example: Let's pretend the contents of the file being returned is only this:
// Contents of yourExternalFile.js
console.log('Executed!');
Now, when you use $.getScript:
$.getScript(pathToPluginJSFile, function( data, textStatus, jqxhr ) {
// The script you retrieved has already executed at this point, and you will find "Executed!" in the console.
console.log('All Done');
});
Console output:
> Executed!
> All Done
The $.getScript method is not meant to be used to return a string of the content of the file. However, while that data is available in the callback, the contents of the file have already been executed. So by taking the string version of the file, and re-executing it with either new Function, or even eval, you are executing it twice on the page (jQuery does it once, and so do you).
Original Post:
Use the Function constructor instead of using eval.
// Your function as a string stored to a variable
var stringFunction = "(function(){console.log('Executed');})()";
// Use the Function constructor to create a new function:
var executableFunction = new Function(stringFunction);
// Now you can execute it
executableFunction(); // logs "Executed"
This snippet from this SO question/answer addresses the difference between eval and new Function.
eval() evaluates a string as a JavaScript expression within the current execution scope and can access local variables.
new Function() parses the JavaScript code stored in a string into a function object, which can then be called. It cannot access local variables because the code runs in a separate scope.
Additional Information (Based on comments)
Yes, you can just get the string contents of the file and store them to a variable without the contents of the file executing. You can have that function to execute anytime. You just need to use the regular get method using jquery, and set the dataType to text. This way, the script will not execute, and you can execute it as you see fit:
var storedFunction;
$.get({url: pathToPluginJSFile, dataType: 'text'})
.done(function (data) {
// Turn the script into a new function and store it
// The information in the script file has not done anything yet
storedFunction = new Function(data);
})
.fail(function () {
console.log('Failed :(');
});
The only thing you will have to watch out for, is making sure that the function was assigned to the storedFunction variable as you are making an api call, and you have to wait for that to finish before attempting to make the function execute.
// Later on, call that function anytime, and as often as you want:
storedFunction();
I am trying to make a simple Chrome Extension.
At the top of the document, I have put var maxLengthVar;.
I have stored a number using the chrome.storage API. I am getting the value using:
chrome.storage.sync.get('maxLength', function (items) {
maxLengthVar = items.maxLength;
console.log(maxLengthVar);
});
The console.log displays the correct value (2). But when I try to use this variable, I get undefined:
console.log(maxLengthVar);
document.getElementById("textToChange").innerHTML = maxLengthVar;
Note: this code is directly below the first piece of code.
I expect to have the console log the number 2 and the div textToChange's content to change to 2, but, instead, I get undefined in the console and the div stays the same. Why is this?
From what I understand, I have a global variable because at the top of the document, I declared the variable, so what makes the document.getElementById... not read the variable?
Another thing is that even though the console.log's I said above are in the same order in the document, they appear in the console reversed (ie. I get undefined first and then 2.) I know because I added some text ('text' + maxLengthVar) to the console.log's. Why is this?
You need to read up on async methods.
Basically the get method is an async method. While this method is executing the remainder of your script will execute printing out undefined
chrome.storage.sync.get('maxLength', function (items) {
maxLengthVar = items.maxLength;
console.log(maxLengthVar);
});
// <- at this point the async GET method may not have finished executing
// but your code continues anyway
console.log(maxLengthVar);
document.getElementById("textToChange").innerHTML = maxLengthVar;
Your code can be re-written using callbacks
chrome.storage.sync.get('maxLength', function (items) {
maxLengthVar = items.maxLength;
document.getElementById("textToChange").innerHTML = maxLengthVar;
});
I have created a stored javascript function in my MongoDB instance that counts the number of records in each collection. If I go to my mongo shell and type:
> db.eval("getTotals()");
it works as expected. if I try to call it through mongo like so:
totals = mongoose.connection.db.eval("getTotals()");
console.log(totals);
undefined gets logged. Does anyone see what I am doing wrong here?
Most mongoose calls do not return in-line like this, but rather expect a callback to be passed in to process the results.
Completely untested, but you probably want something like:
mongoose.connection.db.eval("getTotals()", function(err, retVal) {
console.log(retVal)
});
And in the real world, assign your result to a var outside of that scope or whatever you want to do.