Casper - understanding evaluate with function callbacks - javascript

This is a bit of a tricky question.
I am very familiar with javascript, however I am on a project that auto-crawls a website using PhantomJS and CasperJS. These are entirely new subjects to me.
I was able to figure out how to use Casper and navigate, log in to pages, etc, however it is unweildy as the general flow seems to be:
casper.start('http://google.fr/');
casper.then(function() {
this.echo("I'm in your google.");
});
casper.then(function() {
this.echo('Now, let me write something');
});
casper.then(function() {
this.echo('Oh well.');
});
casper.run();
My problem with this is that I want to do all sorts of things with the website, depending on what data is gotten with it. I can't pre-layout the sequence of navigations and not have it change. I hope this makes sense.
To solve this, I created a Javascript Navigator object with builtin functions. My general concept was:
navigator.logIn(function()
{
navigator.actionA(parameters, function()
{
if (navigator.data.a == navigator.data.b) {
navigator.actionB();
} else {
navigator.actionC();
}
});
});
And embedded in each of these functions would be casper functions.
Here is a shortened version of my actual code, and where things started getting funky:
var casper = require('casper').create({
clientScripts: [ 'jquery.min.js' ],
onError: function(self, m) {
console.log('FATAL:' + m);
self.exit();
},
});
var navigator = new _Navigator();
function _Navigator() { }
_Navigator.prototype.logIn = function(aCallback)
{
var self = this;
casper.start('https://website/login.asp', function()
{
if (1 == 1) {
this.evaluate(function() {
$("input[name=blah]").val('blahblah');
});
// ... A LOT MORE CODE
aCallback();
}
});
}
_Navigator.prototype.search = function(aDataSet, aCallback)
{
var self = this;
console.log('this works');
casper.then(function(){
console.log('this works');
});
var firstName = 'foobar';
casper.then(function(){
console.log('this works');
this.evaluate(function()
{
console.log('this no longer works!!');
$('input[id=blah]').val(firstName);
aCallback();
});
});
}
navigator.logIn(function() {
// LOG IN RUNS, AND CALLS BACK SUCCESSFULLY...
navigator.search({'dataset'}, function()
{
console.log('This never runs');
});
});
casper.run();
You'll notice that in the navigator.login function, I call casper.start(); In this, the evaluation function works fine, however then I do a callback function within that casper.start(); In my callback, I call the next function, navigator.search, which I suppose is still technically executing in the casper.start?
When I try running casper.evaluate within this new function called by the first callback function, everything seems to behave fine with the exception that casper.evaluate no longer works! It seems to eat the function, not printing any console logs or anything.
I have tried everything on this. I am not sure how to do this correctly. Does anyone have any suggestions on what I am doing wrong? Thanks.

I know this is quite old, but: What's going on here is a combination of two issues:
casper.evaluate() seems to eat all errors within the current stack - onError won't run from inside an .evaluate() callback.
Functions used in .evaluate are not standard closures - they're sandboxed, and have no access to variables outside their scope, unless passed as explicit arguments to casper.evaluate. So in the evaluated function where you call aCallback() there's no aCallback in scope, and the function will fail (silently) with a ReferenceError.

casper.evaluate() is as a window onto the headless browser session.
Anything that happens in functions passed to evaluate doesn't appear on your local console.
However, you can either log any value returned from evaluate or print all output by setting up a listener:
casper.on('remote.message', function(message) {
console.log(message);
});

Related

Know every function on scope

I have a function main that has several inner functions like this:
function main_f (params) {
function do_this () {
// do this...
}
function do_that () {
do_this(); // working
main_f.parse_stuff(); // not working
parse_stuff(); // not working
}
do_that();
main_f.parse_stuff = function(){
console.log("success");
}
}
function second_f () {
main_f.parse_stuff(); //working
}
I was expecting that main_f.parse_stuff() would work inside do_that, but that is not the case. My questions are:
-Is it posible to call that method from inside main_f ? how?
EDIT: Execute do_that after parse_stuff is written.
-Why can't I call parse_stuff from main_f?
EDIT: I just realised that the function doesn't read on compilation time, but execution time, therefore it is not visible when do_that is called.
-How can I know every function on scope?
It is not possible by programation but you can do it with the debugger. Just insert a break point on that scope and you can check everything that is global, local and in the closure.
I checked this with chrome dev-tools.

Ember Data's store promises don't work in quint tests

Update
A bit of context into some quirks of the illustrative code below. StoreProxy exists as a model, created by the ApplicationRouter, that has a reference to the store. This lets other objects access the store directly (for singletons, tests, etc). Example:
MyApp.StoreProxy = DS.Model.extend();
MyApp.ApplicationRoute = U.Route.extend({
model: function () {
return this.store.createRecord('storeProxy');
}
});
Before the route is executed, StoreProxy doesn't have a store property. After, it does. I can only assume this is because of some ember-data magic.
I very well realize your reaction to this may be "Ugh! No! You're doing it wrong!". Noted. We'll move to do it the right way from here over time. That said, this is where the code is now. So, given that, and given this method for getting a reference to the current store, why doesn't the code below call its accept or rejection handlers?
Original question
I'm writing a qUnit unit test for ember. I'm using fixture data. The findAll call on the store isn't resolving or rejecting the promise.
test('Find all in store', function() {
expect(1);
var findPromise;
findPromise = MyApp.StoreProxy.store.findAll('rule');
findPromise.then(function(result) {
console.log('yes');
ok(true);
}, function(error) {
console.log('no');
});
});
I tried using async tests mentioned in this question:
testing ember fixture data with quint but the resolve and reject are never called, so the test hangs indefinitely.
I've also tried placing Ember.run calls around my code, in case it's a weird run loop thing. But to no avail.
asyncTest('Find all in store', 1, function() {
var findPromise;
Ember.run(function() {
findPromise = MyApp.StoreProxy.store.findAll('rule');
findPromise.then(function(result) {
console.log('yes');
ok(true);
start();
}, function(error) {
console.log('no');
start();
});
});
});
The code I'm testing runs fine when I run the application normally (fixture adapter or no), so it feels like something with the test environment.
Any thoughts on what to try? I'm stumped.
The way that you're writing your asynchronous tests is incorrect. Check out QUnit's page on async testing. Your test should look something like this:
asyncTest('Find all in store', function() {
var findPromise = ...;
findPromise.then(function(result) {
start();
ok(result);
}, function() {
start();
ok(false);
});
});
Specifically:
You put an extra parameter in the asyncTest function, which likely causes the test to not run at all.
You're using Ember.Application.store, which is not how you should access your store (and probably isn't even a valid store). I'm not sure what your context is, but you should be getting your store from elsewhere.
You're putting the start() calls after your assertions when they should be before.

Recognising variables while assigning a function to a variable in javascript

In my jQuery scripts, when the user closes a menu with an animation, I have to call a function after the closing animation is finished. I want to assign this function dynamically by calling a function openStrip() with a parameter. My code looks like:
var FUNCTION_JUST_AFTER_MENU_CLOSE = function(){};
function openStrip(stripId){
FUNCTION_JUST_AFTER_MENU_CLOSE = function(){
createStrip(stripId);
});
}
if I call openStrip("aStripId"), I expect FUNCTION_JUST_AFTER_MENU_CLOSE to be:
// #1
function(){
createStrip("aStripId");
}
whereas my current code gives:
//#2
function(){
createStrip(stripId);
}
i.e, the parameter passed to the function openStrip() is lost while assigning the function() to the variable FUNCTION_JUST_AFTER_MENU_CLOSE.
How can I avoid this.
EDIT: I discovered that my code is actually working. The problem was elsewhere. I got confused because when I looked at Chrome's debugger, it was showing me the function definition as is (#2 in above). But when it actually went down executing that function later in the code, it did evaluate the values of the passed argument, and endedup executing #1.
Thanks for the answer though. I am marking it correct because that is perhaps a better way of assigning the function.
The best way is to return a function, from openStrip like this
function openStrip(stripId) {
return function() {
createStrip(stripId);
};
}
For example,
function openStrip(stripId) {
return function() {
console.log(stripId);
};
}
openStrip("aStripId")();
# aStripId
openStrip("bStripId")();
# bStripId
You can even assign the function objects returned to different variables and use them later on
var aStrip = openStrip("aStripId");
aStrip();
# aStripId
aStrip();
# aStripId

Why doesn't plain JavaScript work in CasperJS?

I'm new to CasperJS. How come this.echo(this.getTitle()); works but console.log("Page Title ", document.title); doesn't? Also why isn't my document.querySelector working? Does anyone have a good explanation? Where in the CasperJS documentation can I find the answer?
Here's my code:
var casper = require('casper').create();
var url = 'http://www.example.com/';
casper.start(url, function() {
this.echo(this.getTitle()); // works
this.echo(this.getCurrentUrl()); // works
});
casper.then(function(){
this.echo(this.getCurrentUrl()); // works
console.log("this is URL: ", document.URL); // doesn't work
console.log("Page Title ", document.title); // doesn't work
var paragraph = document.querySelectorAll('p')[0].innerHTML;
console.log(paragraph); // doesn't work
});
casper.run();
EDIT:
I'm using casper.thenEvaluate and casper.evaluate now and it's still not working. Any ideas?
var casper = require('casper').create();
var url = 'http://www.example.com/';
casper.start(url, function() {
this.echo(this.getTitle()); // works
this.echo(this.getCurrentUrl()); // works
console.log('page loaded: '); // works
});
casper.thenEvaluate(function(){
var paragraph = document.querySelectorAll('p')[0].innerHTML; // doesn't work
console.log(paragraph); // doesn't work
console.log("Page Title ", document.title); // doesn't work
});
casper.run();
You have to call functions that depend on document with this.evaluate:
var paragraph = this.evaluate(function() {
return document.querySelector('p').innerHtml;
});
When in doubt, consult the docs.
CasperJS has inherited the split between DOM context (page context) and the outer context from PhantomJS. You can only access the sandboxed DOM context through casper.evaluate(). document inside of the evaluate() callback is the variable that you would expect in normal JavaScript, but there is also a document outside of evaluate() which is only a dummy object and doesn't provide access to the DOM of the page.
If you want to access DOM properties, then you need to use evaluate().
var title = casper.evaluate(function(){
return document.title;
});
But this won't work for DOM nodes, because only primitive objects can be passed out of the DOM context. The PhantomJS documentation says the following:
Note: The arguments and the return value to the evaluate function must be a simple primitive object. The rule of thumb: if it can be serialized via JSON, then it is fine.
Closures, functions, DOM nodes, etc. will not work!
If you want to use document.querySelector(), then you need to produce a representation of a DOM node that can be passed outside:
var form = casper.evaluate(function() {
var f = document.querySelector('form');
return { html: f.innerHTML, action: f.action };
});
casper.echo(JSON.stringify(form, undefined, 4));
You can also use all of the available CasperJS functions that can provide representations of DOM nodes such as casper.getElementsInfo().
Also, have a look at Understanding the evaluate function in CasperJS.
this.getTitle() executes getTitle() function on Casper object and runs in Casper context, hence it produces the expected result.
However, 'document' is not available in Casper context. The underlying reason is that Casper is running PhantomJS, which is a web browser. So, 'document' is only available in the browser, which is one level "deeper" than the code that runs in Casper context. There is no direct way to share variables between the two environments but there is a way to pass them as parameters by copying the value.
The "bridge" between the two environments (Casper and Phantom) is Casper's 'evaluate' function. Everything inside the function, passed to 'evaluate' as a parameter, will get executed in the browser context, not in Casper context. That's an important distinction. The documentation is available here, as noted by Blender:
http://docs.casperjs.org/en/latest/modules/casper.html#evaluate
Example below:
casper.evaluate(function(username, password) {
document.querySelector('#username').value = username;
document.querySelector('#password').value = password;
document.querySelector('#submit').click();
}, 'sheldon.cooper', 'b4z1ng4');
In the given example you can see how to pass "username" and "password" parameters from Casper environment to the browser (page) environment.
The anonymous "function(username,password)" will get executed within the browser. Therefore, you can use 'document' inside it.
You can also pass the value back, which can be picked up on Casper side. I.e.
var result = casper.evaluate(function run_in_browser(){
return document.title;
});
Try this.echo(this.fetchText('p')); to get innerhtml. Refer documentation

Using a closure to execute a function in the future?

I'm trying to use a closure (I think that's what it is..), I'd just like to execute a function with a local variable at some point in the future, like this:
function boo() {
var message = 'hello!';
var grok = function() { alert(message); }
foo(grok);
}
function foo(myClosure) {
$.ajax({
timeout: 8000,
success: function(json) {
myClosure();
}
}
}
I could get around this by using global variables and such, but would rather use something like the above because it at least seems a bit cleaner. How (if possible) do you do this?
Thanks
----------- Update --------------------
Sorry I wasn't clear - I was wondering if this is the correct syntax for the closure, I tried it out and it seems ok. Thank you.
Your existing code looks perfectly fine except for that missing paren at the end. ;)
If you're looking to understand the concept of closures more deeply, think of it this way: whenever something in a closured language is defined, it maintains a reference to the local scope in which it was defined.
In the case of your code, the parameter to $.ajax() is a newly-created object ("{ timeout: 8000, etc. }"), which contains a newly-created function (the anonymous "success" function), which contains a reference to a local variable ("myClosure") in the same scope. When the "success" function finally runs, it will use that reference to the local scope to get at "myClosure", even if "foo()" ran a long time ago. The downside to this is that you can end up with a lot of unfreeable data tied up in closures -- the data won't be freed until all references to it have been removed.
In retrospect, I may have confused you more than helped you. Sorry if that's the case. :\
Unless you actually want to make an AJAX call, setTimeout might be more along the lines of what you are looking for:
function foo(myClosure) {
setTimeout(myClosure, 8000); // execute the supplied function after 8 seconds
}
If your question was more along the lines of "Am I creating a closure correctly?", then yes, your function boo is doing the right thing.
Is it what you want?
var boo = (function() {
var message = 'hello!';
return function() {
foo(function() {
alert(message);
});
};
})();
function foo(myClosure) {
$.ajax({
timeout: 8000,
success: function(json) {
myClosure();
}
}
}
or just
function boo() {
$.ajax({
timeout: 8000,
success: function(json) {
alert('hello!');
// do sth with json
// ...
}
}); // <- missed a paren
}
The example is too simple to know what you want btw.

Categories