We Keep Coding

jquery .load() does not load external JS

I am loading external content into a div element using jquery.load() without a selector. If the content loaded has embedded JS, the JS works as expected. HOWEVER, if the content includes a script tag with src=path-to-js-code the js-code is not loaded.
Am I correct in this observation and if so is there a good solution other than embedding the JS in the loaded content?
EDIT :
A few clarifications and observations:
To load the content I am using
$("#DivId").load("path/to/content.php", CallbackFunction(response, status, xhr) {
error checking and post processing code
});
Changing the load code to:
$.get("path/to/content.php", CallbackFunction(response, status, xhr) {
error checking
$("#DivId").html(response);
post processing
});
Does not seem to change the behavior (more on the behavior below)
I have not tried parsing the response to retreive the script src and then using getScript().
Now more on the behavior...
Using Firefox, it seems that the external JS is loaded but only if it has been about 2 min from the last load. I do not see an attempt in Firebug unless the refresh is about 2m after the last load of the external JS. (weird). When I was making JS code changes and hitting refresh, it was not loading my new code and thus the original question.
So i will withdraw my question in light of this clarified behavior (2m caching?).
Thanks.

Both the .load() and .html() jQuery methods utilise the .innerHTML property. This won't execute scripts added with <script> tag. Use a regular AJAX call e.g. .get() then in the callback use .append() to add your HTML string and the scripts will run once it's parsed e.g.
$.get("path/to/content.php", function(response, status, xhr) {
// error checking
$("#DivId").append(response); // Any <script> tags in the response string will execute
// post processing
});
Thing is you need to make sure you're running trusted code if it's added by .append()

I was wondering you can get the script src in the response text of $.load method with regular expressions, then use $.getScript() method to load the script, maybe something like this:
$("#DivId").load("path/to/content.php", function(response, status, xhr) {
var regexp = new RegExp('script.*?src="(.*?)"'),
execresults = regexp.exec(response);
if(execresults.length > 1)
{
// the first result is the entire match including
// the 'script..src=', so abandon it
var matches = execresults.slice(1);
$.each(matches, function(){
$.getScript(this, function(){
// do something after load script
});
});
}
});
Hope this can help

This is the easy way to load an external JS to your jQuery
$.ajax({
type: "GET",
url: "path/to/content.php",
dataType: "script"
success:CallbackFunction(response, status, xhr)
});

getting OnLoad HTML/DOM for an HTML page in PHP

I am trying to get the HTML (ie what you see initially when the page completes loading) for some web-page URI. Stripping out all error checking and assuming static HTML, it's a single line of code:
function GetDisplayedHTML($uri) {
return file_get_contents($uri);
}
This works fine for static HTML, and is easy to extend by simple parsing, if the page has static file dependencies/references. So tags like <script src="XXX">, <a href="XXX">, <img src="XXX">, and CSS, can also be detected and the dependencies returned in an array, if they matter.
But what about web pages where the HTML is dynamically created using events/AJAX? For example suppose the HTML for the web page is just a brief AJAX-based or OnLoad script that builds the visible web page? Then parsing alone won't work.
I guess what I need is a way from within PHP, to open and render the http response (ie the HTML we get at first) via some javascript engine or browser, and once it 'stabilises', capture the HTML (or static DOM?) that's now present, which will be what the user's actually seeing.
Since such a webpage could continually change itself, I'd have to define "stable" (OnLoad or after X seconds?). I also don't need to capture any timer or async event states (ie "things set in motion that might cause web page updates at some future time"). I only need enough of the DOM to represent the static appearance the user could see, at that time.
What would I need to do, to achieve this programmatically in PHP?

To render page with JS you need to use some browser. PhantomJS was created for tasks like this. Here is simple script to run with Phantom:
var webPage = require('webpage');
var page = webPage.create();
var system = require('system');
var args = system.args;
if (args.length === 1) {
console.log('First argument must be page URL!');
} else {
page.open(args[1], function (status) {
window.setTimeout(function () { //Wait for scripts to run
var content = page.content;
console.log(content);
phantom.exit();
}, 500);
});
}
It returns resulting HTML to console output.
You can run it from console like this:
./phantomjs.exe render.js http://yandex.ru
Or you can use PHP to run it:
<?php
$path = dirname(__FILE__);
$html = shell_exec($path . DIRECTORY_SEPARATOR . 'phantomjs.exe render.js http://phantomjs.org/');
echo htmlspecialchars($html);
My PHP code assumes that PhantomJS executable is in the same directory as PHP script.

PhantomJS how to render javascript in html string

I'm trying to get PhantomJS to take an html string and then have it render the full page as a browser would (including execution of any javascript in the page source). I need the resulting html result as a string. I have seen examples of page.open which is of no use since I already have the page source in my database.
Do I need to use page.open to trigger the javascript rendering engine in PhantomJS? Is there anyway to do this all in memory (ie.. without page.open making a request or reading/writing html source from/to disk?
I have seen a similar question and answer here but it doesn't quite solve my issue. After running the code below, nothing I do seems to render the javascript in the html source string.
var page = require('webpage').create();
page.setContent('raw html and javascript in this string', 'http://whatever.com');
//everything i've tried from here on doesn't execute the javascript in the string
--------------Update---------------
Tried the following based on the suggestion below but this still does not work. Just returns the raw source that I supplied with no javascript rendered.
var page = require('webpage').create();
page.settings.localToRemoteUrlAccessEnabled = true;
page.settings.webSecurityEnabled = false;
page.onLoadFinished = function(){
var resultingHtml = page.evaluate(function() {
return document.documentElement.innerHTML;
});
console.log(resultingHtml);
//console.log(page.content); // this didn't work either
phantom.exit();
};
page.url = input.Url;
page.content = input.RawHtml;
//page.setContent(input.RawHtml, input.Url); //this didn't work either

The following works
page.onLoadFinished = function(){
console.log(page.content); // rendered content
};
page.content = "your source html string";
But you have to keep in mind that if you set the page from a string, the domain will be about:blank. So if the html loads resources from other domains, then you should run PhantomJS with the --web-security=false --local-to-remote-url-access=true commandline options:
phantomjs --web-security=false --local-to-remote-url-access=true script.js
Additionally, you may need to wait for the completion of the JavaScript execution which might be not be finished when PhantomJS thought it finished. Use either setTimeout() to wait a static amount of time or waitFor() to wait for a specific condition on a page. More robust ways to wait for a full page are given in this question: phantomjs not waiting for “full” page load

The setTimeout made it work even though I'm not excited to wait a set amount of time for each page. The waitFor approach that is discussed here doesn't work since I have no idea what elements each page might have.
var system = require('system');
var page = require('webpage').create();
page.setContent(input.RawHtml, input.Url);
window.setTimeout(function () {
console.log(page.content);
phantom.exit();
}, input.WaitToRenderTimeInMilliseconds);

Maybe not the answer you want, but using PhantomJsCloud.com you can do it easily, Here's an example: http://api.phantomjscloud.com/api/browser/v2/a-demo-key-with-low-quota-per-ip-address/?request={url:%22http://example.com%22,content:%22%3Ch1%3ENew%20Content!%3C/h1%3E%22,renderType:%22png%22,scripts:{domReady:[%22var%20hiDiv=document.createElement%28%27div%27%29;hiDiv.innerHTML=%27Hello%20World!%27;document.body.appendChild%28hiDiv%29;window._pjscMeta.scriptOutput={Goodbye:%27World%27};%22]},outputAsJson:false} The "New Content!" is the content that replaces the original content, and the "Hello World!" is placed in the page by a script.
If you want to do this via normal PhantomJs, you'll need to use the injectJs or includeJs functions, after the page content is loaded.

Firefox extension development: How do I create a global function inside a content script, so other loaded script files can access it?

In my main.js file, I have a PageMod that includes a base.js file:
pageMod.PageMod({
contentScriptFile: ["./base.js"]
I have a function inside of my base.js file
function setupPayment(){ /* DO STUFF HERE */ }
Inside my base.js file, I'm also loading other JS files
$.getScript("https://checkout.stripe.com/checkout.js", function(){
$.getScript( self.options.stripe );
});
Inside of my stripe.js file, I'm trying to call the setupPayment function that's in my base.js file
var yearhandler = StripeCheckout.configure({
key: "pk_live_...",
image: "image.png",
name: "SHINE",
description: "Subscription",
panelLabel: "Subscribe",
allowRememberMe: false,
token: function(token) {
plan = "yearly";
setupPayment(token,plan);
}
});
But setupPayment returns undefined.
And after doing some testing, it looks like any script included via $.getScript can't access any functions inside of my base.js content script? Is there any way to make a function inside of my base.js content script global across all my other scripts files that I load?
Thanks for your help!
Edit: the reason setupPayment() has to be in the base.js file, is so that it can communicate with the main.js file and store some information.

You can now export a function from the content script into the page, see this blog post for the gory details. The code would look like this:
function setupPayment(args, callback) {
// some code
callback(result);
// your callback should use postMessage to send data back to the
// content script, see these docs:
// https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts/Interacting_with_page_scripts#Communicating_with_page_scripts
}
exportFunction(setupPayment, unsafeWindow, {defineAs: "setupPayment"});

ContentScripts run in sandboxes, I don't know what jquery does internally, but it certainly is not designed to be aware of multiple javascript contexts and mozilla's xray wrappers, so it probably just injects a <script> tag into the DOM which then runs the loaded javascript inside the page context instead of the contentScript sandbox.
And I'm not even sure how you get access to jquery in your sandbox, considering that you only load base.js.
So it looks like a good chunk of your addon code is actually running in/loading scripts into the untrusted page context.
the reason setupPayment() has to be in the base.js file, is so that it can communicate with the main.js file and store some information.
Not quite. You can export a privileged function to the page context via exportFunction, which is available inside the sandbox.

How to prevent a race condition when loading on-demand JavaScript files?

To speed up things, my web application loads JavaScript files for certain features on-demand using jQuery's ajax call. I've encountered a race condition problem, though. For example, file B.js depends on file A.js. I've configured a list like list=[A.js,B.js] and I iterate over this list and asynchronously get the scripts. How can I prevent this race condition?
The code that loads the JavaScript files is something like:
loadJS: function(url) {
$.ajax({url: url, dataType: 'script', success: function(){}});
}

This kind of dependency management is not incredibly hard. Without getting too much into how to properly abstract it, if you have an array of files to load, on your ajax callback (which I assume is downloading the contents of the js file to a string) save the string in another array until you download the contents of all the files. Once all the files are in memory, add them to the dom in order.
You will want to add some extra stuff like handling failures and the like but that is a general outline of how to do it.

Suppose you asynchronously load files 1 and 2 using:
function load(URI) {
$.ajax({
url: URI,
dataType: "script",
});
}
and file 2 has dependency on file 1 then the files can be loaded asynchronously as:
load(url1)
.then(load(url2));
and if file 2 doesn't have dependency on file 1 then they can be loaded as:
$.when(load(url1), load(url2))
.then(function(e) { // both scripts loaded successfully});

You can just add script tags instead of making ajax calls:
var scripts = ['a.js', 'b.js'],
idx;
for(idx in scripts) {
if(scripts.hasOwnProperty(idx)) {
document.write(['<script src="', scripts[idx], '"></script>'].join(''));
}
}
Scripts will be loaded asynchronously, but executed synchronously in the order you added them.