We Keep Coding

getting OnLoad HTML/DOM for an HTML page in PHP

I am trying to get the HTML (ie what you see initially when the page completes loading) for some web-page URI. Stripping out all error checking and assuming static HTML, it's a single line of code:
function GetDisplayedHTML($uri) {
return file_get_contents($uri);
}
This works fine for static HTML, and is easy to extend by simple parsing, if the page has static file dependencies/references. So tags like <script src="XXX">, <a href="XXX">, <img src="XXX">, and CSS, can also be detected and the dependencies returned in an array, if they matter.
But what about web pages where the HTML is dynamically created using events/AJAX? For example suppose the HTML for the web page is just a brief AJAX-based or OnLoad script that builds the visible web page? Then parsing alone won't work.
I guess what I need is a way from within PHP, to open and render the http response (ie the HTML we get at first) via some javascript engine or browser, and once it 'stabilises', capture the HTML (or static DOM?) that's now present, which will be what the user's actually seeing.
Since such a webpage could continually change itself, I'd have to define "stable" (OnLoad or after X seconds?). I also don't need to capture any timer or async event states (ie "things set in motion that might cause web page updates at some future time"). I only need enough of the DOM to represent the static appearance the user could see, at that time.
What would I need to do, to achieve this programmatically in PHP?

To render page with JS you need to use some browser. PhantomJS was created for tasks like this. Here is simple script to run with Phantom:
var webPage = require('webpage');
var page = webPage.create();
var system = require('system');
var args = system.args;
if (args.length === 1) {
console.log('First argument must be page URL!');
} else {
page.open(args[1], function (status) {
window.setTimeout(function () { //Wait for scripts to run
var content = page.content;
console.log(content);
phantom.exit();
}, 500);
});
}
It returns resulting HTML to console output.
You can run it from console like this:
./phantomjs.exe render.js http://yandex.ru
Or you can use PHP to run it:
<?php
$path = dirname(__FILE__);
$html = shell_exec($path . DIRECTORY_SEPARATOR . 'phantomjs.exe render.js http://phantomjs.org/');
echo htmlspecialchars($html);
My PHP code assumes that PhantomJS executable is in the same directory as PHP script.

PHP Get Rendered Javascript Page

I'm developing application using AngularJS. Everything seems to be nice until I meet something that leads me to headache: SEO.
From many references, I found out that AJAX content crawled and indexed by Google bot or Bing bot 'is not that easy' since the crawlers don't render Javascript.
Currently I need a solution using PHP. I use PHP Slim Framework so my main file is index.php which contains function to echo the content of my index.html. My question is:
Is it possible to make a snapshot of rendered Javascript in HTML?
My strategy is:
If the request query string contains _escaped_fragment_, the application will generate a snapshot and give that snapshot as response instead of the exact file.
Any help would be appreciated. Thanks.

After plenty of times searching and researching, I finally managed to solve my problem by mixing PHP with PhantomJS (version 2.0). I use exec() function in PHP to run phantomJS and create Javascript file to take get the content of the targeted URL. Here are the snippets:
index.php
// Let's assume that you have a bin folder under your root folder directory which contains phantomjs.exe and content.js
$script = __DIR__ ."/bin/content.js";
$target = "http://www.kincir.com"; // target URL
$cmd = __DIR__."/bin/phantomjs.exe $script $target";
exec($cmd, $output);
return implode("", $output);
content.js
var webPage = require('webpage');
var system = require('system');
var page = webPage.create();
var url = system.args[1]; // This will get the second argument from $cmd, in this example, it will be the value of $target on index.php which is "http://www.kincir.com"
page.open(url, function (status) {
page.onLoadFinished = function () { // Make sure to return the content of the page once the page is finish loaded
var content = page.content;
console.log(content);
phantom.exit();
};
});

I recently published a project that gives PHP access to a browser. Get it here: https://github.com/merlinthemagic/MTS. It also relies on PhantomJS.
After downloading and setup you would simply use the following code:
$myUrl = "http://www.example.com";
$windowObj = \MTS\Factories::getDevices()->getLocalHost()->getBrowser('phantomjs')->getNewWindow($myUrl);
//now you can either retrive the DOM and parse it, like this:
$domData = $windowObj->getDom();
//this project also lets you manipulate the live page. Click, fill forms, submit etc.

WP8 App misbehaving due to StreamWrite in JavaScript

I would like to save the results calculated on html page in a textfile using javascript.
<script type="text/javascript">
window.onload = function () {
var sw : StreamWriter = new StreamWriter("HTML_Results.txt");
sr.Write('xyz");
*** calculations ******
sr.Write (result);
}
</script>
by doing this, my WP8 App is misbehaving and not displaying images as usual. This app is an Image Fader (calculates FPS).
Also tried:
StreamWriter sr;
try {
sr = new StreamWriter("\HTML5\HTMLResults.txt");
sr.Write("xyz");
File.SetAttributes("HTML5\HTMLResults.txt", FileAttributes.Hidden);
} catch(IOException ex) {
console.write ("error writing"); //handling IO
}
The aim is to:
Extract calculated values of several html pages(after getting loaded
one by one) in a single text file.
A Resultant HTML that reads this
text file and displays results in a tabular format.
Is there a better way to this job or the above can be rectified and used? Appreciate help.

Perhaps I've misunderstood your code but it looks like you're trying to write Java within JavaScript scripting tags. You cannot write Java in an HTML document. As far as I know, client-side JavaScript (which given your <script> tags is I guess what you're trying to write) can't perform the kind of file I/O operations you seem to want here.
You need to use Node JS to use JavaScript for something like that and then you're talking server-side. The closest you can get on client-side is using the new localStorage feature in HTML5 (not supported by all browsers).

How to implement printing in web application

In my desktop application, I developed text-based reporting and printing through a batch file because the printing is in bulk size and some times we have to restrict printing a single copy like Fixed Deposit Receipts etc. We are using lqdsi 5235 dot-matrix printers working file. Present desktop application process as follows:
The Batchfile is like this:
Name of the Batchfile: Dosprint.bat
Type %1 > prn
For Network printing:
Name of the Batchfile: Netprint.bat
Type %1 > \\SharedComputer\SharedPrinterName
And in application I redirect the printing as follows
Public Function printFile(ByVal mFileNamePath As String) As Boolean
Shell(Application.StartupPath & "\Printer\dosprint.bat " & mFileNamePath, AppWinStyle.Hide)
Return True
End Function
Printing value is very high and thousands of papers to be print some times. It is very safe and I can control No.of copies and everything like Fixed Deposit Receipts etc.
Help me if there any way to implement the same process in web application.

if you mean how to run your batch file from a web application, you can do something like this:
System.Diagnostics.Process.Start(file.FullName) //where file is a FileInfo class
As long as the file is reachable by your web application (for instance located inside the bin folder) and
The account on which your app is running has sufficient permissions to execute the file.
UPDATE:
The proper way to handle printing scenarios is to create a page that renders the content in a simple and easy way to be printed.
For example, using a simple table for tabulated data, use white color for most of the presentation to avoid spending print cartridges unnecesarily, pictures that matches your needs in specific sizes like A4 or letter, etc. Then you can call this function in the body tag:
<body onload="window.print();">
<!--content specially designed for proper printing -->
</body>

Thanks to Mr. SachinKumar K, for his article FILE HANDLING AT CLIENT SIDE USING JAVASCRIPT in WWW.CODEPROJECT.COM. In this article he mentioned that the website has to be added to the trusted site list so that the ActiveX object can be created and run.
To the dos print, I followed the below system to get dos based printing from my website on client systems dot-matrix printer.
Configuration on client System: (Internet Explorer/browser configuration)
Open Internet Explorer --> Tools --> Internet Options -->Security (tab).
add the SERVER URL in Trusted Sites
Note: uncheck "Required server verification (https:) for all sites in this zone (if your website is not HTTPS) to allow addition the site.
Next enable ActiveX controls and Plug-ins using Custom Level Tab on the same page
//create a batch file ex: printme.bat. And type the following command in the batch file.
Type %1 > prn
//The Batch file contain only one command line as above. You may change prn keyword to LPT1 or shared printer like \system_name\printer
//If required, Grant IIS_IUSRS, IUSR permission to the folder contains the printme.bat file to access by the browser.
Web Page Tags definition and javascript implementation:
// use PRE tag. It stores raw data (ASCII) in un-formatted manner
<pre id="predata" runat="server" style="display:none;"></pre>
<asp:Button Text="Print Report" runat="server" ID="btnprint" Width="101px" CssClass="buttonstyle" BackColor="DarkSlateGray" ForeColor="Aqua" OnClientClick="dosprint()" />
<%-- JAVA SCRIPT FOR PRINTING --%>
<script>
function dosprint () {
var fso, tempfile,mdata;
var fname = { key: 'value' };
fso = new ActiveXObject("Scripting.FileSystemObject");
function CreateTempFile(fname) {
var tfolder, tfile, tname, fname, TemporaryFolder = 2;
tfolder = fso.GetSpecialFolder(TemporaryFolder);
tname = fso.GetTempName();
fname.key = tfolder + '\\' + tname;
tfile = tfolder.CreateTextFile(tname);
return (tfile);
}
tempfile = CreateTempFile(fname);
mdata = document.getElementById('<%= predata.ClientID %>').innerText;
tempfile.writeline(mdata);
tempfile.close();
objShell = new ActiveXObject("WScript.Shell");
comspec = objShell.ExpandEnvironmentStrings("%comspec%");
objExec = objShell.Exec('c:\\temp\\printme.bat ' + fname.key);
// give double back slash to get a back slash in path
}
</script>
In the above code the PRINTME.BAT batch file is exist on client systems c:\temp directory.
The above system is worked for me.
Thanks everybody and happy coding.

Inject local .js file into a webpage?

I'd like to inject a couple of local .js files into a webpage. I just mean client side, as in within my browser, I don't need anybody else accessing the page to be able to see it. I just need to take a .js file, and then make it so it's as if that file had been included in the page's html via a <script> tag all along.
It's okay if it takes a second after the page has loaded for the stuff in the local files to be available.
It's okay if I have to be at the computer to do this "by hand" with a console or something.
I've been trying to do this for two days, I've tried Greasemonkey, I've tried manually loading files using a JavaScript console. It amazes me that there isn't (apparently) an established way to do this, it seems like such a simple thing to want to do. I guess simple isn't the same thing as common, though.
If it helps, the reason why I want to do this is to run a chatbot on a JS-based chat client. Some of the bot's code is mixed into the pre-existing chat code -- for that, I have Fiddler intercepting requests to .../chat.js and replacing it with a local file. But I have two .js files which are "independant" of anything on the page itself. There aren't any .js files requested by the page that I can substitute them for, so I can't use Fiddler.

Since your already using a fiddler script, you can do something like this in the OnBeforeResponse(oSession: Session) function
if ( oSession.oResponse.headers.ExistsAndContains("Content-Type", "html") &&
oSession.hostname.Contains("MY.TargetSite.com") ) {
oSession.oResponse.headers.Add("DEBUG1_WE_EDITED_THIS", "HERE");
// Remove any compression or chunking
oSession.utilDecodeResponse();
var oBody = System.Text.Encoding.UTF8.GetString(oSession.responseBodyBytes);
// Find the end of the HEAD script, so you can inject script block there.
var oRegEx = oRegEx = /(<\/head>)/gi
// replace the head-close tag with new-script + head-close
oBody = oBody.replace(oRegEx, "<script type='text/javascript'>console.log('We injected it');</script></head>");
// Set the response body to the changed body string
oSession.utilSetResponseBody(oBody);
}
Working example for www.html5rocks.com :
if ( oSession.oResponse.headers.ExistsAndContains("Content-Type", "html") &&
oSession.hostname.Contains("html5rocks") ) { //goto html5rocks.com
oSession.oResponse.headers.Add("DEBUG1_WE_EDITED_THIS", "HERE");
oSession.utilDecodeResponse();
var oBody = System.Text.Encoding.UTF8.GetString(oSession.responseBodyBytes);
var oRegEx = oRegEx = /(<\/head>)/gi
oBody = oBody.replace(oRegEx, "<script type='text/javascript'>alert('We injected it')</script></head>");
oSession.utilSetResponseBody(oBody);
}
Note, you have to turn streaming off in fiddler : http://www.fiddler2.com/fiddler/help/streaming.asp and I assume you would need to decode HTTPS : http://www.fiddler2.com/fiddler/help/httpsdecryption.asp
I have been using fiddler script less and less, in favor of fiddler .Net Extensions - http://fiddler2.com/fiddler/dev/IFiddlerExtension.asp

If you are using Chrome then check out dotjs.
It will do exactly what you want!

How about just using jquery's jQuery.getScript() method?
http://api.jquery.com/jQuery.getScript/

save the normal html pages to the file system, add the js files manually by hand, and then use fiddler to intercept those calls so you get your version of the html file