I have a series of large html pages in development, each uses a common structure: header / content / sidebar.
Header and sidebar sections have code common to all pages. Only content changes.
During the development phase, I want to be able to make changes to the header and sidebar once and see the results replicated on all pages without having to edit each page separately.
I've done a bit of googling, and the simplest solution seems to be to create 2 separate text files with the code for the header and sidebar sections, and then inject this at the appropriate points in the HTML pages I'm editing.
I know this is not recommended for a live project, as it would dramatically increase the load times, but for dev work it would save me a lot of time.
Would you agree? If so, anybody have any idea what the simplest way to do this is?
Thanks
You would be better to do this with some sort of server-side technology, like PHP. This would make the impact on loading times negligible as the pages would be concatenated before they were sent to the browser.
You would accomplish it by doing something like this:
<?php require('header.php'); ?>
<!-- Main content goes here -->
<?php require('sidebar.php'); ?>
And then have the files header.php and sidebar.php in the same directory as the main files.
Use some sort of serverside templating languate that would allow for you to include files within each other.
If, despite the other answers, you insist on doing this from JavaScript, the following is a module I use :
(function xload (slctr) { //================================================== xload ===
function xloader (src, dst) {
if (arguments.length == 1) {
dst = src;
src = dst.getAttribute ('data-source') || '';
}
var req;
try {
src = src.match (/^([^]*?)(!)?(?:\[\[([^]*?)\]\])?$/);
(req = new XMLHttpRequest ()).open ('GET', src[1], !src[2]);
req.onreadystatechange = function (ev) {
if (this.readyState === 4) {
if (typeof dst == 'function')
dst (req);
else {
dst[dst.tagName == 'TEXTAREA' ? 'value' : 'innerHTML'] = this.responseText;
[].forEach.call (dst.getElementsByTagName ('SCRIPT'), function (s, i) {
var script = document.createElement ('script');
script.innerHTML = s.parentNode.removeChild (s).innerHTML;
document.body.appendChild (script);
})
}
}
};
src[3] && req.overrideMimeType &&
req.overrideMimeType (src[3]);
req.send (null);
} catch (err) { console.log ('xloader error : ', err); }
}
[].forEach.call (document.querySelectorAll (slctr), function (el) { xloader (el); });
}) ('[data-source]'); //------------------------------------------------------ xload ---
Any element, a div for example with a data-source attribute is processed. The data-source specifies the url of the file to be included. When he Ajax request completes, the entire contents of the div are replaced with the text fetched. The data-sourc url may optionally be followed by a ! indicating synchronous load and then by a Mime type enclosed in [[ and ]]
Any scripts in the loaded text are extracted and injected into the document body.
Error conditions are reported on the console.
The module is entirely standalone and processes all elements containing the data-source attribute. It should, of course be loaded after the HTML text of the page.
Related
I have an HTML page where several JavaScript, CSS and images files are referenced. These references are dynamically injected and user can manually copy the HTML page and the support files to another machine.
If some JS or CSS are missing, the browser complains in the console. For example:
Error GET file:///E:/SSC_Temp/html_005/temp/Support/jquery.js
I need somehow these errors reported back to me on the inline JavaScript of the HTML page so I can ask user to first verify that support files are copied correctly.
There's the window.onerror event which just inform me that there's a JS error on the page such as an Unexpected Syntax error, but this doesn't fire in the event of a 404 Not Found error. I want to check for this condition in case of any resource type, including CSS, JS, and images.
I do not like to use jQuery AJAX to verify that file physically exists - the I/O overhead is expensive for every page load.
The error report has to contain the name of the file missing so I can check if the file is core or optional.
Any Ideas?
To capture all error events on the page, you can use addEventListener with the useCapture argument set to true. The reason window.onerror will not do this is because it uses the bubble event phase, and the error events you want to capture do not bubble.
If you add the following script to your HTML before you load any external content, you should be able to capture all the error events, even when loading offline.
<script type="text/javascript">
window.addEventListener('error', function(e) {
console.log(e);
}, true);
</script>
You can access the element that caused the error through e.target. For example, if you want to know what file did not load on an img tag, you can use e.target.src to get the URL that failed to load.
NOTE: This technically will not detect the error code, it detects if the image failed to load, as it technically behaves the same regardless of the status code. Depending on your setup this would probably be enough, but for example if a 404 is returned with a valid image it will not trigger an error event.
you can use the onload and onerror attributes to detect the error
for example upon loading the following html it gives alert error1 and error2 you can call your own function e.g onerror(logError(this);) and record them in an Array and once the page is fully loaded post is with single Ajax call.
<html>
<head>
<script src="file:///SSC_Temp/html_005/temp/Support/jquery.js" onerror="alert('error1');" onload="alert('load');" type="text/javascript" ></script>
</head>
<body>
<script src="file:///SSC_Temp/html_005/temp/Support/jquery.js" onerror="alert('error2');" onload="alert('load');" type="text/javascript" ></script>
</body>
</html>
I've put together the code below in pure JavaScript, tested, and it works.
All the source code (html, css, and Javascript) + images and example font is here: on github.
The first code block is an object with methods for specific file extensions: html and css.
The second is explained below, but here is a short description.
It does the following:
the function check_file takes 2 arguments: a string path and a callback function.
gets the contents of given path
gets the file extension (ext) of the given path
calls the srcFrom [ext] object method that returns an array of relative paths that was referenced in the string context by src, href, etc.
makes a synchronous call to each of these paths in the paths array
halts on error, and returns the HTTP error message and the path that had a problem, so you can use it for other issues as well, like 403 (forbidden), etc.
For convenience, it resolves to relative path names and does not care about which protocol is used (http or https, either is fine).
It also cleans up the DOM after parsing the CSS.
var srcFrom = // object
{
html:function(str)
{
var prs = new DOMParser();
var obj = prs.parseFromString(str, 'text/html');
var rsl = [], nds;
['data', 'href', 'src'].forEach(function(atr)
{
nds = [].slice.call(obj.querySelectorAll('['+atr+']'));
nds.forEach(function(nde)
{ rsl[rsl.length] = nde.getAttribute(atr); });
});
return rsl;
},
css:function(str)
{
var css = document.createElement('style');
var rsl = [], nds, tmp;
css.id = 'cssTest';
css.innerHTML = str;
document.head.appendChild(css);
css = [].slice.call(document.styleSheets);
for (var idx in css)
{
if (css[idx].ownerNode.id == 'cssTest')
{
[].slice.call(css[idx].cssRules).forEach(function(ssn)
{
['src', 'backgroundImage'].forEach(function(pty)
{
if (ssn.style[pty].length > 0)
{
tmp = ssn.style[pty].slice(4, -1);
tmp = tmp.split(window.location.pathname).join('');
tmp = tmp.split(window.location.origin).join('');
tmp = ((tmp[0] == '/') ? tmp.substr(1) : tmp);
rsl[rsl.length] = tmp;
}
});
});
break;
}
}
css = document.getElementById('cssTest');
css.parentNode.removeChild(css);
return rsl;
}
};
And here is the function that gets the file contents and calls the above object method according to the file extension:
function check_file(url, cbf)
{
var xhr = new XMLHttpRequest();
var uri = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.onload = function()
{
var ext = url.split('.').pop();
var lst = srcFrom[ext](this.response);
var rsl = [null, null], nds;
var Break = {};
try
{
lst.forEach(function(tgt)
{
uri.open('GET', tgt, false);
uri.send(null);
if (uri.statusText != 'OK')
{
rsl = [uri.statusText, tgt];
throw Break;
}
});
}
catch(e){}
cbf(rsl[0], rsl[1]);
};
xhr.send(null);
}
To use it, simply call it like this:
var uri = 'htm/stuff.html'; // html example
check_file(uri, function(err, pth)
{
if (err)
{ document.write('Aw Snap! "'+pth+'" is missing !'); }
});
Please feel free to comment and edit as you wish, i did this is a hurry, so it may not be so pretty :)
#alexander-omara gave the solution.
You can even add it in many files but the window handler can/should be added once.
I use the singleton pattern to achieve this:
some_global_object = {
error: (function(){
var activate = false;
return function(enable){
if(!activate){
activate = true;
window.addEventListener('error', function(e){
// maybe extra code here...
// if(e.target.custom_property)
// ...
}, true);
}
return activate;
};
}());
Now, from any context call it as many times you want as the handler is attached only once:
some_global_object.error();
I am trying to get the HTML (ie what you see initially when the page completes loading) for some web-page URI. Stripping out all error checking and assuming static HTML, it's a single line of code:
function GetDisplayedHTML($uri) {
return file_get_contents($uri);
}
This works fine for static HTML, and is easy to extend by simple parsing, if the page has static file dependencies/references. So tags like <script src="XXX">, <a href="XXX">, <img src="XXX">, and CSS, can also be detected and the dependencies returned in an array, if they matter.
But what about web pages where the HTML is dynamically created using events/AJAX? For example suppose the HTML for the web page is just a brief AJAX-based or OnLoad script that builds the visible web page? Then parsing alone won't work.
I guess what I need is a way from within PHP, to open and render the http response (ie the HTML we get at first) via some javascript engine or browser, and once it 'stabilises', capture the HTML (or static DOM?) that's now present, which will be what the user's actually seeing.
Since such a webpage could continually change itself, I'd have to define "stable" (OnLoad or after X seconds?). I also don't need to capture any timer or async event states (ie "things set in motion that might cause web page updates at some future time"). I only need enough of the DOM to represent the static appearance the user could see, at that time.
What would I need to do, to achieve this programmatically in PHP?
To render page with JS you need to use some browser. PhantomJS was created for tasks like this. Here is simple script to run with Phantom:
var webPage = require('webpage');
var page = webPage.create();
var system = require('system');
var args = system.args;
if (args.length === 1) {
console.log('First argument must be page URL!');
} else {
page.open(args[1], function (status) {
window.setTimeout(function () { //Wait for scripts to run
var content = page.content;
console.log(content);
phantom.exit();
}, 500);
});
}
It returns resulting HTML to console output.
You can run it from console like this:
./phantomjs.exe render.js http://yandex.ru
Or you can use PHP to run it:
<?php
$path = dirname(__FILE__);
$html = shell_exec($path . DIRECTORY_SEPARATOR . 'phantomjs.exe render.js http://phantomjs.org/');
echo htmlspecialchars($html);
My PHP code assumes that PhantomJS executable is in the same directory as PHP script.
How can we remove this script injector system and clear functions from memory?
Briefing) Recently the malfeasants at Bigcommerce created an analytics injector (JS) under guise of "monitoring" that is locked in a global variable. They have pushed it to all their 50,000 front facing stores without consent from any OP's. This puts in 2 JS libraries and sets up (plain code) triggers for them to track customer, behavior, and store plans throwing data to their shared 3rd party analytics bay. The issue is that although they run the code, they do not own rights to put in 3rd party libraries like this across thousands of domains out of their realm. Does anyone have ideas on how we can kill this + remove from memory? Is this even legal for them to do?
1) The injector is found in the shared global %%GLOBAL_AdditionalScriptTags%% in the HTMLhead.html panel, which means it non-accessible. The AdditionalScriptTags is also dynamic, meaning it loads different JS helpers based on what page is being requested. Removing the variable is a no-go for that reason.
2) The injector uses various DSL variables PHP side to build out its settings. Here is what it looks like in <head> as I browse logged into our store as a customer. This is putting 2 lines for 2 separate libraries which I will define below (note certain tokens hidden as 1234)
(function(){
window.analytics||(window.analytics=[]),window.analytics.methods=["debug","identify","track","trackLink","trackForm","trackClick","trackSubmit","page","pageview","ab","alias","ready","group","on","once","off","initialize"],window.analytics.factory=function(a){return function(){var b=Array.prototype.slice.call(arguments);return b.unshift(a),window.analytics.push(b),window.analytics}};for(var i=0;i<window.analytics.methods.length;i++){var method=window.analytics.methods[i];window.analytics[method]=window.analytics.factory(method)}window.analytics.load=function(){var a=document.createElement("script");a.type="text/javascript",a.async=!0,a.src="http://cdn2.bigcommerce.com/r6cb05f0157ab6c6a38c325c12cfb4eb064cc3d6f/app/assets/js/analytics.min.js";var b=document.getElementsByTagName("script")[0];b.parentNode.insertBefore(a,b)},window.analytics.SNIPPET_VERSION="2.0.8",window.analytics.load();
// uncomment the following line to turn analytics.js debugging on
// shows verbose events and other useful information
// analytics.debug();
var storeId = '123456',
userId = '921';
// initialize with Fornax and Segment.io
var providers = {
Fornax: {
host: 'https://analytics.bigcommerce.com',
cdn: 'http://cdn2.bigcommerce.com/r6cb05f0157ab6c6a38c325c12cfb4eb064cc3d6f/app/assets/js/fornax.min.js',
defaultEventProperties: {
storeId: storeId
}
},
'Segment.io': {
apiKey: '1sbkkbifdq'
}
};
var fornaxEnabled = false;
var segmentIOEnabled = false;
var isStorefront = true;
if (!fornaxEnabled) {
delete providers.Fornax;
}
if (!segmentIOEnabled || isStorefront) {
delete providers['Segment.io'];
}
analytics.initialize(providers);
// identify this user
analytics.identify(
userId || null,
{"name":"Test Dude","email":"test#test.com","storeHash":"123456","storeId":123456,"namespace":"bc.customers","storeCountry":"United States","experiments":{"shopping.checkout.cart_to_paid":"legacy_ui","search.storefront.backend":"mysql"},"storefront_session_id":"6b546880d5c34eec4194b5825145ad60d312bdfe"}
);
})();
3) The output libraries are found as 2 references in the <head> and as you see if you own/demo a BC store, are rather un-touchable:
<script type="text/javascript" async="" src="http://cdn2.bigcommerce.com/r6cb05f0157ab6c6a38c325c12cfb4eb064cc3d6f/app/assets/js/fornax.min.js"></script>
<script type="text/javascript" async="" src="http://cdn2.bigcommerce.com/r6cb05f0157ab6c6a38c325c12cfb4eb064cc3d6f/app/assets/js/analytics.min.js"></script>
How can we break the injector and these trackers and prevent them from loading? Is there a way to remove their functions from memory? Speaking on behalf of many thousands of OP's and segment.io here, we are all at our wits end with this.
I've been hacking away at this too and I found something that works well to disable most/all of it.
Before this line:
%%GLOBAL_AdditionalScriptTags%%
Use this code:
<script type="text/javascript">
window.bcanalytics = function () {};
</script>
So you will end up with something like this:
%%GLOBAL_AdditionalScriptTags%%
<script type="text/javascript">
window.bcanalytics = function () {};
</script>
The <script> tags from part 3 of your question will still load as those are always PREpended before the first non-commented out <script> tag, but most, if not all, the analytics functionality will break, including external calls, and even fornax.js won't load. Hope this helps.
Per the question I linked, for you case to at least remove the scripts from Step 3 this is what you should do :
var xhr = new XMLHttpRequest,
content,
doc,
scripts;
xhr.open( "GET", document.URL, false );
xhr.send(null);
content = xhr.responseText;
doc = document.implementation.createHTMLDocument(""+(document.title || ""));
doc.open();
doc.write(content);
doc.close();
scripts = doc.getElementsByTagName("script");
//Modify scripts as you please
[].forEach.call( scripts, function( script ) {
if(script.getAttribute("src") == "http://cdn2.bigcommerce.com/r6cb05f0157ab6c6a38c325c12cfb4eb064cc3d6f/app/assets/js/fornax.min.js"
|| script.getAttribute("src") == "http://cdn2.bigcommerce.com/r6cb05f0157ab6c6a38c325c12cfb4eb064cc3d6f/app/assets/js/analytics.min.js") {
script.removeAttribute("src");
}
});
//Doing this will activate all the modified scripts and the "old page" will be gone as the document is replaced
document.replaceChild( document.importNode(doc.documentElement, true), document.documentElement);
You must make sure that this is the first thing to run, otherwise the other scripts can and will be executed.
Let's say I have a web page (/index.html) that contains the following
<li>
<div>item1</div>
details
</li>
and I would like to have some javascript on /index.html to load that
/details/item1.html page and extract some information from that page.
The page /details/item1.html might contain things like
<div id="some_id">
picture
map
</div>
My task is to write a greasemonkey script, so changing anything serverside is not an option.
To summarize, javascript is running on /index.html and I would
like to have the javascript code to add some information on /index.html
extracted from both /index.html and /details/item1.html.
My question is how to fetch information from /details/item1.html.
I currently have written code to extract the link (e.g. /details/item1.html)
and pass this on to a method that should extract the wanted information (at first
just .innerHTML from the some_id div is ok, I can process futher later).
The following is my current attempt, but it does not work. Any suggestions?
function get_information(link)
{
var obj = document.createElement('object');
obj.data = link;
document.getElementsByTagName('body')[0].appendChild(obj)
var some_id = document.getElementById('some_id');
if (! some_id) {
alert("some_id == NULL");
return "";
}
return some_id.innerHTML;
}
First:
function get_information(link, callback) {
var xhr = new XMLHttpRequest();
xhr.open("GET", link, true);
xhr.onreadystatechange = function() {
if (xhr.readyState === 4) {
callback(xhr.responseText);
}
};
xhr.send(null);
}
then
get_information("/details/item1.html", function(text) {
var div = document.createElement("div");
div.innerHTML = text;
// Do something with the div here, like inserting it into the page
});
I have not tested any of this - off the top of my head. YMMV
As only one page exists in the client (browser) at a time and all other (virtual/possible) pages are on the server, how will you get information from another page using JavaScript as you will have to interact with the server at some point to retrieve the second page?
If you can, integrate some AJAX-request to load the second page (and parse it), but if that's not an option, I'd say you'll have to load all pages that you want to extract information from at the same time, hide the bits you don't want to show (in hidden DIVs?) and then get your index (or whoever controls the view) to retrieve the needed information from there ... even though that sounds pretty creepy ;)
You can load the page in a hidden iframe and use normal DOM manipulation to extract the results, or get the text of the page via AJAX, grab the part between <body...>...</body>ยจ and temporarily inject it into a div. (The second might fail for some exotic elements like ins.) I would expect Greasemonkey to have more powerful functions than normal Javascript for stuff like that, though - it might be worth to thumb through the documentation.
A slew of pages I've written for one of my web projects share some 144 identical lines of code, reproduced in each file. If I update one of those lines, I have to go back through ALL of the pages that share the code and update for each page. Is there a straightforward way to include HTML from a separate file?
And for bonus points, since so many pages use this code, it would be nice not to have to reload it for each page. Is there an easy way to store it in the browser's cache or only load the "content" section of the pages while leaving the rest of the page static?
Fountains of Thanks for any wisdom on this.
Mike
To include HTML from a separate file, use SSI (Server-Side Includes). This requires SSI support to be installed on the server, however.
You would write something like this in your files:
<!--#include file="included.html" -->
and that would include the file included.html when the page is accessed.
To load only the content of each page, use the XMLHTTPRequest object from JavaScript:
function LoadContent(url)
{
if (typeof(XMLHttpRequest) == "undefined")
{
try
{
xmlhttp = new ActiveXObject("Msxml2.XMLHTTP");
}
catch(e)
{
try
{
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
catch(e)
{
// fallback for browsers without XMLHttpRequest
window.location.href = "no-ajax.php?url="+escape(url);
return;
}
}
}
else
{
xmlhttp = new XMLHttpRequest();
}
xmlhttp.open("GET", url, false); // this request will be synchronous (will pause the script)
xmlhttp.send();
if(xmlhttp.status > 399) // 1xx, 2xx and 3xx status codes are not errors
{
// put error handling here
}
document.getElementById("content").innerHTML = xmlhttp.responseText;
}
If we're assuming that you're talking straight html pages, with no server code (asp.net, php, or server side include ability), then in order to do both the including and the caching, you're going to need to use an iframe.
Each of your pages that duplicate the 144 lines of content should replace it with an iframe like so:
<iframe src="pagewithcontent.html"></iframe>
pagewithcontent.html would obviously be where you move the content to. The browser will cache the page appropriately, so that each parent page will simply get the shared content without making another request.
There's an article here that goes into great depth about html includes, and some javascript methods of doing it. I would strongly recommend against the javascript methods.
My answer reflects the assumption that you can't do anything on the server side. However, by far the best solution is to do so if you can.