I'd like to create a bookmarklet to take the contents of a html page and send it to an external url for processing.
Usually it would be enough to just send document.title to the server and CURL it up serverside, but for various reasons this is not an option here. I tried:
javascript:
(
function()
{
var htmlstuff=document.documentElement.innerHTML;
window.open
(
'http://127.0.0.1/grabhtml.php?url='+escape(document.location)+'&title='+escape(document.title)+'&html='+escape(htmlstuff)+'&lm='+escape(document.lastModified)
,'InfoDialog','menubar=no,toolbar=no,location=no,directories=no,status=no,scrollbars=yes,resizable=yes,dependent=no,width=400,height=480,left=50,top=50'
);
}
)
();
grabhtml.php is just
<? file_put_contents('result.html',$_REQUEST['html']); ?>
As expected Apache doesn't like such long requests:
Request-URI Too Large
The requested URL's length exceeds the capacity limit for this server.
Therefore I thought about sending the document.documentElement.innerHTML via POST instead of GET.
Firefox-WebmasterTools has an option to show the "View Generated Source" rather than the normal "View Source".
I remember last year I've read an article about how an Instapaper-like-service did exactly the same.
I've searched for days for this article, or for bookmarklet examples which POST the "Generated Source" to my form.
My Javascript skills are very basic, but I'm a quick learner. A kick into the right direction would be greatly appreciated.
You can only use POST via AJAX, so it is necessary that your JS script is running on the same domain as grabhtml.php
If it is, you can simply use jQuery for this, and it will look like:
$.post('grabhtml.php', {
url: document.location,
title: document.title,
html: htmlstuff
}, function(response) {
alert('Successfully posted.');
});
If don't, you could try to embed your script into iframe (running at the same domain as php script), send title, body, etc. from the parent frame to this iframe (via window.postMessage) and issue POST request described above ommitting cross-domain restrictions.
You can read more about window.postMessage here:
http://viget.com/extend/using-javascript-postmessage-to-talk-to-iframes
Note: I'm not sure about window.postMessage maximum message size
Related
I've got a website that uses JQuery to grab information from an API located at: https://api.lootbox.eu/. The website I'm making has a cert that's been created and installed with Let's Encrypts tools. (I followed a tutorial on DigitalOcean to set it up)
Now, when I click a button to make the API call and update the website contents, Google Chrome then deems the cert "Not Secure" (shown in the address bar).
Here's my code.
//when the page has fully loaded
$(document).ready(function(){
function parseResponse(data)
{
//parsing JSON...
//lets avoid the "data.data"
response = data.data;
//set the user's general profile details
$("#user_avatar").attr("src", response.avatar);
$("#comp_rank_img").attr("src", response.competitive.rank_img);
$("#comp_rank").html(response.competitive.rank);
$("#qp_level").html(response.level);
$("#qp_playtime").html(response.playtime.quick);
$("#comp_playtime").html(response.playtime.competitive);
$("#qp_wins").html(response.games.quick.wins);
$("#comp_total_games").html(response.games.competitive.played);
$("#comp_wins").html(response.games.competitive.wins);
$("#comp_losses").html(response.games.competitive.lost);
}
//goes to the Overwatch API and grabs the JSON data for processing.
function processAjax()
{
$.ajax({
url: "https://api.lootbox.eu/pc/us/JuicyBidet-1292/profile"
}).then( function(response){
parseResponse(response);
});
}
//set up the button event listener
$("#submit").click(function(e){
//prevent the button from reloading the page
e.preventDefault();
//run the ajax grabber/processor
processAjax();
});
});
(I need to learn how to format code properly in SO questions...).
I don't get any other errors in my Chrome console (other than "favicon" 404ing - which is unrelated).
I've also tried the website in Microsoft Edge and I get the following in their Console:
SCRIPT7002: XMLHttpRequest: Network Error 0x800c0019, Security certificate required to access this resource is invalid.
I'm thinking either the problem is with my code, or now that I've checked, the API website's cert is invalid in both Chrome and Edge.
Any help would be appreciated. Thanks.
(I'm aware the code is scrappy, I'm learning)
I get this warning "This server could not prove that it is api.lootbox.eu; its security certificate expired 6 days ago"
This could happen because of one among following reasons.
1.Certificate is not issued by the third party site.
2.The third party site certificate is not updated.
3.The third party and browser connection is not secure.
The insecure error is because if a site is loaded over https the browser expects all subsequent calls made by the page will be secure, but in your case you are calling a URL http://api.lootbox.eu/pc/us/JuicyBidet-1292/profile which uses http:// and is insecure thus making your website insecure.
Solution
Use https url's everywhere in your pages.
I've wrote a script that previews web pages within an iframe, the problem is that some pages (eg facebook and twitter) won't open within iframes. Is it possible to detect if the page can't be loaded or if the iframe is empty?
Thanks in advance.
I do not think this is possible for sites that are not hosted on the same domain as yours, due to browsers same origin policy.
See if something like this would work for you
jsFiddle
alert($(".1").contents().find("body").length);
alert($(".2").contents().find("body").length);
You will need send request to PHP script to check website headers. For example:
$.getJSON('http://debute.eu/allowed.php?url=' + url, function(data) { });
Here is DEMO.
If you can use PHP script, I will send you my PHP.
I am trying to integrate with the FireShot API to given a URL, grab HTML of another web page into a div then take a screenshot of it.
Some things I will need to do after getting the HTML
grab <link> & <script> from <head>
grab <body> into <div>
But 1st, it seems when I try to do a
$.get("http://google.com", function(data) { ... });
I get a 200 in firebug colored red. I think it has to do with sites not allowing you to grab their page with JS? Then is opening a window the best I can do? But how might I control the other page with jQuery or call fsapi on that page?
UPDATE
I tried to do something like below to do something when the new window is ready, but FireBug says "Permission denied to access property 'document'"
w = window.open($url.val());
setTimeout(function() { // if I dont do this, I always get about:blank, is there a better way around this?
$(w.document).ready(function() {
console.log(w.document.body);
});
}, 1000);
I believe the cross-site security setup within Javascript is basically blocking this. You'd likely have to proxy the content through your own domain.
There are a couple other options I think for break the cross-site security constraints, but I'm not sure I'd promote them.
If the "another page" locates within the same domain of your hosting page, yes, you can. Please refer to jQuery's $().load() API.
Otherwise, you're disallowed to do so by the browser's Cross-Site Security Policy. At this moment, you can choose to use iFrame instead of DIV.
Some jQuery plugins, e.g. thickbox provides ability to load pages to appropriate container automatically.
Unless I am correct, I do not believe you can AJAX a page cross domain (e.g. from domain1.com to domain2.com). To get around this, you can have a PHP "proxy" script that does the "getting" of the page and then pass it to JS.
For example, in JS you would get() http://mydomain.com/get/?domain=http://google.com and then do what you need to do!
I have a html page on my localhost - get_description.html.
The snippet below is part of the code:
<input type="text" id="url"/>
<button id="get_description_button">Get description</button>
<iframe id="description_container" src="#"/>
When the button is clicked the src of the iframe is set to the url entered in the textbox. The pages fetched this way are very big with lots of linked files. What I am interested in the page is a block of text contained in a <div id="description"> element.
Is there a way to mitigate downloading of resources linked in the page that loads into the iframe?
I don't want to use curl because the data is only available to logged in users and the steps to take with curl to get the content is too complicated. The iframe is simple as I use this on a box which sends the right cookies to identify the request as coming from a logged in user, but the problem is that it is very wasteful to get nearly 1 MB of data to keep 1 KB of it and throw out the rest.
Edit
If the proposed method just works in Firefox it is fine, so I added Firefox tag. Also, it is possible that the answer actually is from the realm of Firefox add-on techniques, so I added that tag as well.
The problem is not that I cannot get at what I'm looking for, rather, the problem is the easy iframe method is wasteful.
I know that Firefox does allow loading only the text of a page. If you open a page and press Ctrl+U you are taken to 'view page source' window, There links behave as normal and are clickable, if you click on a link in source view, the source of the new page is loaded into the view source window, without the linked resources being downloaded, exactly what I'm trying to get. But I don't know how to access this behaviour.
Another example is the Adblock add-on. It somehow kills elements before they get loaded. With plain Javascript this is not possible. Because it only is triggered too late to intervene in good time.
The Same Origin Policy forbids any web page to access contents of any other web page in a different domain so basically you cannot do that.
However it seems that with some browsers it is allowed to access web pages content if you are trying to access it from a local web page which seems to be your case.
Safari, IE 6/7/8 are browser that allow a local web page to do so via XMLHttpRequest (source: Google Browser Security Handbook) so you may want to choose to use one of those browsers to do what you need (note that future versions of those browsers may not allow to do so anymore).
A part from this solution I only see two possibities:
If the web pages you need to fetch content from are somehow controlled by you, you can create a simpler interface to let other web pages to get the content you need (for example allowing JSONP requests).
If the web pages you need to fetch content from are not controlled by you the only solution I see is to fetch content server side logging in from the server directly (I know that you don't want to do so, but I don't see any other possibility if the previous I mentioned are not practicable)
Hope it helps.
Actually I've seen Cross Domain jQuery .load request before, here: http://james.padolsey.com/javascript/cross-domain-requests-with-jquery/
The author claims that codes like these found on that page
$('#container').load('http://google.com'); // SERIOUSLY!
$.ajax({
url: 'http://news.bbc.co.uk',
type: 'GET',
success: function(res) {
var headline = $(res.responseText).find('a.tsh').text();
alert(headline);
}
});
// Works with $.get too!
would work. (The BBC code might not work because of the recent redesign, but you get the idea)
Apparently it is using YQL wrapped into a jQuery plugin to do the trick. Now I cannot say I fully understand what he is doing there but it appears to work, and fits the bill. Once you load the data I suppose it is a simple matter of filtering out the data that you need.
If you prefer something that works at the browser level, may I suggest Mozilla's Jetpack framework for lightweight extensions. I've not yet read the documentations in its entirety but it should contain the APIs needed for this to work.
There are various ways to go about this in AJAX, I'm going to show the jQuery way for brevity as one option, though you could do this in vanilla JavaScript as well.
Instead of an <iframe> you can just use a container, let's say a <div> like this:
<div id="description_container"></div>
Then to load it:
$(function() {
$("#get_description_button").click(function() {
$("#description_container").load($("input").val() + " #description");
});
});
This uses the .load() method which takes a string in this format: .load("url selector"), then takes that element in the page and places it's content inside the container you're loading, in this case #description_container.
This is just the jQuery route, mainly to illustrate that yes, you can do what you want, but you don't have to do it exactly like this, just showing the concept is getting what you want from an AJAX request, rather than in an <iframe>.
Your description sounds like you are fetching pages from the same domain (you said that you need to be logged in and have session credentials) so have you tried to use async request via XMLHttpRequest? It might complain if the html on a page is particularly messed up but you chould still be able to get raw text via .responseText and extract what you need with a regex.
I have looked everywhere and surprisingly can't find a good solution to this! I've got the following code that is supposed to read a text file and display it's contents. But it's not reading, for some reason. Am I doing something wrong?
FTR, I can't use PHP for this. It's gotta be Javascript.
var txtFile = new XMLHttpRequest();
txtFile.open("GET", "http://www.mysite.com/todaysTrivia.txt", true);
txtFile.send(null);
txtFile.onreadystatechange = function() {
if (txtFile.readyState == 4) { // Makes sure the document is ready to parse.
alert(txtFile.responseText+" - "+txtFile.status);
//if (txtFile.status === 200) { // Makes sure it's found the file.
var doc = document.getElementById("Trivia-Widget");
if (doc) {
doc.innerHTML = txtFile.responseText ;
}
//}
}
txtFile.send(null);
}
Any good ideas what I'm doing wrong? It just keeps givimg me a zero status.
EDIT: I guess it would be a good idea to explain why I need this code. It's basically a widget that other folks can put on their own websites that grabs a line of text from my website and displays it on theirs. The problem is that it really can't be server-side since I've got zero control over everyone else's sites that use this.
If this is cross domain, you won't be able to do this with an xmlhttprequest due to the same origin policy.
This exmaple contains jQuery code.
var text;
$.get( "proxy.php", function(data) {
text = data.responseText;
});
Then in proxy.php:
<?php
header('Content-type: application/xml');
$daurl = 'http://www.mysite.com/todaysTrivia.txt';
$handle = fopen($daurl, "r");
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
echo $buffer;
}
fclose($handle);
}
Example taken from here:
http://jquery-howto.blogspot.com/2009/04/cross-domain-ajax-querying-with-jquery.html
As explained before, xmlhttp is designed for forbid cross domain requests for security issues. But nothing prevents you from doing this on your server in PHP.
Another example can be found here: http://usejquery.com/posts/9/the-jquery-cross-domain-ajax-guide
Your problem could be with the fact that you can only request XML data from the same domain via Javascript. This is the biggest issue with AJAX calls - if the text file is on another server, you can't get it via AJAX. If it's on the same server, make your request using a relative URL (no http://).
EDIT
Now that I know what you're trying to accomplish ... my recommendation would be to use an iFrame. Build the system on your server using server-side code and allow remote sites to embed an iFrame to display the output on their own sites. NetworkedBlogs uses this for displaying Facebook features on remote sites. iGoogle uses it extensively with their various Apps and Gadgets. It's a fairly tried-and-true method.
The advantage of using an iFrame is that you'll still have control over most of the content of the widget, but you can give end-users control over the styling (just have your iFrame application accept arguments via query variables to change colors, positions, and sizes).
Assuming the AJAX stuff is right (which I haven't confirmed): You say you can't use PHP for this - if you just mean you need it to use javascript asynchronously but can still use server code in some places, what about using PHP (or any server-side language) to do the actual work and return it to the page through AJAX/javascript - this would solve the problem Alex brings up.
So instead of getting from mysite.com/something.txt from javascript, get it from SomeAjaxHelper.php (or aspx or whatever).
For cross domain, you would have to use dynamic script tags to fetch data asynchronously. The todaysTrivia file would be a .js file that stores the data as JSON. Google for "dynamic script tags cross domain" if you want to use this technique.