Getting dynamic content from a website through javascript - javascript

I am creating a firefox plugin and using javascript. I am using XMLHttpRequest to get dynamic content from a remote website and storing it into a file to parse it later. But in responseText i am not receiving any dynamic content. After storing information that i got is only the html page not having dynamic content.
code:
var res="";
var req = new XMLHttpRequest();
req.onload = function(){
res=this.responseText;
}
req.open("GET","www.ebay.com", true);
req.send();

It's can't be done due to same-origin policy, so you can only request content within the origin domain.
Here are some links may be helpful to you:
Getting CORS Working
Cross-domain Ajax with Cross-Origin Resource Sharing
Edit:
Since what you want to get from a website is generated by Ajax, so it's impossible to get the dynamic content. the dynamic content present on the website is after the browser loaded the html page and so the Javascript Event can be triggered to get the dynamic content through Ajax. You can get the html page, but you didn't get the Javascript files. therefor, Can't get the dynmic content.
sorry for my bad English

You can only fetch remote content under your own domain. This is a security issue. You can use JSONP, But for ebay I don't know if there are any json contents available. for more information have a look at this: http://www.devproconnections.com/article/aspnet2/ajax-cross-domain-142169
In the case you are using Javascript with a Chrome extension or a Firefox add-on, then you must set permissions in your manifest file to have access to those sites you want.

Related

How to get the page source of another web page using HTML/JavaScript?

I am trying to write HTML code that hits a URL and fetches me it's page source i.e. the whole content which we see when we right click on the page and select the 'View Page Source' option.
I then want to process this content to extract some relevant values.
I have tried a few options on a W3schools TryIt Editor but nothing worked. I have tried for other URLs also, but no luck.
Can someone please tell if this is possible using HTML and JavaScript and if yes, how?
You can make an HTTP request using the Fetch API pretty simply:
const res = await fetch('https://example.com');
const html = await res.text();
However, this is only going to work for URLs that allow you to fetch their sources cross-origin. For security reasons, this isn't common. (If it were, most any other website could steal content from other sites you're logged into, such as your webmail or your bank!)
The only way around this is to proxy that data server-side where the cross-origin problem doesn't exist.
What it is the main purposes to fetch a different page into your page?
On another hand, did you try to use iframes?
https://www.w3schools.com/tags/tag_iframe.asp
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe
Note: In the end, you could fetch the page but you won't have control over js.

JavaScript - cross origin request blocked - non-server solution

I'm a desktop developer that is trying to learn some web basics on the side. I've previously put together an asp.net mvc website that worked more or less okay and am currently working on a simpler, html/css/js only website.
A number of the pages on the website will contain images, with a number of pieces of data accompanying them, so I thought I'd put together a JSON with all of the data, including the links to the images and generate the image list on page load. The problem that I ran into is the JavaScript cross origin request when trying to fetch the JSON file.
I've looked around at solutions and most of them recommend spinning up a server - either asp.net or node.js to fetch the JSON from. Couple of questions:
If I can write HTML that references image files, why can I not fetch a json from javascript? Is there a fundamental piece of understanding that I'm missing here?
Is there any other way of using a JSON without spinning up a web server? Should I try embedding it into the HTML? Is that a bad idea?
Any other pointers/links to resources with relevant info :)
// My JavaScript:
<script>
$(document).ready(function(){
buildGallery('test.json', '#gallery');
});
// Builds a collection of thumbnails from the json specified inside of specified div
function buildGallery(jsonUrl, galleryDiv){
$.getJSON(jsonUrl, function(data){
// Ensure the data is in correct format
if (typeof(data) !== 'object'){
return;
}
// Build the gallery
$.each(data['images'], function(key, image){
var thumbnail = '<img src="' + image['url'] + '"/>'
$(galleryDiv).append(thumbnail);
});
});
}
</script>
This is based of: https://api.jquery.com/jQuery.getJSON/
Thanks heaps!
There are a couple issues with what you are trying to accomplish with the provided code.
First, you are trying to make an Ajax request to a resource that is not hosted on an http server. Ajax is a wrapper for XMLHttpRequest which was designed for fetching resources using the http protocol. However, it can support other protocols such as file, and ftp.
Second, CORS is not controlled by the browser, it's controlled by the http server. Cross domain origin requests can work, but only if the resource you are requesting responds with an http header that allows your domain to access it. Since the resource you are requesting has nothing to do with http, it will probably throw an error.
So why do images work using the file:// scheme? The <img/> tag supports loading resources using any scheme your browser cares to support. It turns out most browsers support it.
So I can't get json into my app without an http server!#? Yes and no. No because you usually cannot request a resource not served through an http server using XMLHttpRequest. However, you can still request resources through other means.
I recommend using the File API for reading files from the users filesystem.

Is possible to read script tag src URL content using JavaScript/jQuery

Please guide me. Is possible to read script tag src URL content using JavaScript/jQuery. Please don't suggest JSON/Ajax/iframe options.
I have tried like
<script src="http://www.test.com" id="test">
document.getElementById('test').src
but it give only src URL.
My Exact requirements is,
I have two application, each in different web server with www.siteA.com and www.siteB.com.
In Server2, I have cross origin and iframe restriction to access the application. Example: If a request is coming from server1(siteA) to server2(siteB) via Ajax or iframe, it's restricted to access the siteB page content. But If I load siteB in script tag it's loaded, but I'm not able to get the content.
So I mentioned above, don't suggest iframe and Ajax. Help me to achieve this.
The best way really is AJAX, but it does sound like CORS is an issue for you. The only other option I can think of would be to open the page in a new browser window, although I don't know what kind of user experience implications that will have on your application.
You can open a new browser window by:
function showContent(){
var url = document.getElementById("test").src;
window.open(url, "_blank");
}
Although if you take this route, you shouldn't put the url on a script tag, because the browser will download it once (via the script tag), and then a second time on the window.open.

Load a remote XML into a web page and access its content via JavaScript

I'd like to know if it's possible to load a remote XML file through the <script> tag, and access the content using JavaScript.
As the XML is a result of an external website (I'm using TheTVDb API), I can't load it using AJAX.
I'm looking for something like the following, if it's possible (JQuery-like syntax):
<script id="xmlload" type="text/xml" src="...">
<script type="text/javascript">
var xmlcontent = $('#xmlload').content();
// parse xmlcontent
</script>
I don't think that this is possible - you will need to use XmlHttpRequest (AJAX) to use a HTTP-based API. However, it might still be possible to actually do cross-site requests if the TheTVDb server allows this - see HTTP access control on MDN, which describes the relevant W3C specification (Cross-Origin Resource Sharing).
So if you haven't done so yet, I'd recommend you just try if making an AJAX request works. Otherwise, it might be a good idea to ask the TheTVDb folks if they are so kind to implement the mentioned spec.

load cross domain xml by Javascript

Hi
Is it possible to load an XML file from a domain that differs from scripts domain with pure javascript and without using a php/asp/jsp/... script as proxy?
Something like xmlHttpRequest but with ability to manage cross domain requests.
Thanks
You can use something called JSONP. I know the name sucks, because it's not really related to JSON. But this requires you have control over the other domain. You need to wrap your XML inside a function call, or assign it to a javascript variable:
func('<xml></xml>');
or
var myxml = '<xml></xml>';
So if your other domain returns one of these two formats, you can use the <script src="http://otherdomain/yourjsonp"></script> syntax in your html to load that data in JavaScript. It's a little hacky but a lot of people use it.
It is possible with yql! (Yahoo did it for you)
Go to this site and simple at the "select from url='xxx' " replace the xxx with your xml url. Use the url created at the text box below and do a simple xmlrequest. You won't have any cross-domain prolems

Categories