Can I grab specific page HTML code from another webpage through Javascript? - javascript

I've read about how there are multiple methods to grabbing source code from another webpage via jQuery or using Cross-Domain Requests. What I want to try and do is make it so I grab a div that has different code each time a page is loaded and not the source code as a whole. So for example, the greater detail you see when you use 'inspect element' or a tool like firebug to dive deeper into the page code.
Would I be using one of the same methods?

Yes.
If you control BOTH domains you can add the Access_control_CORS
header to allow access of cross domain requests and use a
headless browser like phantomJS to grab a cached version of
rendered HTML page.
If you don't control both domains you will have to write a server
side proxy to grab the page and all its resources (you will have
to parse the page to get or rewrite links to images, javascripts,
stylesheets etc...) then run it through phantomJS to create a
HTML snapshot.`
source:
https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
http://phantomjs.org/
NOTE: despite my best efforts, stack overflow is absolutely convinced these links are code. Sorry for posting as code.

Related

Access cross-domain iframe elements like in Optimizely

I'd like to be able to manually select text from cross-domain iframe with javascript on the frontend.
I know it's pretty difficult to achieve such result as the use of postMessage is required if I decided to use a 100% javascript solution. I'm opened to backend solutions (either php or nodejs)
Optimizely can access any dom element from the page, I'd like to understand how they are doing it. Is it because the user needs to install a widget on his website first? If yes, how does it works exactly?
Here's an example:
Reproduced from #2 of this answer:
Let's say you want to create an experiment on http://www.mypage.com ... The Optimizely Editor loads http://www.mypage.com inside an iframe and uses window.postMessage to communicate with the page. This only works if that page already has a snippet like [<script src="//cdn.optimizely.com/js/XXXXXX.js"></script>] on it. If that's not the case, the editor will timeout while waiting for a message from the iframe'd page, and will load it again via a proxy that actually inserts the snippet onto the page. This loading process allows the editor to work with pages that a. contain an account snippet b. do not contain an account snippet, or c. sit behind a firewall (c. requires the snippet.)

Does anyone have a creative solution that allows me to hide menu items from a web page embedded in my web page?

I fear that the answer will be "Impossible due to browser security policy" but I really need to accomplish the following:
The problem I have is that the content to be embedded in my web page includes some menu items that I need to remove/hide because they trigger operations that I need to prevent. I cannot find a way to address these DOM nodes to hide them.
I have a web page and need to embed a URL from another domain into my web page. I have tried this with and also by using Ajax to fetch the URL contents and insert them into the DOM of my web page. These two methods have different results.
If I use to embed the page from the "foreign" domain I can see the "foreign" domain's content and I can address the node but all attempts to access the nodes underneath return null. There is no error message (in Firefox) but I suspect that I am getting null because the browser is enforcing the same-domain policy.
On the other hand, if I use Ajax to insert the page content into my web page I don't even see the content and in this case there is a CORS error in the Firefox debugger console.
Since I don't control the "foreign" domain I can't modify it to use the window.postMessage(); technique.
Can anyone suggest a way for me to be able to hide menu items that are in content fetched from a "foreign" domain? (Gotta be a way, gotta be a way, ...)
Thank you.

a website preview - loading a webpage using javascript Or Server-side

I want to show website preview on a link similar to facebook when a user post a link. my question has been repeated in the following link ,but I am going to ask specific information throughout my solutions. I have 2 solutions for showing webpage preview which are as follows:1. server side html process 2. client side html process.
1. server-side html process
I used System.Net.WebClient().DownloadString(url) to retrieve the web page data in server side , and I tried to extract the most important information in the page ,but in most cases, main part of the page loads using javascript , therefore I do not have access to that information.
Another solution in server-side html process is to work with webBrowser and WebDocument objects. because I didn't work with these libraries and I don't know how much the Web server performance affect by applying this objects , I only present this solution for discussion .Therefore are there any server-side html graber which fetch all html data including javascript loaded html source?
2. Client Side Html process
The simplest approach for client side is to use the iframe tag, but it has two following problems:
a. I can not access to innerHTML of the frame for the links on other domains.
b. I can not load https webpages such as drop-box and facebook in the iframe
because of "x-frame options" error.
My question is that, is there any other client-side solution to retrieve dynamic html source(loaded by javascript) from 3rd party webpages (usually https)? Or can I solve above problems with some tricks.
I guess server side approach would be most viable option. On client side you can use proxy services which allow to solve cross domain limitation, for example, crossorigin.
To generate a preview, similar to one Facebook provides, you need to get Open Graph information for target page. Libraries to process open graph data available for multiple platforms. OpenGraph-Net could be used on .NET plarform.

Is there any alternative to iframe, object or embed?

I want to include an external (external domain) html page into my page. For SEO reasons iframe, object or embed are not useful, there are just a link in my source file. PHP's include function is very helpful but it causes many problems for UI. I want contents of include function and view of iframe.
How can I do that?
Thanks.
There's no reasonable alternative to <iframe>.
Who knows if you could extract the markup from the site from the server-side and print that HTML inside a <div> in your own site, but I doubt that this could ever work, because if the whole target site does AJAX requests, WebSockets or whatever, it should be secure enough to block you from performing them from other domains than allowed ones by themselves (i.e. their official domains).
If you are adding content from an external source the it should really have 0 impact on your SEO. Needs to be your own content as far as I am aware. You could try scraping the external source and using Ajax add it to your page using $().load() or similar... Wouldn't recommend though!

How to offer a webapp to other sites. (div with javascript, iframe or..?)

I am quite new to web application development and I need to know how would I make other sites use it.
My webapp basically gets a username and returns some data from my DB. This should be visible from other websites.
My options are:
iframe. The websites owners embed an iframe and they pass the userid in the querystring. I render a webpage with the data and is shown inside the iframe.
pros: easy to do, working already.
cons: the websites wont know the data returned, and they may like to know it.
javascript & div. They paste a div and some javascript code in their websites and the div content is updated with the data retrieved by the small javascript.
pros: the webside would be able to get the data.
cons: I could mess up with their website and I don't know wow would I run the javascript code appart from being triggered by a document ready, but I wouldn't like to add jquery libraries to their sites.
There must be better ways to integrate web applications than what I'm thinking. Could someone give me some advice?
Thanks
Iframes cannot communicate with pages that are on a different domain. If you want to inject content into someone else's page and still be able to interact with that page you need to include (or append) a JavaScript tag (that points to your code) to the hosting page, then use JavaScript to write your content into the hosting page.
Context Framework contains embedded mode support, where page components can be injected to other pages via Javascript. It does depend on jQuery but it can always be used in noConflict-mode. At current release the embedded pages must be on same domain so that same-origin-policy is not violated.
In the next release, embedded mode can be extended to use JSONP which enables embedding pages everywhere.
If what you really want is to expose the data, but not the visual content, then I'd consider exposing your data via JSONP. There are caveats to this approach, but it could work for you. There was an answer here a couple of days ago about using a Web Service, but this won't work directly from the client because of the browser's Same Origin policy. It's a shame that the poster of that answer deleted it rather than leave it here as he inadvertently highlighted some of the misconceptions about how browsers access remote content.

Categories