Unable to get iframe using Cheerio - javascript

Hi I am trying to get an iframe from wathfree.to.
Here is the code I am using for this purpose:
function getSecondBody(url2)
{
url2 = 'http://www.watchfree.to/watch-366-Forrest-Gump-movie-online-free-putlocker.html';
request(url2, function(err, resp, body)
{
var $ = cheerio.load(body);
var embedcode = $('.links_left_container');
embedcodetext2 = embedcode.html();
console.log(embedcodetext2);
return embedcodetext2;
});
}
But response returned doesn't contain the iframe that I need.
here is the response I am receiving:
while the actual page looks like this:
Only the iframe part is missing in my response.

view-source:http://www.watchfree.to/watch-366-Forrest-Gump-movie-online-free-putlocker.html
Please take a look the source of the page. There is no iframe. It's generic content. So when you request it, it won't load to DOM.
You need to crawl probably this line if you want openload embed link:
var locations = ["http:\/\/streamin.to\/embed-k2msgp8yhhd8-580x326.html","http:\/\/streamin.to\/embed-cbp8zxw3yo3f-580x326.html","http:\/\/embed.nowvideo.sx\/embed.php?v=2729070a4365a","http:\/\/thevideo.me\/embed-wm815ejm5uvb-580x326.html","https:\/\/estream.to\/embed-6u49dntawgr0.html","http:\/\/thevideo.me\/embed-n14mh2bjj2nz-580x326.html","http:\/\/vidtodo.com\/embed-bmjw5u1e18js.html","http:\/\/vidtodo.com\/embed-z8614uxy1824.html","https:\/\/openload.co\/embed\/xTLPXjSGz7o\/","https:\/\/openload.co\/embed\/F8gJS5Y1o1o\/"];

Related

script to get source code of website (js)

My school blocked CTRL + U, but you can use 'view-source:' before a link to view the code. It takes awhile, so i've been trying to make a script to automatically direct to the source code. However, I keep getting errors because it is not a link
I have tried the following:
var code = fetch(`view-source:https://${location.hostname}${location.pathname}`);
location.href = (code);
and
var code = (`view-source:https://${location.hostname}${location.pathname}`);
location.href = (code);
In the first one, I see a bad request, and in the second, I a blank page with the words "view-source:" followed by the link
view-source: isn't a real protocol you can fetch().
However, just
var resp = await fetch('http://...');
var text = await resp.text();
document.body.textContent = text;
should replace the current document's body with the text contents of that URL...
If you try from frontend to fetch the source code you will run to CORS Problems. But you can use some proxyies like in the example beloow:
fetch('https://api.codetabs.com/v1/proxy?quest=https://stackoverflow.com/questions/75440023/script-to-get-source-code-of-website-js#75440023').then((response) => response.text()).then((text) => console.log(text));

Get response from UrlFetchApp.Fetch() in Google app script and print on Logger

I wanted to get the HTML elements using UrlFetchApp.Fetch on this URL,using following code. However, it seems that only certain URLs work. Other URLs just returns 500 or a Bad request.
function getHTML()
{
var response = UrlFetchApp.fetch("http://www.theoutlet24.com/th/catalogsearch/\
result/index/?cat=0&limit=all&q=4X4Man");
Logger.log(response);
}
Any help in this matter is appreciable .
Add just {muteHttpExceptions:true} as an option after URL.I have tested its working!
function getHTML()
{
var response = UrlFetchApp.fetch("http://www.theoutlet24.com/th/catalogsearch/result/index/?cat=0&limit=all&q=4X4Man",{muteHttpExceptions:true});
Logger.log(response);
}

Modify <div> container on click of a button, insert HTML from GET request

Prerequisites
I have a Website, that displays a page with an input and a button. On the other end is a server that exposes a very basic HTTP API. The API is called like this:
http://127.0.0.1/api/arg1/arg2/arg3
where argX are the arguments. It returns raw HTML. This HTML code needs to be inserted into the Website (another domain). There is a
<div id="container5"></div>
on the website. The HTML needs to be inserted into this container. The code returned by the API is specifically made to be inserted into this container, as it uses CSS classes and scripts from the website, i.e.: the code is not valid for it self.
The Goal
Here is what I have: I've got the API to return what I want, and I got a small JavaScript to run on the website to change the contents of the container:
var element = document.getElementById("container5");
element.innerHTML = "New Contents";
This works so far. Now I need a way to get the HTML from the API to the page. By reading numerous SO questions, it quickly became clear that reading HTML from another URL is close to impossible in JavaScript, due to security constraints.
Is there an easy way to do this with JavaScript or do I need rethink the whole process somehow? One last constraint on my side is that I can only insert JS into the website, I can't - for example - upload a new file to the server.
Edit 1: Workaround!
I solved this for me by using a PHP intermediate file on the requesting server:
<?php
echo file_get_contents('http://example.com');
?>
This will generate a site using the HTML content of any URL. Now the requesting site can read this by using JavaScript:
var getHTML = function ( url, callback ) {
// Feature detection
if ( !window.XMLHttpRequest ) return;
// Create new request
var xhr = new XMLHttpRequest();
// Setup callback
xhr.onload = function() {
if ( callback && typeof( callback ) === 'function' ) {
callback( this.responseXML );
}
}
// Get the HTML
xhr.open( 'GET', url );
xhr.responseType = 'document';
xhr.send();
};
This modifies any element:
var element = document.getElementById("resultpage");
getHTML( 'http://localserver.org/test.php', function (response) {
element.innerHTML = response.documentElement.innerHTML;
});
Checkout CORS https://en.wikipedia.org/wiki/Cross-origin_resource_sharing
also JSONP in same article.

Display in iframe a div with a certain ID from an external page

I want to display the content of a certain div with the ID="example" from an external page in an iframe on my website, is it possible and how can it be done? I have no control over the external page, only on my website. The only thing i know from that external page is the ID of the div i want to show on my website....maybe you have a fiddle example
There are many possibilites to build it. (Server or client side).
One of them is use NodeJS working as web crawler, http://net.tutsplus.com/tutorials/javascript-ajax/how-to-scrape-web-pages-with-node-js-and-jquery/
Here is an example, using NodeJS with express and cheerio (for Jquery manipulation):
var url = 'YOUR_URL';
function getContentJSON(body) {
var cheerio = require('cheerio');
var $ = cheerio.load(body);
var content = $('div#ID').text().trim();
var result = {"content": content}
return JSON.stringify(result);
}
function requestPage(url, res) {
var request = require('request');
request(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
var json = getContentJSON(body);
res.send(statusCodes.OK, json);
}
});
}
exports.get = function(request, response) {
requestPage(url, response);
};
If I understand correctly, you want to display only the div on your website.
As far as I know this is not possible due to browser restrictions. If you just load it into an iframe, you will not be able to manipulate the DOM of the content within that iframe. You could try and see if you can manipulate the scrollbars but I have no knowledge if this is possible or if the same restrictions apply.
Another option would be to load the external page using AJAX. This however will be a cross domain request, requiring the external website to allow CORS requests from your domain. This is highly unlikely. You could use a proxy on your webserver to circumvent this problem.
In short: I don't think this is possible using standard javascript. It could however be done using a server side proxy.

Access NPAPI from pages DOM

I'm attempting to override the default functionality for webkitNotifications.createNotification and via a Chrome extension I'm able to inject a script in the pages DOM that does so. Problem I'm having now is I need access to chrome.extension.sendRequest from the pages DOM in order to push my request to the NPAPI I have embedded in the background page. I previously had the embed object rendered on each page during the execution of the content-script - but believe it's more effective (and safe) if the NPAPI is embedded within the extension not injected on every page.
if (window.webkitNotifications)
{
(function()
{
window.webkitNotifications.originalCreateNotification = window.webkitNotifications.createNotification;
window.webkitNotifications.createNotification = function (iconUrl, title, body) {
var n = window.webkitNotifications.originalCreateNotification(iconUrl, title, body);
n.original_show = n.show;
n.show = function ()
{
console.log("Chrome object", chrome);
console.log("Chrome.extension object", chrome.extension);
chrome.extension.sendRequest({'title' : title, 'body' : body, 'icon' : iconUrl});
}
return n;
}
})();
}
That is what is injected in the DOM as a script element. The background page is as follows:
<embed type="application/x-npapiplugz" id="plugz">
<script>
var osd = document.getElementById('plugz');
function processReq(req, sender, callback)
{
osd.notify(req.title, req.body, req.image);
console.log("NOTIFY!", req.title, req.body, req.image);
};
chrome.extension.onRequest.addListener(processReq);
</script>
Once your extension includes a NPAPI plugin, it is no longer safe :) But your correct, instead of allowing every single page have access to the plugin, it is better to let your extension have it. I assume you know about the "public" property which specifies whether your plugin can be accessed by regular web pages, the default is false.
Below, I will explain whats your problem, it isn't a accessing NPAPI from DOM pages problem, it is basically why can't your notifications access your content script or extension pages.
As you noticed, access to the content scripts and the pages DOM are isolated from each other. The only thing they share, is the DOM. If you want your notifications override to communicate to your content script, you must do so within a shared DOM. This is explained in Communication with the embedding page in the Content Scripts documentation.
You could do it the event way, where your content script listens on such event for data coming from your DOM, something like the following:
var exportEvent = document.createEvent('Event');
exportEvent.initEvent('notificationCallback', true, true);
window.webkitNotifications.createNotification = function (iconUrl, title, body) {
var n = window.webkitNotifications.createNotification(iconUrl, title, body);
n.show = function() {
var data = JSON.stringify({title: title, body: body, icon: iconUrl});
document.getElementById('transfer-dom-area').innerText = data;
window.dispatchEvent(exportEvent);
};
return n;
}
window.webkitNotifications.createHTMLNotification = function (url) {
var n = window.webkitNotifications.createHTMLNotification(url);
n.show = function() {
var data = JSON.stringify({'url' : url});
document.getElementById('transfer-dom-area').innerText = data;
window.dispatchEvent(exportEvent);
};
return n;
};
Then your event listener can send that to the background page:
// Listen for that notification callback from your content script.
window.addEventListener('notificationCallback', function(e) {
var transferObject = JSON.parse(transferDOM.innerText);
chrome.extension.sendRequest({NotificationCallback: transferObject});
});
I added that to my gist on GitHub for the whole extension (https://gist.github.com/771033), Within your background page, you can call your NPAPI plugin.
I hope that helps you out, I smell a neat idea from this :)

Categories