I've encountered a paywall and I'm trying to bypass it using javascript in the console. I did some research and found a few different approaches, one of which is changing the requestheader in order to make a given website believe that you got there through a twitter link (thus allowing you to view the content for free). The function I use aims to change the referer by listening to the onBeforeSendHeaders event as specified on https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onBeforeSendHeaders. It looks like the following (NOTE: This function is typed and executed directly inside of the devtools console):
function setReferer(x){
x.requestHeaders = x.requestHeaders.filter(function(header){
if(header.name === 'Referer'){
return false
return true
)}
x.requestheaders.push(
{
"name: "Referer",
"value": "https://t.co/" //Twitter website
}
return {requestHeaders: x.requestHeaders};
}
//this example uses chrome browser
chrome.webRequest.onBeforeSendHeaders.addListener(setReferer,
{
urls: ["<all_urls>"],
types: ["main_frame"], },
["requestHeaders", "blocking", "extraHeaders"] //extraHeaders meant to bypass CORS protocol
);
Unfortunately upon refreshing the window, this approach gives me folllowing error:
GET <some_url> net:ERR_BLOCKED_BY_CLIENT
Behind this error is the URL to the source code of the article, which I was able to load and copy into word, so I got the article I was looking for anyway. However I wasn't able to view it inside of the browsers main frame. Note that I am doing this only for the purpose of polishing my coding skills. I am trying to get a better understanding of the more complicated facets of the HTTP protocol, especially the way headers get sent clientside and interpreted serverside. If anyone knows more about the subject or knows / has a resource that he or she wants to share, this would me greatly appreciated!
Related
I'm trying to develop a basic Firefox Addon with the "new" WebExtensions system.
I'd like to
Extract some text from a web page (not owned by me)
evaluate it using a remote website
post in the same page the result
The problem is how to make the web request with the addon (point 2). I found that I could use XMLHttpRequest, but as I imagined for security reasons I can't get access to remote paths.
That is because (I guess) the javascript code is run inside the page, even though I had thought that an addon would be... external.
Of course the result would be inside the page, but I assume the addon could work as a proxy to make this request. That said I have no idea how and what should I do.
I don't want to use some strange trick (like removing some security control), I'd like to do it the "right" way.
What I also don't understand is if the addons are bounded to run within the page they are made for.
EDIT: OK, turns out the chrome docs is actually better than the mozilla one. To actually use the XHR to cross-site req you have to put an additional line of code in your manifest.
{...
"permissions": [
"http://random.com/"
],
}..
I'm still not sure if this is the proper way to do what I aim to though.
To actually use the XHR to cross-site req you have to put an additional line of code in your manifest.
{...
"permissions": [
"http://random.com/"
],
}
I just got started with firefox addons to help my team fasten up our work, what i am trying to create:
When being on a specific site (let's call it mysite.com/input) i want to fill out automatically an input with an id: "textinput" from the value that is stored on the clipboard.
Yeah it is simple yet it would be simply enough to paste it, wouldn't it?... now here is the twist:
I need an other form of the value: on the clipboard it is x/y/z. There is a database site (let's call it database.com) on which searching like database.com?s=x/y/z would directly give the page from where it is possible to gain the correct value as it has an id: #result
I got lost how to properly communicate between page and content scripts, i'm not even sure in what order should i use the pagemod and the page-worker
Please help me out! Thank you!
The basic flow is this:
In your content script, you get the value form the form, somehow. I'll leave that up to you.
Still in the content script, you send the data to main.js using self.port.emit:
Code:
self.port.emit('got-my-value', myValue);
In main.js, you would then receive the 'got-my-value' event and make a cross-domain request using the request module.
Code:
require('page-mod').PageMod({
include: 'somesite.com',
contentScriptFile: data.url('somescript.js'),
onAttach: function(worker) {
worker.port.on('got-my-value', function(value) {
require('request').Request({
url: 'http://someurl.com',
onComplete: function(response) {
console.log(response);
// maybe send data back to worker?
worker.port.emit('got-other-data', response.json);
}
}).post();
});
}
});
If you need to receive the data back in the original worker, you would another listener for the event coming back.
Code:
self.port.on('got-other-data', function(value) {
// do something
})
I've been struggling with the same issue for the past 2 days until I found this:
https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts/Cross_Domain_Content_Scripts
They indicate the following:
However, you can enable these features for specific domains by adding
them to your add-on's package.json under the "cross-domain-content"
key, which itself lives under the "permissions" key:
"permissions": {
"cross-domain-content": ["http://example.org/", "http://example.com/"] }
The domains listed must include the scheme
and fully qualified domain name, and these must exactly match the
domains serving the content - so in the example above, the content
script will not be allowed to access content served from
https://example.com/. Wildcards are not allowed. This feature is
currently only available for content scripts, not for page scripts
included in HTML files shipped with your add-on.
That did the trick for me.
I have the following code, which is supposed to be a simple example of using the google api javascript client, and simply displays the long-form URL for a hard-coded shortened URL:
<script>
function appendResults(text) {
var results = document.getElementById('results');
results.appendChild(document.createElement('P'));
results.appendChild(document.createTextNode(text));
}
function makeRequest() {
console.log('Inside makeRequest');
var request = gapi.client.urlshortener.url.get({
'shortUrl': 'http://goo.gl/fbsS'
});
request.execute(function(response) {
appendResults(response.longUrl);
});
}
function load() {
gapi.client.setApiKey('API_KEY');
console.log('After attempting to set API key');
gapi.client.load('urlshortener', 'v1', makeRequest);
console.log('After attempting to load urlshortener');
}
</script>
<script src="https://apis.google.com/js/client.js?onload=load"></script>
except with an actual API key instead of the text 'API_KEY'.
The console output is simply:
After attempting to set API key
After attempting to load urlshortener
but I never see 'Inside makeRequest', which is inside the makeRequest function, which is the callback function for the call to gapi.client.load, leading me to believe that the function is not working (or failing to complete).
Can anyone shed some light on why this might be so and how to fix it?
Thanks in advance.
After spending hours googling the problem, I found out the problem was because I was running this file on the local machine and not on a server.
When you run the above code on chrome you get this error in the developer console "Unable to post message to file://. Recipient has origin null."
For some reason the javascript loads only when running on a actual server or something like XAMPP or WAMP.
If there is any expert who can shed some light to why this happens, it would be really great full to learn.
Hope this helps the others noobies like me out there :D
Short answer (http://code.google.com/p/google-api-javascript-client/issues/detail?id=46):
The JS Client does not currently support making requests from a file:// origin.
Long answer (http://en.wikipedia.org/wiki/Same_origin_policy):
The behavior of same-origin checks and related mechanisms is not well-defined
in a number of corner cases, such as for protocols that do not have a clearly
defined host name or port associated with their URLs (file:, data:, etc.).
This historically caused a fair number of security problems, such as the
generally undesirable ability of any locally stored HTML file to access all
other files on the disk, or communicate with any site on the Internet.
I have a flex app hosted on domain A and served through a webpage at domain B. I have enabled cross-domain scripting; in the webpage at domain B, I have the line
params.allowscriptaccess = "always";
and in the application code I have
flash.system.Security.allowDomain("*");
and for good measure,
flash.system.Security.allowDomain("keonehon.com"); // domain A
ExternalInterface calls seem to work; on startup the app calls a javascript function and the webpage calls a function back to pass a parameter in.
if (ExternalInterface.available){
ExternalInterface.call("SWFLoadComplete");
//lblMessage.text = "Data Sent!";
}
function SWFLoadComplete(){
callNewCarWithUser();
}
function callNewCarWithUser()
{
var user_id = document.getElementById('txtUserId').value;
var room_id = document.getElementById('txtRoomId').value;
getTheFlexApp().newCarWithUser(user_id, room_id);
}
And this works correctly. So, yeah. Seems to be working fine, no?
HOWEVER, deep linking is not working, as setting the #state=____ flex parameter in the URL (either by typing something in, or by navigating backward using the back button) causes a javascript error to be thrown at line 435 of history.js, a.k.a.
getPlayer().browserURLChange(flexAppUrl);
It seems like there is some sort of cross-domain security problem, even though I put in the line flash.system.Security.allowDomain("*");. You can see that there are different behaviors when same-domain vs. cross-comain by comparing two pages with identical html and swf files, just cross-domain in one case and same-domain in the other:
cross-domain: http://keonehon.com/gongos/dreamcar.html.
same-domain: http://rails.mit.edu/gongos/dreamcar.html
What the heck is going on?
I'm not sure if this is related at all, but I use to get this error all the time with ExtJS and the problem actually ended up being a simple syntax error, (missing bracket ] )
Have you tried allowScriptAccess instead? (this might be case-sensitive if I remember correctly)
If that didn't work, it'd help to see more code or a sample page.
chrome.tabs.create({
'url': 'https://www.myserver.com/',
'selected': false
}, function(tab) {
chrome.tabs.executeScript(tab.id, {
'code': "doSomething();"
});
});
Actually I'm unable to execute the code, because there's invalid
certificate on the "myserver.com", so Chrome displays red page, which
I'm unable to skip and run my code.
Is there any way how to skip the red page except adding the
certification authority to trusted = except any neccessary step on the
client side?
You cannot inject or manipulate that page, due to security reasons. Which makes sense since that page is there to protect the user :)
The only way to do something like that is through Native Code, NPAPI. You implement a plugin that bypasses it. But as you know, implementing a plugin makes the whole computer vulnerable since you will have access to the entire host machine.
That is why creating plugins is not favoured, but recommended if you absolutely cannot do what you wanted with the current API and limitations.