In order to triage a problem with a web browser I am trying to determine the initiator of the XXX-xsrfstatemanager.js file (the XXX part seems to be something dynamic like a nonce) that occurs as part of a Google Authentication flow (using OAuth).
When I use Chrome developer tools, it says the below URL is the initiator:
https://accounts.google.com/o/oauth2/v2/auth?approval_state=%21Ch[REDACTED]Q%E2%88%99AJ[REDACTED]xq&as=-aBk[REDACTED]
Looking at the result of the above page see a lot of Javascript, but the string "xsrfstatemanager" is nowhere to be found, nor do I see any other javascript pages being included. Unless there is some really cryptic code that is somehow building this URL, the call is actually coming from some other page.
Does anyone know how I can get the 'real' initiator? Or if the above URL might be correct, if I can get more information like what exact line number of the file initiated the call?
By the way, while I edited the above URL for security reasons, if you go to (for example) www.quora.com and quick "continue with google" it is easy to see the flow in question.
The flow includes a redirection, which is why you cannot see the source code that initiates/references that script.
If you view the source of the original URL that is opened when you click on "Continue with Google", you will see the <script src> that references it. This works in Chrome and probably Safari -
view-source:https://accounts.google.com/o/oauth2/auth?redirect_uri=storagerelay%3A%2F%2Fhttps%2Fwww.quora.com%3Fid%3Dauth488109&response_type=code%20permission%20id_token&scope=email%20profile%20openid&openid.realm=&client_id=917071888555.apps.googleusercontent.com&ss_domain=https%3A%2F%2Fwww.quora.com&access_type=offline&include_granted_scopes=true&prompt=select_account&origin=https%3A%2F%2Fwww.quora.com&gsiwebsdk=2
From the source code -
<script src='https://ssl.gstatic.com/accounts/o/532969778-xsrfstatemanager.js' nonce="IgiKmQiLZIHDwGvce7/q6Q"></script>
You can also use tools like Fiddler to see the source code of the redirect, or check "Preserve log" in the Network panel of the Developer Tools feature of Chrome, or by going to the original URL with JavaScript disabled.
Related
I am doing some test then find below code would pop up a window:
test
looks like it is trying to open an application.
Can anyone tell me what is the usage/purpose of this "script"?
looks like FF, Chrome, and IE all support it.
script is being used as a protocol. No applications handle the script protocol, so clicking it does nothing useful.
You can register a custom protocol handler and if the user accepts it, your application will be allowed to open all links of that type (there are only a few permitted protocols):
window.navigator.registerProtocolHandler('web+test', 'http://example.org/?handler=%s', 'Test Protocol');
Only example.org will be allowed to run the above code, but if you open your dev tools while visiting http://example.org/ and run the above code, you will get a dialog asking you to allow or deny the protocol association. If you accept it, click the following link on any website:
test
It will redirect to http://example.org/?handler=hello
Trying to open an application? That script is just a hyperlink, it does not tell anything else in the script, just that it will open a link into site that tells "test"
I try to get the raw response body inside a Web Extension using Firefox 55.0.3.
Only "solutions" I have seen for now:
Repeat the request (I absolutly don't want to repeat the request)
Using Javascript to get innerHTML attribute of HTML tags such as head and body (tell me if I'm wrong, but with a solution like that I will not always have the whole content, for example I will get nothing in case of response without HTML. So it will never be the real raw response and in some case it will simply not work.)
Also, I saw this answer for Chrome (from 2015) using the debugger, but I wasn't able to do it with Firefox. This kind of solutions are interesting, I read Mozilla documentation about devtools but I didn't find a way of using the network tab of webtools interface with Javascript inside a Web Extension.
To give you more details, my goal is to intercept the full request and response from server (header and body). This is not a problem to do it, except for the response body.
Here an example of code to get the request body: (background script)
browser.webRequest.onBeforeRequest.addListener(
function (e) {
console.log(e);
},
{urls: ["http://*/*", "https://*/*"]},
["requestBody"]
)
Here some documentations that I used (there is more, but these links are all official):
Mozilla documentation about Web Extension
Intercept HTTP requests
webRequest
webRequest.onHeadersReceived
webRequest.onBeforeRequest
webRequest.onBeforeSendHeaders
Here some examples of Web Extensions.
Any ideas, solutions or even explainations "why this is not possible" are welcome, thank you in advance for your time !
Cheers++
This is now available, as of Firefox 57:
browser.webRequest.filterResponseData allows you to add a listener via browser.webRequest.onBeforeRequest which receives, and allows you to modify the response.
You can see an example in the Mozilla github webextensions-examples repo
Firefox 57 is going to provide the API browser.webRequest.filterResponseData. This doesn't seem to be documented yet, but you can look through bug 1255894 for details.
Why is this not possible?
For the simple reason that WebRequest was ported over from Chrome extensions, where this is explicitly impossible.
Requests for such functionality (to edit, or just to read) has been around for a very long time (since 2011 and 2015 respectively); they are challenging from both the security perspective and technical perspective, however a principal agreement that read access is a good idea is there.
However, it's simply not yet implemented. Rob W has been doing some work in this direction but it's not done yet.
Perhaps Firefox has a different implementation?
A cursory glance on Mozilla bugtracker doesn't find any bugs on providing this functionality. So, it's not likely that the implementation will diverge anytime soon.
Any workarounds?
Well, only the debugger-level access can touch actual response data.
Since debugger is not implemented in the WebExtension platform, only a devtools.network-using extension can access it - and only while Dev Tools are open for the tab making said request, which is the main limitation of devtools.* APIs.
What is the easy way to access with the chrome devtools api all the content of the sources tab in the devtools?
I am writing a small program using nightmarejs to scrap some webpages. And I need to do some analysis, both on the rendered html and on the original one.
Nightmarejs doesn't provide an api call to get the source of the page. I am thinking about using the devtools api. But this is not clear to me how to do so. As I can see many files in the Sources tab of the chrome devtools, I thought I could get this content easily.
For now, I have a few leads:
The chrome.devtools.network API.
There is snippet on the documentation:
chrome.devtools.network.onRequestFinished.addListener(
function(request) {
if (request.response.bodySize > 40*1024) {
chrome.devtools.inspectedWindow.eval(
'console.log("Large image: " + unescape("' +
escape(request.request.url) + '"))');
}
});
I thin I could use a listener like this. And get the body if it's available in the response. But I don't find the documentation for this response content. Also, I don't want to store the result of all the requests.
But my main problem is that I don't see the content of the request here. I tried to do a request.getContent(), which returned me null.
chrome.debug
I didn't have time to play with it yet.
Just say I typed in a bad hostname in the address bar.
For example, say I wasn't running a local web server, and I load:
http://localhost/callback_url
In Chrome, this will give me a "This webpage is not available" message.
Is there anyway I can find out what the url is in the address bar from the Javascript console, even though the page failed to load?
I know I can normally use window.location.href to get this, but that returns "data:text/html,chromewebdata" in this instance.
So in this example, I'd like to know if there's some javascript that returns http://localohost/callback_url
EDIT: The main reason I'd like to do this is so I know if server side redirect failed when using ChromeDriver with Selenium. So I'd prefer to avoid using extensions if possible, and am open to Chrome and ChromeDriver specific solutions if applicable! The callback_url may have extra info in it, added by the server, and I'd like to see what this info is. I'd like to avoid running another server to get this data if possible.
The loadTimeData object included in the ERR_CONNECTION_REFUSED page has the failed URL:
> loadTimeData.data_.summary.failedUrl
"http://localhost/foo?request_url=bar"
You can get it from the title of the page.
By typing document.title and doing some regex you can get the URL.
Another way I found is by using the following
var data = loadTimeData.createJsEvalContext();
console.log(data.a.$top.summary.failedUrl);
If you open the developer tools and search for a part of the URL in source code, you will see that Chrome creates the loadTimeData in the "not available page".
In the Net pane of developer tools I can see all the various scripts that are called on a webpage. I can do Ctl+F on the html source to find the script, or link to the script (like with the src attribute).
Sometimes however, scripts call scripts which are called by other scripts, so I can't see the actual reference to the script on the page. Is there anyway I can see exactly what called a particular script. Like a way in the net panel (eg on mainsite.com) I could see b.thomas.com was called by the call a.thomas.com. Then I could see that on the page, the chain started with <script src="a.thomas.com" type="text/javascript"></script>. The header I see in dev tools always simply says mainsite.com.
See my answer at https://stackoverflow.com/a/19565853/432681.
Firebug is currently not able to display the origin of a request directly because of missing APIs in Firefox, though you can use the Referer header of each request as an indication.