I need to get referrer , when user is redirected from other site to my own , i trying get referrer from headers but its empty. All attempts returns null or empty :
Request.UrlReferrer
HttpContext.Current.Response.Headers["Referer"].ToString()
ServerVariables["http_referer"]
and if you look to request headers in browser , i will not find referrer header.
tried get referrer from javascript document.referrer but its returns empty string
can somebody please explain why there is no referrer header and how i can get it ?
Usually, Referrer URLs are passed between two unrelated sites (from one site to another) when navigation occurs by clicking a link or JavaScript-based navigation. Referrer URLs are not sent if the user uses the browsers address bar, back/forward buttons/ etc.. to navigate.
There are several reasons why the Referrer URL is empty in a request.
For some (security/privacy) reasons, the Referrer URL is stripped out
when navigating from a HTTPS site to a HTTP site (e.g. from
https://google.com to http://example.com).
It can also be stripped out using some other JavaScript
and HTML tricks.
once Referrer URL has been stripped out, There is no way to disable this behavior to get it back.
Related
I've been trying to scrape some raw XML data from an internal company site (url excluded for security purposes). I am currently using selenium and beautifulsoup to do so (but am open to any other options). When accessing the site manually, I am prompted with a javascript browser alert for a username and password (see picture). My attempt to automatically validate credentials is below (does not pass authentication):
def main():
#gets specified list of direct reports
# username:password#
url ="http://{username}:{password}#myURL.com"
driver.get(url)
html = driver.page_source
soup = BeautifulSoup(html, "lxml")
# parsing logic follows ...
However, when the script runs I still have to manually enter the username and password in the browsing window controlled by chromedriver and then the rest of the program runs as expected..
Is there a way avoid this manually entry? I've also tried solutions around driver.alert and sending keys & credentials to the browser to no avail.. (I know this may be difficult because the site is not accessible outside of the network, any insight is appreciated!)
Edit: I should mention this method was working a couple weeks ago, but following a chrome update no longer does..
Your login process is likely returning an access token of some kind, either a value in the response body or a header with a token, possibly an Authorization header or a Set-Cookie header.
In most cases, you will need to send that token with every request, either as an authorization header, a body parameter, or whatever the page expects.
Your job is to find that token by inspecting the response from the server when you authenticate, store it somewhere, and send it back each time you make a page request to the server.
How you send it back is dictated by the requirements of the server in question. It may want a request body param or a header, those are the two most likely cases.
Is it possible to load an external website in iframe but without sending HTTP_REFERER ? I just don't want be tracked.
If it is possible then how and if not then is there any workaround using divs or anything else ?
For anchor tag with external link jQuery("a").attr('rel','noreferrer'); is working, but for iframe I've failed to make it work.
Is there any script( js or jQuery ) to make it work ?
Here's a very simple solution.
Use this in you document <head> tag and you are good to go :D
<meta name="referrer" content="none">
The meta referrer tag is placed in the <head> section of your HTML,
and references one of five states, which control how browsers send referrer information from your site.
The five states are:
None: Never pass referral data
None When Downgrade: Sends referrer information to secure HTTPS sites, but not insecure HTTP sites
Origin Only: Sends the scheme, host, and port (basically, the subdomain) stripped of the full URL as a referrer, i.e. moz.com/example.html would simply send moz.com
Origin When Cross-Origin: Sends the full URL as the referrer when the target has the same scheme, host, and port (i.e. subdomain) regardless if it's HTTP or HTTPS, while sending origin-only referral information to external sites. (note: There is a typo in the official spec. Future versions should be "origin-when-cross-origin")
Unsafe URL: Always passes the URL string as a referrer. Note if you have any sensitive information contained in your URL, this isn't the safest option. By default, URL fragments, username, and password are automatically stripped out.
Reference: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta
I came across this on MDN stating that setting the referrerpolicy attribute to no-referrer would accomplish this.
Example:
<iframe src="https://www.whatismyreferer.com/" referrerpolicy="no-referrer"></iframe>
For example, the first search result on this page leads to the older SO question, with the following HTTP request:
GET /questions/4402502/how-does-google-set-the-http-referrer-when-someone-clicks-on-a-search-result-lin HTTP/1.1
Host stackoverflow.com
Referer https://www.google.ru
Note, that:
Only the domain is included in the Referer header, no query string.
Google is open via HTTPS, while SO is open via plain HTTP - nevertherless, the Referer header is sent by the browser.
There are no server-side redirects involved, the first HTTP query to open after the click is to the target site.
The question is, how do they achieve this?
Google makes use of Referrer Policy.
They include the meta tag in the page:
<meta name="referrer" content="origin">
This tells browsers to use "Origin Only" policy, that is, to send domain only information in the Referrer header in any subsequent request.
How do we get the title of referrer URL page? I can get the referrer by
var referrer = document.referrer;
but unfortunately there's no (document.referrer).title available in javascript similar to document.getElementsByTagName('title')[0].innerHTML.
Any ideas how I can get this value?
If any information about the referer is passed to JavaScript then it will be nothing more than the URL.
To get the title of the page you will need to make an HTTP request to it and parse it out of the HTML.
If you make this HTTP request from JavaScript then it will be subject to the Same Origin Policy (with the usual work-arounds applying).
I am working on a login system that I want to work like this:
Attempt to post when not logged in pops up an alert with u/p fields -> submit u/p ajax hits login route -> successful login sets 'logged in' cookie -> attempt to post again now succeeds. all without a page reload.
It seems I have hit a roadblock though with all this CORS/cross-domain-access-control because I am using dynamic subdomains and XHR requests.
You could be anywhere.mysite.com and have to ajax crossdomain to user.mysite.com/login to login.
In order for login() to be allowed to set the cookie you have to specify your origin (no '*' wildcard), set a 'Access-Control-Allow-Credentials = true' header, and submit the ajax with a 'xhrFields: {'withCredentials': true}' parameter. I can get this to work.
My problem is the dynamic domains. Its not possible to set every possible subdomain in the accepted Origins list. Is there any possible way to have it accept .mysite.com as the origin so all subdomains.mysite.com pass?
This post lists the possibility of setting the document.domain = "company.com"; in an iframe to achieve this kind of functionality, but i'm not sure where in my login flow above i would insert an iframe or how that would even work...
Any thoughts?