If I create a cookie with the value 'ë' on the server, it becomes 'ë' when I read it on the client side.
This only occurs with IE (10). I don't get this issue with other browsers (Chrome and Firefox).
The way the cookie is created:
HttpContext.Current.Response.Cookies.Add(new HttpCookie("test", "ë"));
Per RFC 2109 (sec. 4.2.2) and RFC 2616 (sec. 2.2, 4.2), HTTP headers may only be transmitted in ISO-8859-1. (There is an exception, but this is primarily used for MIME and is pretty much never seen in HTTP.) Since ISO-8859-1 is really just a series of octets, the server can choose to use a different encoding (UTF8, in this case) if it wishes, since cookies are intended to be treated opaquely by the client anyway. Thus it should be round-tripped correctly in all cases, but if the client tries parsing the cookie it will get different results based on whether or not it is following the RFCs.
Please try to avoid sending non-ASCII characters in headers if at all possible. The HTTP spec never really accounted for this properly, and as a result it causes pain and heartache when used in practice. :(
Related
This question is related to Cross-Origin Resource Sharing (CORS, http://www.w3.org/TR/cors/).
If there is an error when making a CORS request, Chrome (and AFAIK other browsers as well) logs an error to the error console. An example message may look like this:
XMLHttpRequest cannot load http://domain2.example. Origin http://domain1.example is not allowed by Access-Control-Allow-Origin.
I'm wondering if there's a way to programmatically get this error message? I've tried wrapping my xhr.send() call in try/catch, I've also tried adding an onerror() event handler. Neither of which receives the error message.
See:
http://www.w3.org/TR/cors/#handling-a-response-to-a-cross-origin-request
...as well as notes in XHR Level 2 about CORS:
http://www.w3.org/TR/XMLHttpRequest2/
The information is intentionally filtered.
Edit many months later: A followup comment here asked for "why"; the anchor in the first link was missing a few characters which made it hard to see what part of the document I was referring to.
It's a security thing - an attempt to avoid exposing information in HTTP headers which might be sensitive. The W3C link about CORS says:
User agents must filter out all response headers other than those that are a simple response header or of which the field name is an ASCII case-insensitive match for one of the values of the Access-Control-Expose-Headers headers (if any), before exposing response headers to APIs defined in CORS API specifications.
That passage includes links for "simple response header", which lists Cache-Control, Content-Language, Content-Type, Expires, Last-Modified and Pragma. So those get passed. The "Access-Control-Expose-Headers headers" part lets the remote server expose other headers too by listing them in there. See the W3C documentation for more information.
Remember you have one origin - let's say that's the web page you've loaded in your browser, running some bit of JavaScript - and the script is making a request to another origin, which isn't ordinarily allowed because malware can do nasty things that way. So, the browser, running the script and performing the HTTP requests on its behalf, acts as gatekeeper.
The browser looks at the response from that "other origin" server and, if it doesn't seem to be "taking part" in CORS - the required headers are missing or malformed - then we're in a position of no trust. We can't be sure that the script running locally is acting in good faith, since it seems to be trying to contact servers that aren't expecting to be contacted in this way. The browser certainly shouldn't "leak" any sensitive information from that remote server by just passing its entire response to the script without filtering - that would basically be allowing a cross-origin request, of sorts. An information disclosure vulnerability would arise.
This can make debugging difficult, but it's a security vs usability tradeoff where, since the "user" is a developer in this context, security is given significant priority.
Summary: I would like to make a Range header request from GitHub pages. However, in some browsers this is failing- possibly due to Gzip compression issues. It works in Chrome (v74) but not in FF (v66), Mac OS.
Goal: I would like to reliably make this request in all browsers, such as by forcing the response type to be encoded as text whenever a range request is made.
It is not clear to me whether this behavior is dictated by the browser, the server, or some combination of the two. Knowing origins could help to define a fix- working with Github pages would be nice but not mandatory.
It is also not clear to me whether this represents a bug, or if so, where. (in browser, in spec, etc)
Sample test case:
Possibly because this involves server-side gzip encoding, the sample test case doesn't reproduce locally. You'll need to enter these commands in the JS console at https://abought.github.io/weetabix/ to reproduce.
fetch('https://abought.github.io/weetabix/example_data/McDonald_s.csv', {headers: {range: 'bytes=1-100'}} ).then(resp => resp.text());
In chrome, this fetches the response text. In firefox, it gives a "decoding error".
If I omit resp.text, Firefox can complete the request- the decoding error is in reading the body, rather than any other code. Copying as curl shows that FF adds a --compress flag and Chrome does not.
Investigations
If the byte range is 0-100, the request works in FF. If the range is 1-100, it fails. This section of the file is all ASCII characters.
If I inspect the response headers (Array.from(r.headers.entries())), FF has an extra "content-encoding: gz flag" that I think is causing the issue.
(eg, gzip makes no sense without the secret decoder instructions)
I tried adding 'accept-encoding': 'identity' to the fetch request, but it is a forbidden header and modifying it via code has no effect.
The specs have changed quite recently here. Here is the link to the PR.
TLDR; They now ask the UA that the Acccept-Encoding/Identity header be added to all the Range-requests.
[§5.15]
If httpRequest’s header list contains Range, then append Accept-Encoding/identity to httpRequest’s header list.
Firefox has not yet followed up here, but a bug report has been filled.
For the time being, the Range requests in Firefox indeed are made with the Gzipped data, and thus, you must not break the bytes integrity (for instance the range 0-100 is decode-able in Firefox).
When using the EventSource API in JavaScript, is there any way to send a request body along with the HTTP request initiating the polling?
I need to send a large blob of JSON to the server at the SSE request so that the server can calculate what events to send to the client. It seems daft to do web-sockets when I don't need it or do weird things with cookies or multiple requests.
I worry i'll run in to length limits on query strings if I bundle the data in to that, which may be likely.
Thanks in advance!
The initial SSE request is a fairly ordinary HTTP GET request, so:
Given that SSE is only supported by modern browsers, the maximum URL length should not be assumed to be the old 255 bytes "for old browsers". Most modern browsers allow longer URLs, with IE providing the lowest cap of ~2k. (granted EventSource is not supported on IE anyway, but there's an XHR polyfill...) However, if by large blob you mean several kilobytes, the URL is not reliable. Proxies could also potentially cause problems.
See:
What is the maximum length of a URL in different browsers?,
Is there any limitation on Url's length in Android's WebView.loadUrl method?,
http://www.benzado.com/blog/post/28/iphone-openurl-limit
You could also store information in one or more cookies which will be sent along with the GET request. This could include a cookie you set on the request for the page that uses SSE, or a cookie you set in javascript (prior to creating your EventSource object). The max size for a cookie is specified as being at least 4096 bytes (which is the whole cookie, so somewhat less for your actual data portion) with at least 20 cookies per hostname supported. Emperical testing appears to bear this out: http://browsercookielimits.x64.me/ Worst case you could possibly chunk the information in multiple cookies.
Larger than that, and I think you need an initial request that uploads the JSON and sends back an ID that is referenced by the SSE request.
It is technically possible, but (strongly) discouraged, to send a body with a GET request. See HTTP GET with request body. The EventSource constructor only takes a URL and so does not directly support this.
As dandavis pointed out, you can compress your JSON.
I mocking my API using Apiary.io. But somehow I cannot read any headers from response object using angularJS. And I am sure i have at least Content-Type: application/json correctly set-up by checking in firebug. Code in Angular should read headers correctly too as i can print them when sending request to somewhere else than apiary.io...
$http.get('http://ies.apiary.io/some').then(function(response) {
console.log("ok",response.headers('Content-Type'));
},function(response){console.log("err",response);});
http://plnkr.co/edit/zMO0pXGsIdJkV0fZdBdw
It all boils down to a bug in firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=608735
For CORS request, firefox is not returning anything for req.getAllRequestHeaders(), although req.getRequestHeader('Content-Type') returns properly.
Either FF bug has to be fixed or Angular must work around it (as jQuery does) or you must go deeper and use XmlHttpRequest instance directly.
This was maybe also related, but probably not the core issue:
However, few days ago, Apiary.io was not setting Max-Age in CORS headers. Thus, if you had a minimal blueprint, you might have CORS pre-flight response cached and thus subsequent requests may be disallowed even if you added additional resources to your blueprint.
Max-Age is now set to 10 seconds, so it should work properly. However, depending on your browser, pre-flight cache might still affect you. Try purging it or test this app in another browser.
Forged POST requests can be constructed by untrusted websites by creating a form and posting it to the target site. However, the raw contents of this POST will be encoded by the browser to be in the format:
param1=value1¶m2=value2
Is it possible for untrusted websites to construct forged POSTs which contain arbitrary raw content -- such as stringified JSON?
{param1: value1, param2: value2}
Put another way: Can websites cause the browser to POST arbitrary content to third-party domains?
The POST body of an HTML form’s request is always either application/x-www-form-urlencoded, multipart/form-data, or text/plain as these reflect the valid values for the enctype attribute. Especially text/plain one can be used to form valid JSON data. So form-based CSRF can be used here, however, it requires the server to accept it as text/plain.
Additionally, XHR-based CSRF can be used as the XMLHttpRequest API allows so send arbitrary POST data. The only remaining obstacle with this is the Same-Origin Policy: Only if both have the same origin or your server supports Cross-Origin Request Sharing and allows resource sharing, such valid POST requests can be forged.
Yes!, a POST request is nothing more than text with a specific format sent to a web server. You can use IE or Chrome developer tools to look at what each requests looks like.
So yes, you can create a forged POST request and change whatever you want, however if the request is not well-formed most web servers will reject it.
https://www.rfc-editor.org/rfc/rfc2616
The client side code of a web site would have difficulties to forge a request like that, but the server side code could very easily do that.
As your web site can't tell if the request comes from a browser or a server that behaves just like a browser, the limitations in the browser is no protection.
You can create valid JSON via a regular form post. It's just a matter of creatively naming the form parameters. In particular, parameter names can contain quotes.
http://blog.opensecurityresearch.com/2012/02/json-csrf-with-parameter-padding.html
In the case of pure HTML forms, yes it will always be encoded according to the spec. But there are other encoding schemes such as MIME multipart. There is also the question of Javascript and XMLHttpRequest. Encoding is specifically mentioned in only one case. This strongly implies that there is no encoding applied in the other cases.