I'm trying to optimise the speed of some web page that loads some JSON data using XMLHttpRequest from the same origin. I have added the Link HTTP header to the response of the web page, like this:
Link: </api/endpoint>; rel=preload; as=fetch; crossorigin
This works fine in Chrome. I see that the API endpoint is requested once (due to the preload instruction), and then used for the XMLHttpRequest.
However, in Safari it does not work. I see that the API endpoint is requested twice (once due to the preload instruction, and once more for the XMLHttpRequest). Also, a warning shows up in the console:
[Warning] The resource https://my-origin.com/api/endpoint was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it wasn't preloaded for nothing.
I have tried several variants of the Link headers (without crossorigin and with crossorigin=use-credentials). I also tried setting withCredentials=true on the XHR object. None of those combinations gave a different result.
I read some other question/answer, where it was explained that preloading fetch requests is possible by using mode: "no-cors". However, it is not possible to set that mode for XMLHttpRequest.
Is it possible to preload XMLHttpRequests (from the same origin) in Safari?
I'm using axios in my application, but I'm having a hard time setting the content of the request.
There's currently a call to a URL using $.ajax like this:
$.ajax({
method: 'POST',
data: { 'accountId': accountId },
url: serverUrl,
/* success: ... */
});
And when I look at this request in Chrome dev tools, at the end I see something like this:
Now, I'm trying to do the same thing with axios:
axios.post(serverUrl, { accountId: accountId })
.then(/* ... */);
But, when I look at the request in Chrome dev tools, I have this:
How can I get axios to do the same formatting as jQuery? And maybe the question is that: are they different or it's just the representation?
Also, I noticed that the jQuery call is somehow adding this header: x-requested-with: XMLHttpRequest, but to have the same header in axios, I have to set it manually. Is it normal? Am I missing an axios configuration to add this header?
Thank you
Some frameworks use this header to detect XHR requests, for example. Grails Spring uses this header to identify the query XHR and gives the JSON response or the HTML response as a response.
Most Ajax libraries (Prototype, JQuery and Dojo from version 2.1) include the X-Requested-With header, which indicates that the query was made using XMLHttpRequest instead of running by clicking a regular hyperlink or submitting a form button.
A good reason for security is that it can prevent CSRF attacks, because this header can not be added to the cross domain of the AJAX request without the server's consent through CORS.
Only the following headers are allowed:
To accept
Accept-Language
Content-Language
Last-Event-ID
Content-Type
any others call the "before flight" request in the browsers supported by CORS.
Without CORS, X-Requested-With can not be added to an XHR request with a cross domain.
If the server checks the presence of this header, it knows that the request did not initiate an attempt to make a request on behalf of the user from the attacker's domain using JavaScript.
It also checks that the request was not sent from the usual HTML form, from which it is more difficult to verify that it is not a cross domain without the use of tokens. (However, checking the Origin header can be an option in supported browsers although you leave old browsers vulnerable.)
See also: https://markitzeroday.com/x-requested-with/cors/2017/06/29/csrf-mitigation-for-ajax-requests.html
So also read for greater understanding:
FormData()
https://developer.mozilla.org/en-US/docs/Web/API/FormData/Using_FormData_Objects
https://developer.mozilla.org/en-US/docs/Web/API/FormData/FormData
Request Payload
What's the difference between "Request Payload" vs "Form Data" as seen in Chrome dev tools Network tab
As documented here, You can use the URLSearchParams API to send data in the application/x-www-form-urlencoded format using axios.
Example from offical docs:
var params = new URLSearchParams();
params.append('param1', 'value1');
params.append('param2', 'value2');
axios.post('/foo', params);
I've been stacked with issue which I can't solve
for about month. You can look at the example page here: http://www.7ya.ru/travel/tours/
So, the problem is that little green form is our widget which uses cross-domain XHR and
some people have problem with it. XHR request fails. There is no request exactly. Inspector
just says "canceled". This problem occures in FF, Opera and Chrome on OSX and Win.
BUT! If you'll try to clean cache (for example in Chrome):
you will see that the widget is starts to work like a charm.
Also if you'll try to clean only a cache it wont work.
So, is anybody has ideas about which data may affect XHR requests in most browsers on two different platforms?
P.S. I've been trying to use "vanilla" XHR and jQuery version but it works same.
P.S.S. Bit of code (CoffeeScript):
lt_jq – local jQuery version ($.noConflict())
#inspect - JSON.stringify() with some additional params for formatting
request = lt_jq.ajax({
url : method,
dataType : "json",
data : params,
xhrFields : withCredentials:true
})
request.done (data)->
#request_id = (data.request_id || null)
callback?(data)
request.fail =>
console.log "Request failed [#{method}], #{#inspect(params)}"
QUESTION CLOSED, DETAILS IN COMMENTS
Does the network you're in use a caching proxy, or from the sound of it, do you have an antivirus or some sort of filter on the network which filters the requests through?
It sounds like the files are being cached without the correct headers, so the CORS headers are being cut off...
The easiest way I suggest you check this:
USING CHROME:
Clear your cache, open the dev tools, and watch the request when it
does work. Make sure you have the CORS headers in the request and response headers.
Now that you have the file in your cache, reload your page. Check that it's not working.
Now that it doesn't work, go to chrome://view-http-cache/ to view the individual cache
entries, and look for that file's request. Open it.
Check if the cached file has the CORS headers on it.
If it doesn't... make sure the proxy/antivirus doesn't cut off the CORS headers in your network, or try a different solution
Many analytic and tracking tools are requesting 1x1 GIF image (web bug, invisible for the user) for cross-domain event storing/processing.
Why to serve this GIF image at all? Wouldn't it be more efficient to simply return some error code such as 503 Service Temporary Unavailable or empty file?
Update: To be more clear, I'm asking why to serve GIF image data when all information required has been already sent in request headers. The GIF image itself does not return any useful information.
Doug's answer is pretty comprehensive; I thought I'd add in an additional note (at the OP's request, off of my comment)
Doug's answer explains why 1x1 pixel beacons are used for the purpose they are used for; I thought I'd outline a potential alternative approach, which is to use HTTP Status Code 204, No Content, for a response, and not send an image body.
204 No Content
The server has fulfilled the request
but does not need to return an
entity-body, and might want to return
updated metainformation. The response
MAY include new or updated
metainformation in the form of
entity-headers, which if present
SHOULD be associated with the
requested variant.
Basically, the server receives the request, and decides to not send a body (in this case, to not send an image). But it replies with a code to inform the agent that this was a conscious decision; basically, its just a shorter way to respond affirmatively.
From Google's Page Speed documentation:
One popular way of recording page
views in an asynchronous fashion is to
include a JavaScript snippet at the
bottom of the target page (or as an
onload event handler), that notifies a
logging server when a user loads the
page. The most common way of doing
this is to construct a request to the
server for a "beacon", and encode all
the data of interest as parameters in
the URL for the beacon resource. To
keep the HTTP response very small, a
transparent 1x1-pixel image is a good
candidate for a beacon request. A
slightly more optimal beacon would use
an HTTP 204 response ("no content")
which is marginally smaller than a 1x1
GIF.
I've never tried it, but in theory it should serve the same purpose without requiring the gif itself to be transmitted, saving you 35 bytes, in the case of Google Analytics. (In the scheme of things, unless you're Google Analytics serving many trillions of hits per day, 35 bytes is really nothing.)
You can test it with this code:
var i = new Image();
i.src = "http://httpstat.us/204";
First, i disagree with the two previous answers--neither engages the question.
The one-pixel image solves an intrinsic problem for web-based analytics apps (like Google Analytics) when working in the HTTP Protocol--how to transfer (web metrics) data from the client to the server.
The simplest of the methods described by the Protocol, the simplest (at lest the simplest method that includes a request body) is the GET request. According to this Protocol method, clients initiate requests to servers for resources; servers process those requests and return appropriate responses.
For a web-based analytics app, like GA, this uni-directional scheme is bad news, because it doesn't appear to allow a server to retrieve data from a client on demand--again, all servers can do is supply resources not request them.
So what's the solution to the problem of getting data from the client back to the server? Within the HTTP context there are other Protocol methods other than GET (e.g., POST) but that's a limited option for many reasons (as evidenced by its infrequent and specialized use such as submitting form data).
If you look at a GET Request from a browser, you'll see it is comprised of a Request URL and Request Headers (e.g., Referer and User-Agent Headers), the latter contains information about the client--e.g., browser type and version, browser langauge, operating system, etc.
Again, this is part of the Request that the client sends to the server. So the idea that motivates the one-pixel gif is for the client to send the web metrics data to the server, wrapped inside a Request Header.
But then how to get the client to Request a resource so it can be "tricked" into sending the metrics data? And how to get the client to send the actual data the server wants?
Google Analytics is a good example: the ga.js file (the large file whose download to the client is triggered by a small script in the web page) includes a few lines of code that directs the client to request a particular resource from a particular server (the GA server) and to send certain data wrapped in the Request Header.
But since the purpose of this Request is not to actually get a resource but to send data to the server, this resource should be a small as possible and it should not be visible when rendered in the web page--hence, the 1 x 1 pixel transparent gif. The size is the smallest size possible, and the format (gif) is the smallest among the image formats.
More precisely, all GA data--every single item--is assembled and packed into the Request URL's query string (everything after the '?'). But in order for that data to go from the client (where it is created) to the GA server (where it is logged and aggregated) there must be an HTTP Request, so the ga.js (google analytics script that's downloaded, unless it's cached, by the client, as a result of a function called when the page loads) directs the client to assemble all of the analytics data--e.g., cookies, location bar, request headers, etc.--concatenate it into a single string and append it as a query string to a URL (*http://www.google-analytics.com/__utm.gif*?) and that becomes the Request URL.
It's easy to prove this using any web browser that has allows you to view the HTTP Request for the web page displayed in your browser (e.g., Safari's Web Inspector, Firefox/Chrome Firebug, etc.).
For instance, i typed in valid url to a corporate home page into my browser's location bar, which returned that home page and displayed it in my browser (i could have chosen any web site/page that uses one of the major analytics apps, GA, Omniture, Coremetrics, etc.)
The browser i used was Safari, so i clicked Develop in the menu bar then Show Web Inspector. On the top row of the Web Inspector, click Resources, find and click the utm.gif resource from the list of resources shown on the left-hand column, then click the Headers tab. That will show you something like this:
Request URL:http://www.google-analytics.com/__utm.gif?
utmwv=1&utmn=1520570865&
utmcs=UTF-8&
utmsr=1280x800&
utmsc=24-bit&
utmul=enus&
utmje=1&
utmfl=10.3%20r181&
Request Method:GET
Status Code:200 OK
Request Headers
User-Agent:Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/533.21.1
(KHTML, like Gecko) Version/5.0.5 Safari/533.21.1
Response Headers
Cache-Control:private, no-cache, no-cache=Set-Cookie, proxy-revalidate
Content-Length:35
Content-Type:image/gif
Date:Wed, 06 Jul 2011 21:31:28 GMT
The key points to notice are:
The Request was in fact a request
for the utm.gif, as evidenced by the
first line above: *Request
URL:http://www.google-analytics.com/__utm.gif*.
The Google Analytics parameters are clearly visible in the query string
appended to the Request URL: e.g.,
utmsr is GA's variable name to refer to the client screen
resolution, for me, shows a value of
1280x800; utmfl is the variable
name for flash version, which has a
value of 10.3, etc.
The Response Header called
Content-Type (sent by the server back to the client) also confirms
that the resource requested and
returned was a 1x1 pixel gif:
Content-Type:image/gif
This general scheme for transferring data between a client and a server has been around forever; there could very well be a better way of doing this, but it's the only way i know of (that satisfies the constraints imposed by a hosted analytics service).
Some browsers may display an error icon if the resource could not load. It makes debugging/monitoring the service also a little bit more complicated, you have to make sure that your monitoring tools treat the error as a good result.
OTOH you don't gain anything. The error message returned by the server/framework is typically bigger then the 1x1 image. This means you increase your network traffic for basically nothing.
Because such a GIF has a known presentation in a browser - it's a single pixel, period. Anything else presents a risk of visually interfering with the actual content of the page.
HTTP errors could appear as oversized frames of error text or even as a pop-up window. Some browsers may also complain if they receive empty replies.
In addition, in-page images are one of the very few data types allowed by default in all broswers. Anything else may require explicit user action to be downloaded.
This is to answer the OP's question - "why to serve GIF image data..."
Some users will put a simple img tag to call your event logging service -
<img src="http://www.example.com/logger?event_id=1234">
In this case, if you don't serve an image, the browser will show a placeholder icon that will look ugly and give the impression that your service is broken!
What I do is, look for the Accept header field. When your script is called via an img tag like this, you will see something like following in the header of the request -
Accept: image/gif, image/*
Accept-Encoding:gzip,deflate
...
When there is "image/"* string in the Accept header field, I supply the image, otherwise I just reply with 204.
Well the major reason is to attach the cookie to it so if users go from one side to another we still have the same element to attach cookie to.
#Maciej Perliński is basically correct, but I feel a detailed answer will be beneficial.
why 1x1 GIF and not a 204 No-Content status code?
204 No-Content enables the server to omit all response headers (Content-Type, Content-Length, Content-Encoding, Cache-Control etc...) and return an empty response body with 0 bytes (and saving a lot of unneeded bandwidth).
Browsers know to respect 204 No-Content responses, and not to expect/wait for response headers and response body.
if the server needs to set any response header (e.g. cache-control or cookie), he cannot use 204 No-Content because browsers will ignore any response header by design (according to the HTTP protocol spec).
why 1x1 GIF and not a Content-Length: 0 header with 200 OK status code?
Probably a mix of several issues, just to name a few:
legacy browsers compatibility
MIME type checks on browsers, 0 bytes is not a valid image.
200 OK with 0 bytes might not be fully supported by intermediate proxy servers and VPNs
You don't have to serve an image if you are using the Beacon API (https://w3c.github.io/beacon/) implementation method.
An error code would work if you have access to the log files of your server. The purpose of serving the image is to obtain more data about the user than you normally would with a log file.
I'm jusing jQuery's $.getJSON() function to return a short set of JSON data.
I've got the JSON data sitting on a url such as example.com.
I didn't realize it, but as I was accessing that same url, the JSON data couldn't be loaded. I followed through the console and found that XMLHttpRequest couldn't load due to Access-Control-Allow-Origin.
Now, I've read through, a lot of sites that just said to use $.getJSON() and that would be the work around, but obviously it didn't work. Is there something I should change in the headers or in the function?
Help is greatly appreciated.
It's simple, use $.getJSON() function and in your URL just include
callback=?
as a parameter. That will convert the call to JSONP which is necessary to make cross-domain calls. More info: http://api.jquery.com/jQuery.getJSON/
You may well want to use JSON-P instead (see below). First a quick explanation.
The header you've mentioned is from the Cross Origin Resource Sharing standard. Beware that it is not supported by some browsers people actually use, and on other browsers (Microsoft's, sigh) it requires using a special object (XDomainRequest) rather than the standard XMLHttpRequest that jQuery uses. It also requires that you change server-side resources to explicitly allow the other origin (www.xxxx.com).
To get the JSON data you're requesting, you basically have three options:
If possible, you can be maximally-compatible by correcting the location of the files you're loading so they have the same origin as the document you're loading them into. (I assume you must be loading them via Ajax, hence the Same Origin Policy issue showing up.)
Use JSON-P, which isn't subject to the SOP. jQuery has built-in support for it in its ajax call (just set dataType to "jsonp" and jQuery will do all the client-side work). This requires server side changes, but not very big ones; basically whatever you have that's generating the JSON response just looks for a query string parameter called "callback" and wraps the JSON in JavaScript code that would call that function. E.g., if your current JSON response is:
{"weather": "Dreary start but soon brightening into a fine summer day."}
Your script would look for the "callback" query string parameter (let's say that the parameter's value is "jsop123") and wraps that JSON in the syntax for a JavaScript function call:
jsonp123({"weather": "Dreary start but soon brightening into a fine summer day."});
That's it. JSON-P is very broadly compatible (because it works via JavaScript script tags). JSON-P is only for GET, though, not POST (again because it works via script tags).
Use CORS (the mechanism related to the header you quoted). Details in the specification linked above, but basically:
A. The browser will send your server a "preflight" message using the OPTIONS HTTP verb (method). It will contain the various headers it would send with the GET or POST as well as the headers "Origin", "Access-Control-Request-Method" (e.g., GET or POST), and "Access-Control-Request-Headers" (the headers it wants to send).
B. Your PHP decides, based on that information, whether the request is okay and if so responds with the "Access-Control-Allow-Origin", "Access-Control-Allow-Methods", and "Access-Control-Allow-Headers" headers with the values it will allow. You don't send any body (page) with that response.
C. The browser will look at your response and see whether it's allowed to send you the actual GET or POST. If so, it will send that request, again with the "Origin" and various "Access-Control-Request-xyz" headers.
D. Your PHP examines those headers again to make sure they're still okay, and if so responds to the request.
In pseudo-code (I haven't done much PHP, so I'm not trying to do PHP syntax here):
// Find out what the request is asking for
corsOrigin = get_request_header("Origin")
corsMethod = get_request_header("Access-Control-Request-Method")
corsHeaders = get_request_header("Access-Control-Request-Headers")
if corsOrigin is null or "null" {
// Requests from a `file://` path seem to come through without an
// origin or with "null" (literally) as the origin.
// In my case, for testing, I wanted to allow those and so I output
// "*", but you may want to go another way.
corsOrigin = "*"
}
// Decide whether to accept that request with those headers
// If so:
// Respond with headers saying what's allowed (here we're just echoing what they
// asked for, except we may be using "*" [all] instead of the actual origin for
// the "Access-Control-Allow-Origin" one)
set_response_header("Access-Control-Allow-Origin", corsOrigin)
set_response_header("Access-Control-Allow-Methods", corsMethod)
set_response_header("Access-Control-Allow-Headers", corsHeaders)
if the HTTP request method is "OPTIONS" {
// Done, no body in response to OPTIONS
stop
}
// Process the GET or POST here; output the body of the response
Again stressing that this is pseudo-code.