We Keep Coding

confusion about Cache-Control HTTP header

I have two questions regarding HTTP cache using the Cache-Control header:
How does the browser identify which request can be fulfilled using the existing cache? Is the browser checking if the endpoint matches? But even requests to the same endpoint can have different body or config, I don't quite understand how does the browser know when to use the cache and when to not use the cache given the time it sends out the request is still within the time frame specified by max-age in the response's cache-control header?
I recently learned that both request and response can set max-age in their own cache-control header. I understand that the request's max-age would tell the server (or any intermediate caches) how fresh of response the client is willing to accept from them. The response max-age (or lack thereof) tells the client how long it can consider that response to be fresh. (feel free to correct me if I am wrong). But what would happen in this scenario:
let's say the response has a max-age for one year and then we send another request for the same resources with max-age being 0. Does that make the browser ignore the cache? or Does the browser accept the cache and not send out the request?

You can get information from this specification. According to the document,
The cache entry contains the headers of the request:
As discussed above, caching servers will by default match future
requests only to requests with exactly the same headers and header
values.
This means that you get one entry in your cache every time you make exactly the same request to the server (the cache can be personal or shared, like in a proxy). In practice, for entities that only cache GET requests, the key can be the URI of the request. By the process of normalization, two very similar requests can share a cache entry. The decision to use the cached entry depends on several factors, as detailed below. The figures in the document explain this very well. Bottom line, max-age only determines freshness, not the behavior of the cache.
According to this specification, the cache is never ignored if the entry exists. Even a fresh entry can be discarded to save disk space, and a stale entry can be kept long after it has expired. The diference is that a stale entry is not directly retrieved. In that case, the caching entity (browser/proxy/load_balancer...) sends a freshness request to the server. The server then decides whether the cached page is fresh.
In summary, if a cached page is fresh according to max-age and whatever other modifiers are used, the caching entity decides that the cached resource will be used. If it is stale, the server decided whether the cached resource can be used.
EDIT after comment:
To understand the difference between max-age sent by the client and the server, we need to dig into the http protocol. In section 5.2.1., It says
5.2.1. Request Cache-Control Directives
5.2.1.1. max-age
Argument syntax:
delta-seconds (see Section 1.2.1)
The "max-age" request directive indicates that the client is
unwilling to accept a response whose age is greater than the
specified number of seconds. Unless the max-stale request directive
is also present, the client is not willing to accept a stale
response.
The language seems to indicate that the server is not forced by the directive, but it is expected to honor it. In your example, this would means that the client directive prevails, as it is more restrictive. The client is saying "I do not want any page cached for more than 0 seconds", and the cache server is suposed to contact the server to fulfill the condition.

Is it possible for an HTTP `GET` request with `Cache-Control: no-cache` to not hit the server exactly once? (Levering out idempotency of `GET`.)

In theory, one should use the HTTP GET method only for idempotent requests.
But, for some intricate reasons, I cannot use any other method than GET and my requests are not idempotent (they mutate the database). So my idea is to use the Cache-Control: no-cache header to ensure that any GET request actually hits the database. Also, I cannot change the URLs which means I cannot append a random URL argument to bust caches.
Am I safe or shall I implement some kind of mechanism to ensure that the GET request was received exactly once? (The client is the browser and the server is Node.js.)
What about a GET request that gets duplicated by some middle-man resulting in the same GET request being received twice by the server? I believe the spec allows such situation but does this ever happen in "real life"?
I've never seen a middle man, such as Cloudflare or NGNIX, preventing or duplicating a GET request with Cache-Control: no-cache.

Let's start by saying what you've already pointed out -- GET requests should be idempotent. That is, they should not modify the resource and therefore should return the same thing every time (barring any other methods being used to modify it in the meantime.)
It's worth pointing out, as restcookbook.com notes, that this doesn't mean nothing can change as a result of the request. Rather, the resource's representation should not change. So for instance, your database might log the request, but shouldn't return a different value in the response.
The main concern you've listed is middleware caching.
The danger isn't that the middleware sends the request to your server more than once (you mentioned 'duplicating' a request), but rather that (a) it sends an old, cached, no-longer-accurate response to whatever is making the request, and (b) the request does not reach the server.
For instance, imagine a response returning a count property that starts at 0 and increments when the GET endpoint is hit. Request #1 will return "1" as the count. Request #2 should now return "2" as the count, but if its cached, it might still show as 1, and not hit the server to increase the count to 2. That's 2 separate problems you have (caching, and not updating).
So, will a middleware prevent a request from reaching the server and serve a cached copy instead? We don't know. It depends on the middleware. You can absolutely write one right now that does just that. You can also write one that doesn't.
If you don't know what will be consuming your API, then it's not a great option. But whether it's "safe" depends on the specifics.
As you know, it's always best to follow the set of expectations that comes with the grammar of HTTP requests. Deviating from them sets yourself up for failure in many ways. (For instance, there are different security expectations for requests based on method. A browser may treat a GET request as "simple" from a CORS perspective, while it would never treat a PATCH request as such.)
I would go to great lengths to not break this convention, but if I were forced by circumstances to break this expectation, I would definitely note it in my APIs documentation.

One workaround to ensure that your GET request is only called once is to allow caching of responses and use the Vary header. The spec for the Vary header can be found here.
In summary, a Vary header basically tells any HTTP cache, which parts of the request header to take into account when trying to find the cached object.
For example, you have an endpoint /api/v1/something that accepts GET requests and does the required database updates. Let's say that when successful, this endpoint returns the following response.
HTTP/1.1 200 OK
Content-Length: 3458
Cache-Control: max-age=86400
Vary: X-Unique-ID
Notice the Vary header has a value of X-Unique-ID. This means that if you include the X-Unique-ID header in your request, any HTTP caching layer (be it the browser, CDN, or other middleware) will use the value in this header to determine whether to use a previously cached response or not.
Say your make a first request that includes a X-Unique-ID header with the value id_1 then you make a subsequent request with X-Unique-ID value of id_2. The caching layer will not use a previously cached response for the second request because the value of the X-Unique-ID is different.
However, if you make another request that contains the X-Unique-ID value of id_1 again, the caching layer will not make a request to the backend but instead reuse the cached response for the first request assuming that the cache hasn't expired yet.
One thing you have to consider though is this will only work if the caching layer actually respects the specifications for the Vary header.

The Hypertext Transfer Protocol (HTTP) is designed to enable communications between clients and servers.
where Get method is used to request the data from specified resources.
When we used 'Cache-control: no-cache' it means the cache can't store anything about the client request
or server responses. That Request hits to the server and a full response is downloaded each and every time.

This depends a lot on what's sat in the middle and where the retry logic sits, if there is any. Almost all of your problems will be in failure handling and retry handling - not the basic requests.
Let's say, for example that Alice talks to Bob via a proxy. Let's assume for the sake of simplicity that the requests are small and the proxy logic is pure store-and-forward. i.e. most of the time a request either gets through or doesn't but is unlikely to get stalled half-way through. There's no guarantee this is the case and some proxies will stop requests part-way through by design.
Alice -> Proxy GET
Proxy -> Bob GET
Bob -> Proxy 200
Proxy -> Alice 200
So far so good. Now imagine Bob dies before responding to the proxy. Does the proxy retry? If so, we have this:
Alice -> Proxy GET
Proxy -> Bob GET
Bob manipulates database then dies
Proxy -> Bob GET (retry)
Now we have a dupe
Unlikely, but possible.
Now imagine (much more likely) that the proxy (or even more likely, some bit of the network between the proxy and the client) dies. Does the client retry? If so, we have this:
Alice -> Proxy GET
Proxy -> Bob GET
Bob -> Proxy 200
Proxy or network dies
Alice -> Proxy GET (retry)
Proxy -> Bob GET
Is this a dupe or not? Depends on your point of view
Plus, for completeness there's also the degenerate case where the server receives the request zero times.

Create a Batch HTTP API With multipart response

Actually, I´ve create a Batch HTTP API that receives a JSON array with many different requests to our backend server. The Batch API just call all of these requests to a load balancer, wait for the return of all of them and return a new JSON to the client.
The client receives a huge JSON array response with its indices in the same position as the request, so it is easy to know what response is addressed for what request.
The motivation for this API was to solve the 5 browser simultaneous connections and improve performance as the Batch API has a much more direct access to the server (we do not have a reverse proxy or a SSL server between then).
The service is running pretty well, but now I have some new requirements as it is gaining more use. First, the service can use a lot of memory as it has a buffer for each request that will only be flushed when all responses are ready (I am using an ordered JSON Array). More, since it can take some time to all requests be delivered, the client will need to wait everything be processed before receiving a single byte.
I am planning change the service to return each response as soon it is available (and solve both issues above). And would like to share and validate my ideas with you:
I will change the response from a JSON response to a multipart response.
The server will include, for every part, the index of the response
The server will flush the response once its available
The client XHR will need to understand multipart content type response and be able to process each part as soon as it is available.
I will create a PoC to validate every step, but at this moment I would like to validate the idea and hear some thoughts about it. Here some doubts I have about the solution:
From what I read, I am in doubbt of that content-type is right for the response. multipart/mixed? multipart/digest?
Can I use an accept request header to identify if the client is able to handle the new service implementation? If so, what is the right accept header for this? My plan is to use the same endpoint but very accept header.
How can I develop a XHR client that is able to process many parts of a single response as soon as they are available? I found some ideias on the Web but I am not entirely confident with then.

I will change the response from a JSON response to a multipart
response.
The server will include, for every part, the index of the
response
The server will flush the response once its available
The
client XHR will need to understand multipart content type response and
be able to process each part as soon as it is available.
The XHR protocol will not support this work flow through a single request from the client. Since XHR relies heavily on the HTTP protocol for communications, XHR follows the HTTP connection rules. The first and most important rule: HTTP connections are always initiated by the client. Another rule: XHR returns the entire content-body or fails.
The implications for your workflow is that each part of the multipart response must be requested individually by the client.
From what I read, I am in doubbt of that content-type is right for the
response. multipart/mixed? multipart/digest?
You should be in doubt as there is no provision in the specfication to do this. The response-type attribute is limited to the empty string (default), "arraybuffer", "blob", "document", "json", and "text". it is possible to set the override MIME type header, but that does not change the response type. Event given that case, the XHR spec is very clear about what it will send back. It is one of the types listed above as documented here.
Can I use an accept
request header to identify if the client is able to handle the new
service implementation? If so, what is the right accept header for
this? My plan is to use the same endpoint but very accept header.
Custom HTTP headers are designed to assist us in telling the server what our capabilities are on the client. This is easily done. it doesn't necessarily have to be in the accept header (as that also is a defined list of MIME types).
How
can I develop a XHR client that is able to process many parts of a
single response as soon as they are available? I found some ideias on
the Web but I am not entirely confident with then.
XHR is processed natively by the client and cannot be overridden for all sorts of security reasons. So this is unlikely to be available as a solution for this reason.
Note: ordinarily one might suggest the use of a custom version of Chromium, but your constraints do not allow for that.

Can JavaScript do a POST HTTP request to any domain?

I know it's possible to load any kind of document from any domain from JavaScript (without necessarily being able to peek at its content), but it usually concerns regular GET requests. What about POST?
Is it possible to make an HTTP POST request from JavaScript to any domain name? (I'm specifically interested in form submissions.)
If so, how?

As per some answers on a nearby question, «HTTP GET request in JavaScript?», you might use XMLHttpRequest, since, according to the docs, the POST method is supported, too.
http://www.w3.org/TR/XMLHttpRequest/
https://developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest
A sample code from the above w3.org document:
function log(message) {
var client = new XMLHttpRequest();
client.open("POST", "/log");
client.setRequestHeader("Content-Type", "text/plain;charset=UTF-8");
client.send(message);
}
However, it would seem like in order for it to work with POST requests to domains unrelated to yours (where instead of "/log", a complete http or https URL is specified), the Cross-Origin Resource Sharing may have to be supported and enabled on the target server, as per https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS#Simple_requests.
So, it seems like, at least through XMLHttpRequest, you cannot make form submissions through POST requests (in fact, looks like even GET requests won't fly, either).

Why serve 1x1 pixel GIF (web bugs) data at all?

Many analytic and tracking tools are requesting 1x1 GIF image (web bug, invisible for the user) for cross-domain event storing/processing.
Why to serve this GIF image at all? Wouldn't it be more efficient to simply return some error code such as 503 Service Temporary Unavailable or empty file?
Update: To be more clear, I'm asking why to serve GIF image data when all information required has been already sent in request headers. The GIF image itself does not return any useful information.

Doug's answer is pretty comprehensive; I thought I'd add in an additional note (at the OP's request, off of my comment)
Doug's answer explains why 1x1 pixel beacons are used for the purpose they are used for; I thought I'd outline a potential alternative approach, which is to use HTTP Status Code 204, No Content, for a response, and not send an image body.
204 No Content
The server has fulfilled the request
but does not need to return an
entity-body, and might want to return
updated metainformation. The response
MAY include new or updated
metainformation in the form of
entity-headers, which if present
SHOULD be associated with the
requested variant.
Basically, the server receives the request, and decides to not send a body (in this case, to not send an image). But it replies with a code to inform the agent that this was a conscious decision; basically, its just a shorter way to respond affirmatively.
From Google's Page Speed documentation:
One popular way of recording page
views in an asynchronous fashion is to
include a JavaScript snippet at the
bottom of the target page (or as an
onload event handler), that notifies a
logging server when a user loads the
page. The most common way of doing
this is to construct a request to the
server for a "beacon", and encode all
the data of interest as parameters in
the URL for the beacon resource. To
keep the HTTP response very small, a
transparent 1x1-pixel image is a good
candidate for a beacon request. A
slightly more optimal beacon would use
an HTTP 204 response ("no content")
which is marginally smaller than a 1x1
GIF.
I've never tried it, but in theory it should serve the same purpose without requiring the gif itself to be transmitted, saving you 35 bytes, in the case of Google Analytics. (In the scheme of things, unless you're Google Analytics serving many trillions of hits per day, 35 bytes is really nothing.)
You can test it with this code:
var i = new Image();
i.src = "http://httpstat.us/204";

First, i disagree with the two previous answers--neither engages the question.
The one-pixel image solves an intrinsic problem for web-based analytics apps (like Google Analytics) when working in the HTTP Protocol--how to transfer (web metrics) data from the client to the server.
The simplest of the methods described by the Protocol, the simplest (at lest the simplest method that includes a request body) is the GET request. According to this Protocol method, clients initiate requests to servers for resources; servers process those requests and return appropriate responses.
For a web-based analytics app, like GA, this uni-directional scheme is bad news, because it doesn't appear to allow a server to retrieve data from a client on demand--again, all servers can do is supply resources not request them.
So what's the solution to the problem of getting data from the client back to the server? Within the HTTP context there are other Protocol methods other than GET (e.g., POST) but that's a limited option for many reasons (as evidenced by its infrequent and specialized use such as submitting form data).
If you look at a GET Request from a browser, you'll see it is comprised of a Request URL and Request Headers (e.g., Referer and User-Agent Headers), the latter contains information about the client--e.g., browser type and version, browser langauge, operating system, etc.
Again, this is part of the Request that the client sends to the server. So the idea that motivates the one-pixel gif is for the client to send the web metrics data to the server, wrapped inside a Request Header.
But then how to get the client to Request a resource so it can be "tricked" into sending the metrics data? And how to get the client to send the actual data the server wants?
Google Analytics is a good example: the ga.js file (the large file whose download to the client is triggered by a small script in the web page) includes a few lines of code that directs the client to request a particular resource from a particular server (the GA server) and to send certain data wrapped in the Request Header.
But since the purpose of this Request is not to actually get a resource but to send data to the server, this resource should be a small as possible and it should not be visible when rendered in the web page--hence, the 1 x 1 pixel transparent gif. The size is the smallest size possible, and the format (gif) is the smallest among the image formats.
More precisely, all GA data--every single item--is assembled and packed into the Request URL's query string (everything after the '?'). But in order for that data to go from the client (where it is created) to the GA server (where it is logged and aggregated) there must be an HTTP Request, so the ga.js (google analytics script that's downloaded, unless it's cached, by the client, as a result of a function called when the page loads) directs the client to assemble all of the analytics data--e.g., cookies, location bar, request headers, etc.--concatenate it into a single string and append it as a query string to a URL (*http://www.google-analytics.com/__utm.gif*?) and that becomes the Request URL.
It's easy to prove this using any web browser that has allows you to view the HTTP Request for the web page displayed in your browser (e.g., Safari's Web Inspector, Firefox/Chrome Firebug, etc.).
For instance, i typed in valid url to a corporate home page into my browser's location bar, which returned that home page and displayed it in my browser (i could have chosen any web site/page that uses one of the major analytics apps, GA, Omniture, Coremetrics, etc.)
The browser i used was Safari, so i clicked Develop in the menu bar then Show Web Inspector. On the top row of the Web Inspector, click Resources, find and click the utm.gif resource from the list of resources shown on the left-hand column, then click the Headers tab. That will show you something like this:
Request URL:http://www.google-analytics.com/__utm.gif?
utmwv=1&utmn=1520570865&
utmcs=UTF-8&
utmsr=1280x800&
utmsc=24-bit&
utmul=enus&
utmje=1&
utmfl=10.3%20r181&
Request Method:GET
Status Code:200 OK
Request Headers
User-Agent:Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/533.21.1
(KHTML, like Gecko) Version/5.0.5 Safari/533.21.1
Response Headers
Cache-Control:private, no-cache, no-cache=Set-Cookie, proxy-revalidate
Content-Length:35
Content-Type:image/gif
Date:Wed, 06 Jul 2011 21:31:28 GMT
The key points to notice are:
The Request was in fact a request
for the utm.gif, as evidenced by the
first line above: *Request
URL:http://www.google-analytics.com/__utm.gif*.
The Google Analytics parameters are clearly visible in the query string
appended to the Request URL: e.g.,
utmsr is GA's variable name to refer to the client screen
resolution, for me, shows a value of
1280x800; utmfl is the variable
name for flash version, which has a
value of 10.3, etc.
The Response Header called
Content-Type (sent by the server back to the client) also confirms
that the resource requested and
returned was a 1x1 pixel gif:
Content-Type:image/gif
This general scheme for transferring data between a client and a server has been around forever; there could very well be a better way of doing this, but it's the only way i know of (that satisfies the constraints imposed by a hosted analytics service).

Some browsers may display an error icon if the resource could not load. It makes debugging/monitoring the service also a little bit more complicated, you have to make sure that your monitoring tools treat the error as a good result.
OTOH you don't gain anything. The error message returned by the server/framework is typically bigger then the 1x1 image. This means you increase your network traffic for basically nothing.

Because such a GIF has a known presentation in a browser - it's a single pixel, period. Anything else presents a risk of visually interfering with the actual content of the page.
HTTP errors could appear as oversized frames of error text or even as a pop-up window. Some browsers may also complain if they receive empty replies.
In addition, in-page images are one of the very few data types allowed by default in all broswers. Anything else may require explicit user action to be downloaded.

This is to answer the OP's question - "why to serve GIF image data..."
Some users will put a simple img tag to call your event logging service -
<img src="http://www.example.com/logger?event_id=1234">
In this case, if you don't serve an image, the browser will show a placeholder icon that will look ugly and give the impression that your service is broken!
What I do is, look for the Accept header field. When your script is called via an img tag like this, you will see something like following in the header of the request -
Accept: image/gif, image/*
Accept-Encoding:gzip,deflate
...
When there is "image/"* string in the Accept header field, I supply the image, otherwise I just reply with 204.

Well the major reason is to attach the cookie to it so if users go from one side to another we still have the same element to attach cookie to.

#Maciej Perliński is basically correct, but I feel a detailed answer will be beneficial.
why 1x1 GIF and not a 204 No-Content status code?
204 No-Content enables the server to omit all response headers (Content-Type, Content-Length, Content-Encoding, Cache-Control etc...) and return an empty response body with 0 bytes (and saving a lot of unneeded bandwidth).
Browsers know to respect 204 No-Content responses, and not to expect/wait for response headers and response body.
if the server needs to set any response header (e.g. cache-control or cookie), he cannot use 204 No-Content because browsers will ignore any response header by design (according to the HTTP protocol spec).
why 1x1 GIF and not a Content-Length: 0 header with 200 OK status code?
Probably a mix of several issues, just to name a few:
legacy browsers compatibility
MIME type checks on browsers, 0 bytes is not a valid image.
200 OK with 0 bytes might not be fully supported by intermediate proxy servers and VPNs

You don't have to serve an image if you are using the Beacon API (https://w3c.github.io/beacon/) implementation method.
An error code would work if you have access to the log files of your server. The purpose of serving the image is to obtain more data about the user than you normally would with a log file.