I have just finished a task on updating the database written in JSON from the front end.
I am still not clarified on the use of headers which is a type of AxiosConfig. What is the difference between using axios with it and without it.
For example:
What is the difference between
await axios.post(`${URL}/update`, JSON.stringify({urlSlug: apiHash}),
{
headers: {
'content-type': 'application/json'
}
})
).data
And
await axios.post(`${URL}/update`, {apiHash})).data
I tried both and only the first one works in my case. But, after finishing the task, I still don't really know the difference in how they work.
Headers are part of the HTTP request structure. A request consists of many parts including URL, headers and body. Not everything is considered suitable to be specified in the body like api-keys, tokens, etc. These can be included in the header.
There are some details that the service running in the back-end needs in order to process a request. If the back-end service expects the request containing JSON data in the body to specify its type in header as JSON, it will not work if this isn't specified as there may be no handlers for non JSON type request in the back-end service.
You can read more about HTTP request structure here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages
Related
Why do some fetch requests require you to not only include a URL but include an Object with a property like method or headers?
It seems like every time I've encountered this, the method properties value is always set to 'POST'. Are there instances where this is not true? What do the headers do behind the scenes?
fetch('https://.....', {
method: 'POST',
headers: {
'example': 'example',
'example2': 'example2'
}
}) ....
If you call fetch without the second argument, it will make a default GET call with standard headers.
If you want to make a POST/PUT/PATCH/DELETE call on the other hand, since these verbs send data to the server and cause a database change, you need to send some extra information.
Headers for these verbs usually, amongst the default entries, carry some authorization bearer token, and define the content type of the body.
The body of the request is the most important entry, since it will carry the data that you want send to the server.
Basically the headers are key-value entries that are attached to the HTTP request, and they carry additional information about the type of request:
A request header is an HTTP header that can be used in an HTTP request to provide information about the request context, so that the server can tailor the response. For example, the Accept-* headers indicate the allowed and preferred formats of the response. Other headers can be used to supply authentication credentials (e.g. Authorization), to control caching, or to get information about the user agent or referrer, etc.
MDN
I am trying to get the csv data from a url from the nrel developer website. Unfortunately I can't work with json data because it will send it to the email, while the csv file is immediate. I could achieve this using vba using the following:
With req
.Open "GET", parsedUrl, False
.SetRequestHeader "Content-Type", "text/csv;"
.Send
End With
result = Split(req.responseText, Chr(10))
'do work with result
However, working with javascript and the excel addins documentation I am not able to store the csv data on a variable.
I tried the following code
const res = await fetch(tmyUrl, {
method: "GET",
headers: {
"Content-Type": "text/csv;",
},
});
if (!res.ok) throw new Error("There was a problem trying to get the the weather data.");
But this gives me the following error:
has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
I also tried adding the following:
mode: "no-cors"
But the res.ok is false and the body seems to be empty.
It takes a little time to process this request, so I know it is doing something in the background.
I also could trigger the download using downloadjs but I don't need to download, I just need to temporary store the data in a variable.
Any help or advice please.
I also tried adding [mode: "no-cors"] but... the body seems to be empty.
This is precisely because you're calling the endpoint with mode: no-cors. This behavior is well-documented, and discussed at length in this SO thread. From the linked MDN page (emphasis mine):
no-cors — Prevents the method from being anything other than HEAD, GET or POST, and the headers from being anything other than simple headers. If any ServiceWorkers intercept these requests, they may not add or override any headers except for those that are simple headers. In addition, JavaScript may not access any properties of the resulting Response. This ensures that ServiceWorkers do not affect the semantics of the Web and prevents security and privacy issues arising from leaking data across domains.
The reason you're seeing CORS errors in the first place is that the server that's responding to your request has deliberately been configured this way to prevent these types of requests on its resources. You can bypass this by making the request to your own server (which allows CORS), which can then proxy the ultimate request to the service and return the response to your client. There used to be several prominent drop-in services that provided this functionality, but most have gone by the wayside due to service abuse (CORS Anywhere and Whatever Origin, to name two).
There still exists one service that appears to still proxy these types of requests - All Origins. Drop in your URL according to their documentation to effectively bypass this restriction:
const res = await fetch(`https://allorigins.win/get?url=${encodeURIComponent(tmyUrl)}`, {
method: "GET",
headers: {
"Content-Type": "text/csv;",
},
});
Be forewarned, you really, really need to trust this service to use this in any project. By routing your requests through their service, you are implicitly allowing them to snoop on your traffic uninhibited and potentially return malicious content to your clients. It's a much better idea to do this on your own server that you control.
I am trying to make a POST request to an AWS Lambda Python Script I have set up.
I have already heard about solving the issue of Access-Control-Allow-Origin by adding a header to the Response from AWS lambda, that looks like Access-Control-Allow-Origin:*, and I implemented this in the code I was working on for the get request. Then when I begun creating a POST request, I did the same thing, so that the new POST request originally looked like this:
def post(event, context):
try:
return dict(
statusCode=200,
headers= {'Access-Control-Allow-Origin': "*"},
body="test"
)
except Exception as ex:
return dict(
statusCode=500,
body=str(ex)
)
And this works, in Chrome's Network-Tab I see that we pass a post request and there is no issue, and the header is recognized properly, allowing Cross-Origin calls. But this is pretty much just the GET request I have which is already set up and working. So when I update it to be more similar to a POST METHOD and do what I want the problems begin.
So my code, that I want to be performed in the POST request is simple, I should be able to make the request like such,
def post(event, context):
try:
result=str(event.get("inputOne") + event.get("inputTwo"))
return dict(
statusCode=200,
headers= {'Access-Control-Allow-Origin': "*"},
body=result
)
except Exception as ex:
return dict(
statusCode=500,
body=str(ex)
)
When I test this in the AWS Lambda Management console, my code works, and I get the expected response back in the body.
But then when I try and call the POST endpoint from my React Site, I once again am getting the error:
Access to XMLHttpRequest at 'THE_POST_URL' from origin
'http://localhost:XXX' has been blocked by CORS policy: No 'Access-
Control-Allow-Origin' header is present on the requested resource.
So for some reason, when the body variable is not just a simple string in quotes like "this" <- it fails. I believe this has to do with the Content-Type in the request being of the form application/json;charset=UTF-8 while my request only accepts (as it says in dev-tools) application/json however even when trying to add the headers to accept this format for content-type, I am still getting the same issues.
I am not entirely sure why the header worked on the GET request, but not the POST request, and would appreciate anyone being able to help teach me, thank you!
EDIT: I have also tried adding 'Access-Control-Allow-Methods': "OPTIONS,GET,POST" to the headers of the post response, but this did not work either.
So after reading through the comments on my question, I decided it was worth going and taking a harder look at the way I had my API Gateway, and Lambda service set up on AWS.
I began using the curl utility in order to try and test my post-call from a terminal utility and found that there it was also failing, but I was able to use CloudWatch to debug, the issue and finally get it working.
The problem that I was facing with my curl Request was that I was not properly formatting the JSON input, but even when I was the Lambda was transforming the event improperly, not resulting in a Dict, as expected.
The curl call I was using can be seen:
curl -X POST https://myURL.execute-api.myREGION.amazonaws.com/stage/yourPOST \
-d '{"inputOne":80000,"inputTwo":85}'
But in order to read that input properly, and then work with it, I had to update my code in the Lambda to reflect parsing the input properly, this updated lambda code can be found here. The event object that you are passing to this lambda is a DICT object, so on the first line, we use JSON.loads to decode the "body" value of the event, into a dict that we store as body. Then to get the value of the two attributes, we use the command body.get("yourKey").
def yourPOST(event, context):
body=json.loads(event["body"])
a=body.get("inputOne")
c=body.get("inputTwo")
After making these changes the call from my React site works without error! If anyone has any questions feel free to comment, I hope this helped!
I'm totally new to file uploads....I'm using angular-file-upload which creates an XHR outside of angularjs to upload files to google cloud storage. When I try to upload I keep getting the error below. How can I resolve this?
400 Bad content type. Please use multipart.
Here's my controller setup:
var uploader = $scope.uploader = new FileUploader({
url: 'https://www.googleapis.com/upload/storage/v1/b/bucketsbuckets/o?uploadType=multipart',
headers : {
'Authorization': 'Bearer ya29.lgHmbYk-5FgxRElKafV4qdyWsdMjBFoO97S75p4vB0G0d6fryD5LASpf3JUY8Av9Yzhp9cQP8IQqhA',
'Content-Type': 'multipart/form-data'
},
autoUpload:true
});
For those who are using fetch just remove the content-type on your header.
I found this issue on github and I quote:
Setting the Content-Type header manually means it's missing the boundary parameter. Remove that header and allow fetch to generate the full content type. It will look something like this:
Content-Type: multipart/form-data;boundary=----WebKitFormBoundaryyrV7KO0BoCBuDbTL
Fetch knows which content type header to create based on the FormData object passed in as > the request body content.
^^ this one worked for me.
The problem is that the endpoint you're using is for multipart uploads, but not FORM-based multipart uploads. If you set your Content-Type to "multipart/related" instead of "multipart/form-data", you should be able to proceed.
A multipart upload to that endpoint, "www.googleapis.com/upload/storage/etc?uploadType=multipart", expects a multipart message with exactly two parts, the first part being the metadata of the object, in JSON, and the second part being the data.
More on these requirements are here: https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload
If, however, you'd like to do an upload in the style of a form submission, that's also possible, but the rules are different. In this case, you'll submit a POST to "storage.googleapis.com/BUCKET/OBJECT" with a variety of appropriate form parameters: https://cloud.google.com/storage/docs/reference-methods#postobject
Now, this all assumes you're trying to do an upload as a multipart request for some reason. That might be necessary if you need to set some specific metadata properties on the object, but if not, you may be making things harder for yourself. A simple PUT to "storage.googleapis.com/BUCKET/OBJECT" will work just fine (although I'm not familiar with angular-file-upload and don't know whether it supports non-form-style uploads).
I'm jusing jQuery's $.getJSON() function to return a short set of JSON data.
I've got the JSON data sitting on a url such as example.com.
I didn't realize it, but as I was accessing that same url, the JSON data couldn't be loaded. I followed through the console and found that XMLHttpRequest couldn't load due to Access-Control-Allow-Origin.
Now, I've read through, a lot of sites that just said to use $.getJSON() and that would be the work around, but obviously it didn't work. Is there something I should change in the headers or in the function?
Help is greatly appreciated.
It's simple, use $.getJSON() function and in your URL just include
callback=?
as a parameter. That will convert the call to JSONP which is necessary to make cross-domain calls. More info: http://api.jquery.com/jQuery.getJSON/
You may well want to use JSON-P instead (see below). First a quick explanation.
The header you've mentioned is from the Cross Origin Resource Sharing standard. Beware that it is not supported by some browsers people actually use, and on other browsers (Microsoft's, sigh) it requires using a special object (XDomainRequest) rather than the standard XMLHttpRequest that jQuery uses. It also requires that you change server-side resources to explicitly allow the other origin (www.xxxx.com).
To get the JSON data you're requesting, you basically have three options:
If possible, you can be maximally-compatible by correcting the location of the files you're loading so they have the same origin as the document you're loading them into. (I assume you must be loading them via Ajax, hence the Same Origin Policy issue showing up.)
Use JSON-P, which isn't subject to the SOP. jQuery has built-in support for it in its ajax call (just set dataType to "jsonp" and jQuery will do all the client-side work). This requires server side changes, but not very big ones; basically whatever you have that's generating the JSON response just looks for a query string parameter called "callback" and wraps the JSON in JavaScript code that would call that function. E.g., if your current JSON response is:
{"weather": "Dreary start but soon brightening into a fine summer day."}
Your script would look for the "callback" query string parameter (let's say that the parameter's value is "jsop123") and wraps that JSON in the syntax for a JavaScript function call:
jsonp123({"weather": "Dreary start but soon brightening into a fine summer day."});
That's it. JSON-P is very broadly compatible (because it works via JavaScript script tags). JSON-P is only for GET, though, not POST (again because it works via script tags).
Use CORS (the mechanism related to the header you quoted). Details in the specification linked above, but basically:
A. The browser will send your server a "preflight" message using the OPTIONS HTTP verb (method). It will contain the various headers it would send with the GET or POST as well as the headers "Origin", "Access-Control-Request-Method" (e.g., GET or POST), and "Access-Control-Request-Headers" (the headers it wants to send).
B. Your PHP decides, based on that information, whether the request is okay and if so responds with the "Access-Control-Allow-Origin", "Access-Control-Allow-Methods", and "Access-Control-Allow-Headers" headers with the values it will allow. You don't send any body (page) with that response.
C. The browser will look at your response and see whether it's allowed to send you the actual GET or POST. If so, it will send that request, again with the "Origin" and various "Access-Control-Request-xyz" headers.
D. Your PHP examines those headers again to make sure they're still okay, and if so responds to the request.
In pseudo-code (I haven't done much PHP, so I'm not trying to do PHP syntax here):
// Find out what the request is asking for
corsOrigin = get_request_header("Origin")
corsMethod = get_request_header("Access-Control-Request-Method")
corsHeaders = get_request_header("Access-Control-Request-Headers")
if corsOrigin is null or "null" {
// Requests from a `file://` path seem to come through without an
// origin or with "null" (literally) as the origin.
// In my case, for testing, I wanted to allow those and so I output
// "*", but you may want to go another way.
corsOrigin = "*"
}
// Decide whether to accept that request with those headers
// If so:
// Respond with headers saying what's allowed (here we're just echoing what they
// asked for, except we may be using "*" [all] instead of the actual origin for
// the "Access-Control-Allow-Origin" one)
set_response_header("Access-Control-Allow-Origin", corsOrigin)
set_response_header("Access-Control-Allow-Methods", corsMethod)
set_response_header("Access-Control-Allow-Headers", corsHeaders)
if the HTTP request method is "OPTIONS" {
// Done, no body in response to OPTIONS
stop
}
// Process the GET or POST here; output the body of the response
Again stressing that this is pseudo-code.