Cannot Upload File to FastAPI backend using Fetch API in the frontend - javascript

I'm trying to figure out how to send an image to my API, and also verify a generated token that is in the header of the request.
So far this is where I'm at:
#app.post("/endreProfilbilde")
async def endreProfilbilde(request: Request,file: UploadFile = File(...)):
token=request.headers.get('token')
print(token)
print(file.filename)
I have another function that triggers the change listener and upload function, passing the parameter: bildeFila
function lastOpp(bildeFila) {
var myHeaders = new Headers();
let data = new FormData();
data.append('file',bildeFila)
myHeaders.append('token', 'SOMEDATAHERE');
myHeaders.append('Content-Type','image/*');
let myInit = {
method: 'POST',
headers: myHeaders,
cache: 'default',
body: data,
};
var myRequest = new Request('http://127.0.0.1:8000/endreProfilbilde', myInit);
fetch(myRequest)//more stuff here, but it's irrelevant for the Q
}
The Problem:
This will print the filename of the uploaded file, but the token isn't passed and is printed as None. I suspect this may be due to the content-type, or that I'm trying to force FastAPI to do something that is not meant to be doing.

As per the documentation:
Warning: When using FormData to submit POST requests using XMLHttpRequest or the Fetch_API with the
multipart/form-data Content-Type (e.g. when uploading Files and
Blobs to the server), do not explicitly set the Content-Type
header on the request. Doing so will prevent the browser from being
able to set the Content-Type header with the boundary expression
it will use to delimit form fields in the request body.
Hence, you should remove the Content-Type header from your code. The same applies to sending requests through Python Requests, as described here and here. Read more about the boundary in multipart/form-data.
Working examples on how to upload file(s) using FastAPI in the backend and Fetch API in the frontend can be found here, here, as well as here and here.

So I figured this one out thanks to a helpful lad in Python's Discord server.
function lastOpp(bildeFila) {
let data = new FormData();
data.append('file',bildeFila)
data.append('token','SOMETOKENINFO')
}
#app.post("/endreProfilbilde")
async def endreProfilbilde(token: str = Form(...),file: UploadFile = File(...)):
print(file.filename)
print(token)
Sending the string value as part of the formData rather than as a header lets me grab the parameter.

Related

FastAPI returns "Error 422: Unprocessable entity" when I send multipart form data with JavaScript Fetch API

I have some issue with using Fetch API JavaScript method when sending some simple formData like so:
function register() {
var formData = new FormData();
var textInputName = document.getElementById('textInputName');
var sexButtonActive = document.querySelector('#buttonsMW > .btn.active');
var imagesInput = document.getElementById('imagesInput');
formData.append('name', textInputName.value);
if (sexButtonActive != null){
formData.append('sex', sexButtonActive.html())
} else {
formData.append('sex', "");
}
formData.append('images', imagesInput.files[0]);
fetch('/user/register', {
method: 'POST',
data: formData,
})
.then(response => response.json());
}
document.querySelector("form").addEventListener("submit", register);
And on the server side (FastAPI):
#app.post("/user/register", status_code=201)
def register_user(name: str = Form(...), sex: str = Form(...), images: List[UploadFile] = Form(...)):
try:
print(name)
print(sex)
print(images)
return "OK"
except Exception as err:
print(err)
print(traceback.format_exc())
return "Error"
After clicking on the submit button I get Error 422: Unprocessable entity. So, if I'm trying to add header Content-Type: multipart/form-data, it also doesn't help cause I get another Error 400: Bad Request. I want to understand what I am doing wrong, and how to process formData without such errors?
The 422 response body will contain an error message about which field(s) is missing or doesn’t match the expected format. Since you haven't provided that (please do so), my guess is that the error is triggered due to how you defined the images parameter in your endpoint. Since images is expected to be a List of File(s), you should instead define it using the File type instead of Form. For example:
images: List[UploadFile] = File(...)
^^^^
When using UploadFile, you don't have to use File() in the default value of the parameter. Hence, the below should also work:
images: List[UploadFile]
Additionally, in the frontend, make sure to use the body (not data) parameter in the fetch() function to pass the FormData object (see example in MDN Web Docs). For instance:
fetch('/user/register', {
method: 'POST',
body: formData,
})
.then(res => {...
Please have a look at this answer, as well as this answer, which provide working examples on how to upload multiple files and form data to a FastAPI backend, using Fetch API in the frontend.
As for manually specifying the Content-Type when sending multipart/form-data, you don't have to (and shouldn't) do that, but rather let the browser set the Content-Type—please take a look at this answer for more details.
So, I found that I has error in this part of code:
formData.append('images', imagesInput.files[0]);
Right way to upload multiple files is:
for (const image of imagesInput.files) {
formData.append('images', image);
}
Also, we should use File in FastAPI method arguments images: List[UploadFile] = File(...) (instead of Form) and change data to body in JS method. It's not an error, cause after method called, we get right type of data, for example:
Name: Bob
Sex: Man
Images: [<starlette.datastructures.UploadFile object at 0x7fe07abf04f0>]

how do I send a request using multipart/form-data?

I have an app in ReactJs, using Axios and Papaparse.
I have a page where a user drop a csv file in a box, I automatically download the csv, update and make some change in the data, and send a new csv file to a server.
I did all until I arrive to the part where I need to create a new csv, and upload it to the server.
Here is my code currently :
const data = papaparse.unparse(destinationUpdateData, {
header: true,
skipEmptyLines: true
});
// data is a string in csv format
const file = new File([data as BlobPart], "destination.csv", { type: "text/csv" });
// I get a File type.
const paramsDestination = {
project_id: this.props.projectId,
datastore_id: 'DESTINATIONS',
file: file,
id: ["1", "2","3"]
}
// the data I would like to send is build
Axios.post(`new_item_file_attachment`, params, {headers: {"Content-Type": "multipart/form-data"}})
//I send to the server
The thing is, my server is expecting a request with a content Type of multipart/form-data, but I don't get how manually set my parameter to match this type.
The api call currently don't work, because the data is passed like a json, and the server reject it.
Is it possible to achieve it ?
I tried using FormData, but I can't see how to send boolean and array
Not 100% familiar with Axios but it should be something like this:
var params = new FormData();
params.append("project_id", this.props.projectId);
params.append("datastore_id", 'DESTINATIONS');
params.append("file", file);
params.append("id", JSON.stringify(["1", "2","3"])); // Arrays must be stringified
Axios.post(`new_item_file_attachment`, params)
You definitely need to put everything in FormData object. Last time I was doing this, I also had to remove the "Content-Type": "multipart/form-data" from the header. I believe the correct header should get filled in automatically. Try it with and without that stating a header and let me know if either works.
Here is my solution.
const data = new FormData();
data.append("project_id", id);
data.append("file", file);
axios.post(url, data);
Try and comments when some errors occur.

How to extract the Content-Type of a file sent via multipart/form-data

I receive a Request in my cloudflare worker and want to upload the data to Google cloud storage. My problem is that I can't extract the content type from the multipart/form-data data I receive in order to upload it with the correct content type to GCS.
When I read the request with await req.formData() I can get('file')from the formData and it returns the raw file data that I need for the GCS, but I can't seem to see anywhere the file content-type that I need (I can see it only when looking at the raw Request body).
Here is my (striped down) code :
event.respondWith((async () => {
const req = event.request
const formData = await req.formData()
const file = formData.get('file')
const filename = formData.get('filename')
const oauth = await getGoogleOAuth()
const gcsOptions = {
method: 'PUT',
headers: {
Authorization: oauth.token_type + ' ' + oauth.access_token,
'Content-Type': 'application/octet-stream' //this should by `'Content-Type': file.type`
},
body: file,
}
const gcsRes = await fetch(
`https://storage.googleapis.com/***-media/${filename}`,
gcsOptions,
)
if (gcsRes.status === 200) {
return new Response(JSON.stringify({filename}), gcsRes)
} else {
return new Response('Internal Server Error', {status: 500, statusText: 'Internal Server Error'})
}
})())
Reminder - the code is part of our cloudflare worker code.
It seems to me this should be straight forward, determining the type of file you extract from the multipart/form-data data.
Am I missing something?
Unfortunately, as of this writing, the Cloudflare Workers implementation of FormData is incomplete and does not permit extracting the Content-Type. In fact, it appears our implementation currently interprets all entries as text and return strings, which means binary content will be corrupted. This is a bug which will require care to fix since we don't want to break already-deployed scripts that might rely on the buggy behavior.
Thanks Kenton for your response.
What I ended up doing:
As the Cloudflare Workers don't support the multipart/form-data of Blob or any type other than String, I ended up using the raw bytes in the ArrayBuffer data type. After converting it to an Uint8Array I parsed it byte by byte to determine the file type and the start and end indexes of the file data. Once I found the start and end of the transferred file I was able create an array of the file data, add it to the request and send it to the GCS as I showed above.

Image upload to Imgur API via JS Fetch API is canceled by browser

I wrote a small application on Codepen to upload to a specific Imgur album using Javascript's Fetch API. The application is failing to upload, and I'm not sure why since I'm not actually getting a response from the server. The browser appears to be canceling the request rather than sending it off.
Here's my code:
const form = document.querySelector('form');
form.addEventListener('submit', event => {
const fileInput = event.target.querySelector('input');
const imageFile = fileInput.files[0];
const formData = new FormData();
formData.append('image', imageFile);
formData.append('album', 'EVGILvAjGfArJFI');
fetch('https://api.imgur.com/3/image', {
method: 'POST',
headers: new Headers({
'Content-Type': 'multipart/form-data',
Authorization: 'Client-ID 90ef1830bd083ba',
}),
body: formData
}).then(response => {
if (response.ok) {
alert('Image uploaded to album');
}
}).catch(error => {
console.error(JSON.stringify(error));
alert('Upload failed: ' + error);
});
});
I'm not sure if I need to be reading the file or if I can pass the file object directly into the form data. I've seen examples of both methods online, and I've tried both with the same result. I notice dev tools doesn't seem to show the image data in the request body. Not sure if that means it isn't being sent or if Chrome is just not displaying it since it wouldn't be human-readable anyway. I've tried the request with CORS mode on and off.
Any guidance on what I'm doing wrong here?
For FormData, you don't actually need the Content-Type header. The browser automatically adds that for you when have a FormData type in the body. I think that's because that header needs some additional info (when the browser automatically handles it, it looks like multipart/form-data; boundary=----WebKitFormBoundaryvAmR7mCXOyHiPIH5 on my machine).
Here's the updated pen that looks like it's working:
https://codepen.io/ccnokes/pen/wyBYOd
Also, because you're submitting that data via a <form> but handling the request via fetch, you need to tell the <form> not to do a full page navigation change to POST the form (which cancels the fetch). So I added event.preventDefault() in the submit handler on that form.

cordova-plugin-file-transfer: How do you upload a file to S3 using a signed URL?

I am able to upload to S3 using a file picker and regular XMLHttpRequest (which I was using to test the S3 setup), but cannot figure out how to do it successfully using the cordova file transfer plugin.
I believe it is either to do with the plugin not constructing the correct signable request, or not liking the local file uri given. I have tried playing with every single parameter from headers to uri types, but the docs aren't much help, and the plugin source is bolognese.
The string the request needs to sign match is like:
PUT
1391784394
x-amz-acl:public-read
/the-app/317fdf654f9e3299f238d97d39f10fb1
Any ideas, or possibly a working code example?
A bit late, but I just spent a couple of days struggling with this so in case anybody else is having problems, this is how managed to upload an image using the javascript version of the AWS SDK to create the presigned URL.
The key to solving the problem is in the StringToSign element of the XML SignatureDoesNotMatch error that comes back from Amazon. In my case it looked something like this:
<StringToSign>
PUT\n\nmultipart/form-data; boundary=+++++org.apache.cordova.formBoundary\n1481366396\n/bucketName/fileName.jpg
</StringToSign>
When you use the aws-sdk to generate a presigned URL for upload to S3, internally it will build a string based on various elements of the request you want to make, then create an SHA1 hash of it using your AWS secret. This hash is the signature that gets appended to the URL as a parameter, and what doesn't match when you get the SignatureDoesNotMatch error.
So you've created your presigned URL, and passed it to cordova-plugin-file-transfer to make your HTTP request to upload a file. When that request hits Amazon's server, the server will itself build a string based on the request headers etc, hash it and compare that hash to the signature on the URL. If the hashes don't match then it returns the dreaded...
The request signature we calculated does not match the signature you provided. Check your key and signing method.
The contents of the StringToSign element I mentioned above is the string that the server builds and hashes to compare against the signature on the presigned URL. So to avoid getting the error, you need to make sure that the string built by the aws-sdk is the same as the one built by the server.
After some digging about, I eventually found the code responsible for creating the string to hash in the aws-sdk. It is located (as of version 2.7.12) in:
node_modules/aws-sdk/lib/signers/s3.js
Down the bottom at line 168 there is a sign method:
sign: function sign(secret, string) {
return AWS.util.crypto.hmac(secret, string, 'base64', 'sha1');
}
If you put a console.log in there, string is what you're after. Once you make the string that gets passed into this method the same as the contents of StringToSign in the error message coming back from Amazon, the heavens will open and your files will flow effortlessly into your bucket.
On my server running node.js, I originally created my presigned URL like this:
var AWS = require('aws-sdk');
var s3 = new AWS.S3(options = {
endpoint: 'https://s3-eu-west-1.amazonaws.com',
accessKeyId: "ACCESS_KEY",
secretAccessKey: "SECRET_KEY"
});
var params = {
Bucket: 'bucketName',
Key: imageName,
Expires: 60
};
var signedUrl = s3.getSignedUrl('putObject', params);
//return signedUrl
This produced a signing string like this, similar to the OP's:
PUT
1481366396
/bucketName/fileName.jpg
On the client side, I used this presigned URL with cordova-plugin-file-transfer like so (I'm using Ionic 2 so the plugin is wrapped in their native wrapper):
let success = (result: any) : void => {
console.log("upload success");
}
let failed = (err: any) : void => {
let code = err.code;
alert("upload error - " + code);
}
let ft = new Transfer();
var options = {
fileName: filename,
mimeType: 'image/jpeg',
chunkedMode: false,
httpMethod:'PUT',
encodeURI: false,
};
ft.upload(localDataURI, presignedUrlFromServer, options, false)
.then((result: any) => {
success(result);
}).catch((error: any) => {
failed(error);
});
Running the code produced the signature doesn't match error, and the string in the <StringToSign> element looks like this:
PUT
multipart/form-data; boundary=+++++org.apache.cordova.formBoundary
1481366396
/bucketName/fileName.jpg
So we can see that cordova-plugin-file-transfer has added in its own Content-Type header which has caused a discrepancy in the signing strings. In the docs relating to the options object that get passed into the upload method it says:
headers: A map of header name/header values. Use an array to specify more than one value. On iOS, FireOS, and Android, if a header named Content-Type is present, multipart form data will NOT be used. (Object)
so basically, if no Content-Type header is set it will default to multipart form data.
Ok so now we know the cause of the problem, it's a pretty simple fix. On the server side I added a ContentType to the params object passed to the S3 getSignedUrl method:
var params = {
Bucket: 'bucketName',
Key: imageName,
Expires: 60,
ContentType: 'image/jpeg' // <---- content type added here
};
and on the client added a headers object to the options passed to cordova-plugin-file-transfer's upload method:
var options = {
fileName: filename,
mimeType: 'image/jpeg',
chunkedMode: false,
httpMethod:'PUT',
encodeURI: false,
headers: { // <----- headers object added here
'Content-Type': 'image/jpeg',
}
};
and hey presto! The uploads now work as expected.
I run into such issues with this plugin
The only working way I found to upload a file with a signature is the method of Christophe Coenraets : http://coenraets.org/blog/2013/09/how-to-upload-pictures-from-a-phonegap-app-to-amazon-s3/
With this method you will be able to upload your files using the cordova-plugin-file-transfer
First, I wanted to use the aws-sdk on my server to sign with getSignedUrl()
It returns the signed link and you only have to upload to it.
But, using the plugin it always end with 403 : signatures don't match
It may be related to the content length parameter but I didn't found for now a working solution with aws-sdk and the plugin

Categories