I am trying to download large files (3 gigs) using a post request to my backend. The post request is done to hide the file system from web scraping. It seems like some of the download initially is loaded into memory because the file doesn't initially start downloading, but my ram spikes, then 30 seconds later it begins downloading and slightly lower ram usage. Here is the code I am using.
fetch("http://localhost:5000/api/send", {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({ "root": root, "path": path, "name": name })
})
.then(response => response.blob())
.then(blob => {
var url = window.URL.createObjectURL(blob)
var a = document.createElement("a")
a.href = url
a.download = name
document.body.appendChild(a)
a.click()
a.remove()
})
Is there anyway to implement this without a ram/performance hit? Downloading the file as a GET request and using something like window.location.href = url works no problem, but I would prefer not to use that, and have difficulties specifying file names with symbols like "&" in the URL parameter.
Any ideas are appreciated!
Your server respond with a blob, so it’s downloaded into RAM then referred to as an object url on client side.
You see what the client side really need is a url. You don’t have to create it from a blob. Instead you can create a hashed url on server side like "bit.ly/whatever", respond with that url, then client code do the same trick.
Related
I am getting file link from backend and I need to download the file directly or ask to savsAs on click a save button.
This is the link of one file.
https://unify.eoxyslive.com/images/jobs/image_2023_01_28T09_33_42_235Z.png
I have tried this method to download file and its downloading but its corrupted. and not opening.
const FileDownload = (link, fileName) => {
fetch(link, { method: "get", mode: "no-cors", referrerPolicy: "no-referrer" })
.then((res) => res.blob())
.then((res) => {
const aElement = document.createElement("a");
aElement.setAttribute("download", fileName);
const href = URL.createObjectURL(res);
console.log(href);
aElement.href = href;
aElement.setAttribute("target", "_blank");
// aElement.click();
URL.revokeObjectURL(href);
});
};
FileDownload("https://unify.eoxyslive.com/images/jobs/image_2023_01_28T09_33_42_235Z.png", "attachment.png")
The easiest way to trigger a download is a download link, JavaScript-free.
<a href="file.ext" download>
(the attribute can also be used with a value, to suggest a filename other than the href)
But that can only do a simple GET request with no extra headers.
As soon as you need headers (for authentication, etc.), you will have to use fetch and create a download link (not a link with target=_blank as in your example ; that works but is inferior to download functionally and semantically) to a Blob URL.
A workaround for making download links work despite the need for headers (or another method than GET) is to have a Service Worker handle the extra headers transparently.
Also, if you have any control on the server, make it send a Content-Disposition: attachement response header. That will encourage the browser to trigger a download even with a normal link.
This issue I'm having could be literally anything in my workflow, but I will start with this scope for now...
I have a rest api that you send a JSON structure and it sends back an excel file, but I can't get it to work probably in the browser/Javascript.
My Javascript code in the browser is basically the following:
fetch('myapiurl')
.then(response => response.blob)
.then(blob => downloadBlob(blob))
function downloadBlob(blob){
const a = document.createElement("a");
a.href = URL.createObjectURL(blob);
a.download = 'export.xlsx;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
}
But this generates a corrupted file, there's either something missing or I'm not sending the file in the right encoding.
Here's the python code from the API in the part where it sends the file (it's an AWS lambda):
output = BytesIO()
#Code that puts an excel file in output
return {
'statusCode' : 200,
'headers' : {
'Content-Disposition': 'attachment; filename="export.xlsx"',
'Content-Type': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
'Access-Control-Allow-Origin': '*'
},
#I can't return raw bytes so I transform it into a string
'body' : output.getvalue().hex(),
}
Am I sending the file the wrong way?
I tried sending it as base64 but it still was corrupted
It works if I make a test locally using python and just bytes.fromhex() the api response and write it in a file in byte mode.
I wonder if AWS Api Gateway is automatically encoding my body as base64
Make this change and it might work
- .then(response => response.blob)
+ .then(response => response.blob())
Regarding the response body
#I can't return raw bytes so I transform it into a string
'body' : output.getvalue().hex(),
you can't send back a hex string, you need to send the raw data. otherwise you would have to convert the hex back to binary as well on the client side before making a blob and later a objectURL
As you may already know the browser will not save an attachment if you fetch it with ajax, what you have to do is "navigate" to the file to trigger the download. And since you are posting json data and converting it to excel you have to do a form submit to send the data over to the server instead of using ajax, since it's not a regular GET request?
<form hidden method="post" enctype="text/plain" action="url">
<textarea name="json">{"a":12}</textarea>
<input type="submit">
</form>
This would be better as you don't have to hold all data in the browsers memory before the file can be saved, besides, you can start saving the file much earlier
I am storing images in a static folder in my express server app. So far this works as expected. The code that matters for this is as follows
app.use('/uploads', express.static(Path.join(__dirname, `/../uploads`)))
Now from the client app, I need to do a GET request on these images and display them, as it stands, just doing a request on my image gives me unusable data which I don't know how to handle. Here is a sample of what I am talking about:
IHDRXZ��9�|iCCPICC Profile(�c``*I,(�aa``��+)
rwR���R`������ �`� ��\\��À|����/���J��yӦ�|�6��rV%:���wJjq2#���R��d
Is there a way I can get this data and use it as the image?
I have also read up an alternative way of sending images over the wire between the client and server, which would be to set up my GET request. From this thread Basically using express' .sendFile, this however does not work as I get a 404 not found. Is there something else wrong?
Anyway, I guess the question is how can I get the files in a format that I can display them on my client app?
This is what my client code looks like:
return isomorphicFetch(`${MY_URL}/${imageId}`, {
headers: {
Accept: 'application/json'
}
method: 'GET'
})
.then(res => {
console.log('My fetched image', res)
return res
})
Why not just use an <img> element?
const img = document.createElement('img');
img.src = `${MY_URL}/${imageId}`;
someplace.appendChild(img);
BTW if, for some reason, such as sending custom HTTP headers, you want to use fetch():
fetch(`${MY_URL}/${imageId}` /*...*/)
.then(res => res.blob())
.then(blob => {
const img = document.createElement('img');
img.src = URL.createObjectURL(blob);
somewhere.appendChild(img);
});
I am trying to attach an image file. Somehow my client side didnt send anything if attaching a file there.
It's on React(website) and React-Native Web View(rendering).
Bad part of the game is, I cant track the client log, cause of mobile browser issue.
1)
const formData = new FormData();
formData.append('report[description]', text);
formData.append('token', localStorage.getItem('tempToken'));
formData.append('report[image]', new Blob([this.state.file], { type: 'image/png' }));
axios.post('/accept_report', formData)
2)
const config = {
headers: {
'content-type': 'multipart/form-data'
}
}
formData.append('report[image]', this.state.file);
axios.post("/accept_report", formData, config)
Solved:
Since, It happens only when attaching the file. My Production nginx was restricting big sized files. I was only able to upload around 500kb not over.
Have changed nginx client body size.
http{
#...
client_max_body_size 10M; #or 100M it depends on yr preference.
#...
}
I have a post fetch request coming from my React client to my remote Flask server like so:
fetch(FETCH_URL, {
method: 'POST',
body: data,
headers: {
'Content-Type': 'application/json'
}
}).then((response) => {
var a = response.body.getReader();
a.read().then(({ done, value }) => {
console.log(new TextDecoder("utf-8").decode(value));
}
);
});
response.body comes in the form of a ReadableStream object so I was able to extract a Uint8array which I then decoded to be the contents of the txt file I sent back from my flask server as shown in the code above.
At this point I'm lost, what I'm trying to do is send a request to my remote server with a filename (in the requests' data), and download that file on my computer.
As shown above, I tried a fetch request to my remote server, then in flask, my server finds and opens the file which is stored on the server itself, and sends back the file.
filename = request.get_json()['filename']
f = open(filename)
return f
The problem now is that from what I've read, I can't create a file on my computer just with react. Even so, I don't know if this would work with all types of files or just txt files. Does anyone have any guidance to get to my end goal of downloading a file from a remote flask server.
If your requirement is to create a file with data you received from the response. The below solution should work.
Create the blob object with the text you received
Create Blob Object URL for that blob
Trigger downloading the object using that URL
Since this is pure Javascript solution, it's independent of React or any library you use.
Solution:
fetch(FETCH_URL, {
method: 'POST',
body: data,
headers: {
'Content-Type': 'application/json'
}
}).then((response) => {
var a = response.body.getReader();
a.read().then(({ done, value }) => {
// console.log(new TextDecoder("utf-8").decode(value));
saveAsFile(new TextDecoder("utf-8").decode(value), 'filename');
}
);
});
function saveAsFile(text, filename) {
// Step 1: Create the blob object with the text you received
const type = 'application/text'; // modify or get it from response
const blob = new BlobBuilder([text], {type});
// Step 2: Create Blob Object URL for that blob
const url = URL.createObjectURL(blob);
// Step 3: Trigger downloading the object using that URL
const a = document.createElement('a');
a.href = url;
a.download = filename;
a.click(); // triggering it manually
}
Alternatively, you can use <Button href="${YOUR_FILE_URL}"/> to download the file sent by flask.
To add to Kamalakannan's post, if you are never going to use that element again make sure to removeChild() and revokeObjectURL() after triggering the click().