Here is my workflow as of now:
In a button click event, I have search results being exported to a .csv file, which is saved to the server. Once the file is saved, I want to send it for download to the browser. Using this question How to handle conditional file downloads in meteor.js, I created a method that is called after the method that saves the file returns. Here is that method:
exportFiles: function(file_to_export) {
console.log("to export = "+file_to_export);
Meteor.Router.add('/export', 'GET', function() {
console.log('send '+file_to_export+' to browser');
return [200,
{
'Content-type': 'text/plain',
'Content-Disposition': "attachment; filename=" + this.request.query.file
}, fs.readFileSync( save_path + this.request.query.file )];
});
}
My question, however, is how to invoke that route? Using .Router.to('/export?file=filename.ext') doesn't work, and causes the user to leave the current page. I want this to appear seamless to the user, and I don't want them to have any idea they are being redirected. Before anyone asks, save_path is declared outside of the method, so it does exist.
I have gotten it! However, it required the use of a few additional packages. First, let me describe the workflow a little more clearly:
A user on our site performs a search. On the subsequent search results page, a button exists that allows the user to export his/her search results to a .csv file. The file is then to be exported to the browser for download.
One concern we had was if a file is written to the server, making sure only the user who is exporting the file has the ability to view the file. To control who had visibility on files, I used a meteorite package, CollectionFS (mrt add collectionFS or clone from github). This package writes file buffers to a mongo collection. Supplying an "owner" field when saving gives you control over access.
Regardless of how the file is created, whether saved to the server via an upload form or generated on the fly the way I did using the json2csv package, the file must be streamed to CollectionFS as a buffer.
var userId = Meteor.userId()
var buffer = Buffer(csv.length); //csv is a var holding the data for write
var filename = "name_of_file.csv";
for ( var i=0; i<csv.length; i++ ) {
buffer[i] = csv.charCodeAt(i);
}
CollectionFS.storeBuffer(filename, buffer, {
contentType: 'text/plain',
owner: userId
});
So at this point, I have taken my data file, and streamed it as a buffer into the mongo collection. Because my data exists in memory in the var csv, I stream it as a buffer by looping through each character. If this were a file saved on a physical disk, I would use fs.readFileSync(file) and send the returned buffer to CollectionFS.storeBuffer().
Now that the file is saved as a buffer in mongo with an owner, I can limit through way I publish the CollectionFS collection who can download/update/delete the file or even know the file exists.
In order to read the file from mongo and send the file to the browser for download, another Javascript library is necessary: FileSaver (github).
Using the retrieveBlob method from CollectionFS, pull your file out of mongo as a blob by supplying the _id that references the file in your mongo collection. FileSaver has a method, saveAs that accepts a blob, and exports to the browser for download as a specified file name.
var file = // file object stored in meteor
CollectionFS.retrieveBlob(file._id, function(fileItem) {
if ( fileItem.blob ) saveAs(fileItem.blob, file.filename);
else if ( fileItem.file ) saveAs(fileItem.file, file.filename);
});
I hope someone will find this useful!
If your route works, when when your method returns you could open a new window containing the link to the text file.
You've already added in content disposition headers so the file should always ask to be saved.
Even if you just redirect to the file, because it has these content disposition headers it will ask to be saved and not interrupt your session.
Related
Over the years on snapchat I have saved lots of photos that I would like to retrieve now, The problem is they do not make it easy to export, but luckily if you go online you can request all the data (thats great)
I can see all my photos download link and using the local HTML file if I click download it starts downloading.
Here's where the tricky part is, I have around 15,000 downloads I need to do and manually clicking each individual one will take ages, I've tried extracting all of the links through the download button and this creates lots of Urls (Great) but the problem is, if you past the url into the browser then ("Error: HTTP method GET is not supported by this URL") appears.
I've tried a multitude of different chrome extensions and none of them show the actually download, just the HTML which is on the left-hand side.
The download button is a clickable link that just starts the download in the tab. It belongs under Href A
I'm trying to figure out what the best way of bulk downloading each of these individual files is.
So, I just watched their code by downloading my own memories. They use a custom JavaScript function to download your data (a POST request with ID's in the body).
You can replicate this request, but you can also just use their method.
Open your console and use downloadMemories(<url>)
Or if you don't have the urls you can retrieve them yourself:
var links = document.getElementsByTagName("table")[0].getElementsByTagName("a");
eval(links[0].href);
UPDATE
I made a script for this:
https://github.com/ToTheMax/Snapchat-All-Memories-Downloader
Using the .json file you can download them one by one with python:
req = requests.post(url, allow_redirects=True)
response = req.text
file = requests.get(response)
Then get the correct extension and the date:
day = date.split(" ")[0]
time = date.split(" ")[1].replace(':', '-')
filename = f'memories/{day}_{time}.mp4' if type == 'VIDEO' else f'memories/{day}_{time}.jpg'
And then write it to file:
with open(filename, 'wb') as f:
f.write(file.content)
I've made a bot to download all memories.
You can download it here
It doesn't require any additional installation, just place the memories_history.json file in the same directory and run it. It skips the files that have already been downloaded.
Short answer
Download a desktop application that automates this process.
Visit downloadmysnapchatmemories.com to download the app. You can watch this tutorial guiding you through the entire process.
In short, the app reads the memories_history.json file provided by Snapchat and downloads each of the memories to your computer.
App source code
Long answer (How the app described above works)
We can iterate over each of the memories within the memories_history.json file found in your data download from Snapchat.
For each memory, we make a POST request to the URL stored as the memories Download Link. The response will be a URL to the file itself.
Then, we can make a GET request to the returned URL to retrieve the file.
Example
Here is a simplified example of fetching and downloading a single memory using NodeJS:
Let's say we have the following memory stored in fakeMemory.json:
{
"Date": "2022-01-26 12:00:00 UTC",
"Media Type": "Image",
"Download Link": "https://app.snapchat.com/..."
}
We can do the following:
// import required libraries
const fetch = require('node-fetch'); // Needed for making fetch requests
const fs = require('fs'); // Needed for writing to filesystem
const memory = JSON.parse(fs.readFileSync('fakeMemory.json'));
const response = await fetch(memory['Download Link'], { method: 'POST' });
const url = await response.text(); // returns URL to file
// We can now use the `url` to download the file.
const download = await fetch(url, { method: 'GET' });
const fileName = 'memory.jpg'; // file name we want this saved as
const fileData = download.body; // contents of the file
// Write the contents of the file to this computer using Node's file system
const fileStream = fs.createWriteStream(fileName);
fileData.pipe(fileStream);
fileStream.on('finish', () => {
console.log('memory successfully downloaded as memory.jpg');
});
Short version
I have a webapp using Magnolia, I need to upload a comment with posibility of multiple files, I want to use AJAX, before saving the files as assets I want to be able to check the user's permission, I figured I need a custom Java-based REST endpoint, I made it work, but I have issues saving "jcr:data" into an asset.
Long version
I have a webapp, I have registered users (using PUR), I have different roles for users (for simplicity let's say User and Editor) and I have a Post and a Comment content types. Every User can create a Post and add files, every Post holds the creator's UUID, array of Comment UUIDs and array of file UUIDs (from Assets app), every Post has a comment section and every Comment can have files attached (zero or multiple) to it. Every Editor can post a comment to every Post, but Users can only post comments to their own Posts.
My form for comments looks something like this:
<form action="" method="post" id="comment-form" enctype="multipart/form-data">
<input type="file" name="file" id="file" multiple />
<textarea name="text"></textarea>
<button type="button" onclick="sendComment();">Send</button>
</form>
I tried using Javascript model to process the data and I was able to save the asset correctly, however only one. I couldn't access the other files in the model.
I tried solving it (and improving user experience) by using AJAX and a REST endpoint. I opted not to use the Nodes endpoint API, because I didn't know how to solve the permission issue. I know I can restrict access to REST for each role, but not based on ownership of the Post. So I created my own Java-based endpoint (copied from documentation).
In the sendComment() function in Javascript I create an object with properties like name, extension, mimeType, ..., and data. I read in the documentation that you should send the data using the Base64 format, so I used FileReader() to accomplish that:
var fileObject = {
// properties like name, extension, mimeType, ...
}
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
// this part is easy
};
xhttp.open("PUT", "http://localhost:8080/myApp/.rest/assets/v1/saveAsset", true);
xhttp.setRequestHeader("Content-type", "application/json");
var reader = new FileReader();
reader.onload = function() {
fileObject.data = reader.result;
// I also tried without the 'data:image/png;base64,' part by reader.result.split(",")[1];
xhttp.send(JSON.stringify(fileObject));
};
reader.readAsDataURL(file); //where file is the value of the input element.files[i]
In Java I created a POJO class that has the same properties as the javascript object. Including a String data.
The code for the endpoint looks like this:
public Response saveAsset(Asset file) {
// Asset is my custom POJO class
Session damSession;
Node asset;
Node resource;
try {
damSession = MgnlContext.getJCRSession("dam");
asset = damSession.getRootNode().addNode(file.getName(), "mgnl:asset");
asset.setProperty("name", file.getName());
asset.setProperty("type", file.getExtension());
resource = asset.addNode("jcr:content", "mgnl:resource");
InputStream dataStream = new ByteArrayInputStream(file.getData().getBytes());
ValueFactory vf = damSession.getValueFactory();
Binary dataBinary = vf.createBinary(dataStream);
resource.setProperty("jcr:data", dataBinary);
resource.setProperty("fileName", file.getName());
resource.setProperty("extension", file.getExtension());
resource.setProperty("size", file.getSize());
resource.setProperty("jcr:mimeType", file.getMimeType());
damSession.save();
return Response.ok(LinkUtil.createLink(asset)).build();
} catch (RepositoryException e) {
return Response.ok(e.getMessage()).build(); //I know it's not ok, but that's not important at the moment
}
}
The asset gets created, the properties get saved apart from the jcr:data. If I upload an image and then download it either by the link I get as a response or directly from the Assets app, it cannot be opened, I get the format is not supported message. The size is 0, image doesn't show in the Assets app, seems like the data is simply not there, or it's in the wrong format.
How can I send the file or the file data to the Java endpoint? And how should I receive it? Does anybody know what am I missing? I honestly don't know what else to do with it.
Thank you
The input stream had to be decoded from Base64.
InputStream dataStream = new ByteArrayInputStream(Base64.decodeBase64(file.getData().getBytes()));
...and it only took me less than 3 months.
Noticed it after going through the source code for REST module that for unmarshalling Binary Value it had to be encoded to Base64, so I tried decoding it and it started to work.
In my Vue app I receive a PDF as a blob, and want to display it using the browser's PDF viewer.
I convert it to a file, and generate an object url:
const blobFile = new File([blob], `my-file-name.pdf`, { type: 'application/pdf' })
this.invoiceUrl = window.URL.createObjectURL(blobFile)
Then I display it by setting that URL as the data attribute of an object element.
<object
:data="invoiceUrl"
type="application/pdf"
width="100%"
style="height: 100vh;">
</object>
The browser then displays the PDF using the PDF viewer. However, in Chrome, the file name that I provide (here, my-file-name.pdf) is not used: I see a hash in the title bar of the PDF viewer, and when I download the file using either 'right click -> Save as...' or the viewer's controls, it saves the file with the blob's hash (cda675a6-10af-42f3-aa68-8795aa8c377d or similar).
The viewer and file name work as I'd hoped in Firefox; it's only Chrome in which the file name is not used.
Is there any way, using native Javascript (including ES6, but no 3rd party dependencies other than Vue), to set the filename for a blob / object element in Chrome?
[edit] If it helps, the response has the following relevant headers:
Content-Type: application/pdf; charset=utf-8
Transfer-Encoding: chunked
Content-Disposition: attachment; filename*=utf-8''Invoice%2016246.pdf;
Content-Description: File Transfer
Content-Encoding: gzip
Chrome's extension seems to rely on the resource name set in the URI, i.e the file.ext in protocol://domain/path/file.ext.
So if your original URI contains that filename, the easiest might be to simply make your <object>'s data to the URI you fetched the pdf from directly, instead of going the Blob's way.
Now, there are cases it can't be done, and for these, there is a convoluted way, which might not work in future versions of Chrome, and probably not in other browsers, requiring to set up a Service Worker.
As we first said, Chrome parses the URI in search of a filename, so what we have to do, is to have an URI, with this filename, pointing to our blob:// URI.
To do so, we can use the Cache API, store our File as Request in there using our URL, and then retrieve that File from the Cache in the ServiceWorker.
Or in code,
From the main page
// register our ServiceWorker
navigator.serviceWorker.register('/sw.js')
.then(...
...
async function displayRenamedPDF(file, filename) {
// we use an hard-coded fake path
// to not interfere with legit requests
const reg_path = "/name-forcer/";
const url = reg_path + filename;
// store our File in the Cache
const store = await caches.open( "name-forcer" );
await store.put( url, new Response( file ) );
const frame = document.createElement( "iframe" );
frame.width = 400
frame.height = 500;
document.body.append( frame );
// makes the request to the File we just cached
frame.src = url;
// not needed anymore
frame.onload = (evt) => store.delete( url );
}
In the ServiceWorker sw.js
self.addEventListener('fetch', (event) => {
event.respondWith( (async () => {
const store = await caches.open("name-forcer");
const req = event.request;
const cached = await store.match( req );
return cached || fetch( req );
})() );
});
Live example (source)
Edit: This actually doesn't work in Chrome...
While it does set correctly the filename in the dialog, they seem to be unable to retrieve the file when saving it to the disk...
They don't seem to perform a Network request (and thus our SW isn't catching anything), and I don't really know where to look now.
Still this may be a good ground for future work on this.
And an other solution, I didn't took the time to check by myself, would be to run your own pdf viewer.
Mozilla has made its js based plugin pdf.js available, so from there we should be able to set the filename (even though once again I didn't dug there yet).
And as final note, Firefox is able to use the name property of a File Object a blobURI points to.
So even though it's not what OP asked for, in FF all it requires is
const file = new File([blob], filename);
const url = URL.createObjectURL(file);
object.data = url;
In Chrome, the filename is derived from the URL, so as long as you are using a blob URL, the short answer is "No, you cannot set the filename of a PDF object displayed in Chrome." You have no control over the UUID assigned to the blob URL and no way to override that as the name of the page using the object element. It is possible that inside the PDF a title is specified, and that will appear in the PDF viewer as the document name, but you still get the hash name when downloading.
This appears to be a security precaution, but I cannot say for sure.
Of course, if you have control over the URL, you can easily set the PDF filename by changing the URL.
I believe Kaiido's answer expresses, briefly, the best solution here:
"if your original URI contains that filename, the easiest might be to simply make your object's data to the URI you fetched the pdf from directly"
Especially for those coming from this similar question, it would have helped me to have more description of a specific implementation (working for pdfs) that allows the best user experience, especially when serving files that are generated on the fly.
The trick here is using a two-step process that perfectly mimics a normal link or button click. The client must (step 1) request the file be generated and stored server-side long enough for the client to (step 2) request the file itself. This requires you have some mechanism supporting unique identification of the file on disk or in a cache.
Without this process, the user will just see a blank tab while file-generation is in-progress and if it fails, then they'll just get the browser's ERR_TIMED_OUT page. Even if it succeeds, they'll have a hash in the title bar of the PDF viewer tab, and the save dialog will have the same hash as the suggested filename.
Here's the play-by-play to do better:
You can use an anchor tag or a button for the "download" or "view in browser" elements
Step 1 of 2 on the client: that element's click event can make a request for the file to be generated only (not transmitted).
Step 1 of 2 on the server: generate the file and hold on to it. Return only the filename to the client.
Step 2 of 2 on the client:
If viewing the file in the browser, use the filename returned from the generate request to then invoke window.open('view_file/<filename>?fileId=1'). That is the only way to indirectly control the name of the file as shown in the tab title and in any subsequent save dialog.
If downloading, just invoke window.open('download_file?fileId=1').
Step 2 of 2 on the server:
view_file(filename, fileId) handler just needs to serve the file using the fileId and ignore the filename parameter. In .NET, you can use a FileContentResult like File(bytes, contentType);
download_file(fileId) must set the filename via the Content-Disposition header as shown here. In .NET, that's return File(bytes, contentType, desiredFilename);
client-side download example:
download_link_clicked() {
// show spinner
ajaxGet(generate_file_url,
{},
(response) => {
// success!
// the server-side is responsible for setting the name
// of the file when it is being downloaded
window.open('download_file?fileId=1', "_blank");
// hide spinner
},
() => { // failure
// hide spinner
// proglem, notify pattern
},
null
);
client-side view example:
view_link_clicked() {
// show spinner
ajaxGet(generate_file_url,
{},
(response) => {
// success!
let filename = response.filename;
// simplest, reliable method I know of for controlling
// the filename of the PDF when viewed in the browser
window.open('view_file/'+filename+'?fileId=1')
// hide spinner
},
() => { // failure
// hide spinner
// proglem, notify pattern
},
null
);
I'm using the library pdf-lib, you can click here to learn more about the library.
I solved part of this problem by using api Document.setTitle("Some title text you want"),
Browser displayed my title correctly, but when click the download button, file name is still previous UUID. Perhaps there is other api in the library that allows you to modify download file name.
I have a problem (or may be two) with saving files using HTML5 File API.
A files comes from the server as a byte array and I need to save it. I tried several ways described on SO:
creating blob and opening it in a new tab
creating a hidden anchor tag with "data:" in href attribute
using FileSaver.js
All approaches allow to save the file but with breaking it by changing the encoding to UTF-8, while the file (in current test case) is in ANSI. And it seems that I have to problems: at the server side and at the client side.
Server side:
Server side is ASP.NET Web API 2 app, which controller sends the file using HttpResponseMessage with StreamContent. The ContentType is correct and corresponds with actual file type.
But as can be seen on the screenshot below server's answer (data.length) is less then actual file size calculated at upload (file.size). Also here could be seen that HTML5 File object has yet another size (f.size).
If I add CharSet with value "ANSI" to server's response message's ContentType property, file data will be the same as it was uploaded, but on saving result file still has wrong size and become broken:
Client side:
I tried to set charset using the JS File options, but it didn't help. As could be found here and here Eli Grey, the author of FileUplaod.js says that
The encoding/charset in the type is just metadata for the browser, not an encoding directive.
which means, if I understood it right, that it is impossible to change the encoding of the file.
Issue result: at the end I can successfully download broken files which are unable to open.
So I have two questions:
How can I save file "as is" using File API. At present time I cannot use simple way with direct link and 'download' attribute because of serverside check for access_token in request header. May be this is the "bottle neck" of the problem?
How can I avoid setting CharSet at server side and also send byte array "as is"? While this problem could be hacked in some way I guess it's more critical. For example, while "ANSI" charset solves the problem with the current file, WinMerge shows that it's encoding is Cyrillic 'Windows-1251' and also can any other.
P.S. the issue is related to all file types (extensions) except *.txt.
Update
Server side code:
public HttpResponseMessage DownloadAttachment(Guid fileId)
{
var stream = GetFileStream(fileId);
var message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(stream);
message.Content.Headers.ContentLength = file.Size;
message.Content.Headers.ContentType = new MediaTypeHeaderValue(file.ContentType)
{
// without this charset files sent with bigger size
// than they are as shown on image 1
CharSet = "ANSI"
};
message.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = file.FileName + file.Extension,
Size = file.Size
};
return message;
}
Client side code (TypeScript):
/*
* Handler for click event on download <a> tag
*/
private downloadFile(file: Models.File) {
var self = this;
this.$service.downloadAttachment(this.entityId, file.fileId).then(
// on success
function (data, status, headers, config) {
var fileName = file.fileName + file.extension;
var clientFile = new File([data], fileName);
// here's the issue ---^
saveAs(clientFile, fileName);
},
// on fail
function (error) {
self.alertError(error);
});
}
My code is almost the same as in answers on related questions on SO: instead of setting direct link in 'a' tag, I handle click on it and download file content via XHR (in my case using Angularjs $http service). Getting the file content I create a Blob object (in my case I use File class that derives from Blob) and then try to save it using FileSaver.js. I also tried approach with encoded URL to Blob in href attribute, but it only opens a new tab with a file broken the same way. I found that the problem is in Blob class - calling it's constructor with 'normal' file data I get an instance with 'wrong' size as could be seen on first two screenshots. So, as I understand, my problem not in the way I try to save my file, but in the way I create it - File API
I am creating a node.js app in which user can upload files and can download later.
I am storing file information (original file name that user uploaded, ...) in mongodb document and named that file same as mongodb document id. Now i want my user to be able to download that file with the original file name.
What i want to know is when a user sends a GET request on http://myapp.com/mongoDocument_Id
user gets a file named myOriginalfile.ext
I know about node-static and other modules but i can't rename them before sending file.
i am using koa.js framework.
Here's a simple example using koa-file-server:
var app = require('koa')();
var route = require('koa-route');
var send = require('koa-file-server')({ root : './static' }).send;
app.use(route.get('/:id', function *(id) {
// TODO: perform lookup from id to filename here.
// We'll use a hardcoded filename as an example.
var filename = 'test.txt';
// Set the looked-up filename as the download name.
this.attachment(filename);
// Send the file.
yield send(this, id);
}));
app.listen(3012);
In short:
the files are stored in ./static using the MongoDB id as their filename
a user requests http://myapp.com/123456
you look up that ID in MongoDB to find out the original filename (in the example above, the filename is just hardcoded to test.txt)
the file ./static/123456 is offered as a download using the original filename set in the Content-Disposition header (by using this.attachment(filename)), which will make the browser store it locally as test.txt instead of 123456.