We Keep Coding

Tampermonkey To open multiple javascript in href in new tab [duplicate]

Over the years on snapchat I have saved lots of photos that I would like to retrieve now, The problem is they do not make it easy to export, but luckily if you go online you can request all the data (thats great)
I can see all my photos download link and using the local HTML file if I click download it starts downloading.
Here's where the tricky part is, I have around 15,000 downloads I need to do and manually clicking each individual one will take ages, I've tried extracting all of the links through the download button and this creates lots of Urls (Great) but the problem is, if you past the url into the browser then ("Error: HTTP method GET is not supported by this URL") appears.
I've tried a multitude of different chrome extensions and none of them show the actually download, just the HTML which is on the left-hand side.
The download button is a clickable link that just starts the download in the tab. It belongs under Href A
I'm trying to figure out what the best way of bulk downloading each of these individual files is.

So, I just watched their code by downloading my own memories. They use a custom JavaScript function to download your data (a POST request with ID's in the body).
You can replicate this request, but you can also just use their method.
Open your console and use downloadMemories(<url>)
Or if you don't have the urls you can retrieve them yourself:
var links = document.getElementsByTagName("table")[0].getElementsByTagName("a");
eval(links[0].href);
UPDATE
I made a script for this:
https://github.com/ToTheMax/Snapchat-All-Memories-Downloader

Using the .json file you can download them one by one with python:
req = requests.post(url, allow_redirects=True)
response = req.text
file = requests.get(response)
Then get the correct extension and the date:
day = date.split(" ")[0]
time = date.split(" ")[1].replace(':', '-')
filename = f'memories/{day}_{time}.mp4' if type == 'VIDEO' else f'memories/{day}_{time}.jpg'
And then write it to file:
with open(filename, 'wb') as f:
f.write(file.content)
I've made a bot to download all memories.
You can download it here
It doesn't require any additional installation, just place the memories_history.json file in the same directory and run it. It skips the files that have already been downloaded.

Short answer
Download a desktop application that automates this process.
Visit downloadmysnapchatmemories.com to download the app. You can watch this tutorial guiding you through the entire process.
In short, the app reads the memories_history.json file provided by Snapchat and downloads each of the memories to your computer.
App source code
Long answer (How the app described above works)
We can iterate over each of the memories within the memories_history.json file found in your data download from Snapchat.
For each memory, we make a POST request to the URL stored as the memories Download Link. The response will be a URL to the file itself.
Then, we can make a GET request to the returned URL to retrieve the file.
Example
Here is a simplified example of fetching and downloading a single memory using NodeJS:
Let's say we have the following memory stored in fakeMemory.json:
{
"Date": "2022-01-26 12:00:00 UTC",
"Media Type": "Image",
"Download Link": "https://app.snapchat.com/..."
}
We can do the following:
// import required libraries
const fetch = require('node-fetch'); // Needed for making fetch requests
const fs = require('fs'); // Needed for writing to filesystem
const memory = JSON.parse(fs.readFileSync('fakeMemory.json'));
const response = await fetch(memory['Download Link'], { method: 'POST' });
const url = await response.text(); // returns URL to file
// We can now use the `url` to download the file.
const download = await fetch(url, { method: 'GET' });
const fileName = 'memory.jpg'; // file name we want this saved as
const fileData = download.body; // contents of the file
// Write the contents of the file to this computer using Node's file system
const fileStream = fs.createWriteStream(fileName);
fileData.pipe(fileStream);
fileStream.on('finish', () => {
console.log('memory successfully downloaded as memory.jpg');
});

How to download a file in the browser directly from the node.js server side without any variable?

My app is created with mean and I am a user of docker too. The purpose of my app is to create and download a CSV file. I already created my file, compressed it and placed it in a temp folder (the file will be removed after the download). This part is in the nodejs server side and works without problems.
I already use several things like (res.download) which is supposed to download directly the file in the browser but nothing append. I tried to use blob in the angularjs part but it doesn't work.
The getData function creates and compresses the file (it exists I can reach it directly when I look where the app is saved).
exports.getData = function getData(req, res, next){
var listRequest = req.body.params.listURL;
var stringTags = req.body.params.tagString;
//The name of the compressed CSV file
var nameFile = req.body.params.fileName;
var query = url.parse(req.url, true).query;
//The function which create the file
ApollineData.getData(listRequest, stringTags, nameFile)
.then(function (response){
var filePath = '/opt/mean.js/modules/apolline/client/CSVDownload/'+response;
const file = fs.createReadStream(filePath);
res.download(filePath, response);
})
.catch(function (response){
console.log(response);
});
};
My main problem is to download this file directly in the browser without using any variable because it could be huge (like several GB). I want to download it and then delete it.

There is nothing wrong with res.download
Probably the reason why res.download don't work for you is b/c you are using AJAX to fetch the resource, Do a regular navigation. Or if it requires some post data and another method: create a form and submit.

How to download a URL in JavaScript (Nodejs)?

I used this link to make an original link into a download link:
https://milanaryal.com/2015/direct-linking-to-your-files-on-dropbox-google-drive-and-onedrive/
Now how do I actually use that download link to download the file in JavaScript? I want to do something like:
link = 'https://drive.google.com/uc?export=download&id=FILE_ID';
let x = download(link); //now x is the download file
I looked it up and it seems like there are ways of doing this with HTML/jQuery, but I am not using those because I am working on the server side with Nodejs. I am doing this download thing because I want to check if the file is a pdf or text, parse the text, and then search through it using Elasticsearch.

It's easiest to use a module such as Request to do a HTTP get from a node script.
For example:
var request = require('request');
request.get('https://drive.google.com/uc?export=download&id=FILE_ID',
function(err, res, body){
if(err) return console.log(err);
console.log(body);
});
Once the file has downloaded, the callback function is run with the downloaded file in the body variable

If you only want to download the file, open it, search for data and delete it, you can easily edit this code snippet: https://stackoverflow.com/a/11944984/642977

How to serve static files in node.js after renaming them?

I am creating a node.js app in which user can upload files and can download later.
I am storing file information (original file name that user uploaded, ...) in mongodb document and named that file same as mongodb document id. Now i want my user to be able to download that file with the original file name.
What i want to know is when a user sends a GET request on http://myapp.com/mongoDocument_Id
user gets a file named myOriginalfile.ext
I know about node-static and other modules but i can't rename them before sending file.
i am using koa.js framework.

Here's a simple example using koa-file-server:
var app = require('koa')();
var route = require('koa-route');
var send = require('koa-file-server')({ root : './static' }).send;
app.use(route.get('/:id', function *(id) {
// TODO: perform lookup from id to filename here.
// We'll use a hardcoded filename as an example.
var filename = 'test.txt';
// Set the looked-up filename as the download name.
this.attachment(filename);
// Send the file.
yield send(this, id);
}));
app.listen(3012);
In short:
the files are stored in ./static using the MongoDB id as their filename
a user requests http://myapp.com/123456
you look up that ID in MongoDB to find out the original filename (in the example above, the filename is just hardcoded to test.txt)
the file ./static/123456 is offered as a download using the original filename set in the Content-Disposition header (by using this.attachment(filename)), which will make the browser store it locally as test.txt instead of 123456.

Download files from server using Meteor.js

Here is my workflow as of now:
In a button click event, I have search results being exported to a .csv file, which is saved to the server. Once the file is saved, I want to send it for download to the browser. Using this question How to handle conditional file downloads in meteor.js, I created a method that is called after the method that saves the file returns. Here is that method:
exportFiles: function(file_to_export) {
console.log("to export = "+file_to_export);
Meteor.Router.add('/export', 'GET', function() {
console.log('send '+file_to_export+' to browser');
return [200,
{
'Content-type': 'text/plain',
'Content-Disposition': "attachment; filename=" + this.request.query.file
}, fs.readFileSync( save_path + this.request.query.file )];
});
}
My question, however, is how to invoke that route? Using .Router.to('/export?file=filename.ext') doesn't work, and causes the user to leave the current page. I want this to appear seamless to the user, and I don't want them to have any idea they are being redirected. Before anyone asks, save_path is declared outside of the method, so it does exist.

I have gotten it! However, it required the use of a few additional packages. First, let me describe the workflow a little more clearly:
A user on our site performs a search. On the subsequent search results page, a button exists that allows the user to export his/her search results to a .csv file. The file is then to be exported to the browser for download.
One concern we had was if a file is written to the server, making sure only the user who is exporting the file has the ability to view the file. To control who had visibility on files, I used a meteorite package, CollectionFS (mrt add collectionFS or clone from github). This package writes file buffers to a mongo collection. Supplying an "owner" field when saving gives you control over access.
Regardless of how the file is created, whether saved to the server via an upload form or generated on the fly the way I did using the json2csv package, the file must be streamed to CollectionFS as a buffer.
var userId = Meteor.userId()
var buffer = Buffer(csv.length); //csv is a var holding the data for write
var filename = "name_of_file.csv";
for ( var i=0; i<csv.length; i++ ) {
buffer[i] = csv.charCodeAt(i);
}
CollectionFS.storeBuffer(filename, buffer, {
contentType: 'text/plain',
owner: userId
});
So at this point, I have taken my data file, and streamed it as a buffer into the mongo collection. Because my data exists in memory in the var csv, I stream it as a buffer by looping through each character. If this were a file saved on a physical disk, I would use fs.readFileSync(file) and send the returned buffer to CollectionFS.storeBuffer().
Now that the file is saved as a buffer in mongo with an owner, I can limit through way I publish the CollectionFS collection who can download/update/delete the file or even know the file exists.
In order to read the file from mongo and send the file to the browser for download, another Javascript library is necessary: FileSaver (github).
Using the retrieveBlob method from CollectionFS, pull your file out of mongo as a blob by supplying the _id that references the file in your mongo collection. FileSaver has a method, saveAs that accepts a blob, and exports to the browser for download as a specified file name.
var file = // file object stored in meteor
CollectionFS.retrieveBlob(file._id, function(fileItem) {
if ( fileItem.blob ) saveAs(fileItem.blob, file.filename);
else if ( fileItem.file ) saveAs(fileItem.file, file.filename);
});
I hope someone will find this useful!

If your route works, when when your method returns you could open a new window containing the link to the text file.
You've already added in content disposition headers so the file should always ask to be saved.
Even if you just redirect to the file, because it has these content disposition headers it will ask to be saved and not interrupt your session.