We Keep Coding

Tampermonkey To open multiple javascript in href in new tab [duplicate]

Over the years on snapchat I have saved lots of photos that I would like to retrieve now, The problem is they do not make it easy to export, but luckily if you go online you can request all the data (thats great)
I can see all my photos download link and using the local HTML file if I click download it starts downloading.
Here's where the tricky part is, I have around 15,000 downloads I need to do and manually clicking each individual one will take ages, I've tried extracting all of the links through the download button and this creates lots of Urls (Great) but the problem is, if you past the url into the browser then ("Error: HTTP method GET is not supported by this URL") appears.
I've tried a multitude of different chrome extensions and none of them show the actually download, just the HTML which is on the left-hand side.
The download button is a clickable link that just starts the download in the tab. It belongs under Href A
I'm trying to figure out what the best way of bulk downloading each of these individual files is.

So, I just watched their code by downloading my own memories. They use a custom JavaScript function to download your data (a POST request with ID's in the body).
You can replicate this request, but you can also just use their method.
Open your console and use downloadMemories(<url>)
Or if you don't have the urls you can retrieve them yourself:
var links = document.getElementsByTagName("table")[0].getElementsByTagName("a");
eval(links[0].href);
UPDATE
I made a script for this:
https://github.com/ToTheMax/Snapchat-All-Memories-Downloader

Using the .json file you can download them one by one with python:
req = requests.post(url, allow_redirects=True)
response = req.text
file = requests.get(response)
Then get the correct extension and the date:
day = date.split(" ")[0]
time = date.split(" ")[1].replace(':', '-')
filename = f'memories/{day}_{time}.mp4' if type == 'VIDEO' else f'memories/{day}_{time}.jpg'
And then write it to file:
with open(filename, 'wb') as f:
f.write(file.content)
I've made a bot to download all memories.
You can download it here
It doesn't require any additional installation, just place the memories_history.json file in the same directory and run it. It skips the files that have already been downloaded.

Short answer
Download a desktop application that automates this process.
Visit downloadmysnapchatmemories.com to download the app. You can watch this tutorial guiding you through the entire process.
In short, the app reads the memories_history.json file provided by Snapchat and downloads each of the memories to your computer.
App source code
Long answer (How the app described above works)
We can iterate over each of the memories within the memories_history.json file found in your data download from Snapchat.
For each memory, we make a POST request to the URL stored as the memories Download Link. The response will be a URL to the file itself.
Then, we can make a GET request to the returned URL to retrieve the file.
Example
Here is a simplified example of fetching and downloading a single memory using NodeJS:
Let's say we have the following memory stored in fakeMemory.json:
{
"Date": "2022-01-26 12:00:00 UTC",
"Media Type": "Image",
"Download Link": "https://app.snapchat.com/..."
}
We can do the following:
// import required libraries
const fetch = require('node-fetch'); // Needed for making fetch requests
const fs = require('fs'); // Needed for writing to filesystem
const memory = JSON.parse(fs.readFileSync('fakeMemory.json'));
const response = await fetch(memory['Download Link'], { method: 'POST' });
const url = await response.text(); // returns URL to file
// We can now use the `url` to download the file.
const download = await fetch(url, { method: 'GET' });
const fileName = 'memory.jpg'; // file name we want this saved as
const fileData = download.body; // contents of the file
// Write the contents of the file to this computer using Node's file system
const fileStream = fs.createWriteStream(fileName);
fileData.pipe(fileStream);
fileStream.on('finish', () => {
console.log('memory successfully downloaded as memory.jpg');
});

Saving file with JavaScript File API results wrong encoding

I have a problem (or may be two) with saving files using HTML5 File API.
A files comes from the server as a byte array and I need to save it. I tried several ways described on SO:
creating blob and opening it in a new tab
creating a hidden anchor tag with "data:" in href attribute
using FileSaver.js
All approaches allow to save the file but with breaking it by changing the encoding to UTF-8, while the file (in current test case) is in ANSI. And it seems that I have to problems: at the server side and at the client side.
Server side:
Server side is ASP.NET Web API 2 app, which controller sends the file using HttpResponseMessage with StreamContent. The ContentType is correct and corresponds with actual file type.
But as can be seen on the screenshot below server's answer (data.length) is less then actual file size calculated at upload (file.size). Also here could be seen that HTML5 File object has yet another size (f.size).
If I add CharSet with value "ANSI" to server's response message's ContentType property, file data will be the same as it was uploaded, but on saving result file still has wrong size and become broken:
Client side:
I tried to set charset using the JS File options, but it didn't help. As could be found here and here Eli Grey, the author of FileUplaod.js says that
The encoding/charset in the type is just metadata for the browser, not an encoding directive.
which means, if I understood it right, that it is impossible to change the encoding of the file.
Issue result: at the end I can successfully download broken files which are unable to open.
So I have two questions:
How can I save file "as is" using File API. At present time I cannot use simple way with direct link and 'download' attribute because of serverside check for access_token in request header. May be this is the "bottle neck" of the problem?
How can I avoid setting CharSet at server side and also send byte array "as is"? While this problem could be hacked in some way I guess it's more critical. For example, while "ANSI" charset solves the problem with the current file, WinMerge shows that it's encoding is Cyrillic 'Windows-1251' and also can any other.
P.S. the issue is related to all file types (extensions) except *.txt.
Update
Server side code:
public HttpResponseMessage DownloadAttachment(Guid fileId)
{
var stream = GetFileStream(fileId);
var message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(stream);
message.Content.Headers.ContentLength = file.Size;
message.Content.Headers.ContentType = new MediaTypeHeaderValue(file.ContentType)
{
// without this charset files sent with bigger size
// than they are as shown on image 1
CharSet = "ANSI"
};
message.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = file.FileName + file.Extension,
Size = file.Size
};
return message;
}
Client side code (TypeScript):
/*
* Handler for click event on download <a> tag
*/
private downloadFile(file: Models.File) {
var self = this;
this.$service.downloadAttachment(this.entityId, file.fileId).then(
// on success
function (data, status, headers, config) {
var fileName = file.fileName + file.extension;
var clientFile = new File([data], fileName);
// here's the issue ---^
saveAs(clientFile, fileName);
},
// on fail
function (error) {
self.alertError(error);
});
}
My code is almost the same as in answers on related questions on SO: instead of setting direct link in 'a' tag, I handle click on it and download file content via XHR (in my case using Angularjs $http service). Getting the file content I create a Blob object (in my case I use File class that derives from Blob) and then try to save it using FileSaver.js. I also tried approach with encoded URL to Blob in href attribute, but it only opens a new tab with a file broken the same way. I found that the problem is in Blob class - calling it's constructor with 'normal' file data I get an instance with 'wrong' size as could be seen on first two screenshots. So, as I understand, my problem not in the way I try to save my file, but in the way I create it - File API

HTML anchor tag download attribute not working in Firefox for jpg and png files

In my web application I have supported user to upload any type of document (.png, .jpg, .docx, .xls, ... ) I'm trying to implement download functionality for these documents.
In Google Chrome if you click on Download link Save dialog is shown for all above documents.
In Mozilla Firefox for docx and xls works fine, Save dialog is shown but for .png and .jpg download tag is not working as expected i.e., download dialog or Save dialog does not appear, it directly open that image.
My code:
Download
I have tried almost all solutions mentioned on stackoverflow and suggested by Google. But most of them says that 'check firefox version' and other changes like:
try adding the element to the DOM before triggering the click
Remove filename from download tag it is of boolean type and etc.
I have also tried w3schools lesson on anchor tag and download attribute but nothing seems to be working.
My Mozilla Firefox version is: 38.0.5
P.S.: in chrome as well as in firefox .docs, .xls, .pdf documents work fine, problem is for .png and .jpg in firefox.

Firefox will handle png and jpeg using default handling, which is to inline them in the document. When clicking a link, even if download attribute is defined, seem to make Firefox think it has a new image ignoring the download aspect of it. This may be a temporary bug.
Here is a way, admittedly not super-elegant, to get around this problem forcing the image to be interpreted as an octet-stream.
It does not work inline on Stackoverflow so you have to test it on jsFiddle.
The code does the following:
Scans the document for a-tags.
Those which has data-link set will have a common click-handler attached.
When clicked the link is extracted from the data-link attribute (href is se to #), loaded as an ArrayBuffer via XHR (CORS requirements applies, not a problem in this case), and is converted to an Object-URL with the Blob set to mime-type octet/stream
The Object URL is set as window.location to redirect to this binary data which will make the browser ask user to download the file instead.
var links = document.querySelectorAll("a"), i = 0, lnk;
while(lnk = links[i++]) {
if (lnk.dataset.link.length) lnk.onclick = toBlob;
}
function toBlob(e) {
e.preventDefault();
var lnk = this, xhr = new XMLHttpRequest();
xhr.open("GET", lnk.dataset.link);
xhr.responseType = "blob";
xhr.overrideMimeType("octet/stream");
xhr.onload = function() {
if (xhr.status === 200) {
window.location = (URL || webkitURL).createObjectURL(xhr.response);
}
};
xhr.send();
}
Example tag:
Click to download
The drawback is that you'll loose the extension in the filename.
This is also possible to do using a Data-URL, but a data-url has a 166% overhead compared to using ArrayBuffer and a blob.

I had a similar problem with firefox not handling the download attribute, even for same-domain files.
My target files are actually hosted on AWS, so they are cross-domain. I got around this with a same-domain endpoint that downloads the remote file and pipes it to the client.
const express = require('express')
const {createWriteStream} = require('fs')
const downloadVideo = (url) => { return new Promise((resolve, reject) => {
const filePath = `/tmp/neat.mp4`
const ws = createWriteStream(filePath)
request(url, {}, (error, response, body) => {
if(error) { return reject(error) }
resolve(filePath)
}).pipe(ws)
})}
app.get('/api/download', async (req, res) => {
const videoPath = await downloadVideo(req.query.url)
res.sendFile(videoPath)
})
On the client, I send the file path to the download endpoint to get a blob back, which is then converted to an object url. From there, it's standard download attribute stuff.
async download(remoteFilePath){
const a = document.createElement('a')
const dlURL = `/api/download?url=${encodeURIComponent(remoteFilePath)}`
const blob = await fetch(dlURL).then(res => res.blob())
a.href = URL.createObjectURL(blob)
a.setAttribute('download', 'cool.mp4')
document.body.appendChild(a)
a.click()
a.remove()
}

As you are using HTML5 attribute, each browser handling differently. So use https://github.com/dcneiner/Downloadify for client side forceful download instead of viewing in browser.

Make browser download/save files without user interaction

I need to create a small browser-based application that helps users download/save, and possibly print to the default printer, a large number of files from a webserver we have no control over (but we have all the URIs beforehand).
These files are hidden behind a "single sign-on" (SSO) that can only be performed via a browser and requires a user. Hence, it must be a browser-based solution, where we piggyback on to the session established by the SSO.
The users' platform is Windows 7.
The point is to save the users from going through a lot of clicks per file (download, where to save, etc.) when they need to perform this operation (daily).
At this point all the files are PDF, but that might change in the future.
A browser-agnostic solution is preferred (and I assume more robust when it comes to future browser updates).
But we can base it on a particular browser if needed.
How would you do this from Javascript?

As the comments to my question says, this isn't really allowed by the browsers for security reasons.
My workaround for now (only tested using IE11) is to manually change the security settings of the users browser, and then download the files as a blob into a javascript variable using AJAX, followed by upload of same blob to my own server again using AJAX.
"My own server" is a Django site created for this purpose, that also knows which files to download for the day, and provide the javascript needed. The user goes to this site to initiate the daily download after performing the SSO in a separate browser tab.
On the server I can then perform whatever operations needed for said files.
Many thanks to this post https://stackoverflow.com/a/13887220/833320 for the handling of binary data in AJAX.
1) In IE, add the involved sites to the "Local Intranet Zone", and enable "Access data sources across domains" for this zone to overcome the CORS protection otherwise preventing this.
Of course, consider the security consequences involved in this...
2) In javascript (browser), download the file as a blob and POST the resulting data to my own server:
var x = new XMLHttpRequest();
x.onload = function() {
// Create a form
var fd = new FormData();
fd.append('csrfmiddlewaretoken', '{{ csrf_token }}'); // Needed by Django
fd.append('file', x.response); // x.response is a Blob object
// Upload to your server
var y = new XMLHttpRequest();
y.onload = function() {
alert('File uploaded!');
};
y.open('POST', '/django/upload/');
y.send(fd);
};
x.open('GET', 'https://external.url', true);
x.responseType = 'blob'; // <-- This is necessary!
x.send();
3) Finally (in Django view for '/django/upload/'), receive the uploaded data and save as file - or whatever...
filedata = request.FILES['file'].read()
with open('filename', 'wb') as f:
f.write(filedata)
Thanks all, for your comments.
And yes, the real solution would be to overcome the SSO (that requieres the user), so it all could be done by the server itself.
But at least I learned a little about getting/posting binary data using modern XMLHttpRequests. :)

Actually, I had a problem like it, I wanted to download a binary file(an image) and store it and then use it when I need it, So I decided to download it with Fetch API Get call:
const imageAddress = 'an-address-to-my-image.jpg'; // sample address
fetch(imageAddress)
.then(res => res.blob) // <-- This is necessary!
.then(blobFileToBase64)
.then(base64FinalAnswer => console.log(base64FinalAnswer))
The blobFileToBase64 is a helper function that converts blob binary file to a base64 data string:
const blobToBase64 = blob => {
const reader = new FileReader();
reader.readAsDataURL(blob);
return new Promise(resolve => {
reader.onloadend = () => {
resolve(reader.result);
};
});
};
In the end, I have the base64FinalAnswer and I can do anything with it.

HTML5: drag out a JS generated file

I have a feeling security concerns may not allow this but is it possible to generate a file with JavaScript and allow the user to drag it to the desktop (or file system)?
The following code drags out a file from a server
files[0].addEventListener("dragstart",function(evt){
evt.dataTransfer.setData("DownloadURL", "application/octet-stream:Eadui2.ttf:http://thecssninja.come/demo/gmail_dragout/Eadui.ttf");
},false);
And with the below code I can generate a file and have it download but I can't set the file name or let the user select the location.
var uriContent = "data:application/octet-stream," + encodeURIComponent(JSON.stringify(map));
location.href = uriContent;
Ideally I'd like a magical combination of both.

following code is currently working in Chrome only:
// generate downloadable URL, file name here will not affect stored file
var url = URL.createObjectURL(new File([JSON.stringify(map)], 'file_name.txt'));
// note that any draggable element may be used instead of files[0]
// since JSON.stringify returns a string, we use 'text' type in setData
files[0].addEventListener("dragstart", function(evt) {
evt.dataTransfer.setData("DownloadURL", "text:file_name.txt:" + url);
}, false);
now, dragging our files[0] element from the browser to desktop or file system, will store there a text file called, file_name.txt.
Feel free to choose another file name :)

This is only possible for Chrome, and even in Chrome you can't set the location. If using only Chrome is okay then you will have the following options:
Stick with Drag n' Drop like from the CSS Ninja's tutorial, then you should try Ben's answer. encodeURIComponent is one way, but if you have the file generated using BlobBuilder then you can use window.webkitURL.createObjectURL() to get the file's URL. You can also try using FileWriter() with requestFileSystem(TEMPORARY, ...).
Chrome supports download attribute for anchor tags so you can have regular link for the user to click (dragging also works):
Download
For cross browser support I suggest Downloadify.

You could try sending it to the server, saving the file, checking the return value and firing the download file function, followed by a server file that deletes the file from the server.
Something like this (with jQuery)
$.ajax({
url: 'saveFile.php',
method: 'post',
data: {
Filedata: data// file data variable
},
success: function(d) {
// save file function, where d is the filename
}
})
PHP:
$filename = ;//generate filename
file_put_contents($filename, $_POST['Filedata']);
echo $filename;
Obviously there is more to it but that should be the basics