I used this link to make an original link into a download link:
https://milanaryal.com/2015/direct-linking-to-your-files-on-dropbox-google-drive-and-onedrive/
Now how do I actually use that download link to download the file in JavaScript? I want to do something like:
link = 'https://drive.google.com/uc?export=download&id=FILE_ID';
let x = download(link); //now x is the download file
I looked it up and it seems like there are ways of doing this with HTML/jQuery, but I am not using those because I am working on the server side with Nodejs. I am doing this download thing because I want to check if the file is a pdf or text, parse the text, and then search through it using Elasticsearch.
It's easiest to use a module such as Request to do a HTTP get from a node script.
For example:
var request = require('request');
request.get('https://drive.google.com/uc?export=download&id=FILE_ID',
function(err, res, body){
if(err) return console.log(err);
console.log(body);
});
Once the file has downloaded, the callback function is run with the downloaded file in the body variable
If you only want to download the file, open it, search for data and delete it, you can easily edit this code snippet: https://stackoverflow.com/a/11944984/642977
Related
Over the years on snapchat I have saved lots of photos that I would like to retrieve now, The problem is they do not make it easy to export, but luckily if you go online you can request all the data (thats great)
I can see all my photos download link and using the local HTML file if I click download it starts downloading.
Here's where the tricky part is, I have around 15,000 downloads I need to do and manually clicking each individual one will take ages, I've tried extracting all of the links through the download button and this creates lots of Urls (Great) but the problem is, if you past the url into the browser then ("Error: HTTP method GET is not supported by this URL") appears.
I've tried a multitude of different chrome extensions and none of them show the actually download, just the HTML which is on the left-hand side.
The download button is a clickable link that just starts the download in the tab. It belongs under Href A
I'm trying to figure out what the best way of bulk downloading each of these individual files is.
So, I just watched their code by downloading my own memories. They use a custom JavaScript function to download your data (a POST request with ID's in the body).
You can replicate this request, but you can also just use their method.
Open your console and use downloadMemories(<url>)
Or if you don't have the urls you can retrieve them yourself:
var links = document.getElementsByTagName("table")[0].getElementsByTagName("a");
eval(links[0].href);
UPDATE
I made a script for this:
https://github.com/ToTheMax/Snapchat-All-Memories-Downloader
Using the .json file you can download them one by one with python:
req = requests.post(url, allow_redirects=True)
response = req.text
file = requests.get(response)
Then get the correct extension and the date:
day = date.split(" ")[0]
time = date.split(" ")[1].replace(':', '-')
filename = f'memories/{day}_{time}.mp4' if type == 'VIDEO' else f'memories/{day}_{time}.jpg'
And then write it to file:
with open(filename, 'wb') as f:
f.write(file.content)
I've made a bot to download all memories.
You can download it here
It doesn't require any additional installation, just place the memories_history.json file in the same directory and run it. It skips the files that have already been downloaded.
Short answer
Download a desktop application that automates this process.
Visit downloadmysnapchatmemories.com to download the app. You can watch this tutorial guiding you through the entire process.
In short, the app reads the memories_history.json file provided by Snapchat and downloads each of the memories to your computer.
App source code
Long answer (How the app described above works)
We can iterate over each of the memories within the memories_history.json file found in your data download from Snapchat.
For each memory, we make a POST request to the URL stored as the memories Download Link. The response will be a URL to the file itself.
Then, we can make a GET request to the returned URL to retrieve the file.
Example
Here is a simplified example of fetching and downloading a single memory using NodeJS:
Let's say we have the following memory stored in fakeMemory.json:
{
"Date": "2022-01-26 12:00:00 UTC",
"Media Type": "Image",
"Download Link": "https://app.snapchat.com/..."
}
We can do the following:
// import required libraries
const fetch = require('node-fetch'); // Needed for making fetch requests
const fs = require('fs'); // Needed for writing to filesystem
const memory = JSON.parse(fs.readFileSync('fakeMemory.json'));
const response = await fetch(memory['Download Link'], { method: 'POST' });
const url = await response.text(); // returns URL to file
// We can now use the `url` to download the file.
const download = await fetch(url, { method: 'GET' });
const fileName = 'memory.jpg'; // file name we want this saved as
const fileData = download.body; // contents of the file
// Write the contents of the file to this computer using Node's file system
const fileStream = fs.createWriteStream(fileName);
fileData.pipe(fileStream);
fileStream.on('finish', () => {
console.log('memory successfully downloaded as memory.jpg');
});
I am using a node sever to send a table from a sqlite db to the browser. This table contains filename and path of a pdf file that I want to render on the browser. Until now I was using hard coded paths for the the pdf file and rendering. But now i have setup a get route and a controller in node such that whenever '/content' is hit in browser , the server queries the database and and sends the data to the client. To the send the data I am using
res.render('content/index',{data:queryData});
Now, how do I access this data using client side javascript so that I can pass the path of the pdf file to the function that renders the pdf? I have done research and the nearest answer I got was using XMLHttpRequest. I tried this method
var xhr = new XMLHttpRequest();
const path = "http://localhost:3000/content";
xhr.onreadystatechange = function () {
if (xhr.readyState == 4 && xhr.status == 200)
{
var myResponseText = xhr.responseText;
console.log(myResponseText);
}
};
xhr.open('get', path, true);
xhr.send();
When I do this I get the entire html code for the view. Not the data I expected. How do I solve this issue. I have done some more reading while writing this post and I suppose. I have set a header somewhere? But the documentation says
app.render(view, [locals], callback)
which means res.render can take local variables, shouldn't be setting the headers?
You should return json instead of render template:
app.get('content/index', (req, res) => {
res.json({data: queryData});
});
I am using pdf.js
PDF.js needs the PDF file, e.g.:
pdfjsLib.getDocument('helloworld.pdf')
I'm assuming your queryData goes something like this:
{ filename: 'file.pdf', path: './path/to/file.pdf' }
I'm not sure what's in your content/index or what path this is on, but you obviously need to find a way to make your PDF file ('./path/to/file.pdf') available (as a download). See Express's built-in static server or res.download() to do that.
Once you have the PDF file available as a download, plug that path into PDF.js's .getDocument('/content/file.pdf') and do the rest to render the PDF onto the canvas or whatever.
Hope that helps.
I have two file inputs and button. When the button is pressed, it should send the two files to the server and wait to receive a file back (the two files processed server-side and the result is returned).
HTML
<input id='file-input1' type='file'>
<input id='file-input2' type='file'>
<button id='send-btn'>
JavaScript (Client)
var input1 = document.getElementById('file-input1');
var input2 = document.getElementById('file-input2');
var btn = document.getElementById('send-btn');
var file1 = null;
var file2 = null;
input1.addEventListener('change', () => file1 = input1.files[0]);
input2.addEventListener('change', () => file2 = input2.files[0]);
btn.addEventListener('click', () => {
if (file1 === null || file2 === null) return;
_sendfiles(file1, file2);
});
function _sendfiles(file1, file2) {
let xmlhttp = new XMLHttpRequest();
xml.open("PUT", "/process", true);
xmlhttp.send({'file1': file1, 'file2': file2});
}
JavaScript (Server)
app.put('/process', (req, res) => {
// Get files from request
// Do stuff with them to generate a third file
// Send generated file back
});
I'm not sure how to receive the files on the server-side, nor how to wait to receive the server's file on the client side. The use of third-party modules is discouraged but not completely out of the question. I'm also not married to the idea of using XMLHttpRequest().
To send files from the client I'd suggest using Fetch + FormData API for convenience:
const formData = new FormData()
formData.append('file1', file1)
formData.append('file2', file2)
fetch(`/api/companies/${id}/logo`, {
method: 'PUT',
body: formData
})
Passing FormData instance to body will automatically set Content-Type: multipart/form-data header.
On the server side I'd suggest using multer since you already use express. You can, of course, implement your own middleware to retrieve files from the request stream if you want to (I didn't do it myself so can't help much).
To receive a file on the client you can do the following (I assume you want this file to be downloaded to the user's file system):
Way #1 (simple):
In the response just send a download URL of this file. Then use this solution to create a link and trigger click event on it. The file will be downloaded by a browser.
Way #2 (not so simple):
On the server use res.sendFile method to send a file (if it's located on fs - otherwise you can send a file Buffer like this for instance).
Then on the client you have response.blob() method to access file blob.
Use a similar trick to download this blob into a file with the help of URL.createObjectURL API.
Additionally, Response API allows you to pipe the stream and do other things with it if you need to (see Streams API).
EDIT (the simplest way)
As Endless pointed out there is a much simpler way actually. I guess I spent too much time dealing with AJAX requests... 🤷♂️
You can just submit your HTML form by clicking on submit button, this way a browser will send POST (yeah, can't do PUT this way) request with Content-Type: multipart/form-data automatically since you have inputs with type file:
<form method='post' action='/process'>
<input name='file1' type='file'>
<input name='file2' type='file'>
<button type='submit'>Submit</button>
</form>
So no need to set any event listeners or use any JS in fact.
Then on the server use res.sendFile and add a Content-Disposition: attachment; filename="filename.jpg" header to make sure browser will download it as an attachment and not open it as a webpage.
The biggest disadvantage here is that there is no built-in convenient way in a browser to subscribe to the request's completion event. I.e. there is no success event on the form which you can listen to.
So, if you need it then a nice approach would be to send a cookie from the server along with the file.
On the client set an interval at the moment you submit the form and there check if the cookie exists. If it exists then this means the file is downloaded.
My app is created with mean and I am a user of docker too. The purpose of my app is to create and download a CSV file. I already created my file, compressed it and placed it in a temp folder (the file will be removed after the download). This part is in the nodejs server side and works without problems.
I already use several things like (res.download) which is supposed to download directly the file in the browser but nothing append. I tried to use blob in the angularjs part but it doesn't work.
The getData function creates and compresses the file (it exists I can reach it directly when I look where the app is saved).
exports.getData = function getData(req, res, next){
var listRequest = req.body.params.listURL;
var stringTags = req.body.params.tagString;
//The name of the compressed CSV file
var nameFile = req.body.params.fileName;
var query = url.parse(req.url, true).query;
//The function which create the file
ApollineData.getData(listRequest, stringTags, nameFile)
.then(function (response){
var filePath = '/opt/mean.js/modules/apolline/client/CSVDownload/'+response;
const file = fs.createReadStream(filePath);
res.download(filePath, response);
})
.catch(function (response){
console.log(response);
});
};
My main problem is to download this file directly in the browser without using any variable because it could be huge (like several GB). I want to download it and then delete it.
There is nothing wrong with res.download
Probably the reason why res.download don't work for you is b/c you are using AJAX to fetch the resource, Do a regular navigation. Or if it requires some post data and another method: create a form and submit.
If i have the code below how can i edit the specific file and make the right corrections?
var p4 = require('C:/Program Files/nodejs/node_modules/p4');
var File = process.argv[2];
p4.edit(File, function(err, data) {
if (err) {
console.error(err.message);
}
console.log(data);
});
Your code looks correct to open the file for edit. If that returns any errors when you run it, you should post those here, but I'll assume that it returns a success message ("(file) opened for edit").
Opening the file for edit means that it is made writable on the local filesystem (i.e. the one where this code is running -- the file is the one you passed as an argument to the edit command). To actually modify the file you can use any other function at your disposal.