We Keep Coding

Tampermonkey To open multiple javascript in href in new tab [duplicate]

Over the years on snapchat I have saved lots of photos that I would like to retrieve now, The problem is they do not make it easy to export, but luckily if you go online you can request all the data (thats great)
I can see all my photos download link and using the local HTML file if I click download it starts downloading.
Here's where the tricky part is, I have around 15,000 downloads I need to do and manually clicking each individual one will take ages, I've tried extracting all of the links through the download button and this creates lots of Urls (Great) but the problem is, if you past the url into the browser then ("Error: HTTP method GET is not supported by this URL") appears.
I've tried a multitude of different chrome extensions and none of them show the actually download, just the HTML which is on the left-hand side.
The download button is a clickable link that just starts the download in the tab. It belongs under Href A
I'm trying to figure out what the best way of bulk downloading each of these individual files is.

So, I just watched their code by downloading my own memories. They use a custom JavaScript function to download your data (a POST request with ID's in the body).
You can replicate this request, but you can also just use their method.
Open your console and use downloadMemories(<url>)
Or if you don't have the urls you can retrieve them yourself:
var links = document.getElementsByTagName("table")[0].getElementsByTagName("a");
eval(links[0].href);
UPDATE
I made a script for this:
https://github.com/ToTheMax/Snapchat-All-Memories-Downloader

Using the .json file you can download them one by one with python:
req = requests.post(url, allow_redirects=True)
response = req.text
file = requests.get(response)
Then get the correct extension and the date:
day = date.split(" ")[0]
time = date.split(" ")[1].replace(':', '-')
filename = f'memories/{day}_{time}.mp4' if type == 'VIDEO' else f'memories/{day}_{time}.jpg'
And then write it to file:
with open(filename, 'wb') as f:
f.write(file.content)
I've made a bot to download all memories.
You can download it here
It doesn't require any additional installation, just place the memories_history.json file in the same directory and run it. It skips the files that have already been downloaded.

Short answer
Download a desktop application that automates this process.
Visit downloadmysnapchatmemories.com to download the app. You can watch this tutorial guiding you through the entire process.
In short, the app reads the memories_history.json file provided by Snapchat and downloads each of the memories to your computer.
App source code
Long answer (How the app described above works)
We can iterate over each of the memories within the memories_history.json file found in your data download from Snapchat.
For each memory, we make a POST request to the URL stored as the memories Download Link. The response will be a URL to the file itself.
Then, we can make a GET request to the returned URL to retrieve the file.
Example
Here is a simplified example of fetching and downloading a single memory using NodeJS:
Let's say we have the following memory stored in fakeMemory.json:
{
"Date": "2022-01-26 12:00:00 UTC",
"Media Type": "Image",
"Download Link": "https://app.snapchat.com/..."
}
We can do the following:
// import required libraries
const fetch = require('node-fetch'); // Needed for making fetch requests
const fs = require('fs'); // Needed for writing to filesystem
const memory = JSON.parse(fs.readFileSync('fakeMemory.json'));
const response = await fetch(memory['Download Link'], { method: 'POST' });
const url = await response.text(); // returns URL to file
// We can now use the `url` to download the file.
const download = await fetch(url, { method: 'GET' });
const fileName = 'memory.jpg'; // file name we want this saved as
const fileData = download.body; // contents of the file
// Write the contents of the file to this computer using Node's file system
const fileStream = fs.createWriteStream(fileName);
fileData.pipe(fileStream);
fileStream.on('finish', () => {
console.log('memory successfully downloaded as memory.jpg');
});

How can I upload multiple files into Magnolia CMS assets app from a custom form

Short version
I have a webapp using Magnolia, I need to upload a comment with posibility of multiple files, I want to use AJAX, before saving the files as assets I want to be able to check the user's permission, I figured I need a custom Java-based REST endpoint, I made it work, but I have issues saving "jcr:data" into an asset.
Long version
I have a webapp, I have registered users (using PUR), I have different roles for users (for simplicity let's say User and Editor) and I have a Post and a Comment content types. Every User can create a Post and add files, every Post holds the creator's UUID, array of Comment UUIDs and array of file UUIDs (from Assets app), every Post has a comment section and every Comment can have files attached (zero or multiple) to it. Every Editor can post a comment to every Post, but Users can only post comments to their own Posts.
My form for comments looks something like this:
<form action="" method="post" id="comment-form" enctype="multipart/form-data">
<input type="file" name="file" id="file" multiple />
<textarea name="text"></textarea>
<button type="button" onclick="sendComment();">Send</button>
</form>
I tried using Javascript model to process the data and I was able to save the asset correctly, however only one. I couldn't access the other files in the model.
I tried solving it (and improving user experience) by using AJAX and a REST endpoint. I opted not to use the Nodes endpoint API, because I didn't know how to solve the permission issue. I know I can restrict access to REST for each role, but not based on ownership of the Post. So I created my own Java-based endpoint (copied from documentation).
In the sendComment() function in Javascript I create an object with properties like name, extension, mimeType, ..., and data. I read in the documentation that you should send the data using the Base64 format, so I used FileReader() to accomplish that:
var fileObject = {
// properties like name, extension, mimeType, ...
}
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
// this part is easy
};
xhttp.open("PUT", "http://localhost:8080/myApp/.rest/assets/v1/saveAsset", true);
xhttp.setRequestHeader("Content-type", "application/json");
var reader = new FileReader();
reader.onload = function() {
fileObject.data = reader.result;
// I also tried without the 'data:image/png;base64,' part by reader.result.split(",")[1];
xhttp.send(JSON.stringify(fileObject));
};
reader.readAsDataURL(file); //where file is the value of the input element.files[i]
In Java I created a POJO class that has the same properties as the javascript object. Including a String data.
The code for the endpoint looks like this:
public Response saveAsset(Asset file) {
// Asset is my custom POJO class
Session damSession;
Node asset;
Node resource;
try {
damSession = MgnlContext.getJCRSession("dam");
asset = damSession.getRootNode().addNode(file.getName(), "mgnl:asset");
asset.setProperty("name", file.getName());
asset.setProperty("type", file.getExtension());
resource = asset.addNode("jcr:content", "mgnl:resource");
InputStream dataStream = new ByteArrayInputStream(file.getData().getBytes());
ValueFactory vf = damSession.getValueFactory();
Binary dataBinary = vf.createBinary(dataStream);
resource.setProperty("jcr:data", dataBinary);
resource.setProperty("fileName", file.getName());
resource.setProperty("extension", file.getExtension());
resource.setProperty("size", file.getSize());
resource.setProperty("jcr:mimeType", file.getMimeType());
damSession.save();
return Response.ok(LinkUtil.createLink(asset)).build();
} catch (RepositoryException e) {
return Response.ok(e.getMessage()).build(); //I know it's not ok, but that's not important at the moment
}
}
The asset gets created, the properties get saved apart from the jcr:data. If I upload an image and then download it either by the link I get as a response or directly from the Assets app, it cannot be opened, I get the format is not supported message. The size is 0, image doesn't show in the Assets app, seems like the data is simply not there, or it's in the wrong format.
How can I send the file or the file data to the Java endpoint? And how should I receive it? Does anybody know what am I missing? I honestly don't know what else to do with it.
Thank you

The input stream had to be decoded from Base64.
InputStream dataStream = new ByteArrayInputStream(Base64.decodeBase64(file.getData().getBytes()));
...and it only took me less than 3 months.
Noticed it after going through the source code for REST module that for unmarshalling Binary Value it had to be encoded to Base64, so I tried decoding it and it started to work.

How to access data sent by node sever using client side javascript

I am using a node sever to send a table from a sqlite db to the browser. This table contains filename and path of a pdf file that I want to render on the browser. Until now I was using hard coded paths for the the pdf file and rendering. But now i have setup a get route and a controller in node such that whenever '/content' is hit in browser , the server queries the database and and sends the data to the client. To the send the data I am using
res.render('content/index',{data:queryData});
Now, how do I access this data using client side javascript so that I can pass the path of the pdf file to the function that renders the pdf? I have done research and the nearest answer I got was using XMLHttpRequest. I tried this method
var xhr = new XMLHttpRequest();
const path = "http://localhost:3000/content";
xhr.onreadystatechange = function () {
if (xhr.readyState == 4 && xhr.status == 200)
{
var myResponseText = xhr.responseText;
console.log(myResponseText);
}
};
xhr.open('get', path, true);
xhr.send();
When I do this I get the entire html code for the view. Not the data I expected. How do I solve this issue. I have done some more reading while writing this post and I suppose. I have set a header somewhere? But the documentation says
app.render(view, [locals], callback)
which means res.render can take local variables, shouldn't be setting the headers?

You should return json instead of render template:
app.get('content/index', (req, res) => {
res.json({data: queryData});
});

I am using pdf.js
PDF.js needs the PDF file, e.g.:
pdfjsLib.getDocument('helloworld.pdf')
I'm assuming your queryData goes something like this:
{ filename: 'file.pdf', path: './path/to/file.pdf' }
I'm not sure what's in your content/index or what path this is on, but you obviously need to find a way to make your PDF file ('./path/to/file.pdf') available (as a download). See Express's built-in static server or res.download() to do that.
Once you have the PDF file available as a download, plug that path into PDF.js's .getDocument('/content/file.pdf') and do the rest to render the PDF onto the canvas or whatever.
Hope that helps.

How to execute a function on the node server by using vanilla javascript

Okay so basically I'm trying to create a typing game that uses a JSON file as the database. I'm using Nodejs to manipulate the use of a .json file with the fs module. What I want to do is have a main.js which operates the website's appearance which will contain a table of high scores, and I also want a player.js which will run on the back-end which updates a JSON file containing each player's name and high score.
So I was experimenting with it. I learned the hard way that require() is a node function. I want main.js to call a function from player.js and send it parameters.
var playerArr = player.updatePlayers("Rain", 30);
I know this doesn't work but the updatePlayers() function written with node in player.js returns an array and I want to set it to playerArr within main.js.
require() comes up with an error in the browser because its not a part of vanilla Javascript and I know it has to be run on a node server.
Is there any way to do this? It's for a school project and I'd love to be able to implement Nodejs and JSON into it.
Okay so everything's working except one crucial thing... the array isn't being sent back to the browser. I decided to use routes from express:
app.get("/:name,:score", returnScoreboard);
function returnScoreboard(request, response){
let data = request.params;
playerArr = updatePlayers(data.name, data.score);
response.send(playerArr);
}
Now in my main.js, this is my code:
var playerArr;
var updatePlayer = new XMLHttpRequest();
updatePlayer.onreadystatechange = () => {
playerArr = this.responseText;
}
updatePlayer.open("GET", "/Moo,100", true);
updatePlayer.send();
console.log(playerArr);
And I can't seem to get a response from into the playerArr. The JSON file that I'm using is getting updated correctly with a new object every time ({"name":"Moo", "score":100}) but it's not being assigned to playerArr. Please help!

I suggest you have a Node backend and use something like https://expressjs.com/. You can use it to set up some endpoint(s). These endpoints would be called upon by your frontend JS code using Fetch or something of the like, https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch.
Response to the edits above:
I think you are probably facing an asynchronous issue. I just moved the console.log into the onreadystatechange call. The onreadystatechange executes whenever a response comes back from the server, so by the time your code got to the original console.log the onreadystatechange may not have fired.
var playerArr;
var updatePlayer = new XMLHttpRequest();
updatePlayer.onreadystatechange = () => {
playerArr = this.responseText;
console.log(playerArr);
}
updatePlayer.open("GET", "/Moo,100", true);
updatePlayer.send();

AJAX Upload file straight after downloading it (without storing)

I'm making a JavaScript script that is going to essentially save an old game development sandbox website before the owners scrap it (and lose all of the games). I've created a script that downloads each game via AJAX, and would like to somehow upload it straight away, also using AJAX. How do I upload the downloaded file (that's stored in responseText, presumably) to a PHP page on another domain (that has cross origin headers enabled)?
I assume there must be a way of uploading the data from the first AJAX request, without transferring the responseText to another AJAX request (used to upload the file)? I've tried transferring the data, but as expected, it causes huge lag (and can crash the browser), as the files can be quite large.
Is there a way that an AJAX request can somehow upload individual packets as soon as they're recieved?
Thanks,
Dan.

You could use Firefox' moz-chunked-text and moz-chunked-arraybuffer response types. On the JavaScript side you can do something like this:
function downloadUpload() {
var downloadUrl = "server.com/largeFile.ext";
var uploadUrl = "receiver.net/upload.php";
var dataOffset = 0;
xhrDownload = new XMLHttpRequest();
xhrDownload.open("GET", downloadUrl, true);
xhrDownload.responseType = "moz-chunked-text"; // <- only works in Firefox
xhrDownload.onprogress = uploadData;
xhrDownload.send();
function uploadData() {
var data = {
file: downloadUrl.substring(downloadUrl.lastIndexOf('/') + 1),
offset: dataOffset,
chunk: xhrDownload.responseText
};
xhrUpload = new XMLHttpRequest();
xhrUpload.open("POST", uploadUrl, true);
xhrUpload.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');
xhrUpload.send(JSON.stringify(data));
dataOffset += xhrDownload.responseText.length;
};
}
On the PHP side you need something like this:
$in = fopen("php://input", "r");
$postContent = stream_get_contents($in);
fclose($in);
$o = json_decode($postContent);
file_put_contents($o->file . '-' . $o->offset . '.txt', $o->chunk);
These snippets will just give you the basic idea, you'll need to optimize the code yourself.