We Keep Coding

Decoding url not working properly JS I tried decodeURIComponent

Hi I have a case a URL simillar to this:
https://linkhere?response-content-disposition=inline; filename="placeholder.jpg"; filename*=UTF-8''placeholder.jpg response-content-type=image/jpeg&X-Amz-Algorithm=....sometextcontinueshere
I am trying to decode it like this and I need to take the filename
const decodedUrl = decodeURIComponent("linkhere")
const urlParams = new URLSearchParams(decodedUrl);
const filename = urlParams.get('filename');
console.log(decodedUrl)
But for some reason does not work the decoding properly, do you guys have any idea?

There is nothing built in that is magically going to get the filename since it is not a querystring paramter. The easiest thing you can do it change whatever is building this to have valid querystring parameters so you can parse it.
With what you have, you will need to read the one parameter that has the file name in it. After that you are going to have to parse out the filename from that string.
Basic idea:
var str = `https://linkhere?response-content-disposition=inline; filename="placeholder.jpg"; filename*=UTF-8''placeholder.jpg response-content-type=image/jpeg&X-Amz-Algorithm=....sometextcontinueshere`;
const url = new URL(str);
const parts = url.searchParams.get('response-content-disposition').split(/;\s/);
const fileNameParam = parts.find(x => x.startsWith('filename=')).match(/"([^"]+)"/)[1];
console.log(fileNameParam);

How do I download a file from FastAPI backend using Fetch API in the frontend?

This is my FastAPI(python) code, which returns a .ics file:
#app.get("/latLong/")
async def read_item(lat: float,long:float):
mainFunc(lat,long)
return FileResponse("/tmp/myics.ics")
This is my frontend code in Javascript using Fetch API:
<script>
async function apiCall(long,lat) {
let myObject = await fetch('myapi.com/lat/long');
let myText = await myObject.text();
}
</script>
So from my visor (my api logs), it successfully calls the API. But from the front end, I am trying to get it to return the file.
The end result I would like to achieve is when the user clicks a button, the browser grabs the location, then sends the location to the API, and the API returns a file that the user can download.

First, you need to adjust your endpoint on server side to accept path parameters, as in the way it is currently defined, lat and long are expected to be query parameters; however, in your javascript code you are trying to send those coordinates as path parameters. Thus, your endpoint should look like this:
#app.get("/{lat}/{long}/")
async def read_item(lat: float, long: float):
Next, set the filename in FileResponse, so that it can be included in the Content-Disposition response header, which can later be retrieved on client side:
return FileResponse("/tmp/myics.ics", filename="myics.ics")
If you are doing a cross-origin request (see FastAPI CORS as well), make sure to set the Access-Control-Expose-Headers response header on server side, indicating that the Content-Disposition header should be made available to JS scripts running in the browser; otherwise, the filename won't be accessible on client side. Example:
headers = {'Access-Control-Expose-Headers': 'Content-Disposition'}
return FileResponse("/tmp/myics.ics", filename="myics.ics", headers=headers)
On client side, you could use a similar approach to this answer or this answer. The below example also takes into account scenarios where the filename includes unicode characters (i.e., -, !, (, ), etc.) and hence, comes (utf-8 encoded) in the form of, for instance, filename*=utf-8''Na%C3%AFve%20file.txt (see here for more details). In such cases, the decodeURIComponent() function is used to decode the filename. Working example below:
const url ='http://127.0.0.1:8000/41.64007/-47.285156'
fetch(url)
.then(res => {
const disposition = res.headers.get('Content-Disposition');
filename = disposition.split(/;(.+)/)[1].split(/=(.+)/)[1];
if (filename.toLowerCase().startsWith("utf-8''"))
filename = decodeURIComponent(filename.replace("utf-8''", ''));
else
filename = filename.replace(/['"]/g, '');
return res.blob();
})
.then(blob => {
var url = window.URL.createObjectURL(blob);
var a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a); // append the element to the dom
a.click();
a.remove(); // afterwards, remove the element
});

Md5 always same hash when drag&drop in javascript

I have drag&drop event and I would like to hash the filed dragged. I have this:
var file = ev.dataTransfer.items[i].getAsFile();
var hashf = CryptoJS.SHA512(file).toString();
console.log("hashf", hashf)
But when I drag differents files, "hashf" is always the same string.
https://jsfiddle.net/9rfvnbza/1/

The issue is that you are attempting to hash the File object. Hash algorithms expect a string to hash.
When passing the File Object to the CryptoJS.SHA512() method, the API attempts to convert the object to a string. That conversion results in CryptoJS.SHA512() receiving the same string not matter what File object you send provide it.
The string is [object File] - you can replace file in your code with that string and discover it is the same hash code you've see all along.
To fix this, retrieve the text from the file first and pass that to the hashing algorithm:
file.text().then((text) => {
const hashf = CryptoJS.SHA512(text).toString();
console.log("hashf", hashf);
});
If you prefer async/await, you can put it in an IIFE:
(async() => {
const text = await file.text()
const hashf = CryptoJS.SHA512(text).toString();
console.log("hashf", hashf);
})();

Mechanisms for hashing a file in JavaScript

I am relatively new to JavaScript and I want to get the hash of a file, and would like to better understand the mechanism and code behind the process.
So, what I need: An MD5 or SHA-256 hash of an uploaded file to my website.
My understanding of how this works: A file is uploaded via an HTML input tag of type 'file', after which it is converted to a binary string, which is consequently hashed.
What I have so far: I have managed to get the hash of an input of type 'text', and also, somehow, the hash of an uploaded file, although the hash did not match with websites I looked at online, so I'm guessing it hashed some other details of the file, instead of the binary string.
Question 1: Am I correct in my understanding of how a file is hashed? Meaning, is it the binary string that gets hashed?
Question 2: What should my code look like to upload a file, hash it, and display the output?
Thank you in advance.

Basically yes, that's how it works.
But, to generate such hash, you don't need to do the conversion to string yourself. Instead, let the SubtleCrypto API handle it itself, and just pass an ArrayBuffer of your file.
async function getHash(blob, algo = "SHA-256") {
// convert your Blob to an ArrayBuffer
// could also use a FileRedaer for this for browsers that don't support Response API
const buf = await new Response(blob).arrayBuffer();
const hash = await crypto.subtle.digest(algo, buf);
let result = '';
const view = new DataView(hash);
for (let i = 0; i < hash.byteLength; i += 4) {
result += view.getUint32(i).toString(16).padStart(2, '0');
}
return result;
}
inp.onchange = e => {
getHash(inp.files[0]).then(console.log);
};
<input id="inp" type="file">

Gmail API decoding messages in Javascript

I am having serious problems decoding the message body of the emails I get using the Gmail API. I want to grab the message content and put the content in a div. I am using a base64 decoder, which I know won't decode emails encoded differently, but I am not sure how to check an email to decide which decoder to use -- emails that say they are utf-8 encoded are successfully decoded by the base64 decoder, but not be a utf-8 decoder.
I've been researching email decoding for several days now, and I've learned that I am a little out of my league here. I haven't done much work with coding around email before. Here is the code I am using to get the emails:
gapi.client.load('gmail', 'v1', function() {
var request = gapi.client.gmail.users.messages.list({
labelIds: ['INBOX']
});
request.execute(function(resp) {
document.getElementById('email-announcement').innerHTML = '<i>Hello! I am reading your <b>inbox</b> emails.</i><br><br>------<br>';
var content = document.getElementById("message-list");
if (resp.messages == null) {
content.innerHTML = "<b>Your inbox is empty.</b>";
} else {
var encodings = 0;
content.innerHTML = "";
angular.forEach(resp.messages, function(message) {
var email = gapi.client.gmail.users.messages.get({
'id': message.id
});
email.execute(function(stuff) {
if (stuff.payload == null) {
console.log("Payload null: " + message.id);
}
var header = "";
var sender = "";
angular.forEach(stuff.payload.headers, function(item) {
if (item.name == "Subject") {
header = item.value;
}
if (item.name == "From") {
sender = item.value;
}
})
try {
var contents = "";
if (stuff.payload.parts == null) {
contents = base64.decode(stuff.payload.body.data);
} else {
contents = base64.decode(stuff.payload.parts[0].body.data);
}
content.innerHTML += '<b>Subject: ' + header + '</b><br>';
content.innerHTML += '<b>From: ' + sender + '</b><br>';
content.innerHTML += contents + "<br><br>";
} catch (err) {
console.log("Encoding error: " + encodings++);
}
})
})
}
});
});
I was performing some checks and debugging, so there's leftover console.log's and some other things that are only there for testing. Still, you can see here what I am trying to do.
What is the best way to decode the emails I pull from the Gmail API? Should I try to put the emails into <script>'s with charset and type attributes matching the encoding content of the email? I believe I remember charset only works with a src attribute, which I wouldn't have here. Any suggestions?

For a prototype app I'm writing, the following code is working for me:
var base64 = require('js-base64').Base64;
// js-base64 is working fine for me.
var bodyData = message.payload.body.data;
// Simplified code: you'd need to check for multipart.
base64.decode(bodyData.replace(/-/g, '+').replace(/_/g, '/'));
// If you're going to use a different library other than js-base64,
// you may need to replace some characters before passing it to the decoder.
Caution: these points are not explicitly documented and could be wrong:
The users.messages: get API returns "parsed body content" by default. This data seems to be always encoded in UTF-8 and Base64, regardless of the Content-Type and Content-Transfer-Encoding header.
For example, my code had no problem parsing an email with these headers: Content-Type: text/plain; charset=ISO-2022-JP, Content-Transfer-Encoding: 7bit.
The mapping table of the Base64 encoding varies among various implementations. Gmail API uses - and _ as the last two characters of the table, as defined by RFC 4648's "URL and Filename safe Alphabet"1.
Check if your Base64 library is using a different mapping table. If so, replace those characters with the ones your library accepts before passing the body to the decoder.
1 There is one supportive line in the documentation: the "raw" format returns "body content as a base64url encoded string". (Thanks Eric!)

Use atob to decode the messages in JavaScript (see ref). For accessing your message payload, you can write a function:
var extractField = function(json, fieldName) {
return json.payload.headers.filter(function(header) {
return header.name === fieldName;
})[0].value;
};
var date = extractField(response, "Date");
var subject = extractField(response, "Subject");
referenced from my previous SO Question and
var part = message.parts.filter(function(part) {
return part.mimeType == 'text/html';
});
var html = atob(part.body.data);
If the above does not decode 100% properly, the comments by #cgenco on this answer below may apply to you. In that case, do
var html = atob(part.body.data.replace(/-/g, '+').replace(/_/g, '/'));

Here is the solution:
Gmail API - "Users.messages: get" method has in response message.payload.body.data parted base64 data, it's separated by "-" symbol. It's not entire base64 encoded text, it's parts of base64 text. You have to try to decode every single part of this or make one mono string by unite and replace "-" symbol. After this you can easily decode it to human text.
You can manually check every part here https://www.base64decode.org

I was also annoyed by this point. I discovered a solution through looking at an extension for VSCode. The solution is really simple:
const body = response.data.payload.body; // the base64 encoded body of a message
body = Buffer.alloc(
body.data.length,
body.data,
"base64"
).toString(); // the decoded message
It worked for me as I was using gmail.users.messages.get() call of Gmail API.

Please use websafe decoder for decoding gmail emails and attachments. I got blank pages when I used just base64decoder, had to use this: https://www.npmjs.com/package/urlsafe-base64

I can easily decode using another tool at https://simplycalc.com/base64-decode.php
In JS: https://www.npmjs.com/package/base64url
In Python 3:
import base64
base64.urlsafe_b64decode(coded_string)

Thank #ento 's answer. I explain more why you need to replace '-' and '_' character to '+' and '/' before decode.
Wiki Base64 Variants summary table shows:
RFC 4648 section 4: base64 (standard): use '+' and '/'
RFC 4648 section 5: base64url (URL-safe and filename-safe standard): use '-' and '_'
In short, Gmail API use base64url (urlsafe) format('-' and '_'), But JavaScript atob function or other
JavaScript libraries use base64 (standard) format('+' and '/').
For Gmail API, the document says body use base64url format, see below links:
string/bytes type
MessagePartBody
RAW
For Web atob/btoa standards, see below links:
The algorithm used by atob() and btoa() is specified in RFC 4648, section 4
8.3 Base64 utility methods
Forgiving base64