I've got a blob of audio data confirmed to play in the browser but fails to play after storing, retrieving, and conversion of the same data. I've tried a few methods without success, each time returning the error:
Uncaught (in promise) DOMException: Failed to load because no supported source was found
Hasura notes that bytea data must be passed in as a String, so I tried a couple things.
Converting the blob into base64 stores fine but the retrieval and playing of the data doesn't work. I've tried doing conversions within the browser to base64 and then back into blob. I think it's just the data doesn't store properly as bytea if I convert it to base64 first:
// Storing bytea data as base64 string
const arrayBuffer = await blob.arrayBuffer();
const byteArray = new Uint8Array(arrayBuffer);
const charArray = Array.from(byteArray, (x: number) => String.fromCharCode(x));
const encodedString = window.btoa(charArray.join(''));
hasuraRequest....
`
mutation SaveAudioBlob ($input: String) {
insert_testerooey_one(
object: {
blubberz: $input
}
) {
id
blubberz
}
}
`,
{ input: encodedString }
);
// Decoding bytea data
const decodedString = window.atob(encodedString);
const decodedByteArray = new Uint8Array(decodedString.length).map((_, i) =>
decodedString.charCodeAt(i)
);
const decodedBlob = new Blob([decodedByteArray.buffer], { type: 'audio/mpeg' });
const audio4 = new Audio();
audio4.src = URL.createObjectURL(decodedBlob);
audio4.play();
Then I came across a Github issue (https://github.com/hasura/graphql-engine/issues/3336) suggesting the use of a computed field to convert the bytea data to base64, so I tried using that instead of my decoding attempt, only to be met with the same error:
CREATE OR REPLACE FUNCTION public.content_base64(mm testerooey)
RETURNS text
LANGUAGE sql
STABLE
AS $function$
SELECT encode(mm.blobberz, 'base64')
$function$
It seemed like a base64 string was not the way to store bytea data, so I tried converting the data to a hex string prior to storing. It stores ok, I think, but upon retrieval the data doesn't play, and I think it's a similar problem as storing as base64:
// Encoding to hex string
const arrayBuffer = await blob.arrayBuffer();
const byteArray = new Uint8Array(arrayBuffer);
const hexString = Array.from(byteArray, (byte) =>
byte.toString(16).padStart(2, '0')
).join('');
But using the decoded data didn't work again, regardless of whether I tried the computed field method or my own conversion methods. So, am I just not converting it right? Is my line of thinking incorrect? Or what is it I'm doing wrong?
I've got it working if I just convert to base64 and store as a text field but I'd prefer to store as bytea because it takes up less space. I think something's wrong with how the data is either stored, retrieved, or converted, but I don't know how to do it. I know the blob itself is fine because when generated I can play audio with it, it only bugs out after fetching and attempted conversion its stored value. Any ideas?
Also, I'd really like to not store the file in another service like s3, even if drastically simpler.
Related
In trying to send document scans as binary data over a web socket the first block of code below works fairly well. Since the first byte contains information concerning what exactly to do with the blob, the image data starts at an offset of one byte; and because it appears that blob.slice() returns a copy of the original blob rather than simply reading it, this likely is not the most efficient method of processing the blob because it makes a copy of the entire blob less one byte.
socket.onmessage = function(evt) {
if (evt.data instanceof Blob) {
evt.data.slice(0,1).arrayBuffer()
.then( (b) => {
let v = new DataView(b),
r = v.getUint8(0);
// Do something based on r and then display scan.
let objectURL = URL.createObjectURL( evt.data.slice(1) );
imgscan.onload = () => {
URL.revokeObjectURL( objectURL );
};
imgscan.src = objectURL;
})
If the websocket's binaryType is changed to arraybuffer, it's a bit easier to read the first byte and apparently does not make a copy, but I do not understand how to get the image data out of the buffer to display it; that is, I don't see which method to apply to the DataView to get the raw binary data of the image. Would you please explain or point me to the correct documentation? Thank you.
This SO question seems similar but was not helpful to me anyway.
socket.onmessage = function(evt) {
if ( evt.data instanceof ArrayBuffer ) {
let v = new DataView(evt.data),
r = v.getUint8(0);
// How to get the raw binary image data out of
// the array buffer starting at the first byte?
I am new to React and want to display an image downloaded as binary data. I download the image data from api call to adobe lightroom api. The api call works since the image is displayed in Postman without problems. I can also save the image data to a jpeg-file and it is displayed ok.
In React I want to do <img src={`data:image/jpeg;base64,${theImage}`} /> and for that to work I need to convert the binary data to a base64 encoded string. When i convert the downloaded jpeg using cat image.jpeg|base64 > base64.txt the resulting string works in my React app.
But when I try var theImage = btoa(binarydata) in React I get Unhandled Rejection (InvalidCharacterError): Failed to execute 'btoa' on 'Window': The string to be encoded contains characters outside of the Latin1 range.
After searching the issue I try use var theImage = btoa(unescape(encodeURIComponent( binarydata ))) and similar proposed solution but resulting strings from those does not turn out to be a valid base64 encodings of the jpeg as it seem (I try the result from the conversions in online base64->image services and no image is shown). I have also tried other proposed solution such as base64-js and js-base64 libraries and non of those create a valid base64 valid image that can be shown in my React code.
How do you convert jpeg binary data to valid Base64 image encoding when btoa throws latin1 exception?
You've said you're using axios.get to get the image from the server. What you'll presumably get back will be a Buffer or ArrayBuffer or Blob, etc., but it depends on what you do with the response you get from axios.
I don't use axios (never felt the need to), but you can readily get a data URI for binary data from the server via fetch:
// 1.
const response = await fetch("/path/to/resource");
// 2.
if (!response.ok) {
throw new Error("HTTP error " + response.status);
}
// 3.
const buffer = await response.arrayBuffer();
// 4.
const byteArray = new Uint8Array(buffer);
// 5.
const charArray = Array.from(byteArray, byte => String.fromCharCode(byte));
// 6.
const binaryString = charArray.join("");
// 7.
const theImage = btoa(binaryString);
Or more concisely:
const response = await fetch("/path/to/resource");
if (!response.ok) {
throw new Error("HTTP error " + response.status);
}
const buffer = await response.arrayBuffer();
const binaryString = Array.from(new Uint8Array(buffer), byte => String.fromCharCode(byte)).join("");
const theImage = btoa(binaryString);
Here's how that works:
We request the image data.
We check that the request worked (fetch only rejects its promise on network errors, not HTTP errors; those are reported via the status and ok props.
We read the body of the response into an ArrayBuffer; the buffer will have the binary image data.
We want to convert that buffer data into a binary string. To do that, we need to access the bytes individually, so we create a Uint8Array (using that buffer) to access them.
To convert that byte array into a binary string, we need to convert each byte into its equivalent character, and join those together into a string. Let's do that by using Array.from and in its mapping function (called for each byte), we'll use String.fromCharCode to convert the byte to a character. (It's not really much of a conversion. The byte 25 [for instance] becomes the character with character code 25 in the string.)
Now we create the binary string by joining the characters in that array together into one string.
Finally, we convert that string to Base64.
Looking at the docs, it looks like axios lets you provide the option responseType: "arraybuffer" to get an array buffer. If I'm reading right, you could use axios like this:
const response = await axios.get("/path/to/resource", {responseType: "arraybuffer"});
const binaryString = Array.from(new Uint8Array(response.body), v => String.fromCharCode(v)).join("");
const theImage = btoa(binaryString);
Fetch your image as a Blob and generate a blob:// URI from it.
data:// URLs are completely inefficient and require far more memory space than blob:// URLs. The data:// URL takes 34% more space than the actual data it represents and it must be stored in the DOM + decoded as binary again to be read by the image decoder. The blob:// URI on the other hand is just a pointer to the binary data in memory.
blob:// URLs are not perfect, but until browsers implement srcDoc correctly, it's still the best we have.
So if as per the comments you are using axios in a browser, you can do
const blob = await axios.get("/path/to/resource", {responseType: "blob"});
const theImage = URL.createObjectURL(blob);
And if you want to use the fetch API
const response = await fetch("/path/to/resource");
if (!response.ok) {
throw new Error("HTTP error " + response.status);
}
const blob = await response.blob();
const theImage = URL.createObjectURL(blob);
Ps:
If you don't have any particular reason to fetch this image through AJAX (e.g credentials or special POST params), then pass directly the URL of the resource as the src of your image:
<img src="/path/to/resource" />
I am relatively new to JavaScript and I want to get the hash of a file, and would like to better understand the mechanism and code behind the process.
So, what I need: An MD5 or SHA-256 hash of an uploaded file to my website.
My understanding of how this works: A file is uploaded via an HTML input tag of type 'file', after which it is converted to a binary string, which is consequently hashed.
What I have so far: I have managed to get the hash of an input of type 'text', and also, somehow, the hash of an uploaded file, although the hash did not match with websites I looked at online, so I'm guessing it hashed some other details of the file, instead of the binary string.
Question 1: Am I correct in my understanding of how a file is hashed? Meaning, is it the binary string that gets hashed?
Question 2: What should my code look like to upload a file, hash it, and display the output?
Thank you in advance.
Basically yes, that's how it works.
But, to generate such hash, you don't need to do the conversion to string yourself. Instead, let the SubtleCrypto API handle it itself, and just pass an ArrayBuffer of your file.
async function getHash(blob, algo = "SHA-256") {
// convert your Blob to an ArrayBuffer
// could also use a FileRedaer for this for browsers that don't support Response API
const buf = await new Response(blob).arrayBuffer();
const hash = await crypto.subtle.digest(algo, buf);
let result = '';
const view = new DataView(hash);
for (let i = 0; i < hash.byteLength; i += 4) {
result += view.getUint32(i).toString(16).padStart(2, '0');
}
return result;
}
inp.onchange = e => {
getHash(inp.files[0]).then(console.log);
};
<input id="inp" type="file">
I have a SOAP API that is returning me a file divided in chunks encoded in several base64 strings
i'm not being able to save it to the file system without breaking it
This is the pastebin of a whole file encoded, as is, once i download and chain the responses.
What is the way to save it correctly?
i tried in many ways
var f = Ti.FileSystem.getFile(Ti.FileSystem.tempDirectory, 'test.pdf');
...
var blobStream = Ti.Stream.createStream({ source: fileString, mode: Ti.Stream.MODE_READ });
var newBuffer = Ti.createBuffer({ length: fileString.length });
f.write(fileString);
or
var data = Ti.Utils.base64decode(fileString);
var blobStream = Ti.Stream.createStream({ source: data, mode: Ti.Stream.MODE_READ });
var newBuffer = Ti.createBuffer({ length: data.length });
var bytes = blobStream.read(newBuffer);
f.write(fileString);
or
var data = Ti.Utils.base64decode(fileString);
var blobStream = Ti.Stream.createStream({ source: data, mode: Ti.Stream.MODE_READ });
var newBuffer = Ti.createBuffer({ length: data.length });
var bytes = blobStream.read(newBuffer);
f.write(bytes);
but i'm not understanding which one is the right path
Do I have to convert back to byteArray the string on my own?
What is the right way to save it?
Do I have to create a buffer from the string or ...?
I think that the base64enc for the file is not valid or incomplete, I've tested it using bash and base64 utils. You can perform these steps.
Copy and paste the base64 string on a file called pdf.base64 then run this command:
cat pdf.base64 | base64 --decode >> out.pdf
the output file is not a valid pdf.
You can try to encode and decode a valid pdf file to take a look at the generated binary:
cat validfile.pdf | base64 | base64 --decode >> anothervalidfile.pdf
Try to check if you are chainig chunks correctly or simply make a phone call with the guy who build the soap api.
Before you start downloading your file you need to create the file stream to write too, writing to a blob is not the way to go:
// Step 1
var outFileStream = Ti.Filesystem.getFile('outfile.bin').open(Ti.Filesystem.MODE_WRITE);
After creating your HTTPClient or socket stream and when you receive a chunk of Base64 data from the serve, you need to put that decoded data into a Titanium.Buffer. This would probably go into your onload or onstream in an HTPPClient, :
// Step 2
var rawDecodedFileChunk = Ti.Utils.base64decode(fileString);
var outBuffer = Ti.createBuffer({
byteOrder : // May need to set this
type : // May also need to set this to match data
value: rawDecodedFileChunk
});
Finally you can write the data out to the file stream:
// Step 3
var bytesWritten = outFileStream.write(outBuffer); // writes entire buffer to stream
Ti.API.info("Bytes written:" + bytesWritten); // should match data length
if(outBuffer.length !== bytesWritten) {
Ti.API.error("Not all bytes written!");
}
Generally errors come from having the wrong byte order or type of data, or writing in the wrong order. Obviously, this all depends on the server sending the data in the correct order and it being valid!
You may also want to consider the pump command version of this, which allows you to transfer from input stream to output file stream, minimizing your load.
lets say i have a URL given. I would like to:
1) download it and convert to base64
2) upload it to some key/value storage (as text)
3) download it from key/value storage (with text/plain mimetype), reencode it from base64, display it.
Best Regards
If someone is still searching for downloading images and encoding them in base64 string, I recently find this kind of outdated method but really reliable. The advantage is that it's pure Javascript so there is no need to install any external library. I previously tried using fetch and axios but for some reason the encoded string was not in a correct format.
NB: If you are encoding this image to send it to an API, some of them require to delete the leading data type including the , at the start of the encoded string.
function toDataURL (url, callback) {
const xhRequest = new XMLHttpRequest()
xhRequest.onload = function () {
const reader = new FileReader()
reader.onloadend = function () {
callback(reader.result)
}
reader.readAsDataURL(xhRequest.response)
}
xhRequest.open('GET', url)
xhRequest.responseType = 'blob'
xhRequest.send()
}
const URL = "https://upload.wikimedia.org/wikipedia/commons/f/f7/Stack_Overflow_logo.png"
const logCallback = (base64image) => {
// Base64 encoded string with leading data type like
// data:image/png;base64,RAW_ENCODED_DATA______
console.log(base64image)
}
toDataURL(URL, logCallback)