We Keep Coding

Convert input files to byte[] javascript

I'm working on a REST web application that manages documents between users and uploaders. The backend is written in Java and my Document entity contains, besides various attributes, a byte[] content. I was able to send a file created at server side by
#GET
...
document.setContent(Files.readAllBytes(Paths.get("WEB-INF/testFile.txt")));
return Response.ok(document).build();
and retrieve it at front-end (vueJs) through
async function download(file) {
const fileURL = window.URL.createObjectURL(new Blob([atob(file.content)]));
const fileLink = document.createElement("a");
fileLink.href = fileURL;
fileLink.setAttribute("download",`${file.name}.${file.type}`);
document.body.appendChild(fileLink);
fileLink.click();
fileLink.remove;
window.URL.revokeObjectURL(fileURL);
}
the problem is that when I try to upload a file and then download it, its content is not parsed correctly (is shown undefined, string in Base64 or numbers depending on how I try to solve it). The file is sent by a post request and is retrieved through an input form bound to an onFileSelected function.
function onFileSelected(e) {
var reader = new FileReader();
reader.readAsArrayBuffer(e.target.files[0]);
reader.onloadend = (evt) => {
if (evt.target.readyState === FileReader.DONE) {
var arrayBuffer = evt.target.result;
this.file.content = new Uint8Array(arrayBuffer);
//this.file.content = arrayBuffer;
}
};
}
axios.post(...,document,...)
and I have tried using atob and btoa as well before assigning the value to this.file.content. If I print the file on server Welcome.txt it gives B#ae3b74d and if I use Arrays.toString(welcome.getContent()) it gives an array of numbers but as soon as it passed to the frontend its content become in Base64 welcome: { ... content: IFRoaXMgaXMgYSB0ZXN0IGZpbGUhIAo...}. Any idea? Thank you a lot!

How to save JS audio Blob as a file so it can be sent to a Flask API

I'm building an app for my friend and I need to record audio and store it on a server. I have successfully done it with text and images, but I can't make it work for audio.
I am probably missing something when creating the file (the file is created but not playable). There has to be a problem with the data conversion from the JS Blob to the actual file.
I managed to get the audio blob and even play it back in JS. But when I create a file from the blob it can't be played (I tried to save it locally with the same outcome - got the file but could not play it). I also tried to save it in different formats with no success (wav, mp3). So the problem has to be in the conversion from the blob to the actual file. With text and images, it was straightforward forward and the files were created from the blob just by saving them with a filename. But I guess that with audio isn't that simple.
My understanding is that I have some binary data (JS Blob), that can be saved as a file. But with audio, there has to be some special conversion or encoding so the output file works and can be played.
here is the frontend code (I am using this with some of the variables because its part of a Vue component)
this.mediaRecorder.addEventListener("stop", () => {
// tried to save it as WAV with the same result got the file, but couldn't play it
this.audioBlob = new Blob(this.audioChunks, { 'type' : 'audio/mpeg-3' })
//debugging - playing back the sound in the browser works fine
const audioUrl = URL.createObjectURL(this.audioBlob);
const audio = new Audio(audioUrl);
audio.play();
//adding the blob to the request
let filename = this.$store.state.counter + "-" + this.$store.state.step
const formData = new FormData();
formData.append('file', this.audioBlob, `${filename}.mp3`);
const config = {
headers: { 'content-type': 'multipart/form-data' }
}
//sending it to my Flask API (xxx is the name of the folder it gets saved to on the server)
this.$axios.post('http://localhost:5000/api/v1/file/upload/xxx', formData, config)
})
here is my endpoint on the server
#app.route('/api/v1/file/upload/<test_id>', methods=['POST'])
def upload_file(test_id):
uploaded_file = request.files['file']
filename = secure_filename(uploaded_file.filename)
if filename != '':
uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], test_id, filename))
return jsonify({'message': 'file saved'})
Here is the whole recording code snippet
navigator.mediaDevices.getUserMedia({ audio: true })
.then(stream => {
this.mediaRecorder = new MediaRecorder(stream);
// audio.srcObject = stream
this.mediaRecorder.start();
this.mediaRecorder.addEventListener("dataavailable", event => {
this.audioChunks.push(event.data)
})
this.mediaRecorder.addEventListener("stop", () => {
this.audioBlob = new Blob(this.audioChunks, { 'type' : 'audio/mpeg-3' })
//debugging - playing back the sound in the browser works fine
const audioUrl = URL.createObjectURL(this.audioBlob);
const audio = new Audio(audioUrl);
audio.play();
//adding the blob to the request
let filename = this.$store.state.counter + "-" + this.$store.state.step
const formData = new FormData();
formData.append('file', this.audioBlob, `${filename}.mp3`);
const config = {
headers: { 'content-type': 'multipart/form-data' }
}
//sending it to my Flask API (xxx is the name of the folder it gets saved to on the server)
this.$axios.post('http://localhost:5000/api/v1/file/upload/xxx', formData, config)
})
})

Gcloud API file.save() data format

I'm using the gcloud API on a Nodejs web server to upload files. I'd prefer the files not be uploaded on the client side and instead uploaded on the server. Currently, I am producing a blob on the client side, then converting it to text and passing that to the server through a POST request. All of the information gets successfully passed from the client to the server as expected. This data is also uploaded to gcloud, however, Gcloud does not recognize this as a valid file nor does my computer when I download it.
What is the best way to get the contents of the file to gcloud from the server side? I've tried using dataURIs and reading the orignal file by text and both produce similiar issues. I've also explored piping a readFileStream from the blob on the server end but blobs are not natively supported by node so I have not done so yet.
Client Side
function readSingleFile(e, func, func2){
var file = e.target.files[0];
if(!file){
return; // Add error msg_here
}
var reader = new FileReader();
reader.onload = function(e){
let contents = e.target.result;
let img = document.createElement('img')
let cvs = document.createElement('canvas');
img.onload = ()=>{
cvs.width = img.width;
cvs.height= img.height;
let ctx = cvs.getContext('2d');
ctx.drawImage(img,0,0);
cvs.toBlob((res)=>{res.text().then((text)=>{func2(text)})}, "image/jpeg", 0.92);
}
img.src=contents;
func(contents);
}
reader.readAsDataURL(file);
}
Server Side
function publishPrintjob(dataObj){
try{
var newElemKey = database.ref().child('queue').push().key; // Get random Key
// Create a new blob in the bucket and upload the file data.
const gcloudFile = storage.file('images/' + newElemKey + '.jpg');
gcloudFile.save(dataObj.sockImageFile, function(err) {
if (!err) {
Console.log("File Uploaded!")
}
});
var data = {
date: dataObj.Date,
email: dataObj.email,
design: dataObj.Design,
author: dataObj.Author,
address: dataObj.address,
imageKey: newElemKey,
}
admin.database().ref('queue/' + newElemKey).set(data);
} catch(err){
console.log(err)
}
}
Note: func simply shows the image on the client side, func2 just adds the contents to the POST object.

Uploading a file directly from the computer would be easiest using the storage.bucket(bucketName).upload() function from the cloud storage library. However, this uses location of a file locally and thus will not work unless a file is transferred to the server and saved first. This could be achieved using multi-part form data. Using multipart or uploading locally are better methods for uploading to google storage.
Instead, I solve this by first converting the image to a dataURI, sending the data URI to the server via the body of a GET request, and then converting it to a buffer with a readable stream that can be piped to google storage.
Client
let formData = getFormData('myForm');
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
// Typical action to be performed when the document is ready:
}
};
xhttp.open("POST", "dashboard", true);
xhttp.setRequestHeader('Content-Type', 'application/json');
xhttp.send(JSON.stringify(formData));
xhttp.onload = ()=> {
console.log(JSON.parse(xhttp.response))
// Handle server response here
};
}
Server
// DataObject is the body of the GET request, the property imageFile is the URI from readFileAsURI
function uploadImageOnServer(dataObj){
try{
var newElemKey = database.ref().child('queue').push().key; // Get random Key to use as filename
// Create a new blob in the bucket and upload the file data.
const gcloudFile = storage.file('images/' + newElemKey + '.jpeg');
var fs = require('fs'); // Should be required at the top of the file
var string = dataObj.ImageFile;
var regex = /^data:.+\/(.+);base64,(.*)$/;
var matches = string.match(regex);
var ext = matches[1];
var data = matches[2];
var buffer = Buffer.from(data, 'base64');
// Create the readstream
const readableInstanceStream = new Readable({
read() {
this.push(buffer);
this.push(null);
}
});
readableInstanceStream.pipe(gcloudFile.createWriteStream()) // link to gcloud storage api
.on('error', function(err) {
console.log('error')
})
.on('finish', function() {
console.log('upload complete')
});
} catch(err){
console.log(err)
}
}

Uint8array to readable stream in nodejs [duplicate]

I'm working with the Microsoft Bot Framework (hosting on Azure) using Node.js, and saving files locally isn't an option. I have image data in base64 format, and I need to transfer that data into a readable stream of an image file that I can pump into Azure using their Blob Storage API (which takes a stream).
var base64ImageData; //my base64 image
//Create myStream here
//Then send to Azure
blobService.createBlockBlobFromStream('mycontainer',
nameForBlob, myStream, fileSize,
function (error, result, response){
if(!error)
console.log(response);
else
console.log(error)
});
Any help would be appreciated. I can't figure out how to decode the base64 and make a stream out of it without saving a jpg onto my disk.

Please try something like this:
var azureStorage = require('azure-storage');
var streamifier = require('streamifier');
var base64String = 'iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAEiSURBVDhPxZIhT0NBEITnoBKBrEQgEUhISEiqKiFUYJDI/gBEc8VVY5AYHKKGBIEAVBFN+A8EUVFZUdGk/XbvGtqmpS8ImGTu5vbd7N3tPv0Pog5gzeSGB4oiqgSbCnphNbRQsKEQorbY/YCqaqwrXatl4WIJosrsfELtw3vucOFxsP4J6eRnlJnfOf3S4xk/J0hmO3kPfmE+5er+9ilSAqtoU203TGEFC7pDHcFBNvf82wxSgqAzxhPmDsbdHLtl9FZhbmDuul5AKmLUNuoDtQMH8BGeQ0OXBIckGOX1HL67ELlq6m8pBRyjbF56umEzz9KbPnXM9qBKjhhuMFsdVmKxC/ZzvCbpVW9kvRLzCeydY/9J+sx11laPXyB63/8C0gQlpj3Fc3O2WAAAAABJRU5ErkJggg==';
var blobService = azureStorage.createBlobService('account-name',
'account-key');
var buffer = Buffer.from(base64String, 'base64');
var stream = streamifier.createReadStream(buffer);
blobService.createBlockBlobFromStream('container-name', 'checked.png', stream, buffer.byteLength, function(error, response) {
if (error) {
console.log('Error!');
console.log(error);
} else {
console.log('Blob uploaded successfully!');
console.log(response);
}
});
I had to install streamifier node package to convert buffer to stream.

Read downloaded blob from dropbox API using HTTP

Using dropbox you can create a shortcut by dragging and dropping a URL into your Dropbox folder. This will be saved like this:
Using the /2/files/download HTTP API from dropbox will return an XHR response that looks something like this:
How do you parse this response so that you can grab only the URL and make that a clickable link?

Here is what needs to go into an Angular 1 factory. To use this, you would just call the downloadFile function from a controller and provide the path to the file in your dropbox account.
function downloadFile(filePath) {
if (!filePath) {
console.error('Cannot download file because no file was specified.');
return;
}
return $q(function(fulfill, reject) {
$http({
url: 'https://content.dropboxapi.com/2/files/download',
method: 'POST',
headers: {
'Authorization': 'Bearer {{access-token-goes-here}}',
'Dropbox-API-Arg': `{"path": "${filePath}"}`
},
responseType: 'blob'
}).then(
results => {
// data received from dropbox is binary data saved as a blob
// The FileReader object lets web applications asynchronously read the contents of files
// https://developer.mozilla.org/en-US/docs/Web/API/FileReader
var fileReader = new FileReader();
// function will run after successfully reading the file
fileReader.onload = function() {
var string = this.result; // store the file contents
string = encodeURI(string); // get rid of the paragraph return characters
var endPosition = string.indexOf('%0D%0A', 32); // find the end of the URL, startPosition is 32
var actualURL = string.substring(32, endPosition); // grab only the characters between start and end positions
fulfill(actualURL);
};
fileReader.readAsText(results.data);
},
error => reject(error));
});
}