We Keep Coding

How to send both text and binary data in axios post request?

I would need to find a solution to send via a single axios POST request both of the following:
json structure
binary file (excel file)
How can I achieve this?
let files = event.target.files;
const fileReader = new FileReader();
fileReader.readAsText(files[0], null);
fileReader.onload = () => {
this.fileContent = fileReader.result;
let binaryDataForObject = this.fileContent;
let referenceDataStructure = {
textData: textDataForObject,
binaryData: binaryDataForObject,
referenceDataFileExtension: this.referenceDataFileExtension,
userProvidedDataTypes: this.columnTypes
};
}
this.axios
.post(
"http://url,
referenceDataStructure
)
This works technically but on the java side I couldn't figure out, how to decode the binary data (encoded as a string) so that it is treated as an excel file.
Thank You in advance for any meaningful responses.
Lubos.

With simple POST request you can send only up to 1mb of binary data
To send binary and text in one request you should use FormData
Check out this answer for information
Update 14.12
How I managed to do this in my recent project was using FormData
So firstly you need to get file as a blob:
const fileReader = new FileReader()
// Here we will get the file as binary data
fileReader.onload = () => {
const MB = 1000000;
const Blob = new Blob([fileReader.result], {
// This will set the mimetype of the file
type: fileInputRef.current.files[0].type
});
const BlobName = fileInputRef.current.files[0].name;
if (Blob.size > MB) return new Error('File size is to big');
// Initializing form data and passing the file as a param
const formData = new FormData();
// file - field name, this will help you to read file on backend
// Blob - main data to send
// BlobName - name of the file, default it will be name of your input
formData.append('file', Blob, BlobName);
// Append json data
formData.apped('some-key', someValue)
// then just send it as a body with post request
fetch('/api/submit-some-form-with-file', {
method: 'POST',
body: formData
})
// Handle the rest
.then()
}
fileReader.readAsArrayBuffer(fileInputRef.current.files[0])
You can wrap this example in handle submit function in react and like or use it as is

Passing FormData/File Object from content script to background script in chrome extension with Manifest V3

I'm building a chrome extension where I get a file as input from the user and pass it to my background.js (service worker in case of manifest v3) to save it to my backend. Since making cross-origin requests are blocked from content scripts I have to pass the same to my background.js and use FETCH API to save the file. When I pass the FormData or File Object to the chrome.runtime.sendMessage API it uses JSON Serialization and what I receive in my background.js is an empty object. Refer to the below snippet.
//content-script.js
attachFile(event) {
let file = event.target.files[0];
// file has `File` object uploaded by the user with required contents.
chrome.runtime.sendMessage({ message: 'saveAttachment', attachment: file });
}
//background.js
chrome.runtime.onMessage.addListener((request, sender) => {
if (request.message === 'saveAttachment') {
let file = request.attachment; //here the value will be a plain object {}
}
});
The same happens even when we pass the FormData from the content script.
I referred to multiple solutions suggested by the old StackOverflow questions, to use URL.createObjectURL(myfile); and pass the URL to my background.js and fetch the same file. Whereas FETCH API does not support blob URL to fetch and also XMLHttpRequest is not supported in service worker as recommended here. Can someone help me in solving this? Am so blocked with this behaviour.

Currently only Firefox can transfer such types directly. Chrome might be able to do it in the future.
Workaround 1.
Serialize the object's contents manually to a string, send it, possibly in several messages if the length exceeds 64MB message size limit, then rebuild the object in the background script. Below is a simplified example without splitting, adapted from Violentmonkey. It's rather slow (encoding and decoding of 50MB takes several seconds) so you may want to write your own version that builds a multipart/form-data string in the content script and send it directly in the background script's fetch.
content script:
async function serialize(src) {
const wasBlob = src instanceof Blob;
const blob = wasBlob ? src : await new Response(src).blob();
const reader = new FileReader();
return new Promise(resolve => {
reader.onload = () => resolve([
reader.result,
blob.type,
wasBlob,
]);
reader.readAsDataURL(blob);
});
}
background script, inside onMessage listener:
const [body, type] = deserialize(message.body);
fetch(message.url, {
body,
headers: {
'Content-Type': type,
},
}).then(/*........*/);
function deserialize([base64, type, wasBlob]) {
const str = atob(base64.slice(base64.indexOf(',') + 1));
const len = str.length;
const arr = new Uint8Array(len);
for (let i = 0; i < len; i += 1) arr[i] = str.charCodeAt(i);
if (!wasBlob) {
type = base64.match(/^data:(.+?);base64/)[1].replace(/(boundary=)[^;]+/,
(_, p1) => p1 + String.fromCharCode(...arr.slice(2, arr.indexOf(13))));
}
return [arr, type];
}
Workaround 2.
Use an iframe for an html file in your extension exposed via web_accessible_resources.
The iframe will be able to do everything an extension can, like making a CORS request.
The File/Blob and other cloneable types can be transferred directly from the content script via postMessage. FormData is not clonable, but you can pass it as [...obj] and then assemble in new FormData() object.
It can also pass the data directly to the background script via navigator.serviceWorker messaging.
Example: see "Web messaging (two-way MessagePort)" in that answer.

I have a better solution: you can actually store Blob in the IndexedDB.
// client side (browser action or any page)
import { openDB } from 'idb';
const db = await openDB('upload', 1, {
upgrade(openedDB) {
openedDB.createObjectStore('files', {
keyPath: 'id',
autoIncrement: true,
});
},
});
await db.clear('files');
const fileID = await db.add('files', {
uploadURL: 'https://yours3bucketendpoint',
blob: file,
});
navigator.serviceWorker.controller.postMessage({
type: 'UPLOAD_MY_FILE_PLEASE',
payload: { fileID }
});
// Background Service worker
addEventListener('message', async (messageEvent) => {
if (messageEvent.data?.type === 'UPLOAD_MY_FILE_PLEASE') {
const db = await openDB('upload', 1);
const file = await db.get('files', messageEvent.data?.payload?.fileID);
const blob = file.blob;
const uploadURL = file.uploadURL;
// it's important here to use self.fetch
// so the service worker stays alive as long as the request is not finished
const response = await self.fetch(uploadURL, {
method: 'put',
body: blob,
});
if (response.ok) {
// Bravo!
}
}
});

I found another way to pass files from a content page (or from a popup page) to a service worker.
But, probably, it is not suitable for all situations,
You can intercept a fetch request sent from a content or popup page in a service worker. Then you can send this request through the service-worker, it can also be modified somehow
popup.js:
// simple fetch, but with a header indicating that the request should be intercepted
fetch(url, {
headers: {
'Some-Marker': 'true',
},
});
background.js:
self.addEventListener('fetch', (event) => {
// You can check that the request should be intercepted in other ways, for example, by the request URL
if (event.request.headers.get('Some-Marker')) {
event.respondWith((async () => {
// event.request contains data from the original fetch that was sent from the content or popup page.
// Here we make a request already in the background.js (service-worker page) and then we send the response to the content page, if it is still active
// Also here you can modify the request hoy you want
const result = await self.fetch(event.request);
return result;
})());
}
return null;
});

How to save JS audio Blob as a file so it can be sent to a Flask API

I'm building an app for my friend and I need to record audio and store it on a server. I have successfully done it with text and images, but I can't make it work for audio.
I am probably missing something when creating the file (the file is created but not playable). There has to be a problem with the data conversion from the JS Blob to the actual file.
I managed to get the audio blob and even play it back in JS. But when I create a file from the blob it can't be played (I tried to save it locally with the same outcome - got the file but could not play it). I also tried to save it in different formats with no success (wav, mp3). So the problem has to be in the conversion from the blob to the actual file. With text and images, it was straightforward forward and the files were created from the blob just by saving them with a filename. But I guess that with audio isn't that simple.
My understanding is that I have some binary data (JS Blob), that can be saved as a file. But with audio, there has to be some special conversion or encoding so the output file works and can be played.
here is the frontend code (I am using this with some of the variables because its part of a Vue component)
this.mediaRecorder.addEventListener("stop", () => {
// tried to save it as WAV with the same result got the file, but couldn't play it
this.audioBlob = new Blob(this.audioChunks, { 'type' : 'audio/mpeg-3' })
//debugging - playing back the sound in the browser works fine
const audioUrl = URL.createObjectURL(this.audioBlob);
const audio = new Audio(audioUrl);
audio.play();
//adding the blob to the request
let filename = this.$store.state.counter + "-" + this.$store.state.step
const formData = new FormData();
formData.append('file', this.audioBlob, `${filename}.mp3`);
const config = {
headers: { 'content-type': 'multipart/form-data' }
}
//sending it to my Flask API (xxx is the name of the folder it gets saved to on the server)
this.$axios.post('http://localhost:5000/api/v1/file/upload/xxx', formData, config)
})
here is my endpoint on the server
#app.route('/api/v1/file/upload/<test_id>', methods=['POST'])
def upload_file(test_id):
uploaded_file = request.files['file']
filename = secure_filename(uploaded_file.filename)
if filename != '':
uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], test_id, filename))
return jsonify({'message': 'file saved'})
Here is the whole recording code snippet
navigator.mediaDevices.getUserMedia({ audio: true })
.then(stream => {
this.mediaRecorder = new MediaRecorder(stream);
// audio.srcObject = stream
this.mediaRecorder.start();
this.mediaRecorder.addEventListener("dataavailable", event => {
this.audioChunks.push(event.data)
})
this.mediaRecorder.addEventListener("stop", () => {
this.audioBlob = new Blob(this.audioChunks, { 'type' : 'audio/mpeg-3' })
//debugging - playing back the sound in the browser works fine
const audioUrl = URL.createObjectURL(this.audioBlob);
const audio = new Audio(audioUrl);
audio.play();
//adding the blob to the request
let filename = this.$store.state.counter + "-" + this.$store.state.step
const formData = new FormData();
formData.append('file', this.audioBlob, `${filename}.mp3`);
const config = {
headers: { 'content-type': 'multipart/form-data' }
}
//sending it to my Flask API (xxx is the name of the folder it gets saved to on the server)
this.$axios.post('http://localhost:5000/api/v1/file/upload/xxx', formData, config)
})
})

Converting dataURI to file for upload via REST does not work but normal filefield upload works fine

I have implemented a solution that accepts a single file upload (image for profile) in a Django Rest Framework backend. This route api/people/id/upload_image. Only accepts a parameter with an image. and can be used via HTTP POST.
When uploading via a fileinput field in eg. Postman, the default Django API or via browser fetch() in my Vue.js application is no problem. So it seems as long as it is a default form-upload field it is doing its job.
But in my frond-end (vuejs 3) I am using an image-cropper. Users can upload an image and via the javascript cropper the image can be cropped. This is important for the UI because I need a square image. The cropper uses HTMLCanvasElement.toDataURL() as export format.
And what seemed to be not that difficult gets me stuck for days now. I just can't find a way to convert and POST the cropped image in such a way that is accepted by the upload_image API backend. I am using Fetch() for sending this POST call.
I am not a javascript expert so I get my knowledge via internet and I tried it in several ways; with first creating a BLOB from the dataURI, and by creating a File before feeding it to dataForm and send it as :body in Fetch()
The dataURI seems OK because I am also replacing the cropped image directly in the HTML. And that looks totally fine. The API is responding with an 'HTTP 200 OK'. But the old image is not being replaced.
So my assumption is that there is something wrong with the image send to the API, because via normal fileupload everything works fine. How should I convert this dataURI in a proper way so it can be send and accepted by the API endpoint. And how should the API call look like: headers, body..
this is my last attempt in converting and sending the cropped image: (dataURIimage is OK)
uploadPhoto(context, dataURIimage) {
const blob = dataURItoBlob(dataURIimage);
const resultFile = new File([blob], "picture", {
type: "image/png"
});
const formData = new FormData();
formData.append("image", resultFile);
const headerToken = "Token" + " " + this.getters.getToken;
const url =
"https://workserver-7e6s4.ondigitalocean.app/api/people/" +
this.getters.getProfiel.id +
"/upload_image/";
fetch(url, {
method: "POST",
headers: {
'Content-Type': 'multipart/form-data',
'Authorization': headerToken,
},
body: formData,
})
.then(function(response) {
console.log(response.status)
if (response.ok) {
return response.json();
}
})
.then(function(data) {
console.log(data.image);
})
.catch(function(error) {
alert(error + " " + ":ERROR");
});
},
function dataURItoBlob(dataURI) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
return new Blob([ab], { type: mimeString });
}
Martijn dekker

Read downloaded blob from dropbox API using HTTP

Using dropbox you can create a shortcut by dragging and dropping a URL into your Dropbox folder. This will be saved like this:
Using the /2/files/download HTTP API from dropbox will return an XHR response that looks something like this:
How do you parse this response so that you can grab only the URL and make that a clickable link?

Here is what needs to go into an Angular 1 factory. To use this, you would just call the downloadFile function from a controller and provide the path to the file in your dropbox account.
function downloadFile(filePath) {
if (!filePath) {
console.error('Cannot download file because no file was specified.');
return;
}
return $q(function(fulfill, reject) {
$http({
url: 'https://content.dropboxapi.com/2/files/download',
method: 'POST',
headers: {
'Authorization': 'Bearer {{access-token-goes-here}}',
'Dropbox-API-Arg': `{"path": "${filePath}"}`
},
responseType: 'blob'
}).then(
results => {
// data received from dropbox is binary data saved as a blob
// The FileReader object lets web applications asynchronously read the contents of files
// https://developer.mozilla.org/en-US/docs/Web/API/FileReader
var fileReader = new FileReader();
// function will run after successfully reading the file
fileReader.onload = function() {
var string = this.result; // store the file contents
string = encodeURI(string); // get rid of the paragraph return characters
var endPosition = string.indexOf('%0D%0A', 32); // find the end of the URL, startPosition is 32
var actualURL = string.substring(32, endPosition); // grab only the characters between start and end positions
fulfill(actualURL);
};
fileReader.readAsText(results.data);
},
error => reject(error));
});
}