We Keep Coding

How to send both text and binary data in axios post request?

I would need to find a solution to send via a single axios POST request both of the following:
json structure
binary file (excel file)
How can I achieve this?
let files = event.target.files;
const fileReader = new FileReader();
fileReader.readAsText(files[0], null);
fileReader.onload = () => {
this.fileContent = fileReader.result;
let binaryDataForObject = this.fileContent;
let referenceDataStructure = {
textData: textDataForObject,
binaryData: binaryDataForObject,
referenceDataFileExtension: this.referenceDataFileExtension,
userProvidedDataTypes: this.columnTypes
};
}
this.axios
.post(
"http://url,
referenceDataStructure
)
This works technically but on the java side I couldn't figure out, how to decode the binary data (encoded as a string) so that it is treated as an excel file.
Thank You in advance for any meaningful responses.
Lubos.

With simple POST request you can send only up to 1mb of binary data
To send binary and text in one request you should use FormData
Check out this answer for information
Update 14.12
How I managed to do this in my recent project was using FormData
So firstly you need to get file as a blob:
const fileReader = new FileReader()
// Here we will get the file as binary data
fileReader.onload = () => {
const MB = 1000000;
const Blob = new Blob([fileReader.result], {
// This will set the mimetype of the file
type: fileInputRef.current.files[0].type
});
const BlobName = fileInputRef.current.files[0].name;
if (Blob.size > MB) return new Error('File size is to big');
// Initializing form data and passing the file as a param
const formData = new FormData();
// file - field name, this will help you to read file on backend
// Blob - main data to send
// BlobName - name of the file, default it will be name of your input
formData.append('file', Blob, BlobName);
// Append json data
formData.apped('some-key', someValue)
// then just send it as a body with post request
fetch('/api/submit-some-form-with-file', {
method: 'POST',
body: formData
})
// Handle the rest
.then()
}
fileReader.readAsArrayBuffer(fileInputRef.current.files[0])
You can wrap this example in handle submit function in react and like or use it as is

Passing FormData/File Object from content script to background script in chrome extension with Manifest V3

I'm building a chrome extension where I get a file as input from the user and pass it to my background.js (service worker in case of manifest v3) to save it to my backend. Since making cross-origin requests are blocked from content scripts I have to pass the same to my background.js and use FETCH API to save the file. When I pass the FormData or File Object to the chrome.runtime.sendMessage API it uses JSON Serialization and what I receive in my background.js is an empty object. Refer to the below snippet.
//content-script.js
attachFile(event) {
let file = event.target.files[0];
// file has `File` object uploaded by the user with required contents.
chrome.runtime.sendMessage({ message: 'saveAttachment', attachment: file });
}
//background.js
chrome.runtime.onMessage.addListener((request, sender) => {
if (request.message === 'saveAttachment') {
let file = request.attachment; //here the value will be a plain object {}
}
});
The same happens even when we pass the FormData from the content script.
I referred to multiple solutions suggested by the old StackOverflow questions, to use URL.createObjectURL(myfile); and pass the URL to my background.js and fetch the same file. Whereas FETCH API does not support blob URL to fetch and also XMLHttpRequest is not supported in service worker as recommended here. Can someone help me in solving this? Am so blocked with this behaviour.

Currently only Firefox can transfer such types directly. Chrome might be able to do it in the future.
Workaround 1.
Serialize the object's contents manually to a string, send it, possibly in several messages if the length exceeds 64MB message size limit, then rebuild the object in the background script. Below is a simplified example without splitting, adapted from Violentmonkey. It's rather slow (encoding and decoding of 50MB takes several seconds) so you may want to write your own version that builds a multipart/form-data string in the content script and send it directly in the background script's fetch.
content script:
async function serialize(src) {
const wasBlob = src instanceof Blob;
const blob = wasBlob ? src : await new Response(src).blob();
const reader = new FileReader();
return new Promise(resolve => {
reader.onload = () => resolve([
reader.result,
blob.type,
wasBlob,
]);
reader.readAsDataURL(blob);
});
}
background script, inside onMessage listener:
const [body, type] = deserialize(message.body);
fetch(message.url, {
body,
headers: {
'Content-Type': type,
},
}).then(/*........*/);
function deserialize([base64, type, wasBlob]) {
const str = atob(base64.slice(base64.indexOf(',') + 1));
const len = str.length;
const arr = new Uint8Array(len);
for (let i = 0; i < len; i += 1) arr[i] = str.charCodeAt(i);
if (!wasBlob) {
type = base64.match(/^data:(.+?);base64/)[1].replace(/(boundary=)[^;]+/,
(_, p1) => p1 + String.fromCharCode(...arr.slice(2, arr.indexOf(13))));
}
return [arr, type];
}
Workaround 2.
Use an iframe for an html file in your extension exposed via web_accessible_resources.
The iframe will be able to do everything an extension can, like making a CORS request.
The File/Blob and other cloneable types can be transferred directly from the content script via postMessage. FormData is not clonable, but you can pass it as [...obj] and then assemble in new FormData() object.
It can also pass the data directly to the background script via navigator.serviceWorker messaging.
Example: see "Web messaging (two-way MessagePort)" in that answer.

I have a better solution: you can actually store Blob in the IndexedDB.
// client side (browser action or any page)
import { openDB } from 'idb';
const db = await openDB('upload', 1, {
upgrade(openedDB) {
openedDB.createObjectStore('files', {
keyPath: 'id',
autoIncrement: true,
});
},
});
await db.clear('files');
const fileID = await db.add('files', {
uploadURL: 'https://yours3bucketendpoint',
blob: file,
});
navigator.serviceWorker.controller.postMessage({
type: 'UPLOAD_MY_FILE_PLEASE',
payload: { fileID }
});
// Background Service worker
addEventListener('message', async (messageEvent) => {
if (messageEvent.data?.type === 'UPLOAD_MY_FILE_PLEASE') {
const db = await openDB('upload', 1);
const file = await db.get('files', messageEvent.data?.payload?.fileID);
const blob = file.blob;
const uploadURL = file.uploadURL;
// it's important here to use self.fetch
// so the service worker stays alive as long as the request is not finished
const response = await self.fetch(uploadURL, {
method: 'put',
body: blob,
});
if (response.ok) {
// Bravo!
}
}
});

I found another way to pass files from a content page (or from a popup page) to a service worker.
But, probably, it is not suitable for all situations,
You can intercept a fetch request sent from a content or popup page in a service worker. Then you can send this request through the service-worker, it can also be modified somehow
popup.js:
// simple fetch, but with a header indicating that the request should be intercepted
fetch(url, {
headers: {
'Some-Marker': 'true',
},
});
background.js:
self.addEventListener('fetch', (event) => {
// You can check that the request should be intercepted in other ways, for example, by the request URL
if (event.request.headers.get('Some-Marker')) {
event.respondWith((async () => {
// event.request contains data from the original fetch that was sent from the content or popup page.
// Here we make a request already in the background.js (service-worker page) and then we send the response to the content page, if it is still active
// Also here you can modify the request hoy you want
const result = await self.fetch(event.request);
return result;
})());
}
return null;
});

How to send (POST) a .json file selected via an input file control on client side HTML form

Hi I currently have a form to select a file for upload to an endpoint that requires the payload to be encoded as JSON.
Currently, I have the following form but it is multipart/form-data. I do not think it is possible to encode a form submission as application/json without resorting to something like ajax....
<form action = "/upload" method = "POST"
enctype = "multipart/form-data">
<input type = "file" name = "file" />
<input type = "submit"/>
</form>
Is it possible using ajax to read the file selected in the form and send the contents as a json payload to the POST url endpoint? Can anyone help?

It's possible by using the FileReader API to parse the file to a string and send the string to the server as encoded JSON.
<form id="upload-form" action="/upload" method="POST" enctype="multipart/form-data">
<input type="file" name="file" />
<input type="submit"/>
</form>
First of we'll need a function that takes in a File and parses it to a string with the FileReader. The readFileAsText function returns a Promise which will resolve once the file has been parsed as text.
The result should be the content of the file as a string. That string is the JSON that we'll send to the server.
const readFileAsText = file => new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = ({ target }) => {
resolve(target.result);
};
reader.readAsText(file);
});
Edit: nowadays you can perform this same process of reading the file as string with the Blob.text() method. A File instance has this method as well and is used like this:
/**
* In this context the file variable is an instance of File.
*/
// In an async context
const text = await file.text();
// Outside an async context.
file.text().then(text => {
// handle the text.
});
Next will be the AJAX part of sending your payload to the server. We do this by using the Fetch API. Here we specify the content type in the headers and create a POST request with a payload that contains JSON.
After the request is made it checks the response from the server and expects a JSON response from the server. Be sure to return a JSON response from the server side.
const sendJSON = async json => {
const headers = {
'Content-Type': 'application/json'
};
const response = await fetch('/upload', {
headers,
method: 'POST',
body: json
});
if (!response.ok) {
throw new Error(`Sending file caused an error - ${response.statusText}`);
}
return response.json();
};
Now we'll try to piece it together. In the following snippet we listen for the submit event on the form. When the form submits, stop the default behavior and overwrite what we need to do on submit.
Get the values from the form with the FormData API. This enables us to extract the values from the form without having to select a specific element.
Select the file from the formData object and pass it to the readFileAsText function (or use the text() method on the file). The result should be the JSON in the file.
Now pass the json from the file into the sendJSON function and off it goes.
const form = document.querySelector('#upload-form');
form.addEventListener('submit', async event => {
event.preventDefault();
const formData = new FormData(event.target);
const file = formData.get('file');
// No file size means no file.
if (file.size === 0) {
return; // Do nothing.
}
try {
const json = await readFileAsText(file);
const response = await sendJSON(json);
console.log(response);
} catch(error) {
console.log(error);
}
});

Read downloaded blob from dropbox API using HTTP

Using dropbox you can create a shortcut by dragging and dropping a URL into your Dropbox folder. This will be saved like this:
Using the /2/files/download HTTP API from dropbox will return an XHR response that looks something like this:
How do you parse this response so that you can grab only the URL and make that a clickable link?

Here is what needs to go into an Angular 1 factory. To use this, you would just call the downloadFile function from a controller and provide the path to the file in your dropbox account.
function downloadFile(filePath) {
if (!filePath) {
console.error('Cannot download file because no file was specified.');
return;
}
return $q(function(fulfill, reject) {
$http({
url: 'https://content.dropboxapi.com/2/files/download',
method: 'POST',
headers: {
'Authorization': 'Bearer {{access-token-goes-here}}',
'Dropbox-API-Arg': `{"path": "${filePath}"}`
},
responseType: 'blob'
}).then(
results => {
// data received from dropbox is binary data saved as a blob
// The FileReader object lets web applications asynchronously read the contents of files
// https://developer.mozilla.org/en-US/docs/Web/API/FileReader
var fileReader = new FileReader();
// function will run after successfully reading the file
fileReader.onload = function() {
var string = this.result; // store the file contents
string = encodeURI(string); // get rid of the paragraph return characters
var endPosition = string.indexOf('%0D%0A', 32); // find the end of the URL, startPosition is 32
var actualURL = string.substring(32, endPosition); // grab only the characters between start and end positions
fulfill(actualURL);
};
fileReader.readAsText(results.data);
},
error => reject(error));
});
}

How to get a file or blob from an object URL?

I am allowing the user to load images into a page via drag&drop and other methods. When an image is dropped, I'm using URL.createObjectURL to convert to an object URL to display the image. I am not revoking the url, as I do reuse it.
So, when it comes time to create a FormData object so I can allow them to upload a form with one of those images in it, is there some way I can then reverse that Object URL back into a Blob or File so I can then append it to a FormData object?

Modern solution:
let blob = await fetch(url).then(r => r.blob());
The url can be an object url or a normal url.

As gengkev alludes to in his comment above, it looks like the best/only way to do this is with an async xhr2 call:
var xhr = new XMLHttpRequest();
xhr.open('GET', 'blob:http%3A//your.blob.url.here', true);
xhr.responseType = 'blob';
xhr.onload = function(e) {
if (this.status == 200) {
var myBlob = this.response;
// myBlob is now the blob that the object URL pointed to.
}
};
xhr.send();
Update (2018): For situations where ES5 can safely be used, Joe has a simpler ES5-based answer below.

Maybe someone finds this useful when working with React/Node/Axios. I used this for my Cloudinary image upload feature with react-dropzone on the UI.
axios({
method: 'get',
url: file[0].preview, // blob url eg. blob:http://127.0.0.1:8000/e89c5d87-a634-4540-974c-30dc476825cc
responseType: 'blob'
}).then(function(response){
var reader = new FileReader();
reader.readAsDataURL(response.data);
reader.onloadend = function() {
var base64data = reader.result;
self.props.onMainImageDrop(base64data)
}
})

The problem with fetching the blob URL again is that this will create a full copy of the Blob's data, and so instead of having it only once in memory, you'll have it twice. With big Blobs this can blow your memory usage quite quickly.
It's rather unfortunate that the File API doesn't give us access to the currently linked Blobs, certainly they thought web-authors should store that Blob themselves at creation time anyway, which is true:
The best here is to store the object you used when creating the blob:// URL.
If you are afraid this would prevent the Blob from being Garbage Collected, you're right, but so does the blob:// URL in the first place, until you revoke it. So holding yourself a pointer to that Blob won't change a thing.
But for those who aren't responsible for the creation of the blob:// URI (e.g because a library made it), we can still fill that API hole ourselves by overriding the default URL.createObjectURL and URL.revokeObjectURL methods so that they do store references to the object passed.
Be sure to call this function before the code that does generate the blob:// URI is called.
// Adds an URL.getFromObjectURL( <blob:// URI> ) method
// returns the original object (<Blob> or <MediaSource>) the URI points to or null
(() => {
// overrides URL methods to be able to retrieve the original blobs later on
const old_create = URL.createObjectURL;
const old_revoke = URL.revokeObjectURL;
Object.defineProperty(URL, 'createObjectURL', {
get: () => storeAndCreate
});
Object.defineProperty(URL, 'revokeObjectURL', {
get: () => forgetAndRevoke
});
Object.defineProperty(URL, 'getFromObjectURL', {
get: () => getBlob
});
const dict = {};
function storeAndCreate(blob) {
const url = old_create(blob); // let it throw if it has to
dict[url] = blob;
return url
}
function forgetAndRevoke(url) {
old_revoke(url);
try {
if(new URL(url).protocol === 'blob:') {
delete dict[url];
}
} catch(e){}
}
function getBlob(url) {
return dict[url] || null;
}
})();
// Usage:
const blob = new Blob( ["foo"] );
const url = URL.createObjectURL( blob );
console.log( url );
const retrieved = URL.getFromObjectURL( url );
console.log( "retrieved Blob is Same Object?", retrieved === blob );
fetch( url ).then( (resp) => resp.blob() )
.then( (fetched) => console.log( "fetched Blob is Same Object?", fetched === blob ) );
And an other advantage is that it can even retrieve MediaSource objects, while the fetching solutions would just err in that case.

Using fetch for example like below:
fetch(<"yoururl">, {
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + <your access token if need>
},
})
.then((response) => response.blob())
.then((blob) => {
// 2. Create blob link to download
const url = window.URL.createObjectURL(new Blob([blob]));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', `sample.xlsx`);
// 3. Append to html page
document.body.appendChild(link);
// 4. Force download
link.click();
// 5. Clean up and remove the link
link.parentNode.removeChild(link);
})
You can paste in on Chrome console to test. the file with download with 'sample.xlsx' Hope it can help!

See Getting BLOB data from XHR request which points out that BlobBuilder doesn't work in Chrome so you need to use:
xhr.responseType = 'arraybuffer';

Unfortunately #BrianFreud's answer doesn't fit my needs, I had a little different need, and I know that is not the answer for #BrianFreud's question, but I am leaving it here because a lot of persons got here with my same need. I needed something like 'How to get a file or blob from an URL?', and the current correct answer does not fit my needs because its not cross-domain.
I have a website that consumes images from an Amazon S3/Azure Storage, and there I store objects named with uniqueidentifiers:
sample: http://****.blob.core.windows.net/systemimages/bf142dc9-0185-4aee-a3f4-1e5e95a09bcf
Some of this images should be download from our system interface.
To avoid passing this traffic through my HTTP server, since this objects does not require any security to be accessed (except by domain filtering), I decided to make a direct request on user's browser and use local processing to give the file a real name and extension.
To accomplish that I have used this great article from Henry Algus:
http://www.henryalgus.com/reading-binary-files-using-jquery-ajax/
1. First step: Add binary support to jquery
/**
*
* jquery.binarytransport.js
*
* #description. jQuery ajax transport for making binary data type requests.
* #version 1.0
* #author Henry Algus <henryalgus#gmail.com>
*
*/
// use this transport for "binary" data type
$.ajaxTransport("+binary", function (options, originalOptions, jqXHR) {
// check for conditions and support for blob / arraybuffer response type
if (window.FormData && ((options.dataType && (options.dataType == 'binary')) || (options.data && ((window.ArrayBuffer && options.data instanceof ArrayBuffer) || (window.Blob && options.data instanceof Blob))))) {
return {
// create new XMLHttpRequest
send: function (headers, callback) {
// setup all variables
var xhr = new XMLHttpRequest(),
url = options.url,
type = options.type,
async = options.async || true,
// blob or arraybuffer. Default is blob
dataType = options.responseType || "blob",
data = options.data || null,
username = options.username || null,
password = options.password || null;
xhr.addEventListener('load', function () {
var data = {};
data[options.dataType] = xhr.response;
// make callback and send data
callback(xhr.status, xhr.statusText, data, xhr.getAllResponseHeaders());
});
xhr.open(type, url, async, username, password);
// setup custom headers
for (var i in headers) {
xhr.setRequestHeader(i, headers[i]);
}
xhr.responseType = dataType;
xhr.send(data);
},
abort: function () {
jqXHR.abort();
}
};
}
});
2. Second step: Make a request using this transport type.
function downloadArt(url)
{
$.ajax(url, {
dataType: "binary",
processData: false
}).done(function (data) {
// just my logic to name/create files
var filename = url.substr(url.lastIndexOf('/') + 1) + '.png';
var blob = new Blob([data], { type: 'image/png' });
saveAs(blob, filename);
});
}
Now you can use the Blob created as you want to, in my case I want to save it to disk.
3. Optional: Save file on user's computer using FileSaver
I have used FileSaver.js to save to disk the downloaded file, if you need to accomplish that, please use this javascript library:
https://github.com/eligrey/FileSaver.js/
I expect this to help others with more specific needs.

If you show the file in a canvas anyway you can also convert the canvas content to a blob object.
canvas.toBlob(function(my_file){
//.toBlob is only implemented in > FF18 but there is a polyfill
//for other browsers https://github.com/blueimp/JavaScript-Canvas-to-Blob
var myBlob = (my_file);
})

Following #Kaiido answer, another way to overload URL without messing with URL is to extend the URL class like this:
export class URLwithStore extends URL {
static createObjectURL(blob) {
const url = super.createObjectURL(blob);
URLwithStore.store = { ...(URLwithStore.store ?? {}), [url]: blob };
return url;
}
static getFromObjectURL(url) {
return (URLwithStore.store ?? {})[url] ?? null;
}
static revokeObjectURL(url) {
super.revokeObjectURL(url);
if (
new URL(url).protocol === "blob:" &&
URLwithStore.store &&
url in URLwithStore.store
)
delete URLwithStore.store[url];
}
}
Usage
const blob = new Blob( ["foo"] );
const url = URLwithStore.createObjectURL( blob );
const retrieved = URLwithStore.getFromObjectURL( url );
console.log( "retrieved Blob is Same Object?", retrieved === blob );