Reading multiple files with FileReader and getting array of the filedatas - javascript

So I have FileList of files and I need to read them all to an array ie. [fileData1, fileData2], so that I can make a request with all the files to my backend. I'm stuck with all the async opertions and am not sure how to wait for things to finish. I need a way to detect when I can make the request to the backend. I also want to make this in a functional way of programming. Sorry if the question is unclear.
files.map((file) => {
const fr = new FileReader();
fr.readAsDataURL(file)
fr.addEventListener('load', (event) => {
// This is what I need to get to an array
const fileData = event.target.value.substr(fr.result.indexOf(',') + 1);
})
})
//Here I need to make a single post request with all the files in it

Since you added the functional-programming tag, how about a good old fashioned recursive function:
function read_files(cur_file) {
if (cur_file < files.length) {
// read the next file
const fr = new FileReader();
fr.readAsDataURL(files[cur_file]);
fr.addEventListener('load', (event) => {
// extract the file data from event here
read_files(cur_file+1);
}
}
else {
// we've read all the files
// send the request
}
}
read_files(0);

Related

How to pass image data as base64 to a global variable after the input changes as has happend in an input field in an async function case

I am very new to this concept of async ,await ,promise ..paradigms ..Please help me
I have an input field like this
I have a global variable var base64.How to fill base64 global variable with base encoded value. Inside the function in addEventListener reader.result shows base encoded value.How Can i pass it to outside global variable.I tried a lot .. as it seems to be a async function ..the behavior seems to be quiet different and seems very difficult in catching up and also I am not getting the expected result
<input type="file" id="photo" accept="image/*" required #change="UploadPhoto()" />
The change function of UploadPhoto() is ..{This i got from mozilla}
function UploadPhoto() {
const file = document.querySelector('#photo').files[0];
const reader = new FileReader();
reader.addEventListener("load", function () {
// convert image file to base64 string
console.log(reader.result);
}, false);
if (file) {
reader.readAsDataURL(file);
}
}
Also I will be greatful if you can explain me the working of reader.addEventListener and reader.readAsDataURL(file); with the case of asynchronous behavioured ..I googled a lot but couldnt find an article with detailed explaination of these two functions.
Wanting a global (in this case) is a symptom of not having the tools you need to do without it.
If you really needed a global, you could get it this way...
let b64Encoded;
function UploadPhoto() {
const file = document.querySelector('#photo').files[0];
const reader = new FileReader();
reader.addEventListener("load", function () {
// convert image file to base64 string
b64Encoded = btoa(reader.result);
}, false);
But, as you suppose, the modern way to do it is to use promises. This will make the code simpler to understand, and will prove an even better decision when you get to actually posting the encoded file...
First wrap the file reader in a promise. Here's a fine post on the subject, and here's the punchline (slightly modernized)
async function readAsDataURL(file) {
return new Promise((resolve, reject) => {
const fr = new FileReader();
fr.onerror = reject;
fr.onload = () => {
resolve(fr.result);
}
fr.readAsDataURL(file);
});
}
Use it like this:
// we don't need this anymore :-)
// let b64Encoded;
async function UploadPhoto() {
const file = document.querySelector('#photo').files[0];
const result = await readAsDataURL(file);
const b64Encoded = btoa(result); // stack var, as it should be!
// insert real (promise-based) network code here
return myNetworkLibrary.post(b64Encoded);
}
The addEventListener method on file reader is part of the old school async: It says: file reader is going to do i/o on a separate thread. It will you tell you when it's done (that's the "event" part) by calling a function you specify (that's the "listener" part), pass it a function to call (that's the "add" part).

mediainfo.js query specific data

I got an example working great.
Now Im trying to modify that working example and come up with a way to extract specific data.
For example. Frame rate.
Im thinking the syntax should be something like this with result.frameRate
see below where I tried console.log("Frame: "+ result.frameRate) also tried the Buzz suggestion of result.media.track[0].FrameRate neither suggestion works.
<button class="btn btn-default" id="getframe" onclick="onClickMediaButton()">Get Frame Rate</button>
<script type="text/javascript" src="https://unpkg.com/mediainfo.js/dist/mediainfo.min.js"></script>
const onClickMediaButton = (filename) => {
//const file = fileinput.files[0]
const file = "D:\VLCrecords\Poldark-Episode6.mp4"
if (file) {
output222.value = 'Working…'
const getSize = () => file.size
const readChunk = (chunkSize, offset) =>
new Promise((resolve, reject) => {
const reader = new FileReader()
reader.onload = (event) => {
if (event.target.error) {
reject(event.target.error)
}
resolve(new Uint8Array(event.target.result))
}
reader.readAsArrayBuffer(file.slice(offset, offset + chunkSize))
})
mediainfo
.analyzeData(getSize, readChunk)
.then((result) => {
consoleLog("Frame: " + result.media.track[0].FrameRate);
output222.value = result;
})
.catch((error) => {
output222.value = `An error occured:\n${error.stack}`
})
}
}
but I cant figure out the exact syntax. Can you help point me in the right direction?
Short answer
Use result.media.track[0].FrameRate.
Long answer
The type of result depends on how you instantiated the library. Your example does not provide enough information on how you are using the library.
From the docs:
MediaInfo(opts, successCallback, errorCallback)
Where opts.format can be object, JSON, XML, HTML or text. So, assuming you used format: 'object' (the default), result will be a JavaScript object.
The structure of the result object depends on the data you provide to MediaInfoLib (or in this case mediainfo.js which is an Emscripten port of MediaInfoLib). The information about the framerate will only be available if you feed a file with at least one video track to the library.
Assuming this is the case, you can access the list of tracks using result.media.track. Given that the video track you are interested in has the index 0, the access to the desired property would be result.media.track[0].FrameRate. This is true for a large amount of video files that usually have at least one video track and have this track as the first available track. Note that this won't necessarily work for all video files and you must make sure your code is fault-tolerant in case these properties don't exist on the result object.
Unfortunately, it seems there is no detailed list of available fields in the MediaInfoLib documentation. You could look at the source code. This might get tedious and lengthy and requires you to understand a fair amount of C++. The most convenient way for me though is to just feed MediaInfoLib a file and look at the result.
PS: The question was already answered in this GitHub issue.
Disclaimer: I'm the author of the aforementioned Emscripten port mediainfo.js.
EDIT:
I don't think you are correct when saying my answer is "partial-incomplete" and "does not work."
As a proof here is a working snippet. Obviously you need to use a media file with a video track.
const fileinput = document.getElementById('fileinput');
const onChangeFile = (mediainfo) => {
const file = fileinput.files[0];
if (file) {
const getSize = () => file.size;
const readChunk = (chunkSize, offset) =>
new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = (event) => {
resolve(new Uint8Array(event.target.result));
}
reader.readAsArrayBuffer(file.slice(offset, offset + chunkSize));
});
console.log(`Processing ${file.name}`);
mediainfo
.analyzeData(getSize, readChunk)
.then((result) => {
const frameRate = result.media.track[0].FrameRate; // <- Here we read the framerate
console.log(`Framerate: ${frameRate}`);
})
}
}
MediaInfo(null, (mediainfo) => {
fileinput.addEventListener('change', () => onChangeFile(mediainfo));
fileinput.disabled = false;
})
<script src="https://unpkg.com/mediainfo.js#0.1.4/dist/mediainfo.min.js"></script>
<input disabled id="fileinput" type="file" />

file input files not read onChange on mobile

I'm building a puzzle app in React that allows the user to upload their own puzzles. This works fine on the web (the user clicks the input's label and it opens a dialog. When the user picks a file the onChange event is triggered), but on mobile, or at least on Chrome on Android, the files are not read...
This is where the input is declared:
<div className="file-input-wrapper">
<label for="puzzleUpload" className="button-dark">Upload Puzzle(s)</label>
<input type="file"
accept="application/json"
multiple
id="puzzleUpload"
onChange={handleFiles}/>
</div>
and this is the handleFiles() method
// when a file is uploaded, this checks to see that it's the right type, then adds it to the puzzle list
const handleFiles = () => {
var selectedFiles = document.getElementById('puzzleUpload').files;
// checks if the JSON is a valid puzzle
const validPuzzle = (puzzle) => {
let keys = ["name", "entitySetID", "logic", "size"];
return keys.every((key) => {return puzzle.hasOwnProperty(key)});
};
const onLoad = (event) => {
let puzzle = JSON.parse(event.target.result);
if(validPuzzle(puzzle)) {
appendPuzzleList(puzzle);
}
else {
console.log("JSON file does not contain a properly formatted Logike puzzle")
}
};
//checks the file type before attempting to read it
for (let i = 0; i < selectedFiles.length; i++) {
if(selectedFiles[i].type === 'application/json') {
//creates new readers so that it can read many files sequentially.
var reader = new FileReader();
reader.onload = onLoad;
reader.readAsText(selectedFiles[i]);
}
}
};
A working prototype with the most recent code can be found at http://logike.confusedretriever.com and it's possible to quickly write compatible JSON using the builder in the app.
I've been looking up solutions for the past hour and a half and have come up empty handed, so any help would be greatly appreciated! I read the FileReader docs, and everything seems to be supported, so I'm kind of stumped.
Interestingly, the file IS selected (you can see the filename in the ugly default version of the input once it's selected, but I hide it via CSS), so I'm tempted to implement a mobile-only button to trigger the event, if there isn't a more legit solution...
Chrome uses the OS's list of known MIME Types.
I guess Android doesn't know about "application/json", and at least, doesn't map the .json extension to this MIME type, this means that when you upload your File in this browser, you won't have the correct type property set, instead, it is set to the empty string ("").
But anyway, you shouldn't trust this type property, ever.
So you could always avoid some generic types, like image/*, video/*, but the only reliable way to know if it was a valid JSON file or not will be by actually reading the data contained in your file.
But I understand you don't want to start this operation if your user provides a huge file, like a video.
One simple solution might be to check the size property instead, if you know in which range your generated files might come.
One less simple but not so hard either solution would be to prepend a magic number (a.k.a File Signature)to your generated files (if your app is the only way to handle these files).
Then you would just have to check this magic number only before going to read the whole file:
// some magic-number (here "•MJS")
const MAGIC_NB = new Uint8Array([226, 128, 162, 77, 74, 83]);
// creates a json-like File, with our magic_nb prepended
function generateFile(data) {
const str = JSON.stringify(data);
const blob = new Blob([MAGIC_NB, str], {
type: 'application/myjson' // won't be used anyway
});
return new File([blob], 'my_file.json');
}
// checks whether the provided blob starts with our magic numbers or not
function checkFile(blob) {
return new Promise((res, rej) => {
const reader = new FileReader();
reader.onload = e => {
const arr = new Uint8Array(reader.result);
res(!arr.some((v, i) => MAGIC_NB[i] !== v));
};
reader.onerror = rej;
// read only the length of our magic nb
reader.readAsArrayBuffer(blob.slice(0, MAGIC_NB.length));
});
}
function handleFile(file) {
return checkFile(file).then(isValid => {
if (isValid) {
return readFile(file);
} else {
throw new Error('invalid file');
}
});
}
function readFile(file) {
return new Promise((res, rej) => {
const reader = new FileReader();
reader.onload = e => res(JSON.parse(reader.result));
reader.onerror = rej;
// don't read the magic_nb part again
reader.readAsText(file.slice(MAGIC_NB.length));
});
}
const my_file = generateFile({
key: 'value'
});
handleFile(my_file)
.then(obj => console.log(obj))
.catch(console.error);
And in the same way note that all browsers won't accept all the schemes for the accept attribute, and that you might want to double your MIME notation with a simple extension one (anyway even MIMEs are checked only against this extension).

Read blob contents into an existing SharedArrayBuffer

I'm trying to find the most efficient way to read the contents of a Blob into an existing SharedArrayBuffer originating is a worker waiting for the buffer to be poplated. In my case, I can guarantee that the SharedArrayBuffer is at least long enough to hold the entire contents of the Blob. The best approach I've come up with is:
// Assume 'blob' is the blob we are reading
// and 'buffer' is the SharedArrayBuffer.
const fr = new FileReader();
fr.addEventListener('load', e =>
new Uint8Array(buffer).set(new Uint8Array(e.target.result)));
fr.readAsArrayBuffer(blob);
This seems inefficient, especially if the blob being read is relatively large.
Blob is not a Transferable object. Also, there is no .readAsSharedArrayBuffer method available on FileReader.
However, if you only need to read a Blob from multiple workers simultaneously, I believe you can achieve this with URL.createObjectURL() and fetch, although I have not tested this with multiple workers:
// === main thread ===
let objectUrl = URL.createObjectURL(blob);
worker1.postMessage(objectUrl);
worker2.postMessage(objectUrl);
// === worker 1 & 2 ===
self.onmessage = msg => {
fetch(msg.data)
.then(res => res.blob())
.then(blob => {
doSomethingWithBlob(blob);
});
};
Otherwise, as far as I can tell, there really isn't an efficient way to load data from a file into a SharedArrayBuffer.
I also provide a method here for transferring chunks of a blob from main thread to a single worker. For my use case, the files are too big to read the entire contents into a single array buffer anyway (shared or not), so I use .slice to deal in chunks. Something like this will let you deliver tons of data to a single worker in a stream-like fashion via multiple .postMessage calls using the Transferable ArrayBuffer:
// === main thread ===
let eof = false;
let nextBuffer = null;
let workerReady = true;
let read = 0;
function nextChunk() {
let end = read + chunkSize;
if(end >= file.length) {
end = file.length;
eof = true;
}
let slice = file.slice(read, end);
read = end;
fr.readAsArrayBuffer(slice);
}
fr.onload = event => {
let ab = event.target.result;
if(workerReady) {
worker.postMessage(ab, [ab]);
workerReady = false;
if(!eof) nextChunk();
}
else {
nextBuffer = ab;
}
};
// wait until the worker finished the last chunk
// ... otherwise we'll flood main thread's heap
worker.onmessage = msg => {
if(nextBuffer) {
worker.postMessage(nextBuffer, [nextBuffer]);
nextBuffer = null;
}
else if(!eof && msg.ready) {
nextChunk();
}
};
nextChunk();
// === worker ===
self.onmessage = msg => {
let ab = msg.data;
// ... do stuff with data ...
self.postMessage({ready:true});
};
This will read a chunk of data into an ArrayBuffer in the main thread, transfer that to the worker, and then read the next chunk into memory while waiting for worker to process the previous chunk. This basically ensures that both threads stay busy the whole time.

How to load audio file into AudioContext like stream?

For example i want to load 100MB mp3 file into AudioContext, and i can do that with using XMLHttpRequest.
But with this solution i need to load all file and only then i can play it, because onprogress method don't return data.
xhr.onprogress = function(e) {
console.log(this.response); //return null
};
Also i tried to do that with fetch method, but this way have same problem.
fetch(url).then((data) => {
console.log(data); //return some ReadableStream in body,
//but i can't find way to use that
});
There is any way to load audio file like stream in client JavaScript?
You need to handle the ajax response in a streaming way.
there is no standard way to do this until fetch & ReadableStream have properly been implemented across all the browsers
I'll show you the most correct way according to the new standard how you should deal with streaming a ajax response
// only works in Blink right now
fetch(url).then(res => {
let reader = res.body.getReader()
let pump = () => {
reader.read().then(({value, done}) => {
value // chunk of data (push chunk to audio context)
if(!done) pump()
})
}
pump()
})
Firefox is working on implementing streams but until then you need to use xhr and moz-chunked-arraybuffer
IE/edge has ms-stream that you can use but it's more complicated
How can I send value.buffer to AudioContext?
This only plays the first chunk and it doesn't work correctly.
const context = new AudioContext()
const source = context.createBufferSource()
source.connect(context.destination)
const reader = response.body.getReader()
while (true) {
await reader.read()
const { done, value } = await reader.read()
if (done) {
break
}
const buffer = await context.decodeAudioData(value.buffer)
source.buffer = buffer
source.start(startTime)
}

Categories