We Keep Coding

How to create file in the local fs using modern FileApi?

I have this code:
document.querySelector('#myfile').onchange = function(e) {
var files = this.files;
window.requestFileSystem(window.TEMPORARY, 1024 * 1024, function(fs) {
let file = files[0];
let nem_file_name = file.name + '_copy';
fs.root.getFile(nem_file_name, {
create: true,
exclusive: true
}, function(fileEntry) {
fileEntry.createWriter(fileWriter => {
fileWriter.write(file);
}, () => alert('error 1'));
}, err => alert('error 2 ' + err));
}, () => alert('error 3'));
};
<input type="file" id="myfile" ref="myfile" multiple />
I want to create a copy of my file when I select it with the input control. What's wrong with my code? I got no errors and nothing happens

The "modern" API is called File System Access, it is still only a proposal, but is expected to supersede the previous and deprecated File System API, and is already available in Chromium browsers.
To write a file using this API, you first request a FileHandle using the showSaveFilePicker() method, then you can create a writer from that handle and append data using that writer:
onclick = async (e) => { // needs to be initiated by an user gesture
const handle = await showSaveFilePicker(); // prompt "Save As"
const writer = await handle.createWritable(); // request writable stream
await writer.write( new Blob( [ "some data" ] ) ); // write the Blob directly
writer.close(); // end writing
};
But this API is still overprotected, so it unfortunately can't be ran in cross-origin iframes like the ones used here by StackSnippet or most popular fiddling services. So in order to make a live demo, I had to make a plunker, that you must run in windowed mode.
And if you need to set the name of the file yourself, you need to request for a directory access using showDirectoryPicker() and then to get a FileHandle from that directory handle using its getFileHandle() method. plnkr

The documentation states that this method is non-standard, only in Chrome and already deprecated:
Even compared to the rest of the File and Directory Entries API,
requestFileSystem() is especially non-standard; only Chrome implements
it, and all other browser makers have decided that they will not
implement it. It has even been removed from the proposed
specification. Do not use this method!
You should not use this.

Dealing with IOS web browsers not caching audio

I have a language site that I am working on to teach language. Users can click on objects and hear the audio for what they click on. Many of the people that will be using this are in more remote areas with slower Internet connections. Because of this, I am needing to cache audio before each of the activities is loaded otherwise there is too much of a delay.
Previously, I was having an issue where preloading would not work because iOS devices do not allow audio to load without a click event. I have gotten around this, however, I now have another issue. iOS/Safari only allows the most recent audio file to be loaded. Therefore, whenever the user clicks on another audio file (even if it was clicked on previously), it is not cached and the browser has to download it again.
So far I have not found an adequate solution to this. There are many posts from around 2011~2012 that try to deal with this but I have not found a good solution. One solution was to combine all audio clips for activity into a single audio file. That way only one audio file would be loaded into memory for each activity and then you just pick a particular part of the audio file to play. While this may work, it also becomes a nuisance whenever an audio clip needs to be changed, added, or removed.
I need something that works well in a ReactJS/Redux environment and caches properly on iOS devices.
Is there a 2020 solution that works well?

You can use IndexedDB. It's a low-level API for client-side storage of significant amounts of structured data, including files/blobs. IndexedDB API is powerful, but may seem too complicated for simple cases. If you'd prefer a simple API, try libraries such as localForage, dexie.js.
localForage is A Polyfill providing a simple name:value syntax for client-side data storage, which uses IndexedDB in the background, but falls back to WebSQL and then localStorage in browsers that don't support IndexedDB.
You can check the browser support for IndexedDB here: https://caniuse.com/#search=IndexedDB. It's well supported. Here is a simple example I made to show the concept:
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Audio</title>
</head>
<body>
<h1>Audio</h1>
<div id="container"></div>
<script src="localForage.js"></script>
<script src="main.js"></script>
</body>
</html>
main.js
"use strict";
(function() {
localforage.setItem("test", "working");
// create HTML5 audio player
function createAudioPlayer(audio) {
const audioEl = document.createElement("audio");
const audioSrc = document.createElement("source");
const container = document.getElementById("container");
audioEl.controls = true;
audioSrc.type = audio.type;
audioSrc.src = URL.createObjectURL(audio);
container.append(audioEl);
audioEl.append(audioSrc);
}
window.addEventListener("load", e => {
console.log("page loaded");
// get the audio from indexedDB
localforage.getItem("audio").then(audio => {
// it may be null if it doesn't exist
if (audio) {
console.log("audio exist");
createAudioPlayer(audio);
} else {
console.log("audio doesn't exist");
// fetch local audio file from my disk
fetch("panumoon_-_sidebyside_2.mp3")
// convert it to blob
.then(res => res.blob())
.then(audio => {
// save the blob to indexedDB
localforage
.setItem("audio", audio)
// create HTML5 audio player
.then(audio => createAudioPlayer(audio));
});
}
});
});
})();
localForage.js just includes the code from here: https://github.com/localForage/localForage/blob/master/dist/localforage.js
You can check IndexedDB in chrome dev tools and you will find our items there:
and if you refresh the page you will still see it there and you will see the audio player created as well. I hope this answered your question.
BTW, older versions of safari IOS didn't support storing blob in IndexedDB if it's still the case you can store the audio files as ArrayBuffer which is very well supported. Here is an example using ArrayBuffer:
main.js
"use strict";
(function() {
localforage.setItem("test", "working");
// convert arrayBuffer to Blob
function arrayBufferToBlob(buffer, type) {
return new Blob([buffer], { type: type });
}
// convert Blob to arrayBuffer
function blobToArrayBuffer(blob) {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.addEventListener("loadend", e => {
resolve(reader.result);
});
reader.addEventListener("error", reject);
reader.readAsArrayBuffer(blob);
});
}
// create HTML5 audio player
function createAudioPlayer(audio) {
// if it's a buffer
if (audio.buffer) {
// convert it to blob
audio = arrayBufferToBlob(audio.buffer, audio.type);
}
const audioEl = document.createElement("audio");
const audioSrc = document.createElement("source");
const container = document.getElementById("container");
audioEl.controls = true;
audioSrc.type = audio.type;
audioSrc.src = URL.createObjectURL(audio);
container.append(audioEl);
audioEl.append(audioSrc);
}
window.addEventListener("load", e => {
console.log("page loaded");
// get the audio from indexedDB
localforage.getItem("audio").then(audio => {
// it may be null if it doesn't exist
if (audio) {
console.log("audio exist");
createAudioPlayer(audio);
} else {
console.log("audio doesn't exist");
// fetch local audio file from my disk
fetch("panumoon_-_sidebyside_2.mp3")
// convert it to blob
.then(res => res.blob())
.then(blob => {
const type = blob.type;
blobToArrayBuffer(blob).then(buffer => {
// save the buffer and type to indexedDB
// the type is needed to convet the buffer back to blob
localforage
.setItem("audio", { buffer, type })
// create HTML5 audio player
.then(audio => createAudioPlayer(audio));
});
});
}
});
});
})();

Moving my answer here from the comment.
You can use HTML5 localstorage API to store/cache the audio content. See this article from Apple https://developer.apple.com/library/archive/documentation/iPhone/Conceptual/SafariJSDatabaseGuide/Introduction/Introduction.html.
As per the article,
Make your website more responsive by caching resources—including audio
and video media—so they aren't reloaded from the web server each time
a user visits your site.
There is an example to show how to use the storage.
Apple also allows you to use a database if you need so. See this example: https://developer.apple.com/library/archive/documentation/iPhone/Conceptual/SafariJSDatabaseGuide/ASimpleExample/ASimpleExample.html#//apple_ref/doc/uid/TP40007256-CH4-SW4

Lets explore some browser storage options
localStorage is only good for storing short key/val string
IndexedDB is not ergonomic for it design
websql is deprecated/removed
Native file system is a good canditate but still experimental behind a flag in chrome
localForge is a just booiler lib for a key/value storage wrapped around IndexedDB and promises (good but unnecessary)
That leaves us with: Cache storage
/**
* Returns the cached url if it exist or fetches it,
* stores it and returns a blob
*
* #param {string|Request} url
* #returns {Promise<Blob>}
*/
async function cacheFirst (url) {
const cache = await caches.open('cache')
const res = await cache.match(file) || await fetch(url).then(res => {
cache.put(url, res.clone())
return res
})
return res.blob()
}
cacheFirst(url).then(blob => {
audioElm.src = URL.createObjectURL(blob)
})
Cache storage goes well hand in hand with service worker but can function without it. doe your site needs to be secure, as it's a "power function" and only exist in secure contexts.
Service worker is a grate addition if you want to build PWA (Progressive web app) with offline support, maybe you should consider it. something that can help you on the way is: workbox it can cache stuff on the fly as you need them - like some man in the middle. it also have a cache first strategy.
Then it can be as simple as just writing <audio src="url"> and let workbox do it thing

Is there a `loadedmetadata` for html5 img?

I'm rendering large images which are streamed natively by the browser.
What I need is a Javascript event that indicates that the image's dimensions were retrieved from its metadata. The only event that seems to be firing is the onload event, but this is not useful as the dimensions were known long before that. I've tried loadstart but it does not fire for img elements.
Is there a loadedmetadata event for the img element in html5?

There is not an equivalent of loadedmetadata for img elements.
The most updated specs at the time of writting are the w3 Recommendation (5.2) (or the w3 WD (5.3)) and the WHATWG Living Standard. Although I find easier to browse through all the events in MDN; their docs are more user friendly.
You can check that loadedmetadata is the only event related to metadata and that it applies just to HTMLMediaElements.
You could take advantage of the Streams API to access the streams of data, process them and extract the metadata yourself. It has two caveats, though: it is an experimental technology with limited support and you will need to look for a way to read the image dimensions from the data stream depending on the image format.
I put together an example for PNG images based on MDN docs.
Following the PNG spec, the dimensions of a PNG image are just after the signature, at the beginning of the IHDR chunk (i.e., width at bytes 16-19, height at 20-23). Although it is not guaranteed, you can bet that the metadata of every image format is available in the first chunk that you receive.
const image = document.getElementById('img');
// Fetch the original image
fetch('https://upload.wikimedia.org/wikipedia/commons/d/de/Wikipedia_Logo_1.0.png')
// Retrieve its body as ReadableStream
.then(response => {
const reader = response.body.getReader();
return stream = new ReadableStream({
start(controller) {
let firstChunkReceived = false;
return pump();
function pump() {
return reader.read().then(({
done,
value
}) => {
// When no more data needs to be consumed, close the stream
if (done) {
controller.close();
return;
}
// Log the chunk of data in console
console.log('data chunk: [' + value + ']');
// Retrieve the metadata from the first chunk
if (!firstChunkReceived) {
firstChunkReceived = true;
let width = (new DataView(value.buffer, 16, 20)).getInt32();
let height = (new DataView(value.buffer, 20, 24)).getInt32();
console.log('width: ' + width + '; height: ' + height);
}
// Enqueue the next data chunk into our target stream
controller.enqueue(value);
return pump();
});
}
}
})
}).then(stream => new Response(stream))
.then(response => response.blob())
.then(blob => URL.createObjectURL(blob))
.then(url => console.log(image.src = url))
.catch(err => console.error(err));
<img id="img" src="" alt="Image preview...">
Disclaimer: when I read this question I knew that the Streams API could be used but I've never been in a need to extract metadata so I've never made ANY research about it. It could be that there are other APIs or libraries that do a better job, more straightforward and with wider browser support.

JavaScript: Writing to download stream

I want to download an encrypted file from my server, decrypt it and save it locally. I want to decrypt the file and write it locally as it is being downloaded rather than waiting for the download to finish, decrypting it and then putting the decrypted file in an anchor tag. The main reason I want to do this is so that with large files the browser does not have to store hundreds of megabytes or several gigabytes in memory.

This is only going to be possible with a combination of service worker + fetch + stream
A few browser has worker and fetch but even fewer support fetch with streaming (Blink)
new Response(new ReadableStream({...}))
I have built a streaming file saver lib to communicate with a service worker in other to intercept network request: StreamSaver.js
It's a little bit different from node's stream here is an example
function unencrypt(){
// should return Uint8Array
return new Uint8Array()
}
// We use fetch instead of xhr that has streaming support
fetch(url).then(res => {
// create a writable stream + intercept a network response
const fileStream = streamSaver.createWriteStream('filename.txt')
const writer = fileStream.getWriter()
// stream the response
const reader = res.body.getReader()
const pump = () => reader.read()
.then(({ value, done }) => {
let chunk = unencrypt(value)
// Write one chunk, then get the next one
writer.write(chunk) // returns a promise
// While the write stream can handle the watermark,
// read more data
return writer.ready.then(pump)
)
// Start the reader
pump().then(() =>
console.log('Closed the stream, Done writing')
)
})
There are also two other way you can get streaming response with xhr, but it's not standard and doesn't mather if you use them (responseType = ms-stream || moz-chunked-arrayBuffer) cuz StreamSaver depends on fetch + ReadableStream any ways and can't be used in any other way
Later you will be able to do something like this when WritableStream + Transform streams gets implemented as well
fetch(url).then(res => {
const fileStream = streamSaver.createWriteStream('filename.txt')
res.body
.pipeThrogh(unencrypt)
.pipeTo(fileStream)
.then(done)
})
It's also worth mentioning that the default download manager is commonly associated with background download so ppl sometimes close the tab when they see the download. But this is all happening in the main thread so you need to warn the user when they leave
window.onbeforeunload = function(e) {
if( download_is_done() ) return
var dialogText = 'Download is not finish, leaving the page will abort the download'
e.returnValue = dialogText
return dialogText
}

New solution has arrived: showSaveFilePicker/FileSystemWritableFileStream, supported in Chrome, Edge, and Opera since October 2020 (and with a ServiceWorker-based shim for Firefox—from the author of the other major answer!), will allow you to do this directly:
async function streamDownloadDecryptToDisk(url, DECRYPT) {
// create readable stream for ciphertext
let rs_src = fetch(url).then(response => response.body);
// create writable stream for file
let ws_dest = window.showSaveFilePicker().then(handle => handle.createWritable());
// create transform stream for decryption
let ts_dec = new TransformStream({
async transform(chunk, controller) {
controller.enqueue(await DECRYPT(chunk));
}
});
// stream cleartext to file
let rs_clear = rs_src.then(s => s.pipeThrough(ts_dec));
return (await rs_clear).pipeTo(await ws_dest);
}
Depending on performance—if you're trying to compete with MEGA, for instance—you might also consider modifying DECRYPT(chunk) to allow you to use ReadableStreamBYOBReader with it:
…zero-copy reading from an underlying byte source. It is used for efficient copying from underlying sources where the data is delivered as an "anonymous" sequence of bytes, such as files.

For security reasons, browsers do not allow piping an incoming readable stream directly to the local file system, so you have two ways to solve it:
window.open(Resource_URL): download the resource in a new window with
Content_Disposition set to "attachment";
<a download href="path/to/resource"></a>: using the "download" attribute of
AnchorElement to download stream into the hard disk;
hope these helps :)

How to get download progress of an IFRAME

I would like to display a progress of downloading some document inside an iframe. I've created this JavaScript progress bar and now I want to use it as a download progress indicator. I know it's possible when used with ajax, but what about loading file/document in a traditional way.
I've tried to search inside MSDN for the solution and I'm now wondering if I could use any of these:
window.onreadystatechange / document.onreadystatechange
document.onprogress event - I'm not sure if it's supported by other browsers than IE and if it's applied to download progress at all.
or should I look somewhere else...?

Well... 11 years later, I see it goes unanswered, but your progress bar still exists!
I was looking for the same thing and ended up using a fetch, implementing this enhancement: https://github.com/AnthumChris/fetch-progress-indicators/blob/master/fetch-enhanced/supported-browser.js
This piece of code checks the amount of data read of fetch response against the content-length header, and turns it into an easy to use hook. One can also implement something similar themselves, if preferred.
Then you can do something like:
let progress = 0;
const fetcher = new ProgressReportFetcher({loaded, total} => {
progress = (loaded / total) * 100;
});
let iframeSrc;
fetcher.fetch(resourceUrl)
.then((response) => response.arrayBuffer())
.then((arrayBuffer) => new Blob([arrayBuffer], { type: 'application/pdf' })) // see note below
.then((blob) => window.URL.createObjectURL(blob))
.then((url) => {
iframeSrc = url;
});
(Or, if you want to port it back to javascript 11 years ago, some jquery variant of this...)
N.b., I turned it into a PDF here. You need to set the mimetype when making the blob, otherwise you see garbled data, and the native response.blob() method unfortunately does not support doing this. Hence the extra jump through the arrayBuffer.