We Keep Coding

Check if local file has changed [duplicate]

This question already has answers here:
Check if file has changed using HTML5 File API
(3 answers)
Closed 2 years ago.
I have a web app, where the user can select a local file as input, using the html5 FileReader API. Is there any way I can check if the file has changed, that works in modern browsers?
Historically, this was possible in some browsers by polling the file object and comparing the File.lastModifiedDate (deprecated) or File.lastModified property, as outlined in this QA: Check if file has changed using HTML5 File API. However, the spec says that lastModifiedDate and other file data should be a snapshot of the file as it looked like when the users first selected it, so this should not work (and it seems like recent versions of most browsers indeed follow the spec now, making this hack unavailable).
I was hoping to be able to check for changes by reading the file. This kind of works, but as soon as the file is changed on disk, Chrome and Firefox throw an error saying DOMException: The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired. Is there any way around this?
This is what I tried:
let oldText
setInterval(function () {
const fileObj = document.getElementById('myFileInput').files[0]
const reader = new FileReader()
reader.onload = evt => {
const text = evt.target.result
if (text !== oldText) {
console.log("The file has changed!")
oldText = text
}
}
reader.readAsText(fileObj)
}, 1000)
...or simpler:
const fileObj = document.getElementById('myFileInput').files[0]
const reader = new FileReader()
reader.readAsText(fileObj) // works
// now lets edit the file and try again
reader.readAsText(fileObj) // fails
reader.readAsText() works as expected, until the file is changed, when it throws the error mentioned above. I guess this is a security measure of sorts, though I don't fully understand what it's trying to protect the user from. What can I do instead?

This will be possible again if/when the Native File System API is implemented in browsers. It will be partially enabled in Google Chrome 85, scheduled for release in October 2020.
Unlike the FileReader API it requires a explicit user interaction, so you'd do something like this:
myFileInput.addEventListener('change', async (e) => {
const fh = await window.chooseFileSystemEntries()
// fh is now a FileSystemFileHandle object
// Use fh.getFile() to get a File object
})

Color-Thief Node plugin error: "Image given has not completed loading"

Working in Node/Express, I was trying to get the npm package color-thief to grab the dominant color from an image, and it failed because "image given has not completed loading".
The image was, again, local, so it shouldn't have had this particular problem. And besides that, color-thief returns a promise, and I was using async/await, so it should have waited however long it took for the image to load instead of throwing an error.
Below is my SSCCE code:
const ColorThief = require('color-thief');
let colorThief = new ColorThief();
async function getDominantColor() {
const img = 'public/img/seed/big-waves-2193828__340.webp';
const dominantColor = await colorThief.getColor(img);
console.log(dominantColor);
}
getDominantColor();

The issue turned out to be that the plugin apparently does not support .webp files.
It works fine with .jpg and .png, though the Documentation (which isn't easy to get to) doesn't explicitly state what file types it does/does not support.
I've submitted a feature request on Github to either add support for webp or update the documentation with an explicit list of supported filetypes, but the author states at the very bottom of his blog regarding the project:
"In the short term I'm not planning on doing any more work on the script."
Just figured I would try to save someone else using this in the future some headache and time

As per R Greenstreet answer above, Color-Thief does not support other formats than .jpg or .png.
So to workaround this, you need to convert image on the fly.
The fastest and most convenient way I could find is just to use node sharp module. And the code itself is just a few lines...
const sharp = require('sharp');
let image = await sharp(imageData);
let imageData = await image.metadata();
if (imageData.format === 'webp') {
image = await image.toFormat('png').toBuffer();
} else {
image = await image.toBuffer();
}
I know, this is not the optimal solution, but if you want a stable fix, this should be fine.

HTML video loop re-downloads video file

I have an HTML5 video that is rather large. I'm also using Chrome. The video element has the loop attribute but each time the video "loops", the browser re-downloads the video file. I have set Cache-Control "max-age=15768000, private". However, this does not prevent any extra downloads of the identical file. I am using Amazon S3 to host the file. Also the s3 server responds with the Accepts Ranges header which causes the several hundred partial downloads of the file to be requested with the 206 http response code.
Here is my video tag:
<video autoplay="" loop="" class="asset current">
<source src="https://mybucket.s3.us-east-2.amazonaws.com/myvideo.mp4">
</video>
UPDATE:
It seems that the best solution is to prevent the Accept Ranges header from being sent with the original response and instead use a 200 http response code. How can this be achieved so that the video is fully cached through an .htaccess file?
Thanks in advance.

I don't know for sure what's the real issue you are facing.
It could be that Chrome has a max-size limit to what they'd cache, and if it the case, then not using Range-Requests wouldn't solve anything.
An other possible explanation is that caching media is not really a simple task.
Without seeing your file it's hard to tell for sure in which case you are, but you have to understand that to play a media, the browser doesn't need to fetch the whole file.
For instance, you can very well play a video file in an <audio> element, since the video stream won't be used, a browser could very well omit it completely and download only the audio stream. Not sure if any does, but they could. Most media formats do physically separate audio and video streams in the file and their byte positions are marked in the metadata.
They could certainly cache the Range-Requests they perform, but I think it's still quite rare they do.
But as tempting it might be to disable Range-Requests, you've to know that some browsers (Safari) will not play your media if your server doesn't allow Range-Requests.
So even then, it's probably not what you want.
The first thing you may want to try is to optimize your video for web usage. Instead of mp4, serve a webm file. These will generally take less space for the same quality and maybe you'll avoid the max-size limitation.
If the resulting file is still too big, then a dirty solution would be to use a MediaSource so that the file is kept in memory and you need to fetch it only once.
In the following example, the file will be fetched entirely only once, by chunks of 1MB, streamed by the MediaSource as it's being fetched and then only the data in memory will be used for looping plays:
document.getElementById('streamVid').onclick = e => (async () => {
const url = 'https://upload.wikimedia.org/wikipedia/commons/transcoded/2/22/Volcano_Lava_Sample.webm/Volcano_Lava_Sample.webm.360p.webm';
// you must know the mimeType of your video before hand.
const type = 'video/webm; codecs="vp8, vorbis"';
if( !MediaSource.isTypeSupported( type ) ) {
throw 'Unsupported';
}
const source = new MediaSource();
source.addEventListener('sourceopen', sourceOpen);
document.getElementById('out').src = URL.createObjectURL( source );
// async generator Range-Fetcher
async function* fetchRanges( url, chunk_size = 1024 * 1024 ) {
let chunk = new ArrayBuffer(1);
let cursor = 0;
while( chunk.byteLength ) {
const resp = await fetch( url, {
method: "get",
headers: { "Range": "bytes=" + cursor + "-" + ( cursor += chunk_size ) }
}
)
chunk = resp.ok && await resp.arrayBuffer();
cursor++; // add one byte for next iteration, Ranges are inclusive
yield chunk;
}
}
// set up our MediaSource
async function sourceOpen() {
const buffer = source.addSourceBuffer( type );
buffer.mode = "sequence";
// waiting forward to appendAsync...
const appendBuffer = ( chunk ) => {
return new Promise( resolve => {
buffer.addEventListener( 'update', resolve, { once: true } );
buffer.appendBuffer( chunk );
} );
}
// while our RangeFetcher is running
for await ( const chunk of fetchRanges(url) ) {
if( chunk ) { // append to our MediaSource
await appendBuffer( chunk );
}
else { // when done
source.endOfStream();
}
}
}
})().catch( console.error );
<button id="streamVid">stream video</button>
<video id="out" controls muted autoplay loop></video>

Google chrome has a limit to the size of it's file catch. In this case my previous answer would not work. You should use something like file-compressor this resource may be able to compress the file enough that it makes the file cache eligible. The webbrowser can have a new cache size manually set however this is not doable if the end user has not designated their cache to agree with the space required to hold the long video.

A possibility that people that got here are facing - the dev-tool has a "Disable Cache" Option under Network tab. When enabled (meaning cache is disabled) the browser probably doesn't cache the videos, hence needs to download them again.
disable cache from network tab

Can I set the filename of a PDF object displayed in Chrome?

In my Vue app I receive a PDF as a blob, and want to display it using the browser's PDF viewer.
I convert it to a file, and generate an object url:
const blobFile = new File([blob], `my-file-name.pdf`, { type: 'application/pdf' })
this.invoiceUrl = window.URL.createObjectURL(blobFile)
Then I display it by setting that URL as the data attribute of an object element.
<object
:data="invoiceUrl"
type="application/pdf"
width="100%"
style="height: 100vh;">
</object>
The browser then displays the PDF using the PDF viewer. However, in Chrome, the file name that I provide (here, my-file-name.pdf) is not used: I see a hash in the title bar of the PDF viewer, and when I download the file using either 'right click -> Save as...' or the viewer's controls, it saves the file with the blob's hash (cda675a6-10af-42f3-aa68-8795aa8c377d or similar).
The viewer and file name work as I'd hoped in Firefox; it's only Chrome in which the file name is not used.
Is there any way, using native Javascript (including ES6, but no 3rd party dependencies other than Vue), to set the filename for a blob / object element in Chrome?
[edit] If it helps, the response has the following relevant headers:
Content-Type: application/pdf; charset=utf-8
Transfer-Encoding: chunked
Content-Disposition: attachment; filename*=utf-8''Invoice%2016246.pdf;
Content-Description: File Transfer
Content-Encoding: gzip

Chrome's extension seems to rely on the resource name set in the URI, i.e the file.ext in protocol://domain/path/file.ext.
So if your original URI contains that filename, the easiest might be to simply make your <object>'s data to the URI you fetched the pdf from directly, instead of going the Blob's way.
Now, there are cases it can't be done, and for these, there is a convoluted way, which might not work in future versions of Chrome, and probably not in other browsers, requiring to set up a Service Worker.
As we first said, Chrome parses the URI in search of a filename, so what we have to do, is to have an URI, with this filename, pointing to our blob:// URI.
To do so, we can use the Cache API, store our File as Request in there using our URL, and then retrieve that File from the Cache in the ServiceWorker.
Or in code,
From the main page
// register our ServiceWorker
navigator.serviceWorker.register('/sw.js')
.then(...
...
async function displayRenamedPDF(file, filename) {
// we use an hard-coded fake path
// to not interfere with legit requests
const reg_path = "/name-forcer/";
const url = reg_path + filename;
// store our File in the Cache
const store = await caches.open( "name-forcer" );
await store.put( url, new Response( file ) );
const frame = document.createElement( "iframe" );
frame.width = 400
frame.height = 500;
document.body.append( frame );
// makes the request to the File we just cached
frame.src = url;
// not needed anymore
frame.onload = (evt) => store.delete( url );
}
In the ServiceWorker sw.js
self.addEventListener('fetch', (event) => {
event.respondWith( (async () => {
const store = await caches.open("name-forcer");
const req = event.request;
const cached = await store.match( req );
return cached || fetch( req );
})() );
});
Live example (source)
Edit: This actually doesn't work in Chrome...
While it does set correctly the filename in the dialog, they seem to be unable to retrieve the file when saving it to the disk...
They don't seem to perform a Network request (and thus our SW isn't catching anything), and I don't really know where to look now.
Still this may be a good ground for future work on this.
And an other solution, I didn't took the time to check by myself, would be to run your own pdf viewer.
Mozilla has made its js based plugin pdf.js available, so from there we should be able to set the filename (even though once again I didn't dug there yet).
And as final note, Firefox is able to use the name property of a File Object a blobURI points to.
So even though it's not what OP asked for, in FF all it requires is
const file = new File([blob], filename);
const url = URL.createObjectURL(file);
object.data = url;

In Chrome, the filename is derived from the URL, so as long as you are using a blob URL, the short answer is "No, you cannot set the filename of a PDF object displayed in Chrome." You have no control over the UUID assigned to the blob URL and no way to override that as the name of the page using the object element. It is possible that inside the PDF a title is specified, and that will appear in the PDF viewer as the document name, but you still get the hash name when downloading.
This appears to be a security precaution, but I cannot say for sure.
Of course, if you have control over the URL, you can easily set the PDF filename by changing the URL.

I believe Kaiido's answer expresses, briefly, the best solution here:
"if your original URI contains that filename, the easiest might be to simply make your object's data to the URI you fetched the pdf from directly"
Especially for those coming from this similar question, it would have helped me to have more description of a specific implementation (working for pdfs) that allows the best user experience, especially when serving files that are generated on the fly.
The trick here is using a two-step process that perfectly mimics a normal link or button click. The client must (step 1) request the file be generated and stored server-side long enough for the client to (step 2) request the file itself. This requires you have some mechanism supporting unique identification of the file on disk or in a cache.
Without this process, the user will just see a blank tab while file-generation is in-progress and if it fails, then they'll just get the browser's ERR_TIMED_OUT page. Even if it succeeds, they'll have a hash in the title bar of the PDF viewer tab, and the save dialog will have the same hash as the suggested filename.
Here's the play-by-play to do better:
You can use an anchor tag or a button for the "download" or "view in browser" elements
Step 1 of 2 on the client: that element's click event can make a request for the file to be generated only (not transmitted).
Step 1 of 2 on the server: generate the file and hold on to it. Return only the filename to the client.
Step 2 of 2 on the client:
If viewing the file in the browser, use the filename returned from the generate request to then invoke window.open('view_file/<filename>?fileId=1'). That is the only way to indirectly control the name of the file as shown in the tab title and in any subsequent save dialog.
If downloading, just invoke window.open('download_file?fileId=1').
Step 2 of 2 on the server:
view_file(filename, fileId) handler just needs to serve the file using the fileId and ignore the filename parameter. In .NET, you can use a FileContentResult like File(bytes, contentType);
download_file(fileId) must set the filename via the Content-Disposition header as shown here. In .NET, that's return File(bytes, contentType, desiredFilename);
client-side download example:
download_link_clicked() {
// show spinner
ajaxGet(generate_file_url,
{},
(response) => {
// success!
// the server-side is responsible for setting the name
// of the file when it is being downloaded
window.open('download_file?fileId=1', "_blank");
// hide spinner
},
() => { // failure
// hide spinner
// proglem, notify pattern
},
null
);
client-side view example:
view_link_clicked() {
// show spinner
ajaxGet(generate_file_url,
{},
(response) => {
// success!
let filename = response.filename;
// simplest, reliable method I know of for controlling
// the filename of the PDF when viewed in the browser
window.open('view_file/'+filename+'?fileId=1')
// hide spinner
},
() => { // failure
// hide spinner
// proglem, notify pattern
},
null
);

I'm using the library pdf-lib, you can click here to learn more about the library.
I solved part of this problem by using api Document.setTitle("Some title text you want"),
Browser displayed my title correctly, but when click the download button, file name is still previous UUID. Perhaps there is other api in the library that allows you to modify download file name.

Use FileAPI to download big generated data file

The JavaScript process generates a lot of data (200-300MB). I would like to save this data for further analysis but the best I found so far is saving using this example http://jsfiddle.net/c2U2T/ which is not an option for me, because it looks like it requires all the data being available before starting the downloading. But what I need is something like
var saver = new Saver();
saver.save(); // The Save As ... dialog appears
saver.onaccepted = function () { // user accepted saving
for (var i = 0; i < 1000000; i++) {
saver.write(Math.random());
}
};
Of course, instead of the Math.random() will be some meaningful construction.

#dader - I would build upon dader's example.
Use HTML5 FileSystem API - but instead of writing to the file each and every line (more IO than it is worth), you can batch some of the lines in memory in a javascript object/array/string, and only write it to the file when they reach a certain threshold. You are thus appending to a local file as the process chugs (makes it easy to pause/restart/stop etc)
Of note is the following, which is an example of how you can spawn the dialoge to request the amount of data that you would need (it sounds large). Tested in chrome.:
navigator.persistentStorage.queryUsageAndQuota(
function (usage, quota) {
var availableSpace = quota - usage;
var requestingQuota = args.size + usage;
if (availableSpace >= args.size) {
window.requestFileSystem(PERSISTENT, availableSpace, persistentStorageGranted, persistentStorageDenied);
} else {
navigator.persistentStorage.requestQuota(
requestingQuota, function (grantedQuota) {
window.requestFileSystem(PERSISTENT, grantedQuota - usage, persistentStorageGranted, persistentStorageDenied);
}, errorCb
);
}
}, errorCb);
When you are done you can use Javascript to open a new window with the url of that blob object that you saved which you can retrieve via: fileEntry.toURL()
OR - when it is done crunching you can just display that URL in an html link and then they could right click on it and do whatever Save Link As that they want.
But this is something that is new and cool that you can do entirely in the browser without needing to involve a server in any way at all. Side note, 200-300MB of data generated by a Javascript Process sounds absolutely huge... that would be a concern for whether you are storing the "right" data...

What you actually are trying to do is a kind of streaming. I mean FileAPI is not suited for the task. Instead, I could suggest two options :
The first, using XHR facility, ie ajax, by splitting your data into several chunks which will sequencially be sent to the server, each chunk in its own request along with an id ( for identifying the stream ) and a position index ( for identifying the chunk position ). I won't recommend that, since it adds work to break up and reassemble data, and since there's a better solution.
The second way of achieving this is to use Websocket API. It allows you to send data sequentially to the server as it is generated. Following a usual stream API. I think you definitely need this.
This page may be a good place to start at : http://binaryjs.com/
That's all folks !
EDIT considering your comment :
I'm not sure to perfectly get your point though but, what about HTML5's FileSystem API ?
There are a couple examples here : http://www.html5rocks.com/en/tutorials/file/filesystem/ among which this sample that allows you to append data to an existant file. You can also create a new file, etc. :
function onInitFs(fs) {
fs.root.getFile('log.txt', {create: false}, function(fileEntry) {
// Create a FileWriter object for our FileEntry (log.txt).
fileEntry.createWriter(function(fileWriter) {
fileWriter.seek(fileWriter.length); // Start write position at EOF.
// Create a new Blob and write it to log.txt.
var blob = new Blob(['Hello World'], {type: 'text/plain'});
fileWriter.write(blob);
}, errorHandler);
}, errorHandler);
}
EDIT 2 :
What you're trying to do is not possible using javascript as said on SO here. Tha author nonetheless suggest to use Java Applet to achieve needed behaviour.
To put it in a nutshell, HTML5 Filesystem API only provides a sandboxed filesystem, ie located in some hidden directory of the browser. So if you want to access the true filesystem, using java would be just fine considering your use case. I guess there is an interface between java and javascript here.
But if you want to make your data only available from the browser ( constrained by same origin policy ), use FileSystem API.