We Keep Coding

Check if local file has changed [duplicate]

This question already has answers here:
Check if file has changed using HTML5 File API
(3 answers)
Closed 2 years ago.
I have a web app, where the user can select a local file as input, using the html5 FileReader API. Is there any way I can check if the file has changed, that works in modern browsers?
Historically, this was possible in some browsers by polling the file object and comparing the File.lastModifiedDate (deprecated) or File.lastModified property, as outlined in this QA: Check if file has changed using HTML5 File API. However, the spec says that lastModifiedDate and other file data should be a snapshot of the file as it looked like when the users first selected it, so this should not work (and it seems like recent versions of most browsers indeed follow the spec now, making this hack unavailable).
I was hoping to be able to check for changes by reading the file. This kind of works, but as soon as the file is changed on disk, Chrome and Firefox throw an error saying DOMException: The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired. Is there any way around this?
This is what I tried:
let oldText
setInterval(function () {
const fileObj = document.getElementById('myFileInput').files[0]
const reader = new FileReader()
reader.onload = evt => {
const text = evt.target.result
if (text !== oldText) {
console.log("The file has changed!")
oldText = text
}
}
reader.readAsText(fileObj)
}, 1000)
...or simpler:
const fileObj = document.getElementById('myFileInput').files[0]
const reader = new FileReader()
reader.readAsText(fileObj) // works
// now lets edit the file and try again
reader.readAsText(fileObj) // fails
reader.readAsText() works as expected, until the file is changed, when it throws the error mentioned above. I guess this is a security measure of sorts, though I don't fully understand what it's trying to protect the user from. What can I do instead?

This will be possible again if/when the Native File System API is implemented in browsers. It will be partially enabled in Google Chrome 85, scheduled for release in October 2020.
Unlike the FileReader API it requires a explicit user interaction, so you'd do something like this:
myFileInput.addEventListener('change', async (e) => {
const fh = await window.chooseFileSystemEntries()
// fh is now a FileSystemFileHandle object
// Use fh.getFile() to get a File object
})

Automating Priority-Web-SDK file upload

I would like to create a command line (or other automated) method for uploading files to priority using the Web-SDK. The best solution I have right now seems to be a simple webform activated by a python script.
Are there tools/examples for using Javascript and a file picker without opening the browser? Are there Priority-Web-SDK ports to other environments? C#, Python, etc?
Any other suggestions also welcome.
UPDATE June 14, 2020:
I was able to complete the task for this client using a combination of Javascript, Python and C#. A tangled mess indeed, but files were uploaded. I am now revisiting the task and looking for cleaner solutions.
I found a working and usable Node module to compact the program into an executable to make it a viable option for deployment.
So the question becomes more focused => creating the input for uploadDataUrl() or uploadFile() without a browser form.

You run node locally and use priority SDK.
*As long as you work in an environment that is capable to render JS.
You can send files through the function uploadFile.
The data inside the file object need to be written as 64 base file.

This nodejs script will upload a file to Priority. Make sure that fetch-base64 is npm installed:
"use strict";
const priority = require('priority-web-sdk');
const fetch = require('fetch-base64');
const configuration = {...};
async function uploadFile(formName, zoomVal, filepath, filename) {
try {
await priority.login(configuration);
let form = await priority.formStartEx(formName, null, null, null, 1, {zoomValue: zoomVal});
await form.startSubForm("EXTFILES", null ,null);
let data = await fetch.local(filepath + '/' + filename);
let f = await form.uploadDataUrl(data[0], filename.match(/\..+$/i)[0], () => {});
await form.fieldUpdate("EXTFILENAME", f.file); // Path
await form.fieldUpdate("EXTFILEDES", filename); // Name
await form.saveRow(0);
} catch(err) {
console.log('Something bad happened:');
console.dir(err);
}
}
uploadFile('AINVOICES', 'T9679', 'C:/my/path', 'a.pdf');

NodeJs Microsoft Azure Storage SDK Download File to Stream

I just started working with the Microsoft Azure Storage SDK for NodeJS (https://github.com/Azure/azure-storage-node) and already successfully uploaded my first pdf files to the cloud storage.
However, now I started looking at the documentation, in order to download my files as a node_buffer (so I dont have to use fs.createWriteStream), however the documentation is not giving any examples of how this works. The only thing they are writing is "There are also several ways to download files. For example, getFileToStream downloads the file to a stream:", but then they only show one example, which is using the fs.createWriteStream, which I dont want to use.
I was also not able to find anything on Google that really helped me, so I was wondering if anybody has experience with doing this and could share a code sample with me?

The getFileToStream function need a writable stream as param. If you want all the data wrote to a Buffer instead of a file, you just need to create a custom writable stream.
const { Writable } = require('stream');
let bufferArray = [];
const myWriteStream = new Writable({
write(chunk, encoding, callback) {
bufferArray.push(...chunk)
callback();
}
});
myWriteStream.on('finish', function () {
// all the data is stored inside this dataBuffer
let dataBuffer = Buffer.from(bufferArray);
})
then pass myWriteStream to getFileToStream function
fileService.getFileToStream('taskshare', 'taskdirectory', 'taskfile', myWriteStream, function(error, result, response) {
if (!error) {
// file retrieved
}
});

JavaScript: Writing to download stream

I want to download an encrypted file from my server, decrypt it and save it locally. I want to decrypt the file and write it locally as it is being downloaded rather than waiting for the download to finish, decrypting it and then putting the decrypted file in an anchor tag. The main reason I want to do this is so that with large files the browser does not have to store hundreds of megabytes or several gigabytes in memory.

This is only going to be possible with a combination of service worker + fetch + stream
A few browser has worker and fetch but even fewer support fetch with streaming (Blink)
new Response(new ReadableStream({...}))
I have built a streaming file saver lib to communicate with a service worker in other to intercept network request: StreamSaver.js
It's a little bit different from node's stream here is an example
function unencrypt(){
// should return Uint8Array
return new Uint8Array()
}
// We use fetch instead of xhr that has streaming support
fetch(url).then(res => {
// create a writable stream + intercept a network response
const fileStream = streamSaver.createWriteStream('filename.txt')
const writer = fileStream.getWriter()
// stream the response
const reader = res.body.getReader()
const pump = () => reader.read()
.then(({ value, done }) => {
let chunk = unencrypt(value)
// Write one chunk, then get the next one
writer.write(chunk) // returns a promise
// While the write stream can handle the watermark,
// read more data
return writer.ready.then(pump)
)
// Start the reader
pump().then(() =>
console.log('Closed the stream, Done writing')
)
})
There are also two other way you can get streaming response with xhr, but it's not standard and doesn't mather if you use them (responseType = ms-stream || moz-chunked-arrayBuffer) cuz StreamSaver depends on fetch + ReadableStream any ways and can't be used in any other way
Later you will be able to do something like this when WritableStream + Transform streams gets implemented as well
fetch(url).then(res => {
const fileStream = streamSaver.createWriteStream('filename.txt')
res.body
.pipeThrogh(unencrypt)
.pipeTo(fileStream)
.then(done)
})
It's also worth mentioning that the default download manager is commonly associated with background download so ppl sometimes close the tab when they see the download. But this is all happening in the main thread so you need to warn the user when they leave
window.onbeforeunload = function(e) {
if( download_is_done() ) return
var dialogText = 'Download is not finish, leaving the page will abort the download'
e.returnValue = dialogText
return dialogText
}

New solution has arrived: showSaveFilePicker/FileSystemWritableFileStream, supported in Chrome, Edge, and Opera since October 2020 (and with a ServiceWorker-based shim for Firefox—from the author of the other major answer!), will allow you to do this directly:
async function streamDownloadDecryptToDisk(url, DECRYPT) {
// create readable stream for ciphertext
let rs_src = fetch(url).then(response => response.body);
// create writable stream for file
let ws_dest = window.showSaveFilePicker().then(handle => handle.createWritable());
// create transform stream for decryption
let ts_dec = new TransformStream({
async transform(chunk, controller) {
controller.enqueue(await DECRYPT(chunk));
}
});
// stream cleartext to file
let rs_clear = rs_src.then(s => s.pipeThrough(ts_dec));
return (await rs_clear).pipeTo(await ws_dest);
}
Depending on performance—if you're trying to compete with MEGA, for instance—you might also consider modifying DECRYPT(chunk) to allow you to use ReadableStreamBYOBReader with it:
…zero-copy reading from an underlying byte source. It is used for efficient copying from underlying sources where the data is delivered as an "anonymous" sequence of bytes, such as files.

For security reasons, browsers do not allow piping an incoming readable stream directly to the local file system, so you have two ways to solve it:
window.open(Resource_URL): download the resource in a new window with
Content_Disposition set to "attachment";
<a download href="path/to/resource"></a>: using the "download" attribute of
AnchorElement to download stream into the hard disk;
hope these helps :)

Check if file has changed using HTML5 File API

Okay, so I have a program which outputs some specific data to a tab separated variable file.
I had been using Excel to open and view the file contents, however I found excel's insistence on locking every file it opens to be incredibly annoying as my program would crash if I left the file open in Excel... but I would really like the data to neatly update after each run of the program, so I don't have to close and re-open the file all the time.
So, I decided it would be easiest to use Javascript to parse the file and display it in a html table, and it was. I knocked something together in no time. Now my program doesn't crash if I leave the file on display, however, it still doesn't update... and I have to open the newly generated file each time.
So, I was wondering if there was a mechanism by which my Javascript could be somehow notified of a change to the file by another process? I know this is unlikely, but I would like to avoid simply polling the file for obvious reasons.
I am very familiar with JS, but HTML5 and the new APIs are all new to me.

I don't believe the File API has any event for the file changing, just progress events and the like.
Update August 2020: The alternative below no longer works, and the specification specifically disallows it by saying that the File object's information must reflect the state of the underlying file as of when it was selected. From the spec:
User agents should endeavor to have a File object’s snapshot state set to the state of the underlying storage on disk at the time the reference is taken. If the file is modified on disk following the time a reference has been taken, the File's snapshot state will differ from the state of the underlying storage.
You could use polling. Remember the lastModifiedDate of the File, and then when your polling function fires, get a new File instance for the input and see if the lastModifiedDate has changed.
This works for me on Chrome, for instance: Live Copy | Source
(function() {
var input;
var lastMod;
document.getElementById('btnStart').onclick = function() {
startWatching();
};
function startWatching() {
var file;
if (typeof window.FileReader !== 'function') {
display("The file API isn't supported on this browser yet.");
return;
}
input = document.getElementById('filename');
if (!input) {
display("Um, couldn't find the filename element.");
}
else if (!input.files) {
display("This browser doesn't seem to support the `files` property of file inputs.");
}
else if (!input.files[0]) {
display("Please select a file before clicking 'Show Size'");
}
else {
file = input.files[0];
lastMod = file.lastModifiedDate;
display("Last modified date: " + lastMod);
display("Change the file");
setInterval(tick, 250);
}
}
function tick() {
var file = input.files && input.files[0];
if (file && lastMod && file.lastModifiedDate.getTime() !== lastMod.getTime()) {
lastMod = file.lastModifiedDate;
display("File changed: " + lastMod);
}
}
function display(msg) {
var p = document.createElement('p');
p.innerHTML = msg;
document.body.appendChild(p);
}
})();
<input type='file' id='filename'>
<input type='button' id='btnStart' value='Start'>

There is two solutions to this problem, and <input type="file"> is not one of them. according to the spec, it creates a "snapshot" of the file.
Native File System
This api is experimental and requires flags to be enabled in blink (aka chromium browsers). The idea is that you get a file handle and when you need the file then you call the async "getFile" function to retrieve the actual file.
This feature is a "power feature" and require your site to be secure, and it can't work in sandboxed iframes.
So without testing here is some "code in the dark":
// triggerd on click
async function pickFile () {
const handle = showOpenFilePicker()
let lastModificationTime = 0
async function compare () {
const file = await handle.getFile()
if (file.lastModified > lastModificationTime) {
lastModificationTime = +file.lastModified
console.log(await file.text())
}
}
setInterval(compare, 1000)
}
Get Entry from Drag and drop
Similar to the native file system you can also retrieve a file handle and do the same thing, but this feature works in all browsers today. but this code snippet don't work in stackoverflow since it use some sandboxing do making it incompatible, so here is a fiddle with few comments
function drop(event) {
event.stopPropagation();
event.preventDefault();
// get the file as an fileEntry (aka file handle)
const fileEntry = event.dataTransfer.items[0].webkitGetAsEntry()
let lastModificationTime = 0
async function read (file) {
// use the new async read method on blobs.
console.log(await file.text())
}
function compare (meta) {
if (meta.modificationTime > lastModificationTime) {
lastModificationTime = meta.modificationTime
fileEntry.file(read)
}
}
setInterval(fileEntry.getMetadata.bind(fileEntry, compare), 1000)
}
Edit: there is now also a way to get drag and dropped files as a FileSystemFileHandle that is much nicer to work with
elem.addEventListener('dragover', evt => {
// Prevent navigation.
evt.preventDefault()
})
elem.addEventListener('drop', async evt => {
// Prevent navigation.
evt.preventDefault()
// Process all of the items.
for (const item of evt.dataTransfer.items) {
// kind will be 'file' for file/directory entries.
if (item.kind === 'file') {
const entry = await item.getAsFileSystemHandle();
if (entry.kind === 'file') {
// use same solution as the first Native File System solution
}
}
}
})

While T.J. Crowder's answer is correct, Chrome's implementation appears to break the spec.
Each Blob must have an internal snapshot state, which must be initially set to the state of the underlying storage, if any such underlying storage exists, and must be preserved through structured clone. Further normative definition of snapshot state can be found for files.
When a file is selected the input has a snapshot of the contents at that point. Local changes on disk don't update the snapshot.