I am using lockfile.locksync to lock a file in node.js. But I'd like to know the complete mechanism of this utility. So far every website says that its a "very polite lock file utility" but none explains the internal mechanism of this utility.
Any suggestions?
NODE FS
To understand file options in Node.js, take a look here #fs_file_system_flags and Linux open() syscall which covers options like
O_TRUNC - truncate existing file
O_CREAT - create if not exists
O_WRONLY - access mode, write-only
O_EXCL - ensure that this call creates the file. In pair with O_CREAT triggers EEXIST error if file exists.
But it doesn't offer file lock alternatives similar to "flock"
NPM LOCKFILE INVESTIGATING
I tries to use npm library called "lockfile" as there are too much copy/pasted examples with it.
Verdict. "lockfile" npm library in current implementation "v1.0.4" is incomplete/useless!
From the source:
exports.lockSync = function (path, opts) {
opts = opts || {}
opts.req = opts.req || req++
debug('lockSync', path, opts)
if (opts.wait || opts.retryWait) {
throw new Error('opts.wait not supported sync for obvious reasons')
}
try {
var fd = fs.openSync(path, wx)
locks[path] = fd
try { fs.closeSync(fd) } catch (er) {}
debug('locked sync!', path, fd)
return
So that 5 lines of actual code in "try{}" block just "openSync" in "wx" mode, save "path" to "locks[path]" and "closeSync".
If file exists it fails! due to "wx"!
YOU CAN'T LOCK THE EXISTING FILE with "lockfile" library!
Finally it tries to call "unlockSync" with onExit signal with this code:
exports.unlockSync = function (path) {
debug('unlockSync', path)
// best-effort. unlocking an already-unlocked lock is a noop
try { fs.unlinkSync(path) } catch (er) {}
delete locks[path]
}
And yes, it will DELETE your file after process exited!
This is not the file locking mechanics!
ANY RELIABLE SOLUTION FOR NODEJS FILE LOCK?
I've found fs-ext works just perfect! Open 2 tabs with node process and check what locking file means:
tab1
const fs = require("fs");
const {flockSync} = require('fs-ext');
const fd = fs.openSync('1.txt', 'w');
flockSync(fd, 'ex');
tab2
const fs = require("fs");
const {flockSync} = require('fs-ext');
const fd = fs.openSync('1.txt', 'r');
flockSync(fd, 'sh'); // PENDING!
In tab2 flockSync(fd, 'sh'); will pending until in tab1 flockSync(fd, 'un'); is called!
It really works!
Related
I created a plugin in plugins/index.js
const os = require('os');
module.exports = (on, config) => {
config.env.testing_computer = os.hostname();
return config;
}
Basically I want to store the hostname in the environment variable testing_computer.
However when I try to access it later in a custom command by Cypress.env("testing_computer") it is an empty string. How can I get the actual value?
Fixed. I cleared the rest of the file and left only the new plugin. I think the line at the top <references... may have been causing trouble, though I'm not sure.
I currently have a folder that has two subfolders and about 500 files, totalling ~3GB. In my Electron application I perform a backup of that folder by essentially copy and pasting it to a different drive through NodeJS. However, this process takes about 3 mins when I copy and paste via Windows but around 15 mins when I use NodeJS.
I am using fs-extra to copy the whole folder over:
const path = require('path');
const fs = require('fs-extra');
var source = 'some/source/directory'
var destination = path.join('my/backup/directory','name_of_folder_after_backup')
// copy source folder to destination
await fs.copy(source, destination);
I did consider making this an asynchronous operation with just fs.writeFile() but considering this is running on an HDD at worst, I figured firing 500 write operations and waiting for it to complete might not be the best idea.
My questions are:
Is the the proper way to use NodeJS to do a folder copy operation like this?
Why is the Windows copy operation so much faster than the NodeJS copy operation?
How to reduce the copy time of the folder? Is there some native Electron API I might be missing?
I'm not an expert in this area so I'd appreciate if your answer had some sources I could look at or a bit of a detailed explanation.
Thanks to the comments above I got a solution. I used robocopy on windows to do the copy. Node spawns a child process, launches Robocopy, wraps it in a promise and then awaits it.
const path = require("path");
const util = require("util");
const exec = util.promisify(require("child_process").exec);
const source = 'your/source/dir/here';
const destination = path.join('temp', "PDES-Backup");
// copy source folder to destination
//await fs.copySync(source, destination);
const copyCommand = `Robocopy "${source}" "${destination}" /MIR /LOG:backuplog-${new Date()
.toISOString()
.substr(0, 10)}-${new Date().getTime()}.txt`;
const promise = exec(copyCommand, { stdio: "inherit" });
const child = promise.child;
child.stdout.on("data", (data) => {
console.log("stdout: " + data);
});
child.stderr.on("data", (data) => {
console.log("stderr: " + data);
});
child.on("error", (error) => {
console.error(error);
});
child.on("close", (code) => {
console.warn("closing code: " + code);
});
// i.e. can then await for promisified exec call to complete
const { stdout, stderr } = await promise;
For some reason the first copy gives the wrong closing code but the copy gets completed just fine
I have a local JSON file which I intent to read/write from a NodeJS electron app. I am not sure, but I believe that instead of using readFile() and writeFile(), I should get a FileHandle to avoid multiple open and close actions.
So I've tried to grab a FileHandle from fs.promises.open(), but the problem seems to be that I am unable to get a FileHandle from an existing file without truncate it and clear it to 0.
const { resolve } = require('path');
const fsPromises = require('fs').promises;
function init() {
// Save table name
this.path = resolve(__dirname, '..', 'data', `test.json`);
// Create/Open the json file
fsPromises
.open(this.path, 'wx+')
.then(fileHandle => {
// Grab file handle if the file don't exists
// because of the flag 'wx+'
this.fh = fileHandle;
})
.catch(err => {
if (err.code === 'EEXIST') {
// File exists
}
});
}
Am I doing something wrong? Are there better ways to do it?
Links:
https://nodejs.org/api/fs.html#fs_fspromises_open_path_flags_mode
https://nodejs.org/api/fs.html#fs_file_system_flags
Because JSON is a text format that has to be read or written all at once and can't be easily modified or added onto in place, you're going to have to read the whole file or write the whole file at once.
So, your simplest option will be to just use fs.promises.readFile() and fs.promises.writeFile() and let the library open the file, read/write it and close the file. Opening and closing a file in a modern OS takes advantage of disk caching so if you're reopening a file you just previously opened not long ago, it's not going to be a slow operation. Further, since nodejs performs these operations in secondary threads in libuv, it doesn't block the main thread of nodejs either so its generally not a performance issue for your server.
If you really wanted to open the file once and hold it open, you would open it for reading and writing using the r+ flag as in:
const fileHandle = await fsPromises.open(this.path, 'r+');
Reading the whole file would be simple as the new fileHandle object has a .readFile() method.
const text = await fileHandle.readFile({encoding 'utf8'});
For writing the whole file from an open filehandle, you would have to truncate the file, then write your bytes, then flush the write buffer to ensure the last bit of the data got to the disk and isn't sitting in a buffer.
await fileHandle.truncate(0); // clear previous contents
let {bytesWritten} = await fileHandle.write(mybuffer, 0, someLength, 0); // write new data
assert(bytesWritten === someLength);
await fileHandle.sync(); // flush buffering to disk
I am very new to node.js and I think I understand the basics of how it functions but I feel like I am not seeing something that is vital to how fs.write and buffers function.
I am trying to send a user defined variable over socket.io and write it into an html file. I have a main site that has the button, when clicked it sends the information to the socket in a variable.
The thing I can't figure out is how to insert the variable into the html file.
I can save strings that I type, into a file:
(e.g.) var writeBuffer = new Buffer ('13');
But not variables that I put in:
(e.g.) var writeBuffer = new Buffer ($(newval));
I even tried different encoding methods, I think I am missing something.
Server.js
var newval = "User String";
var fd = fsC.open(fileName, 'rs+', function (error, fd) {
if (error) { throw error }
var writeBuffer = new Buffer($(newval));
var bufferLength = writeBuffer.length;
fsC.write( fd, writeBuffer, 0, bufferLength, 937,
function (error, written) {
if (error) { throw error }
fsC.close(fd, function() {
console.log('File Closed');
});
}
);
});
If you are using a version of jsdom 4.0.0 or later, it will not work with Node.js. As per the jsdom github readme:
Note that as of our 4.0.0 release, jsdom no longer works with
Node.js™, and instead requires io.js. You are still welcome to install
a release in the 3.x series if you use Node.js™.
So I have a personal website, and I have a button that I want to use to open photoshop and run a script for me. How do I do this?
It is not possible. It would pose a huge security risk to allow javascript to open programs on client side.
This has been up for awhile, but if the solution to this in node is to use a child_process.
you can npm install child_process
and the code to run executables would be to do
const exec = require("child-process").execFile;
var process = exec("Photoshop.exe", [*add options here*], {cwd:"C:/*path to photoshop*"});
you can do a lot of cool things afterwards like event handlers
process.on("close", code => {
console.log("process closed with code: "+ code)
})
process.on("exit", code => {
console.log("process exited with code: "+ code)
})
process.stdout.on("data", data => {
console.log(data)
})
You can read the Docs here: https://nodejs.org/api/child_process.html
maybe this
Code will open image in photoshop with the help of javascript. You just need to place your image file into photoshop->sample folder nothing more than that and you done with it.
var fileRef = new File(app.path.toString() + “/Samples/test.jpg”); // ‘samples’ is a folder resides in Program Files\Adobe\Adobe Photoshop CS5\samples
//open (fileRef);
var doc = open(fileRef);
// get document name (and remove file extension)
var name = tempName[0];
// convert to RGB; convert to 8-bpc; merge visible
doc.changeMode(ChangeMode.RGB);
doc.bitsPerChannel = BitsPerChannelType.EIGHT;
doc.artLayers.add();
doc.mergeVisibleLayers();
// rename layer; duplicate to new document
var layer = doc.activeLayer;
layer.name = tempName[0];
layer.duplicate(newDoc, ElementPlacement.PLACEATBEGINNING);
// close imported document
doc.close(SaveOptions.DONOTSAVECHANGES);