My code uses the npm unzipper package to unzip a file into its separate entries and write each individual entry into memory. My code looks like this:
const zipFileEntries = fs.createReadStream(zipFilePath).pipe(unzipper.Parse({ forceStream: true }));
for await (const entry of productFeedsZip) {
try {
// for reference: entry.path returns the file name
entry.pipe(fs.createWriteStream(`/entry/write/path/${entry.path}`));
// ERROR OCCURS HERE AT AUTODRAIN
entry.autodrain();
} catch (err) {
console.log(err)
continue;
}
The .zip file I am reading from changes daily and I read from it upon changing. The issue is that sometimes the .zip file contains a faulty file (something that is out of my control) that causes the following error:
Error: unexpected end of file
at Zlib.zlibOnError [as onerror] (node:zlib:189:17) {
errno: -5,
code: 'Z_BUF_ERROR'
}
All I am looking to do is make it so that when my code encounters this error, the program execution is not halted and the for loop will continue onwards and write the next file entry into storage.
As you can see, I attempted a solution by inserting a try/catch block inside of the for loop, but this does not prevent program execution from halting when it encounters this error.
Anyone know how I can handle this error and allow my for loop to continue iterating over the .zip file entries?
Related
I'm running a function which I've written in JavaScript inside a nodejs/Electron client.
This function is meant to copy a file from the users flash drive to their c:/Windows/System32 (The file is being copied there so that it can be ran from Command Prompt manually next time the computer is touched without having to switch directories)
The problem is, the files are not being copied, and copyFileSync is not throwing an error.
Here is the code I'm specifically having a problem with:
try {
console.log('copying t.bat');
fs.copyFileSync(remote.app.getAppPath() + '\\app\\files\\scripts\\files\\t.bat', 'C:\\Windows\\System32\\t.bat');
} catch(err) {
console.log('could not copy t.bat', err);
$('#mfail_title').text('Could not copy t.bat file');
$('#mfail_data').text(err);
UIkit.modal("#master_fail").show();
return false;
}
As you can see, I have copyFileSync inside a TRY CATCH block. I know this code is running because in the console I get copying t.bat, but nothing else.
How can I get my files to copy, or at least throw an error when it cannot?
This client is running inside OOBE mode on various Windows 10 machines, therefore always has administrator access.
I've tried updating to the async version of copyFile, but I'm having the same issue. Here is my code
var source = remote.app.getAppPath() + '\\app\\files\\scripts\\files\\t.bat';
var destination = 'C:\\Windows\\System32\\t.bat';
fs.copyFile(source, destination, (err) => {
if (err) {
console.log(err);
} else {
source = remote.app.getAppPath() + '\\app\\files\\scripts\\files\\p.bat';
destination = 'C:\\Windows\\System32\\p.bat';
fs.copyFile(source, destination, (err) => {
if (err) {
console.log(err);
} else {
source = remote.app.getAppPath() + '\\app\\files\\scripts\\files\\p.bat';
destination = 'C:\\Windows\\System32\\p.bat';
child = spawn("powershell.exe",['-ExecutionPolicy', 'ByPass', '-File', remote.app.getAppPath() + '\\app\\files\\scripts\\' + type + '.ps1']);
}
});
}
});
This should copy a file, then when it's complete it should copy another file, once that is complete, it should run a powershell script.
Each copyFile checks for an error before moving on, but it never throws an error, and the file is never copied.
I had a similar issue earlier, In which an Antivirus(Comodo) was not allowing electron app to access the hard drive.
Copy and other file operations were successful in that case as well, because electron in such case access the corresponding sandbox
Please check this is not the case with you.
You can actually access 'fs' in console from electron and check other things in the file system.
Looks to me as if you're using fs on then renderer process (client side) which will not work (assuming that your fs is the node.js fs module and (*)). Your first script seems to use jQuery (hints for renderer) and the second one uses remote in the first line.
fs can only (*) be used on the main process and you'll need to create an IRC channel and do something like:
ircRenderer.sendSync('copy-file-sync', {from: '/from/path', to: '/to/path'})
and, of course, implement the handler for that quickly invented 'copy-file' channel on the main process.
(*) Edit: I haven't played around a lot with nodeIntegration = true, so fs may or may not work on the renderer process with that flag set on the BrowserWindow. But the irc messaging should definitely work and if not, the problem is outside electron, probably related to file permissions.
Well I'm trying to get a nodejs process to launch a python script. - And this python script logs while it is busy - as it logs I wish to display this in the console window used by the nodejs process.
The python script is really trivial
from time import sleep
if __name__ == '__main__':
print('small text testing')
sleep(10)
raise Exception('test')
prints 'small text testing', sleeps for 10 seconds(!) and then raises an exception which is uncaught and thus finishes the script.
In node I tried to get this to work with:
const { exec } = require('child_process');
const exec_str = '. BackgroundServer/BackgroundServer/bin/activate && python BackgroundServer/main.py 1';
const child = exec(exec_str,
{
// detachment and ignored stdin are the key here:
detached: true,
stdio: [ 'ignore', 1, 2 ]
});
child.unref();
child.stdout.on('data', function(data) {
console.log(data.toString());
});
child.stderr.on('data', function(data) {
console.error(data.toString());
});
However this "fails" in the sense that it will only print after the python process has finished running.
Now I know it is possible to run a script through spawn but that would require me to create a temporary script, give that script permissions and then execute that script. Not optimal either.
Not knowing much about javascript or node.js I am pretty sure your problem is due to the fact that Python buffers its output if it is run as subprocess.
To fix this issue, you can do either manually ensure that Python flushes the buffer by adding calls to sys.stdout.flush() as
import sys
from time import sleep
if __name__ == '__main__':
print('small text testing')
sys.stdout.flush()
sleep(10)
raise Exception('test')
or you can force Python to not be buffered also when used as a subprocess by calling the intrepreter with the -u argument, thus modifying exec_str to
const exec_str = '. BackgroundServer/BackgroundServer/bin/activate && \
python -u BackgroundServer/main.py 1';
The first solution will always flush the output, if that is desirable and you use it in another place, without you having to think about the -u option. However, I would still recommend the second approach as it still allows the code to run buffered (which sometimes can be what you want) and also when working with longer scripts you may have to insert quite a number of manual sys.stdout.flush() calls otherwise.
Also, as a sidenote, there is no need for raising an exception in the Python script. It will end anyway, when it reaches its last line.
I am writing a log reader program in Node.js and need to check if a log file is used by another process (the process that is writing the log). The program is running on Windows. I've tried the following, but it seems that the log file is not locked when it is open and being written to, and therefore fs.openSync will always succeed.
function checkOpened(filePath){
var fd=0;
try{
fd=fs.openSync(filePath, 'r+');
}catch(err){//cannot lock the file
console.log(err.code);
if(err.code === "EBUSY"){
return true
}
return false;
}
try{
fs.statSync(fileP)
fs.closeSync(fd);
fs.unlinkSync(filePath);
}catch(err){
//Doesn't really care
}
return false;
}
How can I check if a file is opened by another process in Node.js then?
EDIT: Apparently it is possible to use fs.rename to check if a file is being used (it will fail to do so if the file is used by another process). I will settle for this for now.
I have a problem getting fs.readFile to return a css file. I've already read in this SO post that EISDIR error occurs when you try to open a file, but the path given is a directory. But I've checked several times and the console.log inside the getCss function is logging the right path with the correct filename and .css extension. What's the issue? Thanks in advance!
// Path to MUI css
muiCssFile = path.join(muiDistFolder, 'css', 'mui.css'),
// Read css files
function getCss(callback) {
console.log('MUI CSS file path in getCss() :: ', muiCssFile);
// Code fails here....
fs.readFile(muiCssFile, 'utf8', function (err, css) {
if (err) {
return callback(err);
}
fs.readFile(styleFile, function (error, customCss) {
if (error) return callback(error);
callback(null, css + customCss);
});
});
}
Ok, just for anyone stumbling across this error I thought it would help to provide a short answer to my question. The error is most likely caused by one or more of the paths you are trying to use fs.readFile on ending on a directory path and NOT a file.
In my specific case above, the error was actually occurring on the second fs.readFile call so be sure to check ALL paths first and ensure that they do in fact lead to files to uncover the problem path. Hope that helps someone save some time getting past this error.
I am creating a web app in node.js and golang. I need to connect nodejs with golang code which talks to mongodb and returns data to node program. is there any way to connect so? I tried to use gonode API.This is my code using gonode API.
my node.js file contains below code:
var Go = require('gonode').Go;
var options = {
path : 'gofile.go',
initAtOnce : true,
}
var go = new Go(options,function(err){
if(err) throw err;
go.execute({commandText: 'Hello world from gonode!'}, function(result, response) {
if(result.ok) {
console.log('Go responded: ' + response.responseText);
}
});
go.close();
}); `
And this is the code in my gofile.go file:
package main
import(
gonode "github.com/jgranstrom/gonodepkg"
json "github.com/jgranstrom/go-simplejson"
)
func main(){
gonode.Start(process)
}
func process(cmd *json.Json) (response *json.Json) {
response, m := json.MakeMap()
if(cmd.Get("commandText").MustString() == "Hello") {
m["responseText"] = "Well hello there!"
} else {
m["responseText"] = "What?"
}
return
}
This is the error am getting while running as node node.js in terminal
events.js:72
throw er; // Unhandled 'error' event
^
Error: write EPIPE
at errnoException (net.js:905:11)
at Object.afterWrite (net.js:721:19)
Golang from 1.5, you can build go to shared object binary file (*.so). This allows you to connect your go compiled library to be called by nodejs, python, ruby, java etc.
Here is a guide you could refer to: https://medium.com/learning-the-go-programming-language/calling-go-functions-from-other-languages-4c7d8bcc69bf
thanks for the response. I got a solution for this. I made 2 different servers. One for NodeJS and another for Golang. I am calling golang uri in Node server and getting data from golang server.
Based on a very cursive check of the gonode source code, the module seems to spawn go code as a child process and communicate through stdin/-out. EPIPE error means that the other end closed the stream. Based on this it might be that your go process exits prematurely.
You could try to debug the problem by modifying Command.prototype.execute in gonode/lib/command.js to print out the JSON that's sent to the go process. Then you can debug the go program by running it directly and giving it the same input via stdin.