I need to read a text file line by line in JavaScript.
I might want to do something with each line (e.g. skip or modify it) and write the line to another file. But the specific actions are out of the scope of this question.
There are many questions with similar wording, but most actually read the whole file into memory in one step instead of reading line by line. So those solutions are unusable for bigger files.
With Node.js a new function was added in v18.11.0 to read files line by line
filehandle.readLines([options])
This is how you use this with a text file you want to read
import { open } from 'node:fs/promises';
myFileReader();
async function myFileReader() {
const file = await open('./TextFileName.txt');
for await (const line of file.readLines()) {
console.log(line)
}
}
To understand more read Node.js documentation here is the link for file system readlines():
https://nodejs.org/api/fs.html#filehandlereadlinesoptions
The code to read a text file line by line is indeed surprisingly non-trivial and hard to discover.
This code uses NodeJS' readline module to read and write text file line by line. It can work on big files.
const fs = require("fs");
const readline = require("readline");
const input_path = "input.txt";
const output_path = "output.txt";
const inputStream = fs.createReadStream(input_path);
const outputStream = fs.createWriteStream(output_path, { encoding: "utf8" });
var lineReader = readline.createInterface({
input: inputStream,
terminal: false,
});
lineReader.on("line", function (line) {
outputStream.write(line + "\n");
});
Related
Is it possible to return the contents of a static path to a directory instead of using an .
I want to write a script that reads the contents of a directory on the file system to a given time daily. This is integrated in a webapp I can't edit.
Short answer: you can't.
You need to do this server-side. Here is an answer from a similar question, using node.js.
You can use the fs.readdir or fs.readdirSync methods.
fs.readdir
const testFolder = './tests/';
const fs = require('fs');
fs.readdir(testFolder, (err, files) => {
files.forEach(file => {
console.log(file);
});
});
fs.readdirSync
const testFolder = './tests/';
const fs = require('fs');
fs.readdirSync(testFolder).forEach(file => {
console.log(file);
});
The difference between the two methods, is that the first one is asynchronous, so you have to provide a callback function that will be executed when the read process ends.
The second is synchronous, it will return the file name array, but it will stop any further execution of your code until the read process ends.
When using child_process.spawn in Node, it spawns a child process and automatically create stdin, stdout and stderr streams to interact with the child.
const child = require('child_process');
const subProcess = child.spawn("python", ["myPythonScript.py"])
subProcess.stdout.on('data', function(data) {
console.log('stdout: ' + data);
});
I thus imlemented this in my project but the thing is that the subprocess actually write on the output stream only when the buffer reach a certain size. And not when the buffer is set with data (whatever the size of the data).
Indeed, i'd like to receive the subprocess output stream directly when it writes it on the output stream, and not when it has filled the whole buffer. any solution ?
EDIT: As pointed out by t.888, it should actually be working as i expect. And it actually does if I spawn another subprocess. A c++ one this time. But I don't know why it does not work when I spawn my python script. Actually, the python script sends only big chunks of messages via stdout (probably when the buffer is full)
I think that you need readline instead.
const fs = require('fs');
const readline = require('readline');
async function processLineByLine() {
const fileStream = fs.createReadStream('input.txt');
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
// Note: we use the crlfDelay option to recognize all instances of CR LF
// ('\r\n') in input.txt as a single line break.
for await (const line of rl) {
// Each line in input.txt will be successively available here as `line`.
console.log(`Line from file: ${line}`);
}
}
processLineByLine();
From https://nodejs.org/api/readline.html#readline_example_read_file_stream_line_by_line
I solved my problem yesterday. It was actually due to python itself and not child_process function.
I have to do
const subProcess = child.spawn("python", ["-u", "myPythonScript.py"])
instead of
const subProcess = child.spawn("python", ["myPythonScript.py"])
indeed, -u argument tells python to flush data as soon as possible.
I am trying to get json values using nodejs but not working.I have searched some question in stackoverflow related this but always I am getting [Object Object] like this.I do not know Why I am getting like this.Anyone can resolve this issue?
file.json:
{
"scripts": {
"mr": "place",
"kg": "time",
"bh": "sec"
}
}
extension.js:
var fs = require("fs");
var file = JSON.parse(fs.readFileSync("c:\\xampp\\htdocs\\projects\\file.json", "utf8"));
console.log(file);
This is not duplicate. I have tried many ways but not working.
Note:I am using this code inside my visual studio code extension.
In node, you can import JSON like a JavaScript file
const file = require('./file.json')
console.log(file)
See is there a require for json in node.js for more info
const data = require("./file.json")
console.log(data.scripts)
Try this one out it is simple.
const fs = require('fs');
const paht = require('path');
console.log(paht.join(__dirname,'../file.json'));
let file = JSON.parse(fs.readFileSync(paht.join(__dirname,'../file.json'), "utf8"));
__dirname gives you the directory of your current file, I used path.join to make sure I can go further.
I put the json file in upper directory in my case
I want to make a simple node module that can be run from the command line that I can input files into, then it might change every instance of 'red' to 'blue' for example and then save that as a new file. Is there a simple example out there somewhere that I can edit to fit my purposes? I've tried looking but couldn't find one that was sufficiently simple to understand how to modify it. Can anyone help?
A simple example of replace.js (both old and new files are supposed to be in UTF-8 encoding):
'use strict';
const fs = require('fs');
const oldFilePath = process.argv[2];
const newFilePath = process.argv[3];
const oldFileContent = fs.readFileSync(oldFilePath, 'utf8');
const newFileContent = oldFileContent.replace(/red/g, 'blue');
fs.writeFileSync(newFilePath, newFileContent);
How to call:
node replace.js test.txt new_test.txt
Documentation on used API:
process.argv
fs.readFileSync()
fs.writeFileSync()
I have created a zip file of a .txt file using node.js zlib and while trying to unzip the file i used the same node_module zlib but its throwing error.Please check my code snippet and please feel free to edit code.I also checked the file location its same as the
Error :
Error: unexpected end of file
at Unzip.zlibOnError (zlib.js:153:15)
Code :
const fs=require('fs');
const zlib = require('zlib');
const iii = fs.createReadStream('test.txt.gz');
const oo = fs.createWriteStream('test1.txt');
const unzip = zlib.createUnzip()
iii.pipe(unzip).pipe(oo)
I am unable to understand my mistake.Can anyone help me with the same?
Thanks in advance!!
Used below code to zip the file.
code to zip:
const fs=require('fs');
const zlib = require('zlib');
const gzip = zlib.createGzip();
const inp = fs.createReadStream('test.txt');
const out = fs.createWriteStream('test.txt.gz');
inp.pipe(gzip).pipe(out);
I was getting an error because the file got corrupted while trying the zip-unzip.
There was no error in the code.
So after deleting and recreating zip file, i was able to unzip it.
Note:
Whenever you try something like this,if you get such error, then always check file size it should not be zero. If it is zero,then recreate the zip and try again.