We Keep Coding

Write JavaScript object to an array inside a JSON file

How can I write a JavaScript object inside of an array that is inside a JSON file?
What I mean is: I'm making a Discord (message app) BOT, when the user uses the command "/add" the BOT will ask for 2 inputs, a "name" and an "artist" both this inputs make up a song so I'm creating an object called "data" for that song.
I also have a JSON file, that my database, what I want is, everytime this command is used, my object should the pushed inside of an array in my JSON file, so later on I can retrieve a random object inside of this array. How can I do that? I hope the question is not too confusing, thanks!
module.exports={
data: new SlashCommandBuilder()
.setName('add')
.setDescription('Add a song to the database.')
.addStringOption(option =>
option.setName('artist')
.setDescription('The artist of the song')
.setRequired(true))
.addStringOption(option =>
option.setName('name')
.setDescription('The name of the song')
.setRequired(true)),
async execute(interaction){
let name = interaction.options.getString('name');
let artist = interaction.options.getString('artist');
const data = { name: name, artist: artist};
await interaction.reply(`**` + artist + `**` + ` - ` + `**` + name + `**` + ` was added to the database.`)},
};
//WHAT YOU SEE FROM NOW ON IS A DIFFERENT FILE, A JSON FILE CALLED data.json with some examples of what it should look like
[
{
"name":"Die for You",
"artist":"The Weeknd"
},
{
"name":"FEAR",
"artist":"Kendrick Lamar"
}
]

You have to use node filesystem.
Import FS and use writeFile function.
https://nodejs.org/api/fs.html#filehandlewritefiledata-options
Dont forget about JSON.stringify to turn your object into string
const fs = require('fs');
const data = { name: name, artist: artist };
fs.writeFile("output.json", JSON.stringify(data), 'utf8', function (err) {
if (err) {
console.log("An error occured while writing JSON Object to File.");
return console.log(err);
}
console.log("JSON file has been saved.");
});
// Edit (Adding new objects to .json)
You have to read data from your file, add something and save again.
let rawdata = fs.readFileSync('data.json');
let data = JSON.parse(rawdata);
data.push({name: name, artist: artist});
// to use push() function data have to be an array, edit your .json file to " [] ",
// now you can add elements.
fs.writeFile("output.json", JSON.stringify(data), 'utf8', function (err) {
if (err) {
console.log("An error occured while writing JSON Object to File.");
return console.log(err);
}
console.log("JSON file has been saved.");
});

If you want to add something to an array, you .push it in there.
var arr = [];
arr.push(123);
arr.push({a:1,b:2});
arr.push(null);
arr.push("string");
console.dir(arr);

How can I duplicate a file (completely) but change it's name?

So I need to generate the smaller previews of the image files that will be uploaded and I have to append "_preview" at the end of each file name.
Currently I'm doing this:
uploadFile.map((file) => {
if (file.type.includes('image')) {
console.log('Generating thumbnail for ' + file.name)
const fileName = file.name.split('.').slice(0, -1).join('.')
const fileExtension = file.name.split('.').pop()
const compressedFile = new File(
[file.slice(0, file.size, file.type)],
fileName + '_preview.' + fileExtension,
)
console.log('Generated file:', compressedFile)
convert({
file: compressedFile,
width: 300,
height: 300,
type: fileExtension,
})
.then((resp) => {
uploadFile.push(resp)
})
.catch((error) => {
// Error
console.error('Error compressing ', file.name, '-', error)
})
}
})
The problem is that "compressedFile" is missing some fields which were present in the original file and so the convert functoin throws the error "File type not supported". As you can see "type" and "webkitRelativePath" are not copied.
Can anybody suggest how I can retain all the information from the original file and just append _preview at the end of file name?

I realized File API provides an option to pass "options" object as well which can specify the file type. For instance:
const file = new File(["foo"], "foo.txt", {
type: "text/plain",
});
Source: https://developer.mozilla.org/en-US/docs/Web/API/File/File

for copy code in js or duplicate, you can use this code
//copyfile.js
const fs = require('fs');
// destination will be created or overwritten by default.
fs.copyFile('C:\folderA\myfile.txt', 'C:\folderB\myfile.txt', (err) => {
if (err) throw err;
console.log('File was copied to destination');
});

npm package csvtojson CSV Parse Error: Error: unclosed_quote

Node version: v10.19.0
Npm version: 6.13.4
Npm package csvtojson Package Link
csvtojson({
"delimiter": ";",
"fork": true
})
.fromStream(fileReadStream)
.subscribe((dataObj) => {
console.log(dataObj);
}, (err) => {
console.error(err);
}, (success) => {
console.log(success);
});
While trying to handle large CSV file (about 1.3 million records) I face error "CSV Parse Error: Error: unclosed_quote." after certain records(e.g. after 400+ records) being processed successfully. From the CSV file i don't see any problems with data formatting there, however the parser might be raising this error because of "\n" character being found inside the column/field value.
Is there a solution already available with this package? or
is there a workaround to handle this error? or
is there a way to skip such CSV rows having any sort of errors not only this one, to let the
entire CSV to JSON parsing work without the processing getting stuck?
Any help will be much appreciated.

I've played about with this, and it's possible to hook into this using a CSV File Line Hook, csv-file-line-hook, you can check for invalid lines and either repair or simply invalidate them.
The example below will simply skip the invalid lines (missing end quotes)
example.js
const fs = require("fs");
let fileReadStream = fs.createReadStream("test.csv");
let invalidLineCount = 0;
const csvtojson = require("csvtojson");
csvtojson({ "delimiter": ";", "fork": true })
.preFileLine((fileLineString, lineIdx)=> {
let invalidLinePattern = /^['"].*[^"'];/;
if (invalidLinePattern.test(fileLineString)) {
console.log(`Line #${lineIdx + 1} is invalid, skipping:`, fileLineString);
fileLineString = "";
invalidLineCount++;
}
return fileLineString
})
.fromStream(fileReadStream)
.subscribe((dataObj) => {
console.log(dataObj);
},
(err) => {
console.error("Error:", err);
},
(success) => {
console.log("Skipped lines:", invalidLineCount);
console.log("Success");
});
test.csv
Name;Age;Profession
Bob;34;"Sales,Marketing"
Sarah;31;"Software Engineer"
James;45;Driver
"Billy, ;35;Manager
"Timothy;23;"QA

This regex works better
/^(?:[^"\]|\.|"(?:\.|[^"\])")$/g
Here is a more complex working script for big files by reading each line
import csv from 'csvtojson'
import fs from 'fs-extra'
import lineReader from 'line-reader'
import { __dirname } from '../../../utils.js'
const CSV2JSON = async(dumb, editDumb, headers, {
options = {
trim: true,
delimiter: '|',
quote: '"',
escape: '"',
fork: true,
headers: headers
}
} = {}) => {
try {
log(`\n\nStarting CSV2JSON - Current directory: ${__dirname()} - Please wait..`)
await new Promise((resolve, reject) => {
let firstLine, counter = 0
lineReader.eachLine(dumb, async(line, last) => {
counter++
// log(`line before convert: ${line}`)
let json = (
await csv(options).fromString(headers + '\n\r' + line)
.preFileLine((fileLineString, lineIdx) => {
// if it its not the first line
// eslint-disable-next-line max-len
if (counter !== 1 && !fileLineString.match(/^(?:[^"\\]|\\.|"(?:\\.|[^"\\])*")*$/g)) {
// eslint-disable-next-line max-len
console.log(`Line #${lineIdx + 1} is invalid. It has unescaped quotes. We will skip this line.. Invalid Line: ${fileLineString}`)
fileLineString = ''
}
return fileLineString
})
.on('error', e => {
e = `Error while converting CSV to JSON.
Line before convert: ${line}
Error: ${e}`
throw new BaseError(e)
})
)[0]
// log(`line after convert: ${json}`)
if (json) {
json = JSON.stringify(json).replace(/\\"/g, '')
if (json.match(/^(?:[^"\\]|\\.|"(?:\\.|[^"\\])*")*$/g)) {
await fs.appendFile(editDumb, json)
}
}
if (last) {
resolve()
}
})
})
} catch (e) {
throw new BaseError(`Error while converting CSV to JSON - Error: ${e}`)
}
}
export { CSV2JSON }

Updating content from JSON files by using the filesystem module in NodeJs

I have a small JSON file
{
"users": [
{
"id": 1111,
"name": "Foo",
"gold": 2
},{
"id": 2222,
"name": "Bar",
"gold": 7
}
]
}
and want to manipulate the data of one specific object, selected by its id.
I want to
read the data from the file
manipulate the data
write the new data back to the file
send a response to the client
so I went for this route, called by using Ajax
app.get('/incG/:id', function (req, res) {
fs.readFile('./database.json', 'utf8', function (err, data) {
var json = JSON.parse(data); // Read the data
var users = json.users; // Get all users
var user = users.find(u => u.id === Number(req.params.id)); // get the user by id
user.gold++; // increase his value
fs.writeFile('./database.json', , (err) => { // the second parameter is missing!
res.send(user.gold); // send a response to the client
});
});
});
As you can see, when using fs.writeFile(database, , (err) => { the second parameter is missing.
What do I have to pass in there? I just want to update one specific user object (one specific value).
EDIT
When passing in JSON.stringify(user) as a parameter I delete all the data in the file and just write down the new object. So this might not work this way.

Try this,
fs.readFile('./database.json', 'utf8', function (err, data) {
var json = JSON.parse(data); // Read the data
var users = json.users; // Get all users
var userGold;
users.find(u => {
if (u.id === Number(req.params.id)) {
userGold = u.gold++;
}
});
users = JSON.stringify(users);
fs.writeFile('./database.json', users, (err) => {
res.send(userGold);
});
});

fs.writeFile(file, data[, options], callback)
Pass stringified JSON as second argument.
app.get('/incG/:id', function(req, res) {
fs.readFile('./database.json', 'utf8', function(err, data) {
var json = JSON.parse(data); // Read the data
var users = json.users; // Get all users
var user = users.find(u => u.id === Number(req.params.id)); // get the user by id
user.gold++; // increase his value
var dataToWrite = JSON.stringify(user);
fs.writeFile('./database.json', dataToWrite, (err) => { // the second parameter is missing!
res.send(user.gold); // send a response to the client
});
});
});

node.js code to append data to a file

How can I append data to a file using node.js
I already have a file named myfile.json with data. I want to check if the file name exists and then append some data to that file.
I'm using following code
var writeTempFile = function (reportPath, data, callback) {
fs.writeFile(reportPath, data, function (err) {
//if (err) //say(err);
callback(err);
});
}
writeTempFile(reportDir + '_' + query.jobid + ".json", data, function (err) {
context.sendResponse(data, 200, {
'Content-Type': 'text/html'
});

You can use jsonfile
var jf = require('jsonfile');
var yourdata;
var file = '/tmp/data.json';
jf.readFile(file, function(err, obj) {
if(!err) {
var finalData = merge(obj, yourdata);
jf.writeFile(file, finalData, function(err) {
console.log(err);
});
}
});
You need to implement your merging logic in merge(object1, object2)
https://npmjs.org/package/jsonfile

Check out the following code.
function addToFile(reportPath, data, callback){
fs.appendFile(reportPath, data, function (err) {
callback(err);
});
}

Node offers fs module to work with file system.
To use this module do
var fs = require('fs')
To append some data to file you can do:
fs.appendFile('message.txt', 'data to append', function (err) {
if (err) throw err;
console.log('The "data to append" was appended to file!');
});
Node offers you both synchronous and asynchronous method to append data to file, For more information please refer to this documentation