We Keep Coding

csv-parse is throwing Invalid Opening Quote: a quote is found inside a field at line

I know there are other posts out there but none of them seem to fix my issues. I am using csv-parse with node js. This is the CSV header and record that I'm trying to parse.
sku,description,productUnitOfMeasure,unitCost,retailPrice,wholesalePrice,dropShipPrice,assemblyCost,planner,comments,productId,fileUpdate,SkuStatus,Master Planning Family,Category,Sub-Category,Brand,ShortCode,Import/Domestic,Inventory Value,Master Pack Quantity,Pallet / TI / HI,40HC Quantity,Product Group ID
032406021945-GreenBay,TFAL B2080264 INIT TNS GRY SAUTE PN 8",EA,7.72,13.99,0.00,0.00,0,Whitney Ehlke-2307,,032406021945,2022-01-25,New,COOKWARE,OPENSTOCK,NONE,T-FAL,B2080264,Domestic,208.44,3,0/0/0,0,23
I have no control over this file. I just need to be able to parse it. You will see that there is a double quote at the end of the description: TFal B2080264 INI TNS GRY SAUTE PN 8".
I need the double quote to stay there and for that to parse as one field. I keep getting this error:
Invalid Opening Quote: a quote is found inside a field at line 2.
The quote is not an opening. It's technically a closing. But regardless, it will not parse.
This is currently my code:
const parser = fs.createReadStream(filePath).pipe(
parse({ columns: true, relax_quotes: true, escape: '\\', ltrim: true, rtrim: true })
)
I have removed some of the params and tried others, to no avail. Any ideas??

This code works fine with the latest csv-parse version (5.0.4). Which version of the csv-parse package are you using? I ask because it looks like the option may have been renamed from relax to relax_quotes only recently.
So, I think the solution is either:
upgrade to the latest csv-parse, and indicate relax_quotes, or
stay with your current version of csv-parse, and indicate relax
Just to be sure relax_quotes works with the current library, I tested the following code and it worked as expected:
const csv = require('csv-parse');
const fs = require('fs');
const parser = fs.createReadStream("70880341.csv").pipe(
csv.parse({ columns: true, relax_quotes: true, escape: '\\', ltrim: true, rtrim: true })
)
const records = [];
parser.on('readable', function() {
let record;
while ((record = parser.read()) !== null) {
records.push(record);
}
});
parser.on('error', function(err) {
console.error(err.message);
});
parser.on('end', function() {
console.log(records);
});
Result:
[{
sku: '032406021945-GreenBay',
description: 'TFAL B2080264 INIT TNS GRY SAUTE PN 8"',
productUnitOfMeasure: 'EA',
...
}]

For me full error message was: CsvError: Invalid Opening Quote: a quote is found on field 0 at line 1, value is "" (utf8 bom)
And adding "bom: true" parameter for csv.parse - helped.

How to apply regular expression for Javascript

I am trying to get message log from Azure application Insight like this
az monitor app-insights --app [app id] --analystics-query [condition like specific message id]
Then I got a message like this
"message": [
"Receiving message: {"type":"CTL","traceId":"f0d11b3dbf27b8fc57ac0e40c4ed9e48","spanId":"a5508acb0926fb1a","id":{"global":"GLkELDUjcRpP4srUt9yngY","caller":null,"local":"GLkELDUisjnGrSK5wKybht"},"eventVersion":"format version","timeStamp":"2021-10-01T14:55:59.8168722+07:00","eventMetadata":{"deleteTimeStamp":null,"ttlSeconds":null,"isFcra":null,"isDppa":true,"isCCPA":true,"globalProductId":null,"globalSubProductId":null,"mbsiProductId":null},"eventBody":{"sys":"otel","msg":"Testing Centralized Event Publisher with App1 (using logback)","app":{"name":"otel","service":"postHouse","status":"status name","method":"POST","protocol":"HTTP","resp_time_ms":"250","status_code":"4"},}}"
] }
So that I would like to apply Regular Expression for this message to get only the message from {"type.....to "status_code":"4"},}} and also convert it to JSON format
I have code like this in my .js file
Then('extract json from {string}', function(message){
message = getVal(message, this);
const getmess = message.match(/{(.*)}/g);
const messJson = JSON.parse(getmess);
console.log(messJson);
})
But it doesn't work for me
SyntaxError: Unexpected token \ in JSON at position 1
How can I apply this in my code on Javascript? Thank you so much for your help

Try this. But keep in mind, that current regex is binded with provided program output syntax. If output will be different in wrapper structure, this regex might not work any more.
// Text from app
const STDOUT = `
"message": [ "Receiving message: {"type":"CTL","traceId":"f0d11b3dbf27b8fc57ac0e40c4ed9e48","spanId":"a5508acb0926fb1a","id":{"global":"GLkELDUjcRpP4srUt9yngY","caller":null,"local":"GLkELDUisjnGrSK5wKybht"},"eventVersion":"format version","timeStamp":"2021-10-01T14:55:59.8168722+07:00","eventMetadata":{"deleteTimeStamp":null,"ttlSeconds":null,"isFcra":null,"isDppa":true,"isCCPA":true,"globalProductId":null,"globalSubProductId":null,"mbsiProductId":null},"eventBody":{"sys":"otel","msg":"Testing Centralized Event Publisher with App1 (using logback)","app":{"name":"otel","service":"postHouse","status":"status name","method":"POST","protocol":"HTTP","resp_time_ms":"250","status_code":"4"},}}"
] }
`;
// Match JSON part string
let JSONstr = /.*\[\s*\"Receiving message:\s*(.*?)\s*\"\s*]\s*}\s*$/.exec(STDOUT)[1];
// Remove trailing comma(s)
JSONstr = JSONstr.replace(/^(.*\")([^\"]+)$/, (s, m1, m2) => `${m1}${m2.replace(/\,/, "")}`);
// Convert to object
const JSONobj = JSON.parse(JSONstr);
// Result
console.log(JSONobj);

Try this one:
/.*?({"type":.*?,"status_code":"\d+"\})/
When used in Javascript, the part covered by the parentheses counts as Group 1, i.e.,:
const messJson = JSON.parse(message.match(/.*?({"type":.*?,"status_code":"\d+"\})/)[1]);
Reference here: https://regexr.com/66mf2

JS Build Error, Lint Error no-constant-condition

Im getting error Unexpected constant condition no-constant-condition when trying to build this code. (Error is on line indicated with >>>)
The code runs off a user input, commandVariable0 & commandVariable1 take data from the input, in this case b2b on or b2b off, if they user should put b2b test then it should give the error warning: commandVariable1 is not a valid settings option.
Can anyone help me understand why this is not working, if a user puts b2b test it accepts it as valid.
Any help would be appriciated.
const b2b = function(data)
{
const commandVariable0 = data.cmd.params.split(" ")[0].toLowerCase();
const commandVariable1 = data.cmd.params.split(" ")[1].toLowerCase();
>>> if (commandVariable1 === "on"||"off")
{
b2bvar = commandVariable1;
var output =
"**```Bot 2 Bot Command Here```**\n" +
`Embeded Messages : ${b2bvar}\n\n` +
`Command Variable 0 : ${commandVariable0}\n` +
`Command Variable 1 : ${commandVariable1}\n`;
data.color = "info";
data.text = output;
return botSend(data);
}
data.color = "error";
data.text =
":warning: **`" + commandVariable1 +
"`** is not a valid settings option.";
return botSend(data);
};
Please ignore the sloppy code.

Change your line
if (commandVariable1 === "on"||"off")
to
if (commandVariable1 === "on" || commandVariable1 === "off")
What's the problem?
In the above line JavaScript will first check if commandVariable1 equals "on" and if not then it will convert "off" into a boolean and check if that's true.
Since the string "off" is a constant and will always result to true in JavaScript, your linter will complain about the unneccessary OR condition.

Javascript code output differs when ran from node.js shell and from a file

I have this strange behavior of node.js. Basically the title says it all, I would only like to add, that this wasn't the case before, with the exactly the same code.
I suspect that this started to happen after I ran sudo chown -R $USER /usr/local/lib/node_modules because I wanted to install typescript and got writing permissions error, however I cannot be 100% sure. Interestingly enough my test passes, and eventually everything works fine and the result is correct, but I don't know why my console.log() outputs are different. What I get when I put code in node shell and console.log it is a concatenated string from recursive calls. When I run code from a file, I only get the string from the first call.
Here's my code:
const fields = {
0: "D:0:0:0:1:20:9",
1: "D:1:0:22:1:95:9",
2: "D:2:0:117:1:27:9"
};
function getRequiredFields(key, trailing=false) {
// key is a number
if (typeof trailing !== "boolean") {
throw new Error("<trailing> must be of type 'boolean'. Default is false");
}
if (typeof key === "number" && !fields[key]) {
throw new Error("Field is not defined: <key> must be in range [0, 9]");
}
if (typeof key !== "number") {
throw new Error("<key> must be a number");
}
const lastByte = trailing ? "\r" : "";
if (key === 0) {
return fields[key] + lastByte;
} else {
return getRequiredFields(key-1) + "\r" + fields[key] + lastByte;
}
};
console.log(getRequiredKeys(2, true));
and here's my test (mocha+chai):
describe("Expects a positive number to build required fields, the optional 2nd argument puts a trailing carriage return if true, default is false", () => {
it("Returns a combined string for required fields", () => {
expect(getRequiredFields(2)).to.be.a("string");
expect(getRequiredFields(2, true)).to.equal("D:0:0:0:1:20:9\rD:1:0:22:1:95:9\rD:2:0:117:1:27:9\r");
expect(getRequiredFields(2, false)).to.equal("D:0:0:0:1:20:9\rD:1:0:22:1:95:9\rD:2:0:117:1:27:9");
});
it("Throws an error if <trailing> is not a boolean", () => {
expect(getRequiredFields.bind(this, 10, "foo")).to.throw("<trailing> must be of type 'boolean'");
});
it("Throws an error if <key> is higher than 9", () => {
expect(getRequiredFields.bind(this, 10, true)).to.throw("Field is not defined: <key> must be in range [0, 9]");
});
it("Throws an error if <key> is NaN", () => {
expect(getRequiredFields.bind(this, "f", false)).to.throw("<key> must be a number");
});
});

The reason is the control character \r (carriage return), which move the cursor to the leftmost point, so the further characters overwrite existing ones. Since this only effect output, the real content of the string is not effected, so all thing should be fine.
If you intended to use newline, you should use \n instead (or require('os').EOL for OS-specific one).
You can also try out the following, which would output 789456 or 123456789 depend on the enviroment.
console.log('123456\r789')

Are if and else if statements bugged in Google Apps Script?

I'm fairly new to GAS and Javascript in general and have searched far and wide to find an answer to why this isn't working, but haven't found a solution. I was wondering if one of you guys could find the problem. Here's my code:
function maintenanceRequest() {
var findFirstRequest = GmailApp.search("to:censored label:ems-request AND is:unread", 0, 1)[0];
var firstRequest = findFirstRequest.getMessages()[0];
var parseRequest = firstRequest.getPlainBody();
var requestString = String(parseRequest);
if ("Mark archived mail as read" == requestString) {
markArchivedAsRead();
findFirstRequest.moveToArchive();
}
else if ("Cleanup" == requestString) {
weeklyCleanup();
findFirstRequest.moveToArchive();
}
else {
GmailApp.sendEmail("censored", "Failure to parse command", "The EMS has recieved your request but has failed to recognize the command '" + parseRequest + "'. Please try again, using the terms 'Mark archived as read' or 'Cleanup'. If you would like to add an eligible command, please refer to function 'maintenanceRequest'.", {
name: "Email Maintenance Service",
from: "censored"
})
//Add moveToArchive line here after debugging
}
}
The code always skips the if and else if statements and jumps to the else statement, regardless of the email's content. I've tried using both == and === to no avail, and have tried switching the sides that the arguments are on. To no avail. I even created a new var, requestString to convert parseRequest to a string, even though I'm like 99% certain that it already is a string.. so what gives? Where's the problem?

Try adding trim to the string: requestString = requestString.trim()
It's usually a safe bet to use trim on any string you're getting from another service.