I am using Node.js to read data from an XML file. But when I try to compare the data from the file with a literal, it is not matching, even though it looks the same:
const parser: xml2js.Parser = new xml2js.Parser();
const suggestedMatchesXml: any
= fs.readFileSync(`${inputSuggMatchXmlFile}`, 'utf8');
parser.parseString(suggestedMatchesXml, (_parseErr: any, result: any) => {
// console.debug(`result = ${util.inspect(result, false, null)}`);
suggestedMatchesObjFromXml = JSON.parse(JSON.stringify(result));
// console.debug(`suggestedMatchesObjFromXml BEFORE = ${JSON.stringify(suggestedMatchesObjFromXml)}`);
});
const destinations: Array<any> = suggestedMatchesObjFromXml.suggestedmatches.destination;
let docIdForUrl: string | undefined;
_.each(destinations, (destination: any) => {
const { url }: { url: string } = destination;
if (!destination.docId) {
console.debug(`processInputSmXmlFile(): url = ${url} ; index = ${_.indexOf(url, 'meetings')}`);
Here's the log:
processInputSmXmlFile(): url = https://apps.na.collabserv.com/meetings/ ; index = -1
I'm not sure how this could happen, unless one of those strings is unicode, and the other isn't.
How can I convert this one way or the other so that the data matches?
Because I was doing a JSON.parse(), url was not a string - it was an object. When I did a _.toString(url), and replaced _.indexOf(url, 'meetings') with _.includes(url, 'meetings') (since Lodash indexOf() is only for arrays), now everything is working.
Related
I have the above string I want to remove gsi along with the value. After removing gsi my string should look like given below.
Scenario 1
/product/Abrasives?gsi=1
Expected output
/product/Abrasives
Scenario 2
/product/Abrasives?query=search&gsi=1
Expected output
/product/Abrasives?query=search
Scenario 3
/product/Abrasives?query=search&gsi=1&fname=abc
Expected output
/product/Abrasives?query=search&fname=abc
You could use the URL constructor and delete the parameter:
const paths = ["/product/Abrasives?gsi=1", "/product/Abrasives?query=search&gsi=1", "/product/Abrasives?query=search&gsi=1&fname=abc"]
function removeParam(path, name) {
const url = new URL(path, "https://a.com")
url.searchParams.delete(name)
return url.pathname + url.search
}
console.log(
paths.map(p => removeParam(p, "gsi"))
)
Or you could split at ? get the querystring and pathname separately. Use the URLSearchParams constructor and delete the parameter
const paths = ["/product/Abrasives?gsi=1", "/product/Abrasives?query=search&gsi=1", "/product/Abrasives?query=search&gsi=1&fname=abc"]
function removeParam(path, name) {
const [pathName, queryString] = path.split('?');
const searchParams = new URLSearchParams(queryString);
searchParams.delete(name);
return pathName + searchParams
}
console.log(
paths.map(p => removeParam(p, "gsi"))
)
How can I convert this text to JSON by nodejs?
Input :
---
title:Hello World
tags:java,C#,python
---
## Hello World
```C#
Console.WriteLine(""Hello World"");
```
Expected output :
{
title:"Hello World",
tags:["java","C#","python"],
content:"## Hello World\n```C#\nConsole.WriteLine(\"Hello World\"");\n```"
}
What I've tried to think :
use regex to get key:value array, like below:
---
{key}:{value}
---
then check if key equals tags then use string.split function by , to get tags values array else return value.
other part is content value.
but I have no idea how to implement it by nodejs.
If the input is in a known format then you should use a battle tested library to convert the input into json especially if the input is extremeley dynamic in nature, otherwise depending on how much dynamic is the input you might be able to build a parser easily.
Assuming the input is of a static structure as you posted then the following should do the work
function convertToJson(str) {
const arr = str.split('---').filter(str => str !== '')
const tagsAndTitle = arr[0]
const tagsAndTitleArr = tagsAndTitle.split('\n').filter(str => str !== '')
const titleWithTitleLabel = tagsAndTitleArr[0]
const tagsWithTagsLabel = tagsAndTitleArr[1]
const tagsWithoutTagsLabel = tagsWithTagsLabel.slice(tagsWithTagsLabel.indexOf(':') + 1)
const titleWithoutTitleLabel = titleWithTitleLabel.slice(titleWithTitleLabel.indexOf(':') + 1)
const tags = tagsWithoutTagsLabel.split(',')
const result = {
title: titleWithoutTitleLabel,
tags,
content: arr[1].slice(0, arr[1].length - 1).slice(1) // get rid of the first new line, and last new line
}
return JSON.stringify(result)
}
const x = `---
title:Hello World
tags:java,C#,python
---
## Hello World
\`\`\`C#
Console.WriteLine(""Hello World"");
\`\`\`
`
console.log(convertToJson(x))
Looks like you're trying to convert markdown to JSON. Take a look at markdown-to-json.
You can also use a markdown parser (like markdown-it) to get tokens out of the text which you'd have to parse further.
In this specific case, if your data is precisely structured like that, you can try this:
const fs = require("fs");
fs.readFile("input.txt", "utf8", function (err, data) {
if (err) {
return console.log(err);
}
const obj = {
title: "",
tags: [],
content: "",
};
const content = [];
data.split("\n").map((line) => {
if (!line.startsWith("---")) {
if (line.startsWith("title:")) {
obj.title = line.substring(6);
} else if (line.startsWith("tags")) {
obj.tags = line.substring(4).split(",");
} else {
content.push(line);
}
}
});
obj.content = content.join("\n");
fs.writeFileSync("output.json", JSON.stringify(obj));
});
Then you just wrap the whole fs.readFile in a loop to process multiple inputs.
Note that you need each input to be in a separate file and structured EXACTLY the way you mentioned in your question for this to work. For more general usage, probably try some existing npm packages like others suggest so you do not reinvent the wheel.
I need to get data from a csv file hosted on a url and convert it to json array. Till now I'm using this.
import request from "request-promise";
import encode from "nodejs-base64-encode";
let url = 'https://example.com/filename.csv';
function conversion(url) {
return request.get(url, {encoding: null})
.then( function (res) {
//var base64File = encode.encode(res, 'base64');
//how to convert the received data to json
});
}
I have tried converting to base64 first and then decoding the string but it just gives me string like below.
name,value,email
jason,23,email#email.com
jeremy,45,email#email.com
nigel,23,email#email.com
jason,23,email#email.com
I need it as a json array.
You can include a package like csvtojson to handle this conversion for you.
/** csv file
a,b,c
1,2,3
4,5,6
*/
const csvFilePath='<path to csv file>'
const csv=require('csvtojson')
csv()
.fromFile(csvFilePath)
.then((jsonObj)=>{
console.log(jsonObj);
/**
* [
* {a:"1", b:"2", c:"3"},
* {a:"4", b:"5". c:"6"}
* ]
*/
})
// Async / await usage
const jsonArray=await csv().fromFile(csvFilePath);
You're halfway there.
You just need to:
split the string by end-of-line \n. You get an array of strings,
each representing a line in your csv file.
split each line by ,.
And in your case, don't forget to ignore the first line (the header).
Check this article out:
https://medium.com/#sanderdebr/converting-csv-to-a-2d-array-of-objects-94d43c56b12d
Split the string by \r\n to get an array of strings. Use the .map in array to make it an array of objects.
Here is an example:
var newArray = decodedString.split('\r\n').slice(1)map(row => {
var temp = row.split(',')
return {
name: temp[0],
value: temp[1],
email: temp[2]
}
})
Hope this helps :)
Include a package called csvtojson
const request=require('request')
const csv=require('csvtojson')
csv()
.fromStream(request.get('http://mywebsite.com/mycsvfile.csv'))
.subscribe((json)=>{
return new Promise((resolve,reject)=>{
// long operation for each json e.g. transform / write into database.
})
},onError,onComplete);
With Async and Await
async function ConvertCSVtoJSON{
const jsonArray = await csv().fromStream(request.get('http://mywebsite.com/mycsvfile.csv'));
console.log(jsonArray);
}
DCMTK dcm2json produces invalid JSON for broken DS or IS values see https://support.dcmtk.org/redmine/issues/769
The JSON will contain values like this example:
"00291003": {"vr":"IS","Value":[F]},
Where 'F' is clearly not a number.
This can be worked around in a Nodejs environment by first running the raw unparsed JSON through the following.
const jsonRepair = (rawJson) => {
let regex1 = /\[([^"]+)\]/g
let isNumRegex = /^[-.0-9]*\d$/g
let matches = undefined
while ((matches = regex1.exec(rawJson)) !== null) {
if (!isNumRegex.test(matches[1])) {
rawJson = rawJson.replace(matches[0], `["${matches[1]}"]`)
console.log(matches[0], `[${matches[1]}]`)
}
}
return rawJson
}
I need to filter out params from my URL.
So for example:
URL = test1/test2/:test3
some filter output = test3
URL = test1/:test2/test3
some filter output = test2
URL = :test1/test2/test3
some filter output = test1
URL = :test1/test2/test3/:test4
some filter output = test1, test4
I've tried some REGEX and substring filters myself, but I can't really get it to work considering all possible URL examples above in 1 filter.
Something like this?:
https://regex101.com/r/Jh0BVJ/3/
regex = new RegExp(/:(?<match>(?:[\w\-.~:?#\[\]#!$&'()*+,;=%]*)\d+)+/gm)
url = ":test1/test2/test3/:test4"
console.log(url.match(regex))
You are actually just creating another router:
https://github.com/krasimir/navigo
I've never used navigo, but it looks powerful enough:
router
.on('/user/:id/:action', function (params) {
// If we have http://example.com/user/42/save as a url then
// params.id = 42
// params.action = save
})
.resolve();
router.notFound(function (query) {
// ...
});
Now use it:
router.navigate('/products/list');
//Or even with FQDN:
router.navigate('http://example.com/products/list', true);
const REGEX = /\:([A-z0-9-._~:?#\[\]\#\!\$\&\'\(\)\*\+\,\;\=]+)/g;
function findMatches(re, str, matches = []) {
const m = re.exec(str)
m && matches.push(m[1]) && findMatches(re, str, matches)
return matches
}
const urls = [
'test1/test2/:test3',
'test1/:test2/test3',
':test1/test2/test3',
':test1/test2/test3/:test4',
];
const values = urls.map(url => findMatches(REGEX, url));
console.log(values);