I'm trying to read some data from a locally stored JSON file, process it into separate JS objects, and append it to a queue. For the life of me, I can't find a way to test that my parsing function is working. To test that it works, I'm trying to pull the data from the local JSON file, and and print it to console. I think there's something conceptually that I don't understand, or a method to test this that I don't know of.
I cannot run the fs module in the browser even with Browserify. Firefox tells me that createReadStream is not a function.
index.html
<!DOCTYPE html>
<div id="container" style="position: relative; width: 100%; height: 700px;"></div>
<!-- <link rel="stylesheet" type="text/css" href="index.css" /> -->
<body>
<script type="text/javascript" src="parse.js"></script>
</body>
</html>
parse.js
//Assume JSON output = {attack source, attack destination, attack type}
//Required modules
//#ts-ignore: undeclared module
const watchStream = require('fs-watch-stream')
const es = require('event-stream')
const stream = require('JSONStream')
const fs = require('fs');
//Create websocket to receive json data instead?
//Save for later
//https.createServer(attacks).listen(9090)
function initial (){
var filepath = 'sample.json';
//Creates readable stream for JSON file parsing
var stream = fs.createReadStream(filepath, { encoding: 'utf8' }),
parser = stream.parse('*.');
//Send read data to parser function
return stream.pipe(parser);
}
initial()
.pipe(es.mapSync(function (data) {
console.log(data);
}));
sample.json
{
"employees": {
"employee": [
{
"id": "1",
"firstName": "Tom",
"lastName": "Cruise",
"photo": "https://jsonformatter.org/img/tom-cruise.jpg"
},
{
"id": "2",
"firstName": "Maria",
"lastName": "Sharapova",
"photo": "https://jsonformatter.org/img/Maria-Sharapova.jpg"
},
{
"id": "3",
"firstName": "Robert",
"lastName": "Downey Jr.",
"photo": "https://jsonformatter.org/img/Robert-Downey-Jr.jpg"
}
]
}
}
There's no need to involve the browser; just run node parse.js. Node.js has console.log too, and you will see the output in the console.
If you install fs-extra, you can use fs.readJson() to get the JSON as an object directly.
Related
Right, so I am trying to wrap my head around editing (appending data) to a JSON file.
The file (users.json) looks like this:
{
"users": {
"id": "0123456789",
"name": "GeirAndersen"
}
}
Now I want to add users to this file, and retain the formatting, which is where I can't seem to get going. I have spent numerous hours now trying, reading, trying again... But no matter what, I can't get the result I want.
In my .js file, I get the data from the json file like this:
const fs = require('fs').promises;
let data = await fs.readFile('./test.json', 'utf-8');
let users = JSON.parse(data);
console.log(JSON.stringify(users.users, null, 2));
This console log shows the contents like it should:
{
"id": "0123456789",
"name": "GeirAndersen"
}
Just to test, I have defined a new user directly in the code, like this:
let newUser = {
"id": '852852852',
"name": 'GeirTrippleAlt'
};
console.log(JSON.stringify(newUser, null, 2));
This console log also shows the data like this:
{
"id": "852852852",
"name": "GeirTrippleAlt"
}
All nice and good this far, BUT now I want to join this last one to users.users and I just can't figure out how to do this correctly. I have tried so many version and iterations, I can't remember them all.
Last tried:
users.users += newUser;
users.users = JSON.parse(JSON.stringify(users.users, null, 2));
console.log(JSON.parse(JSON.stringify(users.users, null, 2)));
console.log(users.users);
Both those console logs the same thing:
[object Object][object Object]
What I want to achieve is: I want to end up with:
{
"users": {
"id": "0123456789",
"name": "GeirAndersen"
},
{
"id": "852852852",
"name": "GeirTrippleAlt"
}
}
When I get this far, I am going to write back to the .json file, but that part isn't an issue.
That's not really a valid data structure, as you're trying to add another object to an object without giving that value a key.
I think what you're really looking for is for 'users' to be an array of users.
{
"users": [
{
"id": "0123456789",
"name": "GeirAndersen"
},
{
"id": "852852852",
"name": "GeirTrippleAlt"
}
]
}
You can easily create an array in JS and the push() new items into your array. You JSON.stringify() that with no issue.
const myValue = {
users: []
};
const newUser = {
'id': '0123456789',
'name': "GeirAndersen'
};
myValue.users.push(newUser);
const strigified = JSON.stringify(myValue);
I'm working on a command that will automatically fetch a file from a link once a day and extract two of the elements in it and send that as a message in a channel.
My issue here is that I'm having issues actually getting the file downloaded. I've been trying several different functions to fetch the file but nothing has worked so far. I have attached one of the functions I've tried below.
async function getQuote () {
const url = "https://quotes.rest/qod?category=inspire";
const path = Path.resolve(__dirname, 'temp', 'qod.json')
const writer = fs.CreateWriteStream(path)
const response = await axios({
url,
method: 'GET',
responseType: 'stream'
})
response.data.pipe(writer)
getQuote();
return new Promise((resolve, reject) => {
writer.on('finish', resolve)
writer.on('error', reject)
})
}
fs.readFile('./temp/qod.json', 'utf8', function (err, data) {
if (err) throw err;
var obj = JSON.parse(data);
msg.channel.send(data);
})
The file I'm trying to work with here looks something like this:
{
"success": {
"total": 1
},
"contents": {
"quotes": [
{
"quote": "What you do speaks so loudly that I cannot hear what you say.",
"length": "61",
"author": "Ralph Waldo Emerson",
"tags": [
"action",
"inspire",
"leadership",
"management",
"tod"
],
"category": "inspire",
"language": "en",
"date": "2020-08-23",
"permalink": "https://theysaidso.com/quote/ralph-waldo-emerson-what-you-do-speaks-so-loudly-that-i-cannot-hear-what-you-say",
"id": "eZ0NtMPtGp8c5eQJOBfJmweF",
"background": "https://theysaidso.com/img/qod/qod-inspire.jpg",
"title": "Inspiring Quote of the day"
}
]
},
"baseurl": "https://theysaidso.com",
"copyright": {
"year": 2022,
"url": "https://theysaidso.com"
}
}
It wants to download as a json file, but when visiting the link, it is listed as a xml document.
How would I go about getting this downloaded and extracting two lines from it? If you're wondering, the two lines are the quote and author lines.
Thanks!
I copy your code and run my local machine and everythin fine.
Limitations are like mirages created by your own mind. When you realise that limitation do not exist, those around you will also feel it and allow you inside their space. - Stephen Richards
Looks like you are trying to write the result to a file and then read from the file which is not efficient. Here's a much simpler way of doing it.
async function getQuote() {
const url = "https://quotes.rest/qod?category=inspire";
const response = await axios(url);
const result = response.data;
/*
result =
{
"success": {
"total": 1
},
"contents": {
"quotes": [
{
"quote": "Limitations are like mirages created by your own mind. When you realise that limitation do not exist, those around you will also feel it and allow you inside their space. ",
"length": "171",
"author": "Stephen Richards",
"tags": [
"inspire",
"motivational",
"positive-thinking",
"self-empowerment",
"self-help",
"self-improvement",
"wealth",
"wealth-creation"
],
"category": "inspire",
"language": "en",
"date": "2020-08-24",
"permalink": "https://theysaidso.com/quote/stephen-richards-limitations-are-like-mirages-created-by-your-own-mind-when-you",
"id": "OLSVpLiSwrWplvCcFgPPiweF",
"background": "https://theysaidso.com/img/qod/qod-inspire.jpg",
"title": "Inspiring Quote of the day"
}
]
},
"baseurl": "https://theysaidso.com",
"copyright": {
"year": 2022,
"url": "https://theysaidso.com"
}
}
*/
//this is an array of quote objects
const quotes = result.contents.quotes;
//extracting first quote object from the array
const quoteObject = quotes[0];
//extracting quote text and author from quote object
const quote = quoteObject.quote;
const author = quoteObject.author;
//the >>> will make it look like a quote in discord.
console.log(`>>> ${quote}\n- ${author}`);
//send the formatted quote to the channel
msg.channel.send(`>>> ${quote}\n- ${author}`);
//if for some reason you want to save the result to a file
fs.writeFile(filePath, result, function(err) {
if (err) throw err;
console.log('Saved!');
});
}
getQuote();
<script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script>
I would suggest simply reading the quote to an object, then creating a string using interpolation and send it on the discord channel:
async function getQuote () {
const url = "https://quotes.rest/qod?category=inspire";
console.log("getQuote: Reading quote...");
// Get the response as an object
const response = await axios({
url,
method: 'GET'
})
// Use destructuring to get the quote and author
let { quote, author } = response.data.contents.quotes[0];
// Format our quote
let data = `${quote} - ${author}`;
// Add a console.log for debugging purposes..
console.log("getQuote: Sending quote:", data);
// Send the quote on the channel
msg.channel.send(data);
}
Todays quote would then look like so:
Limitations are like mirages created by your own mind. When you realise that limitation do not exist, those around you will also feel it and allow you inside their space. - Stephen Richards
I have a JSON file that I am using javascript to read, I am able to print it out in console, but I have to manually code the number of objects in the JSON file, the file is really simple with only 3 objects, I would like to create a function that checks how many objects are in JSON file.
Json Code
{
"items": [
{
"fname": "Kali",
"lname": "Flower",
"age": "19"},
{
"fname": "JD",
"lname": "Wyatt",
"age": "19"
}]
}
I'm trying to write a javascript function showing how many objects are in it
Use JSON.parse to convert the content to object then use .length to get the size:
JSON.parse(fs.readFileSync(file)).items.length
Explained:
const fs = require("fs"); // require fs
const content = fs.readFileSync(file); // read the file content as string
const obj = JSON.parse(content); // convert string to object
const length = obj.items.length;
Example:
Before:
{
"firstName": "CSS",
"lastName": "HTML",
"isAlive": true,
"age": 5,
"height_cm": 111.12,
}
After:
{
"firstName": "JAVA",
"lastName": "nodeJS",
"isAlive": true,
"age": 5,
"height_cm": 111.12,
}
I want to make changes in Json object(firstname and lastname) and save it into to onother Json file
Could be done like that:
var fs = require("fs");
// read
var content = fs.readFileSync("./before.json", "utf-8");
var object = JSON.parse(content);
// change
object.firstName = "JAVA";
object.lastName = "nodeJS";
// write
var newContent = JSON.stringify(object, null, 4);
fs.writeFileSync("./after.json", newContent, "utf-8");
Note: A prettier approach would use readFile and writeFile instead of their synchronous versions, but this shows the general idea in a cleaner way.
I have created an Azure function to save data in SQL database from Iot Hub that is working fine, Now I want to save Exception and Error to Azure storage Table so for that I have added try{ } catch(err){} but that is not working. please correct me. Thanks!
my function is here
module.exports = function (context, iotHubMessage) {
try {
var strMsg = JSON.stringify(iotHubMessage);
context.log('Message received: ' + strMsg);
var ob1 = { "employee_idw": 444, "last_name": "Teller", "first_name": "Saara", "age": 34, "salary": 87000 };
//I misspelled 'employee_idw' to generate error
var ob2 = { "employee_id": 555, "last_name": "Teller", "first_name": "Saara", "age": 31, "salary": 87000 };
ob1.EventProcessedUtcTime = new Date;
ob2.EventProcessedUtcTime = new Date;
var arr = [];
arr.push(ob1);
arr.push(ob2);
context.bindings.outputTable = arr;
context.done();
} catch (err) {
context.log('CCC Error' + err); // even can not see this message in log
context.bindings.error= { "partitionKey": partitionKey, "rowKey": rowKey, "data": err };
}
};
see this is JSON file
{
"bindings": [
{
"type": "eventHubTrigger",
"name": "myEventHubMessage",
"path": "myeventhub",
"consumerGroup": "$Default",
"connection": "PBCorIOTHub_events_IOTHUB",
"cardinality": "many",
"direction": "in"
},
{
"type": "apiHubTable",
"name": "outputTable",
"dataSetName": "default",
"tableName": "employees",
"connection": "sql_SQL",
"direction": "out"
},
{
"type": "table",
"name": "error",
"tableName": "dddtTest",
"connection": "cccteststr_STORAGE",
"direction": "out"
}
],
"disabled": false
}
Are you using Azure SQL or Azure table storage to store the data? From your code it looks like you are using Azure table storage. The reason i ask is because a changed property name would not cause an error in function. Instead the table storage would create a new property with misspelled name.
Like Mikhail suggested the to store an error caused inside of a function all you have to do is create another output binding and assign the exception to it.
However not all exceptions occur inside of a function context. For example an error in function.json configuration could cause a error connecting to storage. This would cause function execution to fail outside of function code context. Azure functions has direct integration with Application Insights and can help monitor what you are looking for. Here is a blog post that can shows how to configure Application Insights.
https://blogs.msdn.microsoft.com/appserviceteam/2017/04/06/azure-functions-application-insights/