We Keep Coding

processing json with fetch returns undefined for valid json key

I need to request data from my REST server to populate my UI (frontend). In doing so, I need to request some data from my and other servers. One such request is to get a list of states (provinces), process each one and add them to a select HTML component. I use fetch() and .json() amongst other tools to do this.
Problem:
In calling my REST server for json data, I receive the following data (taken from Chrome console):
{provinces:[Eastern Cape,Mpumalanga,Western Cape,Gauteng,KwaZulu Natal,North West,Northern Cape,Free
State,Limpopo]}
I intend to add each of these as an option to a select. While attempting to acquire the value for the provinces key, I get undefined.
I am making this call using:
fetch("http://localhost:3443/app/location/provinces").then(e => e.json()).then(e => console.log(e.provinces));
Further, since I can directly refer to json keys using the [] operator, I attempt this using
fetch("http://localhost:3443/app/location/provinces").then(e => e.json()).then(e => console.log(e['provinces']));
which as you may have guessed aswel, also returns undefined.
For the record, the full Chrome Console output is
Promise {<pending>}
undefined
Looking over some SO examples, I believe my call(s) may be correct, this one, and this one, and this one all which confirm its validity.
What else have I tried:
This SO post and this one suggested to use the json data response inside of the same then() call e.g.
fetch("http://localhost:3443/app/location/provinces").then(e => {
e.json().then(s => {
console.log(s['provinces']);
});
});
and
fetch("http://localhost:3443/app/location/provinces").then(e => {
e.json().then(s => {
console.log(s.provinces);
});
});
both which return:
Promise {<pending>}
undefined
What am I missing / doing wrong?
Update
Screenshot of Chrome console in order of commands listed above:
Resource file za-province-city.json
NodeJS express code:
const express = require('express');
const router = express.Router();
const fs = require('fs');
const raw = fs.readFileSync("./res/za-province-city.json");
const map = JSON.parse(raw);
const mapProvinceCity = {};
map.forEach(item => {
if (!mapProvinceCity.hasOwnProperty(item.ProvinceName)) {
mapProvinceCity[item.ProvinceName] = [];
}
mapProvinceCity[item.ProvinceName].push(item.City);
});
for (let key in mapProvinceCity) {
mapProvinceCity[key].sort((a, b) => a.toLocaleString().localeCompare(b.toLowerCase()));
}
router.get('/location/provinces', function (req, res, next) {
let strings = Object.keys(mapProvinceCity);
let json = JSON.stringify({provinces: strings}).replace(/"/g, '');
return res.json(json);
});
router.get('/location/:province/cities', function (req, res, next) {
let province = req.param('province');
let cities = mapProvinceCity[province];
let json = JSON.stringify({cities: cities}).replace(/"/g, '');
return res.json(json);
});
module.exports = router;
Note: if you are wondering about the replace(), each time I requested data in postman, I got

I think your issues all stem from a misunderstanding of Express' res.json().
This is basically a shortcut for
res.set("Content-type: application/json")
res.status(200).send(JSON.stringify(data))
I imagine your problems started when you thought you needed to stringify your data. What happens then is that your data is double-encoded / double stringified, hence the extra quotes. Removing the quotes though mangles your data.
console.log() is not a particularly good debugging tool as it obfuscates a lot of information. In your code, s is actually a string
"{provinces:[Eastern Cape,Mpumalanga,...]}"
I suggest you use the actual debugger instead.
The simple solution is to use res.json() as intended
router.get('/location/provinces', function (req, res, next) {
return res.json({ provinces: Object.keys(mapProvinceCity) });
});
with your client-side code looking like
fetch("http://localhost:3443/app/location/provinces")
.then(res => {
if (!res.ok) {
throw res
}
return res.json()
})
.then(data => {
console.log('Provinces:', data.provinces)
})
This goes for all your Express routes. Do not use JSON.stringify().

Using `filter` lambda in gremlin with javascript

I am using OrientDb with JavaScript and I have tried with startingWith, containing, endingWith, notContaining, notEndingWith, notStartingWith predicates unsuccessfully. Maybe is a wrong implementation from my side but I have not found documentation about how to use.
I've been looking for a way to filter with lambdas to get a sql like behavior but have not been successful. I tried to use the method described in this answer, but it is not working on JavaScript. When using the predicates the answer is an error.
I've tried that too:
What is the equivalent of the gremlin queries in gremlin javascript?
My current JavaScript code:
import * as gremlin from 'gremlin';
const traversal = gremlin.process.AnonymousTraversalSource.traversal;
const DriverRemoteConnection = gremlin.driver.DriverRemoteConnection;
const TextPredicated = gremlin.process.TextP;
const authenticator = new gremlin.driver.auth.PlainTextSaslAuthenticator('usr', 'pwd');
const remote = new DriverRemoteConnection(
'ws://localhost:8182/gremlin', {
authenticator,
traversalSource: 'g'
});
remote.addListener('socketError', (error) => { console.log(`socketError: ${error}`); });
(async () => {
try {
remote.open();
const g = await traversal().withRemote(remote);
const results = await g.V()
.where('username', TextPredicated.containing('john'))
.toList();
console.log(results);
remote.close();
} catch (error) {
console.log(error);
} finally {
remote.close();
}
})();

You don't say what your error is, but I think your Gremlin should use has() rather than where():
const results = await g.V()
.has('username', TextPredicated.containing('john'))
.toList();
Also note that TextP did not become available until TinkerPop 3.4.0 so you'd need to be sure that your graph (in your case, OrientDB) supports at least this version of TinkerPop.

Not getting error from async method in node

I use the following code
require('./ut/valid').validateFile()
});
in the validate file when I found some duplicate in config I send error
like following
module.exports = {
validateFile: function (req) {
...
if(dup){
console.log("Duplicate found: ");
return new Error("Duplicate found: ");
}
dup is true and the error should be thrown, how should I "catch" it in async method ?
I tried also like following
require('./ut/valid').validateFile(function() {
process.exit(1);
});
what I miss here, I was able to see the console log...

Your approach doens't work because you're doing something asynchronous. A common solution is to use callbacks. In node.js it's common to use a pattern called error first callbacks.
This means you need to pass a callback function to your file validation method and either return an error or your file:
// './utils/validate.js'
module.exports = {
/**
* Validates a file.
*
* #param {Function} next - callback function that either exposes an error or the file in question
*/
file: function (next) {
// ...
if (duplicate) {
console.log('Duplicate found!');
var error = new Error('Duplicate File');
// Perhaps enrich the error Object.
return next(error);
}
// Eveything is ok, return the file.
next(null, file);
}
};
The you can use it like this:
// './app.js'
var validate = require('./utils/validate');
validate.file(function (err, file) {
if (err) {
// Handle error.
process.exit(1);
}
// Everything is ok, use the file.
console.log('file: ', file);
});

I don't have much knowledge in node, but I can tell you, you are returning an error object, which is not an error from JS perspective, it's just an object, to get an error you have to throw an error like:
throw true;
or
throw new Error("Duplicate found: ");
that way it is handled as an error not as a return value

TypeError: Converting circular structure to JSON in nodejs

I am using request package for node.js
Code :
var formData = ({first_name:firstname,last_name:lastname,user_name:username, email:email,password:password});
request.post({url:'http://localhost:8081/register', JSON: formData}, function(err, connection, body) {
exports.Register = function(req, res) {
res.header("Access-Control-Allow-Origin", "*");
console.log("Request data " +JSON.stringify(req));
Here I am getting this error :
TypeError: Converting circular structure to JSON
Can anybody tell me what is the problem

JSON doesn't accept circular objects - objects which reference themselves. JSON.stringify() will throw an error if it comes across one of these.
The request (req) object is circular by nature - Node does that.
In this case, because you just need to log it to the console, you can use the console's native stringifying and avoid using JSON:
console.log("Request data:");
console.log(req);

I also ran into this issue. It was because I forgot to await for a promise.

Try using this npm package. This helped me decoding the res structure from my node while using passport-azure-ad for integrating login using Microsoft account
https://www.npmjs.com/package/circular-json
You can stringify your circular structure by doing:
const str = CircularJSON.stringify(obj);
then you can convert it onto JSON using JSON parser
JSON.parse(str)

I was able to get the values using this method, found at careerkarma.com
Output looks like this.
I just run this code in the debugger console. Pass your object to this function.
Copy paste the function also.
const replacerFunc = () => {
const visited = new WeakSet();
return (key, value) => {
if (typeof value === "object" && value !== null) {
if (visited.has(value)) {
return;
}
visited.add(value);
}
return value;
};
};
JSON.stringify(circObj, replacerFunc());

I forgotten to use await keyword in async function.
with the given systax
blogRouter.put('/:id', async (request, response) => {
const updatedBlog = Blog.findByIdAndUpdate(
request.params.id,
request.body,
{ new: true }
);
response.status(201).json(updatedBlog);
});
Blog.findByIdAndUpdate should be used with the await keyword.

use this https://www.npmjs.com/package/json-stringify-safe
var stringify = require('json-stringify-safe');
var circularObj = {};
circularObj.circularRef = circularObj;
circularObj.list = [ circularObj, circularObj ];
console.log(stringify(circularObj, null, 2));
stringify(obj, serializer, indent, decycler)

It's because you don't an async response For example:
app.get(`${api}/users`, async (req, res) => {
const users = await User.find()
res.send(users);
})

This is because JavaScript structures that include circular references can't be serialized with a"plain" JSON.stringify.
https://www.npmjs.com/package/circular-json mentioned by #Dinesh is a good solution. But this npm package has been deprecated.
So use https://www.npmjs.com/package/flatted npm package directly from the creator of CircularJSON.
Simple usage. In your case, code as follows
import package
// ESM
import {parse, stringify} from 'flatted';
// CJS
const {parse, stringify} = require('flatted');
and
console.log("Request data " + stringify(req));

If you are sending reponse , Just use await before response
await res.json({data: req.data});

Came across this issue in my Node Api call when I missed to use await keyword in a async method in front of call returning Promise. I solved it by adding await keyword.

I was also getting the same error, in my case it was just because of not using await with Users.findById() which returns promise, so response.status().send()/response.send() was getting called before promise is settled (fulfilled or rejected)
Code Snippet
app.get(`${ROUTES.USERS}/:id`, async (request, response) => {
const _id = request.params.id;
try {
// was getting error when not used await
const user = await User.findById(_id);
if (!user) {
response.status(HTTP_STATUS_CODES.NOT_FOUND).send('no user found');
} else {
response.send(user);
}
} catch (e) {
response
.status(HTTP_STATUS_CODES.INTERNAL_SERVER_ERROR)
.send('Something went wrong, try again after some time.');
}
});

For mongodb
so if you are getting errors while fetching data from MongoDB then the problem is async
previously
app.get('/users',(req,res,next)=>{
const user=chatUser.find({});
if(!user){
res.status(404).send({message:"there are no users"});
}
if(user){
res.json(user);
}
})
After
app.get('/users',async(req,res,next)=>{
const user=await chatUser.find({});
if(!user){
res.status(404).send({message:"there are no users"});
}
if(user){
res.json(user);
}
})

I came across this issue when not using async/await on a asynchronous function (api call). Hence adding them / using the promise handlers properly cleared the error.

This error message "TypeError: Converting circular structure to JSON" typically occurs when you try to stringify an object that contains circular references using JSON.stringify().
A circular reference occurs when an object references itself in some way. For example, consider the following code:
const obj = { foo: {} };
obj.foo.obj = obj;
In this example, obj contains a circular reference because the foo property of obj contains a reference to obj itself.
When you try to stringify an object like this using JSON.stringify(), it will fail with the error message "TypeError: Converting circular structure to JSON".
To solve this issue, you can use a third-party library like flatted or circular-json, which are specifically designed to handle circular references in JavaScript objects. Here's an example using flatted:
const flatted = require('flatted');
const obj = { foo: {} };
obj.foo.obj = obj;
const str = flatted.stringify(obj);
console.log(str);
In this example, we use flatted.stringify() instead of JSON.stringify(), and it successfully converts the object to a string without throwing an error.
Alternatively, you can modify your object to remove the circular reference before trying to stringify it. For example:
const obj = { foo: {} };
obj.foo.bar = 'baz';
// add circular reference
obj.foo.obj = obj;
// remove circular reference
obj.foo.obj = undefined;
const str = JSON.stringify(obj);
console.log(str);
In this example, we add the circular reference and then remove it before trying to stringify the object. This approach works well if you don't need to preserve the circular reference in the stringified object.

I had a similar issue:-
const SampleFunction = async (resp,action) => {
try{
if(resp?.length > 0) {
let tempPolicy = JSON.parse(JSON.stringify(resp[0]));
do something
}
}catch(error){
console.error("consoleLogs.Utilities.XXX.YYY", error);
throw error;
}
.
.
I put await before JSON.parse(JSON.stringify(resp[0])).
This was required in my case as otherwise object was read only.
Both Object.create(resp[0]) and {...resp[0]} didn't suffice my need.

If an object has a different type of property like mentioned in the above image, JSON.stringify() will through an error.

Try this as well
console.log(JSON.parse(JSON.stringify(req.body)));

TypeError: Converting circular structure to JSON in nodejs:
This error can be seen on Arangodb when using it with Node.js, because storage is missing in your database. If the archive is created under your database, check in the Aurangobi web interface.

nodejs prepending to a file

For Node.js, what is the best way to prepend to a file in a way SIMILAR to
fs.appendFile(path.join(__dirname, 'app.log'), 'appendme', 'utf8')
Personally, the best way really revolves around a asynchronous solution to create a log where I can basically push onto the file from the top.

This solution isn't mine and I don't know where it's from but it works.
const data = fs.readFileSync('message.txt')
const fd = fs.openSync('message.txt', 'w+')
const insert = Buffer.from("text to prepend \n")
fs.writeSync(fd, insert, 0, insert.length, 0)
fs.writeSync(fd, data, 0, data.length, insert.length)
fs.close(fd, (err) => {
if (err) throw err;
});

It is impossible to add to a beginning of a file. See this question for the similar problem in C or this question for the similar problem in C#.
I suggest you do your logging in the conventional way (that is, log to the end of file).
Otherwise, there is no way around reading the file, adding the text to the start and writing it back to the file which can get really costly really fast.

It seems it is indeed possible with https://www.npmjs.com/package/prepend-file

Here is an example of how to prepend text to a file using gulp and a custom built function.
var through = require('through2');
gulp.src('somefile.js')
.pipe(insert('text to prepend with'))
.pipe(gulp.dest('Destination/Path/'))
function insert(text) {
function prefixStream(prefixText) {
var stream = through();
stream.write(prefixText);
return stream;
}
let prefixText = new Buffer(text + "\n\n"); // allocate ahead of time
// creating a stream through which each file will pass
var stream = through.obj(function (file, enc, cb) {
//console.log(file.contents.toString());
if (file.isBuffer()) {
file.contents = new Buffer(prefixText.toString() + file.contents.toString());
}
if (file.isStream()) {
throw new Error('stream files are not supported for insertion, they must be buffered');
}
// make sure the file goes through the next gulp plugin
this.push(file);
// tell the stream engine that we are done with this file
cb();
});
// returning the file stream
return stream;
}
Sources: [cole_gentry_github_dealingWithStreams][1]

Its possible by using the prepend-file node module. Do the following:
npm i prepend-file -S
import prepend-file module in your respective code.
Example:
let firstFile = 'first.txt';
let secondFile = 'second.txt';
prependFile(firstFile, secondFile, () => {
console.log('file prepend successfully');
})