We Keep Coding

Defined variable is undefined in mongoDB $where's function

Long story short, I have in my MongoDB database a collection of posts and with node.js, express and mongoose, I'm trying to find all the documents using the $where method.
So I try this and it works perfectly, all the results, if their name includes Jo get returned.
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: function () {
return this.name.includes("Jo");
},
}).then((data) => res.send(data));
});
But if I do something like this, it throws me an error
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: function () {
return this.name.includes(nameSearch);
},
}).then((data) => res.send(data));
});
Also, if I do this, it says that $where requires a string or a function.
function runThis(post) {
return post.name.includes("Jo");
}
app.get("/posts", (req, res) => {
Post.find({
$where: runThis(this),
}).then((data) => res.send(data));
});
Even weirder, I think, is that if I change the function inside the $where to an arrow function, no matter what I put in the function, it will return all the results
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: () => {
return this.name.includes("Jo");
},
}).then((data) => res.send(data));
});

Caveat: I don't use Mongoose or MongoDb.
But the documentation says the function you pass for $where isn't executed locally, it's executed on the Mongoose server. So it doesn't have access to the variable.
But a search (1, 2) suggests that the usual way to find a substring within a field is to use $regex. If so, and you have a user-entered string (which might contain characters that have special meaning to $regex), you'll want to escape them. So for instance:
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
name: {
$regex: new RegExp(escapeRegex(nameSearch)),
}),
}).then((data) => res.send(data));
});
...where escapeRegex is from that linked question's answers. (Apparently JavaScript regular expression objects are supported, as well as strings using PCRE syntax instead.)
If for some reason you can't do that, you can also pass a string for $where, so you could create the string dynamically:
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: `this.name.includes(${JSON.stringify(nameSearch)})`,
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
}).then((data) => res.send(data));
});
Normally, writing JavaScript code in strings is a bad idea (even though Mongoose has to convert whatever function you give it to a string to send it to the server anyway), but if you need to do this kind of substitution...

Few things from the documentation
$where docs
About scope
Starting in MongoDB 4.4,
$where
no longer supports the deprecated BSON type JavaScript code with scope (BSON type 15)
You should avoid using $where
$where
evaluates JavaScript and cannot take advantage of indexes. Therefore, query performance improves when you express your query using the standard MongoDB operators (e.g.,
$gt
,
$in
).
In general, you should use
$where
only when you cannot express your query using another operator. If you must use
$where
, try to include at least one other standard query operator to filter the result set. Using
$where
alone requires a collection scan.
About variables
I have no idea, but you can try using $let

Fetching data from mongo via express, build object, and send to React

I am currently stuck in asynchronous hell.
In my React, I have a page /menu, that would load data from my mongo instance via expressjs api.
In my database, called menu, i have collections which represent a meal-type eg "breakfast", "lunch" etc. In those collections, the documents for every item looks like this bread collection example:
{
_id: 2398jcs9dn2f9f,
name: "Ciabatta",
desc: "Italian bread",
imageURI: "image01.jpg",
reviews: []
}
This is my api that would be called when the page loads
exports.getAllFoods = (req, res, next) => {
const db = mongoose.connection
const allCollections = {}
try {
db.db.listCollections().toArray((err, collections) => {
collections.forEach((k) => {
allCollections[k.name] = []
})
Object.keys(allCollections).map(k => {
let Meal = mongoose.model(k, MealSchema)
meal = Meal.find((err, docs) => {
allCollections[k] = docs
console.log(allCollections)
})
})
res.send(allCollections)
})
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
The last output of the console.log(allCollections) produces this:
{ snacks:
[ { review: [],
tags: [],
_id: 5fcec3fc4bc5d81917c9c1fe,
name: 'Simosa',
description: 'Indian food',
imageURI: 'image02.jpg',
__v: 0 } ],
breads:
[ { review: [],
tags: [],
_id: 5fcec41a4bc5d81917c9c1ff,
name: 'Ciabatta',
description: 'Italian bread',
imageURI: 'image02.jpg',
__v: 0 } ],
}
This is exactly what I need, but I am stuck in figuring out how to send to React. What am I to do to send the above json? The res.send(allCollections) gives me this:
{
"snacks": [],
"breads": [],
"drinks": []
}
I understand why the above is being sent, but I dont know what I need to do to address it.
This is my React on page load
useEffect(() => {
axios
.get('http://localhost:8888/api/allFoods')
.then((res) => {
setMealTypes(res.data)
})
.catch((err) => [
console.log(err)
])
}, [])
Ultimately, I need the json outputted in console as I wanted to loop through that data and use the key as a title, and then list the values from the value array eg
<div>
<h2>Breads</h2>
<img src=image01.jpg/>
<h3>Ciabatta</h3>
<p>Italian bread</p>
...
</div>
...
I'd appreciate any help, and any docs I should read to help and improve my javascript understandings

I'd prefer to solve this using async/await and Promise.all, replacing most callbacks.
Because you're calling the DB when you're iterating through an array, you have the most annoying callback situation: how do you issue a bunch of async things and then get the results after? You'll need something else to ensure all callbacks are called before sending the results.
Async/await means we can declare a function is async, and await the results of an async operation. async/await is annoying in JS because it abstracts away callbacks and is actually creating a Promise underneath. Complicating things further, async/await doesn't solve issuing multiple async functions, so again we have to rely on this fancy Promise.all() function combined with map-ing the desired input array to async functions.
Original:
Object.keys(allCollections).map(k => {
let Meal = mongoose.model(k, MealSchema)
meal = Meal.find((err, docs) => {
allCollections[k] = docs
console.log(allCollections)
})
});
Suggested async/await:
await Promise.all(Object.keys(allCollections).map(async k => {
let Meal = mongoose.model(k, MealSchema)
let docs = await Meal.find();
allCollections[k] = docs;
console.log(allCollections);
}));
Another advantage is error handling. If any errors happen in the callback of the original example, they won't be caught in this try/catch block.
async/await handles errors like you'd expect, and errors will end up in the catch block.
...
// Now that we have awaited all async calls above, this should be executed _after_ the async calls instead of before them.
res.send(allCollections);
})
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
Technically Promise.all() returns an array of results, but we can ignore that since you're formatting an Object anyway.
There is plenty of room to optimize this further. I might write the whole function as something like:
exports.getAllFoods = async (req, res, next) => {
const db = mongoose.connection.db;
try {
let collections = await db.listCollections().toArray();
let allCollections = {};
collections.forEach((k) => {
allCollections[k.name] = [];
})
// For each collection key name, find docs from the database
// await completion of this block before proceeding to the next block
await Promise.all(Object.keys(allCollections).map(async k => {
let Meal = mongoose.model(k, MealSchema)
let docs = await Meal.find();
allCollections[k] = docs;
}));
// allCollections should be populated if no errors occurred
console.log(allCollections);
res.send(allCollections);
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
Completely untested.
You might find these links more helpful than my explanation:
https://javascript.info/async-await
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
https://medium.com/dailyjs/the-pitfalls-of-async-await-in-array-loops-cf9cf713bfeb

I hope this will help you : You need to first use the stringify method before sending the collections from the express api and then use JSON.parse on the React front end to restore the object.
PS: can you do a console.log(allCollections) one line above res.send(allCollections)?

You need to send it to the front-end in a JSON format.
replace res.send(allCollections) with res.json(allCollections)

Why this promise chain is stuck at one of then calls and catch returns empty error object?

I'm making a post request and it requires several things to happen. It's native JavaScript promise not any library. Initially used nested promise and it was working but code was not that good. So, I decided to go with Promise chain and I'm stuck. The post route always returns {success:false, err:{}}, which it should when something goes wrong. But the err object is empty object. Why is that? After some tests I found the problem is in the second then where I'm returning AvaiablexForX.findOne({isX:false});. Don't worry about variables names, for the sake idea I have changed the actual names.
router.post("/delevery_request",
passport.authenticate("jwt", {session:false}),
(req, res) => {
const requestInputFields = {};
const foundxProfile = {};
const xProfileId = "";
const newxRequestId = "";
requestInputFields.isAccepted = false;
XProfile.findOne({user:req.user.id})
.then(xProfile => {
foundxProfile= xProfile;
requestInputFields.xId = xProfile._id;
return AvaiablexForX.findOne({isX:false});
})
.then( avaiablexForX =>
{
// this does not reach here
console.log("available x for X", avaiablexForX);
requestInputFields.xToBeDonateId = avaiablexForX._id;
xProfileId = avaiablexForX.xProfileId;
return requestInputFields;
})
.then( result => new RequestxY(result).save()).
then( requestForxY => {
foundxProfile.requestedxDeleivery.unshift(requestForxY._id);
return foundxProfile.save();
}).
then( xProfile => res.json({success:true}))
.catch(err => {
// output in body of request: {success:false, err:{}}
res.status(404).json({success:false, err:err})
}
);
});

Probably the problem is you trying to set a new value for const:
foundxProfile= xProfile;
This cause error and broke the chain. Try to replace all const by let.

Short answer: as already pointed out members declared with const can't be reassigned.
Long answer: you would benefit from a better strategy for accessing previous promise results in a .then() chain
With ref to the linked topic, you are using the "inelegant and rather errorprone" mutable contextual state.
You might consider one of the other approaches :
Nesting (and) closures
Break the chain
Explicit pass-through
For example, Nesting (and) closures would give you something like this :
router.post('/delevery_request', passport.authenticate('jwt', { 'session': false }), (req, res) => {
XProfile.findOne({ 'user': req.user.id })
.then(xProfile => {
return AvaiablexForX.findOne({ 'isX': false })
.then(avaiablexForX => {
return new RequestxY({
'isAccepted': false,
'xId': xProfile._id,
'xToBeDonateId': avaiablexForX._id
}).save();
})
.then(requestForxY => {
xProfile.requestedxDeleivery.unshift(requestForxY._id);
return xProfile.save();
});
})
.then(() => res.json({ 'success': true }))
.catch(err => {
res.status(404).json({
'success': false,
'err': err
});
});
});
Due to closure, xProfile is available to the first and second nested .then().
What was requestInputFields is composed on the fly where it is used.
You lose the nice flat line up of then()s but gain by not needing a bunch of messy outer members.

How do I skip a callback when running .map in JavaScript/Node.js?

Kind of a sequel to this question, I need to accept multiple objects in a POST request and then for each object process it, save it, and then return the saved object to the frontend (so that the client can see which columns were successfully edited).
When I use .map, it does save to the database and I can confirm this. However, I have two problems:
It does not execute res.locals.retval.addData(dtoObject); correctly, and my returning payload has no data transfer objects inside of it.
My object validation cannot be done inside of the callback of map. I initially tried reduce, but that didn't work at all and just saved all the same values to each database object. How can I exclude invalid JSON objects while I'm mapping them?
var jsonObjects = req.body;
//for (var n in req.body) {
var promises = jsonObjects.map((jsonObject) => {
var transform = new Transform();
// VALIDATION OF jsonObject VARIABLE IS HERE
if (jsonObject.id == 0) {
var databaseObject = Database.getInstance().getModel(objectName).build(jsonObject);
transform.setNew(true);
transform.setJsonObject(jsonObject);
transform.setDatabaseObject(databaseObject);
transform.baseExtract()
.then(() => transform.extract())
.then(() => transform.clean())
.then(() => transform.getDatabaseObject().save())
.then(function(data) {
// PROCESSING DATA
}).catch((e) => {
// ERROR
});
} else {
var queryParameters = {
where: {id: jsonObject.id}
};
console.log("Query parameters: ");
console.log(queryParameters);
Database.getInstance().getModel(objectName).findOne(queryParameters).then((databaseObject) => {
transform.setJsonObject(jsonObject);
transform.setDatabaseObject(databaseObject);
})
.then(() => transform.baseExtract())
.then(() => transform.extract())
.then(() => transform.clean())
.then(() => transform.getDatabaseObject().save())
.then((data) => {
// PROCESSING DATA
}).catch((e) => {
// ERROR
});
}
});
Promise.all(promises)
.then((results) => {
return next();
}).catch((e) => {
throw e;
});
Here's the resulting payload:
{
"errors": [],
"warnings": [],
"data": []
}

As #KevinB said in the comments, you are missing the return calls inside of your arrow functions so the database saves are going through because they are part of the Promise chain, but pushes to the response are stalled waiting for the return, and then the Express.js call resolves before the Promises do. Add return Database.getInstance() and return transform.baseExtract() to your code to fix this.
Use Array.prototype.filter() to remove elements you want to ignore since you won't ever need to execute Promises on them, then call Array.prototype.map() on the resulting array. If you don't want to use the arrow functions, you can specify this as a parameter to filter and map:
jsonObjects.filter(function(jsonObject) {
}, this);
var promises = jsonObjects.map(function(jsonObject) {
}, this);

"Iterating" throw promises does not let to generate different ids

Reading some amazing tutorials about promises, I've discovered that, if I need to interate throw some promises, I can't use forEach or some other "traditional" iteration mechanisms, I have to use Q library for node, I've to "iterate" using Q.all.
I've written a simple example in Nodejs Express in order to be sure I've understood promises:
var form = [
{'name':'FORM_NAME_1.1',
'label2':'FORM_LABEL_1.2'
},
{'name':'FORM_NAME_2.1',
'label2':'FORM_LABEL_2.2'
}
];
var params = ['params1','params2'];
var meta = ['meta1','meta2'];
app.get('/', (req,res) => {
return Q.all([
form.map((currentValue,index,arr) => {
req.id = Math.random(); //Random ID to be used in the next promises
console.log(currentValue);
return Form.insert(currentValue,req);
}),
params.map((currentValue,index,arr) => {
console.log(req.id);
return Field.insert(currentValue,req.id);
}),
meta.map((currentValue,index,arr) => {
console.log(req.id);
return Meta.insert(currentValue,req.id);
})
])
.catch((err) => next(err))
.done(() => console.log('It\'s done'));
});
Form.insert code simply is a promise with a console.log call, the same for Field.insert and Meta.insert
var Form = {
insert: (param1,req) => {
var deferred = Q.defer();
console.log('Goes throw Form');
deferred.resolve();
return deferred.promise;
}
}
The problem is that seems to iterate right but the dynamicly generated id does not change along the promises, this is the console output:
Listening at port 3000...
{ name: 'FORM_NAME_1.1', label2: 'FORM_LABEL_1.2' }
Goes throw Form
{ name: 'FORM_NAME_2.1', label2: 'FORM_LABEL_2.2' }
Goes throw Form
0.3757301066790548
Goes throw Field
0.3757301066790548
Goes throw Field
0.3757301066790548
Goes throw Meta
0.3757301066790548
Goes throw Meta
It's done
Any ideas about what is going wrong? Thanks!!

the reason it is not working is because in first for loop, the req.id is set multiple times before other promises are started and and all of them end up using the last randomly generated value, change your code to:
app.get('/', (req,res) => {
let process = (currentValue,index,arr) => {
let reqCopy = {id: Math.random()}
for(let attr in req) // copy all the request attributes
if(attr && attr!='id')
reqCopy[attr] = req[attr]
return Q.all([
Form.insert(form[index],reqCopy),
Field.insert(params[index],reqCopy),
Meta.insert(meta[index],reqCopy)
])
}
return Q.all(form.map(process))
.catch(next)
.done(() => console.log('It\'s done'));
})
you would notice that I am copying all the attributes of req to clone reqCopy for I am not sure what attributes of req are required by the subsequent methods, but at the same time, single req.id would not work thanks to the async nature of code