Long story short, I have in my MongoDB database a collection of posts and with node.js, express and mongoose, I'm trying to find all the documents using the $where method.
So I try this and it works perfectly, all the results, if their name includes Jo get returned.
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: function () {
return this.name.includes("Jo");
},
}).then((data) => res.send(data));
});
But if I do something like this, it throws me an error
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: function () {
return this.name.includes(nameSearch);
},
}).then((data) => res.send(data));
});
Also, if I do this, it says that $where requires a string or a function.
function runThis(post) {
return post.name.includes("Jo");
}
app.get("/posts", (req, res) => {
Post.find({
$where: runThis(this),
}).then((data) => res.send(data));
});
Even weirder, I think, is that if I change the function inside the $where to an arrow function, no matter what I put in the function, it will return all the results
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: () => {
return this.name.includes("Jo");
},
}).then((data) => res.send(data));
});
Caveat: I don't use Mongoose or MongoDb.
But the documentation says the function you pass for $where isn't executed locally, it's executed on the Mongoose server. So it doesn't have access to the variable.
But a search (1, 2) suggests that the usual way to find a substring within a field is to use $regex. If so, and you have a user-entered string (which might contain characters that have special meaning to $regex), you'll want to escape them. So for instance:
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
name: {
$regex: new RegExp(escapeRegex(nameSearch)),
}),
}).then((data) => res.send(data));
});
...where escapeRegex is from that linked question's answers. (Apparently JavaScript regular expression objects are supported, as well as strings using PCRE syntax instead.)
If for some reason you can't do that, you can also pass a string for $where, so you could create the string dynamically:
app.get("/posts", (req, res) => {
const nameSearch = "Jo";
Post.find({
$where: `this.name.includes(${JSON.stringify(nameSearch)})`,
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
}).then((data) => res.send(data));
});
Normally, writing JavaScript code in strings is a bad idea (even though Mongoose has to convert whatever function you give it to a string to send it to the server anyway), but if you need to do this kind of substitution...
Few things from the documentation
$where docs
About scope
Starting in MongoDB 4.4,
$where
no longer supports the deprecated BSON type JavaScript code with scope (BSON type 15)
You should avoid using $where
$where
evaluates JavaScript and cannot take advantage of indexes. Therefore, query performance improves when you express your query using the standard MongoDB operators (e.g.,
$gt
,
$in
).
In general, you should use
$where
only when you cannot express your query using another operator. If you must use
$where
, try to include at least one other standard query operator to filter the result set. Using
$where
alone requires a collection scan.
About variables
I have no idea, but you can try using $let
Related
Hello Stackoverflow users,
Many peoples like me searched for how to pass extra arguments to a callback function. The questions have similar titles but actually they have different challenges and many ways to solve. Plus, it is always a pleasure to share practices to be more experienced.
Recently, I faced a pretty simple challenge in my node js project. One of the APIs I communicate with has an SDK that works synchronically. And I used to pass callback functions every time (which is annoying when you have requests depending on each other and some data needs to transfer within the app layers).
Imagine a plan payment flow that goes like this, a client sends a request to the server including the selected plan and his ID. When the server API layer receives the request data, it passes it to a third-party service function ( .create(...) ). The third-party service function receives a callback with 2 parameters function(err, plan_document). And then, the callback is supposed to apply the selected plan logic on the client by the ID in the request.
** We need to pass the client's and the plan's data to the callback function to apply the logic. The third-party service provides to the callback a plan_document parameter and we still need to somehow pass the client id from the API layer to the service.
The code will look like this.
const create_plan_agreement = (req, res) => {
// some code
var client_id = req.auth.client_id;
third_party.plan_agreement.create({}, update_plan_agreement);
};
const update_plan_agreement = (err, plan_document, client_id) => {
/*
The third-party `third_party.plan_agreement.create` function passes the first
two parameters and somehow we need to add the client_id
*/
console.log('client plan activated');
active_client_plan(plan_document, client_id);
};
------------------ EDIT ------------------
I wonder what if the flow was longer and I need the client id farther than the update function like this.
const create_plan_agreement = (req, res) => {
// some code
var client_id = req.auth.client_id;
third_party.plan_agreement.create({}, update_plan_agreement);
};
const update_plan_agreement = (err, plan_document) => {
console.log('plan activated, send notification to the client');
third_party.plan_agreement.update(plan_document, send_agreement_notification);
};
const send_agreement_notification = (err, plan_document) => {
console.log('client plan activated');
active_client_plan(plan_document, this.client_id);
};
What should I do in this case? Should I keep repeating the.bind({'client_id': client_id}) function until the last step in the flow?
If you want to support older people, you can easily bind using a containing callback, like this:
const create_plan_agreement = (req, res) => {
// some code
var client_id = req.auth.client_id;
third_party.plan_agreement.create({}, function(params, from, create) {
update_plan_agreement(params, from, create, client_id)
});
};
const update_plan_agreement = (err, plan_document, client_id) => {
/*
The third-party `third_party.plan_agreement.create` function passes the first
two parameters and somehow we need to add the client_id
*/
console.log('client plan activated');
active_client_plan(plan_document, client_id);
};
The traditional way is to use a closure. Define the functions inside the parent's scope so that they can access the client_id as an enclosed variable (kind of like global variables):
const create_plan_agreement = (req, res) => {
// some code
var client_id = req.auth.client_id;
const update_plan_agreement = (err, plan_document) => {
console.log('plan activated, send notification to the client');
third_party.plan_agreement.update(plan_document, send_agreement_notification);
};
const send_agreement_notification = (err, plan_document) => {
console.log('client plan activated');
// Note: this function can access client_id
// because it is in scope
active_client_plan(plan_document, client_id);
};
third_party.plan_agreement.create({}, update_plan_agreement);
};
Closures are to scopes what objects are to classes. A closure is an instance of a scope. So unlike regular global variable, each call to create_plan_agreement() will create its own closure with its own copy of client_id.
With modern javascript it is often easier to handle this with Promises. Convert legacy functions to return a Promise and then you can use async/await:
const create_plan_agreement = async (req, res) => {
// some code
var client_id = req.auth.client_id;
try {
var plan_document = await plan_agreement_create({});
var updated_plan_document = await update_plan_agreement(plan_document);
send_agreement_notification(updated_plan_document, client_id);
}
catch (err) {
// handle errors here.
}
};
const plan_agreement_create = (arg) {
return new Promise ((ok, fail) => {
third_party.plan_agreement.create({}, (err, result) => {
if (err) {
return fail(err);
}
ok(result);
});
})
}
const update_plan_agreement = (plan_document) => {
return new Promise ((ok, fail) => {
third_party.plan_agreement.update(plan_document, (err, result) => {
if (err) return fail(err);
ok(result);
});
});
};
const send_agreement_notification = (plan_document, client_id) => {
active_client_plan(plan_document, client_id);
};
Or even without async/await Promises still make callbacks easier to use:
const create_plan_agreement = async (req, res) => {
// some code
var client_id = req.auth.client_id;
plan_agreement_create({})
.then(doc => update_plan_agreement(doc));
.then(doc => {
send_agreement_notification(doc, client_id)
})
.catch(err => {
// handle errors here.
});
};
I followed the youtube Node & Express project tutorial and Im confused facing these code:
This is in async js file:
const asyncHWrapper = (fn) => {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
};
module.exports = asyncHWrapper;
And this is the usage:
const Task = require("../models/taskModel");
const asyncWrapper = require("../middleware/async");
const { createCustomError } = require("../errors/customErrors");
const getAllTasks = asyncWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
});
Im just confused about these questions:
Does is necessary to return an arrow function in the asyncWrapper? Why doesn't just call the function?
Where do the params (req,res) in the asyncWrapper function come from? In the "fn" function declaration?
Why should I write two pairs of async and await in the wrapper and when I call it?
Thanks a lot!
The youtube tutorial link :Node.js / Express Course - Build 4 Projects
Let me analyze the code first, and then I'll answer your question.
So here is your wrapper
const asyncHWrapper = (fn) => {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
};
and here is how it is used
const getAllTasks = asyncWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
});
the asyncWrapper accept an fn param, in this case, it is this function:
async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
}
after asyncHWrapper is called with the above function, it will return another function, in this case, the return function is assigned as the name getAllTasks.
Now for your question:
Does is necessary to return an arrow function in the asyncWrapper? Why doesn't just call the function?
Well basically you can write it like this
const asyncHWrapper = async (fn, req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
And call it like this
await asyncHWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
}, req, res, next)
But in this case, it's just a normal function with callback, it's not a wrapper, and its name shouldn't be asyncHWrapper.
Where do the params (req,res) in the asyncWrapper function come from? In the "fn" function declaration?
No, it comes from getAllTasks, your fn is just consuming two values (req, res), and the next param will be used for error handling. So when you call getAllTask, you must pass in three params like this getAllTasks(req, res, next)
Why should I write two pairs of async and await in the wrapper and when I call it?
I'm not sure what you meant by "two pairs of async and await". I assume you're referring to await when calling getAllTasks and await when calling fn?
That's just because both of them are async functions.
I hope that this answer can help you think about the concept of a "wrapper". Let's say we have a function divide:
const divide = (a,b) => a/b;
You can use this in normal code quite easily:
x = divide(10,5); // sets x to 2.
But you may decide that you care about the possibility of errors. In this case, division by zero is a possibility. You could certainly include some error handling code where you define the divide function. But you could also choose to "wrap" divide with an error handler. This way, we can keep the error handling aspects away from the main division logic. We would like to be able to define a safeDivide function like this:
const safeDivide = catchErrors(divide);
In the same way that divide is a function that takes two arguments, safeDivide also has to be a function that takes two arguments. So the catchErrors wrapper will have to return a function. We will start with something like this:
const catchErrors = (fn) => {
return (p,q) => fn(p,q);
}
If you pass a function fn to catchErrors, it will return a function. That returned function takes two arguments p and q, and returns fn(p,q). So far, this doesn't really achieve anything (except limiting the number of arguments). But now, we can add a try/catch block. I'll do it in a couple of steps, because the notation can be confusing.
The first step is to put an explicit return inside the inner arrow function.
const catchErrors = (fn) => {
return (p,q) => {
return fn(p,q);
}
}
This is technically the same code - it just looks slightly different. Now we add the try/catch.
const catchErrors = (fn) => {
return (p,q) => {
try {
return fn(p,q);
} catch (e) {
console.log("Error occurred. Continuing with null result.");
return null;
}
}
}
So now, the function returned from catchErrors will do the same as the original function, except when an exception is thrown, in which case it will return null. (I'm not saying that this is the best general way to handle exceptions, but it's useful as an example, and it's related to the original question.)
So now look again at where we use this catchErrors wrapper function.
const safeDivide = catchErrors(divide);
When you call the wrapper catchErrors with function divide, it doesn't actually do any dividing - it doesn't yet have any numbers to divide. Instead, it builds up a new function that, whenever it is called, would do the dividing, and catch any exception that arises. I hope that answers your first question.
Your second question is where req and res come from. They are names given to arguments that will be passed to the function. They can be passed to the wrapped function (along with a 3rd argument next), and they will also be passed to the inner (nameless) function which includes calls to Task.find and res.status(200). Those arguments will be provided by the Express (or other) web framework.
I will leave your 3rd question, on the async/await aspects of the wrapper, for another answer.
I am trying to display the number of documents in my MongoDB database whenever the user retrieves the homepage of my web application. The following diagram shows how I wanted to implement this: https://liveuml.com/view/5db6af5e663178088afee61e
Here is the relevant code snippet for the Router
app.route('/')
.get(
(req, res, next) => {
res.locals.countOfWords = countAllWords();
next();
},
(req, res, next) => {
renderIndex(req, res);
}
);
And the relevant code snippet for the Controller
function countAllWords() {
myModel.estimatedDocumentCount({}, (err, result) => {
return result; // returns an object instead of an integer
});
}
function renderIndex(req, res) {
res.render('index', {
'countOfWords' : res.locals.countOfWords
});
}
However, the result that the Controller returns is a Query object and not an integer. So, I am seeing There are [object Object] documents in your database on the web page instead of something like There are 12 documents in your database.
What makes it even more confusing to me : When I replace the return result statement with console.log(result), I see the expected number in the console.
function countAllWords() {
myModel.estimatedDocumentCount({}, (err, result) => {
console.log(result); // displays the number as expected
});
}
My question is, how can ensure that I pass the number back to the Router instead of an object so that it can be displayed on the web page ?
I am using the latest versions of NodeJS, ExpressJS and Mongoose.
Many thanks for your help.
simply use async await and count() in mongoose:
async countAllWords()=>{
let countOfmymodel = await userModel.count({})
}
Im creating a live search box on express and it shows 2 errors.
(1) TypeError (2) Unhandled Promise rejection
CODE:
router. post('/search-phrasing', async (req, res) => {
const {
phrasing
} = req.body;
const
phrasingArray = phrasing.trim().split(' ');
phrasingArray.map(async (phrasing) => {
let suggestions = [];
await Response.find({
entities: {
$regex: new RegExp(phrasing)
}
}).sort({
phrasing: 'asc'
}).then((data) => {
if (data[0]) {
suggestions.push({
id: data[0]._id,
phrasing: data[0].phrasing
});
res.send(suggestions);
}
}).catch((err) => console.log(err));
});
});
Don't attempt to loop async functions this way as it is not required, and certainly don't send responses in a loop. Instead you should .map() the list of regular expressions to $in:
router.post('/search-phrasing', (req, res) => {
const { phrasing } = req.body;
if (phrasing == undefined || ( typeof(phrasing) != 'string' ) ) {
console.error("phrasing is required as a string");
return res.end(); // really should have better error handling
}
const phrasingArray = phrasing.trim().split(' ');
Response.find({ entities: { $in: phrasingArray.map(e => RegExp(e)) })
.sort('phrasing')
.select('phrasing')
.then(suggestions => res.send(suggestions))
.catch(err => console.error(err));
})
The $in operator accepts an array of arguments to match. It also happens to accept regular expressions as those arguments. It's basically shorthand for the $or operator but always applying to the one field.
Attempting to do this otherwise is executing multiple statements with the database, awaiting various promises and attempting to construct a single response from all of that. It's simply not necessary when there are query expressions which already handle this.
Also check your input types. Don't blindly presume you supplied the required data to the POST body. Check for it being present as is shown here, otherwise you get exceptions
This is how I am currently getting data from mongodb:
users.get(base_URL, (req, res) => {
UserModel.find({}, (err, docs) => {
res.render("Users/index", {
title: "All Users here",
user_list: docs
});
});
});
Now, as you can see this is an express application. What I would like, is to simple call a function so that I can get the value from the docs variable inside the mongodb model callback. How do I do this, ideally, I want to see something like this:
users.get(base_URL, (req, res) => {
res.render('<some_jade_file_here>', {
title: "Yes, got it right",
user_list: getAllUsers();
});
});
Ideally, I just want to call a function. How can I do this, since having to put render inside of a mongodb call is a problem, since you may want to query a bunch of things from the database, and it might not even be just one database. I'm struggling a little since I'm not all that used to callbacks.
Any help would be deeply appreciated. If you're wondering about this syntax () => {}, thats just an anonymous function in typescript.
You can't do it without callbacks, but you can use an async flow control library like async to help manage the nest of callbacks. In this case you probably want to use async.parallel.
Using that you can do something like:
users.get(base_URL, (req, res) => {
var data = {
title: "Yes, got it right"
};
async.parallel([
(callback) => {
UserModel.find({}, (err, docs) {
data.user_list = docs;
callback(err);
});
},
(callback) => {
// Other query that populates another field in data
}
], (err, results) => {
// Called after all parallel functions have called their callback
res.render('<some_jade_file_here>', data);
});
});