Background
I have Mongoose Schema about Surveys, that needs to check if the Survey belongs to a set of countries that is in another collection.
Code
To check this, I have a surveySchema, a countrySchema, and a file where I create the models and connect to the DB.
To perform the check that a survey belongs to a valid country, I am using Mongoose async validators in surveySchema like the following:
surveySchema.js:
"use strict";
const mongoose = require("mongoose");
const surveySchema = {
subject: { type: String, required: true },
country: {
type: String,
validate: {
validator: {
isAsync: true,
validator: async function(val, callback) {
const {
Country
} = require("./models.js").getModels();
const countriesNum = await Country.find({"isoCodes.alpha2": val}).count();
callback(countriesNum === 1);
}
},
message: "The country {VALUE} is not available in the DB at the moment."
}
}
};
module.exports = new mongoose.Schema(surveySchema);
module.exports.surveySchema = surveySchema;
countrySchema.js:
"use strict";
const mongoose = require("mongoose");
const countrySchema = {
name: { type: String, required: true },
isoCodes:{
alpha2: { type: String, required: true }
}
}
};
module.exports = new mongoose.Schema(countrySchema);
module.exports.countrySchema = countrySchema;
models.js:
"use strict";
const mongoose = require("mongoose");
const fs = require("fs");
const DB_CONFIG = "./config/dbConfig.json";
/**
* Module responsible for initializing the Models. Should be a Singleton.
*/
module.exports = (function() {
let models;
const initialize = () => {
//Connect to DB
const {
dbConnectionURL
} = JSON.parse(fs.readFileSync(DB_CONFIG, "utf8"));
mongoose.connect(dbConnectionURL);
mongoose.Promise = global.Promise;
//Build Models Object
models = {
Country: mongoose.model('Country', require("./countrySchema.js")),
Survey: mongoose.model('Survey', require("./surveySchema.js"))
};
};
const getModels = () => {
if (models === undefined)
initialize();
return models;
};
return Object.freeze({
getModels
});
}());
The idea here is that I am using the models.js file in other places as well. Because this file is also responsible for connecting to the DB, I decided to make it a Singleton. This way, I should only connect once, and all further requests will always return the same Models, which would be ideal.
Problem
The problem here is that I have a circular dependency that results in:
RangeError: Maximum call stack size exceeded at exports.isMongooseObject (/home/ubuntu/workspace/server/node_modules/mongoose/lib/utils.js:537:12)
...
The flow of code leading to this error is:
Code runs getModels()`
getModels() checks that models is undefined and runs initialize()
initialize() tries to create the models.
When creating the Survey model Survey: mongoose.model('Survey', require("./surveySchema.js")) it runs into the validator function, which again requires models.js
Infinite loop begins
Questions
Is there any other way to check if a Survey's country is part of the county's collection without making a async validation?
How can I structure/change my code so this doesn't happen?
As said in the comments, I think you are a bit confused about how you are using your models.js module. I think this is what is happening:
You are exporting a single function from models.js:
models.js
module.exports = function() { ... };
Therefore, when you require it, you just get that single function:
surveySchema.js
const models = require("./models.js");
models is now a function. Which means every time you call it, you run through the code in models.js and create a new variable let models;, and also new functions initialize() and getModels().
You could move the let models out of the anonymous function into the global scope which would probably fix it, but for my money you only want to run the anonymous function in models.js once, so I would invoke it immediately and set the exports of the module to its result:
models.js
module.exports = (function() {
// codez here
return Object.freeze({
getModels
});
})(); // immediately invoke the function.
Use it:
// models is now the frozen object returned
const { Survey } = models.getModels();
As for options to validation, there's no reason why you can't add your own middleware validation code if normal async validation doesn't do it for you using serial or parallel mechanisms as described in the docs.
Update after comments
The second problem as you point out is that during first execution of getModels() -> initialize() you call require('./surveySchema.js'), but this calls getModels() which is still in the process of being called and hasn't yet initialized models, so initialize() is re-entered.
I think what you're trying to achieve is fine (survey schema depends on customer model), because you can still draw an object graph without any circular dependencies, and it's just the way you've implemented it that you've ended up with one. The simplest way to deal with this I think is actually to keep the circular reference, but defer the point at which you call getModels() in surveySchema.js:
"use strict";
const mongoose = require("mongoose");
const models = require("./models.js");
const surveySchema = {
subject: { type: String, required: true },
country: {
type: String,
validate: {
validator: {
isAsync: true,
validator: async function(val, callback) {
// !! moved from the global scope to here, where we actually use it
const {
Country
} = models.getModels();
const countries = await Country.find({"isoCodes.alpha2": val});
callback(countries.length === 1);
}
},
message: "The country {VALUE} is not available in the DB at the moment."
}
}
};
module.exports = new mongoose.Schema(surveySchema);
module.exports.surveySchema = surveySchema;
A neater and probably more extensible way of approaching it, though, might be to separate out the connection code from the models code, since it's a different concept altogether.
Update #2 after more comments
The infinite stack you're seeing is because you have not used the API correctly. You have:
const surveySchema = {
country: {
validate: {
validator: {
isAsync: true,
validator: async function(val, callback) {...}
},
},
...
}
};
You should have:
const surveySchema = {
country: {
validate: {
isAsync: true,
validator: async function(val, callback) {...}
},
...
}
};
As per the docs.
Related
I am adding query pagination to my GCF query function and am writing unit tests for the pagination query, I am writing a mock function using Node, and the firestore-mock npm library. Note, I can't use Sinon and no third party modules or libraries are allowed aside from firestore-mock.
I am extending the firestore-mock library to add query pagination to our firestore query. Write now, I have 3 methods that extend Firestore-mock that I need to write. Two of which are completed.
orderBy
limit
startAfter
So far I have successfully written mocks methods for limit and orderBy but am stuck on how to implement startAfter
Ideally this startAfter function would have the same functionality as the firebase built in startAfter function.
The request I am using for the query pagination has the following structure:
const paginatedQuery = await f.getQueryPagination({
customerId: ['62005'],
startDate: '2021-11-04T00:00:00.000Z',
endDate: '2021-11-05T00:00:00.000Z',
next: '1657667147.865000000',
},
'wal-com', ['62005'],
logger
);
I have looked at the docs for the startAfter cursor query and in my request I want the startAfter method to set the cursor to be set to the document that starts with the timestamp that next was converted from.
So for the above example: next: '1657667147.865000000' would be equivelant to the next document having a eventDateTime value of
TimeStamp: {
_seconds: 1657667147
_nanoSeconds: 865000000
}
EDIT: Here is the code I have right now for startAfter but it doesn't work as intended
const FirebaseMock = require('firebase-mock');
const QueryMock = require('firestore-mock/mock_constructors/QueryMock');
QueryMock.prototype._startAfter = function(doc) {
const startAfter = doc;
const startAfterId = startAfter.id;
const startAfterData = startAfter.data();
const startFinder = function(doc) {
const docId = doc.id;
const docData = doc.data();
if (docId === startAfterId) {
return true;
}
if (docData === startAfterData) {
return true;
}
return false;
};
const buildStartFinder = function() {
return startFinder;
};
return buildStartFinder;
};
QueryMock.prototype.startAfter = function(value) {
// if value is passed as a parameter
// set the cursor for the query to the document with the timestamp equal to value
if (value) {
if (value.constructor.name === 'DocumentSnapshot') {
this._docs = this.firestore._startAfter(value, this._docs, this.id);
} else if (value.constructor.name === 'Timestamp') {
this._docs = this.firestore._startAfter(
new TimestampMock(value.seconds, value.nanoseconds),
this._docs,
this.id
);
} else {
throw new Error(
'startAfter() only accepts a DocumentSnapshot or a Timestamp'
);
}
}
};
module.exports = {
FirestoreMock
};
To improve our program and reduce code redundancy, we wish to create some inheritance inside the models..
Now take a typical User model, it has a name and password field as "baseclass" and several subclasses can improve upon this depending in the specific application's needs.
So a baseuser would look like:
module.exports = {
attributes: {
username: {
type: 'string',
required: true,
unique: true
},
password: {
type: 'string',
required: true,
},
},
beforeCreate: async function(user, cb) {
const hash = await bcrypt.hash(user.password, 10);
user.password = hash;
cb();
},
}
This bare class doesn't correspond to any database table in its own. Now in derived class from this, VerifyableUser (a model for users that must have verification links), there are a few extra fields, one which is a verify url.
Now to "extend" classes lodash' _.merge function is used, as explained in this question .
const BaseUser = require("../../BaseUser.js");
module.exports = _.merge({}, BaseUser, {
attributes: {
verify_key: {
type: 'string'
}
},
beforeCreate: async function(user, cb) {
user.verify_key = 'helloworld'; //crypto used to generate...
cb();
}
};
Now the problem should be obvious, the derived class' beforeCreate overwrites the original beforeCreate: in a normal OO environment this isn't a big problem either, as I could just call Base.beforeCreate() or something similar.
However can something be done using lodash' merge? Or should I use another way to extend objects? (Or do I really have to repeat myself and retype the beforeCreate?).
or something similar:
// VerifyableUser
async beforeCreate(user, cb) {
await BaseUser.beforeCreate(user, () => 0);
//...
}
You could also use _.mergeWith to check what each property being merged is and if it is a function just pick the object and not the source (in your case the source is BaseUser):
const BaseUser = require("../../BaseUser.js");
let obj = {
attributes: {
verify_key: {
type: 'string'
}
},
beforeCreate: async function(user, cb) {
user.verify_key = 'helloworld'; //crypto used to generate...
cb();
}
}
module.exports = _.mergeWith(
obj,
BaseUser,
(objValue, srcValue, key, object, source) => _.isFunction(objValue) ? objValue : _.merge(object[key], source[key])
)
Here is a test:
var data = {
food: "chicken",
foo: () => console.log('chicken!')
}
var source = {
prop1: 1,
prop2: 1,
foo: () => console.log('foo!')
}
var result = _.merge(data, source, (objValue, srcValue, key, object, source) => _.isFunction(objValue) ? objValue : _.merge(object[key], source[key]))
console.log(result)
result.foo()
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.10/lodash.min.js"></script>
What is the appropriate way to load a child model property of an Ampersand model when doing a fetch on the collection?
I have a Task object:
{
"id": 1
"projectId": 2 // child entity id
"name": "This task for project 1"
}
And I have created an Ampersand model:
var AmpModel = require('ampersand-model');
var Project = require('../project');
module.exports = AmpModel.extend({
props: {
id: ['number'],
projectId: ['number'],
name: ['string']
},
children: {
project: Project
}
});
And a collection:
var AmpCollection = require('ampersand-rest-collection');
var Task = require('./task');
module.exports = AmpCollection.extend({
model: Task,
url: '/api/task'
});
My API is very simple, as the app only stores input in memory:
var _ = require('lodash');
var store = require('../data'); // this is a file with some init data
function get(id) {
return _.findWhere(store.tasks.data, { id: parseInt(id + '', 10) });
}
exports.list = function (req, res) {
res.send(store.tasks.data);
};
exports.add = function (req, res) {
var item = req.body;
item.id = store.tasks.id++;
store.tasks.data.push(item);
res.status(201).send(item);
};
exports.get = function (req, res) {
var found = get(req.params.id);
res.status(found ? 200 : 404);
res.send(found);
};
In Entity Framework I would have included the child entity when retrieving. The projectId field would map to the projects table and the framework would take care of it for me.
After reading the doco and some help in the JavaScript chat room, I realise I have been looking at this architecture the wrong way.
My API should read the tasks data source and use the retrieved project ID to load the project data. The project object would be assigned to the projects property on the task and the whole task object returned. Ampersand will then take over and assemble my model as expected.
What are different ways to insert a document(record) into MongoDB using Mongoose?
My current attempt:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var notificationsSchema = mongoose.Schema({
"datetime" : {
type: Date,
default: Date.now
},
"ownerId":{
type:String
},
"customerId" : {
type:String
},
"title" : {
type:String
},
"message" : {
type:String
}
});
var notifications = module.exports = mongoose.model('notifications', notificationsSchema);
module.exports.saveNotification = function(notificationObj, callback){
//notifications.insert(notificationObj); won't work
//notifications.save(notificationObj); won't work
notifications.create(notificationObj); //work but created duplicated document
}
Any idea why insert and save doesn't work in my case? I tried create, it inserted 2 document instead of 1. That's strange.
The .save() is an instance method of the model, while the .create() is called directly from the Model as a method call, being static in nature, and takes the object as a first parameter.
var mongoose = require('mongoose');
var notificationSchema = mongoose.Schema({
"datetime" : {
type: Date,
default: Date.now
},
"ownerId":{
type:String
},
"customerId" : {
type:String
},
"title" : {
type:String
},
"message" : {
type:String
}
});
var Notification = mongoose.model('Notification', notificationsSchema);
function saveNotification1(data) {
var notification = new Notification(data);
notification.save(function (err) {
if (err) return handleError(err);
// saved!
})
}
function saveNotification2(data) {
Notification.create(data, function (err, small) {
if (err) return handleError(err);
// saved!
})
}
Export whatever functions you would want outside.
More at the Mongoose Docs, or consider reading the reference of the Model prototype in Mongoose.
You can either use save() or create().
save() can only be used on a new document of the model while create() can be used on the model. Below, I have given a simple example.
Tour Model
const mongoose = require("mongoose");
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, "A tour must have a name"],
unique: true,
},
rating: {
type: Number,
default:3.0,
},
price: {
type: Number,
required: [true, "A tour must have a price"],
},
});
const Tour = mongoose.model("Tour", tourSchema);
module.exports = Tour;
Tour Controller
const Tour = require('../models/tourModel');
exports.createTour = async (req, res) => {
// method 1
const newTour = await Tour.create(req.body);
// method 2
const newTour = new Tour(req.body);
await newTour.save();
}
Make sure to use either Method 1 or Method 2.
I'm quoting Mongoose's Constructing Documents documentation:
const Tank = mongoose.model('Tank', yourSchema);
const small = new Tank({ size: 'small' });
small.save(function (err) {
if (err) return handleError(err);
// saved!
});
// or
Tank.create({ size: 'small' }, function (err, small) {
if (err) return handleError(err);
// saved!
});
// or, for inserting large batches of documents
Tank.insertMany([{ size: 'small' }], function(err) {
});
TLDR: Use Create (save is expert-mode)
The main difference between using the create and save methods in Mongoose is that create is a convenience method that automatically calls new Model() and save() for you, while save is a method that is called on a Mongoose document instance.
When you call the create method on a Mongoose model, it creates a new instance of the model, sets the properties, and then saves the document to the database. This method is useful when you want to create a new document and insert it into the database in one step. This makes the creation an atomic transaction. Therefore, the save method leaves the potential to create inefficiencies/inconsistencies in your code.
On the other hand, the save method is called on an instance of a Mongoose document, after you have made changes to it. This method will validate the document and save the changes to the database.
Another difference is that create method can insert multiple documents at once, by passing an array of documents as parameter, while save is intended to be used on a single document.
So, if you want to create a new instance of a model and save it to the database in one step, you can use the create method. If you have an existing instance of a model that you want to save to the database, you should use the save method.
Also, if you have any validation or pre-save hook in your content schema, this will be triggered when using the create method.
I'm in the case of a function being able to search by different fields depending on the situation.
It returns the same dataset, it just searches by different fields: either userId or tagId. Therefore, in my code I have something like this:
var findByMethod;
if (searchBy === 'userId') {
findByMethod = UserArticleModel.findByUser;
}
else {
findByMethod = UserArticleModel.findByTag;
}
findByMethod(idToSearch, function (err, articles) {…});
findByUser and findByTag are static methods defined in the UserArticleModel.js
UserArticleModel.js
var mongoose = require('mongoose');
var userArticleSchema = new mongoose.Schema({
…
}
});
userArticleSchema.statics.findByUser = function (userId, callback) {
this.find({userId: userId}, function () {…});
};
userArticleSchema.statics.findByTag = function (tagId, callback) {…};
module.exports = mongoose.model('UserArticle', userArticleSchema);
Back in my controller, when I do:
UserArticleModel.findByTag(idToSearch, function (err, articles) {…});
All is well and things go right. But when I dynamically call the method via my variable:
findByMethod(idToSearch, function (err, articles) {…});
Things go wrong as node returns an error:
DOMAINE ERROR CAUGHT: TypeError: Object #<Object> has no method 'find'
I suspect this not to be be bound to the correct scope but I don't really understand why as findByMethod === UserArticleModel.findByUser // true
I think you are making this more involved that it need be. Though it is an easy trap to fall into by "too literally" following documented API examples and thinking essentially "this is how I need to hardcode this, because the docs say this is how you do it".
JavaScript objects are, well "Objects", and therefore just assigning "named" static methods which are really only object properties is just a basic process of "looping" the defined "schema paths" from that already defined "Object" and setting up the properties for the "findByFieldName" methods you want.
It's just "assigning named properties" and nothing more obscure or complex or even as "terse" than that.
If that "sounds like a mouthful" then the actual process of iterating object properties and "setting other properties" related to that within an overall object structure is not really as hard as you might think.
As a brief example:
var async = require('async'),
pascal = require('to-pascal-case'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
var testSchema = new Schema({
fieldA: String,
fieldB: Number
});
function setStatics(schema) {
Object.keys(schema.paths).filter(function(key) {
return key != '_id';
}).forEach(function(key) {
schema.statics['findBy' + pascal(key)] = function(arg,callback) {
var query = {};
query[key] = arg;
this.findOne(query,callback);
};
});
};
// Set up findByFieldName other than _id
setStatics(testSchema);
var Test = mongoose.model( 'Test', testSchema, "test" );
mongoose.connect('mongodb://localhost/test');
async.series(
[
function(callback) {
Test.remove({},callback);
},
function(callback) {
Test.create([
{ "fieldA": "a", "fieldB": 1 },
{ "fieldA": "b", "fieldB": 2 }
],callback);
},
function(callback) {
Test.findByFieldA("a",function(err,doc) {
console.log(doc);
callback(err);
});
},
function(callback) {
Test.findByFieldB(2,function(err,doc) {
console.log(doc);
callback(err);
});
}
],
function(err) {
if (err) throw err;
mongoose.disconnect();
}
);
Which proves that they work by "testing them" with the output:
{ _id: 55f2ae1b7d8315f40b1a2b77, fieldA: 'a', fieldB: 1, __v: 0 }
{ _id: 55f2ae1b7d8315f40b1a2b78, fieldA: 'b', fieldB: 2, __v: 0 }
And that is all there is to it.
Of course for fields like "Arrays" you want to get a little more involved, but this is the basic premise as a listing you can try out yourself ( or selves for the community in general ).
I could also note that there are already a few things out there such as Bluebird via it's own .promisifyAll() call which interacts with objects to set new up "named methods" on the object in a similar way. Or at least it should be similar in principle, as I have not actually looked at that code.