Ensure data integrity with nodejs, restify and mongoose - javascript

I'm writing an analytics application that collects events and associates it with visitors.
My Visitor mongoose model as follows:
var visitorSchema = new Schema({
created_at: { type: Date, default: Date.now },
identifier: Number,
client_id: Number,
account_id: Number,
funnels: [String],
goals: [Goal],
events: [Event]
});
The api accept a mixed of visitor info and the event
{
"identifier": 11999762224,
"client_id": 1,
"account_id": 1,
"event": {
"context": "Home",
"action": "Click red button",
"value": ""
}
}
When restify receives a request it checks if the visitor exists, and if it exists the ap just push the event as follows:
server.post('/event', function (req, res, next) {
Visitor.findOne({
identifier: req.params.identifier,
client_id: req.params.client_id,
account_id: req.params.client_id
}, function(err, visitor) {
if(err) {
console.log(err);
res.send(500, visitor);
}
if(visitor) {
visitor.events.push(req.params.event);
visitor.save();
} else {
visitor = new Visitor({
identifier: req.params.cpf,
client_id: req.params.client_id,
account_id: req.params.client_id,
funnels: req.params.funnels,
events: req.params.event
});
visitor.save();
}
res.send(200, visitor);
});
});
By using this method, and I trigger several concurrent requests I get duplicated visitors instead of one visitor with multiple events.
How can I solve this issue? Whats the best approach?

Add a unique index on identifier in mongoose model. This way the second request will break with unique index violation. Just make sure you handle that error.
If you are using single page framework on the client side (angular, backbone etc) make sure you disable the button when you make an api call, and enable it on server response.

Related

Mongo findById() only works sometimes even when passed a valid ID

I am having a strange issue querying a Mongo DB collection. I am using findById() to get a single item that works sometimes and not others.
I have checked the id being passed to the server route and in all cases, they match perfectly with the targeted document in the collection.
Here is the basic code:
router.get("/:postId", async (req, res) => {
console.log('id : ', req.params.postId)
console.log('type: ', typeof(req.params.postId)) // id is a string
try {
const post = await Post.findById(req.params.postId).exec();
console.log('post :', post) // sometimes null
res.json(post);
} catch (err) {
res.json({ message: err });
}
});
In the above route, only certain posts will be found while others come back null. This happens regardless of whether the id passed is correct and the document exists with the exact id.
If anyone has any ideas about what could be going wrong here I'd much appreciate the help!
EDIT
I have done some more debugging and think it is something to do with the Schema for the Post model.
For example, this object will be found:
{
"tags": ["foo"],
"_id": "8394839483fhg020834903",
"title": "bar",
"content": "baz",
"isPrivate": true,
}
But this one will not because of the missing isPrivate property.
{
"tags": [],
"_id": "5e0fdc631ef5c46b285a4734",
"title": "New post",
"content": "Some content here",
}
I have tested this across multiple queries and it appears to the root of the problem.
I have tried adding
isPrivate: {
required: false
}
To the Schema but it doesn't seem to solve the issue.
Here is the full Schema
const postSchema = mongoose.Schema({
title: {
type: String,
required: true
},
content: {
type: String,
required: true
},
tags: [{ type: String }],
date: {
type: Date,
default: Date.now
},
isPrivate: {
type: Boolean
required: false
}
});
I'm not a Mongo/Mongoose expert, so any guidance would be much appreciated.
If post id match with any record it return data, otherwise it will return null. You should handle the exception
router.get("/:postId", async (req, res) => {
try {
const post = await Post.findById(req.params.postId).exec();
if(post) {
return res.json(post);
}
res.json({ message:'No Post found' });
} catch (err) {
res.json({ message: err });
}
});
You can manually check is record exists against a post id. You can use MongoDB Compass for gui browse the record
I believe the issue might be with your _id as per mongo standard _id should be a String is of 12 bytes or a string of 24 hex characters.
We can check if the _id is valid using mongoose.isValidObjectId()
I did run this check on your objects that you posted and indeed 1 is invalid while other is valid
const mongoose = require('mongoose');
console.log(`is '8394839483fhg020834903' valid - ${mongoose.isValidObjectId('8394839483fhg020834903')}`);
console.log(`is '5e0fdc631ef5c46b285a4734' valid - ${mongoose.isValidObjectId('5e0fdc631ef5c46b285a4734')}`);
It gives me
You will have to check what is modifying your ID's in the code, you can upload your schema to get a better understanding as well.

Waterline relations in SailsJS

I write an application in SailsJS and i have a problem with relations,
some code at the beginning:
User.js
module.exports = {
tableName: 'user',
attributes: {
schedules: { collection: 'userworkstationschedule', via: 'user' },
}
UserWorkstationSchedule.js
module.exports = {
tableName: 'user_workstation_schedule',
attributes: {
user: { model: 'user', required: true },
}
}
After run my code in Postaman, in JSON response i get:
{
...
"user": 2,
...
}
I get only ID of my user, but i want to get a whole object of User model with his firstname, lastname, etc.
Ccould anyone help me?
I'm more accustomed to a sails 0.12, but I know there you can configure your app so that population happens automatically, but it is not the default - to do this, go into config/blueprints.js and set the populate option to true.
However, I recommend against this - population may not be something you need on every page / every api call. You can make your api calls with population requests built in, like so:
/api/userworkstationschedule/[workstationid]?populate=user
That should make the populate happen just for that api call.

Loopback. How to get all roles for one user?

I'm creating a Loopback application and have created a custom user model, based on built-in User model.
{
"name": "user",
"base": "User",
"idInjection": true,
"properties": {
"test": {
"type": "string",
"required": false
}
},
"validations": [],
"acls": [],
"methods": []
}
Then in boot script I'm creating (if not exists) new user, new role and a roleMapping.
User.create(
{ username: 'admin', email: 'admin#mail.com', password: 'pass' }
, function (err, users) {
if (err) throw err;
console.log('Created user:', users);
//create the admin role
Role.create({
name: 'admin'
}, function (err, role) {
if (err) throw err;
//make user an admin
role.principals.create({
principalType: RoleMapping.USER,
principalId: users.id
}, function (err, principal) {
if (err) throw err;
console.log(principal);
});
});
});
Then in custom remote method I'm trying to get all roles for User, using user's id. Loopbacks' documentation on this topic says that
Once you define a “hasMany” relation, LoopBack adds a method with the relation name to the declaring model class’s prototype automatically. For example: Customer.prototype.orders(...).
And gives this example:
customer.orders([filter],
function(err, orders) {
...
});
But when I am trying to use User.roles() method, (const User = app.models.user;) I get the next error:
TypeError: User.roles is not a function
But when I'm making a remote request http://localhost:9000/api/users/5aab95a03e96b62718940bc4/roles, I get the desired roleMappings array.
So, i would appreciate if someone could help get this data using js. I know I can probably just query the RoleMappings model, but I've wanted to do it the documentation-way.
Loopback documentation suggests to extend the built-in user model
to add more properties and functionalities.
A good practice is creating a model Member that extends the built-in model User. In the new model declare the following relationship:
"relations": {
"roles": {
"type": "hasMany",
"model": "RoleMapping",
"foreignKey": "principalId"
}
}
Now, you can get all the user roles:
user.roles(function (err, roles) {
// roles is an array of RoleMapping objects
})
where user is an instance of Member.
This is an old question, but I faced the same issue and was able to solve it by having the relation Antonio Trapani suggested and accessing the roles like this:
const userInstance = await User.findById(userId);
const roles = await userInstance.roles.find();
Roles is not a function, it is an object. By the way this is using loopback 3.

Rooms/Channels and userId in Meteor.js

I'm building a multiplayer, turn-based game using meteor.js. The application will handle multiple games, so I'd like to separate my users into rooms.
I've done it before using socket.io channels, but I'm struggling to understand how it should be done in Meteor.
The flow I'd like to achieve is:
User visits http://localhost:3000/join/userId
I make a server-side call to an external API using "sessionId" as parameter, getting user's userId, his assigned roomId and an array of allowed userId's for this room
I'd like to create a room with roomId for the user or join him to an existing one. I know I should create a 'Rooms' collection, but I don't know how to tie users to my rooms and publish messages only to those present in the given room.
I'd like to avoid using 'accounts' package, because I don't need authorisation on my side - it'll be handled by step #2 mentioned above - but if the easiest and cleanest way of doing it involves adding this package, I can change my mind.
Your Rooms collection could look like:
{
_id: "<auto-generated>",
roomId: "roomId",
users: [ "user1", "user2", "user3", ... ],
messages: [
{ message: "", userId: "" },
{ message: "", userId: "" },
{ message: "", userId: "" },
...
]
}
The server-side API call returns
userId and roomId among other information.
So you can do a
Rooms.update({ roomId: roomId }, { $push: { users: userId } }, { upsert: true });
This would push the user into the exiting room or create a new room and add the user.
Your publish function could look like:
Meteor.publish("room", function(roomId) {
// Since you are not using accounts package, you will have to get the userId using the sessionId that you've specified or some other way.
// Let us assume your function getUserId does just that.
userId: getUserId( sessionId );
return Rooms.find({ roomId: roomId, users: userId });
// Only the room's users will get the data now.
});
Hope this helps.

MongoError,err:E11000 duplicate key error

I have a MongoDb schema like this
var User = new Schema({
"UserName": { type: String, required: true },
"Email": { type: String, required: true, unique: true },
"UserType": { type: String },
"Password": { type: String }
});
I am trying to create a new user
This is done in NodeJs using mongoose ODM
And this is the code for creating:
controller.createUser = function (req, res) {
var user = new models.User({
"UserName": req.body.UserName.toLowerCase(),
"Email": req.body.Email.toLowerCase(),
"UserType": req.body.UserType.toLowerCase()
});
models.User.findOne({ 'Email': user.Email }, function (err, olduser) {
if (!err) {
if (olduser) {
res.send({ 'statusCode': 409, 'statusText': 'Email Already Exists' });
}
else if (!olduser) {
user.setPassword(req.body.Password);
user.save(function (err, done) {
if (!err) {
console.log(user);
res.send({ 'statusCode': 201, 'statusText': 'CREATED' });
}
else {
res.send({ 'Status code': 500, 'statusText': 'Internal Server Error' });
}
});
}
}
else {
res.send({ 'statusCode': 500, 'statusText': 'ERROR' });
}
});
};
The for creating new user,I am giving attributes and values as follows:
{
"UserName": "ann",
"Email": "ann#ann.com",
"UserType": "normaluser",
"Password":"123456"
}
And I am getting error like this:
{"Status code":500,"statusText":"Internal Server Error","Error":{"name":"MongoError","err":"E11000 duplicate key error index: medinfo.users.$UserName_1 dup key: { : \"ann\" }","code":11000,"n":0,"connectionId":54,"ok":1}}
I understand that this error is because UserName is duplicated ,but I haven't set UserName with unique constraint.Whenever I add a new row,I need only email to be unique,UserName can be repeated.How to achieve this??
#ManseUK Is probably right, that looks like UserName is a 'key' - in this case an index. The _id attribute is the "primary" index that is created by default, but mongodb allows you to have multiple of these.
Start a mongo console and run medinfo.users.getIndexes()? Something must have added an index on 'UserName'.
required: true wouldn't do that, but you might have played with other settings previously and the index hasn't been removed?
There should be an index that is blocking.
You can try the db.collection.dropIndex() method
medinfo.users.dropIndexes()
I got the similar issue on my project. I tried to clear out all the documents and the dup issue still keep popping up. Until I dropped this collection and re-start my node service, it just worked.
What I had realized is that my data-structures were changing -- this is where versioning comes in handy.
You may need to get a mongoose-version module, do a thing.remove({}, ...) or even drop the collection: drop database with mongoose
I use RoboMongo for an admin tool (and I highly recommend it!) so I just went in and right-clicked/dropped collection from the console.
If anyone knows how to easily version and/or drop a collection from within the code, feel free to post a comment below as it surely helps this thread ( and I :) ).

Categories