We Keep Coding

why is mongoose TTL deleting document, even when partial filter does not match the option

my partial filter is deleting document, but user is not matching that requirement, am I using partial filter incorectly?
Thanks
const postSchema = new mongoose.Schema(
{
title: { type: String },
description: { type: String },
image: { type: String },
price: { type: String },
location: { type: String },
image: { type: Array },
author: {
type: String,
ref: 'User'
},
authorPremium: {
type: Boolean,
default: true,
index:true
},
reported: {
type: Boolean,
default: false
},
reportClear: {
type: Boolean,
default: false
}
},
{
timestamps: true
}
);
// users who are not premium will have posts deleted after 20 seconds
postSchema.index({ createdAt: 1 }, { expireAfterSeconds: 20, partialFilterExpression: { authorPremium: false } });
module.exports = mongoose.model('Post', postSchema);
partial filer should not allow the authorPremium which is true to be deleted, but only delete is authorPremium is false... please advise.
return from mongo index
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "test.posts"
},
{
"v" : 2,
"key" : {
"createdAt" : 1
},
"name" : "createdAt_1",
"ns" : "test.posts",
"expireAfterSeconds" : 120,
"background" : true
},
{
"v" : 2,
"key" : {
"authorPremium" : 1
},
"name" : "authorPremium_1",
"ns" : "test.posts",
"background" : true
},
{
"v" : 2,
"key" : {
"timestamps" : 1
},
"name" : "timestamps_1",
"ns" : "test.posts",
"expireAfterSeconds" : 20,
"background" : true
}
]
it seems when I use mongo cmd some of my old setting remained.. and some new? So how can I completly clear these old ttl settings when I am testing and ensure only the ones I want are there?

It looks like your .index(...) does not work because you already have old index on createdAt field and mongoose won't drop the old one. To synchronize the indexes with your schema you can use Model.syncIndexes
More on how .index() works and why .syncIndexes() was introduced can be found here.

Realm - Value not convertible to a number

Hello community :) I implemented lots of good working functionality with firebase -> realm. Now i tried to edit a structure and I am running through the wildest error messages.
What is right for sure:
Firebase sends the data
Data is Converted (e.g. Firebase has "brands" as array -> is converted to a string for Realm Schema)
The error appears when firebase updates
Not every firebase content has all fields (e.g. Like you can see out of Realm Schema some fields are optional: true)
Fields where i maybe expect an issue:
Maybe its not possible to say that the ReferentList is optional (or i implemented it wrong): See Realm Schema const ReferentsList
What i tried
Debug before realm.create (Realm set) Result: Every data came in the right format
Checked all input values if they are int, string, ...
Hopefully someone can help me here because i got completely stuck with this issue and its necesarry to continue for my project. I want to know:
The solution why or what to do
A posibility to debug realm in a better way
Thank you in advance for your time and help :)
Error message: Value not convertible to a number
Firebase datastructure
"begin" : "2017-05-15T15:50:00.000Z",
"description" : "abc",
"end" : "2017-05-15T16:15:00.000Z",
"id" : 6,
"language" : [ 1 ],
"location" : "L 1.02",
"member" : 20,
"referent" : [ 1, 3 ],
"register" : true,
"title" : "Sound of Silence",
"track" : 6,
"type" : 3,
"brands" : [ 1, 2, 3 ]
Realm Schema
const ReferentListSchema = {
name: 'ReferentList',
properties: {
id: {
type: 'int',
optional: true
}
}
}
const LanguageListSchema = {
name: 'LanguageList',
properties: {
id: 'int'
}
}
const EventSchema = {
name: 'Events',
primaryKey: 'id',
properties: {
id: 'int',
begin: {
type: 'date',
optional: true
},
end: {
type: 'date',
optional: true
},
title: 'string',
description: 'string',
register: 'bool',
member: {
type: 'int',
optional: true
},
language: {
type: 'list',
objectType: 'LanguageList'
},
location: 'string',
referent: {
type: 'list',
objectType: 'ReferentList'
},
type: 'int',
track: {
type: 'int',
optional: true
},
img: {
type: 'string',
optional: true
},
brands:{
type: 'string',
optional: true
}
}
}
Realm set
set(obj) {
realm.write(() => {
if(obj.referent){
obj.referent = obj.referent.map(function(id) {
return {id};
})
}
if (obj.language){
obj.language = obj.language.map(function(id) {
return {id};
})
}
realm.create('Events', obj, true);
});
}

Solved:!
The issue got solved through wrong data at firebase. Some Date Objects hasent been set correct.
How i got to the solution
When i tried to debugg the code i made a try/catch block around:
try{
realm.create('Events', obj, true);
}catch(error){
console.log(obj);
console.log(error);
}
Through this debug i found the right data wich was wrong. Before it just showed me all objects and afterwards the error.
I wont close this question because of the chance to help someone with the same issues.-

Sails.js waterline and mysql adapter, can't get populate() with one-to-many associations working

So I've been at this for awhile and can't see how my code is different from the documentation.
I've also checked out this question, this question, this question, and this unanswered different question.
For my admin panel I'm trying to query to get all the information associated with a user and display a 'master' user profile to the admin.
My User model looks like this:
module.exports = {
autoPK: true,
attributes : {
id: {
type: 'integer',
primaryKey: true,
unique: true
},
email : {
type : 'email',
unique : true,
required : true,
},
password : {
type : 'string',
minLength : 8,
required : true
},
admin:{
type: 'bool'
},
user_profile:{
collection: 'userprofile',
via: 'user_id',
},
properties: {
collection: 'subjectproperties',
via: 'user_id'
},
employment_info: {
collection: 'employmentinfo',
via: 'user_id'
},
file_uploads: {
collection: 'fileupload',
via: 'user_id'
},
nearest_living_relatives:{
collection: 'nearestlivingrelative',
via: 'user_id'
},
mortgage_info: {
collection: 'mortgageinfo',
via: 'user_id'
},
user_progression_state:{
collection: 'userprogressionstate',
via: 'user_id'
},
users_applied_loan_values:{
collection: 'usersappliedloanvalues',
via: 'user_id'
}
}
}
I don't want to list out all the belongs to user models cause there are a lot of them, but here is one of the simpler one's.
EmploymentInfo.js
module.exports = {
tableName: "employment_info",
attributes : {
employers_name:{
type: 'string',
required: true
},
employers_address:{
type: 'string',
required: true
},
employers_city:{
type: 'string',
required: true
},
employers_state:{
type: 'string',
required: true
},
employers_zip:{
type: 'string',
required: true
},
job_position:{
type: 'string',
required: true
},
years_in_position:{
type: 'string',
required: true
},
years_in_industry:{
type: 'integer',
required: true
},
user_id:{
model:'user'
}
}
};
And as for my controller:
create_admin_user_profile: function(req, res){
var user_id = req.query.userId;
User.find({'id': user_id}).populateAll().exec(function(err, user){
if(err || user.length === 0){
sails.log.verbose(err);
}else{
sails.log.verbose(user);
}
});
},
It doesn't error out but all I see in the terminal is this for the above:
[ { user_profile: [],
properties: [],
employment_info: [],
file_uploads: [],
nearest_living_relatives: [],
mortgage_info: [],
user_progression_state: [],
users_applied_loan_values: [],
id: 5,
email: 'test#test.com',
admin: 1 } ]
Even though there is an entry in all of those tables for that user.
If I change the line:
User.find({'id': user_id}).populateAll().exec(function(err, user){
To:
User.find({'id': user_id}).populate('employment_info').exec(function(err, user){
Same but shorter result:
[ { employment_info: [],
id: 5,
email: 'test#test.com',
admin: 1 } ]
I've tried changing the case, I've tried adding columnName to the user_id attribute, I've tried changing the column name across the entire breadth of the project to not have an under_score in it, though that never seemed to be issue in it picking up the names correctly, but nothing I've done seems to work. I've also tried uninstalling sails, and the sails-mysql adapter and clearing my npm cache.
At this point my just stuck, I really can't see a reason why it's not working.
As for project info:
Sails v: 0.12.11
npm v: 3.10.9
node v: 7.2.0
Additional info asked for in comments:
SQL row taken right from db for user 5
employers_name, employers_address, employers_city, employers_state, employers_zip, job_position, years_in_position, years_in_industry, user_id
'Company', 'Mill Steet', 'SLC', 'Utah', '88888', 'Developer', '2', '2', '5'
And json format returned by find method in EmploymentInfo.js controller
{
"employmentInfo": {
"employers_name": "Company",
"employers_address": "Mill Steet",
"employers_city": "SLC",
"employers_state": "Utah",
"employers_zip": "88888",
"job_position": "Developer",
"years_in_position": "2",
"years_in_industry": 2,
"user": 5
}
}
The reason the last param is user and not user_id is because I rename it in the find method to serve the front-end mvc which also has the ability to work with associations. It's also why the JSON has the format it does.
Code from the find method that replaces user_id:
EmploymentInfo.find({'user_id': user_id}).exec(function(err, profile){
if(err || !profile.length){
return res.json(err);
}else{
res.status(200);
profile[0].user = profile[0].user_id;
delete profile[0].user_id;
res.send({'employmentInfo': profile[0]});
}
});
However I've tried not renaming it; I've also tried getting rid of my find override and just relying on the blueprint find method, neither of those worked either.

Mongoose create a new document with catergorized sub documents

I've been going through the Mongoose docs, and I think I'm missing some fundamental understanding in how it works.
What I'm trying to do
I'm making a third party API call that returns a structure that looks like
Route
|__Train 1 on Route
|__Upcoming Station (with ID)
| |__Time to this station
|__Upcoming Station (with ID)
| |__Time to this station
...
|__Train 2
...
And my goal is to format it in a document as such
tableId : String,
stations : [{
stopId : String,
incoming : [{
vehicleId : String,
timeAway : { type: Number, min: 0, max: 3000 },
lastUpdated : { type: Date, default: Date.now }
}]
}],
What I'm trying currently is going through the received data for each train, and in that each upcoming station, and plug the estimated arrival time into the list of stations. The important part is that Train 1 and Train 2 may both be arriving at a given station, and I only want one station element with multiple predictions. The problem is, I can't do a findOneAndUpdate with an upsert, as the document doesn't exist yet.
From the doc on subdocs (here), I've tried push and addToSet, but these just create a subdocument for each prediction. For example I'll get:
[{
stopId: 1234,
incoming : [{
vehicleId : 11,
timeAway : 200
}]
},
stopId: 1234,
incoming : [{
vehicleId : 22,
timeAway : 400
}]
}]
Where I'm trying to get:
[{
stopId: 1234,
incoming : [{
vehicleId : 11,
timeAway : 200
},{
vehicleId : 22,
timeAway : 400
}]
}]
I feel like I'm missing some fundamental aspect of creating this document.

For data schema,
var StationSchema = new mongoose.Schema({
tableId: String,
stations: [{
stopId: String,
incoming: [{
vehicleId: String,
timeAway: {type: Number, min: 0, max: 3000},
lastUpdated: {type: Date, default: Date.now}
}]
}]
});
Save data through
var s = new Station({
tableId: '2'
});
s.save(function(err) {
Result
{ "_id" : ObjectId("56e68bcf851a00680832ef13"), "tableId" : "2", "stations" : [ ], "__v" : 0 }
We know the default value of stations is empty array, which is design behavior of mongoose. The upsert: true will add one new document not for sub-document.
To insert station subdocument, we can first check whether the stopId exists, if not, insert new station subdocument. otherwise, we can insert new incoming subdocument into stations. Here are sample codes
Station
.findOneAndUpdate({tableId: '2', 'stations.stopId': {$exists: false}},
{$addToSet: {stations: {stopId: '1234', incoming: []}}},
function (err, doc){
if (err)
console.log(err);
else{
Station
.findOneAndUpdate(
{'stations.stopId': 1234},
{$addToSet: {'stations.$.incoming': {vehicleId: 22, timeAway: 400}}},
function(err, doc) {
if (err)
console.log(err);
else
console.log(doc);
});
}
});

KeystoneJS - Create new Item throws duplicate key error

I have seen some similar questions related to this but have not found an answer.
I am attempting to create a Gallery in my Keystone Project that is similar to a post, where there will be a list of galleries and in it a gallery with a set of selected images:
var keystone = require('keystone'),
Types = keystone.Field.Types;
/**
* Gallery Model
* =============
*/
var Gallery = new keystone.List('Gallery', {
map: { name: 'name' },
autokey: { path: 'slug', from: 'name', unique: true }
});
Gallery.add({
name: { type: String, required: true},
published: {type: Types.Select, options: 'yes, no', default: 'no', index: true},
publishedDate: { type: Types.Date, index: true, dependsOn: { published: 'yes' } },
description: { type: String },
heroImage : { type: Types.Relationship, ref: 'Image' },
images : { type: Types.Relationship, ref: 'Image', many: true }
});
Gallery.defaultColumns = 'title, published|20%, publishedDate|20%';
Gallery.register();
I am able to create one gallery successfully - but any subsequent galleries throw the error:
There was an error saving your changes: insertDocument :: caused by ::
11000 E11000 duplicate key error index: site-name.galleries.$key_1 dup
key: { : null } (MongoError)
I am at a loss for what I need to change to this model to allow unique slugs for my galleries to be directly linked to etc.

Completely delete the model from MongoDB and restart. I typically see this error when making changes to a model with indexes that have been defined previously

with insert new item,you should set an default value for 'name', because name is set unique=true.
in my case, key is set to unique.
before:
MongoDB Enterprise > db.categories.save({_id: 89,name: 'xxx'})
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 11000,
"errmsg" : "E11000 duplicate key error collection: sku.productcategories index: key_1 dup key: { : null }"
}
})
after:
MongoDB Enterprise > db.categories.save({_id: 89,name: 'xxx', key:'xx'})
WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 89 })