We Keep Coding

MongoDB update many using the values of the user

I've written a function to execute hourly which looks up a user and finds some values and then pushes those values into a history collection that records the hourly updated values. I've written this so far as a test just finding a user by their ID but now I need to roll this out to my entire database of 50,000+ users.
From what I've read using updateMany is a lot more performant but I'm not entirely sure how to retrieve the document detail of the record that is being updated at the time.
Here is my code so far, which you can see I'm first looking up the user and then grabbing their valuation details which I'd like to then push into a history collection.
exports.updateUserValuationHistoric = () => {
User.find({ _id: "609961fdd989613914ef7216" })
.populate('UserValuationHistory')
.exec((err, userDoc) => {
if (err){
console.log('[ERROR]: ', err)
}
const updatedValuationHistory = {
totalValuation: userDoc[0].valuation.totalValuation,
comicsValuation: userDoc[0].valuation.comicsValuation,
collectiblesValuation: userDoc[0].valuation.collectiblesValuation,
omiValuation: userDoc[0].valuation.omiValuation
}
UserValuationHistory.findOneAndUpdate(
{ user: userDoc[0]._id },
{ $push: {
'history': updatedValuationHistory
}},
{upsert: true, new: true}
)
.exec((error, updated) => {
if (error){
console.log('[ERROR]: Unable to update the user valuation history.')
} else {
console.log('[SUCCESS]: User historic valuation has been updated.')
}
})
})
}
Any help is greatly appreciated!
User model:
https://pastebin.com/7MWBVHf3
Historic model:
https://pastebin.com/nkTGztJY

Querying items between a range of dates using Mongoose

I'm trying to get my query working correctly. It's supposed to return 5 items which have a utcTime that falls between two dates. I've verified that documents in my mongoDB do have correct dates and should be returned. This query works if I remove the .where line. In fact it works if I remove the gte and lt lines. The date gets parsed correctly so I'm not sure why this query is failing me.
I've also tried the alternative which is to put a filter in the find function like this
{
"utcTime": {
"$gte": new Date(req.params.startdate),
"$lt": new Date(req.params.enddate)
}
}
But that doesn't work either.
// Get the top 5 items sorted in decreasing order by karma in between
// the specified dates
listItemRoute.route('/top5/:startdate/:enddate').get((req, res) => {
console.log(req.params.startdate) //2020-05-13T07:00:00Z
console.log(req.params.enddate) //2020-05-20T07:00:00Z
ListItem
.find()
.limit(5)
.where('utcTime').gte( new Date(req.params.startdate) ).lt( new Date(req.params.enddate) )
.sort({karma: -1})
.exec((error, data) => {
if (error) {
return next(error)
} else {
res.json(data)
}
})
})
Here's my model
// Define collection and schema
let ListItemSchema = new Schema({
name: {
type: String,
required: true
},
karma: {
type: Number
},
utcTime: {
type: Date
}
})
Everything looks correct and I've tried many other stackoverflow suggestions to no avail. Any help would be awesome.
Edit: example of DB data
{
"name": "TITLE BLAH",
"karma": 11502,
"utcTime": "2020-05-17T09:50:21Z"
}

Mongoose find inside nested schema

In mongoose we are deeply searching inside a nested schema, without much success. Every time we run this function we always get an empty array returned.
function findAlarms(lastUpdate = new Date(0), record = Record) {
// For docs on find http://mongoosejs.com/docs/queries.html
return record
.find({
// Date due must be less than "now"
'documents.alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'documents.alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.populate('documents')
.exec();
}
Our schemas, greatly summarized, look like this:
const RecordSchema = new mongoose.Schema({
documents: [
{
type: Schema.Types.ObjectId,
ref: 'Document',
},
],
});
And our documents schema, similarly summarized looks like this:
const DocumentSchema = new mongoose.Schema({
alarm: {
date_due: { type: Date },
date_reminder: [
{
reminder: { type: Date },
},
],
},
});
This search returns no matching elements, even though we know there are documents that match. If we modify our findAlarms method to use the documents schema:
function findAlarms(lastUpdate = new Date(0), document = Document) {
// For docs on find http://mongoosejs.com/docs/queries.html
return document
.find({
// Date due must be less than "now"
'alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.exec();
}
It will return all of our matching documents. However, having records is essential for our needs. Now, I could use a hack and then find records using the array of document._ids that return.
Nonetheless, I would love to know if there's an approach where we can find using the records directly, since adding that extra step feels really hacky, and this operation runs every 5 minutes so I'd love to be more efficient wherever posible.

sequelize interprets date with hour = 2

I'm trying to use sequelize to search for a record in my database based in a date field:
date_from2 is this value:
2017-01-09
and sequelize interprets its as a date with hour = 2:
SELECT `id`, `user_id`, `date`, `total_visits`, `created_at`, `updated_at` FROM `table1` AS `table1` WHERE `table1`.`user_id` = 123 AND `table1`.`date` = '2017-01-09 02:00:00' LIMIT 1;
And it creates a new record everytime, instead of updating it.
When it inserts, the date is inserted with this value:
2017-01-09 00:00:00
This is my code:
where = { user_id: user_id,
date: date_from2
};
values = {
user_id: user_id,
date: date_from2,
total_visits: total_visits
};
Model.findOne({where: where}).then(function (record) {
if (!record) {
// Item not found, create a new one
Model.create(values)
.then(function () {
console.log('created!');
}).error(function (err) {
console.log('error on create');
});
} else {
// Found an item, update it
Model.update(values, {where: where})
.then(function () {
console.log('updated!');
})
.catch(function (err) {
console.log('error on update');
});
}
});

Query by just using the user's user_id and not the date if you are trying to get a specific user:
Model.findOne({where: {user_id: user_id}})
The issue is that it's storing the date differently then what you are passing in. Because of this you were never able to find the existing user and you kept creating new ones. There is nothing wrong with using UTC timezone and you should be able to convert this to any other time zone.

Updating doc in Pouch DB

I have a DB in PouchDB, and I need to be able to update documents. So when I click "update" in the table I get the data from the fields into a form, then I want to change data in the form and press "Save Updated" button and have the fields updated. Here's what I've tried:
function saveUpdated(){
var vaucherID = window.document.VaucherForm.vaucherID.value;
var date = window.document.VaucherForm.date.value;
var invoiceNumber = window.document.VaucherForm.invoiceNumber.value;
var vendorID = window.document.VaucherForm.vendorID.value;
var amount = window.document.VaucherForm.amount.value;
var fund = window.document.VaucherForm.fund.value;
var deptID = window.document.VaucherForm.deptID.value;
var descript = window.document.VaucherForm.descript.value;
//I get idValue from when I have all values from DB get into form
db.get('idValue').then(function(doc) {
return db.put({
_id: 'idValue',
_rev: doc._rev,
vaucherID: vaucherID,
date: date,
invoiceNumber: invoiceNumber,
vendorID: vendorID,
amount: amount,
fund: fund,
deptID: deptID,
descript: descript
});
}).then(function(response) {
// handle response
}).catch(function (err) {
console.log(err);
});
}
So as I do this I get this error
o {status: 404, name: "not_found", message: "missing", error: true, reason: "missing"}

Hm, maybe it's because the doc._rev is undefined/null, so this is treated differently than just not including the _rev field at all? Do you have a live example to reproduce?