We Keep Coding

Dealing with JSON data returned from a DynamoDB query

I am storing data in a DynamoDb table. I have 3 columns:
hub_id (edit: primary partition key)
on_time (sort key - this is a date time stored as a string)
details this contains my JSON data (information from a sensor):
{ "hub_id" : { "S" : "PnKVrm12" }, "temperature" : { "S" : "23" }, "sensor_name" : { "S" : "Tech Lab" }}
Using react-native and aws amplify I can query the table using the following code and return the correct number of rows :
// setup query params
var params = {
TableName : "hiddenprojectv-mobilehub-1542262168-hidden",
ProjectionExpression:"hub_id, details.temperature, details.sensor_name",
KeyConditionExpression: "hub_id = :hid",
ExpressionAttributeValues: {
":hid":"PnKVrm12"
}
};
//execute query using params
db.query(params, function(err, data) {
if (err) {
console.log("Unable to query. Error:", JSON.stringify(err, null, 2));
} else {
console.log("Query succeeded.");
data.Items.forEach(function(details) {
console.log(details.hub_id,details.sensor_name);
});
}
});
but I am struggling to get at the data held in the Json object (details column). The query above returns:
16:44:45: PnKVrm12 undefined
I expected it to return:
16:44:45: PnKVrm12 Tech Lab
for some reason I can never access data in either temperature or sensor_name.
I would appreciate some advice if you have time.

Your ProjectionExpression is accessed in your forEach loop and provides:
details = { hub_id: 'x', details: { temperature: 'x', sensor_name: 'x' } };
You are not using your data structure properly. To access the sensor_name property you must use:
`details.details.sensor_name'

mongodb array data store using node js

I have to try to store my child info into MongoDB via using postman tool. But it shows me this message "message": "child info validation failed"
in postman console. child info is my collection name where I store my child info.
my requirement is to store the result in array form as below schema mentioned inside MongoDB
1). This is js child schema
userId:{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
quiz:[
{
questionId:{
type: mongoose.Schema.Types.ObjectId,
ref: 'questions'
},
score:{type:String},
time:{type:String}
}
]
2). This is node js
try {
var quizArr = [];
var quizObj = {
'questionId': req.params.questionId,
'score': req.params.score,
'time': new Date().toISOString()
};
quizArr.push(quizObj);
var userObj = {
'userid': req.params.userId,
'quiz': quizArr
};
//var parseObj = Json.stringify(userObj); This line is comment
var childinfoSave = new QuizChildInfo(userObj);
childinfoSave.save(function (err) {
if (err) return next(err);
res.send("Child questionId score and date saved successfully")
console.log("Child questionId score and date saved successfully");
});
}catch(err){console.log(err);}
3). Output of postman screen
{
"message": "childinfos validation failed"
}
4). Output of console
Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html
5). Mongo console
{
"_id" : ObjectId("57bc483169e718642ac0ac44"),
"levelsAttempted" : [ ],
"quiz" : [ ],
"__v" : 0
}

For the problem of your console,
put mongoose.Promise = global.Promise; in the file where you have established your server connection( like in my case its index.js).
And i think you may have not posted the whole code as i couldnt find childinfos in your code.

complex mongoose query with $where

Im having a real headache with a mongoose query, I am still quite new to it and wondered if anyone could suggest a solution.
I have the following fields in a gallery collection.
status (value of 'public' or 'restricted'),
restriction (value of 'canview' or 'cantview'),
canview (an array of user ids (ObjectIds) of users permitted to view gallery)
I then check fields against the current userID, to see what items are viewable.
how would it be possible to select ALL of the following items
a) all items with 'status' = 'public'
b) all items with 'status' = 'restricted' WHERE 'restricted' = 'canview' AND 'canview' contains the UserID
I wasnt able to do this using $and and $or, so tried with $where instead. Using the following global function
var existsInArray = function(str,arr) {
// nb: remember we have typeof ObjectID, so lets convert to strings first!
var newarr = arr.toString().split(',');
// now compare..
if(newarr.inArray(str)) {
return true;
}
return false;
};
I was hoping to do something like this...
exports.listviewable = function(req, res) {
var reqID = req.user.id;
Gallery
.find()
.$where(function () {
return this.status === "public" || ( this.status === "restricted" && this.restriction === "canview" && existsInArray(reqID, this.canview));
})
.sort('-created')
.exec(function(err, galleries) {
if(err) {
return res.status(400).send({message:getErrorMessage(err)})
} else {
res.json(galleries);
}
});
};
but this wasnt working - it appears I wasnt able to use the global existsInArray function in the $where clause ?
Is it possible to do it like this ( similar unresolved question here how to call external function in mongoose $where and passing this variable), or is there a better way to do it with AND and OR ?

Hey I'd recommend hitting the mongo docs - they have alot of info on this.
a)
Gallery.find({status: 'public'}, function(err, data){
console.log(data)
});
b)
Gallery.find({status: 'restricted', restricted: 'canview', canview: reqID }, function(err, data){
});
or both together with sorting...
Gallery.find({$or:[
{status: 'public'},
{status: 'restricted', restricted: 'canview', canview: reqID }
]}, {
sort:{created:-1}
}, function(err, data){
console.log(data)
});

A $where function can't refer to your local existsInArray function because the $where function is executed on the server where that local function doesn't exist.
But you shouldn't be using $where anyway, as you can do with with a simple $or query that takes advantage of the fact that you can directly query array fields like canview with "contains" behavior:
Gallery
.find({
$or: [
{status: 'public'},
{status: 'restricted', restriction: 'cantview', canview: reqId}
]
})
.sort('-created')
.exec(function(err, galleries) {...});

an error about tailable cursor for a query in mongoose

User model contian SubscriptionSchema and AccessToken Schema, I had defined this two plugin schemas with {capped : 234556} too.
var User = new Schema({
email : String
, firstName : String
, password : String
, isAdmin : Boolean
, lastSeen : Date
, subscriptions : [ SubscriptionSchema ]
, accessTokens : [ AccessToken ]
}, {
toObject : { virtuals : true }
, toJSON : { virtuals : true }
, capped : 234556
});
var streamTest = User.find().limit(1).tailable().stream();
When I try to run the above code, I still get the error:
MongoError: tailable cursor requested on non capped collection

That doesn't look like a correct usage of a capped collection or tailable stream. But perhaps a little code first to demonstrate a working example:
var async = require('async'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
var userSchema = new Schema({
email: String,
},{
capped: 2048
});
var User = mongoose.model( "User", userSchema );
mongoose.connect('mongodb://localhost/atest');
var stream;
async.waterfall(
[
function(callback) {
var user = new User({ email: "existing" });
user.save(function(err,doc) {
if (err) throw err;
callback();
});
},
function(callback) {
User.find({}).sort({ "$natural": -1 }).limit(1).exec(function(err,docs) {
if (err) throw err;
console.log( docs );
callback(err,docs[0]);
});
},
function(doc,callback) {
stream = User.find({ "_id": { "$gt": doc._id } }).tailable().stream();
stream.on("data",function(doc) {
console.log( "Streamed:\n%s", doc );
});
callback();
},
function(callback) {
async.eachSeries(
['one','two','three'],
function(item,callback) {
var user = new User({ email: item });
user.save(function(err,doc) {
if (err) throw err;
console.log( "Saved:\n%s", doc );
callback();
});
},
function(err) {
if (err) throw err;
callback();
}
);
}
]
);
First things first, there really needs to be something in the capped collection for anything to work. This presumes that the collection does not exist and it is going to be initialized as a capped collection. Then the first step is making sure something is there.
Generally when you want to "tail", you just want the new documents that are inserted to appear. So before setting up the tailable cursor you want to find the "last" document in the collection.
When you know the last document in the collection, the "tailable stream" is set up to look for anything that is "greater than" that document, which are the new documents. If you did not do this, your first "data" event on the stream would empty all of the current collection items. So the options to .sort() and .limit() here do not apply. Tailing cursors initialize and "follow".
Now that the streaming interface is set up and an listener established, you can add items to the stream. These will then log accordingly, yet as this is "evented", there is no particular sequence to the logging here as either the "save" or the "stream" data event may actually fire first.
Now onto your implementation. These two lines stand out:
, subscriptions : [ SubscriptionSchema ]
, accessTokens : [ AccessToken ]
Those contain embedded arrays, they are not "external" documents in another collection, even though it would not matter if it even did.The general problem here is that you are (at least in some way) introducing an array, which seems to imply some concept of "growth".
Unless your intention is to never "grow" these arrays and only ever insert the content with the new document and never update it, then this will cause problems with capped collections.
Documents in capped collections cannot "grow" beyond their initial allocated size. Trying to update where this happens will result in errors. If you think you are going to be smart about it and "pad" your documents, then this is likely to fail when replicated secondary hosts "re-sync". All documented with capped collections.

How to display the results of an aggregation in jade

How do you display the results of a mongodb aggregation query using node.js, express & jade
I'm not sure what I've done wrong or even if this is the correct approach.
When I attempt to access the page - it just times out?
Any idea? Or is there a better way?
app.js
var mongo = require('mongodb');
var monk = require('monk');
var db = monk('localhost:27017/soundevents');
var timeroute = require('./routes/timers');
app.get('/time/starttimer', timeroute.starttimer);
my routes/timers.js looks like this:
exports.starttimer = function(db) {
return function(req, res) {
var aEvent = db.get('event');
aEvent.aggregation([{$group:{_id:{"EventName":"$EventName"}}}],{}, function(e,docs) {
res.render('time/starttimer', {title: 'Stat Timer',
"eventlist" : docs});
});
};
};
output from mongo
db.event.aggregate([{$group:{_id:{"EventName":"$EventName"}, UpdateTime:{$max: "$UpdateTime"}}}])
{"result" : [{"_id" : {
"EventName" : "MtnBikeRace 1"},
"UpdateTime" : 1392265180.069293},
{"_id" : {
"EventName" : "Student League"},
"UpdateTime" : 1392047321724}],
"ok" : 1}

This example can help you.
https://github.com/visionmedia/express/blob/master/examples/jade/index.js
// Dummy users
var users = [
new User('tj', 'tj#vision-media.ca')
, new User('ciaran', 'ciaranj#gmail.com')
, new User('aaron', 'aaron.heckmann+github#gmail.com')
];
app.get('/', function(req, res){
res.render('users', { users: users });
})
// user.jade
extends ../layout
block content
h1 Users
#users
for user in users
.user
h2= user.name
.email= user.email

I landed here because I was looking for a way to aggregate results using monk.
For future reference: as of now, monk has no .aggregate() method, therefore this part of your code will not work (as you've indicated in a comment):
// there's no aggregation method.
aEvent.aggregation([{$group:{_id:{"EventName":"$EventName"}}}],{}, function(e,docs) {
res.render('time/starttimer', {title: 'Stat Timer',
"eventlist" : docs});
});
However,
this answer shows the correct way to perform an aggregation with monk. It uses the underlying native driver. In your case this would be something like the following:
aEvent.col.aggregate([{$group:{_id:{"EventName":"$EventName"}}}],{}, function(e,docs) {
res.render('time/starttimer', {title: 'Stat Timer',
"eventlist" : docs});
});
If you aggregate this way, I guess you can accept fundon's answer.