We Keep Coding

Mongodb findOneAndUpdate - check if new document added, or just updated

I have the following code which is finding and updating an employee-store record (if it exists, otherwise it creates one).
I have lots of employees in different stores, and they can choose to change store at any point (as long as the location is in America)
Below is my code so far:
employee = await this.employeeStoreModel.findOneAndUpdate(
{ employee: employeeRecord._id, location: "America" },
{
employee: employeeRecord._id,
store: employeeRecord.store,
location: "America",
},
{ new: true, upsert: true }
);
This works correctly, however I am trying to return some messages from this based on what is being updated. It could be any of the following messages:
If it's a completely new employee-store record being added, then return "{{StoreID}} has a new employee - {{EmployeeID}}"
If it's a change of store on an existing employee-store record, then return "{{EmployeeID}} has changed from {{old StoreID}} to {{new StoreID}}"
Is this possible to do? Can anyone guide me on how I could start this?

In the Options field,
rawResult: true
You must add the parameter.
The result will be as in the example below.
{ response:
{ n: 1,
updatedExisting: false,
upserted: 5e6a9e5ec6e44398ae2ac16a },
value:
{ _id: 5e6a9e5ec6e44398ae2ac16a,
name: 'Will Riker',
__v: 0,
age: 29 },
ok: 1 }
Depending on whether there is an update or insert in the updatedExisting field, you can return any message you want.

AWS Javascript SDK : Greengrass createFunctionDefinition

I'm formatting my parameters according to this https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Greengrass.html#createFunctionDefinition-property
But for whatever reason its giving me a key error when for "Execution" as well as DefaultConfig
Response:
Request ID:
"3ed83472-39af-493b-9df7-7f82d2f14636"
Function Logs:
r: Unexpected key \'Execution\' found in params.InitialVersion.Functions[0].FunctionConfiguration.Environment',
code: 'MultipleValidationErrors',
errors:
[ { UnexpectedParameter: Unexpected key 'DefaultConfig' found in params.InitialVersion
at ParamValidator.fail (/var/runtime/node_modules/aws-
and the code
GG.createFunctionDefinition({
InitialVersion: {
DefaultConfig: {
Execution: {
IsolationMode: "NoContainer"}
},
Functions: [
{
FunctionArn: "arn:aws:lambda:us-west-2:644226108543:function:SahmCumminsTelemetryTest:1",
FunctionConfiguration: {
MemorySize: 524288,
Pinned: true,
Timeout: 600,
Environment: {
AccessSysfs: false,
Execution: {
IsolationMode: "NoContainer",
RunAs: {
Gid: 0,
Uid: 0
}
}
}
},
Id: "function_definition",
},
],
},
Name: "function_definition",
}, function (err, data) {
if (err) {
console.log(err, err.stack);
}
else {
funcArn = data.LatestVersionArn;
};

I think that the issue is that the configuration data specifies two mutually exclusive options. There is a memory size value specified AND it says it should use "NoContainer". When the Greengrass container isn't in use memory size isn't a valid option.
Try removing memory size and see if that fixes it.
The provisioning code I've shared on Github "scrubs" functions when NoContainer is set to get around this issue. The scrubbing process is to set the memory size to NULL so when it is serialized to JSON the field is missing.
https://github.com/awslabs/aws-greengrass-provisioner/blob/e2608654b65682ca9b5b03da962cc8cb29ea1cbf/src/main/java/com/awslabs/aws/greengrass/provisioner/implementations/helpers/BasicGreengrassHelper.java#L390

How to improve or avoid find / fetch cycle in meteor's publication?

TL;DR:
Chat is one collection. ChatMess another one that has messages refering to a Chat's _id. How do I get the last messages from a list of chats with the less computation possible ? Here, find / fetch cycle in a loop is way too heavy and long.
I have this publication that is used to return a set of cursor to the user :
The chats sessions he takes part in (from Chat collection)
The last message from each of the chat session referenced in the first cursor (from ChatMess collection)
Currently, the logic is to :
Get the list of chat sessions from the user profile
Find the Chat sessions and loop through it
In the loop, I findOne the last message from this chat session and store its _id in an array. In addition, I store all the other users _ids.
Then, I find the messages which _id match the ones in my array.
Here is my main problem :
Isn't there a way more faster way to get the last messages from each of my chat session ? With that algo, I easily reach the 8000ms of response time, which is a way too heavy computation time, as much of this time is spent to find / fetch the chat messages's _id (cf linked screen from Kadira).
Meteor.publish("publishNewChat", function() {
this.unblock();
// we get a list of chat _id
let chatIdList = _get_all_the_user_chats_ids(this.userId);
if (!chatList)
return ;
// get the chat sessions objects
let chats_cursor = Modules.both.queryGet({
type : 'chat',
method : 'find',
query : { _id: { $in: chatIdList } },
projection : { sort: { _id: 1 }, limit : 1000 }
});
let array_of_fetched_chats = chats_cursor.fetch();
let chat_ids = [];
// and here we loop through the chat documents in order to get the last message that's been attached to each of them
array_of_fetched_chats.forEach(function(e) {
let lastMess = Modules.both.queryGet({
type : 'chatMess',
method : 'findOne',
query : { chatId: e._id },
projection : { sort: { date: -1 } }
});
if (lastMess)
chat_ids.push(lastMess._id);
});
return ([
chats_cursor,
Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { _id: { $in: chat_ids } },
projection : { sort: { date: -1 }, limit: 1000 }
})
]);
});
Finally, it also add latence to all my DDP request that follows. I currently use a this.unblock() to avoid that, but I'd prefer not to use it here.
FYI, I have another publish that is updated each time the client change his current active chat session : on the client, routing to a new chat add its _id in a reactive array that update my getChatMess subscription in order to get on the client the messages from every chats the user visited in this since he connected. The goal is obviously to spare the server the sending of every message from every chat session the user have visited in his life.
Unfortunately, I lack ideas to improve that algo without breaking all my chat logic :S. Have you any idea ? How would you do ?
Thanks you.
EDIT: here is a screen from kadira that clearly show the problem :

Have you considered using the reywood/publishComposite package?
With this package you can publish related data in the same method without having to do a bunch of logic to get the correct data published.
The below code should get you started:
Meteor.publishComposite("publishNewChat", function() {
return [{
find:function(){
return Users.find({ _id: this.userId },{fields:{"profile.chat":1}});
},
children:[{
find:function(user){ //this function is passed each user returned from the cursor above.
return UserChats.find({userId:user._id},{fields:{blah:1,blah:1}}); //find the user chats using whatever query
},
children:[
//if there are any children of user chats that you need to publish, do so here...
{
find:function(userchat){
return Chats.find({_id:userchat.chatId})
},
children:[
{
find:function(chat){
return ChatMess.find({chatId:chat._id},{ sort: { date: -1 } });
},
children:[
{
find:function(chatMess){
var uids = _.without(chatMess.participants, this.userId);
return Users.find({_id:{$in:uids}});
}
}
]
}
]
}
]
},
]
}]
This will publish the cursors for all of the documents related to each of the parent documents. It is pretty fast, I use this package on a production platform high traffic and large datasets with no problems. On the client you could then query the documents as normal to get the ones you need to display.
Something like:
Users.findOne({_id:Meteor.userId()});
UserChats.find({userId:Meteor.userId()});
etc...

Here is a solution I developped :
Meteor.publish("publishNewChat", function() {
this.unblock();
let user = Modules.both.queryGet({
type : 'users',
method : 'findOne',
query : { _id: this.userId },
projection : { fields: { "profile.chat": true } }
});
let thisUserschats = tryReach(user, "profile", "chat").value;
if (!thisUserschats)
return ;
thisUserschats = thisUserschats.map(function(e) { return (e.chatId); });
let chats = Modules.both.queryGet({
type : 'chat',
method : 'find',
query : { _id: { $in: thisUserschats } },
projection : { sort : { _id: 1 },
limit : 1000
}
});
let chatArray = chats.fetch(),
uids = cmid = [];
let messages_id_list = [],
i = chatArray.length;
let _parallelQuery = index => {
Meteor.setTimeout(function () {
let tmp = Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { chatId: chatArray[index]._id },
projection: { limit: 1, sort: { date: -1 } }
});
tmp.forEach(doc => {
messages_id_list.push((doc && doc._id) ? doc._id : null);
});
}, 1);
}
while (--i >= 0)
_parallelQuery(i);
let cursors = {
chats : chats,
chatMessages : null
}
let interval = Meteor.setInterval(function () {
if (messages_id_list.length === chatArray.length)
{
Meteor.clearInterval(interval);
cursors.chatMessages = Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { _id: { $in: messages_id_list } },
projection : { sort: { date: -1 }, limit: 1000 }
});
cursors.chats.observeChanges({
// ...
});
cursors.chatMessages.observeChanges({
// ...
});
self.ready();
self.onStop(() => subHandle.stop(); );
}
}, 10);
});
I used async function with Meteor.setTimeout to parallelize the queries and save an index refering to a chat _id to look for. Then, when a query is finished, I add the last message to an array. With a Meteor.setInterval, I check the array length to know when all the queries are done. Then, as I can't return cursors anymore, I use the Meteor publication low level API to handle the publishing of the documents.
FYI : in a first attempt, I was using 'findOne' in my _parallelQueries, which divided my computation time by 2/3. But then, thanks to a friend, I tried the cursor.foreach() function, which allowed me to divide the computation time by 2 again !
In production, the benchmarks allowed me to go from a 7/8 second response time to an average response time of 1.6 second :)
Hope this will be usefull to you people ! :)

Enyo Collection Merge Strategy

Below is the function I have used to fetch more photos from a service provider once clicked on more button:
showMore: function(){
this.$.resultList.collection.fetch({strategy:"merge",rpp:50});
}
This will call the fetch method defined in collection,
fetch: function(opts) {
this.params = {
feature: this.methodType || "popular",
image_size: 3,
sort: "created_at",
rpp: opts && opts.rpp || 25
};
return this.inherited(arguments);
},
This is working fine, but the problem is once more button is clicked, it should fetch another set of 25 photos and append it to my collection, but what actually happening is sometimes, it shows only say 5 photos along with previous photos .
What I understand by "merge" strategy is, if the records received is same as previous records, it will take only those records which are different from previously fetched records and updates the primarykey of the duplicate records.So one reason i am able to figure out is that, may be, it is fetching 25 photos next time, but because most of them are same as before, it is showing only those which are different from the previous fetched photos.
If i go with the "add" strategy, it works fine for one time, i.e, it shows another set of 25 photos appended to the collection, most of them are again same. But if click on more button one more time, no records are being added to the collection.No idea why this is happening.
How should i approach, if i want to fetch only new photos and append it to the collection.

Using the merge strategy is the right approach. Your description of merge is mostly accurate except that it doesn't update the primary key but instead updates the data for existing records with the same primary key.
It's difficult to say why using "add" doesn't always work. If the records don't have a primary key (which is id by default), "add" and "merge" should always add the records to the collection (unless you're using mergeKeys). If they do have a primary key, it's possible that you're trying to add duplicate records which Enyo will complain about and abort. Check your console log.
Without code, the only other suggestion is to set breakpoints and step through enyo.Collection.merge.
Here's an example of fetching records into a collection. If you comment out setting the id, merge and add strategies will always add records. If you comment out the merge strategy, the code will eventually error when requesting more records.
enyo.kind({
name: "ex.MockSource",
kind: "enyo.Source",
fetch: function(rec, opts) {
if(rec instanceof enyo.Model) {
rec.setObject(Faker.Helpers.createCard());
} else if (rec instanceof enyo.Collection) {
var count = opts && opts.count || 25;
var cards = [];
for(var i=0;i<count;i++) {
var card = Faker.Helpers.createCard();
// artificial id to force merges
card.id = enyo.now()%40;
cards.push(card);
}
opts.success(cards);
}
}
});
enyo.store.addSources({
mock: "ex.MockSource"
});
enyo.kind({
name: "ex.App",
kind: "Scroller",
bindings: [
{from: ".data", to: ".$.list.collection"},
{from: ".data.length", to: ".$.count.content", transform: function(v) {
return enyo.format("Displaying %s records", v);
}}
],
components: [
{name: "count"},
{name: "list", kind: "DataRepeater", components: [
{kind: "onyx.Item", components: [
{name: "name"},
{name: "phone"}
], bindings: [
{from: ".model.name", to: ".$.name.content"},
{from: ".model.phone", to: ".$.phone.content"}
]}
]},
{kind: "onyx.Button", content: "More", ontap: "moreTapped"}
],
create: enyo.inherit(function(sup) {
return function() {
sup.apply(this, arguments);
this.set("data", new enyo.Collection({
defaultSource: "mock"
}));
this.fetchRecords();
};
}),
fetchRecords: function() {
this.data.fetch({
count: 5,
strategy: "merge"
});
},
moreTapped: function() {
this.fetchRecords();
}
});
new ex.App().renderInto(document.body);

Get object within JSON object only by knowing its position, and not name

I'm currently looking into the Twitter-API - specifically the daily trends-API (http://search.twitter.com/trends/current.json).
Example return from Twitter
{
trends: {
2009-11-19 14:29:16: [
{
name: "#nottosayonfirstdate",
query: "#nottosayonfirstdate"
},
{
name: "New Moon",
query: ""New Moon""
},
{
name: "#justbecause",
query: "#justbecause"
}
]
}
}
I wonder how I can return the values within there without knowing the exact date at the time of the call, since it won't be possible to synchronise the client-time with the server-time.
Normally, I'd go for trends.something[1].name to get the name, but since the timestamp will change all the time, how can I get it when trends is no array?

you can use this:
for (var i in trends) {
alert (i); // "2009-11-19 14:29:16"
alert (trends[i][0].name); // "#nottosayonfirstdate"
}