We Keep Coding

How do I do a MongoDB left join, keeping documents with no match on the right?

I'm a little new to this, so forgive me if this is actually trivial. I'm trying to do a left outer join in MongoDB/NodeJS. I would like all matching documents from the left table, even if they have no match on the right. I have a collection of questions:
{_id: 'question1', text: 'This is a question.'}
{_id: 'question2', text: 'This is another question.'}
And a collection of responses, each tied to a user and a question:
{_id: 'response1', QID: 'question1', UID: 'player1', response: 'This is my answer.'}
Now, I'd like to get a list of questions and a user's response to each one, including ones where there is no recorded response, so what I'd want from the documents above might be...
{_id: 'question1', text: 'This is a question.', response: {_id: 'response1', QID: 'question1', UID: 'player1', response: 'This is my answer'}}
{_id: 'question2', text: 'This is another question.', response: []}
Is there a way to do this in the aggregation pipeline? When I use lookup to join responses to questions, and then match the UID, the second question disappears because there's no response tied to it (and thus, fails to satisfy the UID match).
Edit: another thing I tried was to use a let/pipeline in the lookup stage:
const month = 11;
const year = 2020;
const UID = 'aaaaaaa';
//Only get responses by UID 'aaaaaaa'
const myResponses = await Question.aggregate([
{ $match: { year, month } },
{
$lookup: {
from: Response.collection.name,
let: { questionID: '$_id' },
pipeline: [
{
$match: {
$expr: {
$and: [{ $eq: ['$$questionID', '$QID'] }, { $eq: ['$UID', UID] }],
},
},
},
],
as: 'Response',
},
},
This was close, but for some reason, matching the UIDs doesn't seem to work, as it returns an empty array for the Response on every clue. Taking out the last $eq condition and just matching $$questionID and $QID gets every response to every question, as I would've expected, but trying to check for equality to a constant isn't working.
Thanks in advance!

https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/ per the docs
Performs a left outer join

Why javascript array sort doesn't compare all elements with each other?

I'm trying to understand why Javascript array sort doesn't work with the following logic. I have no problems making my own algorithm to sort this array, but I'm trying to make it with the Javascript sort built-in method to understand it better.
In this code, I want to push entities that "belongs to" another entity to the bottom, so entities that "has" other entities appear on the top. But apparently, the sort method doesn't compare all elements with each other, so the logic doesn't work properly.
Am I doing something wrong, or it is the correct behavior for the Javascript sort method?
The code I'm trying to execute:
let entities = [
{
name: 'Permission2',
belongsTo: ['Role']
},
{
name: 'Another',
belongsTo: ['User']
},
{
name: 'User',
belongsTo: ['Role', 'Permission2']
},
{
name: 'Teste',
belongsTo: ['User']
},
{
name: 'Role',
belongsTo: ['Other']
},
{
name: 'Other',
belongsTo: []
},
{
name: 'Permission',
belongsTo: ['Role']
},
{
name: 'Test',
belongsTo: []
},
]
// Order needs to be Permission,
let sorted = entities.sort((first, second) => {
let firstBelongsToSecond = first.belongsTo.includes(second.name),
secondBelongsToFirst = second.belongsTo.includes(first.name)
if(firstBelongsToSecond) return 1
if(secondBelongsToFirst) return -1
return 0
})
console.log(sorted.map(item => item.name))
As you can see, "Role" needs to appear before "User", "Other" before "Role", etc, but it doesn't work.
Thanks for your help! Cheers

You're running into literally how sorting is supposed to work: sort compares two elements at a time, so let's just take some (virtual) pen and paper and write out what your code is supposed to do.
If we use the simplest array with just User and Role, things work fine, so let's reduce your entities to a three element array that doesn't do what you thought it was supposed to do:
let entities = [
{
name: 'User',
belongsTo: ['Role', 'Permission2']
},
{
name: 'Test',
belongsTo: []
},
{
name: 'Role',
belongsTo: ['Other']
}
]
This will yield {User, Test, Role} when sorted, because it should... so let's see why it should:
pick elements [0] and [1] from [user, test, role] for comparison
compare(user, test)
user does not belong to test
test does not belong to user
per your code: return 0, i.e. don't change the ordering
we slide the compare window over to [1] and [2]
compare(test, role)
test does not belong to role
role does not belong to test
per your code: return 0, i.e. don't change the ordering
we slide the compare window over to [2] and [3]
there is no [3], we're done
The sorted result is {user, test, role}, because nothing got reordered
So the "bug" is thinking that sort compares everything-to-everything: as User and Role are not adjacent elements, they will never get compared to each other. Only adjacent elements get compared.

Comparing two unsorted collections in MongoDB

I am trying to compare a large number of documents in two collections. To give you an estimate, I have around 1300 documents in each of the two collections.
I want to generate a diff comparison report after comparing the two collections. I do not need to point out exactly what is missing or what new content has been added, I just need to be able to identify that there is in fact some difference between the two documents. Yes, I do have a unique identifier for each documents other than Mongo's ObjectId ("_id").
Note: I have implemented the database using the denormalized data model, which means I have embedded documents (documents within documents).
What would you say is the best way to go about implementing a solution for the same?
Thank you in advance for your time samaritans!

You should use $lookup and $eq on all the fields you care about.
db.collection1.aggregate([
{
$lookup:
{
from: "collection2",
let: { unique_id: "$unique_id", field1: "$field", field2: "$field", ... },
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$unique_id_in_2", "$$unique_id" ] }
{ $eq: [ "$field_to_match", "$$field1" ] },
{ $eq: [ "$field_to_match.2", "$$field2" ] }
]
}
}
},
],
as: "matches"
}
},
{
$match: {
'matches.0': {$exists: false}
}
}
])
** mongo 3.6+ syntax for lookup.

Apollo Client replaces an array of objects with the same id and different values with an array of copies of the first object

Our GraphQL server responds to a query with data that includes an array of objects each of which shares the same id and different values for a different key. For instance, we might have an array that looks like:
[
{ id: 123, name: 'foo', type: 'bar', cost: 5 },
{ id: 123, name: 'foo', type: 'bar', cost: 6 },
{ id: 123, name: 'foo', type: 'bar', cost: 7 },
{ id: 123, name: 'foo', type: 'bar', cost: 8 }
]
We can see in the Network tab that the response from the server has the correct data in it. However, by the time it goes through processing by the Apollo Client module the array has been transformed into something that might look like this:
[
{ id: 123, name: 'foo', type: 'bar', cost: 5 },
{ id: 123, name: 'foo', type: 'bar', cost: 5 },
{ id: 123, name: 'foo', type: 'bar', cost: 5 },
{ id: 123, name: 'foo', type: 'bar', cost: 5 }
]
Essentially what we're seeing is that if all of the objects in an array share the same value for id then all objects in the array become copies of the first object in the array.
Is this the intended behavior of Apollo Client? We thought maybe it had something to do with incorrect caching, but we were also wondering if maybe Apollo Client assumed that subsequent array members with the same id were the same object.

It looks like this is behavior as intended. The Apollo Client normalizes on id.

As the other answer suggests this happens because Apollo normalises by ID. There's a very extensive article on the official blog that explains the rationale of it, along with the underlying mechanisms.
In short, as seen by Apollo's cache, your array of objects contains 4 instances of the same Object (id 123). Same ID, same object.
This is a fair assumption on Apollo's side, but not so much in your case.
You have to explicitly tell Apollo that these are indeed 4 different items that should be treated differently.
In the past we used dataIdFromObject, and you can see an example here.
Today, you would use typePolicies and keyfields:
const cache = new InMemoryCache({
typePolicies: {
YourItem: {
// Combine the fields that make your item unique
keyFields: ['id', 'cost'],
}
},
});
Docs

It works for me:
const cache: InMemoryCache = new InMemoryCache({ dataIdFromObject: o => false )};
previous answer solves this problem too!
Also you can change the key name(for example id => itemId) on back-end side and there won't be any issue!

I have the same issue. My solution is to set fetchPolicy: "no-cache" just for this single API so you don't have to change the InMemoryCache.
Note that setting fetchPolicy to network-only is insufficient because it still uses the cache.
fetchPolicy document

Having trouble wrapping my head around complex $group'ing/aggregation

I have a schema that is something like this:
{
_id: <objectid>
customer: <objectid>
employee: <objectid>
date: <Month/day/year>
amount: <Number>
}
Using angular, I'm trying to make a page that pulls that data and builds separate tables for each day. So something like I would have a tab for yesterday, that would open up a view for a table that has all of my employees listed and the sum of their for the day. Something like this:
[{
date: 10/29/2019
dataFromThisDate: [
{
employee: <name>
sumAmount: <sum(amount for this date)>
list: [<array of all of the transaction _ids
},
{
employee: <name 2>
//etc
}]
},
{
date: 10/30/2019
dataFromThisDate: //etc
}]
Basically as far as I've gotten is just:
MyCollection.aggregate(
[{
$group: {
_id: "$date"
}
}],function(err, result) { //blah }
)
But I'm not sure how to even do nested grouping (first by date, then by employee on that date). Just thinking through it, it feels like I would have to group by date, then pass on all the data to a new grouping pipeline?
Sorry I don't have more of what I've tried, this whole aggregation thing is just completely new to me and I can't find good examples that are similar enough to what I'm trying to do to learn from. I looked at the api docs for mongodb and I understand their basic examples and play around with them, but I'm just having a hard time coming up with how to do my more complex example.

You can try something like this. This uses two groups. First group by date and employee, summing the amount and adding the transaction ids. Second group by date and add the employees with their total amount and transactions list.
aggregate([{
$group: {
_id: {
date: "$date",
employee: "$employee"
},
amount: {
$sum: "$amount"
},
transactionIds: {
$push: "$_id"
}
}
}, {
$group: {
_id: "$_id.date",
dataFromThisDate: {
$push: {
employee: "$_id.employee",
sumAmount: "$amount",
list: "$transactionIds"
}
}
}
}])
Output
{
"_id": "12/21/2016",
"dataFromThisDate": [{
"employee": "employee1",
"sumAmount": 100,
"list": [ObjectId("58151e881ac3c9ce82782663")]
}, {
"employee": "employee2",
"sumAmount": 73,
"list": [ObjectId("58151e881ac3c9ce82782665"), ObjectId("58151e881ac3c9ce82782666")]
}]
}