This code is sorting by different fields within the document and some of those fields are arrays of objects with the key I want to sort by. I don't understand the behavior that I am seeing when I run the following queries. I don't have a lot of experience with mongodb and didn't write these queries.
const dbCursor = connection.collection('container').find({});
cursor.sort({ "titles.title": 1 });
// These don't happen one after the other as shown here. It's either or.
cursor.sort({ "dates.start": 1 });
Titles and dates are both arrays of objects containing the key passed. Dates appears to be sorting by start date even though there may be multiple object with the the key start in it. Title is not sorting alphabetically and appears to be very random. I don't understand what is actually happening when this type of sort is performed in MongoDB.
How is Mongodb handling the array?
Is it only checking the first element in the array?
Is it checking all the elements in the array?
Is there a better way to perform this type of sorting when dealing with arrays?
// one
{ "titles": [{ "title": "Zippy Mississippi Race" }, { "title": "Wacky
Races"}] }
// two
{ "titles": [{ "title": "New Looney Tunes" }, { "title": "Your Bunny
or Your Life/Misjudgment Day" }] }
// three
{ "titles": [{ "title": "Why Oh Why Wyoming" }, { "title": "Wacky Races" }] }
Returns in this order
Update:
So I have discovered that its sorts all the elements in the array. I just don't understand how it determines who the winner is. Can anyone explain why this order is correct based on sorting all the elements in the array?
Related
I'm working on a POC to pull data from various liquidity pools (paired tokens, i.e. WEI/USDT from various exchanges.
In trying to create something like the DAI chart seen here:
I am trying to come up with a query and data model in JavaScript to contain this data.
The given would be "DAI". First get Uniswap results with DAI pools (any pool pairs containing "DAI"). Then get a list of results from SushiSwap of matching "WETH". Since both sources will likely not have all matching pools, with these two lists in memory, create a list of all items that match, i.e. USDT/WETH (matching in green in the image above).
I initially was going to create an associative array with a list of tokens to match:
poolList["Uniswap"] = { collection of pool objects }
poolList["Sushiswap"] = { collection of pool objects }
Where the collection data would looks something like
{
"data": {
"pools": [
{
"token0": {
"id": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2",
"name": "Wrapped Ether",
"symbol": "WETH"
},
"token1": {
"id": "0xd1063ee5ec2891991a29fefb52bcc448cd386844",
"name": "BanDogge Mastiff",
"symbol": "DOGGE"
}
},
{
"token0": {
"id": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2",
"name": "Wrapped Ether",
"symbol": "WETH"
},
How would one store the data from various exchanges so that either a filters list exists of common pairs, or create some sort of 2D array reflecting how the chart above appears?
I went about looking at this at the ground level: What kind of data problem is this?
In addressing this, I was able to work through a suggested response to my question, What can be used to identify the source collection for common elements from n-number of collections? and came up with a solution that renders the data in a workable format:
The table does not exactly reflect the initial question, however,
(index) represents the pool name or token pairs, i.e. USDT/ETH. Subsequent columns represent exchanges and their properties that contain these token pairs or liquidity pools.
To maintain the context of the original question and to assist others, the full code solution can be found here.
I am trying to construct a JSON schema that meets the following:
Declares a top-level object with at least one property
The value of each property will be an array, each of which must contain exactly N items
Array items must be integers taken from the closed interval [J, K], or null
Integer items in each array must be unique within that array
There is no uniqueness constraint applied to null (so no implied relationship between N and the interval size K-J)
The problem I am running into is #4 and #5. It is easy enough to meet the first 3 requirements, plus part of the 4th, using this schema:
{
"$schema": "http://json-schema.org/draft/2019-09/schema#",
"type": "object",
"minProperties": 1,
"additionalProperties": {
"type": "array",
"minItems": N,
"maxItems": N,
"items": {
"anyOf": [
{
"type": "integer",
"minimum": J,
"maximum": K
},
{
"type": "null"
}
]
},
"uniqueItems": true
}
}
I am not sure how (or if it's even possible) to specify an array that applies the uniqueItems constraint to only a subset of the allowable items. I tried moving uniqueItems to lower levels of the schema with the hope that it might operate with restricted scope, but that doesn't work.
This might be possible using conditionals, but I haven't gone down that road yet since I'm not sure it will actually work, and I am hoping there is an easier approach that I have overlooked.
So, my question is: Is there a way to specify a JSON schema array that selectively enforces a uniqueness constraint only on the items that are not null?
this is beyond the capabilities of uniqueItems and not a constraint JSON Schema is able to express. you will need to check this requirement elsewhere in your application's business logic.
I'm building an app where I need to delete items stored in the database. Here's a (shortened) example of user data I have in my DynamoDB table called 'registeredUsers':
{
"userId": "f3a0f858-57b4-4420-81fa-1f0acdec979d"
"aboutMe": "My name is Mary, and I just love jigsaw puzzles! My favourite jigsaw category is Architecture, but I also like ones with plants in them.",
"age": 27,
"email": "mary_smith#gmail.com",
"favourites": {
"imageLibrary": [
{
"id": "71ff8060-fcf2-4523-98e5-f48127d7d88b",
"name": "bird.jpg",
"rating": 5,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/bird.jpg"
},
{
"id": "fea4fd2a-851b-411f-8dc2-1ae0e144188a",
"name": "porsche.jpg",
"rating": 3,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/porsche.jpg"
},
{
"id": "328b913f-b364-47df-929d-925676156e97",
"name": "rose.jpg",
"rating": 0,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/rose.jpg"
}
]
}
}
I want to be able to delete the item 'rose.jpg' in the user.favourites.imageLibrary array. In order to select the correct user, I can provide the userId as the primary key. Then, in order to select the correct image in the array, I can pass the AWS.DocumentClient the 'id' of the item in order to delete it. However, I'm having trouble understanding the AWS API Reference docs. The examples given in the developer guide do not describe how to delete an item by looking at one of it's attributes. I know I have to provide an UpdateExpression and an ExpressionAttributeValues object. When I wanted to change a user setting, I found it pretty easy to do:
const params = {
TableName: REGISTERED_USERS_TABLE,
Key: { userId },
UpdateExpression: "set userPreferences.difficulty.showGridOverlay = :d",
ExpressionAttributeValues: {
":d": !showGridOverlay
},
ReturnValues: "UPDATED_NEW"
};
To conclude, I need a suitable Key, UpdateExpression and ExpressionAttributeValues object to access the rose.jpg item in the favourites array.
Unfortunately, the UpdateExpression syntax is not as powerful as you would have liked. It supports entire nested documents inside the item, but not sophisticated expressions to search in them or to modify them. The only ability it gives you inside a list is to access or modify its Nth element. For example:
REMOVE #favorites.#imagelibrary[3]
Will remove the 3rd element of imagelibrary (note that the "#imagelibrary" will need to be defined in ExpressionAttributeNames), and you can also have a condition on #favorites.#imagelibrary[3].#id, for example, in ConditionExpression. But unfortunately, there is no way to specify more complex combinations of conditions and updates, such as "find me the i where #favorites.#imagelibrary[i].#id is equal something, and then REMOVE this specific element".
Your remaining option is to read the full value of the item (or with ProjectionExpression just the #favorties.#imagelibrary array), and then in your own code find which of the elements you want to remove (e.g., discover that it is the 3rd element), and then in a separate update, remove the 3rd element.
Note that if there's a possibility that some other parallel operation also changes the item, you must use a conditional update (both UpdateExpression and ConditionExpression) for the element removal, to ensure the element that you are removing still has the id you expected. If the condition fails, you need to repeat the whole operation again - read the modified item again, find the element again, and try to remove it again. This is an example of the so-called "optimistic locking" technique which is often used with DynamoDB.
I have done tutorials and have seen examples where the array list item that is dynamically generated works exactly as intended. For some reason, in the code I am having to map in this particular array list is not generating the array list in any discernible order.
Reason being... From what I seen every time an action is added the key and index and other parameter is "adjusted" on the fly... leading to something completely random.
***Updated the description to be much more accurate to what is occuring based on my code base. I have learned we are using imuttable.js
More info here https://facebook.github.io/immutable-js/ and because of this there are certain things happening prior to the object being mapped...
For example we are using filter for a json object in our this.props
it is written as follows:
const filteredConstant = jsonObjects.filter((jsonObject) => {
return (jsonObject.status === status.APPROVED)
}
From the documentation in immutable it says filter will do the following
Seq({a:1,b:2,c:3,d:4}).filter(x => x % 2 === 0)
// Seq { b: 2, d: 4 }
From here we can rework the const / variable and now use map() to map the object. The below code did not work in the sense that open adding items to the object of jsonObject2 would produce random results. The key and even if I added an index there would be no rhyme or reason as to what the order would be... intended result would be oldest - newest data entry or ascending.
filteredConstant.valueSeq().map((jsonObjectNew, //index) => {
// code here
key={jsonObjectNew.id} //index={index} (//doesn't work either way)
}
apparently valueSeq() doesn't have an affect. Introducing a timestamp appended to a unique id doesn't work either.
Like I said previously it seems as if every-time there is a new entry it "remaps" and thus reorders randomly what is going on. I could be interpreting what the key actually is... I still haven't figured out how and where the key is being generated ideally from the server side. But even in terms of a key that is base64 for example would still have to go in some type of order based on letters and numerics? Am I wrong to think this?
Alas, I was able to create a fix and I will share that below. But more insight into this would be appreciated.
In jsfiddle I cannot replicate the issue but if any help can be offered on what exactly controls the array order and or new data entering into the object would said action have on the effect of the overall order?
So I was able to do a fix and there is something that I had to grasp in order to understand what the map'ed list was not in any type of order.
The answer lies in the fact that I was mapping an object... It wasn't an array or map object that was being inserted to dynamically... It was simply an object that for all intents and purposes is a dictionary of dictionaries.
Per the example react gives they dynamically build and array that is of an order per insertion... new item at the bottom. here is what that array looks like.
[
{
"id": 1388534400000,
"author": "Pete O'malley",
"text": "Hey there!"
},
{
"id": 1420070400000,
"author": "Paul O’Shannessy",
"text": "React is *great*!"
},
{
"id": 1470769287060,
"author": "Christian",
"text": "how are you"
}
] = JSON.stringify(this.props.data, null, 4)
and so on...
My object that was being used was not like this at all... It is just an object of objects.
Here is an example with 2 entries.
{
"93b1keyId": {
"xxxxx": "",
"xxxxx": {
"xxxx": "xxxx",
"xxxxx": [
{
"xxx": "xxxx",
"xxxxx": "xxxxx "
}
],
"xxxxx": "",
"xxxxx": "",
//code here
},
"Date": "2016-08-16",
"xxxxxx": "xxxxxxxx",
"timeCreated": "2016-08-16 17:49:39 pm -0400",
"xxxxx": {
"xxxxxx": "xxxxxx",
"xxxxxxx": "xxxx"
},
"xxxxxx": xxxxxx,
"id": "93b1KeyId",
"xxxxxx": "xxxxx"
},
"ed79KeyId": {
"xxxxx": "",
"xxxxx": {
"xxxx": "xxxx",
"xxxxx": [
{
"xxx": "xxxx",
"xxxxx": "xxxxx "
}
],
"xxxxx": "",
"xxxxx": "",
//code here
},
"Date": "2016-08-16",
"xxxxxx": "xxxxxxxx",
"xxxxx": {
"xxxxxx": "xxxxxx",
"xxxxxxx": "xxxx"
},
"xxxxxx": xxxxxx,
"id": "ed79613e-d4bf-4fb4-993f-c212993d5d3b",
"xxxxxx": "xxxxx"
}
} = JSON.stringify(jsonObjectOriginal, null, 4)
So as you see this is not an array... it is just a dictionary of dictionaries. objects in objects.
Per this entry I am able to confirm with reasonable assurance that order of dictionary objects are not ordered.
Stack question regarding property order
In particular see section 12.6.4 of the ECMAScript specification:
The mechanics and order of enumerating the properties ... is not specified.
&
4.3.3 Object
An object is a member of the type Object. It is an unordered collection of properties each of which contains a primitive value, object, or function. A function stored in a property of an object is called a method.
What was confusing about the situation is that it is a little misleading to consider "mapping" to be iterative of insertion order if what it is mapping is an unordered object.
Per the mozilla their documentation states the following.
A Map object iterates its elements in insertion order — a for...of loop returns an array of [key, value] for each iteration.
Again, that is only if you are inserting to the actuall map object itself... which for me was not the case.
So, the answer...
I had to re-sort the object upon mapping the React function which displays to the UI...
I used sortBy which is an immutable sub method of the map() method.
Doc information can be found here: immutable sortBy method
{filteredConstant.sortBy(sortBy => sortBy.timeCreated).map((jsonObject2) => {
//code here
So now, the new mapped object is reordered correctly based on insertion time and displayed to the UI correctly. I enjoyed learning / working through this one!!! but I feel mozilla should perhaps update that page.
I verified my "theory" by studying the jsonObjectOriginal and jsonObjectNew... The original was the one taking in random order.
I am quite new to this languages and trying to make sense of what's going on. I have managed to get data from an external JSON file and create a list from it.
This is the contents from the JSON file:
{
"player": [
{
"name": "John",
"country": "USA",
"score": 102400
},
{
"name": "Mary",
"country": "Australia",
"score": 80001
},
{
"name": "Jane",
"country": "England",
"score": 103900
}
]
}
Now here is the fiddle with the HTML and js.
http://jsfiddle.net/tusika_/ut3NZ/
As you can see, every ul is wrapped in a div with class "player". What I would like to achieve is to be able to sort those divs of class "player", by sorting alphabetically the name (default) or country or descending score of the players.
After two days of research and finding answers to similar questions, I managed to put the data into an array, and when I use the sort method and the function in the js, i see in the console that the objects do get sorted differently, however they only sort alphabetically for the first three objects and then the last two get not sorted (in the original file I have many more players than three).
Also I do not undestand how to reprint of screen that new order. (it should replace the current output each time)
I would appreciate a response that indicates where the error of the logic is and doesn't only provide the code but helps me understand why the code is such.
Thank you very much!!!
The issue is (as you have noticed) the disconnection of the array order and the DOM order. You only use the array to create the DOM elements. They are not somehow linked so that what happens to one affect the other.
You will have to manually redraw the dom by either emptying the container and redrawint the element, or by re-arranging the existing DOM elements. For example you could have a function that will clear the #list element and then append the sorted nodes.
function displayData(array) {
var list = $("#list").empty();
$.each(array, function () {
list.append("<div class='player'><ul><li>" + this['name'] + "</li><li>" + this['country'] + "</li><li>" + this['score'] + "</li></ul></div>");
});
}
Also you do not need to sort the array while adding each element, just sort the whole of the array once.
So you can use the above code like this
var sorted = data.player.sort(byCountry);
displayData(sorted);
You can see a simple demo at http://jsfiddle.net/gaby/YhvTt/
When you're doing array.push(key, value);, you're pushing both the key and the value in the array (at position i and i+1).
OH, and BTW, you can simply do: data.player.sort(compare);