We Keep Coding

MongoDB get ArrayFilters reference to update value

I recently started using MongoDB and I'm trying to update a few properties from a document but not being able to get the object reference to update a value.
Please consider the following data:
const data = {
weekplanId: 'someid',
days: [
{label: 'Monday', cost: 20, comment: "some comment" },
{label: 'Tuesday', cost: 40, comment: "..." }
]
}
const update = await weekplan.updateOne(
{
_id: new ObjectId(data.weekplanId),
},
{
$set: {
"weekdays.$[i].cost": data.days.$[i].cost,
"weekdays.$[i].comment": data.days.$[i].comment,
"weekdays.$[i].date": new Date(),
"weekdays.$[i].someproperty": "something",
}
},
{
arrayFilters: [
{
"i.label": {
$in: data.days.map(p => p.label),
},
}]
}
);
How can I reference the array object to set the property value?
I know data.days.$[i].cost and data.days.$[i].comment are wrong, they are just an example of what I'm trying to achieve.
Setting the date and someproperty works as expected, since the values are not dependent on the source data.
I would like to try to do this without using JS.
Is arrayFilters even appropriate for this? I'd appreciate some guidance as to where to look at.
Thanks in advance!
###EDIT:
Expected output:
"_id": {"someid"},
"weekdays": [
{
"day": "Monday",
"cost": 20,
"comment": "some comment",
"date": 2021-08-01T19:51:45.057Z
"someproperty": "something"
},
{
"day": "Tuesday",
"cost": 40,
"comment": "...",
"date": 2021-08-01T19:51:45.057Z
"someproperty": "something"
},
...
...
...
]
The rest of the week days (Wednesday, Thursday, Friday) would remain untouched on this update.

In that example code data.days.$[i].cost is evaluated on the client side, before the query is submitted, so whatever value (or lack thereof) that is has will be assigned to the corresponding field of the $set object when the server receives it.
The data object will not be sent to the server with the query, so even if it were able to do array lookup on input values, the input value would not be there.
The way to accomplish this is to iterate the array on the client side, and programmatically build the update query. Perhaps something like:
let labelChar = 105;
let setOps = {};
let filters = {};
data.days.forEach( function(day) {
let char = String.fromCharCode(labelChar++);
setOps['weekdays.$[' + char + '].cost'] = day.cost;
setOps['weekdays.$[' + char + '].comment'] = day.comment;
setOps['weekdays.$[' + char + '].date'] = new Date();
setOps['weekdays.$[' + char + '].someproperty'] = "something";
let filterObj = {};
filterObj[char + '.label'] = day.label;
filters.push(filterObj);
});
const update = await weekplan.updateOne(
{
_id: new ObjectId(data.weekplanId),
},
{
$set: setOps
},
{
arrayFilters: filters
}
);
For the provided sample input, this will give the update:
.updateOne(
{
_id: new ObjectId(data.weekplanId),
},
{
$set: {
'weekdays.$[i].cost': 20,
'weekdays.$[i].comment': 'some comment',
'weekdays.$[i].date': ISODate(),
'weekdays.$[i].someproperty': 'something',
'weekdays.$[j].cost': 40,
'weekdays.$[j].comment': '...',
'weekdays.$[j].date': ISODate(),
'weekdays.$[j].someproperty': 'something'
}
},
{
arrayFilters: [
{'i.label': 'Monday'},
{'j.label': 'Tuesday'}
]
}
);

Elasticsearch - find closest number when scoring results

I need a way to match the closest number of an elasticsearch document.
I'm wanting to use elastic search to filter quantifiable attributes and have been able to achieve hard limits using range queries accept that results that are outside of that result set are skipped. I would prefer to have the closest results to multiple filters match.
const query = {
query: {
bool: {
should: [
{
range: {
gte: 5,
lte: 15
}
},
{
range: {
gte: 1979,
lte: 1989
}
}
]
}
}
}
const results = await client.search({
index: 'test',
body: query
})
Say I had some documents that had year and sales. In the snippet is a little example of how it would be done in javascript. It runs through the entire list and calculates a score, then based on that score it sorts them, at no point are results filtered out, they are just organized by relevance.
const data = [
{ "item": "one", "year": 1980, "sales": 20 },
{ "item": "two", "year": 1982, "sales": 12 },
{ "item": "three", "year": 1986, "sales": 6 },
{ "item": "four", "year": 1989, "sales": 4 },
{ "item": "five", "year": 1991, "sales": 6 }
]
const add = (a, b) => a + b
const findClosestMatch = (filters, data) => {
const scored = data.map(item => ({
...item,
// add the score to a copy of the data
_score: calculateDifferenceScore(filters, item)
}))
// mutate the scored array by sorting it
scored.sort((a, b) => a._score.total - b._score.total)
return scored
}
const calculateDifferenceScore = (filters, item) => {
const result = Object.keys(filters).reduce((acc, x) => ({
...acc,
// calculate the absolute difference between the filter and data point
[x]: Math.abs(filters[x] - item[x])
}), {})
// sum the total diffences
result.total = Object.values(result).reduce(add)
return result
}
console.log(
findClosestMatch({ sales: 10, year: 1984 }, data)
)
<script src="https://codepen.io/synthet1c/pen/KyQQmL.js"></script>
I'm trying to achieve the same thing in elasticsearch but having no luck when using a function_score query. eg
const query = {
query: {
function_score: {
functions: [
{
linear: {
"year": {
origin: 1984,
},
"sales": {
origin: 10,
}
}
}
]
}
}
}
const results = await client.search({
index: 'test',
body: query
})
There is no text to search, I'm using it for filtering by numbers only, am I doing something wrong or is this not what elastic search is made for and are there any better alternatives?
Using the above every document still has a default score, and I have not been able to get any filter to apply any modifiers to the score.
Thanks for any help, I new to elasticsearch links to articles or areas of the documentation are appreciated!

You had the right idea, you're just missing a few fields in your query to make it work.
It should look like this:
{
"query": {
function_score: {
functions: [
{
linear: {
"year": {
origin: 1984,
scale: 1,
decay: 0.999
},
"sales": {
origin: 10,
scale: 1,
decay: 0.999
}
}
},
]
}
}
}
The scale field is mandatory as it tells elastic how to decay the score, without it the query just fails.
The decay field is not mandatory, however without it elastic does not really know how to calculate the new score to documents so it will end up giving a default score only to documents in the range of origin + scale which is not useful for us.
source docs.
I also recommend you limit the result size to 1 if you want the top scoring document, otherwise you'll have add a sort phase (either in elastic or in code).
EDIT: (AVOID NULLS)
You can add a filter above the functions like so:
{
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"bool": {
"filter": [
{
"bool": {
"must": [
{
"exists": {
"field": "year"
}
},
{
"exists": {
"field": "sales"
}
},
]
}
}
]
}
},
{
"match_all": {}
}
]
}
},
"functions": [
{
"linear": {
"year": {
"origin": 1999,
"scale": 1,
"decay": 0.999
},
"sales": {
"origin": 50,
"scale": 1,
"decay": 0.999
}
}
}
]
}
}
}
Notice i have a little hack going on using match_all query, this is due to filter query setting the score to 0 so by using the match all query i reset it back to 1 for all matched documents.
This can also be achieved in a more "proper" way by altering the functions, a path i choose not to take.

How can I give my JSON dictionary a main key, without implementing it into my loop?

I am translating some code from python into Javascript (I am inexperienced in JS), as part of it I need to write a JSON, currently when writing in JS it looks like this(just a brief sample):
[
{
"Num": "000000",
"Status": 1,
},
{
"Num": "00001",
"Status": 0,
},
]
However I need it to look like this:
{
"mydata": [
{
"Num": "00000",
"Status": 1,
},
{
"Num": "00001",
"Status": 0,
},
]
}
How can I adapt my code to generate this single main key for the whole JSON, here is what I have so far:
var jsondata = []
for (result in results) {
jsondata.push({
'Num' : idnum,
'Status': results[result].StatusID,
})
}
let data = JSON.stringify(jsondata, null, 2)
fs.writeFileSync('testingjson.json', data)
This code here sits in a for loop, so I cannot just write the key in the push, it would generate the mydata key for every iteration of the loop.
I need to know how I can pre-define the mydata, has anyone got a good method to do this?

Just define mydata as an array, and then at the end, create an object where mydata is one of the keys:
const mydata = []
for (const result in results) {
mydata.push({
'Num' : idnum,
'Status': results[result].StatusID,
});
}
const json = JSON.stringify({ mydata }, null, 2);
If you want to use a different key name, then change it when stringifying, eg:
const json = JSON.stringify({ someKeyName: mydata }, null, 2);
If results is an object, then you can make the code more concise with Object.values and map:
const Num = idnum;
const mydata = Object.values(results)
.map((item => ({ Num, Status: item.StatusID }));
const json = JSON.stringify({ mydata }, null, 2);
fs.writeFileSync('testingjson.json', data)

Mongoose - Increment a value inside an array of objects

I am having hard time figuring out how to increment a value in an object within an array
For instance I have this document based on Poll schema:
{
"_id": "584b2cc6817758118e9557d8",
"title": "Number of Skittles",
"description": "Test1",
"date": "Dec 9, 2016",
"__v": 0,
"labelOptions": [
{
"Bob": 112
},
{
"Billy": 32
},
{
"Joe": 45
}
]
}
Using express, I am able to get this far:
app.put('/polls/:id', function(req, res){
let id = req.params.id;
let labelOption = req.query.labelOption;
Poll.findOneAndUpdate(
{'_id' : id},
{$inc: {`labelOptions.$.${labelOption}`: 1 }},
function(err){
console.log(err)
})
where labelOption is the one that I would like to increment its value
To be more concise, I am having trouble transversing inside the document.

It is not possible to directly increment the value in the .find query if labelOptions is an Array of Object. To make it easier, you should change the labelOptions type from Array of Objects to Object:
"labelOptions": {
"Bob": 112,
"Billy": 32,
"Joe": 45
};
Also consider using .findByIdAndUpdate instead of .findOneAndUpdate if you are querying by the document's _id. And then, you can achieve what you want by:
Poll.findByIdAndUpdate(
id,
{$inc: {`labelOptions.${labelOption}`: 1 }},
function(err, document) {
console.log(err);
});
UPDATE: If you are persistent on using Array of Objects for labelOptions, there is a workaround:
Poll.findById(
id,
function (err, _poll) {
/** Temporarily store labelOptions in a new variable because we cannot directly modify the document */
let _updatedLabelOptions = _poll.labelOptions;
/** We need to iterate over the labelOptions array to check where Bob is */
_updatedLabelOptions.forEach(function (_label) {
/** Iterate over key,value of the current object */
for (let _name in _label) {
/** Make sure that the object really has a property _name */
if (_label.hasOwnProperty(_name)) {
/** If name matches the person we want to increment, update it's value */
if (_name === labelOption) ++_label._name;
}
}
});
/** Update the documents labelOptions property with the temporary one we've created */
_poll.update({labelOptions: _updatedLabelOptions}, function (err) {
console.log(err);
});
});

There is another way to do this which allows a more flexible document model. If you add a field to your object like:
{
"_id": "584b2cc6817758118e9557d8",
"title": "Number of Skittles",
"description": "Test1",
"date": "Dec 9, 2016",
"__v": 0,
"labelOptions": [
{
"name": "Bob",
"number": 112
},
{
"name": "Billy",
"number": 32
},
{
"name": "Joe"
"number": 45
}
]
}
Then you can do:
app.put('/polls/:id', function(req, res){
let id = req.params.id;
let labelOption = req.query.labelOption;
Poll.findOneAndUpdate(
{
'_id' : id,
'labelOptions.name
},
{$inc: {
`labelOptions.$.number`: 1
}},
function(err){
console.log(err)
})

Populate Highchart Column From JSON

I have to create a column chart in my project using Highchart. I am using $.ajax to populate this data. My current JSON data is like this :
[{
"city": "Tokyo",
"totalA": "10",
"totalB": "15"
},
{
"city": "Seoul",
"totalA": "20",
"totalB": "27"
},
{
"city": "New York",
"totalA": "29",
"totalB": "50"
}]
How to resulting JSON string look like this:
[{
"name": "city",
"data": ["Tokyo", "Seoul", "New York"]
}, {
"name": "totalA",
"data": [10, 20, 29]
}, {
"name": "totalB",
"data": [15, 27, 50]
}]
Thank you.

Assuming all the elements look the same (they all have the same fields): Live Example
// src is the source array
// Get the fields from the first element
var fields = Object.keys(src[0]);
// Map each key to...
var result = fields.map(function(field) {
// Grab data from the original source array
var data = src.reduce(function(result, el) {
// And create an array of data
return result.concat(el[field]);
}, []);
// Format it like you want
return {name: field, data: data};
});
console.log(result);
If they aren't, the code is slightly more complicated: Live Example
// Work out the fields by iterating all of the elements
// and adding the keys that weren't found yet to an array
var fields = src.reduce(function (fields, current) {
return fields.concat(Object.keys(current).filter(function (key) {
return fields.indexOf(key) === -1;
}));
}, []);
var result = fields.map(function (field) {
// Need another step here, filter out the elements
// which don't have the field we care about
var data = src.filter(function (el) {
return !!el[field];
})
// Then continue like in the example above.
.reduce(function (result, el) {
return result.concat(el[field]);
}, []);
return {
name: field,
data: data
};
});
console.log(result);