Parsing Exception error when using Terms in ElasticSearch - javascript

I'm getting an error on this elastic search for terms. The error message is
"[parsing_exception] [terms] unknown token [START_ARRAY] after [activeIds], with { line=1 & col=63 }"
Active Ids is an array of unique ids. It sort of looks like
const activeIds = [ '157621a1-d892-4f4b-80ca-14feddb837a0',
'd04c5c93-a22c-48c3-a3b0-c79a61bdd923',
'296d40d9-f316-4560-bbc9-001d6f46858b',
'2f8c6c37-588d-4d24-9e69-34b6dd7366c2',
'ba0508dd-0e76-4be8-8b6e-9e938ab4abed',
'ab076ed9-1dd5-4987-8842-15f1b995bc0d',
'ea6b0cff-a64f-4ce3-844e-b36d9f161e6f' ]
let items = await es.search({
"index": table,
"body": {
"from": 0, "size": 25,
"query": {
"terms" : {
"growerId" : {
activeIds
}
},
"bool": {
"must_not": [
{ "match":
{
"active": false
}
},
],
"must": [
{ "query_string" :
{
"query": searchQuery,
"fields": ["item_name"]
}
}
],
}
}
}
})
Appreciate the help!
Edit: Answering this question- "What's the expected result? Can you elaborate and share some sample data? – Nishant Saini 15 hours ago"
I'll try to elaborate a bit.
1) Overall I'm trying to retrieve items that belong to active users. There are 2 tables: user and items. So I'm initially running an ES that returns all the users that contain { active: true } from the user table
2) Running that ES returns an array of ids which I'm calling activeIds. The array looks like what I've already displayed in my example. So this works so far (let me know if you want to see the code for that, but if I'm getting an expected result then I don't think we need that now)
3) Now I want to search through the items table, and retrieve only the items that contain one of the active ids. So an item should look like:
4) expected result is retrieve an array of objects that match the growerId with one of the activeIds. So if I do a search query for "flowers", a single expected result should look like:
[ { _index: 'items-dev',
_type: 'items-dev_type',
_id: 'itemId=fc68dadf-21c8-43c2-98d2-cf574f71f06d',
_score: 11.397207,
_source:
{ itemId: 'fc68dadf-21c8-43c2-98d2-cf574f71f06d',
'#SequenceNumber': '522268700000000025760905838',
item_name: 'Flowers',
grower_name: 'Uhs',
image: '630b5d6e-566f-4d55-9d31-6421eb2cff87.jpg',
dev: true,
growerId: 'd04c5c93-a22c-48c3-a3b0-c79a61bdd923',
sold_out: true,
'#timestamp': '2018-12-20T16:09:38.742599',
quantity_type: 'Pounds',
active: true,
pending_inventory: 4,
initial_quantity: 5,
price: 10,
item_description: 'Field of flowers' } },
So here the growerId matches activeIds[1]
But if I do a search for "invisible", which is created by a an inactive user, I get:
[ { _index: 'items-dev',
_type: 'items-dev_type',
_id: 'itemId=15200473-93e1-477c-a1a7-0b67831f5351',
_score: 1,
_source:
{ itemId: '15200473-93e1-477c-a1a7-0b67831f5351',
'#SequenceNumber': '518241400000000004028805117',
item_name: 'Invisible too',
grower_name: 'Field of Greens',
image: '7f37d364-e768-451d-997f-8bb759343300.jpg',
dev: true,
growerId: 'f25040f4-3b8c-4306-9eb5-8b6c9ac58634',
sold_out: false,
'#timestamp': '2018-12-19T20:47:16.128934',
quantity_type: 'Pounds',
pending_inventory: 5,
initial_quantity: 5,
price: 122,
item_description: 'Add' } },
Now that growerId does not match any of the ids in activeIds.
5) Using the code you helped with, it's returning 0 items.
Let me know if you need more detail. I've been working on this for a bit too long :\

Terms query accept array of terms so the terms query should be defined as below:
"terms": {
"growerId": activeIds
}
You might face other errors as well after making the above correction. So below is full query which might help you:
{
"from": 0,
"size": 25,
"query": {
"bool": {
"must_not": [
{
"match": {
"active": false
}
}
],
"must": [
{
"query_string": {
"query": searchQuery,
"fields": [
"item_name"
]
}
},
{
"terms": {
"growerId": activeIds
}
}
]
}
}
}

Related

Match by multiple objects in array in Mongoose

My data looks like this
{
"_id": "62f77d806f24c09f0acae163",
"name": "Test product",
"attributes": [
{
"attribute_name": "Shape",
"attribute_value": "Square"
},
{
"attribute_name": "Color",
"attribute_value": "Red"
}
]
}
I am using the aggregate method to filter results where I want to find products where "attribute_name" is "shape" and the "attribute_value" is "Square" AND "attribute_name" is "Color" and the "attribute_value" is "Red"
Basically I am building a filter feature in my application and basis the data passed to the API I want to get the products.
I have tried this:
let lookup = {
$match: {
$and: [
{
'attributes.attribute_label': 'Shape',
'attributes.attribute_value': {
$in: ['Square']
},
},
{
'attributes.attribute_label': 'Color',
'attributes.attribute_value': {
$in: ['Red']
},
}
],
}
};
let products = await productsModel.aggregate(lookup);
At first it seemed like it worked, but then I noticed it doesn't work properly, it matches
'attributes.attribute_value': {
$in: ['Red']
},
so if it finds "Red" in "attribute_label" which can be anything other than "Color" it will still return the results.
Any help is appreciated
I want to be able to get results based on the values for each attribute name
For e.g data passed might be this
Shape=Square,Color=Red,Green
I want to get the products which matches this, where the object with attribute_label of Color contains the attribute_value of Red or Green.
Is it a typo? Once you are using "attributes.attribute_label" and once "attributes.attribute_name".
This should work with attributes.attrubite_name (not label!)
[
{
'$match': {
'$and': [
{
'attributes.attribute_name': 'Shape'
}, {
'attributes.attribute_value': {
'$in': [
'Square'
]
}
}, {
'attributes.attribute_name': 'Color'
}, {
'attributes.attribute_value': {
'$in': [
'Red'
]
}
}
]
}
}
]

Elasticsearch returns results greater than specified in range

I'm use Elasticsearch (Version: 6.8.4) with MongoDB (4.0.3)
I want find all the documents where price between 1 and 800 and sale_date is greater than Date.now(), but I have problem with my query:
{
"query": {
"bool": {
"must": [],
"filter": {
"term": {
"sold_out": false
},
"bool": {
"should": [
{
"range": {
"sale_date": {
"gt": Date.now()
}
}
},
{
"range": {
"price": {
"gt": 1,
"lte": 800
}
}
}
]
}
}
}
},
"from": 10,
"size": 200
}
It's query returns me results with products where some of them have price greater than 800
Price field stored in Elasticsearch as long
Whant I'm try:
use from: to: and got the same results
change "should" to "must" in filter and it's returns empty results
remove from query { "range": { "sale_date": { "gt": Date.now(), } }
} and it's returns right results!
What I'm doing wrong ?
First remove the uneccessary nesting ofthe two bool objects, try having all the clauses in the same "bool" field like this.
Remeber that with must a document must have the term you are making a comparison on to be included in the result, with should a match will only improve the score, if the condition is met, so in a certain sense must is like AND and should similar to an OR.
In your example changing the filter from should to must made the query return nothing maybe because you don't have any element that has both the date and price you want, or maybe it was the problem was the bool nesting.
Try this:
{
"query": {
"bool": {
"must": [{
"range": {
"sale_date": {
"gt": Date.now()
}
}
},
{
"range": {
"price": {
"gt": 1,
"lte": 800
}
}
}],
"filter": {
"term": {
"sold_out": false
},
}
}
},
"from": 10,
"size": 200
}
This will only get the elements with price lte 800 and sale_date gt Date.now().
Change Date().now() to "gt": "now/d".

MongoDB aggregate conditional push with fixed array length

Scenario: Members can choose (yes/no) from 4 different activities available.
Based on the following input,
[
{
name:"member1",
activity:"activity1",
selected:true
},
{
name:"member1",
activity: "activity3",
selected:false
},
{
name:"member2",
activity:"activity2",
selected:true
},
{
name:"member2",
activity: "activity4",
selected:false
}
]
need a result as follows, showing member's choice on all the 4 activities in the order of activity 1 to 4 (including the activities which the user has not made a decision yet)
[
{
name:"member1",
activities:[true,null,false,null]
},
{
name:"member2",
activities:[null,true,null,false]
}
]
I tried the following code,
db.collection("MemberActivities").aggregate(
[
{
$group:
{
_id: "$MemberName",
activities: { $push: "$selected"}
}
}
]
but, it contain only the activities the user has made a decision (yes/no).
[
{
_id:"member1",
activities:[true,false]
},
{
_id:"member2",
activities:[true,false]
} ]
Please guide on how to get desired result.

Manipulating MongoDB response NodeJS

I made a "little" query for mongodb to join two collections and retrieve data.
The game: insert 2 or 3 params on a URL
-include can be 0,1 or 2.
0 exclusive
1 inclusive
2 return all
-netcode: is a key to filter data
-group: another optional keys, that works with the first param "include"
-My query works perfectly, returns in a way how much times a event happened in a certain group.
-The problem? I can't work with the result of mongo db, i need to parse it to JSON.
I'm not so clever at JS, so i don't know where to put it. Since i work in corporation, some of the code was already done.
Well my output is this:
{
"events": [
{
"_id": {
"group": "GFS-CAJEROS-INFINITUM-TELDAT-M1",
"event": "SNMP DOWN"
},
"incidencias": 1
},
{
"_id": {
"group": "GFS-CAJEROS-MPLS",
"event": "Proactive Interface Input Utilisation"
},
"incidencias": 1209
},
{
"_id": {
"group": "GFS-CAJEROS-MPLS",
"event": "Proactive Interface Output Utilisation"
},
"incidencias": 1209
},
{
"_id": {
"group": "GFS-CAJEROS-MPLS",
"event": "Proactive Interface Availability"
},
"incidencias": 2199
},
{
"_id": {
"group": "GFS-SUCURSALES-HIBRIDAS",
"event": "Proactive Interface Output Utilisation"
},
"incidencias": 10
},
But i want it fused in a JSON format, like this: check the int value is next for the name of the event.
[
{
"group": "GFS-CAJEROS-MPLS",
"Proactive Interface Input Utilisation" : "1209",
"Proactive Interface Output Utilisation" : "1209",
"Proactive Interface Availability" : "2199",
},
{
"group": "GFS-SUCURSALES-HIBRIDAS",
"Proactive Interface Output Utilisation" : "10",
},
I'm using Nodejs and the mongodb module, since i dont know how this function exactly works, i don't know how to manage the response, ¿there is a better way to do this? like to get the json file, using another js to generate it?
This is the code i'm using, basically is the important part:
var events = db.collection('events');
events.aggregate([
{ $match : { netcode : data.params.netcode } },
{
$lookup:
{
from: "nodes",
localField: "name",
foreignField: "name",
as: "event_joined"
}
},
{ $unwind: {path: "$event_joined"} },
{ $match : {"event_joined.group" :
{$in:
[
groups[0] ,
groups[1] ,
groups[2] ,
groups[3] ,
groups[4] ,
groups[5] ,
groups[6] ,
groups[7] ,
groups[8] ,
groups[9] ,
]
}
}
},
{ $group : { _id : {group:"$event_joined.group", event:"$event"}, incidencias: { $sum: 1} } },
])
.toArray( function(err, result) {
if (err) {
console.log(err);
} else if (result) {
data.response.events = result;
} else {
console.log("No result");
}
You should add another $group to your pipeline {_id: "$_id.group", events: {$push : {name: "$_id.event", incidencias: "$incidencias"}}}
Then change the structure of your data on the JS code with "Array.map".
data.response.events = data.response.events.map(function (eve){
var obj = {
"group": eve.group
};
eve.events.forEach(function (e){
obj[e.name] = e.incidencias
})
return obj;
})

Filter subdocument array while still returning parent data if empty

I am using the method from this question How to filter array in subdocument with MongoDB
It works as expected except when none of the elements in the array match the test. In that case, I just get an empty array with no parent data.
SAMPLE DATA
{
"_id": "53712c7238b8d900008ef71c",
"dealerName": "TestDealer",
"email": "test#test.com",
"address": {..},
"inventories": [
{
"title": "active",
"vehicles": [
{
"_id": "53712fa138b8d900008ef720",
"createdAt": "2014-05-12T20:08:00.000Z",
"tags": [
"vehicle"
],
"opts": {...},
"listed": false,
"disclosures": {...},
"details": {...}
},
{
"_id": "53712fa138b8d900008ef720",
"createdAt": "2014-05-12T20:08:00.000Z",
"tags": [...],
"opts": {...},
"listed": true,
"disclosures": {...},
"details": {...}
}
]
},
{
"title": "sold",
"vehicles": []
}
]
}
TRYING TO DO
In my query I would like to return the user (document) top-level info (dealerName, email) and a property called vehicles containing all the vehicles in the "active" inventory that have the property listed set to true.
HOW FAR I GOT
This is my query. (I use Mongoose but use mostly native Mongo features)
{
$match:
email: params.username
}
{
$unwind: '$inventories'
}
{
$match:
'inventories.title': 'active'
}
{
$unwind:
'$inventories.vehicles'
}
{
$match:
'inventories.vehicles.listed':
$eq: true
}
{
$group:
_id: '$_id'
dealerName:
$first: '$dealerName'
email:
$first: '$email'
address:
$first: '$address'
vehicles:
$push: '$inventories.vehicles'
}
THE PROBLEM
At first, I thought my query was fine, however, if none of the vehicles are marked as listed, the query just returns an empty array. This makes sense since
{
$match:
'inventories.vehicles.listed':
$eq: true
}
Doesn't match anything but I would still like to get the dealerName as well as his email
DESIRED OUTPUT IF NO VEHICLES MATCH
[{"dealerName": "TestDealer", "email": "test#test.com", vehicles : []}]
ACTUAL OUTPUT
[]
You could use $redact instead of $match in this case, like this
db.collectionName.aggregate({
$redact:{
$cond:{
if:{$and:[{$not:"$dealerName"},{$not:"$title"},{$eq:["$listed",false]},
then: "$$PRUNE",
else: "$$DESCEND"
}
}
})
We need first condition to skip top level documents, second condition to skip second level and third one to prune vehicles. No $unwind needed in this case!
One more thing: $redact available only in 2.6

Categories