We Keep Coding

How to search for specific word and exact match in ElasticSearch

sample data for title
actiontype test
booleanTest
test-demo
test_demo
Test new account object
sync accounts data test
default Mapping for title
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
tried with this query search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": "test"
}
}
]
}
},
}
here my expectation
with specific word(e.g. : test ) it should return following titles
expect
actiontype test
booleanTest
test-demo
test_demo
Test new account object
sync accounts data test
But
got
actiontype test
test-demo
test_demo
Test new account object
sync accounts data test
With exact match (e.g. : sync accounts data test ) it should return only this(sync accounts data test) but got all records those contains this words (sync,account,data,test).
What should I do to make this happen ? Thanks.

I am not sure which ES version you're using but the following should give you an idea.
Using your mapping you can get all title text with test, including booleanTest using query string query type. Eg.
GET {index-name}/{mapping}/_search
{
"query": {
"query_string": {
"default_field": "title",
"query": "*test*"
}
}
}
However, for this to work, make sure you give your title field an analyzer with a lowercase analyzer filter (see below settings example). Your current mapping will not work since it's just pure text as is... test /= TEST by default.
There are other ways, if you're interested to know the workings of ES... Eg. You can also match booleanTest in your match query by writing a custom nGram filter to your index settings. Something like,
{
"index": {
"settings": {
"index": {
"analysis": {
"filter": {
"nGram": {
"type": "nGram",
"min_gram": "2",
"max_gram": "20"
}
},
"ngram_analyzer": {
"filter": [
"lowercase",
"nGram"
],
"type": "custom",
"tokenizer": "standard"
}
}
}
}
}
}
NB: ngram_analyzer is just a name. You can call it whatever.
min_gram & max_gram: Pick numbers that work for you.
Learn more about n-gram filter, the goods and bad here: N-GRAM
Then you can add the analyzer to your field mapping like,
{
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256,
"analyzer": "ngram_analyzer"
}
}
}
}
Lastly for exact matches, these work on type keyword. So based on your mapping, you already have the keyword field so you can use term query to get the exact match by searching on the title.keyword field;
GET {index-name}/{mapping}/_search
{
"query": {
"term": {
"title.keyword": {
"value": "sync accounts data test"
}
}
}
}
Also, you will want to read/ learn more about these solutions and decide on the best solution based on your indexing setup and needs. Also, there may be more ways to achieve what you need, this should be a good start.

When should I use the read/writeQuery and when the read/writeFragment while working with the Apollo Cache?

I'm working with the Apollo local cache and I found that the read/writeQuery works in the same way as read/writeFragment. Both of them do the same thing and the only difference at my opinion is just that the readQuery returns an object that contains the query name as a key and the query result as a value. The readFragment returns the query result object itself.
Based on the official documentation about readQuery I could assume that it executes the query and returns the data.
About the readFragment things look the same. The only difference is that in the case of Fragment I should pass the composed ID of the required item that Apollo cache uses itself.
But the result is totally the same as in the case of readQuery except that it returns the object of values.
Here is the code example.
Sorry, there are will be a lot of cumbersome boilerplate. Simple but a lot to demonstrate the problem in very detail.
The schema of the Track (I'll omit other types to reduce the code boilerplate):
type Track {
id: ID!
title: String!
author: Author!
thumbnail: String
length: Int
modulesCount: Int
description: String
numberOfViews: Int
modules: [Module!]!
}
The readQuery example with the all the Track fields:
const { track } = client.readQuery({
query: gql`
query getTrack($trackId: ID!) {
track(id: $trackId) {
id
title
author {
id
name
photo
}
thumbnail
length
modulesCount
numberOfViews
modules {
id
title
length
}
description
}
}
`,
variables: {
trackId: 'c_1',
},
});
The result is:
{
"__typename": "Track",
"id": "c_1",
"title": "FAMOUS CATSTRONAUTS",
"author": {
"__typename": "Author",
"id": "cat-2",
"name": "Grumpy Cat",
"photo": "https://images.unsplash.com/photo-1593627010886-d34828365da7?ixlib=rb-1.2.1&q=80&fm=jpg&crop=entropy&cs=tinysrgb&w=1080&fit=max&ixid=eyJhcHBfaWQiOjExNzA0OH0"
},
"thumbnail": "https://res.cloudinary.com/dety84pbu/image/upload/v1598474100/famous_cats_epuqcr.jpg",
"length": 1916,
"modulesCount": 5,
"numberOfViews": 26,
"modules": [
{
"__typename": "Module",
"id": "l_10",
"title": "Neil Armstrong",
"length": 321
},
{
"__typename": "Module",
"id": "l_11",
"title": "Yuri Gagarin",
"length": 470
},
{
"__typename": "Module",
"id": "l_12",
"title": "Buzz Aldrin",
"length": 545
},
{
"__typename": "Module",
"id": "l_13",
"title": "John Glenn",
"length": 357
},
{
"__typename": "Module",
"id": "l_14",
"title": "Chris Hadfield",
"length": 223
}
],
"description": "Be inspired by famous catstronauts who have made their names legend from across the galaxies. Special guest appearance included, so hold on to your boots!"
}
Now the readFragment example (all fields):
const track = client.readFragment({
id: 'Track:c_1',
fragment: gql`
fragment MyTrack on Track {
id
title
author {
id
name
photo
}
thumbnail
length
modulesCount
numberOfViews
modules {
id
title
length
}
description
}
`
});
The result is the same as in the case of the readyQuery.
Now the example with the partial fields.
For the readQuery:
const { track } = client.readQuery({
query: gql`
query getTrack($trackId: ID!) {
track(id: $trackId) {
id
title
}
}
`,
variables: { // Provide any required variables here. Variables of mismatched types will return `null`.
trackId: 'c_1',
},
});
The result will be:
{
"__typename": "Track",
"id": "c_1",
"title": "FAMOUS CATSTRONAUTS"
}
The readFragment (partial fields)
const track = client.readFragment({
id: 'Track:c_1',
fragment: gql`
fragment MyTrack on Track {
id
title
}
`
});
The result will be the same as readyQuery!
My question: When should I use the readQuery and when the readFragment? It looks like they provide the same result.
The same for writeQuery and the writeFragment. They do the same at the first look.
writeQuery partial update code example:
client.writeQuery({
query: gql`
query WriteTrack($trackId: ID!) {
track(id: $trackId) {
id
title
},
}`,
data: { // Contains the data to write
track: {
__typename: 'Track',
id: 'c_1',
title: 'Buy grapes 🍇',
},
},
variables: {
id: 'c_1'
}
})
writeFragment partial update code example:
client.writeFragment({
id: 'Track:c_1',
fragment: gql`
fragment MyTrack on Track {
id
title
}
`,
data: {
__typename: 'Track',
id: 'c_1',
title: 'Buy grapes 🍇',
}
})
The result is the same. Both update the title. So what is the difference? When should I use the writeQuery and when the writeFragment?
Thanks for any help and your patience and time!

ElasticSearch EJS query to fetch missing field

I have an existing ejs query as below:
let queryBody = ejs.Request()
.size(0)
.query(
ejs.BoolQuery()
.must(
ejs.RangeQuery('hour_time_stamp').gte(this.lastDeviceDate).lte(this.lastDeviceDate)
)
)
.agg(ejs.TermsAggregation('market_agg').field('market').order('sum', 'asc').size(50000)
.agg(ejs.SumAggregation('sum').field('num_devices'))
)
currently the field('market') returns the values where data for market is present. There is data in the database for missing values for market as well, which I need to access. How do I do that?
EDIT:
Values for market in ES is either null or field is missing. I wrote ES query to get all those fields but I am not able to incorporate an ejs query for the same. Any idea how this can be done?
{
"query": {
"bool": {
"should": [
{
"exists": {
"field": "market"
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "market"
}
}
]
}
}
]
}
}
}

As per your problem you need a way to group the empty market fields too.
So for that you can use the "missing" value parameter. It defines how the values which are missing(as in your case) are grouped. So you query in json form will be modified like below :-
{
"query":
{
"must": [
"range": {
"hour_time_stamp": {
"gte": lastDeviceDate,
"lte": lastDeviceDate
}
}
]
},
"aggs": {
"market_agg" : {
"market": {
"missing": "empty_markets",
"order": { "sum": "asc" }
}
},
"sum_agg": {
"sum" : { "field" : "num_devices" }
}
}
}
Or in your code it could be done by adding missing parameter like this.
let queryBody = ejs.Request()
.size(0)
.query(
ejs.BoolQuery()
.must(
ejs.RangeQuery('hour_time_stamp').gte(this.lastDeviceDate).lte(this.lastDeviceDate)
)
)
.agg(ejs.TermsAggregation('market_agg').field('market').missing('empty_markets').order('sum', 'asc').size(50000)
.agg(ejs.SumAggregation('sum').field('num_devices'))
)

Graphql: how to query using prefix string

I have the following query in GraphQL:
{
allBibliografia(nome: "Bibliografia 1") {
id
nome
}
}
which works. But I need to query my backend using a prefix string, as the user is typing. For example: if I type "Bibliografia 1" as above, my query returns
{
"data": {
"allBibliografia": [
{
"id": "1",
"nome": "Bibliografia 1"
}
]
}
}
but if I input just "Bibliografia" or "Bibliografia*", I get this
{
"data": {
"allBibliografia": []
}
}
I was hoping that "Bibliografia*" would work just like a wildcard for SQL, but it didn't. Does anybody knows how to achieve that?

Manipulating MongoDB response NodeJS

I made a "little" query for mongodb to join two collections and retrieve data.
The game: insert 2 or 3 params on a URL
-include can be 0,1 or 2.
0 exclusive
1 inclusive
2 return all
-netcode: is a key to filter data
-group: another optional keys, that works with the first param "include"
-My query works perfectly, returns in a way how much times a event happened in a certain group.
-The problem? I can't work with the result of mongo db, i need to parse it to JSON.
I'm not so clever at JS, so i don't know where to put it. Since i work in corporation, some of the code was already done.
Well my output is this:
{
"events": [
{
"_id": {
"group": "GFS-CAJEROS-INFINITUM-TELDAT-M1",
"event": "SNMP DOWN"
},
"incidencias": 1
},
{
"_id": {
"group": "GFS-CAJEROS-MPLS",
"event": "Proactive Interface Input Utilisation"
},
"incidencias": 1209
},
{
"_id": {
"group": "GFS-CAJEROS-MPLS",
"event": "Proactive Interface Output Utilisation"
},
"incidencias": 1209
},
{
"_id": {
"group": "GFS-CAJEROS-MPLS",
"event": "Proactive Interface Availability"
},
"incidencias": 2199
},
{
"_id": {
"group": "GFS-SUCURSALES-HIBRIDAS",
"event": "Proactive Interface Output Utilisation"
},
"incidencias": 10
},
But i want it fused in a JSON format, like this: check the int value is next for the name of the event.
[
{
"group": "GFS-CAJEROS-MPLS",
"Proactive Interface Input Utilisation" : "1209",
"Proactive Interface Output Utilisation" : "1209",
"Proactive Interface Availability" : "2199",
},
{
"group": "GFS-SUCURSALES-HIBRIDAS",
"Proactive Interface Output Utilisation" : "10",
},
I'm using Nodejs and the mongodb module, since i dont know how this function exactly works, i don't know how to manage the response, ¿there is a better way to do this? like to get the json file, using another js to generate it?
This is the code i'm using, basically is the important part:
var events = db.collection('events');
events.aggregate([
{ $match : { netcode : data.params.netcode } },
{
$lookup:
{
from: "nodes",
localField: "name",
foreignField: "name",
as: "event_joined"
}
},
{ $unwind: {path: "$event_joined"} },
{ $match : {"event_joined.group" :
{$in:
[
groups[0] ,
groups[1] ,
groups[2] ,
groups[3] ,
groups[4] ,
groups[5] ,
groups[6] ,
groups[7] ,
groups[8] ,
groups[9] ,
]
}
}
},
{ $group : { _id : {group:"$event_joined.group", event:"$event"}, incidencias: { $sum: 1} } },
])
.toArray( function(err, result) {
if (err) {
console.log(err);
} else if (result) {
data.response.events = result;
} else {
console.log("No result");
}

You should add another $group to your pipeline {_id: "$_id.group", events: {$push : {name: "$_id.event", incidencias: "$incidencias"}}}
Then change the structure of your data on the JS code with "Array.map".
data.response.events = data.response.events.map(function (eve){
var obj = {
"group": eve.group
};
eve.events.forEach(function (e){
obj[e.name] = e.incidencias
})
return obj;
})