I am trying to find an exact search for an url with ElasticSearch ("#elastic/elasticsearch": "^7.5.0").
I have configured my mapping like so:
const schema = {
userId: {
type: "keyword"
},
url: {
type: "keyword",
index: false,
analyzer: 'keyword'
},
pageTitle: {
type: 'text',
},
pageText: {
type: 'text',
}
};
await client.indices.putMapping({
index,
type,
include_type_name: true,
body: {
properties: schema
}
})
I have tried different queries, and they looks like this:
body: {
query: {
bool: {
must: {
match: {
query: 'test stack',
analyzer: 'keyword',
}
}
}
}
}
Or second attempt:
body: {
query: {
constant_score: {
filter: {
bool: {
must: {
term: {
url: 'test stack'
}
}
}
}
},
}
}
None of them work. I want to get only the results where the exact string 'test/stack' is found. Any help would be highly appreciated.
Example of data I'm trying to add:
[
{"url": "test stack",
"userId": "anotherTest",
"pageTitle": "not important",
"pageText": "not important",
"log": [1, 3, 7]
},
{"url": "test stack",
"userId": "anotherTest",
"pageTitle": "not important",
"pageText": "not important",
"log": [1, 3, 7]
},
{"url": "test stack",
"userId": "anotherTest",
"pageTitle": "not important",
"pageText": "not important",
"log": [1, 3, 7]
}
]
Thanks.
I managed to make this work. Steps are:
1. Delete the index.
2. Delete the custom mapping function.
3. Create the index (with client.indices.create)
4. Index the first item (with client.index).
5. At this point, you can check in postman the dynamic mappings created by ElasticSearch (only visible after 1st item is indexed, by what I could tell). You can make a get request at http://localhost:9200/history/_mappings, and the response should look something like this:
{
"history": {
"mappings": {
"properties": {
"fullTitle": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"log": {
"properties": {
"startTime": {
"type": "long"
},
"timeSpent": {
"type": "long"
}
}
},
"protocol": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"totalTimeSpent": {
"type": "long"
},
"totalVisits": {
"type": "long"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"userId": {
"type": "long"
}
}
}
}
}
As you can see, any field indexed as text has attached another field, called keyword, which can be used for exact matches.
6. The query to get the exact matches looks like this:
const result = await esClient.search({
index: 'history',
body: {
query: {
term: {
'url.keyword': {
value: toInsert.url
}
}
}
}
})
At this point you should receive results only in case of exact match for the field "url" in my case. Hope this helps somebody else. Thanks #ibexit for trying to help me.
I see two problems:
The mapping defined for the url field says
url: {
type: "keyword",
index: false,
analyzer: 'keyword'
},
If you define index: false, the field will not be searchable at all. Using the following mapping should work properly:
url: {
type: "keyword"
}
See https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html for more detailed information
The keyword mapped fields will not match using the match query which is designed to query text fields. Please use the term query instead for keyword fields. Please notice the example below using the Elasticseaech Query API:
GET /_search
{
"query": {
"term": {
"url": { <<= the field to search
"value": "test stack" <<= the searched value
}
}
}
}
Here is the according documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html
BTW: keep in mind that you need to reindex the data after a mapping change
Related
I'm new in Gatsby Development, in my website I'm using a community theme and modifying it in some things.
My problem started when I modified a JSON file called 'settings.json', where I added a field in the siteConfiguration object, the original file was this:
{"siteConfiguration": {
"logo": { "text": "chrisley"},
"navigation": {
"header": [
{ "label": "About", "url": "/#about" },
{ "label": "Blog", "url": "/blog" },
{ "label": "Features", "url": "/#features" },
{ "label": "Github", "url": "/#github" }
],
"ctaButton": {
"openNewTab": true,
"label": "Resume",
"url": "/resume.pdf"
},
"footer": [
{ "label": "Privacy", "url": "/privacy" },
{ "label": "Imprint", "url": "/imprint" }
]
},
"featureToggles": {
"useDarkModeAsDefault": false,
"useDarkModeBasedOnUsersPreference": true,
"useCookieBar": false
}
}
}
And the one where I added the field 'img' inside "logo" is the following:
{"siteConfiguration": {
"logo": { "text": "chrisley","img":"/content/images/logo-dark.png"},
"navigation": {
"header": [
{ "label": "About", "url": "/#about" },
{ "label": "Blog", "url": "/blog" },
{ "label": "Features", "url": "/#features" },
{ "label": "Github", "url": "/#github" }
],
"ctaButton": {
"openNewTab": true,
"label": "Resume",
"url": "/resume.pdf"
},
"footer": [
{ "label": "Privacy", "url": "/privacy" },
{ "label": "Imprint", "url": "/imprint" }
]
},
"featureToggles": {
"useDarkModeAsDefault": false,
"useDarkModeBasedOnUsersPreference": true,
"useCookieBar": false
}
}
}
And when I trying to get the img field in GraphQL with this Query:
query SiteConfiguration {
allSettingsJson: allContentJson {
settings: nodes {
siteConfiguration {
logo {
text
img
}
}
}
}
}
I get the next error:
{
"errors": [
{
"message": "Cannot query field \"img\" on type \"Logo\".",
"locations": [
{
"line": 7,
"column": 11
}
],
"extensions": {
"stack": [
"GraphQLError: Cannot query field \"img\" on type \"Logo\".",
" at Object.Field (/Users/chrisley/Documents/Development/Gatsby/chrisley_dev_website/node_modules/graphql/validation/rules/FieldsOnCorrectTypeRule.js:48:31)",
" at Object.enter (/Users/chrisley/Documents/Development/Gatsby/chrisley_dev_website/node_modules/graphql/language/visitor.js:323:29)",
" at Object.enter (/Users/chrisley/Documents/Development/Gatsby/chrisley_dev_website/node_modules/graphql/utilities/TypeInfo.js:370:25)",
" at visit (/Users/chrisley/Documents/Development/Gatsby/chrisley_dev_website/node_modules/graphql/language/visitor.js:243:26)",
" at validate (/Users/chrisley/Documents/Development/Gatsby/chrisley_dev_website/node_modules/graphql/validation/validate.js:69:24)",
" at graphqlMiddleware (/Users/chrisley/Documents/Development/Gatsby/chrisley_dev_website/node_modules/express-graphql/index.js:98:38)",
" at processTicksAndRejections (node:internal/process/task_queues:95:5)"
]
}
}
]
}
Hope you can help me guys, as always haha 😅
I'll be answering myself because I found the problem hahaha.
The problem was that I was missing updating the Schema of the theme, in particular of this theme was the file gastby/node/createSchemaCustomization.js
So.. after updating the file with this lines:
module.exports = ({ actions }) => {
actions.createTypes(`
...
type Logo {
text: String
img: String
}
...
`);
};
GraphQL detected my new field 'img' of the 'logo' object.
After some research I'm assuming you are using the following starter: https://github.com/konstantinmuenster/gatsby-theme-portfolio-minimal/tree/main/gatsby-theme-portfolio-minimal
Have you tried stopping your development process and cleaning Gatsby cache by:
gatsby clean
Or manually deleting the .cache folder.
I have a nodejs app that calls ES's updateByQuery API like below:
elasticsearchClient.updateByQuery({
index: 'logstash-dev-2019.05.14',
body: {
"query": {
"ids": {
"values": alert.esId
}
},
"script": {
"source": `ctx._source.alertObject.violation_status = 'closed'`,
"lang": "painless"
}
}
});
However occasionally, I run into version_conflict_engine_exception. I want to proceed with the conflict by using conflict=proceed as mentioned in the documentation. Where do I fit this into my update call?
You can use querystring object as provided in the docs, please scroll to the end of the link.
elasticsearchClient.updateByQuery({
index: 'logstash-dev-2019.05.14',
querystring: {
"conflict": "proceed"
},
body: {
"query": {
"ids": {
"values": alert.esId
}
},
"script": {
"source": `ctx._source.alertObject.violation_status = 'closed'`,
"lang": "painless"
}
}
});
I have the following data:
makeStr: xerox
modelStr: Designjet 1050C
I want it to match
xerox
Designjet 1050C Plus Printer
but it is matching
canon
DesignJet 1050C
and currently I have this query
"query": {
"bool": {
"should":
{
"multi_match": {
"query": modelStr,
"type": "most_fields",
"fields": ['model.alphanum']
}
}
,
"filter": [
{
"match": {
"make.blur": makeStr
}
},
{
"match": {
"model.blur": modelStr
}
}
]
}
},
"functions": [{
"field_value_factor": {
"field": "isMpsSupported",
"factor": 1,
"missing": 0
}
}],
"boost_mode": "sum"
}
How do I give preference for makeStr such that it considers both makeStr and modelStr during search.
More preference can be given by using boost. Refer here
Something like makeStr^2 should work.
Moving from D3 to Highcharts and this is eluding me. I have a fairly complex object that contains a clickthrough object which needs to be accessed in a function on a point click in the series. I'm creating the series array with the data and name just fine with a small conversion, but I need to attach this object to the data points as well. No idea how.
Quick example. original data:
[
{
"key": "Super Cool Thing",
"format": ".2f",
"values": [
{
"label": "01",
"value": 9.5,
"format": ".2f",
"order": 0,
"tooltip": "numerator = 133, denominator = 14",
"clickthrough": {
"output_format": "json",
"metrics": "",
"options": {
"columns": [
{
"order": 1,
"display_as": "Brand",
"format": "{0}",
"name": "brand",
"data_type": "string"
},
{
"order": 2,
"display_as": "Last Submit Time (Month)",
"format": "%m",
"name": "last-submit-time-month",
"data_type": "datetime"
},
{
"order": 3,
"display_as": "Agent Things",
"format": "{0}",
"name": "agent-thing-values",
"data_type": "string"
}
]
},
"cut_y": "brand",
"type": "",
"filter": { },
"cut_x": "last-submit-time-month"
},
"metrics": [
{
"name": "Agent - Feel Appreciated Mean",
"slug": "qcustomersatr4-mean"
}
]
}
]
}
]
run through a (super quick POC) funct:
for(let i = 0; i < data.length; i++){
var values = [];
var xcuts = [];
data[i].values.forEach(val => {
values.push(val.value);
xcuts.push(val.label);
});
chart.addSeries({
name: data[i].key,
data: values
})
chart.xAxis[0].setCategories(xcuts);
}
and this all works fine. But I need the clickthrough object so I can do something like:
plotOptions: {
series: {
allowPointSelect: true,
cursor: 'pointer',
point: {
events: {
click: function (event) {
console.log('CLICKTHROUGH DATA HERE');
console.log(event.point);
}
}
}
},
},
I'm unsure how to format the series data to include additional data that's accessible in an event function later down the line. I currently do this via d3 and it's fine, but am struggling with the Highcharts method to do the same. It seems I can't just add whatever I want to the series or data, so is this possible?
Have it. I have to set the y value explicitly and then I can add whatever else which is then avail in the event.
example:
data[i].values.forEach(val => {
values.push({link: val.clickthrough, y:val.value});
xcuts.push(val.label);
});
chart.addSeries({
name: data[i].key,
data: values
})
I wounder if there any way or setting to perform an alphabetical sort in elasticsearch. I've got a field and I want to perform sort in descending order over it. Elastic performs it lexicographically. What I get:
Company name
Customer name
company address
What I want to get:
Company name
company address
Customer name
I found that I can create a custom analyser, but maybe there can be a better option?
use multifields to index the text field as lowercased keyword with fielddata true where you can sort.
{
"settings": {
"analysis": {
"analyzer": {
"keyword_lowercase": {
"tokenizer": "keyword",
"filter": ["lowercase"]
}
}
}
},
"mappings": {
"my_type": {
"properties": {
"text": {
"type": "text",
"fields": {
"raw": {
"type": "text",
"analyzer": "keyword_lowercase",
"fielddata": true
}
}
}
}
}
}
}
Query
{
"sort": [
{
"text.raw": {
"order": "asc"
}
}
]
}