Search inside array using Elastic Search - javascript

I am using Elastic version 6.8, created one index into whose schema is as follow:
{
"properties": {
"title": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"tags": {
"type": "keyword",
"fields": {
"raw": {
"type": "text"
}
}
}
}}
and I have added following documents into it
[{
"title": "one",
"tags": ["html", "css", "javascript"]
}, {
"title": "two",
"tags": ["java", "jsp", "servlet"]
}, {
"title": "three",
"tags": ["spring", "java"]
}, {
"title": "four",
"tags": ["react", "angular", "javascript"]
}, {
"title": "five",
"tags": ["java"]
}, {
"title": "six",
"tags": []
}]
now I have more than 10 millions document in elastic search. Now I want to search following cases:
List all tags. with unique result (using skip, limit) skip value change but limit is fixed.so here I want result like
html,
css,
javascript,
java,
jsp,
servlet,
spring,
react,
angular
Partil search inside tags, it means if I search using act then it should give result as : react this also using skip limit.
How I can get this using Elastic search query. please help me here?

You can find unique possible value by using term aggregation.
GET yourindex/_search
{
"size": 0,
"aggs": {
"all_tags": {
"terms": {
"field": "tags",
"size": 100
}
}
}
}
"size":100 Get at most 100 unique values. Default is 10. You can increase more but it will include cost. You can check more on doc.
For partial search you can use wildcard query OR you can try N-Gram Tokeninzer. Both will allow to do partial search but wildcard query will be costly. You can evaluate according to your use case.

Related

Microsoft power automate: JSON "Object reference not set to an instance of an object"

I am attempting to create a JSON Object from an array to pass into a Microsoft product. The format in which the JSON object is accepted is shown beneath (content-type: "application/json"):
{
"value": [
{
"activityGroupNames": [],
"confidence": 0,
"description": "This is a canary indicator for demo purpose. Take no action on any observables set in this indicator.",
"expirationDateTime": "2019-03-01T21:44:03.1668987+00:00",
"externalId": "Test--8586509942423126760MS164-0",
"fileHashType": "sha256",
"fileHashValue": "b555c45c5b1b01304217e72118d6ca1b14b7013644a078273cea27bbdc1cf9d6",
"killChain": [],
"malwareFamilyNames": [],
"severity": 0,
"tags": [],
"targetProduct": "Azure Sentinel",
"threatType": "WatchList",
"tlpLevel": "green",
},
{
"activityGroupNames": [],
"confidence": 0,
"description": "This is a canary indicator for demo purpose. Take no action on any observables set in this indicator.",
"expirationDateTime": "2019-03-01T21:44:03.1748779+00:00",
"externalId": "Test--8586509942423126760MS164-1",
"fileHashType": "sha256",
"fileHashValue": "1796b433950990b28d6a22456c9d2b58ced1bdfcdf5f16f7e39d6b9bdca4213b",
"killChain": [],
"malwareFamilyNames": [],
"severity": 0,
"tags": [],
"targetProduct": "Azure Sentinel",
"threatType": "WatchList",
"tlpLevel": "green",
}
]
}
I making use of an inline code script in Microsoft automate that performs the following in JavaScript:
var threat = workflowContext.actions.Compose.outputs;
var value = Object.values(threat);
return value;
The workflowContext.actions.Compose.outputs line pulls an array consisting of objects shown in the following snippet:
[{"id": "1", "activityGroupNames": "test2"}, {"id": "2", "activityGroupNames": "test3"}, {"id": "3", "activityGroupNames": "test4"}]
This is my output:
{
"body": [
{
"id": "1",
"action": "alert",
"activityGroupNames": "test2"
},
{
"id": "2",
"action": "alert",
"activityGroupNames": "test3"
},
{
"id": "3",
"action": "alert",
"activityGroupNames": "test2"
}
]
}
it is pretty much identical to the format described my Microsoft shown in the first snippet. (https://learn.microsoft.com/en-us/graph/api/tiindicator-submittiindicators?view=graph-rest-beta&tabs=http) at the bottom.
I am unsure as to how I can change the key name from "body" to "value" and think maybe this will resolve my issue. Either way, I'd appreciate any other help on the matter, if any more context is required, please ask.
EDIT: The image beneath shows that the returned return value; is in fact being used as the input for a POST request to the Microsoft graph API

Query language for JSON or how to use Javascript/Lua in Python?

I'm looking solution to query JSON that extracts specific data.
This solution will be embedded inside our Python software (more exactly Python AWS Lambda) and the user should have the ability to specify such query string in GUI.
For example, we have such JSON and I would extract "nazwisko_weterynarza.value" from "files" array if "typ_dokument" is equal to "szczepienie" and "data_szczepienia" is equal to 3434343.
I've tried this using JSON Path but I've failed.
What about embedded Lua/Javascript in Python, but how to create a 'sandbox' to be sure that such a solution will be safe?
{
"files" :
[
{
"meta": {
"updated": 555555,
"attributes": [
{
"typ_dokumentu":
{
"formula_type": "NONE",
"formula": "",
"value_type": "string",
"value": "szczepienie",
"updated": 5345435435345
}
},
{
"data_szczepienia":
{
"formula_type": "NONE",
"formula": "",
"value_type": "int",
"value": 3434343,
"updated": 5345435225345
},
},
{
"nazwisko_weterynarza":
{
"formula_type": "NONE",
"formula": "",
"value_type": "string",
"value": "Nowak",
"updated": 5345435225345
}
}
]
}
},
{
"meta": {
"updated": 555555,
"attributes": [
{
"typ_dokumentu" :
{
"formula_type": "NONE",
"formula": "",
"value_type": "string",
"value": "Certyfikat urodzin",
"updated": 5345435435345
}
},
{
"data_urodzin" :
{
"formula_type": "NONE",
"formula": "",
"value_type": "int",
"value": 8888888,
"updated": 5345435225345
}
}
]
}
}
]
}
The AWS JSON (reference) path queries are not capable enough. Some more advanced JSONPath processors can do it, eg. using Jayway's JsonPath we could run a query like this:
$.files..meta.[?(#.attributes[*].typ_dokumentu.value contains 'szczepienie' && #.attributes[*].data_szczepienia.value contains '3434343')]..nazwisko_weterynarza.value
Reurns ["Nowak"] (You can try it online).
I don't know what you mean by embedded JavaScript/Lua, but in any event, you could create a web service that does the querying for you leveraging an implementation of your choice. Security wouldn't be an issue as long as your API is properly secured.

Dialogflow html/js card json value v1

I am trying to access the card json value, to no avail.
In my scenario, I am asking the bot about "weather in London" and it replies back with "It is currently 9 degrees celcius in London." via the webhook.
Which is correct and dynamic.
However, I am trying to also pass the values to a card too.
In the json reply, I do get the card as so
{
"id": "REMOVED",
"timestamp": "2017-12-05T11:10:52.033Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "weather in london",
"action": "sayWeather",
"actionIncomplete": false,
"parameters": {
"geo-city": "London"
},
"contexts": [],
"metadata": {
"intentId": "REMOVED",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"webhookResponseTime": 626,
"intentName": "Weather"
},
"fulfillment": {
"speech": "It is currently 9 degrees celcius in London.",
"source": "agent",
"displayText": "It is currently 9 degrees celcius in London.",
"messages": [
{
"type": 0,
"speech": "It is currently 9 degrees celcius in London."
}
],
"data": {
"items": [
{
"simpleResponse": {
"textToSpeech": "This is the first simple response for a basic card"
}
},
{
"basicCard": {
"title": "Title: this is a title",
"formattedText": "This is a basic card. Text in a\n basic card can include \"quotes\" and most other unicode characters\n including emoji 📱. Basic cards also support some markdown\n formatting like *emphasis* or _italics_, **strong** or __bold__,\n and ***bold itallic*** or ___strong emphasis___ as well as other things\n like line \nbreaks",
"subtitle": "This is a subtitle",
"image": {
"url": "https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png",
"accessibilityText": "Image alternate text"
},
"buttons": [
{
"title": "This is a button",
"openUrlAction": {
"url": "https://assistant.google.com/"
}
}
]
}
},
{
"simpleResponse": {
"textToSpeech": "This is the 2nd simple response ",
"displayText": "This is the 2nd simple response"
}
}
]
}
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success",
"webhookTimedOut": false
},
"sessionId": "REMOVED"
}
Accessing the value of speech using data.result.fulfillment.speech works fine.
However, when using data.result.fulfillment.data.items.basicCard.image.url it just doesnt work. And if I go up several levels, I do get:
[object Object]
Your help is appreciated.
The items attribute is a list and not an object. As such, you'll have to use a numerical index to retrive the data. In the example you provided the index of the basicCard object is second so your code should look something like this:
data.result.fulfillment.data.items[1].basicCard.image.url
Notice the [1] after items.
Bear in mind that if the order of this list changes you may no longer be retrieving a basicCard object so you may want to add some checking to make sure you're retrieving the data you want.

How to retrieve linked fields in Contentful query

I'm using Contentful and have a content model which uses a series of related fields. I am querying my content in NodeJS using the JS api. If I call get entries, like so
contentfulClient.getEntries({
content_type: 'homePage'
})
it fetches all the content of type homePage, and includes the actual field data for each related field, like so
"subField": {
"sys": {
"space": {
"sys": {
"type": "Link",
"linkType": "Space",
"id": "#######"
}
},
"id": "#######",
"type": "Entry",
"createdAt": "2017-03-10T15:58:25.697Z",
"updatedAt": "2017-03-10T15:58:25.697Z",
"revision": 1,
"contentType": {
"sys": {
"type": "Link",
"linkType": "ContentType",
"id": "homeSubField"
}
},
"locale": "en-GB"
},
"fields": {
"category": {
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "#######"
}
},
"subFieldContent": "Some field content"
}
},
However, if I call a specific entry with an ID, like
contentfulClient.getEntry('1234567890')
Then each linked field is just returned as a reference with an ID, like
"subField": {
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "#######"
}
},
How can I get the full text when calling getEntry, like I do with getEntries?
Unfortunately Contentful does not include referenced content when fetching a content item by id directly. One way around this is to instead use the getEntries method, but filter by sys.id. That way you'll get the same entry back, although in an array, but it will also include referenced content.
contentfulClient.getEntries({
content_type: 'homePage',
sys.id: '1234567890'
})
This also results in a single request instead of multiple as you would end up with when using the GetEntry method and then resolving each referenced content item manually.

Iterate through nested Javascript Objects from API response

I've tried 100 different things, and spend days looking through Google and Stackoverflow, but I can't find a solution to this problem. Everything I call after the body of this API response returns undefined!
The response from Facebook SDK looks like this:
[
{
"body": "[
"data": [
{
"name": "Larry Syid Wright",
"administrator": false,
"id": "xxx"
}, {
"name": "Melissa Long Jackson",
"administrator": false,
"id": "xxx"
}, {
"name": "Charlotte Masson",
"administrator": false,
"id": "xxx"
}
],
"paging": {
"next": "url"
}
]"
},{
"body": "{
"data": [
{
"id": "xxx_xxx",
"message": "In honor of Halloween, how many of you have your own ghost stories? Who believes in ghosts and who doesn't?",
"type": "status",
"created_time": "2014-10-31T20:02:01+0000",
"updated_time": "2014-11-01T02:52:51+0000",
"likes": {
"data": [
{
"id": "xxx",
"name": "Joe HerBatman Owenby Jr."
}
],
}
"paging": {
"cursors":
{
"after": "xxx",
"before": "xxx"
}
}
}
},{
"id": "xxx_xxx",
"from": {
"id": "xxx",
"name": "Jessica Starling"
},
"message": "Watching the "Campaign" and I can't help but notice what a fantastic job they did (Will ferrell and all) with that North Carolina accent! Ya'll know we sound different than other southern states ;)",
"type": "status",
"created_time": "2014-11-01T02:36:21+0000",
"updated_time": "2014-11-01T02:36:21+0000",
"likes": {
"data": [
{
"id": "xxx",
"name": "Scott Williams"n
}
]
}
}
],
"paging": {
"previous": "xxx",
"next": "xxx"
}
}"
}
]
This response is from a batch call. If I call them separately, I can easily iterate through the responses, and get everything from them. When I call them in the batch though, I can't get past "body", and I need to use a batch call.
console.log(response[0].body); will return the object inside the body of the first part of the response, but console.log(response[0].body.data); returns undefined. I just don't get it. This should be simple but it's like there's a lock on the door and I don't have the right key.
I normally have no issue iterating through objects, so I don't need a generalized answer. I need help seeing whatever it is here that I don't see. Why does the console show undefined when I call anything after the body, and what do I need to be doing to get any of these values?
That JSON contains nested JSON. body seems to be a string. Use
var body = JSON.parse(response[0].body);
The values from the body are just strings.which are embedded as json.So firstly you would need to parse them using JSON.parse.
The code would be like
var body = JSON.parse(response[0].body);

Categories