I want to extract the created_utc from the JSON object can anyone tell me how can I do that in javascript? I do not want to convert it into string and parse word by word is there a quicker method to obtain the created_utc field?
var temp = {
"responseHeader":{
"status":0,
"QTime":44,
"params":{
"q":"*:*",
"indent":"on",
"wt":"json",
"_":"1491759117819"}},
"response": {
"numFound": 2, "start": 0, "docs": [
{
"id": "1",
"_version_": 1564222440512421888
},
{
"author": ["illusorywall"],
"link_id": ["t3_3yy560"],
"created_utc": [1451606530],
"subreddit": ["bloodborne"],
"score": [5],
"body": ["I don't think it's always the same kind of wall. It can be a larger, sort of \"arched\" section of wall, or a small rectangular section of wall, like a normal doorway. I have seen these two spaces use the same texture though, if that's what you mean by it always being the same type of wall, but I think it can also have a different texture. I intend to take a screenshot of every one I find to be sure."],
"id": "4f9a4bc0-4b7b-4aba-8820-187bfd2e6645",
"_version_": 1564222753474609152
}]
}
};
You should be using:
temp.response.docs[1].created_utc[0]
Snippet:
var temp = {
"responseHeader": {
"status": 0,
"QTime": 44,
"params": {
"q": "*:*",
"indent": "on",
"wt": "json",
"_": "1491759117819"
}
},
"response": {
"numFound": 2,
"start": 0,
"docs": [{
"id": "1",
"_version_": 1564222440512421888
},
{
"author": ["illusorywall"],
"link_id": ["t3_3yy560"],
"created_utc": [1451606530],
"subreddit": ["bloodborne"],
"score": [5],
"body": ["I don't think it's always the same kind of wall. It can be a larger, sort of \"arched\" section of wall, or a small rectangular section of wall, like a normal doorway. I have seen these two spaces use the same texture though, if that's what you mean by it always being the same type of wall, but I think it can also have a different texture. I intend to take a screenshot of every one I find to be sure."],
"id": "4f9a4bc0-4b7b-4aba-8820-187bfd2e6645",
"_version_": 1564222753474609152
}
]
}
};
console.log(temp.response.docs[1].created_utc[0])
try this
temp["response"]["docs"][1]["created_utc"][0]
Related
He there, I use to work with Relational DBS, but right now trying to implement e-commerce shop and use mongodb.
I need the product, sub-products and description (multi lang);
I prefere to separate everything by 3 collection (maybe its not a good idea because mongo use 1 collection for one entity, in my example, 3 collection for 1 entity)
"content": [{
pid: 1,
lang: "ru",
title: "Привет"
},
{
pid: 1,
lang: "en",
title: "Hello"
},
{
pid: 2,
lang: "ru",
title: "Пока"
},
{
pid: 2,
lang: "en",
title: "Bye"
}
],
"products": [{
"_id": 1,
"item": "almonds",
"price": 12,
},
{
"_id": 2,
"item": "pecans",
"price": 20,
},
],
"sub": [{
"_id": 11,
"pid": 1,
"features": {
"color": ["red"],
"size": 42
},
"qt": 5
},
{
"_id": 12,
"pid": 1,
"features": {
"color": ["red"],
"size": 43
},
"qt": 2
},
{
"_id": 13,
"pid": 1,
"features": {
"color": ["yellow"],
"size": 44
},
"qt": 3
},
{
"_id": 21,
"pid": 2,
"features": {
"color": ["yellow"],
"size": 41
},
"qt": 6
},
{
"_id": 22,
"pid": 2,
"features": {
"color": ["red"],
"size": 47
},
"qt": 10
}
]
Products should have sub-products in order to use filter, for example when i want to filter items i will seek into sub-product collection find all the yellow t-short for example with size 44, then i just $group the items by main productId and make $lookup with main products and return it.
Also in order to receive main product with description I should to do $lookup with content collection.
Is it a great idea or should i use 1 collection for product and content?
Like:
"products": [{
"_id": 1,
"item": "almonds",
"price": 12,
"content": [{
lang: "ru",
title: "Привет"
},
{
lang: "en",
title: "Hello"
},
},
]
and maybe should I include sub-item also to main product, like:
"products": [{
"_id": 1,
"item": "almonds",
"price": 12,
"content": [{
lang: "ru",
title: "Привет"
},
{
lang: "en",
title: "Hello"
},
},
"sub": [{
"features": {
"color": ["red"],
"size": 42
},
"qt": 5
},
{
"features": {
"color": ["red"],
"size": 43
},
"qt": 2
},
]
]
The main Question is it good idea to compare everything and don't care about size of the collection? And if so how should I do a filter on nested documents ('sub-products')(previously 'sub-products' collection was like a plain collection and I could make aggregation in order to find all items by color for example: {"features.color": { $in: ['red'] }}) how can i manage it with nested document, and it will not be overwhelmed operation?
Your question about the data model design decision of how to split up your entities/documents cannot be answered in a useful way with just the information given, but I can give you some aspects to think about for the decision.
MongoDb is a key-value store and as such is most useful if you can design your data in a way that uses mostly key-based lookups. This can be extended with creating additional indexes on document fields other than _id, which is indexed by default. Everything that is not indexed will result in collection scans and be very inefficient to query. Indexes can substantially increase the amount of storage required, so depending on your scenario cost may be a prohibiting factor to just index every field you want to query by later. That means when designing entities you will also have to consider estimated size of collections and the possibility to reference between collections (by guid for example).
So in order to make the right design decisions for your data model we cannot judge just based on the data schema you want to store, but rather based on the queries you are looking to perform later and the expected collection sizes / future scaling requirements. Those are aspects that you only touch very lightly in your question. For example if you plan to query all kind of complex joins and combinations of property values across all your entities, you may ask yourself if you can afford the extra storage cost of non-normalized data and additional indexes, or if a traditional (or modern) SQL-RDB, or maybe a graph database may be more suitable than a key-value store for your use case. Whereas if your database scale will be small at all times and your main concern is developer productivity these considerations may be worthless.
Your specific question about accessing "sub"-documents within an array of the parent "products" can be answered by that it is supported by using an elemmatch. For example:
db.products.find({sub: {$elemMatch: {'features.size':43, 'features.color':'red'}}})
Please note again that these queries will only be efficient if you index the fields in your query. In case of array sub-documents that means looking into multi-key indexes.
In order to acquire more knowledge for better decisioning around DB models and your questions about queries in MongoDB I recommend reading the official MongoDB data model design guide, the documentation for querying arrays, as well as googling some articles on normalization and the motivation of SQL vs noSQL in terms of scaling/sharding, ACID and eventual consistency.
I am trying to access the card json value, to no avail.
In my scenario, I am asking the bot about "weather in London" and it replies back with "It is currently 9 degrees celcius in London." via the webhook.
Which is correct and dynamic.
However, I am trying to also pass the values to a card too.
In the json reply, I do get the card as so
{
"id": "REMOVED",
"timestamp": "2017-12-05T11:10:52.033Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "weather in london",
"action": "sayWeather",
"actionIncomplete": false,
"parameters": {
"geo-city": "London"
},
"contexts": [],
"metadata": {
"intentId": "REMOVED",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"webhookResponseTime": 626,
"intentName": "Weather"
},
"fulfillment": {
"speech": "It is currently 9 degrees celcius in London.",
"source": "agent",
"displayText": "It is currently 9 degrees celcius in London.",
"messages": [
{
"type": 0,
"speech": "It is currently 9 degrees celcius in London."
}
],
"data": {
"items": [
{
"simpleResponse": {
"textToSpeech": "This is the first simple response for a basic card"
}
},
{
"basicCard": {
"title": "Title: this is a title",
"formattedText": "This is a basic card. Text in a\n basic card can include \"quotes\" and most other unicode characters\n including emoji 📱. Basic cards also support some markdown\n formatting like *emphasis* or _italics_, **strong** or __bold__,\n and ***bold itallic*** or ___strong emphasis___ as well as other things\n like line \nbreaks",
"subtitle": "This is a subtitle",
"image": {
"url": "https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png",
"accessibilityText": "Image alternate text"
},
"buttons": [
{
"title": "This is a button",
"openUrlAction": {
"url": "https://assistant.google.com/"
}
}
]
}
},
{
"simpleResponse": {
"textToSpeech": "This is the 2nd simple response ",
"displayText": "This is the 2nd simple response"
}
}
]
}
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success",
"webhookTimedOut": false
},
"sessionId": "REMOVED"
}
Accessing the value of speech using data.result.fulfillment.speech works fine.
However, when using data.result.fulfillment.data.items.basicCard.image.url it just doesnt work. And if I go up several levels, I do get:
[object Object]
Your help is appreciated.
The items attribute is a list and not an object. As such, you'll have to use a numerical index to retrive the data. In the example you provided the index of the basicCard object is second so your code should look something like this:
data.result.fulfillment.data.items[1].basicCard.image.url
Notice the [1] after items.
Bear in mind that if the order of this list changes you may no longer be retrieving a basicCard object so you may want to add some checking to make sure you're retrieving the data you want.
I am building a web app with the MEAN stack and Yelp API that returns an array of objects, where each object is a local business. I work with this data in the front-end, but before I send a response I want to check if a particular object exists in the MongoDB database and I am struggling with how to do that.
Here is an object that is returned from the API:
[
{
"name": "Arendsnest",
"url": "https://www.yelp.com/biz/arendsnest-amsterdam-2?adjust_creative=ycRBsh7KEkNFq3wJvKoL6Q&utm_campaign=yelp_api&utm_medium=api_v2_search&utm_source=ycRBsh7KEkNFq3wJvKoL6Q",
"snippet_text": "The reigning Lord of Amsterdam beer bars. Popular and seats go fast...come early. Ask for the massive all-Dutch beer list and prepare to have your...",
"image_url": "https://s3-media2.fl.yelpcdn.com/bphoto/FurcfTuqaYBv_q34bGTK5g/ms.jpg"
},
{
"name": "Bar Oldenhof",
"url": "https://www.yelp.com/biz/bar-oldenhof-amsterdam?adjust_creative=ycRBsh7KEkNFq3wJvKoL6Q&utm_campaign=yelp_api&utm_medium=api_v2_search&utm_source=ycRBsh7KEkNFq3wJvKoL6Q",
"snippet_text": "So I'm not much of a drinker. My taste is highly selective and I usually prefer not to drink alcohol altogether. But my husband is the opposite so on a...",
"image_url": "https://s3-media4.fl.yelpcdn.com/bphoto/1k57z7ziIW8MyAWHlXWGdg/ms.jpg"
},
{
"name": "Beer Temple",
"url": "https://www.yelp.com/biz/beer-temple-amsterdam?adjust_creative=ycRBsh7KEkNFq3wJvKoL6Q&utm_campaign=yelp_api&utm_medium=api_v2_search&utm_source=ycRBsh7KEkNFq3wJvKoL6Q",
"snippet_text": "This is a great place to stop in and have some American craft beer. With 30+ taps and a seemingly never ending list of bottle selections, you have many...",
"image_url": "https://s3-media1.fl.yelpcdn.com/bphoto/yxUiYre1Y6ULqMhQ30NPOA/ms.jpg"
},
{
"name": "Tales & Spirits",
"url": "https://www.yelp.com/biz/tales-en-spirits-amsterdam?adjust_creative=ycRBsh7KEkNFq3wJvKoL6Q&utm_campaign=yelp_api&utm_medium=api_v2_search&utm_source=ycRBsh7KEkNFq3wJvKoL6Q",
"snippet_text": "This is exactly what every high-end cocktail bar should strive to have and be.\n\nFriendly staff: From the bartenders to the manager to the waitress. Everyone...",
"image_url": "https://s3-media4.fl.yelpcdn.com/bphoto/IElXytpbY0bpp7ZdjFdGvA/ms.jpg"
}
]
This exists in the MongoDB database:
{
"_id": {
"$oid": "57da26d8dcba0f51172f47b1"
},
"name": "Arendsnest",
"url": "https://www.yelp.com/biz/arendsnest-amsterdam-2?adjust_creative=ycRBsh7KEkNFq3wJvKoL6Q&utm_campaign=yelp_api&utm_medium=api_v2_search&utm_source=ycRBsh7KEkNFq3wJvKoL6Q",
"snippet_text": "The reigning Lord of Amsterdam beer bars. Popular and seats go fast...come early. Ask for the massive all-Dutch beer list and prepare to have your...",
"image_url": "https://s3-media2.fl.yelpcdn.com/bphoto/FurcfTuqaYBv_q34bGTK5g/ms.jpg"
}
How can I write a query in Node to loop through my array using name property and do a check on every object if it exists in the database and return the data?
No need to iterate the array, use the $or operator with a mapped array that has the fields you want to query.
Take the following example where you want to search for a match of two properties:
var yelp = [
{
"name": "Arendsnest",
"url": "url1",
"snippet_text": "foo",
"image_url": "bar.jpg"
},
{
"name": "Bar Oldenhof",
"url": "abc",
"snippet_text": "efg",
"image_url": "ms.jpg"
},
{
"name": "Beer Temple",
"url": "https://www.yelp.com/",
"snippet_text": "test",
"image_url": "ms.jpg"
},
{
"name": "Tales & Spirits",
"url": "https://www.yelp.com/",
"snippet_text": "This is exactly...",
"image_url": "ms.jpg"
}
],
query = yelp.map(function(item){ return { name: item.name, url: item.url }; });
db.collection.find({ "$or": query });
This will create an array that you can use as the $or expression in your find() method, equivalent to :
db.collection.find({
"$or": [
{
"name": "Arendsnest",
"url": "url1"
},
{
"name": "Bar Oldenhof",
"url": "abc"
},
{
"name": "Beer Temple",
"url": "https://www.yelp.com/"
},
{
"name": "Tales & Spirits",
"url": "https://www.yelp.com/"
}
]
})
For querying on single properties, say for instance you want to query on just the name field, better use the $in operator which is better optimised for such:
query = yelp.map(function(item){ return item.name; });
db.collection.find({ "name": { "$in": query } });
I want to store links and text. The links can be of image, video or gifs. And i want to store them in order the user specifies.
Ex: img1, some text, video, img2 - This should be saved as json.
So far I have come up with 2 options:
First:
{
"content": {
"0": ["image", "image-link-here"],
"1": ["text", "this is some text here"],
"2": ["image", "2nd-image-link"],
"3": ["video", "video-link"]
}
}
Second: In this case, i can know if its image, video, gif or text by extension
{
"content": ["https://somelink.com/pic.jpg", "this is a text", "https://somelink.com/pic2.png", "https://somelink.com/vid.mp4"]
}
I need to store these in DynamoDB. So which one would be good and correct keeping in mind that I expect DB to grow?
If both these approaches are bad, please suggest the good way to do it.
You should try to make arrays where each element has the same type/structure, as it will enable better searches.
The second solution is not specific enough. As a text could end with ".jpg", you'll need a more elaborate test to determine whether it is just text or not. The text might even look a lot like a URL...
The first is better, but it is not really helpful to have numerical keys. Instead you should combine per type, and use that type as the key name, and put the actual value(s) in an array value, like this:
{
"content": {
"images": ["image-link-here", "2nd-image-link"],
"texts": ["this is some text here"],
"videos": ["video-link"]
}
}
This structure should allow for most practical searches.
As you indicated in comments that you need to know the order of each item, then I would suggest to define content as an array, where the occurrence of elements in that array represent the order of them:
{
"content": [
{ "image": "image-link-here" },
{ "text": "this is some text here" },
{ "image": "2nd-image-link" },
{ "video": "video-link" }
]
}
Or, to make each of the objects have the same properties:
{
"content": [
{ "type": "image", "value": "image-link-here" },
{ "type": "text", "value": "this is some text here" },
{ "type": "image", "value": "2nd-image-link" },
{ "type": "video", "value": "video-link" }
]
}
The choice depends on which kind of queries you intend to do.
I want to convert the data from an Excel file to a JSON file. However, I'm not sure about the design of my JSON code (i.e. is it organized in a proper way in order to process it easily?)
I will use this JSON file with D3.js.
This a small part of my Excel file:
I'd like to convert this data into a JSON file in order to use it with D3.js. This is what I have so far:
So my question is: is this a good design (way) for organizing the data in order to use it with D3.js?
This is a sample output:
Thanks in advance!
This is a somewhat subjective question, but from my experience, there is a better way:
Since you're working in d3, you're probably doing something like this:
d3.selectAll('div')
.data(entities)
.enter()
.append('div')
...
So you want entities to be an array. The question is what are your entities? Is there a view where entities are all the countries in the world? Is there a view where entities are all the countries plus all the regions plus the whole world? Or, are all the views going to be simply all the countries in a selected region, not including the region itself?
The unless the JSON structure you're proposing matches the combinations of entities that you plan to display, your code will have to do a bunch of concat'ing and/or filtering of arrays in order to get a single entities array that you can bind to. Maybe that's ok, but it will create some unnecessary amount of coupling between your code and the structure of the data.
From my experience, it turns out that the most flexible way (and also probably the simplest in terms of coding) is to keep the hierarchy flat, like it is in the excel file. So, instead of encoding regions into the hierarchy, just have them in a single, flat array like so:
{
"immigration": [
{
"name": "All Countries"
"values: [
{ "Year": ..., "value": ... },
{ "Year": ..., "value": ... },
...
]
},
{
"name": "Africa"
"values: [
{ "Year": ..., "value": ... },
{ "Year": ..., "value": ... },
...
]
},
{
"name": "Eastern Africa"
"continent": "Africa"
"values": [
{ "Year": ..., "value": ... },
{ "Year": ..., "value": ... },
...
]
},
{
"name": "Burundi"
"continent": "Africa"
"region": "East Africa"
"values": [
{ "Year": ..., "value": ... },
{ "Year": ..., "value": ... },
...
]
},
{
"name": "Djibouti"
"continent": "Africa"
"region": "East Africa"
"values": [
{ "Year": ..., "value": ... },
{ "Year": ..., "value": ... },
...
]
},
...
]
}
Note that even though the array is flat, there is still a hierarchy here -- the region and sub-region properties.
You'll have to do a bit of filtering to extract just the countries/regions you want to show. But that's simpler than traversing the hierarchy you're proposing:
var africanEntities = data.immigration.filter(function(country) {
return country.continent == "Africa";
}); // Includes the region "East Africa"
var justCountries = data.immigration.filter(function(country) {
return country.continent != null && country.region != null;
});
Also, d3 has the awesome d3.nest(), which lets you turn this flat data into hierarchical one with little effort:
var countriesByContinent = d3.nest()
.key(function(d) { return d.continent; })
.map(data.immigration);
var africanEntities = countriesByContinent['Africa'];
Hope that helps....