Mapping a JSON to MongoDB using Nodejs - javascript

I am writing a script in Nodejs to map a json to mongodb but I am really confused on how to tackle this as the structure is really complex. Any help would be appreciated! So basically the structure looks like this.
{
"4.1": {
"picture": "",
"code": {
"PEK": {
"number": 4.1,
"text": "XYZ."
},
"DXB": {
"number": 5.1,
"text": "XYZ"
},
"LAX": {
"number": 6.1,
"text": "XYZ"
},
"HND": {
"number": 6.1,
"text": "XYZ"
}
}
},
"4.2": {
"picture": "",
"province": {
"PEK": {
"number": 4.2,
"text": "XYZ"
},
"DXB": {
"number": 5.2,
"text": "XYZ"
},
"LAX": {
"number": 6.2,
"text": "XYZ"
},
"HND": {
"number": 6.2,
"text": "XYZ"
}
}
}
}
So I have put two instances for example just so it's easier to understand. What I want is that I am using the the number of "PEK" as a key to access the right object. When I get response from the server I get a number, and the code, for example 4.1 and DXB. Using this information I want to show DXB under 4.1 and also its respective image. All the objects under 4.1 are tied to a common image which is linked to 4.1 but they have their own respective number and text which I want to show at runtime depending on the response. What would be the smartest way to map this to mongodb. Any help would be appreciated, thanks!!

Related

Google Sheets not receiving json data properly from Woocommerce Webhook

I hope you guys are having a wonderful day.
I have set up a webhook in my woocommerce that sends JSON data to Google sheets. The webhook has been working great for months now, just today, I am having some trouble with it. I have tracked the issue to be in google sheets receiving the JSON data, but I don't know why this is happening.
Let me explain.
https://docs.google.com/spreadsheets/d/18G-yVDjYeccl6kznpZgSuRTysRMAu57pwY2oGf6-KWI/edit?usp=sharing
This is the google sheet, when it gets Woocommerce JSON data, it populates a new row.
The problem
Sometimes google sheets doesn't populate the row upon receiving a new order. The problem doesn't lie with woocommerce, because I have checked woocommerce with reqbin and the webhook fires with every order.
Furthermore, when I send requests from reqbin.com to my sheet, the sheet performs the operation successfully 5-6 out of 10 times. Other times it shows an error.
The Error
The error is due to google sheets not being able to parse JSON data, because the JSON data it receives 5 out of 10 times is not proper JSON data. Other 5 times, it is just as it should be. I have put a catch statement if the sheet is unable to parse JSON. Instead of appending new row with the parsed data, it appends the raw received data to the sheet.
It is clear now that there is some issue with google sheets handling that JSON data because when the same data is sent from reqbin.com to webhook.site, it is perfectly as it should be 10/10 times.
How to reproduce the issue
Open this google sheet. https://docs.google.com/spreadsheets/d/18G-yVDjYeccl6kznpZgSuRTysRMAu57pwY2oGf6-KWI/edit?usp=sharing
Open reqbin.com and webhook.site, and send the following JSON from reqbin.com to webhook.site 10 times to see if any kind of error occurs.
{ "id": 47222, "parent_id": 0, "status": "processing", "currency": "PKR", "version": "5.1.0","prices_include_tax": false, "date_created": "2021-06-10T01:23:46", "date_modified": "2021-06-10T01:23:46", "discount_total": "0", "discount_tax": "0", "shipping_total": "150", "shipping_tax": "0", "cart_tax": "0", "total": "1850", "total_tax": "0", "customer_id": 0, "order_key": "wc_order_7gIuR7px6MX9C", "billing": { "first_name": "Name", "last_name": "", "company": "", "address_1": "Address", "address_2": "", "city": "City", "state": "", "postcode": "", "country": "PK", "email": "email#email.com", "phone": "1234" }, "shipping": { "first_name": "Name", "last_name": "", "company": "", "address_1": "Address", "address_2": "", "city": "City", "state": "", "postcode": "", "country": "Country" }, "payment_method": "cod", "payment_method_title": "Cash on delivery", "transaction_id": "", "customer_ip_address": "8.8.8.8", "customer_user_agent": "Mozilla/5.0 (Linux; Android 11; M2102J20SG) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.88 Mobile Safari/537.36", "created_via": "checkout", "customer_note": "", "date_completed": null, "date_paid": null, "cart_hash": "64d834c72eecc8e32b9d83fd67d10d9c", "number": "47222", "meta_data": [ { "id": 869388, "key": "_shipping_calculator", "value": "" }, { "id": 869389, "key": "is_vat_exempt", "value": "no" }, { "id": 869391, "key": "_wfacp_report_data", "value": { "wfacp_total": "0.00" } }, { "id": 869392, "key": "_woofunnel_cid", "value": "4" }, { "id": 869393, "key": "_wfacp_post_id", "value": "24852" }, { "id": 869394, "key": "_wfacp_source", "value": "https://website.com/checkouts/checkout-page/" }, { "id": 869395, "key": "_wfacp_timezone", "value": "Asia/Karachi" }, { "id": 869396, "key": "order_comments", "value": "" }, { "id": 869412, "key": "_new_order_email_sent", "value": "true" }, { "id": 869424, "key": "_woofunnel_custid", "value": "4" }, { "id": 869425, "key": "_pys_purchase_event_fired", "value": "1" }, { "id": 869426, "key": "_wfob_stats_ids", "value": [] }, { "id": 869427, "key": "_wfocu_thankyou_visited", "value": "yes" } ], "line_items": [ { "id": 35114, "name": "MTECH Ultra Resilient Knife", "product_id": 11074, "variation_id": 0, "quantity": 1, "tax_class": "", "subtotal": "1700", "subtotal_tax": "0", "total": "1700", "total_tax": "0", "taxes": [], "meta_data": [], "sku": "", "price": 1700, "parent_name": null } ], "tax_lines": [], "shipping_lines": [ { "id": 35115, "method_title": "Fast Shipping (2-4 Days)", "method_id": "flat_rate", "instance_id": "1", "total": "150", "total_tax": "0", "taxes": [], "meta_data": [ { "id": 275053, "key": "Items", "value": "MTECH Ultra Resilient Knife × 1", "display_key": "Items", "display_value": "MTECH Ultra Resilient Knife × 1" } ] } ], "fee_lines": [], "coupon_lines": [], "refunds": [], "date_created_gmt": "2021-06-09T20:23:46", "date_modified_gmt":"2021-06-09T20:23:46", "date_completed_gmt": null, "date_paid_gmt": null, "currency_symbol": "₨","_links": { "self": [ { "href": "https://website.com/wp-json/wc/v3/orders/47222" } ],"collection": [ { "href": "https://website.com/wp-json/wc/v3/orders" } ] } }
Now send the same data to the following google sheet to see if it appends the row correctly each time.
https://script.google.com/macros/s/AKfycbxupm9bje86F4PQQkyys_LWtXs_kj279R0ipgnZ-cLd7aiEADf1AN_prhk28vOPW9JsRQ/exec
How do I solve the issue? Please let me know if you need any more information. Thanks.
Edit:
Instead of getting a full JSON body like mentioned above, the google sheets seems to be getting the following JSON.
{contextPath=, queryString=, parameter={}, postData=FileUpload, parameters={}, contentLength=3981.0}
I would like to know why the google sheets default parameter (e) contains this instead of a full JSON body sent to it.
Edit # 2
I would like to know why the google sheets default parameter (e) contains this instead of a full JSON body sent to it.
This is because (e) has a body which will always contain those parameters. The error is due to Google Sheets receiving an empty JSON body. I am still unable to understand why this happens. When I send the same JSON to API testing sites, they always receive full JSON body. Google sheets, in some cases, does not. Why is that?
I managed to solve the issue with some trial and error. For anyone facing the same issue in the future, here is what worked for me.
I was using e.postData.contents to get the JSON body but this seems to have stopped working, which was causing the JSON body to be empty. I tried e.postData.getDataAsString(); which seems to be working just fine and the issue has been resolved.

Query language for JSON or how to use Javascript/Lua in Python?

I'm looking solution to query JSON that extracts specific data.
This solution will be embedded inside our Python software (more exactly Python AWS Lambda) and the user should have the ability to specify such query string in GUI.
For example, we have such JSON and I would extract "nazwisko_weterynarza.value" from "files" array if "typ_dokument" is equal to "szczepienie" and "data_szczepienia" is equal to 3434343.
I've tried this using JSON Path but I've failed.
What about embedded Lua/Javascript in Python, but how to create a 'sandbox' to be sure that such a solution will be safe?
{
"files" :
[
{
"meta": {
"updated": 555555,
"attributes": [
{
"typ_dokumentu":
{
"formula_type": "NONE",
"formula": "",
"value_type": "string",
"value": "szczepienie",
"updated": 5345435435345
}
},
{
"data_szczepienia":
{
"formula_type": "NONE",
"formula": "",
"value_type": "int",
"value": 3434343,
"updated": 5345435225345
},
},
{
"nazwisko_weterynarza":
{
"formula_type": "NONE",
"formula": "",
"value_type": "string",
"value": "Nowak",
"updated": 5345435225345
}
}
]
}
},
{
"meta": {
"updated": 555555,
"attributes": [
{
"typ_dokumentu" :
{
"formula_type": "NONE",
"formula": "",
"value_type": "string",
"value": "Certyfikat urodzin",
"updated": 5345435435345
}
},
{
"data_urodzin" :
{
"formula_type": "NONE",
"formula": "",
"value_type": "int",
"value": 8888888,
"updated": 5345435225345
}
}
]
}
}
]
}
The AWS JSON (reference) path queries are not capable enough. Some more advanced JSONPath processors can do it, eg. using Jayway's JsonPath we could run a query like this:
$.files..meta.[?(#.attributes[*].typ_dokumentu.value contains 'szczepienie' && #.attributes[*].data_szczepienia.value contains '3434343')]..nazwisko_weterynarza.value
Reurns ["Nowak"] (You can try it online).
I don't know what you mean by embedded JavaScript/Lua, but in any event, you could create a web service that does the querying for you leveraging an implementation of your choice. Security wouldn't be an issue as long as your API is properly secured.

Ember Difference between Restadapter vs Jsonapiadapter

Ember uses Restadapter & Jsonapiadapter for the adapters.
What are the exact differences between the 2 in terms of data formats for request/response ?
Any other things we need to ensure when using any of these 2.
The JSONAPIAdapter conforms to the JSONApi spec
Use RESTAdapter when you have an JSON API that follows a REST endpoint with pluralized object names and has a root node using the name of the object being returned.
Examples below:
Example JSONAPI spec object:
{
"data": [{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!"
},
"relationships": {
"author": {
"links": {
"self": "http://example.com/articles/1/relationships/author",
"related": "http://example.com/articles/1/author"
},
"data": { "type": "people", "id": "7" }
}
},
}],
"included": [{
"type": "people",
"id": "7",
"attributes": {
"name": "Dave",
"twitter": "kiwiupover"
}
}]
}
Example Rest json api object:
{
"posts": {
"id": 5,
"title": "An API that gets bikeshed for months ",
"author": "kiwiupover",
"comments": [1]
},
"comments": [{
"id": 1,
"name": "Dave",
}]
}
Ember Data provides straightforward methods for adapting your DS.adapter to your specific JSON API shape.
There is a third adapter from which the previously mentioned adapters are extended from.

Iterate through nested Javascript Objects from API response

I've tried 100 different things, and spend days looking through Google and Stackoverflow, but I can't find a solution to this problem. Everything I call after the body of this API response returns undefined!
The response from Facebook SDK looks like this:
[
{
"body": "[
"data": [
{
"name": "Larry Syid Wright",
"administrator": false,
"id": "xxx"
}, {
"name": "Melissa Long Jackson",
"administrator": false,
"id": "xxx"
}, {
"name": "Charlotte Masson",
"administrator": false,
"id": "xxx"
}
],
"paging": {
"next": "url"
}
]"
},{
"body": "{
"data": [
{
"id": "xxx_xxx",
"message": "In honor of Halloween, how many of you have your own ghost stories? Who believes in ghosts and who doesn't?",
"type": "status",
"created_time": "2014-10-31T20:02:01+0000",
"updated_time": "2014-11-01T02:52:51+0000",
"likes": {
"data": [
{
"id": "xxx",
"name": "Joe HerBatman Owenby Jr."
}
],
}
"paging": {
"cursors":
{
"after": "xxx",
"before": "xxx"
}
}
}
},{
"id": "xxx_xxx",
"from": {
"id": "xxx",
"name": "Jessica Starling"
},
"message": "Watching the "Campaign" and I can't help but notice what a fantastic job they did (Will ferrell and all) with that North Carolina accent! Ya'll know we sound different than other southern states ;)",
"type": "status",
"created_time": "2014-11-01T02:36:21+0000",
"updated_time": "2014-11-01T02:36:21+0000",
"likes": {
"data": [
{
"id": "xxx",
"name": "Scott Williams"n
}
]
}
}
],
"paging": {
"previous": "xxx",
"next": "xxx"
}
}"
}
]
This response is from a batch call. If I call them separately, I can easily iterate through the responses, and get everything from them. When I call them in the batch though, I can't get past "body", and I need to use a batch call.
console.log(response[0].body); will return the object inside the body of the first part of the response, but console.log(response[0].body.data); returns undefined. I just don't get it. This should be simple but it's like there's a lock on the door and I don't have the right key.
I normally have no issue iterating through objects, so I don't need a generalized answer. I need help seeing whatever it is here that I don't see. Why does the console show undefined when I call anything after the body, and what do I need to be doing to get any of these values?
That JSON contains nested JSON. body seems to be a string. Use
var body = JSON.parse(response[0].body);
The values from the body are just strings.which are embedded as json.So firstly you would need to parse them using JSON.parse.
The code would be like
var body = JSON.parse(response[0].body);

Unable to publish a Open Graph Action and I receive a weird response

My code used to work, but now I am not able to publish any actions. Everything seems to be in order but the response I get from Facebook is a JSON list of the last 25 actions I took. I know my action isn't published because I'm watching it live in a browser. I'm also trying to post these actions with a browser.
Here is my code:
https://graph.facebook.com/me/NAMESPACE:ACTION?OBJECT=http://www.example.com/pumpkinpie.html&access_token=TOKEN
The response is 25 of these:
{
"data": [
{
"id": ""****"",
"from": {
"id": ""****"",
"name": "****"
},
"start_time": "2012-05-05T04:56:38+0000",
"end_time": "2012-05-05T04:56:38+0000",
"publish_time": "2012-05-05T04:56:38+0000",
"application": {
"id": ""****"",
"name": ""****"
},
"data": {
"website": {
"id": ""****"",
"url": "****",
"type": "website",
"title": "****"
}
},
"likes": {
"count": 0
},
"comments": {
"count": 0
}
},
of course the ** are actual values...
You cannot do HTTP POST for actions in a browser, only HTTP GET, which is why you receive the list of previous actions.
Please use cURL or the Graph API Explorer to accomplish creating an action.

Categories