We Keep Coding

How do I break up a JSON object to insert into a MySQL table?

Summary
I have some JSON coming back me from this:
// Now Get Only New Listings
finalArray = _.difference(existingResults, items);
and I need to insert those values into this:
Some of the data coming in is not available to fill in all columns. In that case, I would like to leave them blank.
Here is what the data looks like coming in (one of many presumably)...
{
price: '$6,000',
title: '1999 Jeep Wrangler SE Sport Utility 2D',
location: 'Auburndale, Florida',
miles: '12K miles',
imgUrl: 'http://www.example.com/pic.jpg',
itemURL: '/products/item/36427/'
}
So I need to figure out how to go through every JSON object, and insert into the correct columns.
var insertQuery = 'Your code here! :) ';
I suppose that query needs to be in a loop somewhere? Or one big insert statement?
Any advice or code would be really appreciated. Thanks.
Note
Using: NodeJS and JawsDB MySQL ( in case that info is important )

The following approach will only work if your payload doesn't have any nested objects.Used ES6 Maps to do the work. I have used connection.escape() to remove any injectable scripts within the JSON to prevent SQL injection attacks. I hope that you have already install mysql connector for node.js.
const mysql = require('mysql');
const connection = mysql.createConnection({
host : 'localhost',
user : 'me',
password : 'secret',
database : 'my_db'
});
connection.connect();
const payload = {
price: '$6,000',
title: '1999 Jeep Wrangler SE Sport Utility 2D',
location: 'Auburndale, Florida',
miles: '12K miles',
imgUrl: 'http://www.example.com/pic.jpg',
itemURL: '/products/item/36427/'
}
const map = new Map(Object.entries(payload));
const keys = map.keys();
const values = map.values();
const insertQuery = `
INSERT INTO TABLE_NAME (${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)})
VALUES (${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)});
`
connection.query(insertQuery, function (error, results, fields) {
if (error) throw error;
// ...
});
console.log(insertQuery);

Parsing JSON in node express - ends up with \n between delimiters?

I'm sending an Ajax request to an express endpoint, like so:
var postData = {
method: "POST",
url: "/order/address",
data: { order: JSON.stringify(addressFields) },
cache: false
};
updateAjax = $.ajax(postData).done(function(data) {
The address object being sent is a JSON array. I have confirmed it is not already stringified prior to doing this in the post data object.
It posts to a server running node express, with a path of /order/address.
The express route is as follows, only showing up to the point the json gets parsed, as it's the only relevant parts of this:
router.route('/')
.post(function (req, res, next) {
// prevents caching of customer information
res.header('Cache-Control', 'private, no-cache, no-store, must-revalidate');
const addresses = JSON.parse(req.body.order);
Prior to being parsed, req.body.order is :
[{\"isDefaultShipping\":true,\"firstName\":\"\",\"lastName\":\"\",\"street1\":\"\",\"street2\":\"\",\"city\":\"\",\"postcode\":\"\",\"region\":\"\",\"country\":\"GB\",\"phone\":\"\",\"isDefaultBilling\":true}]
And, here is the problem I'm having, after parsing it ends up as an invalid JSON object:
[\n {\n isDefaultShipping: true,\n firstName: '',\n lastName: '',\n street1: '',\n street2: '',\n city: '',\n postcode: '',\n region: '',\n country: 'GB',\n phone: '',\n isDefaultBilling: true\n }\n]
I have been unable to remove the line breaks - and I guess there's line breaks that are not visible, within the stringified data that is being parsed - but I can't for the life of me remove them?
EDIT: Someone has kindly pointed out my stupidity (politely), that the output above, with the \n linebreaks, is my logger, so ignore that part.
Leaving it in there - well, because it would be rude to remove this part of my question that I got an answer on.
Postman shows the following issue as it hits the JSON.parse line in the route:
SyntaxError: Unexpected token u in JSON at position 0\n at JSON.parse (<anonymous>)

Just replace \n
router.route('/')
.post(function (req, res, next) {
// prevents caching of customer information
res.header('Cache-Control', 'private, no-cache, no-store, must-revalidate');
const addresses = JSON.parse(req.body.order).replace(\[\n]+\, '');

How to add continuation backslash or character to MongoDB script

I need to write a script to create multiple users and roles in MongoDB from the command line..
Below is an example of one of the scripts.
mongo $DB --eval "db.createUser({ user: '$USER', pwd: '$PASS', roles: [ { role: '$ROLE', db: '$DB' } ] });"
I know in bash, you can simple add back slash to condense a lengthy script
mongo $DB --eval \
"db.createUser({ user: '$USER', pwd: '$PASS',\
roles: [ { role: '$ROLE', db: '$DB' } ] });"
I tried the something similar to bash but the script is treating the slash as a literal string. Any idea how to fix the issue?

The problem is with a space character present after the \ character in the second line. The below command will work for you.
mongo $DB --eval \
"db.createUser({ user: '$USER', pwd: '$PASS',\
roles: [ { role: '$ROLE', db: '$DB' } ] });"

Sequelize convert date from UTC to local

I have a problem with queries in sequelize.js.
In database I store model with myDate field which is stored in UTC. I also have a query where time period is specified:
AND p.myDate BETWEEN :dateStart AND :dateEnd
Using sequelize.query, I replace parameters with:
dateStart: 2018-09-11T22:00:00.000Z
dateEnd: 2018-09-14T14:15:40.609Z
and now the problem is that I see query is executing however with localTime values:
AND p.myDate BETWEEN '2018-09-12 00:00:00' AND '2018-09-14 16:15:40'
Why did it convert it to the local time? I was not able to find the right answer.

You can specify the time zone in which you want to read/write in the config of Sequelize like this.
development: {
username: 'postgres',
password: 'postgres',
database: 'your_database_name',
host: '127.0.0.1',
port: 5432,
dialect: 'postgres',
dialectOptions: {
useUTC: false, // for reading from database
},
timezone: '+05:30', // for writing to database
},

Inquiry on Node.js HTTP requests/responses and URL as primary key for this JSON-formatted document

I have a script that uses Node.js to request headers from a specific site.
var http = require("http");
var fs = require("fs");
var hostNames = ['www.google.com'];
var options = {
host: hostNames[i],
path: '/'
};
http.get(options, function(res) {
var obj = {};
obj.statusCode = res.statusCode;
obj.headers = res.headers;
console.log(JSON.stringify(obj, null, 4));
})
The output, for the URL "www.google.com" would be attached below:
{
"statusCode": 200,
"headers": {
"date": "Mon, 04 Mar 2013 16:43:39 GMT",
"expires": "-1",
"cache-control": "private, max-age=0",
"content-type": "text/html; charset=ISO-8859-1",
"set-cookie": [
"PREF=ID=cfa31a2cae817ca6:FF=0:TM=1362415419:LM=1362415419:S=m-sNTevwPhFFWVpv; expires=Wed, 04-Mar-2015 16:43:39 GMT; path=/; domain=.google.com",
"NID=67=AKMqJ9Q94GtcmF0kTOAOLgFLqz9XAnSwVe4jzzXFVhvxuxRJP_l9QEwbjR3F7d506thF9BURyGJUz5DuNTEzXesit50Dm7FlOoVuL2qGRt9XZwRMGjAlxL5heO4vIATp; expires=Tue, 03-Sep-2013 16:43:39 GMT; path=/; domain=.google.com; HttpOnly"
],
"p3p": "CP=\"This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info.\"",
"server": "gws",
"x-xss-protection": "1; mode=block",
"x-frame-options": "SAMEORIGIN",
"transfer-encoding": "chunked"
}
}
My question is in-regards to JSON. I am trying to store the output into MongoDB. MongoDB stores JSON-like documents. From my understanding, SQL-based databases have a primary key. This is where my confusion comes in. I would like to use the URL, in this case, 'www.google.com' as the primary key. How do I achieve this? This is my first time using JSON-like storing structures, and the multiple articles I have read do not really apply to my specific situation.
When I search for "www.google.com" in the database, the plan is to have the headers show up, under "www.google.com." I don't know - I think I am still thinking in the SQL mindset. Can someone share some insight to this?

Here is official docs on object ids.
So you can create your own object id for a record using anything with appropriate format (hex number) and length, so this will work
db.names.insert({"_id": new ObjectId("012345678901234567890123"), "name" : "my name" })
but this dont
db.names.insert({"_id": new ObjectId("my reallllly long string"), "name" : "my name" })
you will need to use hash of your url if you want to using at object id.
However mongo gives you another option. leave _id field alone and create url field for url, and than set index on url field
db.scrapedPages.ensureIndex({ 'url': 1})
UPDATE: more specifically to your example.
You are not going to set/change _id property, mongo does it for you.
Instead you are going set url property of document to save, and reasonable thing to use here is your options object, as it defines the page you are parsing.
So I think you'll endup with something like that ( I expect you use mongo native driver and have mongo connection open )
var options = {
host: hostNames[i],
path: '/'
};
http.get(options, function(res) {
var obj = {
url: options.host + options.path // or whatever else is
statusCode : res.statusCode,
headers : res.headers
}
save(obj, function(err, objects) {
if (err) console.warn(err.message);
})
})
function save(doc, callback) {
var collection = new mongodb.Collection(client, 'test_collection')
, cb = callback || function() {}
collection.insert(doc, {safe:true}, cb);
}

The primary key in an SQL table is a column that is used to uniquely identify a particular row. In mongodb _id is the field which is the primary key. mongodb adds it automatically if you don't specify it and assigns an ObjectId (12 byte long BSON identifier) to it. You can check the details here.