Bulk Insert into Mysql from Object Array NodeJS - javascript

I'm not sure what I'm missing, I thought putting Games in brackets would let me do bulk insert to MySQL. However when I do this it comes back as (OBJECT,OBJECT). If I remove the brackets then I only get one single insert but it works! Any suggestions to do bulk insert?
Short sample of the Array
[{title:xxx,link:https://xxxx,description:xxx.,xxx:xxxxx},{title:xxx,link:https://xxxx,description:xxx.,xxx:xxxxx}]
Javascript
// Writing the Games inside a json file
fs.writeFile("feed.json", JSON.stringify(Games), function(err) {
if (err) throw err;
console.log("Saved!");
});
con.connect(function(err) {
if (err) throw err;
console.log("Connected!");
var sqlquery = "INSERT INTO gtable (title, link, description, company) VALUES ?";
let queryData = {
sql: sqlquery,
values: [Games]
}
con.query(queryData, function (err, result) {
if (err) throw err;
console.log("Number of records inserted: " + result.affectedRows);
});
});

I believe your issue may be that you are passing an array of objects while MySQL is expecting an array of arrays.
Your format is basically:
var sqlquery = "INSERT INTO gtable (title, link, description, company) VALUES ?";
var values = [
[{title:xxx,link:https://xxxx,description:xxx.,xxx:xxxxx}
{title:xxx,link:https://xxxx,description:xxx.,xxx:xxxxx}]
];
When it should be:
var sqlquery = "INSERT INTO gtable (title, link, description, company) VALUES ?";
var values = [
[[title1,link1,description1,xxx1],
[title2,link2,description2,xxx2]]
];
You would need to convert each object to an array of just its values. Which you could do with something like this I think:
let sqlValues = Games.map(obj => Object.values(obj));
EDIT: I had the incorrect format for the mapping, fixed now.

I believe #otw has a solid solution (and an awesome explanation of why this is happening), but to offer an alternative solution you could do something like this:
Games.forEach(game => {
con.query('INSERT INTO gtable SET ?', game, insertErr => {
if (insertErr) {
throw new Error("Error inserting game " + game.title + "! " + insertErr);
}
console.log(`Inserted ${game.title}!`);
});
})

Related

Functions out of order when getting results from SQL

When an existing SQL record exists I want to use it rather than adding another, but if it doesn't yet exist I want to add it. The issue I am having is that when my Node.js app's endpoint is called it's not executing in the correct order so the SQL lookup to find existing records is happening after I check it's length to see if I need to add a new record.
// it does this second
let existingGet = "SELECT * FROM Items WHERE name = '" + productName + "'";
let existingItem = async () => {
db.query(existingGet, function (err, rows, fields) {
return rows;
});
};
// it does this first
if (await existingItem().length > 0) {
// Existing found, use existing
itemId = existingItem.ID;
} else {
// Item not found, create new
var sql = "INSERT INTO Items (...) VALUES (...)";
await db.query(sql, async function (err, result) {
itemId = existingItem.affectedRows.ID;
});
}
The desired outcome is that it does the first section before the second section because the second section needs the results of the first.
Try removing the outer brackets so that the existingItem will receive the result from the query
// it does this second
let existingGet = "SELECT * FROM Items WHERE name = '" + productName + "'";
//removed outer brackets
let existingItem = async () =>
db.query(existingGet, function (err, rows, fields) {
return rows;
});
// it does this first
if (await existingItem().length > 0) {
// Existing found, use existing
itemId = existingItem.ID;
} else {
// Item not found, create new
var sql = "INSERT INTO Items (...) VALUES (...)";
await db.query(sql, async function (err, result) {
itemId = existingItem.affectedRows.ID;
});
}
Do it in a single db call using sql command
"INSERT INTO Items (...) VALUES (...)
WHERE NOT EXISTS (SELECT 1 FROM Items WHERE name = ...)"
And use sql command parameters instead of concatenation to avoid sql injection.

Output always gives []

I am trying to filter some data by "averageRating".
This is my code for the method:
filterr(request, respond) {
var averageRating = request.params.rating;
var sql = "SELECT * FROM shopreview.shops WHERE averageRating = ?";
db.query(sql, [averageRating], function (error, result) {
if (error) {
throw error;
}
else {
respond.json(result);
}
});
}
My sql statement is working when I test it against my database. However, I keep getting [] as my result. Can someone please help identify what the problem is? Thanks a lot!
the problem is that "?" since the db is unable to parse it.
either add that avarageRating variable like so:
var sql = "SELECT * FROM shopreview.shops WHERE averageRating = ${parseInt(avarageRating)}";
or if you're using couchbase you could parse it like this:
var sql = `SELECT * FROM shopreview.shops WHERE averageRating = $1`;
where $1 is the first variable in the array of variables.

In nodejs SQLite3 .each() method, how can I add mulitple arguments to work with multiple "WHERE ... = ?" queries?

I have a table that has a column named CRS_TITLE and I would like to search a list of course titles within that column. Right now my code only works with one argument at a time. I suppose I can write a simple loop, and during each iteration I can just call db.each() again with a new class name. I know this way is inefficient, so I would like to check all the arguments within one pass.
Things I have tried:
1) Adding another '?' in WHERE and other variations.
2) Using the new spread operator from javascript like [...classes] when passing in the arguments.
None of the above works due to syntax error.
Anyways, here is my code below.
let sqlite3 = require('sqlite3').verbose()
// open the database
let db = new sqlite3.Database('./classes.db', (err) => {
if (err) {
console.error(err.message);
}
console.log('Connected to the classes database.');
});
let sql = `SELECT CRS_CDE course_code,
Column3 start_time,
Column4 end_time
FROM spring_schedule
WHERE CRS_TITLE = ?`;
// I want to do multiple "WHERE = ?" here.
// with multiple arguments
let className1 = 'Spanish I'
// I want to change the line above to this instead.
// let classNames = ['Accounting I', 'English', 'Math'...etc]
db.each(sql, [className1]/*pass an array of items to look up instead of just 1 item.*/, (err, row) => {
if (err) {
throw err;
}
console.log(`${row.course_code} ${row.start_time} - ${row.end_time}`);
// Will use this result to update frontend.
});
// close the database connection
db.close();
EDITED: working code below.
let sql = `SELECT CRS_CDE course_code,
Column3 start_time,
Column4 end_time
FROM spring_schedule
WHERE CRS_TITLE IN `;
let classNames = ['Spanish I', 'Accounting II', 'College Algebra']
let where = '(?' + ',?'.repeat(classNames.length-1) + ')';
sql += where;
db.each(sql, [...classNames], (err, row) => {
if (err) {
throw err;
}
console.log(`${row.course_code} ${row.start_time} - ${row.end_time}`);
// Will use this result to update frontend.
});
db.close();
You may try building a dynamic WHERE IN clause using the exact number of parameters to match your input array:
let sql = `SELECT CRS_CDE course_code, Column3 start_time, Column4 end_time
FROM spring_schedule
WHERE CRS_TITLE IN `;
let classNames = ['Accounting I', 'English', 'Math'];
let where = '(?' + ',?'.repeat(classNames.length-1) + ')';
sql += where;
db.all(sql, [], (err, rows) => {
if (err) {
throw err;
}
rows.forEach((row) => {
console.log(row.name);
});
});
db.close();

filter[0].ID is undefined

I am making a Discord Bot add a database entry when a new user joins. To avoid duplicate entries I'm checking if the database row ID already has the members ID in it. My problem is if the member is not in the database it comes back as undefined.
var userID = member.id.toString();
var UserName = member.user.username.toString();
// var NickName = member.nickname.toString();
var DateJoined = new Date();
con.query("SELECT ID FROM listAllUsers", function (err, selectResult, fields) {
var filter = selectResult.filter(m => m.ID === userID);
console.log(filter[0].ID)
if(filter[0].ID == userID) {
console.log(`That user all ready exists in the database.`)
} else {
var sql = `INSERT INTO listAllUsers (ID, UserName, NickName, DateJoined) VALUES ('${ID}', '${UserName}', 'none', '${DateJoined}')`;
con.query(sql, function (err, result) {
if (err) throw err;
console.log(`User: ${UserName} -- ${ID} has joined the server. Added to the Database`);
});
}
});
var filter = selectResult.filter(m => m.ID === userID);
This checks if the ID is equal to any of the ID's in the database. But this is also my problem. Because if this comes back empty it will make this:
console.log(filter[0].ID) undefined. Well actually the .ID bit comes back undefined.
Well if you have an entry, than you already know the id exists....
if (filter.length > 0) {
// you have a user with the id
}
But wouldn't it make more sense to just query the user id instead of returning everyone? SELECT ID FROM listAllUsers WHERE ID=userID

sql is only returning 1 row, how do I access the rest?

sql.get(`SELECT * FROM scores ORDER BY points DESC`).then(allScores => {
console.log(allScores);
});
This should give me all of the rows ordered by points, but I'm only getting the first row.
How do I access all of the other rows using javascript?
Use sql.all instead of sql.get refer http://www.sqlitetutorial.net/sqlite-nodejs/query/
Define your db instance as follows:
const sqlite3 = require('sqlite3').verbose();
// open the database
let db = new sqlite3.Database('./db/yourCoolDB');
And then,
let sql = `SELECT * FROM scores ORDER BY points DESC`;
db.all(sql, [], (err, rows) => {
if (err) {
throw err;
}
rows.forEach((row) => {
console.log(row.Id); // You can use row.yourAnotherAttributeName
});
});

Categories