We Keep Coding

Mongodb returns an empty array while retrieving data through nodejs

let mongodb = require('mongodb').MongoClient;
let express = require("express")
let app = express()
let connectionString = 'mongodb://ToDoAppUser:ToDoAppUserPassword#ac-u9kgapm-shard-00-00.8rdkdoi.mongodb.net:27017,ac-u9kgapm-shard-00-01.8rdkdoi.mongodb.net:27017,ac-u9kgapm-shard-00-02.8rdkdoi.mongodb.net:27017/?ssl=true&replicaSet=atlas-68qno6-shard-0&authSource=admin&retryWrites=true&w=majority'
let db
mongodb.connect(connectionString,function(err,client){
if (err) throw err
db = client.db()
app.listen(3000)
console.log("Database connected.");
})
app.use(express.urlencoded({extended : false}))
Trying to retrieve data from MongodB
As you can see that , I am trying to retrieve data from MongoDB collection named #item and want to print it. But it shows an empty array. I am stuck on this. kindly help me to resolve this issue.
app.get("/", function(req, res){
**// this collectio method of mongodb returns empty array.
// however, mongodb got connected, but i cannot retreive data from mongodb**
db.collection('items').find().toArray(function(err, items) {
if(err) console.log(err)
console.log(items)
})

You need to use the following format.
async function findOneListingByName(client, nameOfListing) {
const result = await client.db("sample_airbnb").collection("listingsAndReviews").findOne({ name: nameOfListing });
if (result) {
console.log(`Found a listing in the collection with the name '${nameOfListing}':`);
console.log(result);
} else {
console.log(`No listings found with the name '${nameOfListing}'`);
}
}
This code above is worked for me.
By the way, you can read their documentation here for more examples:
https://www.mongodb.com/developer/languages/javascript/node-crud-tutorial/

My guess is that the call to fetch items from DB is asynchronous, and you're trying to use items synchronous manner.
Try adding async to the controller function and using await for the DB request. Like this:
app.get("/", async function(req, res){
/* Mongo documents don't show any parameters for the toArray method
* Read here https://www.mongodb.com/docs/manual/reference/method/cursor.toArray/#mongodb-method-cursor.toArray
*
*/
const itemsFromDB = await db.collection('items').find().toArray()
conssole.log('items are:' itemsFromDB )
})

How to send data from NodeJS server side to the JS client side, only when data is ready?

On my website, when the user clicks on a button, some user's data will be stored in a database and after that I want the server to send notification data to the Javascript frontend file to change the UI.
Right now, the Js file (index.js) receives data right after the website loads (always false). I want it to be received only when the data is ready on the server.
I searched a lot but couldn't find an answer to my problem?
I appreciate any help :)
server.js
var requestValidation = false;
app.post("/", function(req, res){
var name = req.body.personName;
var email = req.body.personEmail;
var collabTopic = req.body.collabTopic;
const newUser = new User({ //mongoDB schema
name: name,
email: email,
collabTopic: collabTopic
});
newUser.save(function(err){ //adding data to mongoDB
if(!err){
requestValidation = true;
}
});
});
app.get("/succ", function(req, res){
res.json(requestValidation);
});
index.js
const url = "http://localhost:3000/succ";
const getData = async (url) => {
try {
const response = await fetch(url);
const json = await response.json();
console.log(json);
} catch (error) {
console.log(error);
}
};
getData(url);

I'm not sure this is completely the answer you're looking for, but it's definitely a tool/feature to consider as you rework your approach.
app.post("/", async (req, res) => {
let result = await INSERT MONGODB UPDATE OR INSERT FUNCTION;
res.render("YOUR TEMPLATE", result);
});
You probably can't plug and play this, but when you finish a MongoDB operation, it returns a json object with some details on whether or not there was success. For example, a MongoDB insert operation returns something like this (stored in the variable result that I created)
{ "acknowledged" : true, "insertedId" : ObjectId("5fd989674e6b9ceb8665c57d") }
and then you can pass this value on as you wish.
Edit: This is what tkausl referred to in a comment.

Here is an example if you want to pass the content of a txt file to the client with express and jquery:
in express:
app.get('/get', (req, res) => {
fs.readFile('test.txt', (err, data) => {
if (err) throw err;
return res.json(JSON.parse(data));
})
})
jquery in client side:
$.getJSON( "http://localhost:3000/get", function( data ) {
geojsondata1 = JSON.stringify(data)
}
now you can do anything you want with the variable data

SQL to MongoDB Migration using nodejs script

Im new to nodejs and Im currently doing an sql to mongodb migration. I have created a script to load data to mongodb from sql queries. I created the script with the sample code from Google and it is working. But im facing below issue and need a workaround for this.
I have an sql query array and I don't need to run those queries if any of the queries has any syntax issues or any errors in the query result. (Say if the second query has syntax issue then no need to load the data of first query to mongo, currently its loading in my case). Basically if any of the query has any issue then no need to load the result in the mongo collection. And also if any issues from the mongo side no need to commit the transactions.
I have used the mongo transactions here to roll back the data if any errors. please find the below code and any help would be much appreciated.The sql and mongo credentials are mock data only.
config file code
var mongoCollection = 'collectionName';
exports.mongoCollection = mongoCollection;
var queryList = [
'sample query one',
'sample query two '
];
exports.queryList = queryList;
main script code
var MongoClient = require('mongodb').MongoClient;
var sql = require('mysql');
const config = require('./assets/config');
var sqlConfig = {
user: 'username',
password: 'password',
server: 'servername',
database: 'databasename',
port: 'portname',
multipleStatements: true
};
async function transaction() {
const mongodbUrl = 'mongourl';
const client = await MongoClient.connect(mongodbUrl, {useNewUrlParser: true}, {useUnifiedTopology:
true});
const db = client.db();
config.queryList.forEach(query => {
new sql.ConnectionPool(sqlConfig).connect().then(pool => {
return pool.request().query(query)
}).then(result => {
(async()=>{
const session = client.startSession();
session.startSession({
readConcers: {level: 'snapshot'},
writeConcern: {w: 'majority'}
});
try {
const collection = client.db('mongodbName').collection(config.mongoCollection);
await collection.insertMany(result.recordset, {session});
await session.commitTransaction();
session.emdSession();
console.log('transaction completed');
}catch(error){
await session.abortTransaction();
session.endSession();
console.log('transaction aborted');
throw error;
}
});
sql.close();
}).catch(error => {
sql.close();
throw error;
})
});
};
transaction();

Depending on the volume of data, you might look at breaking the process into two parts
Get the data from mySql
If no errors, load into Mongo
That would save you having to roll back the mongo writes
You can also take advantage of the default mongo pool size (5) and use pool on the mySQL side too.
Currently, this code is creating a pool for every select, which isn't optimal
config.queryList.forEach(query => {
new sql.ConnectionPool(sqlConfig).connect().then(pool => {//<-New pool per query?
return pool.request().query(query)
})
})
Instead, you can set up a pool once, per the mySql documentation
It looks like that driver only has a callback api, but you can promisfy the query to make it easier to work with.
So to put it all together, you could try something like this (this isn't working/tested code, just a suggestion)
var MongoClient = require('mongodb').MongoClient;
var sql = require('mysql');
const config = require('./assets/config');
var pool = sql.createPool({
connectionLimit : 5,
host : 'servername',
user : 'username',
password : 'password',
database : 'databasename'
});
async function transaction() {
try{
const mongodbUrl = 'mongourl';
const client = await MongoClient.connect(mongodbUrl, {useNewUrlParser: true}, {useUnifiedTopology: true});
const db = client.db();
const collection = client.db('mongodbName').collection(config.mongoCollection);
//Map your query list to an array of runSql promises
//this will complete when all queries return, and jump to the catch if any fail
let results = await Promise.all(config.queryList.map(runSql))
//Map the results to an array of mongo inserts
let inserts = await Promise.all(results.map(r=>collection.insertMany(r.recordset)))
//Close all connections
pool.end((err)=>err?console.err(err):console.log('MySQL Closed'))
client.close((err)=>err?console.err(err):console.log('MongoDB Closed'))
}
catch(err){
console.error(err)
}
};
transaction();
function runSql(queryStr){
return new Promise((resolve, reject)=>{
pool.query(queryStr, function (error, results, fields){
error?reject(error):resolve(results)
})
})
}
If data volume is a concern, you might want to look at getting streams from your mySql selects instead of just running them

Exporting data from the function [mysql, node.js, discord.js]

I have such a problem here with exporting data from functions. I do not know if it can be done at all, but I do not see any other solution here. My problem is that I am exporting a function and I would like to export the result of this function, so in this case I have MYSQL. I cannot add rows to module.exports = {sql, rows} because I get a message that rows is undefined. I am asking for help or for some other solution.
//------------------------------
📁 index.js
//------------------------------
const mysql = require('mysql')
const db = require('./database')
var con = mysql.createConnection({
host: db.host,
user: db.user,
password: db.password,
database: db.database
})
con.connect(err => {
if(err) console.log(err)
})
function sql(sql){
con.query(sql, (err, rows) => {
if(err) console.log(err)
})
}
module.exports = { sql, rows }
// con.end()
//------------------------------
📁 Command file
//------------------------------
const sql = require('./../config/test')
sql.sql("SELECT * FROM `servers`")
console.log(sql.rows)
//------------------------------
📁 Console error
//------------------------------
(node:30132) UnhandledPromiseRejectionWarning: ReferenceError: rows is not defined

You can make use of a Callback function in which you can use the data how you want. By passing a function as a parameter to the sql function, you can invoke that callback function with the rows fetched from the database as a parameter.
Take a look at the code I edited for you:
Your index.js
/* Skipped all code above that isn't important for this example */
function sql(query, callback){
con.query(query, (err, rows) => {
if(err) console.log(err);
callback(rows);
});
}
module.exports = {sql}
Your Command file
const queryHandler = require('./../config/test');
// Call the sql function and pass a callback as the 2nd parameter
queryHandler.sql("SELECT * FROM `servers`", (rows) => {
// Do whatever you want with the rows
console.log(rows);
});
Give it a try and see how it goes

Read SQLite database with Node.js

Having this SQLite DB I'm trying to read the data from it. So, from table Athlete I want to read the first 3 columns.
This is the code (app.js):
var sqlite3 = require('sqlite3').verbose();
var db = new sqlite3.Database('ocs_athletes');
db.serialize(function () {
db.each('SELECT athlete_id, name, surname FROM Athlete', function (err, row) {
console.log('User: ', row.athlete_id, row.name, row.surname);
});
});
db.close();
The file app.js is in the same folder as the db file - ocs_athletes.
Running in cmd node app.js returns this error message:
/home/dd/Documents/Projects/OCSapp/app.js:6
console.log('User: ', row.athlete_id, row.name, row.surname);
^
TypeError: Cannot read property 'athlete_id' of undefined
at /home/dd/Documents/Projects/OCSapp/app.js:6:31
at replacement (/home/dd/Documents/Projects/OCSapp/node_modules/sqlite3/lib/trace.js:25:27)
at Statement.errBack (/home/dd/Documents/Projects/OCSapp/node_modules/sqlite3/lib/sqlite3.js:14:21)
Why is this happening?

Try connecting to db like this.
let db = new sqlite3.Database('./ocs_athlete.db', (err) => {
if (err) {
console.error(err.message);
}
console.log('Connected to the my database.');
});
Give path of .db file. This will work.
There are three opening modes:
sqlite3.OPEN_READONLY: open the database for read-only.
sqlite3.OPEN_READWRITE : open the database for reading and writting.
sqlite3.OPEN_CREATE: open the database, if the database does not exist, create a new database.
To open the chinook sample database for read and write, you can do it as follows:
let db = new sqlite3.Database('./ocs_athlete.db', sqlite3.OPEN_READWRITE, (err) => {
if (err) {
console.error(err.message);
}
console.log('Connected to the ocs_athlete database.');
});