We Keep Coding

Res.send sends an empty object, even though console.log shows it isn't empty?

I am trying to use the google-sheets api with express and don't have much experience with javascript. I'm attempting to use pass a json object from express to react, but it seems that whenever I finally send the object, it just renders as empty on the frontend?
I've tried using res.body/res.data, but the object doesn't seem to have either. I've also tried to put as many awaits as I can everywhere to make sure the object is loaded in before sending, but nothing seems to do the trick. If I use res.json or res.send with just the response object, I get a circular structure converting to JSON error. Here is the code I'm working with.
async function docShit() {
// Initialize the sheet - doc ID is the long id in the sheets URL
const doc = new GoogleSpreadsheet(
"--SPREADSHEET ID--"
);
// Initialize Auth - see https://theoephraim.github.io/node-google-spreadsheet/#/getting-started/authentication
await doc.useServiceAccountAuth({
// env var values are copied from service account credentials generated by google
// see "Authentication" section in docs for more info
client_email: process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL,
private_key: process.env.GOOGLE_PRIVATE_KEY,
});
await doc.loadInfo(); // loads document properties and worksheets
const sheet = doc.sheetsByTitle[--WORKSHEET TITLE--];
const rows = await sheet.getRows(); // can pass in { limit, offset }
return rows;
}
app.get("/home", async (req, res) => {
try {
await docShit()
.then((response) => {
res.send(Promise.resolve(response)); //console log shows the object, but res.send just sends nothing??
})
.catch((err) => console.log(err));
} catch (err) {
console.error(err.message);
}
});

There is no res.send at all in your code. Also, you use await and .then together, but I consider them alternatives. Try the following:
app.get("/home", async (req, res, next) => {
try {
var response = await docShit();
console.log(response);
/* If response is circular, decide which parts of it you want to send.
The following is just an example. */
res.json(response.map(function(row) {
return {id: row.id, cells: row.cells.map(function(cell) {
return {id: cell.id, value: cell.value};
};
})};
} catch (err) {
console.error(err.message);
next(err);
}
});

return results from .then()

Ok, so I need to connect to a MySQL database through SSH and the connection works fine. I am able to execute queries with no problem. I can also get the results and print it out. The thing is, I need something simpler since I will have to send a lot of queries to this database. Below is the code for a promise which creates a connection to this database.
const SSHConnection = new Promise((resolve, reject) => {
sshClient.on('ready', () => {
sshClient.forwardOut(
forwardConfig.srcHost,
forwardConfig.srcPort,
forwardConfig.dstHost,
forwardConfig.dstPort,
(err, stream) => {
if (err) reject(err);
const updatedDbServer = {
...dbServer,
stream
};
const connection = mysql.createConnection(updatedDbServer);
connection.connect((error) => {
if (error) {
reject(error); // Return error
}
resolve(connection); // OK : return connection to database
});
});
}).connect(tunnelConfig);
});
Then I have this piece of code that gives me access to said connection and allows me to send queries. The problem is that I need the return value of my queries and be able to use it in other modules for my project. For example, export a single function to be used to send queries like sendQuery('Enter SQL here').
function sendQuery(sql) {
SSHConnection.then(
function(connection) {
connection.query(
sql,
function(err, results, fields) {
return results; // <---------- I want to return this from the function 'sendQuery()'
}
);
},
function(error) {
console.log("Something wrong happened");
}
);
}
I can work with the results inside SSHConnection.then() but this isn't functional for me.
I would like to have something similar below to work.
// Main function
(async function() {
let res = sendQuery(`SELECT 23+2 AS Sum;`);
console.log(res); // Outputs Sum: 25
})();
So to my question. Is there a way to access the results from a query inside of a promise.then(), from the outside?

I think the problem is you need to add another return statement to your code.
function sendQuery(sql) {
return SSHConnection.then(
function(connection) {
return connection.query(
sql,
function(err, results, fields) {
return results; // <---------- I want to return this from the function 'sendQuery()'
}
);
},
function(error) {
console.log("Something wrong happened");
}
);
}
This should return the results from the query properly IF connection.query returns a promise. I'm not sure if it does. If it does then you can just execute the function like so.
// Main function
(async function() {
let res = await sendQuery(`SELECT 23+2 AS Sum;`);
console.log(res); // Outputs Sum: 25
})();
If connection.query does not return a promise then I suppose you could wrap it in a promise like so.
function sendQuery (sql) {
return SSHConnection.then(
function (connection) {
return new Promise((resolve, reject) => {
connection.query(
sql,
function (err, results, fields) {
if (err)reject(err)
resolve(results) // <---------- I want to return this from the function 'sendQuery()'
}
)
})
},
function (error) {
console.log('Something wrong happened')
}
)
}

love the name by the way...really liked that movie. As to your question, I'd suggest a couple of things:
if you have a lot of queries coming, you might consider moving the connection independent of the query, so that the connection setup and teardown isn't part of the cost of time for the query itself. If you have a single connection, single DB, etc., then you could instantiate the connection once, at startup, and then leave the connection open, and reference it in your queries.
your question:
function sendQuery(sql) {
const resValues = await SSHConnection.then(
function(connection) {
connection.query(
sql,
function(err, results, fields) {
return results; // <---------- I want to return this from the function 'sendQuery()'
}
);
},
function(error) {
console.log("Something wrong happened");
}
);
return(resValues); // or handle error cases
}
Note I added a return value from the ".then()" call, which captures your "results" return value, and then returns that from the parent function (sendQuery)

Reading data off JSON file and saving it to database

I am trying to upload a JSON file to my express app, read data from the JSON file and save it as a new document in my MongoDB database.
The documents get saved but the fields don't get added, and that is due to the fields being undefined for some reason, How do I fix this?
When I console.log(obj) i can see the data but when I console.log(obj['DataSource_Name']) for example I get undefined why?
app.post("/upload-config", uploads.single("txt"), async (req, res) => {
// Read the file and send to the callback
fs.readFile(req.file.path, handleFile);
let obj;
// Write the callback function
function handleFile(err, data) {
try {
obj = JSON.parse(data);
console.log(obj["DataSource_Name"]);
} catch (err) {
console.log(err);
}
}
const config = new ConfigM({
DataSource_Name: obj.DataSource_Name,
DataSource_ID: obj.DataSource_ID,
DataSource_Type: obj.DataSource_Type,
DataSource_ICON: obj.DataSource_ICON,
DoubleQuotes_Text: obj.DoubleQuotes_Text,
Delimeter: obj.Delimeter,
Field_list: obj.Field_list,
});
try {
await config.save();
res.send(obj);
} catch (err) {
console.error(err);
}
});

obj is only going to be defined when fs.readFile is ready which is NOT as soon as you called it. That happens asyncronously. So after fs.readFile() is called, the code just continues executing, creating the config object and obj is still not defined.
If you would like to use callback functions as you have here, you probably want your config creation, config.save() and res.send(obj) to be part of the readFile callback. Maybe to try before refactoring you could simply move the closing } right after the } of your final catch.
app.post("/upload-config", uploads.single("txt"), async (req, res) => {
// Read the file and send to the callback
fs.readFile(req.file.path, handleFile);
let obj;
// Write the callback function
function handleFile(err, data) {
try {
obj = JSON.parse(data);
console.log(obj["DataSource_Name"]);
} catch (err) {
console.log(err);
}
const config = new ConfigM({
DataSource_Name: obj.DataSource_Name,
DataSource_ID: obj.DataSource_ID,
DataSource_Type: obj.DataSource_Type,
DataSource_ICON: obj.DataSource_ICON,
DoubleQuotes_Text: obj.DoubleQuotes_Text,
Delimeter: obj.Delimeter,
Field_list: obj.Field_list,
});
try {
await config.save();
res.send(obj);
} catch (err) {
console.error(err);
}
}
});
I am not certain what you precisely are doing by reading the file locally while you got it from the post. Be aware that you will want a lot more security guards all over the place and not accepting path names from the request to decide the path name on your server. But I guess that's all a topic for another stackoverflow question :)

Error only callback in async npm module

I am using this async module for asynchronously requesting
web content with the help of another module request, as this is an asynchronous call.
Using async.each method, for requesting data from each link,
the result is also successfully returned by the scrap() function (which I have wrote to scrap returned html data
and return it as array of fuel prices by state).
Now, the problem is that when I try to return prices back to async.each() using cb(null, prices), it shows console.log(prices) as undefined
but logging inside the _check_fuel_prices(), works fine. It seems the callback works with only one argument
(or error only callback, as show as an example in the async.each link above). What if I want to it return prices (I can change it with error like cb(prices), but I also want to log error).
router.get('/someRoute', (req, res, next) => {
const fuels = ['diesel', 'petrol'];
async.each(fuels, _check_fuel_prices, (err, prices) => {
if (!err) {
res.statusCode = 200;
console.log(prices);
return res.json(prices);
}
res.statusCode = 400;
return res.json(err);
});
function _check_fuel_prices(fuel, cb) {
let prices = '';
const url_string = 'http://some.url/';
request(`${url_string}-${fuel}-price/`, (error, response, html) => {
if (error) {
cb(error, null);
return;
}
if (response.statusCode === 404) {
console.log(response.statusCode);
cb('UNABLE TO FIND PAGE', null);
return;
}
prices = scrap(html, fuel);
console.log(prices);
cb(null, prices);
return;
});
}
});

As #generalhenry points out, I was able to get the prices by using async.map which returns error first callback instead of error only apart from that async.series can be used here by slightly changing the code.

Mocha js Calling After() too Soon?

New to Mocha unit testing, I have a few Mocha examples that are running fine, but I have been trying for hours to get this one to run and no matter what I do, after() is called way earlier than I feel it should. Here's an example:
var dummyData = require('./dummyData.js')
describe('mochaTest', function() {
after(function(done) {
dummyData.cleanDb(function(){
done();
})
});
it('should hit the db to get dummy data and send', function(done) {
dummyData.createDummyData(function(data1, data2, Lookup) {
Lookup.lookup({
data1: data1,
data2: data2
}, function(err, result) {
done();
});
});
});
})
And then in dummyData.js:
exports.createDummyData = function(cb){
doSomeStuff(function (err, patient) {
// Connect to db, get some data to pass.
var Lookup = require(./Lookup.js);
cb(data1, data2, Lookup);
})
}
exports.cleanDb = function(cb) {
// Clear db connections.
cb();
}
The problem is that right after the test is run, the after() function gets called and the Lookup function can't hit the db, because the db connection has been cleared. Why is after being called so early, it shouldn't be called until the it statement calls done() right?

This is an old question, but I have experienced the same issue and could not find any explanation to solve this. Unfortunately I do not have enough reputation to answer as a comment, so I'll share how I solved my issue here.
In my controller, I had a method outlined as follows:
exports.insert = (request, response) => {
UserModel.createUser(request.body)
.then(() => {
respond.status(201).send({message: 'User created successfully
});
};
I realized the issue here, was that my test method wasn't waiting for a response from my User.insert(), because this function is void -- it does not return a value. So the following test would jump straight to calling done(); since there was no reason to wait for a response from User.insert(). This was causing my after hook to run prematurely and close the connection to my database before my UserModel could populate.
// Code is shortened, but my request and response was formed using node-mocks-http package
it('should return status 201', function (done) {
User.insert(request, response);
assert(response._getStatusCode() === 201);
done();
};
I hate to say how long this took me to realize this. But, I needed to return a promise from my controller, so that my test function would have something to wait on. By changing my controller to this:
exports.insert = async (request, response) => {
await UserModel.createUser(request.body)
.then(() => {
response.status(201).send({message: "User created successfully"});
})
.catch(() => {
response.status(400).send({message: "There is already an account with that email"});
});
return response;
And with my test looking something like this:
it('should return a status code of 201', async function () {
await User.insert(request, response);
assert(response._getStatusCode() === 201);
});
The database populates successfully, and my test will wait for my promise to fulfill, thus updating the response before calling my assert. Then the after hook works when expected.
I hope this helps someone else who stumbles across this old thread.