We Keep Coding

Res.send sends an empty object, even though console.log shows it isn't empty?

I am trying to use the google-sheets api with express and don't have much experience with javascript. I'm attempting to use pass a json object from express to react, but it seems that whenever I finally send the object, it just renders as empty on the frontend?
I've tried using res.body/res.data, but the object doesn't seem to have either. I've also tried to put as many awaits as I can everywhere to make sure the object is loaded in before sending, but nothing seems to do the trick. If I use res.json or res.send with just the response object, I get a circular structure converting to JSON error. Here is the code I'm working with.
async function docShit() {
// Initialize the sheet - doc ID is the long id in the sheets URL
const doc = new GoogleSpreadsheet(
"--SPREADSHEET ID--"
);
// Initialize Auth - see https://theoephraim.github.io/node-google-spreadsheet/#/getting-started/authentication
await doc.useServiceAccountAuth({
// env var values are copied from service account credentials generated by google
// see "Authentication" section in docs for more info
client_email: process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL,
private_key: process.env.GOOGLE_PRIVATE_KEY,
});
await doc.loadInfo(); // loads document properties and worksheets
const sheet = doc.sheetsByTitle[--WORKSHEET TITLE--];
const rows = await sheet.getRows(); // can pass in { limit, offset }
return rows;
}
app.get("/home", async (req, res) => {
try {
await docShit()
.then((response) => {
res.send(Promise.resolve(response)); //console log shows the object, but res.send just sends nothing??
})
.catch((err) => console.log(err));
} catch (err) {
console.error(err.message);
}
});

There is no res.send at all in your code. Also, you use await and .then together, but I consider them alternatives. Try the following:
app.get("/home", async (req, res, next) => {
try {
var response = await docShit();
console.log(response);
/* If response is circular, decide which parts of it you want to send.
The following is just an example. */
res.json(response.map(function(row) {
return {id: row.id, cells: row.cells.map(function(cell) {
return {id: cell.id, value: cell.value};
};
})};
} catch (err) {
console.error(err.message);
next(err);
}
});

Next.js API Routes response empty

I have try some method to fetch database using mysql. Here what i try:
export default function handler(req, res) {
const mysql = require('mysql')
const dbConn = mysql.createConnection({
host: '*******',
user: '*******',
password: '*******',
database: '*******'
})
dbConn.connect((err) => {
if (err) throw err
})
var data
dbConn.query(`SELECT * FROM training`, (err, result, fields) => {
data = JSON.stringify(result)
})
res.end(data)
}
I got nothing:
Response
Please I don't know how to fetch database directly with next.js

You are sending/ending the response before the database query has resolved. Instead move the end/send to inside the callback:
dbConn.query(`SELECT * FROM training`, (err, result, fields) => {
res.end(JSON.stringify(result));
// res.status(200).json(result)
})
Hopefully that helps!

Looks like an issue with Scope.
You will need to hand data into your Query function, or you can call res.end(data) inside your .Query function.
Handing in data might look like something like this: (without looking at the Docs)
dbConn.query(`SELECT * FROM training`, (err, result, fields, data) => {
data = JSON.stringify(result)
})
Calling res.end() inside your .Query function
dbConn.query(`SELECT * FROM training`, (err, result, fields) => {
res.end(JSON.stringify(result));
})

How to run request sequentially in a loop?

This is a pseudo code of what I am trying to achieve. First I need to get a list of URLs from the request body then pass those URLs to request function (using request module) which will get the data from each url and then save those data to MongoDB. After all the requests are finished including saving data to the server only then it should send a response.
app.post('/', (req, resp) => {
const { urls } = req.body;
urls.forEach((url, i) => {
request(url, function (err, resp, body) {
if (err) {
console.log('Error: ', err)
} else {
// function to save data to MongoDB server
saveUrlData(body);
console.log(`Data saved for URL number - ${i+1}`)
}
})
});
// Should be called after all data saved from for loop
resp.send('All data saved')
})
I have tried this code and of course the resp.send() function will run without caring if the request has completed. Using this code I get a result on the console like this:
Data saved for URL number - 3
Data saved for URL number - 1
Data saved for URL number - 5
Data saved for URL number - 2
Data saved for URL number - 4
I could write them in nested form but the variable urlscan have any number of urls and that's why it needs to be in the loop at least from my understanding. I want the requests to run sequentially i.e. it should resolve 1st url and then second and so on and when all urls are done only then it should respond. Please help!

app.post('/', async (req, resp) => {
const {
urls
} = req.body;
for (const url of urls) {
try {
const result = await doRequest(url)
console.log(result)
} catch (error) {
// do error processing here
console.log('Error: ', err)
}
}
})
function doRequest(url) {
return new Promise((resolve, reject) => {
request(url, function(err, resp, body) {
err ? reject(err) ? resolve(body)
})
})
}
using async await

You should look at JavaScript Promises
Otherwise, you can do a recursive request like so:
app.post('/', (req, resp) => {
const { urls } = req.body;
sendRequest(urls, 0);
})
function sendRequest(urlArr, i){
request(urlArr[i], function (err, resp, body) {
if (err) {
console.log('Error: ', err)
}
else {
saveUrlData(body);
console.log(`Data saved for URL number - ${i+1}`)
}
i++;
if(i == urlArr.length) resp.send('All data saved') //finish
else sendRequest(urlArr, i); //send another request
})
}
All I had to do is create a separate function I can call over and over again, passing the url array and a base index 0 as arguments. Each success callback increments the index variable which I pass in the same function again. Rinse and repeat until my index hits the length of the url array, I'll stop the recursive loop from there.

You want to wait till all api response you get and stored in db, so you should do async-await and promisify all the response.
You can use Request-Promise module instead of request. So you will get promise on every requested api call instead of callback.
And use promise.all for pushing up all request(module) call inside array.
Using async-await you code execution will wait till all api call get response and stored in db.
const rp = require('request-promise');
app.post('/', async (req, res) => {
try{
const { urls } = req.body;
// completed all will have all the api resonse.
const completedAll = await sendRequest(urls);
// now we have all api response that needs to be saved
// completedAll is array
const saved = await saveAllData(completedAll);
// Should be called after all data saved from for loop
res.status(200).send('All data saved')
}
catch(err) {
res.status(500).send({msg: Internal_server_error})
}
})
function sendRequest(urlArr, i){
const apiCalls = [];
for(let i=0;i < urlArr.length; i++){
apiCalls.push(rp(urlArr[i]));
}
// promise.all will give all api response in order as we pushed api call
return Promise.all(apiCalls);
}
You can refer these links:
https://www.npmjs.com/package/request-promise
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

Looking at the intention(a crawler) you can use Promise.all because the urls are not dependant upon each other.
app.post('/', (req, resp) => {
const { urls } = req.body;
const promises = urls.map((url, i) => {
return new Promise((resolve, rej)=>{
request(url, function (err, resp, body) {
if (err) {
rej(err);
} else {
resolve(body);
}
})
})
.then((body)=>{
//this should definitely be a promise as you are saving data to mongo
return saveUrlData(body);
})
});
// Should be called after all data saved from for loop
Promise.all(promises).then(()=>resp.send('All data saved'));
})
Note: Need to do error handling as well.

there are multiple ways to solve this.
you can use async/await
Promises
you can also use the async library
app.post('/', (req, res, next) => {
const { urls } = req.body;
async.each(urls, get_n_save, err => {
if (err) return next(err);
res.send('All data saved');
});
function get_n_save (url, callback) {
request(url, (err, resp, body) => {
if (err) {
return callback(err);
}
saveUrlData(body);
callback();
});
}
});

Socket.io-emit in callback gives SyntaxError: Unexpected end of JSON input

So I'm exporting a callback-function in a module like this:
(function() {
let request = require("request");
module.exports = function GithubApi(url, callback) {
let options = {
uri: url,
headers: {
"User-Agent": "Me",
"Content-Type": "application/x-www-form-urlencoded"
}
};
request(options, function(err, body) {
let context = {
issues: JSON.parse(body.body).map(function(issue) {
return {
title: issue.title,
comments: issue.comments,
};
})
};
callback(context) // The callback
});
};
}());
And this callback works perfectly fine when I'm using it in my GET-request with express.js:
app.get("/", (req, res) => {
let url = "some url";
GithubApi(url, (data) => {
res.render("../some-views", data);
});
});
But when I add a socket-emit, the callback-function returns SyntaxError: Unexpected end of JSON input
app.get("/", (req, res) => {
let url = "some url";
GithubApi(url, (data) => {
io.socket.emit("update", {message: data}); // adding this
res.render("../some-views", data);
});
});
Can't understand why the socket would interfere with the request and return an error with JSON. Can anybody help?

The probablem must be caused by the fact that body.body doesn't contain a valid JSON string.
When you run code like this:
JSON.parse(body.body)
you should always use try/catch because JSON.parse throws exceptions on bad JSON.
See those answers for more details:
Calling a JSON API with Node.js
Node JS ignores undefined check
How to extract inner string from the array string
Node server crashes in parsing JSON

So the problem was with the io.sockets.emit("update", {message: data});. For some reason, that interfered with the request(still don't know why tough). I guess it has something to do with the socket broadcasting to all channels, and that causes some kind of error, read something about it here.
So I changed the call to the callback-function to this:
GithubApi(orgs, repo, token, (data) => {
io.of("/").emit("update", {message: data}); // This line made it work
res.render("../views/home", data);
});

restful API using node not returning anything

In my server.js I do:-
app.post('/app/getSingleOrderDetail', function(req,res,next){
orderController.getSingleOrderDetail(req.body.order_id);
})
then in models
exports.getSingleOrderDetail = function(order_id, res, res) {
Orders.find({'_id':order_id}).exec(function(err,result){
console.log('result: '+result) //it's ok!!
res.json(result);
});
};
I'm expecting the result with this $http call in angularjs
$http({
url: '/app/getSingleOrderDetail',
method: "POST",
data: {'order_id' : id}
}).then(function(response){
console.log(response.data)
vm.sales_detail = response.data;
}).catch(function(response) {
alert('Error!');
console.log(response);
});
Everything is passed correctly but I just couldn't get the data back to client side in angularjs.

In getSingleOrderDetail you're expecting the arguments (order_id, res, res), When you're invoking the function though, you're only passing in a value to the first argument, order_id and nothing for res. Also, you've defined res twice which is going to cause an issue in your code when trying to access res.
You should fix up those issues like so
Route
app.post('/app/getSingleOrderDetail', orderController.getSingleOrderDetail);
Model
exports.getSingleOrderDetail = function(req, res) {
let order_id = req.body.order_id;
Orders.find({'_id': order_id}).exec(function(err,result) {
if (err) return res.status(500).send(err);
console.log('result: ' + result); //it's ok!!
return res.status(200).json(result);
});
};
Just a side note, from the looks of your route name, this wouldn't be considered RESTful, it would be more of an RPC (Remote Procedure Call) style API.