We Keep Coding

Express Node JS POST. How do I add values to my req.body without using url parameters

I'm building a rest api using Express NodeJS with mysql. I'm having no issues at all using GET and using req.params.value and passing values using URL parameters.
Now I'm trying to use POST to insert some data into my database. Using POSTMAN I have no problems doing this because obviously you can set the BODY variables to be used. But it dawned on me in my applications I won't be able to use POSTMAN to do this. My quesiton (which may be silly) is how do I pass these BODY variables to my api? Do I still pass them through the url as parameters? If so, would I use req.body.value or req.params.value? Here is my POST code:
// Add new record
router.post('/editablerecords/add', function (req, res) {
let qb_TxnID = req.body.txnid
let type = req.body.type;
let margin = req.body.margin;
if (!qb_TxnID || !type || !margin ) {
return res.status(400).send({ error:true, message: 'Please provide qb_TxnID, type, or margin' });
}
// res.send(qb_TxnID + ' ' + type + ' ' + margin);
connection.query("INSERT INTO pxeQuoteToClose SET ? ", { 'qb_TxnID': qb_TxnID, 'type': type, 'margin': margin }, function (error, results, fields) {
if(error){
res.send(JSON.stringify({"status": 500, "error": error, "response": null}));
//If there is error, we send the error in the error section with 500 status
} else {
res.send(JSON.stringify({ error: false, data: results, message: 'New record has been created successfully.' }));
//If there is no error, all is good and response is 200OK.
}
});
});

Well it depends on how you will make your request, as an XMLHttpRequest call or via a form, but it will definitely not be using params in that case. Here are three options to send data to your API depending on your requirement, in your case I would advice the last one.
1 - You can use a form and make the action point to to your endpoint. In that case you'll have to add a bodyParser middleware for x-www-form-urlencoded data to your express application. If your using the body-parser library it is as simple as app.use(bodyParser.urlencoded([options])). Data will be available in req.body.
2 - You can send all data in your url but in the query string, not the params. For example: https://yourapi.com/editablerecords/add?qb_TxnID=data_id&type=data_type&margin=data_margin. All data will then be available in the req.query object. No need to add any parser.
3 - Last but not least, I would advise to send your data as a json body using an XMLHttpRequest. To help you out you may use a library like axios or fecth but the principle stay the same: you set your body with your data object and you retrieve it on req.bodyon your api. Seeing your code I do assume that your are using a body parser but if that would not be the case, you should add a middleware using app.use(bodyParser.json()).
I hope I have answer your question and that it will help you out.

Hide an API key (in an environment variable perhaps?) when using Angular

I'm running a small Angular application with a Node/Express backend.
In one of my Angular factories (i.e. on the client side) I make a $http request to Github to return user info. However, a Github-generated key (which is meant to be kept secret) is required to do this.
I know I can't use process.env.XYZ on the client side. I'm wondering how I could keep this api key a secret? Do I have to make the request on the back end instead? If so, how do I transfer the returned Github data to the front end?
Sorry if this seems simplistic but I am a relative novice, so any clear responses with code examples would be much appreciated. Thank you

Unfortunately you have to proxy the request on your backend to keep the key secret. (I am assuming that you need some user data that is unavailable via an unauthenticated request like https://api.github.com/users/rsp?callback=foo because otherwise you wouldn't need to use API keys in the first place - but you didn't say specifically what you need to do so it is just my guess).
What you can do is something like this: In your backend you can add a new route for your frontend just for getting the info. It can do whatever you need - using or not any secret API keys, verify the request, process the response before returning to your client etc.
Example:
var app = require('express')();
app.get('/github-user/:user', function (req, res) {
getUser(req.params.user, function (err, data) {
if (err) res.json({error: "Some error"});
else res.json(data);
});
});
function getUser(user, callback) {
// a stub function that should do something more
if (!user) callback("Error");
else callback(null, {user:user, name:"The user "+user});
}
app.listen(3000, function () {
console.log('Listening on port 3000');
});
In this example you can get the user info at:
http://localhost:3000/github-user/abc
The function getUser should make an actual request to GitHub and before you call it you can change if that is really your frontend that is making the request e.g. by cheching the "Referer" header or other things, validate the input etc.
Now, if you only need a public info then you may be able to use a public JSON-P API like this - an example using jQuery to make things simple:
var user = prompt("User name:");
var req = $.getJSON('https://api.github.com/users/'+user);
req.then(function (data) {
console.log(data);
});
See DEMO

RESTful Get Request with Angular & NodeJS Express

I'm just following tutorials and figuring out how to handle get requests in NodeJS.
Here are snippets of my code:
NodeJS:
router.get('/test', function(request, response, next) {
console.log("Received Get Request");
response.jsonp({
data: 'test'
});
});
Angular:
$http.get("http://localhost:3000/test").
success(function(response) {
alert("OK");
}).
error(function(response) {
alert("FAIL");
});
If I try to access the link directly # localhost:3000/test, I'm able to receive the JSON message correctly. But when I use angularJS $http call, the request always fails and I'll find this error in the network inspector (Response)
SyntaxError:JSON.parse:unexpected end of data at line 1 column 1 of
the JSON data
The reason for that is because the response is empty but the response code is 200 in both cases.
I've tried searching for hours but maybe someone can enlighten me on this?

you could try and send
res.send('test')
and then on your http request you can use 'then'
$http.get("http://localhost:3000/test").then(function(res) {
console.log(res);
})
unlike success, then will give you a complete object (with 'test' - string as res.data)
success will bring you only the data;
then will bring you the whole object (with the status and such)..
now about that jsonp .. it's used to override a json response. you could simply use 'res.json({data: 'test'})' and it should also work for you..
hope it helps

You're using jsonp in node, which you probably don't need to. This adds extra characters to the response and so the JSON parser fails to parse (that's what the error is telling you, the JSON is malformed)
Try changing the server to look like
response.json({
data: 'test'
});
If you look in the Network pane of the developer tools, you should be able to see the raw response. It should look something like:
{"data" : "test"}

Node JS app only display the first JSON object. Why?

I am trying to write a json object in my node application, integrating the Twilio API. When console logging the object all objects are returned properly but when I write it to the document only the first object is written. Why? How should I change the code to see the same written response as in my console log.
var express = require('express');
var app = express();
app.use(express.bodyParser());
app.get('/', function(req, res) {
var accountSid = 'xxx';
var authToken = 'xxx';
var client = require('twilio')(accountSid, authToken);
client.messages.list({
from: "xxx",
to: "xxx"
}, function(err, data) {
data.messages.forEach(function(message) {
console.log(message.body); // THIS WILL DISPLAY ALL OBJECTS
res.json(message.body); // THIS WILL ONLY DISPLAY THE FIRST OBJECT
});
});
});
app.listen(1337);
I am new to Node JS and think this is easy to solve, but I still can’t find the solution.

res.json(...); sends back the response. You are doing that in the first iteration over the array, hence the client only gets the first message.
If you want to extract body from all messages and send all of them back, then do that. Create an array with the data you want and send it back. Example:
res.json(data.messages.map(function(message) {
return message.body;
}));

You can only call res.json once per request. You're calling it multiple times in a loop. The first time you call it, the browser receives the response, and you'll get a headers already sent exceptions (or something like that) for all other res.json calls.

res.json actually does a data conversion to JSON. I'd be willing to bet there is something it is not dealing with, or it's simply screwing it up. If the response from Twilio is already json, you probably don't need to do that. Try res.send, instead, which just returns whatever you got back.

writestream and express for json object?

I might be out of depth but I really need something to work. I think a write/read stream will solve both my issues but I dont quite understand the syntax or whats required for it to work.
I read the stream handbook and thought i understood some of the basics but when I try to apply it to my situation, it seems to break down.
Currently I have this as the crux of my information.
function readDataTop (x) {
console.log("Read "+x[6]+" and Sent Cached Top Half");
jf.readFile( "loadedreports/top"+x[6], 'utf8', function (err, data) {
resT = data
});
};
Im using Jsonfile plugin for node which basically shortens the fs.write and makes it easier to write instead of constantly writing catch and try blocks for the fs.write and read.
Anyways, I want to implement a stream here but I am unsure of what would happen to my express end and how the object will be received.
I assume since its a stream express wont do anything to the object until it receives it? Or would I have to write a callback to also make sure when my function is called, the stream is complete before express sends the object off to fullfill the ajax request?
app.get('/:report/top', function(req, res) {
readDataTop(global[req.params.report]);
res.header("Content-Type", "application/json; charset=utf-8");
res.header("Cache-Control", "max-age=3600");
res.json(resT);
resT = 0;
});
I am hoping if I change the read part to a stream it will allievate two problems. The issue of sometimes receiving impartial json files when the browser makes the ajax call due to the read speed of larger json objects. (This might be the callback issue i need to solve but a stream should make it more consistent).
Then secondly when I load this node app, it needs to run 30+ write files while it gets the data from my DB. The goal was to disconnect the browser from the db side so node acts as the db by reading and writing. This due to an old SQL server that is being bombarded by a lot of requests already (stale data isnt an issue).
Any help on the syntax here?
Is there a tutorial I can see in code of someone piping an response into a write stream? (the mssql node I use puts the SQL response into an object and I need in JSON format).
function getDataTop (x) {
var connection = new sql.Connection(config, function(err) {
var request = new sql.Request(connection);
request.query(x[0], function(err, topres) {
jf.writeFile( "loadedreports/top"+x[6], topres, function(err) {
if(err) {
console.log(err);
} else {
console.log(x[6]+" top half was saved!");
}
});
});
});
};

Your problem is that you're not waiting for the file to load before sending the response. Use a callback:
function readDataTop(x, cb) {
console.log('Read ' + x[6] + ' and Sent Cached Top Half');
jf.readFile('loadedreports/top' + x[6], 'utf8', cb);
};
// ...
app.get('/:report/top', function(req, res) {
// you should really avoid using globals like this ...
readDataTop(global[req.params.report], function(err, obj) {
// setting the content-type is automatically done by `res.json()`
// cache the data here in-memory if you need to and check for its existence
// before `readDataTop`
res.header('Cache-Control', 'max-age=3600');
res.json(obj);
});
});