I am trying to write a framework for Node without using any Express code. I want to be able to get the body from a HTTP request using the http module, but nothing I try seems to work. This is my code currently:
import http from "http";
import url from "url";
import Route from "./Route";
import HTTPRequest from "./HTTPRequest";
class HextecCreator {
static createApp = (routes: Array<Route>) => {
return {
getRoutes: () => {
return routes;
},
run: (port: number) => {
http
.createServer(function (req, res) {
let data: any = [];
req
.on("data", (chunk) => data.push(chunk))
.on("end", () => {
data = Buffer.concat(data).toString();
});
var correctRoute;
for (var route of routes) {
if (route.getUrl() === req.url) {
correctRoute = route;
break;
} else {
correctRoute = "Route not found";
}
}
if (typeof correctRoute != "object") {
res.write("Route not found");
res.end();
} else {
res.write(
correctRoute
.getHandlerFunc()(
new HTTPRequest(
req.method,
req.url,
url.parse(req.url as string, true).query,
data
)
)
.getResponse()
);
res.end();
}
})
.listen(port);
},
};
};
}
export default HextecCreator;
This should work, but when I use postman to actually get the body of the request, it is empty. How can I fix this?
The end event gets triggered after your entire function is complete. To to do something with the entire HTTP request body, you will need to trigger this functionality from within the end handler.
This means that the rest of your logic needs to live inside the end event handler, or you need to call a function that "does the rest", but still from within that end event.
I also wrote a framework from scratch, and this is my async/await based version:
https://github.com/curveball/core/blob/master/src/node/request.ts#L36
Note that this still delegates turning an entire stream to a buffer to an external package.
I am receiving my data from my POST request. I want to use that data as a variable in a GET request.The data from the POST request is user inputted to make different queries on the 3rd party API> How can I save my data as a variable to do so? You can see my request.body is being saved as const data. And I need it used as ${data} in my get request. I am pretty new to this, so any suggestions on best practice would be appreciated. Thanks for helping out.
Server.js
app.get(`/getmovies`, (req, res) => {
request(`http://www.omdbapi.com/?t=${data}&apikey=${API_KEY}`,
function (error, response, body) {
if (!error && response.statusCode == 200) {
var parsedBody = JSON.parse(body);
res.send(parsedBody)
} else {
console.log("error in the server")
}
}
)
})
app.post('/postmovie', (request, response) => {
console.log("I got a request!")
console.log(request.body);
const data = request.body;
response.json({
status: 'success',
name: data
})
})
///Client
postMovie = async (e) => {
e.preventDefault();
const data = this.state.movie;
const options = {
method: 'Post',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(data)
};
const response = await fetch('/postmovie', options);
const json = await response.json();
console.log(json);
};
getMovies = (e) => {
e.preventDefault();
const { movie } = this.state;
axios.get(`/getmovies`)
.then(response => this.setState({ movies: response.data }))
// .then(response => this.setState({ movies: response.data }))
.catch(err => console.error(err))
}
render() {
const { movie } = this.state;
return (
<div>
<section className="form-container">
<div className="movie-form-div">
< form>
<input className="form-input" type="text" name="name"
value={movie.name}
onChange={e => this.setState({ movie: { ...movie, name: e.target.value } })} />
<button onClick={this.getMovies}>Search</button>
<button onClick={this.postMovie}>Post</button>
</form >
</div>
</section>
As I have mentioned in the comments, your question is worded such that you make it sound like you absolutely need two requests.
Per the comments, OP has clarified that they don't necessarily need two requests so on that note here are a few things that you can try.
1) Set up an application.conf
Configuration files allow us to parameterize and configure variables that our applications need to function. You can vary the config files between environments (dev/stage/prod etc.) and deployments.
This will assume your API_KEY doesn't change frequently. Think of it as hard coding your api key like
var apiKey = "...."
but a better and more secure way of doing so.
In your case this won't work but it is good to be aware of it. You can use this package in the future
2) Get request query parameters.
This is your best bet
Query parameters, as the name suggest, allow us to pass in variables with the get requests to filter the results in the server side. This can be something like a user-id, page number etc.
Your get url from the client side would look like something like this
mydomain.com/getmovies?apikey={YourKeyGoesHere}
I am sure you have seen that ? in many of the urls you visit.
If you are using express your end point will look something like this.
app.get(`/getmovies`, (req, res) => {
var apiKey = req.query.apikey
request(`http://www.omdbapi.com/?t=${data}&apikey=${apikey}`,
function (error, response, body) {
if (!error && response.statusCode == 200) {
var parsedBody = JSON.parse(body);
res.send(parsedBody)
} else {
console.log("error in the server")
}
}
)
})
See How to get GET (query string) variables in Express.js on Node.js?
This will allow you to pass it from the client side.
If you are not using express please see https://nodejs.org/api/querystring.html
3) If you absolutely need two endpoints...
Use a cookie. Once a cookie is set, it is transported back and forth with every request, until it expires.
See this How to set cookie in node js using express framework? or this
Once you receive the post request you would set a cookie
app.post('/postmovie', (request, response) => {
response.cookie("apikey", data, maxAge: 900000, httpOnly: true });
response.json({
status: 'success'
})
})
Once this cookie is set, it will be transported to the client and back with all the requests including the get.
So in your get request you can do:
app.get(`/getmovies`, (req, res) => {
var apikey = req.cookies.apikey;
request(`http://www.omdbapi.com/?t=${data}&apikey=${apikey}`,
function (error, response, body) {
if (!error && response.statusCode == 200) {
var parsedBody = JSON.parse(body);
res.send(parsedBody)
} else {
console.log("error in the server")
}
}
)
})
So what's the difference between #2 and #3?
The query parameters that are encoded in the url are public and can be intercepted. This is fine for things like page numbers but if the API key is a paid one, this isn't very safe. Also once you submit the query, it is gone (unless you save the api key in the client side to the memory or local storage) so the user will have to potentially re-enter the api key everything.
Cookies on the other hand are private. The http-only flag we set in the code, prevents even the browser from reading it. Only the issuing server can see the contents. Plus cookies get stored in the browser until the expiration date so they can be reused with multiple requests.
Pick the one that works for you.
P.S. I didn't test the code above, you may need to tweak it a little bit. The answer is to give you a general idea on how these things are usually handled
How can I make an HTTP request from within Node.js or Express.js? I need to connect to another service. I am hoping the call is asynchronous and that the callback contains the remote server's response.
Here is a snippet of some code from a sample of mine. It's asynchronous and returns a JSON object. It can do any form of GET request.
Note that there are more optimal ways (just a sample) - for example, instead of concatenating the chunks you put into an array and join it etc... Hopefully, it gets you started in the right direction:
const http = require('http');
const https = require('https');
/**
* getJSON: RESTful GET request returning JSON object(s)
* #param options: http options object
* #param callback: callback to pass the results JSON object(s) back
*/
module.exports.getJSON = (options, onResult) => {
console.log('rest::getJSON');
const port = options.port == 443 ? https : http;
let output = '';
const req = port.request(options, (res) => {
console.log(`${options.host} : ${res.statusCode}`);
res.setEncoding('utf8');
res.on('data', (chunk) => {
output += chunk;
});
res.on('end', () => {
let obj = JSON.parse(output);
onResult(res.statusCode, obj);
});
});
req.on('error', (err) => {
// res.send('error: ' + err.message);
});
req.end();
};
It's called by creating an options object like:
const options = {
host: 'somesite.com',
port: 443,
path: '/some/path',
method: 'GET',
headers: {
'Content-Type': 'application/json'
}
};
And providing a callback function.
For example, in a service, I require the REST module above and then do this:
rest.getJSON(options, (statusCode, result) => {
// I could work with the resulting HTML/JSON here. I could also just return it
console.log(`onResult: (${statusCode})\n\n${JSON.stringify(result)}`);
res.statusCode = statusCode;
res.send(result);
});
UPDATE
If you're looking for async/await (linear, no callback), promises, compile time support and intellisense, we created a lightweight HTTP and REST client that fits that bill:
Microsoft typed-rest-client
Try using the simple http.get(options, callback) function in node.js:
var http = require('http');
var options = {
host: 'www.google.com',
path: '/index.html'
};
var req = http.get(options, function(res) {
console.log('STATUS: ' + res.statusCode);
console.log('HEADERS: ' + JSON.stringify(res.headers));
// Buffer the body entirely for processing as a whole.
var bodyChunks = [];
res.on('data', function(chunk) {
// You can process streamed parts here...
bodyChunks.push(chunk);
}).on('end', function() {
var body = Buffer.concat(bodyChunks);
console.log('BODY: ' + body);
// ...and/or process the entire body here.
})
});
req.on('error', function(e) {
console.log('ERROR: ' + e.message);
});
There is also a general http.request(options, callback) function which allows you to specify the request method and other request details.
Request and Superagent are pretty good libraries to use.
note: request is deprecated, use at your risk!
Using request:
var request=require('request');
request.get('https://someplace',options,function(err,res,body){
if(err) //TODO: handle err
if(res.statusCode === 200 ) //etc
//TODO Do something with response
});
You can also use Requestify, a really cool and very simple HTTP client I wrote for nodeJS + it supports caching.
Just do the following for GET method request:
var requestify = require('requestify');
requestify.get('http://example.com/api/resource')
.then(function(response) {
// Get the response body (JSON parsed or jQuery object for XMLs)
response.getBody();
}
);
This version is based on the initially proposed by bryanmac function which uses promises, better error handling, and is rewritten in ES6.
let http = require("http"),
https = require("https");
/**
* getJSON: REST get request returning JSON object(s)
* #param options: http options object
*/
exports.getJSON = function (options) {
console.log('rest::getJSON');
let reqHandler = +options.port === 443 ? https : http;
return new Promise((resolve, reject) => {
let req = reqHandler.request(options, (res) => {
let output = '';
console.log('rest::', options.host + ':' + res.statusCode);
res.setEncoding('utf8');
res.on('data', function (chunk) {
output += chunk;
});
res.on('end', () => {
try {
let obj = JSON.parse(output);
// console.log('rest::', obj);
resolve({
statusCode: res.statusCode,
data: obj
});
}
catch (err) {
console.error('rest::end', err);
reject(err);
}
});
});
req.on('error', (err) => {
console.error('rest::request', err);
reject(err);
});
req.end();
});
};
As a result you don't have to pass in a callback function, instead getJSON() returns a promise. In the following example the function is used inside of an ExpressJS route handler
router.get('/:id', (req, res, next) => {
rest.getJSON({
host: host,
path: `/posts/${req.params.id}`,
method: 'GET'
}).then(({ statusCode, data }) => {
res.json(data);
}, (error) => {
next(error);
});
});
On error it delegates the error to the server error handling middleware.
Unirest is the best library I've come across for making HTTP requests from Node. It's aiming at being a multiplatform framework, so learning how it works on Node will serve you well if you need to use an HTTP client on Ruby, PHP, Java, Python, Objective C, .Net or Windows 8 as well. As far as I can tell the unirest libraries are mostly backed by existing HTTP clients (e.g. on Java, the Apache HTTP client, on Node, Mikeal's Request libary) - Unirest just puts a nicer API on top.
Here are a couple of code examples for Node.js:
var unirest = require('unirest')
// GET a resource
unirest.get('http://httpbin.org/get')
.query({'foo': 'bar'})
.query({'stack': 'overflow'})
.end(function(res) {
if (res.error) {
console.log('GET error', res.error)
} else {
console.log('GET response', res.body)
}
})
// POST a form with an attached file
unirest.post('http://httpbin.org/post')
.field('foo', 'bar')
.field('stack', 'overflow')
.attach('myfile', 'examples.js')
.end(function(res) {
if (res.error) {
console.log('POST error', res.error)
} else {
console.log('POST response', res.body)
}
})
You can jump straight to the Node docs here
Check out shred. It's a node HTTP client created and maintained by spire.io that handles redirects, sessions, and JSON responses. It's great for interacting with rest APIs. See this blog post for more details.
Check out httpreq: it's a node library I created because I was frustrated there was no simple http GET or POST module out there ;-)
For anyone who looking for a library to send HTTP requests in NodeJS, axios is also a good choice. It supports Promises :)
Install (npm): npm install axios
Example GET request:
const axios = require('axios');
axios.get('https://google.com')
.then(function (response) {
// handle success
console.log(response);
})
.catch(function (error) {
// handle error
console.log(error);
})
Github page
Update 10/02/2022
Node.js integrates fetch in v17.5.0 in experimental mode. Now, you can use fetch to send requests just like you do on the client-side. For now, it is an experimental feature so be careful.
If you just need to make simple get requests and don't need support for any other HTTP methods take a look at: simple-get:
var get = require('simple-get');
get('http://example.com', function (err, res) {
if (err) throw err;
console.log(res.statusCode); // 200
res.pipe(process.stdout); // `res` is a stream
});
Use reqclient: not designed for scripting purpose
like request or many other libraries. Reqclient allows in the constructor
specify many configurations useful when you need to reuse the same
configuration again and again: base URL, headers, auth options,
logging options, caching, etc. Also has useful features like
query and URL parsing, automatic query encoding and JSON parsing, etc.
The best way to use the library is create a module to export the object
pointing to the API and the necessary configurations to connect with:
Module client.js:
let RequestClient = require("reqclient").RequestClient
let client = new RequestClient({
baseUrl: "https://myapp.com/api/v1",
cache: true,
auth: {user: "admin", pass: "secret"}
})
module.exports = client
And in the controllers where you need to consume the API use like this:
let client = require('client')
//let router = ...
router.get('/dashboard', (req, res) => {
// Simple GET with Promise handling to https://myapp.com/api/v1/reports/clients
client.get("reports/clients")
.then(response => {
console.log("Report for client", response.userId) // REST responses are parsed as JSON objects
res.render('clients/dashboard', {title: 'Customer Report', report: response})
})
.catch(err => {
console.error("Ups!", err)
res.status(400).render('error', {error: err})
})
})
router.get('/orders', (req, res, next) => {
// GET with query (https://myapp.com/api/v1/orders?state=open&limit=10)
client.get({"uri": "orders", "query": {"state": "open", "limit": 10}})
.then(orders => {
res.render('clients/orders', {title: 'Customer Orders', orders: orders})
})
.catch(err => someErrorHandler(req, res, next))
})
router.delete('/orders', (req, res, next) => {
// DELETE with params (https://myapp.com/api/v1/orders/1234/A987)
client.delete({
"uri": "orders/{client}/{id}",
"params": {"client": "A987", "id": 1234}
})
.then(resp => res.status(204))
.catch(err => someErrorHandler(req, res, next))
})
reqclient supports many features, but it has some that are not supported by other
libraries: OAuth2 integration and logger integration
with cURL syntax, and always returns native Promise objects.
If you ever need to send GET request to an IP as well as a Domain (Other answers did not mention you can specify a port variable), you can make use of this function:
function getCode(host, port, path, queryString) {
console.log("(" + host + ":" + port + path + ")" + "Running httpHelper.getCode()")
// Construct url and query string
const requestUrl = url.parse(url.format({
protocol: 'http',
hostname: host,
pathname: path,
port: port,
query: queryString
}));
console.log("(" + host + path + ")" + "Sending GET request")
// Send request
console.log(url.format(requestUrl))
http.get(url.format(requestUrl), (resp) => {
let data = '';
// A chunk of data has been received.
resp.on('data', (chunk) => {
console.log("GET chunk: " + chunk);
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on('end', () => {
console.log("GET end of response: " + data);
});
}).on("error", (err) => {
console.log("GET Error: " + err);
});
}
Don't miss requiring modules at the top of your file:
http = require("http");
url = require('url')
Also bare in mind that you may use https module for communicating over secured network. so these two lines would change:
https = require("https");
...
https.get(url.format(requestUrl), (resp) => { ......
## you can use request module and promise in express to make any request ##
const promise = require('promise');
const requestModule = require('request');
const curlRequest =(requestOption) =>{
return new Promise((resolve, reject)=> {
requestModule(requestOption, (error, response, body) => {
try {
if (error) {
throw error;
}
if (body) {
try {
body = (body) ? JSON.parse(body) : body;
resolve(body);
}catch(error){
resolve(body);
}
} else {
throw new Error('something wrong');
}
} catch (error) {
reject(error);
}
})
})
};
const option = {
url : uri,
method : "GET",
headers : {
}
};
curlRequest(option).then((data)=>{
}).catch((err)=>{
})
PHP dev here, I'm wrestling with NodeJS for a while and I still can't wrap my head around the idea of asynchrony is JS/Node.
Look at this (ExpressJS):
router.get('/:id', function (req, res, next) {
var id = req.params.id;
var db = new Firebase('some_firebase_db');
db.child("users/"+id).once('value', function (snapshot) {
var user = snapshot.val();
if (user) {
db.child("messages/"+id).once('value', function (snapshot) {
res.render('user/messages', {
'user': user,
'messages': snapshot.val()
});
});
} else {
res.render('404');
}
});
});
To make database value accessible for a res object I need to render views inside the fetching data callbacks.
So when I need, say, make 6 requests to my DB I will have to embed my res object in 6 callbacks?
Or is there different approach that will make the code more readable keeping it asynchronous?
Ultimately, I need a way to fetch data from db multiple times in one request that will not make my code look like Christmas tree.
You can make it more readable even without using async or Promise:
router.get('/:id', function(req, res, next) {
var id = req.params.id;
var db = new Firebase('some_firebase_db');
db.child("users/" + id).once('value', userResult);
function userResult(snapshot) {
var user = snapshot.val();
if (user) {
db.child("messages/" + id).once('value', messageResult);
} else {
res.render('404');
}
}
function messageResult(snapshot) {
res.render('user/messages', {
'user': user,
'messages': snapshot.val()
});
}
});
But using async or Promise would be a better solution.