I am using the following code in my application to check if some headers are provided .The code works fine in localhost but not when the application is deployed to server . Basically I am trying to check if headers are present in the request. On the server , I keep getting invalid request . When I pass accesstoken instead of access_token , the request goes through successfully . So by changing if ((request.headers.access_token && request.headers.refresh_token && request.headers.id_token) || request.headers.token)
to
The code works , my question is why is this happening
const Hapi = require('hapi');
const Path = require('path');
const axios = require('axios');
var tokenValidation = function (request, reply) {
if ((request.headers.access_token && request.headers.refresh_token && request.headers.id_token) || request.headers.token) {
if (request.headers.access_token != undefined) {
//do something
}
else {
return reply.continue();
}
} else
return reply.continue();
}
else {
var err = Boom.badRequest(‘Invalid request.');
reply(err);
}
}
server.ext('onRequest', tokenValidation);
Missing (disappearing) HTTP Headers
If you do not explicitly set underscores_in_headers on;, NGINX will silently drop HTTP headers with underscores (which are perfectly valid according to the HTTP standard). This is done in order to prevent ambiguities when mapping headers to CGI variables as both dashes and underscores are mapped to underscores during that process.
https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/#missing--28disappearing-29-http-headers
We have to explicitly underscores_in_headers on in NGINX , else they will be ignored
Related
I've never received an error like this before,
I have a file that defines functions for making API calls, currently I'm reading the endpoint base URLs from the environment variables:
/**
* Prepended to request URL indicating base URL for API and the API version
*/
const VERSION_URL = `${process.env.NEXT_PUBLIC_API_BASE_URL}/${process.env.NEXT_PUBLIC_API_VERSION}`
I tried to make a quick workaround because environment variables weren't being loaded correctly, by hardcoding the URLS incase the variable wasn't defined.
/**
* Prepended to request URL indicating base URL for API and the API version
*/
const VERSION_URL = `${process.env.NEXT_PUBLIC_API_BASE_URL || 'https://hardcodedURL.com'}/${process.env.NEXT_PUBLIC_API_VERSION || 'v1'}`
In development and production mode when running on my local machine it works fine (docker container). However, as soon as it's pushed to production, I then get the following screen:
This is the console output:
framework-bb5c596eafb42b22.js:1 TypeError: Path must be a string. Received undefined
at t (137-10e3db828dbede8a.js:46:750)
at join (137-10e3db828dbede8a.js:46:2042)
at J (898-576b101442c0ef86.js:1:8158)
at G (898-576b101442c0ef86.js:1:10741)
at oo (framework-bb5c596eafb42b22.js:1:59416)
at Wo (framework-bb5c596eafb42b22.js:1:68983)
at Ku (framework-bb5c596eafb42b22.js:1:112707)
at Li (framework-bb5c596eafb42b22.js:1:98957)
at Ni (framework-bb5c596eafb42b22.js:1:98885)
at Pi (framework-bb5c596eafb42b22.js:1:98748)
cu # framework-bb5c596eafb42b22.js:1
main-f51d4d0442564de3.js:1 TypeError: Path must be a string. Received undefined
at t (137-10e3db828dbede8a.js:46:750)
at join (137-10e3db828dbede8a.js:46:2042)
at J (898-576b101442c0ef86.js:1:8158)
at G (898-576b101442c0ef86.js:1:10741)
at oo (framework-bb5c596eafb42b22.js:1:59416)
at Wo (framework-bb5c596eafb42b22.js:1:68983)
at Ku (framework-bb5c596eafb42b22.js:1:112707)
at Li (framework-bb5c596eafb42b22.js:1:98957)
at Ni (framework-bb5c596eafb42b22.js:1:98885)
at Pi (framework-bb5c596eafb42b22.js:1:98748)
re # main-f51d4d0442564de3.js:1
main-f51d4d0442564de3.js:1 A client-side exception has occurred, see here for more info: https://nextjs.org/docs/messages/client-side-exception-occurred
re # main-f51d4d0442564de3.js:1
Viewing the source at t (137-10e3db828dbede8a.js:46:750)
I'm completely at a lost at what this means or what is happening. Why would hardcoding in a string for the path result in this client error? The lack of a readable source code is making this impossible for me to understand what's happening.
Quick googling suggests that I should upgrade some package, but the error is so vague, I'm not even sure what package is giving the issue.
This is the roughly the how the version URL path is being used
/**
* Send a get request to a given endpoint
*
* **Returns a Promise**
*/
function GET(token, data, parent, api) {
return new Promise((resolve, reject) => {
try {
let req = new XMLHttpRequest()
let endpoint = `${VERSION_URL}/${parent}/${api}` // base url with the params not included
let params = new URLSearchParams() // URLSearchParams used for adding params to url
// put data in GET request params
for (const [key, value] of Object.entries(data)) {
params.set(key, value)
}
let query_url = endpoint + "?" + params.toString() // final query url
req.open("GET", query_url, true)
req.setRequestHeader("token", token) // put token into header
req.onloadend = () => {
if (req.status === 200) {
// success, return response
resolve([req.response, req.status])
} else {
reject([req.responseText, req.status])
}
}
req.onerror = () => {
reject([req.responseText, req.status])
}
req.send()
} catch (err) {
reject(["Exception", 0])
}
})
}
From my experience, this problem can happen for multiple reasons. The most common one is because you didn't put the data accessing checker properly when data comes from an API. Sometimes this things we don't see in browser but it gives such error when you deploy.
For example:
const response = fetch("some_url");
const companyId = response.data.companyId; ❌
const companyId = response?.data?.companyId; ✔️
My former server.js is like:
After running the server I could see my index.html
var connect = require('connect');
var serveStatic = require('serve-static');
connect().use(serveStatic(__dirname)).listen(5000, '192.168.xx.xx', function(){
console.log('Server running on 5000');
});
I want to create http login and password to secure the website, so I found online the information of http module: if I put right login and password, I could see congratulations message:
var http = require('http');
var server = http.createServer(function(req, res) {
// console.log(req); // debug dump the request
// If they pass in a basic auth credential it'll be in a header called "Authorization" (note NodeJS lowercases the names of headers in its request object)
var auth = req.headers['authorization']; // auth is in base64(username:password) so we need to decode the base64
console.log("Authorization Header is: ", auth);
if(!auth) { // No Authorization header was passed in so it's the first time the browser hit us
// Sending a 401 will require authentication, we need to send the 'WWW-Authenticate' to tell them the sort of authentication to use
// Basic auth is quite literally the easiest and least secure, it simply gives back base64( username + ":" + password ) from the browser
res.statusCode = 401;
res.setHeader('WWW-Authenticate', 'Basic realm="Secure Area"');
res.end('<html><body>Need authorization</body></html>');
}
else if(auth) { // The Authorization was passed in so now we validate it
var tmp = auth.split(' '); // Split on a space, the original auth looks like "Basic Y2hhcmxlczoxMjM0NQ==" and we need the 2nd part
var buf = new Buffer(tmp[1], 'base64'); // create a buffer and tell it the data coming in is base64
var plain_auth = buf.toString(); // read it back out as a string
console.log("Decoded Authorization ", plain_auth);
// At this point plain_auth = "username:password"
var creds = plain_auth.split(':'); // split on a ':'
var username = creds[0];
var password = creds[1];
if((username == 'admin') && (password == 'admin')) { // Is the username/password correct?
res.statusCode = 200; // OK
res.end('<html><body>Congratulations, feel free to explre!</body></html>');
}
else {
res.statusCode = 401; // Force them to retry authentication
res.setHeader('WWW-Authenticate', 'Basic realm="Secure Area"');
// res.statusCode = 403; // or alternatively just reject them altogether with a 403 Forbidden
res.end('<html><body>You shall not pass</body></html>');
}
}
});
server.listen(5000, function() { console.log("Server Listening on http://localhost:5000/"); });
I am new to nodejs, I want to know how to combine this 2 js? In order to realize my function of adding authorization to my web.
Could I do something to show my index instead of showing congratulation message after putting the login and password?
Thanks a lot.
In order to show HTML page instead of congratulation message, you can follow these steps:
Get request path by req.url, such as / or /introduction.html.
According to the above path, read the corresponding HTML file in server disk, using fs.readFile().
Return HTML file content to browser if the read is successful. Otherwise, return 404 error page.
Here is some example code for above steps:
if((username == 'admin') && (password == 'admin')) { // Is the username/password correct?
res.statusCode = 200; // OK
// res.end('<html><body>Congratulations, feel free to explre!</body></html>');
var requestURL = req.url; // e.g. / or /a or /a.html
var requestFilePath = getFilePathFromRequestURL(requestURL); // you need to implement this logic yourself, such as "/" mapping to "./index.html"
fs.readFile(requestFilePath, function(error, data) {
if (error) {
res.statusCode = 404;
res.write('File not found.');
} else {
res.statusCode = 200;
res.write(data);
}
res.end();
});
}
However, unless you want to write some low-level node.js code to better understand this language, I highly recommend using node.js web framework such as Express. Serve HTTP request using low-level node.js would be tedious, especially in production code.
Also, please note that using WWW-Authenticate Basic for authentication is neither secure nor user-friendly. You need some other way to implement authentication, such as JSON Web Tokens
I am using IBM Bluemix to make a web service for a school project.
My project needs to request a JSON from an API, so I can use the data it provides. I use the http get method for a data set, and I am not sure if it is working properly.
When I run my code, I get the message:
Error: Protocol "https:" not supported. Expected "http:"
What is causing it and how can I solve it?
Here is my .js file:
// Hello.
//
// This is JSHint, a tool that helps to detect errors and potential
// problems in your JavaScript code.
//
// To start, simply enter some JavaScript anywhere on this page. Your
// report will appear on the right side.
//
// Additionally, you can toggle specific options in the Configure
// menu.
function main() {
return 'Hello, World!';
}
main();/*eslint-env node*/
//------------------------------------------------------------------------------
// node.js starter application for Bluemix
//------------------------------------------------------------------------------
// HTTP request - duas alternativas
var http = require('http');
var request = require('request');
// cfenv provides access to your Cloud Foundry environment
// for more info, see: https://www.npmjs.com/package/cfenv
var cfenv = require('cfenv');
//chama o express, que abre o servidor
var express = require('express');
// create a new express server
var app = express();
// serve the files out of ./public as our main files
app.use(express.static(__dirname + '/public'));
// get the app environment from Cloud Foundry
var appEnv = cfenv.getAppEnv();
// start server on the specified port and binding host
app.listen(appEnv.port, '0.0.0.0', function() {
// print a message when the server starts listening
console.log("server starting on " + appEnv.url);
});
app.get('/home1', function (req,res) {
http.get('http://developers.agenciaideias.com.br/cotacoes/json', function (res2) {
var body = '';
res2.on('data', function (chunk) {
body += chunk;
});
res2.on('end', function () {
var json = JSON.parse(body);
var CotacaoDolar = json["dolar"]["cotacao"];
var VariacaoDolar = json["dolar"]["variacao"];
var CotacaoEuro = json["euro"]["cotacao"];
var VariacaoEuro = json["euro"]["variacao"];
var Atualizacao = json["atualizacao"];
obj=req.query;
DolarUsuario=obj['dolar'];
RealUsuario=Number(obj['dolar'])*CotacaoDolar;
EuroUsuario=obj['euro'];
RealUsuario2=Number(obj['euro'])*CotacaoEuro;
Oi=1*VariacaoDolar;
Oi2=1*VariacaoEuro;
if (VariacaoDolar<0) {
recomend= "Recomenda-se, portanto, comprar dólares.";
}
else if (VariacaoDolar=0){
recomend="";
}
else {
recomend="Recomenda-se, portanto, vender dólares.";
}
if (VariacaoEuro<0) {
recomend2= "Recomenda-se, portanto, comprar euros.";
}
else if (VariacaoEuro=0){
recomend2="";
}
else {
recomend2="Recomenda-se,portanto, vender euros.";
}
res.render('cotacao_response.jade', {
'CotacaoDolar':CotacaoDolar,
'VariacaoDolar':VariacaoDolar,
'Atualizacao':Atualizacao,
'RealUsuario':RealUsuario,
'DolarUsuario':DolarUsuario,
'CotacaoEuro':CotacaoEuro,
'VariacaoEuro':VariacaoEuro,
'RealUsuario2':RealUsuario2,
'recomend':recomend,
'recomend2':recomend2,
'Oi':Oi,
'Oi2':Oi2
});
app.get('/home2', function (req,res) {
http.get('https://www.quandl.com/api/v3/datasets/BCB/432.json?api_key=d1HxqKq2esLRKDmZSHR2', function (res3) {
var body = '';
res3.on('data', function (chunk) {
body += chunk;
});
res3.on('end', function () {
var x=json.dataset.data[0][1];
console.log("My JSON is "+x); });
});
});
});
});
});
Here is a print of the error screen I get:
When you want to request an https resource, you need to use https.get, not http.get.
https://nodejs.org/api/https.html
As a side note to anyone looking for a solution from Google... make sure you are not using an http.Agent with an https request or you will get this error.
The reason for this error is that you are trying to call a HTTPS URI from a HTTP client. The ideal solution would have been for a generic module to figure out the URI protocol and take the decision to use HTTPS or HTTP internally.
The way I overcame this problem is by using the switching logic on my own.
Below is some code which did the switching for me.
var http = require('http');
var https = require('https');
// Setting http to be the default client to retrieve the URI.
var url = new URL("https://www.google.com")
var client = http; /* default client */
// You can use url.protocol as well
/*if (url.toString().indexOf("https") === 0){
client = https;
}*/
/* Enhancement : using the URL.protocol parameter
* the URL object , provides a parameter url.protocol that gives you
* the protocol value ( determined by the protocol ID before
* the ":" in the url.
* This makes it easier to determine the protocol, and to support other
* protocols like ftp , file etc)
*/
client = (url.protocol == "https:") ? https : client;
// Now the client is loaded with the correct Client to retrieve the URI.
var req = client.get(url, function(res){
// Do what you wanted to do with the response 'res'.
console.log(res);
});
Not sure why, but the issue for me happened after updating node to version 17, i was previously using version 12.
In my setup, i have node-fetch using HttpsProxyAgent as an agent in the options object.
options['agent'] = new HttpsProxyAgent(`http://${process.env.AWS_HTTP_PROXY}`)
response = await fetch(url, options)
Switching back to node 12 fixed the problem:
nvm use 12.18.3
I got this error while deploying the code.
INFO error=> TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
at new NodeError (node:internal/errors:372:5)
To fix this issue, I have updated the "https-proxy-agent" package version to "^5.0.0"
Now the error was gone and it's working for me.
I want to validate a URL of the types:
www.google.com
http://www.google.com
google.com
using a single regular expression, is it achievable? If so, kindly share a solution in JavaScript.
Please note I only expect the underlying protocols to be HTTP or HTTPS. Moreover, the main question on hand is how can we map all these three patterns using one single regex expression in JavaScript? It doesn't have to check whether the page is active or not. If the value entered by the user matches any of the above listed three cases, it should return true on the other hand if it doesn't it should return false.
There is no need to use a third party library.
To check if a string is a valid URL
const URL = require("url").URL;
const stringIsAValidUrl = (s) => {
try {
new URL(s);
return true;
} catch (err) {
return false;
}
};
stringIsAValidUrl("https://www.example.com:777/a/b?c=d&e=f#g"); //true
stringIsAValidUrl("invalid"): //false
Edit
If you need to restrict the protocol to a range of protocols you can do something like this
const { URL, parse } = require('url');
const stringIsAValidUrl = (s, protocols) => {
try {
new URL(s);
const parsed = parse(s);
return protocols
? parsed.protocol
? protocols.map(x => `${x.toLowerCase()}:`).includes(parsed.protocol)
: false
: true;
} catch (err) {
return false;
}
};
stringIsAValidUrl('abc://www.example.com:777/a/b?c=d&e=f#g', ['http', 'https']); // false
stringIsAValidUrl('abc://www.example.com:777/a/b?c=d&e=f#g'); // true
Edit
Due to parse depreciation the code is simplified a little bit more. To address protocol only test returns true issue, I have to say this utility function is a template. You can adopt it to your use case easily. The above mentioned issue is covered by a simple test of url.host !== ""
const { URL } = require('url');
const stringIsAValidUrl = (s, protocols) => {
try {
url = new URL(s);
return protocols
? url.protocol
? protocols.map(x => `${x.toLowerCase()}:`).includes(url.protocol)
: false
: true;
} catch (err) {
return false;
}
};
There's a package called valid-url
var validUrl = require('valid-url');
var url = "http://bla.com"
if (validUrl.isUri(url)){
console.log('Looks like an URI');
}
else {
console.log('Not a URI');
}
Installation:
npm install valid-url --save
If you want a simple REGEX - check this out
The "valid-url" npm package did not work for me. It returned valid, for an invalid url. What worked for me was "url-exists"
const urlExists = require("url-exists");
urlExists(myurl, function(err, exists) {
if (exists) {
res.send('Good URL');
} else {
res.send('Bad URL');
}
});
Using the url module seems to do the trick.
Node.js v15.8.0 Documentation - url module
const url = require('url');
try {
const myURL = new URL(imageUrl);
} catch (error) {
console.log(`${Date().toString()}: ${error.input} is not a valid url`);
return res.status(400).send(`${error.input} is not a valid url`);
}
Other easy way is use Node.JS DNS module.
The DNS module provides a way of performing name resolutions, and with it you can verify if the url is valid or not.
const dns = require('dns');
const url = require('url');
const lookupUrl = "https://stackoverflow.com";
const parsedLookupUrl = url.parse(lookupUrl);
dns.lookup(parsedLookupUrl.protocol ? parsedLookupUrl.host
: parsedLookupUrl.path, (error,address,family)=>{
console.log(error || !address ? lookupUrl + ' is an invalid url!'
: lookupUrl + ' is a valid url: ' + ' at ' + address);
}
);
That way you can check if the url is valid and if it exists
I am currently having the same problem, and Pouya's answer will do the job just fine. The only reason I won't be using it is because I am already using the NPM package validate.js and it can handle URLs.
As you can see from the document, the URL validator the regular expression based on this gist so you can use it without uing the whole package.
I am not a big fan of Regular Expressions, but if you are looking for one, it is better to go with a RegEx used in popular packages.
I need to receive requests in Node JS that almost identical to HTTP requests, but have a different word to HTTP in the header for example the first line of the request is:
POST / RTSP/1.0
(instead of POST / HTTP/1.0)
The rest of the request format is identical to HTTP in every way.
Is there a way of making the http server parser ignore that the protocol is called HTTP in the first line of the request? So I can use http.createServer etc to receive and respond to these "non-HTTP" requests?
(I know I could use the net module rather than the http module, but then I'd have to implement the header parsing etc, all myself. )
Simplest way would be to implement tcp server in node that acts as a proxy replacing initial POST:
( note: I haven't tested this code, but it should be good enough to illustrate idea)
var net = require('net');
net.createServer(function(s)
{
var sawRequest = false;
var buff = "";
var requestText = "";
var connected = false;
var cli = net.createConnection(yourHttpServerPort);
s.on('data', function(d) {
if (!sawRequest) {
requestText += d.toString();
if (requestText.match(/POST \/ RTSP\/1.0/)) {
requestText.replace('POST / RTSP/1.0', 'POST / HTTP/1.0');
buff = requestText;
sawRequest = true;
}
} else {
buff += d.toString();
}
if (connected)
{
if (buff != '')
cli.write(buff);
cli.write(d);
} else {
buff += d.toString();
}
});
cli.once('connect', function() {
connected = true;
cli.write(buff);
});
cli.pipe(s);
}).listen(publicFacingRTSPport);
You could probably do the same with HAProxy