Customize cors-anywhere - javascript

I have deployed (hosted) cors-anywhere in Heroku, but i don't know how to customize it.
For example, I want to add a site link in whitelist.
I get data from this link: http://fmon.asti.dost.gov.ph/dataloc.php?param=rv&dfrm=null&dto=null&numloc=1&data24=1&locs[]=711
How will I do it? I have tried touching the server.js file:
// Listen on a specific host via the HOST environment variable
var host = process.env.HOST || '0.0.0.0';
// Listen on a specific port via the PORT environment variable
//var port = process.env.PORT || 443;
var port = process.env.PORT || 8080;
// Grab the blacklist from the command-line so that we can update the blacklist without deploying
// again. CORS Anywhere is open by design, and this blacklist is not used, except for countering
// immediate abuse (e.g. denial of service). If you want to block all origins except for some,
// use originWhitelist instead.
var originBlacklist = parseEnvList(process.env.CORSANYWHERE_BLACKLIST);
//var originWhitelist = ['http://fmon.asti.dost.gov.ph/dataloc.php','https://fmon.asti.dost.gov.ph/dataloc.php','http://fmon.asti.dost.gov.ph','https://fmon.asti.dost.gov.ph'];
var originWhitelist = parseEnvList(process.env.CORSANYWHERE_WHITELIST);
function parseEnvList(env) {
if (!env) {
return [];
}
return env.split(',');
}
// Set up rate-limiting to avoid abuse of the public CORS Anywhere server.
var checkRateLimit = require('./lib/rate-limit')(process.env.CORSANYWHERE_RATELIMIT);
var cors_proxy = require('./lib/cors-anywhere');
cors_proxy.createServer({
originBlacklist: originBlacklist,
originWhitelist: originWhitelist,
requireHeader: ['origin', 'x-requested-with'],
checkRateLimit: checkRateLimit,
removeHeaders: [
'cookie',
'cookie2',
// Strip Heroku-specific headers
'x-heroku-queue-wait-time',
'x-heroku-queue-depth',
'x-heroku-dynos-in-use',
'x-request-start',
],
redirectSameOrigin: true,
httpProxyOptions: {
// Do not add X-Forwarded-For, etc. headers, because Heroku already adds it.
xfwd: false,
},
}).listen(port, host, function() {
console.log('Running CORS Anywhere on ' + host + ':' + port);
});
but when I access the data and look at the console log, it returns a 403 error which is forbidden.

NOTE: When you say self hosted CORS it will only work for your site to
proxy. CORS setting on your server is for you not for the list of
sites you mentioned. They will be having their own CORS filters setup.
403 actually refers to the forbidden resource rather than a CORS Issue. Cors issue will look something like as follows:-
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'yourUrl' is therefore not allowed access. The response had HTTP status code 400
For cors-anywhere the whitelisting code is pretty simple as mentioned below:-
// Listen on a specific host via the HOST environment variable
var host = process.env.HOST || '0.0.0.0';
// Listen on a specific port via the PORT environment variable
var port = process.env.PORT || 8080;
var cors_proxy = require('cors-anywhere');
cors_proxy.createServer({
originWhitelist: ['http://fmon.asti.dost.gov.ph'], // Allow all origins
requireHeader: ['origin', 'x-requested-with'],
removeHeaders: ['cookie', 'cookie2']
}).listen(port, host, function() {
console.log('Running CORS Anywhere on ' + host + ':' + port);
});
This should Ideally work for you with your application calling this somewhere.
If you are getting 403 while accessing this URL from your application
then be sure the URL you mentioned is protected and you must get
proper authentication done before requesting it.

I spent 3 days looking for the cause, got a 403 error for some sites and realized that the problem may be that they can not accept requests from "origin" different from theirs.
I just tried removing those headers on the proxy server and everything worked!
removeHeaders: ['origin', 'referer']

Related

NodeJS Koa: How to map a hostname to a htttps server

I have created https server on port 3000 however, currently it is using IP in place of hostname (app.example.com). Hence all the requests happen at https://12.34.56.78:3000.
My current code:
let options = {
key: fs.readFileSync(path.resolve(process.cwd(), 'certs/abc.key'), 'utf8').toString(),
cert: fs.readFileSync(path.resolve(process.cwd(), 'certs/abc.crt'), 'utf8').toString(),
};
const httpsServer = https.createServer(options,app.callback())
.listen(process.env.PORT, "0.0.0.0", listening)
function listening () {
console.log("listening");
}
I want that all request to happen at https://app.example.com:3000.
How can I achieve this?
NOTE:
I have a registered hostname (for example app.example.com)
I have this hostname already running my html,css, and js static files on IIS
I want to use the same hostname for serving APIs of the same website. Hence same hostname

express-http-proxy still gets blocked by CORS policy

I have an express server statically serving my Polymer project. I have a REST API query that I need to make, but if I make it from the client it will be blocked by CORS. So I used express-http-proxy to try to get around that; I send my request to it, and it redirects to the server that has the REST API endpoint on it. This is the entirety of my server code that's running with node server.js:
var express = require('express');
var proxy = require('express-http-proxy');
var server = express();
server.use('/', express.static(__dirname + '/'));
server.listen(8080);
server.use('/rest/api/2/search', proxy('restserver:8877'));
console.log("Server listening on localhost:8080");
When I access restserver:8877/rest/api/2/search in a browser it returns a bunch of json as a 'default' search.
On the client side, I have iron-ajax making this request:
<iron-ajax
id="getBugs"
url="/rest/api/2/search"
params=''
on-response="handleResponse"
debounce-duration="300">
</iron-ajax>
And in the script section, I'm using this.$.getBugs.generateRequest() in the ready function to send the request. So I load this up, expecting the request to not be blocked by CORS, since... it's being proxied by the server. Instead, Chrome devtools gives me this:
XMLHttpRequest cannot load http://restserver:8877/secure/MyJiraHome.jspa. Redirect from 'http://restserver:8877/secure/MyJiraHome.jspa' to 'http://restserver:8877/secure/Dashboard.jspa' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8080' is therefore not allowed access.
I don't understand why it's giving me those URLs, since I never reference them, and why it's blocking due to CORS, since it's going from the server, not the client, that being the whole point of the proxy.
It may be that express-http-proxy simply forwards the Origin header coming from your client which is http://localhost:8080, causing the end server to reject it.
Try modifying it with proxyReqOptDecorator:
server.use('/rest/api/2/search', proxy('restserver:8877', {
proxyReqOptDecorator(proxyReqOpts) {
proxyReqOpts.headers['Origin'] = 'http://accepted.origin.com';
return proxyReqOpts;
}
}));
Never used express-http-proxy and did not test it tho, so tell me if it's not a solution. Also I think using cors as other people suggested may simplify things a lot. But I don't know your development constraints, so I could be wrong.
Server is probably returning a 302 redirect which is not handled correctly in the used middleware.
Read more how the redirect works: https://en.wikipedia.org/wiki/HTTP_location
You can modify the Location response header to overcome the CORS issue or you can try:
var proxy = require('http-proxy-middleware');
var restServerProxy = proxy({target: 'http://restserver:8877', autoRewrite: true});
server.use('/rest/api/2/search', restServerProxy);
The above example should handle redirects automatically.
You don't need any proxy. Since you are calling endpoint on your server, you can just whitelist your client side for calling your server. You can do that with cors package.
https://www.npmjs.com/package/cors
First, define your CORS policy logic in one file (let's name it cors-policy-logic.js), and then export it so you can use it in other files.
const cors = require('cors');
const whitelist = ['http://localhost:8080', 'http://localhost:your_client_url'];
var corsOptionsDelegate = (req, callback) => {
var corsOptions;
if (whitelist.indexOf(req.header('Origin')) !== -1) {
corsOptions = { origin: true };
} else {
corsOptions = { origin: false };
}
callback(null, corsOptions);
};
exports.cors = cors();
exports.corsWithOptions = cors(corsOptionsDelegate);
Now, import it and use it anywhere were you define some endpoint:
var express = require('express');
const cors = require('./cors-policy-logic.js');
var server = express();
server.use('/', express.static(__dirname + '/'));
server.listen(8080);
server.route('/rest/api/2/search')
.options(cors.corsWithOptions, (req, res) => { res.sendStatus(200); })
.get(cors.cors, (req, res, next) => {
//Your business logic
});
console.log("Server listening on localhost:8080");
Alternative solution would be to use http-proxy-middleware as mentioned by #chimurai.
If you want to proxy to an https server to avoid CORS:
const proxy = require('http-proxy-middleware');
app.use('/proxy', proxy({
pathRewrite: {
'^/proxy/': '/'
},
target: 'https://server.com',
secure: false
}));
Here secure: false needs to be set to avoid UNABLE_TO_VERIFY_LEAF_SIGNATURE error.

Configuring domain name instead of IP address while using restify in node app

I am using restify module to generate REST APIs for my application. I want to know how i could configure my domain name directly in this code. I want my ip_addr something like "domainname.com" instead of 127.0.0.1 .
var ip_addr = 'domainname.com';
var port = '80';
var connection_string = '127.0.0.1:27017/myapp';
var db = mongojs(connection_string, ['myapp']);
var jobs = db.collection("jobs")
var server = restify.createServer({
name : "myapp"
});
server.use(restify.queryParser());
server.use(restify.bodyParser());
server.use(restify.CORS());
server.listen(port ,ip_addr, function(){
console.log('%s listening at %s ', server.name , server.url);
});
I am getting an error EADDRNOTAVAIL whenever I use "domainname.com" instead of "127.0.0.1" can someone help, how can I configure it to my domain name in this node app?
There is some misunderstanding. It asks to specify IP address to listen to.
Example:
Server has address 100.1.1.2 and 100.1.1.3
mydomain.com is registered to 100.1.1.3
set ip_addr = '100.1.1.3'
In this case, the server will respond to all requests to this ip, and all domains registered to this IP. Connection through 100.1.1.2 will be timed out.
Shortly, if you want to listen to domainname.com, just specify the IP it is registered to.
And if you do not want to worry about the address and listen to all available network interfaces, then do not specify IP address at all:
server.listen(port, function() { .... Server API

Integrating Backbone with Nodejs

Trying to fetch Nodejs response from Backbone Model.
Update
Changed Model code as following and getting error
XMLHttpRequest cannot load http://localhost:3000/getDifficulty. No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'null' is therefore not allowed access.
var Bitcoin = Backbone.Model.extend({
url:'http://localhost:3000/getDifficulty'
});
var info = new Bitcoin ();
info.fetch();
Node JS is very straight forward and works fine for the url http://localhost:3000/getDifficulty
Server side Node JS
var http = require('http'),
express = require('express'),
bitcoin = require('bitcoin');
var app = express();
var client = new bitcoin.Client({
host: 'localhost',
port: 8332,
user: 'himanshuy',
pass: 'xxx'
});
app.get('/getDifficulty', function(req, res) {
client.getInfo(function(err, info) {
if(err) {
res.send('Bitcoin error: '+ err);
} else {
res.send('Difficulty: ' + info);
}
});
});
app.listen(3000);
Client Side Backbone Model
var Bitcoin = Backbone.Model.extend({
urlRoot:'http://localhost:3000/getDifficulty'
});
var info = new Bitcoin();
It works fine if give some value to the model like this
var info = new Bitcoin({version:"1.0.0.", balance:"20.03"});
Which means that model is not getting the result from url.
Please help.
Note: I am fairly new to both backbone and Nodejs
Try using
var Bitcoin = Backbone.Model.extend({
url:'getDifficulty'
});
var info = new Bitcoin();
info.fetch();
And take a look at the console to see the response from the server. From the looks of it, the above won't work, because you're not returning a json object res.send('Difficulty: ' + info). Backbone expects JSON data to be provided (and only JSON) so it can load the values into the model instance.
If what you're returning from the server needs to be modified before being loaded into the modekl, you need to implement the parsefunction (see http://backbonejs.org/#Model-parse):
var Bitcoin = Backbone.Model.extend({
url:'getDifficulty',
parse: function(response){
// the return value should be what needs to loaded into the model
// for example, if we need to only have the `data`attribute in the model:
return response.data;
}
});
If you are getting errors like
No 'Access-Control-Allow-Origin' header is present on the requested resource.
Origin 'null' is therefore not allowed access.
You may need to configure your browser to disable CORS or web security settings. This Chrome extension should get around that error.
In Firefox, Force CORS extension is helpful.
Launching Chrome with a certain command line argument can also prevent these type of errors, but at the cost of vulns like XSS becoming easier to suffer: On Windows in the target field, this works - with the consequence of lower browser security as well:
"C:\Program Files\Google\Chrome\Application\chrome.exe" --disable-web-security

Measuring traffic with node.js http-proxy

I wrote a small reverse proxy for hosting my applications on the same computer using http and node-http-proxy modules. For example:
I have:
proxy running on port 80
website1.com running on port 3000
website2.com running on port 3001
website3.com running on port 3002
If I access the website1.com domain, the proxy will serve the contents from server running on port 3000 using node-http-proxy.
But now I need to measure the bandwidth used for each domain (both incoming/outgoing, or at least outgoing)
I've tried listening for 'data' events on request object, but in documentation they said that readable events isn't emitted on IncomignMessage for some reason.
I wrote a little module for the "base" functionality too, it can be found here:
https://npmjs.org/package/reproxy
See example/example.js
So, how can I accomplish this measure, using the current setup?
The solution I found was setting and 'end' event on RoutingProxy object and grabbing the socket information in the event callback.
var proxy = new require('http-proxy').RoutingProxy();
proxy.on('end', function(req, res, response) {
var host = req.headers.host;
var bytesIn = response.socket._bytesDispatched;
var bytesOut = response.socket.bytesRead;
console.log('request to ' + host);
console.log('request: ' + bytesIn + ' bytes.');
console.log('response: ' + bytesOut + ' bytes.');
});
Note that this is not optimal solution, because the request size includes the headers added by the reverse proxy, such as "x-" headers.

Categories