We Keep Coding

Why does this http request time out?

I am sending a GET request inside an OpenShift pod to a backend application. In the console of the openshift pod I can run the following command succesfully:
wget -qO- --no-check-certificate \
--header 'Accept: */*' \
--header 'Authorization: Basic <base64 encoded auth>' \
https://my-url.nl:8221/path/to/my/resource/organisations
I can also call the API endpoint from postman without issue (from a different server).
However, when I run the following fetch I get a timeout.
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
const credentials = Buffer.from(
`USERNAME:PASSWORD`
).toString('base64');
fetch(
'https://my-url.nl:8221/path/to/my/resource/organisations',
{
method: 'GET',
headers: {
Authorization: `Basic ${credentials}`,
Accept: '*/*',
},
}
).then((data) => {
console.log(data.json());
}).catch((reason) => {
console.log(reason);
});
I get the following error after a long wait
GET https://my-url.nl:8221/path/to/my/resource/organisations net::ERR_CONNECTION_TIMED_OUT
Uncaught (in promise) TypeError: Failed to fetch
Both the preflight and actual API call fail.
Am I forgetting something simple? Does this have to do with CORS? Or something with skipping the certificate validation? What kind of API call would return a timeout rather than a failed (400/500) response.

It's simply not going to work if you're doing it from the browser, as seen in the photos.
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
These two lines do nothing in the browser. They only work in a NodeJS environment.
The only way to make it work is to change the backend endpoint, to a valid SSL cert (not self-signed). The easiest way to do that is probably cloudflare.
EDIT: I suspect you're compiling this from Node via react, vue, etc. In that case, it absolutely won't work because process.env is setting a system environment variable. Browsers don't have environment variables, so this isn't going to work, those 2 lines will do nothing.

Thank you for the comments & answers to my post. It made me question how I was calling the endpoint. I am working with NextJs (React framework). This allows you to perform API calls, either client side or server side. I was performing these calls client side rather than server side (which won't work). The solution is to use the NextJs GetServerSideProps function.
My full working solution for anyone interested:
import { PageLayoutWithSideMenu } from '#pcss-cargonaut/ui-lib';
import NextLink from 'next/link';
import { GetServerSideProps } from 'next';
type Organisation = {
accountNumber: number;
accountSequenceNumber: number;
name: string;
role: string;
};
type Organisations = Organisation[];
export const getServerSideProps: GetServerSideProps = async () => {
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
const credentials = Buffer.from(
`USERNAME:PASSWORD`
).toString('base64');
const res = await fetch(
'https://my-url.nl:8221/path/to/my/resource/organisations',
{
method: 'GET',
headers: {
Authorization: `Basic ${credentials}`,
Accept: '*/*',
},
}
);
const organisations: Organisations = await res.json();
return { props: { organisations } };
};
const Organizations = ({ organisations }: { organisations: Organisations }) => {
console.log(organisations);
return (
<PageLayoutWithSideMenu>
...
</PageLayoutWithSideMenu>
);
};
export default Organizations;

Fetch returns a promise that resolves with a Response object and the response.json() returns, in turn, another promise that resolves with the result of parsing the response body text as JSON.
So, you have to add another .then() after the first one.
A general example below:
fetch('http://example.com/movies.json')
.then(response => response.json())
.then(data => console.log(data));

Fetch problem with node.js express server

I'm having some trouble with the fetch and node.js.
In my frontend when i click a button, i would like to send a post request in order to receive an array from my backend as answer. I'n my backend i'm using node.js with express, in my frontend i'm using the fetch function.
The error that occours is the following:
Access to fetch at 'http://localhost:8080/api' from origin 'real localhost address' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Code Here
const getArray = async() => {
const data = await fetch ("http://localhost:8080/api");
const dataJson = await data.json();
console.log(dataJson)
}
getArray();
In my server i've got
app.post("/api", (req,res) => {
res.sendFile(JSON.stringify(arr));
});

You need to add request options. Please refer to the MDN docs for further information.

As #Kudah said, you should read the docs.
Fetch (and XMLHttpRequest) follow the same-origin policy. This means that browsers restrict cross-origin HTTP requests from within scripts. A cross-origin request occurs when one domain (for example http://example2.com/) requests a resource from a separate domain (for example http://example1.com/).
The easiest way to solve this, (If you don't want to dig too much into this)
const whiteList = [ "https://myRealBackendUrl-1", "https://myRealBackendUrl-2" ];
// you can also pass a string here instead here instead of array
const corsOptions = {
credentials: true,
origin: process.env.NODE_ENV !== production ? "http://localhost:4000" : whiteList
// if you are in a dev environment, you probably want something like localhost
// http://localhost:4000 is just a demo backend. replace it with your own.
// if you are in a production environment, for example heroku then your backend
// url will be something like http://example.herokuapp.com
// in that case `const whiteList = [ "http://example.herokuapp.com" ];`
};
app.use(cors(corsOptions));
The above code should be enough for the normal use case.
There is also callback function, it is if you want to run some function of your own. Don't read it if you dont plan to use any dynamic checking
var corsOptionsDelegate = async (req, callback) => {
var corsOptions = { origin: false };
try {
// you can do some dynamic check here
// For example: check database for some conditions then allow access
if( myDatabaseSays == true ) corsOptions.origin = true;
else corsOptions.origin = false;
} catch (err) {
console.log(err);
// corsOptions.origin = false;
}
callback(null, corsOptions) // chain it
}
Anyway read the docs properly for more info
[1]: https://expressjs.com/en/resources/middleware/cors.html

How do I set the origin in the DeepL API on javascript?

What else should I try?
I'm currently sending a request to the DeepL API in axios, but I'm getting a 403 response due to a CORS issue.
And tried to set the option using querystring as shown here, but it didn't work. https://github.com/funkyremi/deepl/blob/master/index.ts
Also, using the library at the URL above returns 403.
Furthermore, there is no origin setting in the account settings of DeepL.
I tried using 'Content-Type': 'application/x-www-form-urlencoded' for axios headers: {}, and I also tried setting options for params: { } and not using querystring, but they didn't work.
import axios from 'axios'
import querystring from 'querystring';
export const translateDeepL = async() => {
const options = {
"auth_key": process.env.DEEPL_AUTH_KEY,
"text": 'everyday is birthday.',
"target_lang": 'JA',
};
const url = "https://api-free.deepl.com/v2/translate";
const data = await axios.post(url, querystring.stringify(options)).then(r => r);
console.log(data);
}
VM3451:1 POST https://api-free.deepl.com/v2/translate 403
the request use https with ngrok did't work also.
I also tried the GET method for "https://api-free.deepl.com/v2/usage" but got the same result.
It is definitely api-free.deepl.com since I am using the free plan.
By the way, the above code is executed as a component in React.

the DeepL API does not support being used directly from within browser-based apps. The API Key is not supposed to be shared publicly as well and should always be kept secret.
The best approach is to use a backend proxy for the API Calls.

I was encountering this same issue and couldn't find an answer. This API just didn't seem to want to talk to me via a browser.
My 'solution' was to set up an API proxy in node.
It works fine fetching from a back-end + now I can add some rate limiting etc
C.J on coding garden can explain this way better than I ever can.

You might be being blocked because of sending a request from http (your localhost) to https, try using the proxy axios config, like
const response = await axios
.get("https://api-free.deepl.com/v2/translate", {
params: {
auth_key: x,
text: y,
target_lang: z
},
proxy: {
host: "localhost",
port: 8080
}
});
return response;
};

Axios request receiving a CORS error only when inside a callback/async function

I'm using VueJS for an app I am building. The server I have is written in Golang and has been set to accept CORS. In the app, in one of my components, searchBar, I have set it to fetch some data before it is created.
var searchBar = {
prop: [...],
data: function() {
return { ... };
},
beforeCreate: function() {
var searchBar = this;
axios.request({
url: '/graphql',
method: 'post',
data: {
'query': '{courses{id, name}}'
}
})
.then(function(response) {
searchBar.courses = response.data.data.courses;
});
},
methods: { ... },
template: `...`
};
Axios works perfectly here. It gets the data I need. searchBar has a button which causes it to emit an event which is then picked up by another component, searchResults. Upon receiving the event, searchResults will fetch some data.
var searchResults = {
data: function() {
return { ... }
},
mounted: function() {
var sr = this;
this.$bus.$on('poll-server', function(payload) {
var requestData = {
url: '/graphql',
method: 'post',
data: { ... },
...
};
...
axios.request(requestData)
.then(function(response) {
console.log(response);
}
);
});
},
template: `...`
};
Note that my Axios request call is now inside a callback function. When this call is performed, I receive a CORS error:
Access to XMLHttpRequest at 'http://127.0.0.1:9000/graphql' from origin 'http://127.0.0.1:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
My server is located at http://127.0.0.1:9000, with the client in http://127.0.0.1:8080. Here is the content of the OPTIONS request of the second request call.
For comparison, here is the request header of the first request call (that works!).
I have already set my Golang server to support CORS via go-chi/cors. This is how set it up.
router := chi.NewRouter()
...
// Enable CORS.
cors := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"POST", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "X-CSRF-Token"},
ExposedHeaders: []string{"Link"},
AllowCredentials: true,
MaxAge: 300,
})
router.Use(
render.SetContentType(render.ContentTypeJSON),
middleware.Logger,
middleware.DefaultCompress,
middleware.StripSlashes,
middleware.Recoverer,
cors.Handler,
)
router.Post("/graphql", gqlServer.GraphQL())
return router, db
What is causing the error I am having and how can it be solved?

This CORS error is expected. The CORS plugin you are using does request filtering for you. If you look at the list of allowed headers, you can see it's missing the header called snb-user-gps-location that you are trying to send in your axios call.
Either add that header to the allowed list, or don't send it from the front end.

I still suspect the go-chi CORS setup. I would suggest looking at setting up CORS by hand. It's not that difficult. This page will get a basic setup: https://flaviocopes.com/golang-enable-cors/
If that works with your nested API setup, we can then work backwards to determine the go-chi config issue.
Update:
I would also investigate the other middleware steps - commenting out all non-essential ones.
Middleware handlers normally inspect the r *http.Request or write headers to the w http.ResponseWriter and then the final handler will write to the response body. But throughout the middleware chain the following header/body write flow should look like one of these two flows:
Success:
w.Header().Set(...) // headers
w.Write(...) // body
Note the above flow will issue an implicit http status code write, to keep headers appearing first and body second:
w.Header().Set(...) // headers
w.WriteHeader(http.StatusOK) // implicit success http status code
w.Write(...) // body
Failure:
In the event of reporting a runtime error, the flow should be:
w.Header().Set(...) // headers
w.WriteHeader(http.StatusInternalServerError) // some unrecoverable error
w.Write(...) // optional body
The reason I bring this up, I've seen 3 types of bugs which mess up this flow:
bad middleware handlers write headers after the body causing client confusion
calling http.Error thinking that stops the API dead - instead of returning immediately after the http.Error and ensuring no subsequent middleware handlers are called
write the same header twice. Rewriting headers in a subsequent handler will cause the client to see the last version (thus clobbering any previous versions)
So to fully trace things, I would log.Println all header/body writes for your API to ensure the above flow is correct and no intended values are being overwritten.

My node.js https client always works regardless of certificate validity

This test program connects to an https server and gets some content. I've checked my server in browsers and with curl and the certificate is working correctly. If I run curl to grab data from the server it correctly complains about the certificate being unknown unless I pass it in with --cacert or turn security off with -k.
So the problem I am having is that although I think my client should be doing certificate authentication and I am telling it where the public certificate is, it just always works. If I remove the ca: option so it has no idea what the certificate is from the server then it silently works. I would like to catch the authentication error but I can't seem to do so.
var https = require('https');
var fs = require('fs');
function main() {
var data = '';
var get = https.get({
path: '/',
host: 'localhost',
port: 8000,
agent: false,
ca: [ fs.readFileSync('https_simple/cacert.pem') ]
}, function(x) {
x.setEncoding('utf8');
x.on('data', function(c) {data += c});
x.on('error', function(e) {
throw e;
});
x.on('end', function() {
console.log('Hai!. Here is the response:');
console.log(data);
});
});
get.on('error', function(e) {throw e});
get.end();
}
main();

In order to make this work I needed to upgrade to v0.7.8 (although any v0.7 should be fine) where the rejectUnauthorized functionality has been added to https.get
This combination of options is needed:
agent: false, // or you can supply your own agent, but if you don't you must set to false
rejectUnauthorized: true,
ca: [ fs.readFileSync('https_simple/cacert.pem') ]
Now if the authentication fails you will get an 'error' event and the request will not go ahead.
See the https.request documentation for details on making your own Agent
The bug fix was committed in this change: https://github.com/joyent/node/commit/f8c335d0

As per the documentation for https.request, the ca option of both https.get and https.request is an option from tls.connect. The documentation for the options to the tls.connect module function states:
ca: An array of strings or Buffers of trusted certificates. If this is
omitted several well known "root" CAs will be used, like VeriSign.
These are used to authorize connections.
Digging into the node.js source, the root certs used can be found here: https://github.com/joyent/node/blob/master/src/node_root_certs.h
So in short, with no authority cert provided as an option to https.get the tls module will attempt to authenticate the connection using the list of root certs anyway.

I do this in npm, using the request module. It goes like this:
var cacert = ... // in npm, this is a config setting
var request = require("request")
request.get({ url: "https://...",
ca: cacert,
strictSSL: true })
.on("response", function (resp) { ... })
.on("error", function (er) { ... })
The error event will be raised if the ssl isn't valid.

In V 0.6.15 you need to explicitly check whether or not the certificate validation passed or failed.
if (x.connection.authorized === false) {
console.log('SSL Authentication failed');
} else if (x.connection.authorized === true) {
console.log('SSL Authentication succeeded');
}