Request HTTP libraries are slow - javascript

I'm making a project using Node.js (8.9.4) and Express (4.16.2). Basically, with Node.js i'm consulting an external API and then with Express make a route to consume this result. My surprise was that when I used Axios library, it make my response to delay up to ~30s. I wanted to check if it was my problem or was from the API... I've checked it with PostMan and it returns in less than 300ms. Then I've thought it was any problem related with Axios, so I've decided to use request-promise but...again 30s. The last test I've made is using Node.js native 'https' util and...yes, less than 300ms again.
Anyone knows whats the problem with those packages? Am I condemned to use callbacks instead of promises?
Here's my base code... (AXIOS, Request-Promise... 30s delay in response)
const rp = require('request-promise);
const BASE_URL = 'my https url';
const AUTH_TOKEN = 'my auth token';
const options = {
uri: BASE_URL + '/my-route',
qs: { myQS: true },
headers: { authorization: AUTH_TOKEN }
method: 'GET'
};
rp(options)
.then(response => response)
.catch(error => error);
Here's my base code with HTTPS.... 300ms delay in response
const https = require('https');
const AUTH_TOKEN = 'my auth token';
const options = {
hostname: 'my hostname',
port: 443,
path: 'my path',
headers: { authorization: AUTH_TOKEN },
method: 'GET'
};
https.get(options, (res) => {
res.on('data', d => d);
};

Just a guess, but sounds like Axios is going via a different route in your network. Are both requests configured to use the same proxy setup?

Related

Change IP using Jest and Supertest

I'm following the documentation of nestjs to create my e2e tests.
Now I'd like to create requests and change the IP, because I have an interceptor accepts requests only from a range of ip.
Now I try to create it in my e2e class
const req = require('express/lib/request');
const CLIENT_IP = '1.2.3.4';
beforeEach(() => {
// Mock the `ip` property on the `req` object
let ip = jest.spyOn(req, 'ip', 'get').mockReturnValue(CLIENT_IP);
});
but the ip remains the same (my error: 'Not a valid IP ::ffff:127.0.0.1').
I tried to change the ip in my init setup like:
app.set('Remote-Addr', '1.1.1.1')
but the error is the same (and I don't like this approach.
How can I decide a specific ip for each requests?
I'm using express
I am not really sure why you need to mock the client ip address, you should be sending it as a http header e.g. x-client-id or x-forwarded-for.
Which means that if you are writing a NestJS e2e test using supertest, you should be able to manually set the ip-address in the header e.g.
import { IncomingHttpHeaders } from 'http';
import { Test, TestingModule } from '#nestjs/testing';
it('Create a new user -> (201) success', async () => {
const requestHeaders: IncomingHttpHeaders = {
'x-client-id': '1.1.1.1',
'content-type': 'application/json',
}
const requestBody: User = {
name: 'joe blogs',
age: '65'
}
const { statusCode, body } = await app.inject({
method: 'POST',
url: '/api/v1/users',
headers: requestHeaders,
payload: requestBody,
});
expect(statusCode).toEqual(HttpStatus.CREATED);
exoect(body).toBeDefined();
});

Axios POST working in Node.js but not Electron

So I have a package with a function that uploads a file to Twilio:
const FD = require('form-data');
const axios = require('axios');
async function createFunctionResource(serviceUid, functionUid, client){
let collect_file = "Hello World"
let url = `https://serverless-upload.twilio.com/v1/Services/${serviceUid}/Functions/${functionUid}/Versions`
let form = new FD();
collect_file = "test"
form.append("Path", "collect");
form.append("Visibility", "public");
form.append("Content", collect_file, "collect.js");
form.append("contentType", "application/javascript");
await axios.post(url, form, {
headers: {
Authorization: 'Basic ' + Buffer.from(`${client.accountSid}:${client.password}`).toString('base64'),
...form.getHeaders(),
},
})
}
This works completely fine in node.js and it gets uploaded with the message "Hello World" in the file.
I'm trying to put this into an electron app so I preload this package in preload.js with nodeIntegration set to true but whenever I try to upload a file I get:
Request failed with status code 400
With the error response being:
{"message":"No file attached to request","code":70002,"user_error":true,"http_status_code":400,"params":{}}
Does preloading a package make it act exactly the same as it does in node.js?
Can u add cotent-type in headers section and check .
"content-type": "application/json"
Even though you may try and preload a package with axios hoping it runs in a node environment, requests are done under XHR (browser).
To fix this you must specify the adapter to be HTTP by adding adapter: require('axios/lib/adapters/http')
await axios.post(url, form, {
headers: {
Authorization: 'Basic ' + Buffer.from(`${client.accountSid}:${client.password}`).toString('base64'),
...form.getHeaders(),
},
adapter: require('axios/lib/adapters/http'),
})
}

client (fetch) and server (nodejs http) don't understand each other?

I try to wrap my mind around nodejs at the moment.
So I've created a client:
let myHeaders = {
'Content-Type': 'application/json',
'Accept': 'application/json'
};
let myBody = {
aString: "Test"
};
fetch("http://localhost:8099/", {
method: 'post',
mode: 'no-cors',
headers: myHeaders,
body: JSON.stringify(myBody)
})
.then(result => {
return result.text();
})
.then(text => {
// do stuff with text from server
});
And I have created a server:
// request needed modules
const http = require('http');
// init server
let server = http.createServer(logic);
server.listen(8099);
// server logic
function logic (req, res) {
var body = req.body;
res.end("Hello");
}
Two problems:
1) The sever does not get the body (req.body is undefined).
UPDATE
See my answer below.
--
2) The client does not receive "Hello" (result.text() returns "").
UPDATE
2 is solved by:
Changing this on the client
fetch("http://localhost:8099/", {
method: 'post',
mode: 'no-cors', <-- CHANGE to: mode: 'cors'
...
Adding this on server
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Headers', 'Content-Type');
--
I don't get what I do wrong here...
Your Node.js code has nothing which would generate the HTML document containing the JS that calls fetch.
This means you must be making a cross-origin request (this is supported by the absolute URL you pass to fetch).
You also set mode: 'no-cors' which means "Don't throw a security exception for trying to access a cross-origin resource, and don't make the response available to JS".
Consequently, when you try to read the response: You can't.
Set the mode to "cors" and change the Node.js code to follow the CORS specification to grant permission to the page trying to read the data.
I try to wrap my mind around nodejs at the moment.
There is nothing particular to Node.js here. The problems are related to security restrictions on what JavaScript running in the browser can do unless granted permission by the HTTP server.
To not completely mess up my question, I post my solution for problem number one as separate answer:
SOLUTION 1) The sever does not get the body (req.body is undefined)
As request is a stream, I need to treat it like one (notice "req.on('data'...)
This is how the server works as expected:
// request needed modules
const http = require('http');
// init server
let server = http.createServer(handler);
server.listen(8099);
// server logic
function handler (req, res) {
// Set CORS headers
let headers = {
'Access-Control-Allow-Origin' : '*',
'Access-Control-Allow-Methods' : 'POST, OPTIONS',
'Access-Control-Allow-Headers' : 'Content-Type, Accept'
};
res.writeHead(200, headers);
if(req.method == 'POST'){
var body = '';
req.on('data', data => {
body += JSON.parse(data).aString;
});
req.on('end', () => {
res.end(body.toString().toUpperCase());
});
} else if (req.method == 'OPTIONS'){
res.end();
}
}

Axios HTTP requests returns into an error (Access-Control-Allow-Origin)

I'm trying to make http post requests with Axios in JavaScript. The request was working fine, but then I tried to use cookies. As my backend I'm using an Express/Nodejs Server on http://localhost:8000, while my frontend is a react npm test server on http://localhost:3000.
My backend looks like this:
const express = require('express');
const cookieparser = require('cookie-parser');
const cors = require('cors');
const app = express();
app.use(cookieparser());
app.use(cors());
app.post("/request/status/check", (req, res) => {
if(req.cookies.gitEmployee != null){
res.status(200).send({res: 1, employeeName: req.cookies.gitEmployee.username, fullname: req.cookies.gitEmployee.fullname});
} else if(req.cookies.gitCompany != null){
res.status(200).send({res: 2, companyName: req.cookies.gitCompany.companyName, fullname: req.cookies.gitCompany.fullname});
}else{
res.status(200).send({res: 0});
}
});
app.post("/request/testcookie", (req, res) => {
res.cookie("gitEmployee", null);
res.cookie("gitEmployee", {
username: "testusername",
fullname: "Test Username"
}).send({res: 1});
});
So, as a short description: I'm setting a test cookie by posting a request to http://localhost:8000/request/testcookie. The response should be an JSON object where res = 1. Also, I'm trying to get information out of the cookie by posting a request to http://localhost:8000/request/status/check. In this case the response should be the object {res:1 , employeeName: "testusername", fullname: "Test Username"}.
I tried this concept with a REST Client called Insomnia (something like Postman) and it worked perfectly.
Then I wrote a helper-class for my React Application and for the Http request I'm using Axios.
import axios from 'axios';
class manageMongo {
authstate(){
return new Promise((resolve, reject) => {
axios("http://localhost:8000/request/status/check", {
method: "post",
data: null,
headers: {
"Access-Control-Allow-Origin": "*"
},
withCredentials: true
})
.then(res => {
console.log(res.data);
if(res.data.res === 0){
resolve(false);
}
if(res.data.res === 1){
resolve(true);
}
if(res.data.res === 2){
resolve(true);
}
});
});
}
setTestCookie(){
axios("http://localhost:8000/request/testcookie", {
method: "post",
data: null,
headers: {"Access-Control-Allow-Origin": "*"},
withCredentials: true
})
.then(res => { console.log(res)});
}
}
export default manageMongo.prototype;
When I execute these functions, I'm getting the same error of both of them (of course with different urls):
Failed to load http://localhost:8000/request/testcookie: Response to
preflight request doesn't pass access control check: The value of the
'Access-Control-Allow-Origin' header in the response must not be the
wildcard '*' when the request's credentials mode is 'include'
I already know that it's because of the withCredentials setting in the requests. I added these settings because I want to pass cookies through these requests and if I don't add withCredentials, the /request/status/check request always returns {res: 0} even if I set a cookie before.
I don't know, if this will change if the I set withCredentials = true but i read that in multiple threads. If you know an other working method to pass cookies through these requests even without axios please share it here! Because that is, what I want to achieve.
The problem seems to be you have set
'Access-Control-Allow-Origin': *
Try setting it to your actual origin, for example
'Access-Control-Allow-Origin': 'http://localhost:8000'
or
'Access-Control-Allow-Origin': 'http://localhost:3000'
Whichever the request originates from.

Seneca-web timeout configuration

First of all I would like to say that I am new in senecajs.
I am testing this configuration.
I have configured Senecjs microservice running on port 9007, which is running and handling request correctly. When I request this service directly I receive response after cca 10s (it is request for oracle db data).
But when I request for same data but through the Hapi + Seneca-web I receive this error: "statusCode":504,"error":"Gateway Time-out"
["client","invalid_origin",{"port":9007,"pin":"mc:bankgtw","pg":"mc:bankgtw","type":"web","id":"pg:mc:bankgtw,pin:mc:bankgtw,port:9007","role":"transport","hook":"client","plugin$":{"name":"client$"},"fatal$":true,"meta$":{"mi":"wbn8u45tb7uh","tx":"o3f8eyia3f4n","id":"wbn8u45tb7uh/o3f8eyia3f4n","pattern":"hook:client,role:transport,type:web","action":"(q1yytemztu3k)","plugin_name":"transport","plugin_tag":"-","prior":{"chain":[],"entry":true,"depth":0},"start":1487199713842,"sync":true},"tx$":"o3f8eyia3f4n","host":"0.0.0.0","path":"/act","protocol":"http","timeout":5555,"max_listen_attempts":11,"attempt_delay":222,"serverOptions":{}},{"kind":"res","res":null,"error":{"isBoom":true,"isServer":true,"output":{"statusCode":504,"payload":{**"statusCode":504,"error":"Gateway Time-out**","message":"Client request timeout"},"headers":{}}},"sync":true,"time":{"client_recv":1487199799177}}]
A few seconds before microservice return data.
And this is my configuration:
const Hapi = require('hapi');
const Seneca = require('seneca');
const SenecaWeb = require('seneca-web');
const config = {
adapter: require('seneca-web-adapter-hapi'),
context: (() => {
const server = new Hapi.Server();
server.connection({
port: 3001,
routes: {
cors: true,
payload:{timeout:60000},
timeout:{server: 60000, socket:90000}
}
});
server.route({
path: '/routes',
method: 'get',
handler: (request, reply) => {
const routes = server.table()[0].table.map(route => {
return {
path: route.path,
method: route.method.toUpperCase(),
description: route.settings.description,
tags: route.settings.tags,
vhost: route.settings.vhost,
cors: route.settings.cors,
jsonp: route.settings.jsonp,
server: server.info
}
})
reply(routes)
}
});
return server;
})()
};
const seneca = Seneca({timeout: 99999})
.use(SenecaWeb, config)
.use(require('./hapi_api.js'))
.client({ port:9007, pin:'mc:bankgtw' })
.ready(() => {
const server = seneca.export('web/context')();
server.start(() => {
server.log('server started on: ' + server.info.uri);
});
});
What I am doing wrong or what timeout is causing this?
I've had the same issue, fixed it, but its VERY BAD PRACTICE.
Go to 'transport.js' at seneca-transport folder.
You will see 'timeout: 5555'
Go ahead and change that to whatever you need.
I'm not sure why this is not getting USER defaults.
To the best of my knowledge, this is referring to client timeout. make sure you still use server timeout.

Categories