I'm using Angular's HttpClient to make HTTP requests and I'd like to specify a timeout for them.
I know I can use HTTPInterceptors and add a timeout to RxJS operators, however, these apply to the whole request which I don't want to abort if data transfer is in progress, only if the browser is hanging while trying to connect.
The kind of timeout I need is available in Node.js for example which is well explained here:
Let's say you called socket.setTimeout(300) to set the timeout as 300
ms, and it took 100 ms for DNS lookup, 100 ms for making a connection
with a remote server, 200 ms for the remote server to send response
headers, 50 ms for transferring the first half of the response body
and another 50 ms for the rest. While the entire request & response
took more than 500 ms, timeout event is not emitted at all.
Is it possible to have a timeout like this in an Angular app?
I looked at the source code for the HttpClient. The code that actually deals with the underlying XMLHttpRequest is the class HttpXhrBackend, in source file named xhr.ts
Unfortunately, HttpXhrBackend just uses the default settings of XMLHttpRequest, and does not provide a way to set the XMLHttpRequest's timeout value.
I have seen suggestions for using RxJS operators to shorten the effective timeout,
but that's a bit of a hack, and doesn't really do what you are asking for.
So, technically, the answer to your question is "No", not with the stock Angular HttpClient, but I suppose that you could create your own implementation of HttpBackend and attempt to inject that.
P.S. This article shows how to provide a custom HttpBackend implementation.
For those still seeking a 'hacky' solution you can create an observable and insert an empty/failure after your desired timeout period:
handleError(error: HttpErrorResponse) {
console.warn('HTTPErrorResponse caught', error);
return observableOf({});
}
async __sendCommandHTTP(cmd: SoftAPCommand) {
const URL = this.options.host + cmd.name;
let result: SoftAPResponse = {
name: cmd.name,
payload: {},
error: false,
};
this.logger.debug('[softap-setup starting request');
await new Promise(resolve => {
const httpEvent: Subject<any> = new Subject<any>();
let returned = false;
const sub = this.http
.get<any>(URL, {})
.pipe(catchError(this.handleError))
.subscribe(data => httpEvent.next(data));
// Our cheeky method to ensure a timeout
setTimeout(async () => {
if (!returned) {
this.logger.info('[softap-setup] timeout on ' + result.name);
httpEvent.next({});
}
}, 5000);
httpEvent.subscribe(data => {
this.logger.info('[softap-setup] response ', data);
returned = true;
switch (cmd.name) {
case 'scan-ap':
if (Object.prototype.hasOwnProperty.call(data, 'scans') && data.scans.length) {
result.payload = data.scans;
} else {
result.error = true;
}
break;
default:
result.payload = data;
break;
}
httpEvent.complete();
resolve();
});
});
return result;
}
Basically either the response or the timeout flags there has been a result. The handleError function also neatly handles any eventual errors that may come along, ie the host isn't available. You could apply other logic in there or even pass along the HTTPErrorResponse object.
Related
I am having a problem with a Node.js server using Fastify.
At some point during the execution of a request, the server seems to be closing the connection and the client is getting a socket hang up error.
The logic in the server is:
Fastify client calling a service.
Service sending HTTP request using Axios to get certain information. The service implements a retry mechanism and after every retry, it waits for 15 seconds to make sure the information is available.
The code is as follows:
Fastify server:
fastify.post('/request', async (request, reply) => {
try {
const result = await service.performOperation(request.body);
return result;
} catch(error) {
console.error('Error during operation: %s', error.toString());
throw error;
}
})
fastify.addHook('onError', (request, reply, error, done) => {
console.error('onError hook: %o', error);
done();
})
Service:
async function performOperation(request) {
let attempt = 0;
let latestErrorMessage;
while(attempt++ < 5) {
try {
await waitBeforeAttempt();
return await getInfoFromServer(request);
} catch (error) {
latestErrorMessage = getErrorMessage(error);
if (attempt < 5) {
console.log(`Re-attempting after error: ${latestErrorMessage}`);
}
}
}
throw new Error(`Error after 5 attempts. Last error: ${latestErrorMessage}`);
}
function waitBeforeAttempt() {
return new Promise(resolve => setTimeout(resolve, 15000));
}
async function getInfoFromServer(request) {
const response = await axios.post('http://localhost:3000/service', request, {timeout: 120000});
return response.data.toString();
}
The problem is that the server seems to be closing the connection.
According to the logs, this is happening after waiting for 15 seconds and before the call via Axios, before finishing the first attempt.
You can see in the logs that after closing the connection, the logic continues and finishes with all the attempts with no problems whatsoever.
There is nothing in the logs as to why the connection is closed, not even from the Fastify onError hook declared.
Nothing from Axios either. I guess if there were any timeouts that would throw an exception and be logged.
Important note
Noted that connections are not dropped if I change the waitBeforeAttempt implementation to implement a busy wait instead of setTimeout ie:
function waitBeforeAttempt() {
const start = new Date();
let now;
while (true) {
now = new Date();
if (now - start >= 15000) {
break;
}
}
}
Is there anything I'm doing wrong that is causing the connections to be dropped? Perhaps the 15 seconds wait is too high? I have other setTimeout in the code via Puppetter (same implementation as mine) that don't seem to be causing the problem.
Just answering my own question. The problem turned out to be nothing related to the wait or timeouts.
This wasn't happening when the Node.js service was running locally, only happening intermittently when running on Kubernets + Nginx.
Nginx was just restarting without any apparent reason.
Nginx has been updated and the issue is not showing anymore.
I am running a cron job every 5 mins to get data from 3rd party API, It can be N number of request at a time from NodeJS application. Below are the details and code samples:
1> Running cron Job every 5 mins:
const cron = require('node-cron');
const request = require('request');
const otherServices= require('./services/otherServices');
cron.schedule("0 */5 * * * *", function () {
initiateScheduler();
});
2> Get the list of elements for which I want to initiate the request. Can receive N number of elements. I have called request function (getSingleElementUpdate()) in the forEach loop
var initiateScheduler = function () {
//Database call to get elements list
otherServices.moduleName()
.then((arrayList) => {
arrayList.forEach(function (singleElement, index) {
getSingleElementUpdate(singleElement, 1);
}, this);
})
.catch((err) => {
console.log(err);
})
}
3> Start initiating the request for singleElement. Please note I don't need any callback if I received a successful (200) response from the request. I just have to update my database entries on success.
var getSingleElementUpdate = function (singleElement, count) {
var bodyReq = {
"id": singleElement.elem_id
}
var options = {
method: 'POST',
url: 'http://example.url.com',
body: bodyReq,
dataType: 'json',
json: true,
crossDomain: true
};
request(options, function (error, response, body) {
if (error) {
if (count < 3) {
count = count + 1;
initiateScheduler(singleElement, count)
}
} else{
//Request Success
//In this: No callback required
// Just need to update database entries on successful response
}
});
}
I have already checked this:
request-promise: But, I don't need any callback after a successful request. So, I didn't find any advantage of implementing this in my code. Let me know if you have any positive point to add this.
I need your help with the following things:
I have checked the performance when I received 10 elements in arrayList of step 2. Now, the problem is I don't have any clear vision about what will happen when I start receiving 100 and 1000 of elements in step 2. So, I need your help in determining whether I need to update my code for that scenario or not or is there anything I missed out which degrade the performance. Also, How many maximum requests I can make at a time. Any help from you is appreciable.
Thanks!
AFAIK there is no hard limit on a number of request. However, there are (at least) two things to consider: your hardware limits (memory/CPU) and remote server latency (is it able to respond to all requests in 5 mins before the next batch). Without knowing the context it's also impossible to predict what scaling mechanism you might need.
The question is actually more about app architecture and not about some specific piece of code, so you might want to try softwareengineering instead of SO.
I am currently using node-serialport module for serial port communication. I will send a command ATEC and it will respond with ECHO.
However, this process of sending and receiving data is async(after i send the data, i will not know when the data will arrive in the data event), the example code is below:
//Register the data event from the serial port
port.on('data', (data) => {
console.log(data);
});
//Send data using serialport
port.write('ATEC');
Is there anyway I could write it in this way?
//When i send the command, I could receive the data
port.write('ATEC').then((data)=> {
console.log(data);
});
Is this possible to achieve?
In http communication using request client, we could do something like
request.get('http:\\google.com')
.on('response', (res) => {
console.log(res);
});
I want to replicate the same behaviour using serialport
I wrap a promise in the serial data receive
function sendSync(port, src) {
return new Promise((resolve, reject) => {
port.write(src);
port.once('data', (data) => {
resolve(data.toString());
});
port.once('error', (err) => {
reject(err);
});
});
}
Please take note, the event is using once instead of on to prevent event from stacking (please check the comments below for more information - thanks #DKebler for spotting it)
Then, I could write the code in sync as below
sendSync(port, 'AThello\n').then((data) => {
//receive data
});
sendSync(port, 'ATecho\n').then((data) => {
//receive data
});
or I could use a generator, using co package
co(function* () {
const echo = yield sendSync(port, 'echo\n');
const hello = yield sendSync(port, 'hello 123\n');
return [echo, hello]
}).then((result) => {
console.log(result)
}).catch((err) => {
console.error(err);
})
We have a similar problem in a project I'm working on. Needed a synchronous send/receive loop for serial, and the serialport package makes that kinda weird.
Our solution is to make some sort of queue of functions/promises/generators/etc (depends on your architecture) that the serial port "data" event services. Every time you write something, put a function/promise/etc into the queue.
Let's assume you're just throwing functions into the queue. When the "data" event is fired, it sends the currently aggregated receive buffer as a parameter into the first element of the queue, which can see if it contains all of the data it needs, and if so, does something with it, and removes itself from the queue somehow.
This allows you to handle multiple different kinds of architecture (callback/promise/coroutine/etc) with the same basic mechanism.
As an added bonus: If you have full control of both sides of the protocol, you can add a "\n" to the end of those strings and then use serialport's "readline" parser, so you'll only get data events on whole strings. Might make things a bit easier than constantly checking input validity if it comes in pieces.
Update:
And now that code has been finished and tested (see the ET312 module in http://github.com/metafetish/buttshock-js), here's how I do it:
function writeAndExpect(data, length) {
return new Promise((resolve, reject) => {
const buffer = new Buffer(length);
this._port.write(data, (error) => {
if (error) {
reject(error);
return;
}
});
let offset = 0;
let handler = (d) => {
try {
Uint8Array.from(d).forEach(byte => buffer.writeUInt8(byte, offset));
offset += d.length;
} catch (err) {
reject(err);
return;
}
if (offset === length) {
resolve(buffer);
this._port.removeListener("data", handler);
};
};
this._port.on("data", handler);
});
}
The above function takes a list of uint8s, and an expected amount of data to get back, returns a promise. We write the data, and then set ourselves up as the "data" event handler. We use that to read until we get the amount of data we expect, then resolve the promise, remove ourselves as a "data" listener (this is important, otherwise you'll stack handlers!), and finish.
This code is very specific to my needs, and won't handle cases other than very strict send/receive pairs with known parameters, but it might give you an idea to start with.
I'm accessing an API where you authorize first, and you get an access token back, which you use in successive calls. At some point (hours later), the access token expires. So I'm caching the token on the server so any users using the web app will initiate API calls using that token.
But when it expires, I do a recursive call after updating the access token. So, for example (in pseudo-JS):
function getDetails (id) {
data = HTTP.get(url, {params: ...});
if (!data.success) {
updateToken(function () {
return getDetails(id);
});
} else { /*.. we're good*/ }
}
There would also be a recursion depth check in there too. If there's a better way to do this, I'd love to hear it. Basically:
Call API
(failure)
Update token
Call API again
That is exactly how I set it up. However, if you have access to promises (e.g., bluebird or babel.js), it becomes even nicer syntactically. Here's your code rewritten with promises and es6:
function getDetails (id) {
// the return is optional; return the promise if you want to chain to `getDetails`
return HTTP.get(URL, id, params)
.catch((err) => {
if (err.VALIDATION_ERROR) {
return updateToken().then(() => HTTP.get(URL, id, params))
} else {
throw err
}
}).then((yourRequest) => {
// data here
}).catch((err) => {
// handle fatal error
})
}
The best way would be to keep a timer, since you know how long the session lasts for. That was you can preemptively ask for a new token before your current token/session expires. Using this method also ensures that you never have an interruption of service (unless your request for a new token fails of course). Other than that, you seem to have the rest.
Of course, you will still need to implement the code you have shown us to deal with unexpected session expiration (maybe the API owners decide to invalidate all sessions because of a data breach, etc).
So I'm implementing an application that requires messages to be sent to the browser in real time. Currently this is working fine. When I receive a message I an Untyped Actor similar to the clock example.
What my issue is though is I would like to be able to reconnect the web page when the comet socket gets disconnected. Currently in chrome with comet sockets the loading icon continuously spins. Is there a way I can catch a disconnect message for the iframe/comet socket? Or is there something that I can poll in javascript/jquery? So i can then just reload the page?
If you want to reconnect "the web page" (in other words, make your browser send another request to server, with window.location.reload() or some other method), standard play.libs.Comet.onDisconnected handler is of no use to you - its domain is a server-side, not a client-side.
To make your client-side manage possible blackouts by itself, you may need to implement 'heartbeat' scheme. The client application will ping your server when too much time has been passed since the previous message. One possible way to do this:
var CometProcessor = {
processMessage: function(message) {
console.log(message);
},
handleHeartbeatTimeout: function() {
alert('Heartbeat process timeout');
},
handleHeartbeatError: function() {
alert('Heartbeat process error');
},
timeoutPeriod: 10000, // in milliseconds
timeoutId: 0, // will contain an ID of the current 'checking' timeout
checkHandler: null, // will contain a link to XHR object
checkBeat: function() {
// storing the reference to created XHR object:
this.checkHandler = $.ajax({
url: your_server_url,
// set it to the URL of ping script, that will respond instantly
timeout: 1000,
// this is configurable, but obviously it makes little sense setting this param
// higher than `timeoutPeriod`
success: $.proxy(function() {
// so this particular heartbeat request check went through ok,
// but the next may not be so lucky: we need to schedule another check
this.timeoutId = window.setTimeout(
$.proxy(this.checkBeat, this), this.timeoutPeriod);
}, this),
error: $.proxy(function(x, t) {
if (t === 'timeout') {
this.handleHeartbeatTimeout();
}
else {
this.handleHeartbeatError();
}
}, this)
});
},
message: function(message) {
// when we receive a message, link is obviously functioning,
// so we need to stop all the checking procedures
if (this.checkHandler) {
this.checkHandler.abort();
window.clearTimeout(this.timeoutId);
this.checkHandler = null;
}
processMessage(message); // this is where the actual processing takes place
// when we done with processing, we need to setup the heartbeat again:
this.timeoutId = window.setTimeout(
$.proxy(this.checkBeat, this), this.timeoutPeriod);
}
};
Leveraging this object at server-side is quite easy: you just have to replace the line similar to the one in this example...
Ok.stream(events &> Comet(callback = "parent.cometMessage"))
... with this:
Ok.stream(events &> Comet(callback = "parent.CometProcessor.message"))