We Keep Coding

How to handle HTTPs json format errors from APIs in Node.js?

INTRODUCTION
I am implementing a function for making any kind of https request to any endpoint (using the https native module). When I make a request to a specific API I get an error response in JSON format. Like this:
{
"error": {
"code": 404,
"message": "ID not found"
}
}
How can I handle this kind of errors? At a first moment, I supposed that they were handled in
request.on("error", (err) => {
reject(err);
});
HTTPs Request function code
I have comment '<---------' in the relevant parts of the code
const https = require("https");
exports.httpsRequest = function (options, body = null) {
/*
This function is useful for making requests over the HTTPs protocol
*/
return new Promise((resolve, reject) => {
const request = https.request(options, (response) => {
// Get the response content type
const contentType =
response.headers["content-type"] &&
response.headers["content-type"].split(";")[0];
// Cumulate data
let chuncks = [];
response.on("data", (chunck) => {
chuncks.push(chunck);
});
response.on("end", () => {
// Concat all received chunks
let response = Buffer.concat(chuncks);
// Some responses might be in JSON format...
if (contentType === "application/json") {
// Jsonify the response
response = JSON.parse(response);
}
// (For the future) TODO - Check and parse more content types if needed.
// Resolve the promise with the HTTPs response
resolve(response); // <--------- The JSON format error responses are resolved too!!
});
});
// Reject on request error
request.on("error", (err) => {
// <------------- At a first moment, I supposed that all error responses were handled in this part of the code
reject(err);
});
// Write the body
if (body) {
request.write(body);
}
// Close HTTPs connection.
request.end();
});
};
Question
Why the error response is not handled in request.on("error", ...) ?
Thank you. I would appreciate any help or suggestion.

You need to create a different code path for when the content type isn't what you were expecting in which you call reject() and you also need to try/catch around JSON parsing errors so you can properly catch them and reject on them too. You can solve those issues with this code:
exports.httpsRequest = function (options, body = null) {
/*
This function is useful for making requests over the HTTPs protocol
*/
return new Promise((resolve, reject) => {
const request = https.request(options, (response) => {
// Get the response content type
const contentType =
response.headers["content-type"] &&
response.headers["content-type"].split(";")[0];
// Cumulate data
let chuncks = [];
response.on("data", (chunck) => {
chuncks.push(chunck);
});
response.on("end", () => {
// Concat all received chunks
let response = Buffer.concat(chuncks);
// Some responses might be in JSON format...
if (contentType === "application/json") {
try {
// Jsonify the response
response = JSON.parse(response);
resolve(response);
return;
} catch(e) {
reject(e);
return;
}
}
reject(new Error("Not JSON content-type"))
});
});
// Reject on request error
request.on("error", (err) => {
reject(err);
});
// Write the body
if (body) {
request.write(body);
}
// Close HTTPs connection.
request.end();
});
};
FYI, libraries such as got() and others listed here, all do this work for you automatically and have a lot of other useful features. You don't really need to build this yourself.

Calling a Promise function inside Google Cloud Function

I am trying to issue an HTTP request to another web service, from a Google Cloud Function (GCF) that I have created. I need the HTTP request to complete and return that result inside of my GCF so that I can do something else with it.
My question is; What is the best way to use Promise inside a Google Cloud Function? Is what I am trying to do possible?
My code currently looks like this:
export const MyGCF = functions.https.onRequest((request, response) => {
let dayOfTheWeek: any;
const request1 = require('request');
const url = 'http://worldclockapi.com/api/json/pst/now';
function getDay() {
return new Promise((resolve, reject) => {
request1(url, { json: true }, (err: any, res: any, body: any) => {
if (err) {
return reject(err);
}
resolve(body.dayOfTheWeek);
});
});
}
getDay().then((data) => {
dayOfTheWeek = data;
console.log(dayOfTheWeek);
});
});

In general your approach will work, and you can define additional functions inside of your MyGCF handler, in the same way that you have defined getDay(). One problem with you current code however is that you're forgetting to "write a response" for the request being processed by MyGCF.
You can write a response for the request by calling send() on the second res argument of your MyGCF request handler. A simple example would be:
/* Sends a response of "hello" for the request */
res.send("hello");
With respect to your code, you can use res.send() in your .then() callback to send a response back to the client after getDay() has completed (see code below). Note also to include a .catch() clause and callback for the error case (with an error status) to ensure the client receives an appropriate error response if the call to getDay() fails:
export const MyGCF = functions.https.onRequest((req, res) => {
const request = require('request');
const url = 'http://worldclockapi.com/api/json/pst/now';
function getDay() {
return new Promise((resolve, reject) => {
request(url, {
json: true
}, (err: any, r: any, body: any) => {
if (err) {
reject(err);
} else {
resolve(body.dayOfTheWeek);
}
});
});
}
getDay().then((dayOfTheWeek) => {
/* Send a response once the getDay() request complete */
res.send(dayOfTheWeek);
})
.catch(err => {
/* Don't forget the error case */
res.status(500).send(err);
});
});

promise not working for me http.get

I am using a promise to get my response.
but he value is not coming in my console.
can you tell me whats the problem.
providing my code below
const https = require('https');
/*
* Complete the function below.
* Use console.log to print the result, you should not return from the function.
*/
function getMovieTitles(substr) {
return new Promise((resolve,reject) => {
https.get('https://jsonmock.hackerrank.com/api/movies/search/?Title=spiderman&page=1', (res) =>{
// var {statusCode} = res;
//var contentType = res.headers['content-type'];
console.log('res---->', res);
})
})
}

Here you go,
new Promise(function(resolve, reject) {
https.get('your_url_here', (res) =>{
// var {statusCode} = res;
// var contentType = res.headers['content-type'];
resolve(res);
})
}).then(function(res){
console.log("Response",res);
})
To Test Please Follow - https://jsfiddle.net/qdjkreo5/3741/
Alternatives :
You can deal with the data returned by https module in its callback itself as https also returns a callback.
const https = require('https');
https.get('https://encrypted.google.com/', (res) => {
console.log('response', res);
}).on('error', (e) => {
console.error(e);
});
Apart from that To Combine REST API calls with JavaScript Promises in node.js you can make use of the npm module request-promise.
var request = require('request-promise');
request({
"method":"GET",
"uri": "https://api.github.com/",
"json": true,
"headers": {
"User-Agent": "My little demo app"
}
}).then(console.log, console.log);

Building on the prior answers, here's a promise framework that handles both resolve (response, but not necessarily 200) and reject (no response). The axios package does this but hides the details. I like this a little better because it gives me more control over the logic.
const p1 = new Promise((resolve, reject) => {
https.get(url, (res) => {
let {statusCode} = res;
let contentType = res.headers['content-type'];
resolve(`status=${statusCode} type=${contentType}`); // response, so resolve
}).on('error', (err) => {
reject(err); // no response, so reject
});
});
p1.then(res => console.log(`${url}: ${res}`)) // resolve
.catch(err => console.log(err)) // reject
.finally(console.log('Finally')); // always
Test on the following URLs,
https://google.com
https://x.com
http://x.com

Is there some way to make recursive fetch requests?

I want my fetch request to have some sort of retry system if it somehows fails based on the HTTP code of the response (for example: not 200). It looks something like this:
fetch('someURLWithAJSONfile/file.json')
.then(function (res) {
console.log(res.status);
if (res.status !== 200) {
console.log("There was an error processing your fetch request. We are trying again.");
// Recursive call to same fetch request until succeeds
} else {
return res.json();
}
}).then(function (json) {
data = json;
}).catch(function (err) {
console.log(`There was a problem with the fetch operation: ${err.message}`);
});
Is there a way to put the fetch request inside a custom Promise and make it call itself after checking its http response status?

Here the simple ES6 solution (since you are using fetch). The limit option means how many times you want to try your request.
var doRecursiveRequest = (url, limit = Number.MAX_VALUE) =>
fetch(url).then(res => {
if (res.status !== 200 && --limit) {
return doRecursiveRequest(url, limit);
}
return res.json();
});
doRecursiveRequest('someURLWithAJSONfile/file.json', 10)
.then(data => console.log(data))
.catch(error => console.log(error));

You can do this by wrapping your call to fetch in a named function that returns the promise created by fetch. Consider:
function fetchWithRetry(url, retryLimit, retryCount) {
retryLimit = retryLimit || Number.MAX_VALUE;
retryCount = Math.max(retryCount || 0, 0);
return fetch(url).then(function (res) {
console.log(res.status);
if (res.status !== 200 && retryCount < retryLimit) {
console.log("There was an error processing your fetch request. We are trying again.");
return fetchWithRetry(url, retryLimit, retryCount + 1);
} else {
return res.json();
}
});
}
fetchWithRetry('someURLWithAJSONfile/file.json', 10).then(function (json) {
data = json;
}).catch(function (err) {
console.log(`There was a problem with the fetch operation: ${err.message}`);
});
This code wraps your existing call and takes advantage of closure scope to maintain a retry limit and count which are both optional. You then call the fetchWithRetry function with a URL just like you did your previous call to fetch. If you do not pass a retry limit it will continue endlessly. The final retryCount variable is really only used for recursion purposes and is meant to be called internally.

Reading the body returned for fetch() response which contains an HTTP error code [duplicate]

I have an HTTP API that returns JSON data both on success and on failure.
An example failure would look like this:
~ ◆ http get http://localhost:5000/api/isbn/2266202022
HTTP/1.1 400 BAD REQUEST
Content-Length: 171
Content-Type: application/json
Server: TornadoServer/4.0
{
"message": "There was an issue with at least some of the supplied values.",
"payload": {
"isbn": "Could not find match for ISBN."
},
"type": "validation"
}
What I want to achieve in my JavaScript code is something like this:
fetch(url)
.then((resp) => {
if (resp.status >= 200 && resp.status < 300) {
return resp.json();
} else {
// This does not work, since the Promise returned by `json()` is never fulfilled
return Promise.reject(resp.json());
}
})
.catch((error) => {
// Do something with the error object
}

// This does not work, since the Promise returned by `json()` is never fulfilled
return Promise.reject(resp.json());
Well, the resp.json promise will be fulfilled, only Promise.reject doesn't wait for it and immediately rejects with a promise.
I'll assume that you rather want to do the following:
fetch(url).then((resp) => {
let json = resp.json(); // there's always a body
if (resp.status >= 200 && resp.status < 300) {
return json;
} else {
return json.then(Promise.reject.bind(Promise));
}
})
(or, written explicitly)
return json.then(err => {throw err;});

Here's a somewhat cleaner approach that relies on response.ok and makes use of the underlying JSON data instead of the Promise returned by .json().
function myFetchWrapper(url) {
return fetch(url).then(response => {
return response.json().then(json => {
return response.ok ? json : Promise.reject(json);
});
});
}
// This should trigger the .then() with the JSON response,
// since the response is an HTTP 200.
myFetchWrapper('http://api.openweathermap.org/data/2.5/weather?q=Brooklyn,NY').then(console.log.bind(console));
// This should trigger the .catch() with the JSON response,
// since the response is an HTTP 400.
myFetchWrapper('https://content.googleapis.com/youtube/v3/search').catch(console.warn.bind(console));

The solution above from Jeff Posnick is my favourite way of doing it, but the nesting is pretty ugly.
With the newer async/await syntax we can do it in a more synchronous looking way, without the ugly nesting that can quickly become confusing.
async function myFetchWrapper(url) {
const response = await fetch(url);
const json = await response.json();
return response.ok ? json : Promise.reject(json);
}
This works because, an async function always returns a promise and once we have the JSON we can then decide how to return it based on the response status (using response.ok).
You would error handle the same way as you would in Jeff's answer, however you could also use try/catch, an error handling higher order function, or with some modification to prevent the promise rejecting you can use my favourite technique that ensures error handling is enforced as part of the developer experience.
const url = 'http://api.openweathermap.org/data/2.5/weather?q=Brooklyn,NY'
// Example with Promises
myFetchWrapper(url)
.then((res) => ...)
.catch((err) => ...);
// Example with try/catch (presuming wrapped in an async function)
try {
const data = await myFetchWrapper(url);
...
} catch (err) {
throw new Error(err.message);
}
Also worth reading MDN - Checking that the fetch was successful for why we have to do this, essentially a fetch request only rejects with network errors, getting a 404 is not a network error.

I found my solution at MDN:
function fetchAndDecode(url) {
return fetch(url).then(response => {
if(!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
} else {
return response.blob();
}
})
}
let coffee = fetchAndDecode('coffee.jpg');
let tea = fetchAndDecode('tea.jpg');
Promise.any([coffee, tea]).then(value => {
let objectURL = URL.createObjectURL(value);
let image = document.createElement('img');
image.src = objectURL;
document.body.appendChild(image);
})
.catch(e => {
console.log(e.message);
});

Maybe this option can be valid
new Promise((resolve, reject) => {
fetch(url)
.then(async (response) => {
const data = await response.json();
return { statusCode: response.status, body: data };
})
.then((response) => {
if (response.statusCode >= 200 && response.statusCode < 300) {
resolve(response.body);
} else {
reject(response.body);
}
})
});