We Keep Coding

Node JS Express how can I send a 404 error if a bad request is made to third party API?

In my Node JS server I have this route handler that sends a request to a third party API to get a username:
app.get('/players/:player', apiLimiter, function(request, response) {
const player = request.params.player;
const api_url = `https://api.com/shards/steam/players?filter[playerNames]=${player}`;
var options = {
method: "GET",
observe: 'body',
};
let apiRequest = https.request(api_url, options, function (res) {
let data = "";
res.on("data", chunk => {
data += chunk;
})
res.on("end", () => {
let objectParsed = JSON.parse(JSON.stringify(data));
response.send(objectParsed);
})
if(!player) {
res.status(404).send("Not found.");
}
})
apiRequest.end();
})
This works fine to get a user that exists. However, if I put in a fake username to my /players page, that page still loads with a 200 status instead of getting a 404 response. The page loads and looks broken because it's not actually getting any data from the API.
I feel like this is a dumb question .. In my research I have found how to handle errors if it's just the route, and not if it's the route dependent on the path parameter as in /players/:player
I found a question that was similar to mine (How to throw a 404 error in express.js?) and I tried using an If statement: if (!player){res.status(404).send("Not found."); } but no dice. Am I using this if statement in the wrong place?
How can I get my Node JS server to respond with a 404 if the user from the database doesn't exist?

You have to check the result of the API call and see if you got valid data back and send the 404 there. I also added a check to make sure something was passed for the player name and send back a 400 (bad request) if there's no player specified at all:
app.get('/players/:player', apiLimiter, function(request, response) {
const player = request.params.player;
if (!player) {
res.status(400).send("No player specified.");
return;
}
const api_url = `https://api.com/shards/steam/players?filter[playerNames]=${player}`;
var options = {
method: "GET",
observe: 'body',
};
let apiRequest = https.request(api_url, options, function(res) {
let data = "";
res.on("data", chunk => {
data += chunk;
})
res.on("end", () => {
let objectParsed = JSON.parse(data);
// test objectParsed here
if (!some condition in objectParsed) {
res.status(404).send("No data for that player name.");
} else {
response.send(objectParsed);
}
});
});
apiRequest.end();
});
Also, you don't want JSON.parse(JSON.stringify(data)) here. Your data is already a string. Just do JSON.parse(data).
FYI, if you use a small http request library such as got(), this code gets a lot simpler as it accumulates the response and parses the JSON for you in one line of code as in:
let data = await got(options).json()

Log images after successful http request using Node.js

I'm creating a script that will make a request 2 times per second to a localserver of cameras network and after it gets a positive response that camera detected someone I want to log three images.
In the json config file I have the triggerURL of the server, the interval port, the dataDir where logged images should be saved and a track array which contains the url of those images and the fileName they should receive.
This is the code of the script I use after reading the JSON file:
var configGet = {
host: config.triggerURL
, port: config.interval
, method: 'GET'
};
setInterval(function () {
var request = http.request(configGet, function (response) {
var content = "";
// Handle data chunks
response.on('data', function (chunk) {
content += chunk;
});
// Once we're done streaming the response, parse it as json.
response.on('end', function () {
var data = JSON.parse(response);
if (data.track.length > 0) {
//log images
var download = function (uri, filename, callback) {
request.head(uri, function (err, res, body) {
request(uri)
.pipe(fs.createWriteStream(filename))
.on('close', callback);
});
};
for (var image in data.track) {
var path = config.dataDir + '/' + image.fileName
download(image.url, path.format(config.timestamp), function () {
console.log('done');
});
}
}
});
// Report errors
request.on('error', function (error) {
console.log("Error while calling endpoint.", error);
});
request.end();
}, 500);
});
I have the following questions:
This method produces some kind of error with the download process of the images.Can you identify it?
Is there a better way of doing this process?

Without running the code or deeper inspection; should not "data = JSON.parse(response)" rather be "data = JSON.parse(content)"? Also, if data is undefined or does not contain "track" the "if (data.track.length > 0)" will throw an error. This can be fixed with "if (data && data.track && data.track.length > 0)".
I can not think of a very much better way. I would break it up more in functions to make the code more clear though.

Getting rid of irrelevant console errors (404 (Not Found))

I have a simple program that is scraping a web site for some items. I am using Angular $http service to call the below C# method to get the markup from the page and then handling everything else with JS. Everything is working perfectly fine with the exception of a minor annoyance: a bunch of 404 errors.
The 404 errors are being displayed in the developer tools once the http get call completes. It's almost like the javascript is trying to interpret the HTML and then fails on all the get requests for the images in the browser:
What I'm trying to figure out is how to get the 404 errors to go away or fail silently (not display in the console). I'm not finding anything in my research but am assuming there is some way to handle this whether it be on the server or client side
C#
public static string GetPageSource()
{
JObject result = new JObject();
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://awebpage.html");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
result["data"] = reader.ReadToEnd();
result["success"] = true;
reader.Close();
response.Close();
}
catch (Exception ex)
{
result["data"] = ex.Message;
result["success"] = false;
}
return JsonConvert.SerializeObject(result);
}
JS
$scope.getPageSource = function () {
var ajaxProcessor = Utils.ajaxMessage('Scraping Beer Menu From Source');
ajaxProcessor.start();
$http({
method: 'POST',
url: 'AJAX/MenuHandler.aspx/GetPageSource',
contentType: 'application/json; charset=utf-8',
dataType: 'json',
data: {}
}).then(function (response) {
ajaxProcessor.stop();
var result = $.parseJSON(response.data.d);
if (result.success === false) {
Utils.showMessage('error', result.data);
} else {
var beerMenu = new BeerMenu(result.data, $scope.loggedInUser, function (beerMenu) {
$scope.buildDisplayMenu(beerMenu);
});
}
}, function (err) {
ajaxProcessor.stop();
console.log(err);
Utils.showMessage('error', err.data.Message);
});
};
UPDATE
Thanks to #dandavis, my issue is narrowed down to calling $.parseHTML within the buildDisplayMenu function (which calls buildCurrentMenu). Is there anyway to make it ignore the images or any get request?
buildCurrentMenu: function () {
var html = $.parseHTML(this.pageSource);
var menuDiv = $(html).find('.TabbedPanelsContent')[0];
var categories = $(menuDiv).find('h2');
var categegoryItems = [];
var beerArray = [];
for (var i = 0; i < categories.length; i++) {
...
}
return beerArray;
}

The resolution is to remove any img tags (or any other tag that should be ignored) from the page source before calling $.parseHTML
this.pageSource = this.pageSource.replace(/<img[^>]*>/g, "");

How do I capture an aborted call or is setting the timeout to 0 correct?

I have a JavaScript client that works in Chrome and Firefox, but fails in IE. Looking at the network trace in the IE debugger it shows that multiple of the AJAX calls have been aborted.
I've been able to get around it by setting the timeout to 0. I'd like to know if this is the correct way to handle my requests being aborted? Basically what could go wrong?
My initial thought was that I should capture and resend on error, and if multiple resubmits do not result in a completed request, finally alert the user. I'd still like to know how to do this even if the setTimeout is the proper way to address my immediate issue.
Also the application will process an excel workbook of addresses, call a web service to add some data to them and then allow the user to download the enhanced file.
This is what I have so far, first in the app.js
var requestWithFeedback = function (args) {
$(".loader").removeClass('hidden');
var oldConfig = args.config || function () { };
args.config = function (xhr) {
xhr.setRequestHeader("Authorization", "Bearer " + localStorage.token);
oldConfig(xhr);
extract: extract;
};
var deferred = m.deferred();
setTimeout(function () { // <== This solved in IE, but is this the way to handle this?
m.request(args).then(deferred.resolve, function(err){
if (err === "Invalid token!"){
m.route('/');
}
})}, 0);
$(".loader").addClass('hidden');
return deferred.promise;
}
From the model.js
app.MarkedAddresses.ProcessAddressBatch = function () {
var requestData = {
Addresses: app.MarkedAddresses.vm.addresses
}
return requestWithFeedback({
method: "POST"
, url: "API/server.ashx"
, data: requestData
, deserialize: function (value) { return value; }
})
.then(function (value) {
var responseJSON = $.parseJSON(value);
$.merge(app.MarkedAddresses.vm.results, responseJSON)
app.MarkedAddresses.vm.currentRecord(app.MarkedAddresses.vm.results.length);
app.MarkedAddresses.vm.progress(Math.max(app.MarkedAddresses.vm.progress(), ~~(app.MarkedAddresses.vm.currentRecord() / app.MarkedAddresses.vm.totalRecords() * 100)));
m.redraw(); //Force redraw for progress bar
return value;
},
function (error) { console.log(error) } // <== I thought error would show up here, but I never hit a breakpoint here.
);
}
Added loops
function process_wb(wb) {
app.MarkedAddresses.vm.results.length = 0;
$('.descending').removeClass("descending");
$('.ascending').removeClass("ascending");
app.MarkedAddresses.vm.progress(.1);
m.redraw();
var header = mapHeader(wb);
var addressJSON = to_json(wb, header);
app.MarkedAddresses.vm.totalRecords(addressJSON.length);
for (var i = 0; (i < addressJSON.length + 1) ; i += 1000) {
app.MarkedAddresses.vm.addresses = addressJSON.slice(i, Math.min(((i) + 1000), addressJSON.length));
app.MarkedAddresses.vm.response(new app.MarkedAddresses.vm.processAddressBatch());
}
}
Why isn't the error triggered in the section of the code?
It seems like I should add a deferred section here, but anything I've tried has been a syntax error.

Parse Resize Image not Saving Properly

I am trying to resize an image in Parse Cloud Code following the Cloud Modules Guide.
The basic idea is as follows: when afterSave is called on a User, check if the small profile pic is null and standard profile pic is not null. If true, get the standard profile pic from Parse, read into buffer, create file, save file, then add file to User and Save. Unfortunately, the file doesn't seem to be saving properly.
Here is the cloud afterSave function:
Parse.Cloud.afterSave(Parse.User, function(request) {
...
Parse.Cloud.httpRequest({
url: request.object.get("profilePicture").url(),
success: function(response) {
// The file contents are in response.buffer.
var image = new Image();
console.log("Buffer: " + response.buffer );
console.log("Length " + response.buffer.length);
image.setData(response.buffer);
var imgData = image.data();
// Save the image into a new file.
var base64 = imgData.toString("base64");
var scaled = new Parse.File("thumbnail.png", { base64: base64 });
scaled.save().then(function() {
request.object.set("profilePictureSmall", scaled);
request.object.save();
}, function(error) {
console.log( "The file either could not be read, or could not be saved to Parse.");
});
}
});
...
});
The User object seems to save fine, but the image file that is saved is a broken image.
The strange thing is that console.log("Length " + response.buffer.length); outputs the proper size to the console.
console.log("Buffer: " + response.buffer ); Gives output: �PNG
Any idea what's going on here?

The issue is that setData() is an async call, you need to wait for it to finish before you do the next bit.
http://parse.com/docs/js/symbols/Image.html#setData
Here's a snippet:
Parse.Cloud.httpRequest({
url: request.object.get("profilePicture").url()
}).then(function(response) {
var image = new Image();
return image.setData(response.buffer);
}).then(function(image) {
// make it fit in 100x100
var width = 100, height = 100;
if (image.width() > image.height()) {
// work out scaled height
height = image.height() * (100/image.width());
} else {
// work out scaled width
width = image.width() * (100/image.height());
}
console.log("..scale to " + width + "x" + height);
return image.scale({
width: width,
height: height
});
}).then(function(scaledImage) {
// get the image data in a Buffer
return scaledImage.data();
}).then(function(buffer) {
// save the image to a new file
console.log("..saving file");
var base64 = buffer.toString("base64");
var name = "Thumbnail.png";
var thumbnail = new Parse.File(name, { base64: base64 });
return thumbnail.save();
}).then(function(thumbnail) {
// attach the image file to the original object
console.log("..attaching file");
request.object.set("profilePictureSmall", thumbnail);
return request.object.save();
}) /* further chaining here for after-save */;
You basically have to chain your promises together to allow the previous step to finish.