We Keep Coding

Why am I getting "Cannot access 'server' before initialization" error in NodeJS?

I am getting the dreaded Cannot access 'server' before initialization error in code that is identical to code that's running in production.
The only things that have changed are my OS version (macOS 10.11->10.14) my NodeJS version (10->12) and my VSCode launch.json, but I cannot see anything in either that would cause an issue. My Node version went from 10 to 12, but in production it went from 8 to 15 without issue. I routinely keep launch.json pretty sparse, and the same error happens using node server in Terminal.
Here is the offending code. The issue occurs because I have shutdown() defined before server and it references server. It's written to add an event-handler and then cause the event. Yes, it could be refactored but it already works. It works, really. In 21 instances spread over 7 servers.
I have tried changing the declaraion/init of server from const to var but that does not fix it. As mentioned, this is code that's running in prod! What's wrong with my environment?
Maybe a better question is: why did this ever work?
'use strict'
const fs = require('fs');
const https = require('https');
const cyp = require('crypto').constants;
const stoppable = require('./common/stoppable.js');
const hu = require('./common/hostutil');
process.on('uncaughtException', err => {
wslog.error(`Uncaught Exception: ${err} ${err.stack}`);
shutdown();
});
process.on('unhandledRejection', (reason, p) => {
wslog.error(`Unhandled Promise Rejection: ${reason} - ${p}`);
});
// 'shutdown' is a known static string sent from node-windows wrapper.js if the service is stopped
process.on('message', m => {
if (m == 'shutdown') {
wslog.info(`${wsconfig.appName} has received shutdown message`);
shutdown();
}
});
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
process.on('SIGHUP', shutdown);
function shutdown() {
httpStatus = 503; // Unavailable
wslog.info(`${wsconfig.appName} httpStatus now ${httpStatus} - stopping server...`);
// Error happens on this next line; It should not execute till after server is running already
server.on('close', function () {
wslog.info(`${wsconfig.appName} HTTP server has stopped, now exiting process.`);
process.exit(0)
});
server.stop();
}
// Init and start the web server/listener
var combiCertFile = fs.readFileSync(wsconfig.keyFile, 'utf8');
var certAuthorityFile = fs.readFileSync(wsconfig.caFile, 'utf8');
var serverOptions = {
key: combiCertFile,
cert: combiCertFile,
ca: certAuthorityFile,
passphrase: wsconfig.certPass,
secureOptions: cyp.SSL_OP_NO_TLSv1 | cyp.SSL_OP_NO_TLSv1_1
};
var server = https.createServer(serverOptions, global.app)
.listen(wsconfig.port, function () {
wslog.info(`listening on port ${wsconfig.port}.`);
});
server.on('clientError', (err, socket) => {
if (err.code === 'ECONNRESET' || !socket.writable) { return; }
// ECONNRESET was already logged in socket.on.error. Here, we log others.
wslog.warn(`Client error: ${err} ${err.stack}`);
socket.end('HTTP/1.1 400 Bad Request\r\n\r\n');
});
server.on('error', (err)=>{
if ( err.code === 'EADDRINUSE' ) {
wslog.error(`${err.code} FATAL - Another ${wsconfig.appName} or app is using my port! ${wsconfig.port}`);
} else {
wslog.error(`${err.code} FATAL - Server error: ${err.stack}`);
}
shutdown();
})
combiCertFile = null;
certAuthorityFile = null;
// Post-instantiation configuration required (may differ between apps: need an indirect way to plug in app-specific behavior)
stoppable(server, wsconfig.stopTimeout);
// Load all RESTful endpoints
const routes = require('./routes/');

This is a runtime error, which happens only in a very specific situation. But actually this exact error shouldn't happen with var server = ... but only with const server = ... or let server = .... With var server = ... the error message should say "Cannot read properties of undefined"
What happens
You have an error handler for uncaughtException which is calling shutdown() and in shutdown() you are referencing your server. But consider what happens if your code throws an exception before you initialized your server. For instance if your cert or key cannot be read from the disk, cert or key are invalid ... So nothing will be assigned to server, and an exception will be raised.
Then the handler for your uncaught exception will fire and call the shutdown() function, which then tries to access the server, which of course hasn't been initialized yet.
How to fix
Check what the unhandled exception is, that is thrown before your server is initialized and fix it. In your production environment, there is probably no exception, because the configuration and environment is properly set up. But there is at least one issue in your develepment environment, which causes an exception.
Difference between var and const
And the difference between var server = ... and const server = ... is quite a subtle one. For both, the declaration of the variable is hoisted up to the top of their respective scope. In your case it's always global, also for const. But variables declared as var are assigned a value of undefined whereas variables declared as let/const are not initialized at all.
You can easily reproduce this error if you uncomment either error1 or error2 in the following code. But error3 alone won't produce this ReferenceError because bar will already be initialized. You can also replace const bar = with var bar = and you will see, that you get a different error message.
process.on("uncaughtException", err => {
console.log("uncaught exception");
console.log(err);
foo();
});
function foo() {
console.log("foo");
console.log(bar.name);
}
function init() {
// throw new Error("error1");
return { name: "foobar"}
}
// throw new Error("error2");
const bar = init();
//throw new Error("error3");

Node HTTP request hangs forever

We've got a Node.js script that is run once a minute to check the status of our apps. Usually, it works just fine. If the service is up, it exits with 0. If it's down, it exits with 1. All is well.
But every once in a while, it just kinda stops. The console reports "Calling status API..." and stops there indefinitely. It doesn't even timeout at Node's built-in two-minute timeout. No errors, nothing. It just sits there, waiting, forever. This is a problem, because it blocks following status check jobs from running.
At this point, my whole team has looked at it and none of us can figure out what circumstance could make it hang. We've built in a start-to-finish timeout, so that we can move on to the next job, but that essentially skips a status check and creates blind spots. So, I open the question to you fine folks.
Here's the script (with names/urls removed):
#!/usr/bin/env node
// SETTINGS: -------------------------------------------------------------------------------------------------
/** URL to contact for status information. */
const STATUS_API = process.env.STATUS_API;
/** Number of attempts to make before reporting as a failure. */
const ATTEMPT_LIMIT = 3;
/** Amount of time to wait before starting another attempt, in milliseconds. */
const ATTEMPT_DELAY = 5000;
// RUNTIME: --------------------------------------------------------------------------------------------------
const URL = require('url');
const https = require('https');
// Make the first attempt.
make_attempt(1, STATUS_API);
// FUNCTIONS: ------------------------------------------------------------------------------------------------
function make_attempt(attempt_number, url) {
console.log('\n\nCONNECTION ATTEMPT:', attempt_number);
check_status(url, function (success) {
console.log('\nAttempt', success ? 'PASSED' : 'FAILED');
// If this attempt succeeded, report success.
if (success) {
console.log('\nSTATUS CHECK PASSED after', attempt_number, 'attempt(s).');
process.exit(0);
}
// Otherwise, if we have additional attempts, try again.
else if (attempt_number < ATTEMPT_LIMIT) {
setTimeout(make_attempt.bind(null, attempt_number + 1, url), ATTEMPT_DELAY);
}
// Otherwise, we're out of attempts. Report failure.
else {
console.log("\nSTATUS CHECK FAILED");
process.exit(1);
}
})
}
function check_status(url, callback) {
var handle_error = function (error) {
console.log("\tFailed.\n");
console.log('\t' + error.toString().replace(/\n\r?/g, '\n\t'));
callback(false);
};
console.log("\tCalling status API...");
try {
var options = URL.parse(url);
options.timeout = 20000;
https.get(options, function (response) {
var body = '';
response.setEncoding('utf8');
response.on('data', function (data) {body += data;});
response.on('end', function () {
console.log("\tConnected.\n");
try {
var parsed = JSON.parse(body);
if ((!parsed.started || !parsed.uptime)) {
console.log('\tReceived unexpected JSON response:');
console.log('\t\t' + JSON.stringify(parsed, null, 1).replace(/\n\r?/g, '\n\t\t'));
callback(false);
}
else {
console.log('\tReceived status details from API:');
console.log('\t\tServer started:', parsed.started);
console.log('\t\tServer uptime:', parsed.uptime);
callback(true);
}
}
catch (error) {
console.log('\tReceived unexpected non-JSON response:');
console.log('\t\t' + body.trim().replace(/\n\r?/g, '\n\t\t'));
callback(false);
}
});
}).on('error', handle_error);
}
catch (error) {
handle_error(error);
}
}
If any of you can see any places where this could possibly hang without output or timeout, that'd be very helpful!
Thank you,
James Tanner
EDIT: p.s. We use https directly, instead of request so that we don't need to do any installation when the script runs. This is because the script can run on any build machine assigned to Jenkins without a custom installation.

Aren't you missing the .end()?
http.request(options, callback).end()
Something like explained here.

Inside your response callback your not checking the status..
The .on('error', handle_error); is for errors that occur connecting to the server, status code errors are those that the server responds with after a successful connection.
Normally a 200 status response is what you would expect from a successful request..
So a small mod to your http.get to handle this should do..
eg.
https.get(options, function (response) {
if (response.statusCode != 200) {
console.log('\tHTTP statusCode not 200:');
callback(false);
return; //no point going any further
}
....

Parse SDK JavaScript with Node.js ENOTFOUND

So I want to fetch a large amount of data on Parse, a good solution I found is to make a recursive function: when the data are successfully found, launch another request. The way I'm doing it is pretty simple:
var containedLimit = 1000, // Also tried with 500, 300, and 100
parseUsersWaiting = {
// A lot of Users
},
parseUsers = {}, // Recipt for Users
getPlayers = function (containedIn) {
var count = 0;
if (containedIn == undefined) {
containedIn = [];
for (var i in parseUsersWaiting) {
count++;
if (count > containedLimit) {
break;
}
containedIn.push(parseUsersWaiting[i].id);
delete parseUsersWaiting[i];
}
}
var UserObject = Parse.Object.extend('User'),
UserQuery = new Parse.Query(UserObject);
UserQuery.limit(containedLimit);
UserQuery.containedIn('objectId', containedIn);
UserQuery.find({
success: function (results) {
if (results) {
for (var i in results) {
if (parseUsers[results[i].id] == undefined) {
parseUsers[results[i].id] = results[i];
}
// Other stuff
if (results.length < containedLimit) {
// End of process
} else {
getPlayers();
}
}
} else {
// Looks like an end of process too
}
}, error: function (error) {
console.log(error.message);
getPlayers(containedIn);
}
});
};
Now, here is what I would like to call the "issue": it happen, very frequently, that the "error" callback is launched with this:
Received an error with invalid JSON from Parse: Error: getaddrinfo ENOTFOUND api.parse.com
at errnoException (dns.js:44:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26)
(With code 107, of course) So I searched on the Parse Documentation, and it says the exact same thing: "Received an error with invalid JSON". Yeah.
I'm using the Parse SDK provided by Parse.com (npm install parse)
I also tried to modify a bit the Parse code with replacing the host key by 'hostname' on line 361 of the file parse/node_modules/xmlhttprequest/lib/XMLHttpRequest.js (Parse package version: 1.5.0), and it didn't worked, so I removed my changes.
(By the way, I found a solution talking about using ulimit to change memory usage limit that could solve the problem, but actually, I haven't the necessary rights to execute this command)

This error occurs when it can not connect to the API (technically when it cannot lookup the IP address of the API server). Perhaps your internet connection was lost for a brief moment or their SDK server were unavailable (or maybe the server is denying your request due to rate limits).
Either way it is good to code some resilience into your application. I see your on error function retries the API call, but perhaps it would be worth adding a timeout before you do that to give the problem a chance to recover?
error: function (error) {
console.log(error.message);
setTimeout(getPlayers, 10000, containedIn);
}
If the sever is rate limiting your requests, this will also help.

In Node, what is a good way to create a http connection with a 'wait for connection' timeout

From a unit test I need to create an http connection (to a Chrome's --remote-debugging-port) and need a way to wait for the http based json service to be available.
I was not able to find an npm package that supported that case, the http://nodejs.org/api/http.html#http_http_request_options_callback doesn't seem to support connection timeouts, and the couple examples I saw where based on:
trying to make a connection to the target url
capture req.on('error', function (err) {
check for err.code === 'ECONNREFUSED'
try again for x seconds at a y ms interval
Is that the best way to do it?
Update 1: setTimeout doesn't work because we get an exception if the target URL is not available
it 'connect_To_Chrome', (done)->
url_Debug = "http://localhost:#{nodeWebKit.port_Debug}/json"
url_Debug.GET (data)->
assert_Is_Null(data)
http = require('http')
# throws error if the value is less that 60 (i.e. 60ms)
(100).eval_After -> # setTimeout
socket = http.get url_Debug, -> # create socket
console.log 'inside get' # socket was created
socket.setTimeout 400, -> # try setTimeout
console.log 'inside timeout' # trigered after 400ms
done()
in the code sample shown above, if the initial timeout is for example 10 (or not used at all), we will get the following exceptions
Error: socket hang up
at createHangUpError (http.js:1476:15)
at Socket.socketCloseListener (http.js:1526:23)
at Socket.emit (events.js:117:20)
at TCP.close (net.js:465:12)
Error: connect ECONNREFUSED
at errnoException (net.js:904:11)
at Object.afterConnect [as oncomplete] (net.js:895:19)
update 2: here is a fist pass at a solution
it 'get Chrome Remote Debugging /json', (done)->
wait_For_Http_GET = (url, timeout, callback)->
delay = 10;
try_Http_Get = (next) =>
url.GET (data) => if data is null then (delay).invoke_After next else callback(data)
run_Tests = (test_Count)=> if test_Count.empty() then url.GET (callback) else try_Http_Get ()->run_Tests(test_Count.splice(0,1))
run_Tests([0.. ~~(timeout/delay)])
url_Debug = "http://localhost:#{nodeWebKit.port_Debug}/json"
url_Debug.GET (data)->
assert_Is_Null(data)
wait_For_Http_GET url_Debug,100, (html)->
data = JSON.parse(html)
data.assert_Is_Array().assert_Size_Is(1)
data[0].description .assert_Is('')
data[0].devtoolsFrontendUrl .assert_Is("/devtools/devtools.html?ws=localhost:#{nodeWebKit.port_Debug}/devtools/page/#{data[0].id}")
data[0].id .split('-').assert_Size_Is(5)
data[0].title .assert_Is('')
data[0].type .assert_Is('page')
data[0].url .assert_Is('nw:blank')
data[0].webSocketDebuggerUrl.assert_Is("ws://localhost:#{nodeWebKit.port_Debug}/devtools/page/#{data[0].id}")
done()

There is request.setTimeout() which allows you to set an idle timeout that fires if no data is sent or received on the underlying socket within the period of time you specify. This should also work before a connection is made.
You will also need to listen for the error event on the request object in case the connection is rejected sooner.

I'm going to choose my update 2 answer since it does works as I expected and answers the question
wait_For_Http_GET = (url, timeout, callback)->
delay = 10;
try_Http_Get = (next) =>
url.GET (data) => if data is null then (delay).invoke_After next else callback(data)
run_Tests = (test_Count)=> if test_Count.empty() then url.GET (callback) else try_Http_Get ()->run_Tests(test_Count.splice(0,1))
run_Tests([0.. ~~(timeout/delay)])
test
it 'get Chrome Remote Debugging /json', (done)->
url_Debug = "http://localhost:#{nodeWebKit.port_Debug}/json"
url_Debug.GET (data)->
assert_Is_Null(data)
wait_For_Http_GET url_Debug,100, (html)->
data = JSON.parse(html)
data.assert_Is_Array().assert_Size_Is(1)
data[0].description .assert_Is('')
data[0].devtoolsFrontendUrl .assert_Is("/devtools/devtools.html?ws=localhost:#{nodeWebKit.port_Debug}/devtools/page/#{data[0].id}")
data[0].id .split('-').assert_Size_Is(5)
data[0].title .assert_Is('')
data[0].type .assert_Is('page')
data[0].url .assert_Is('nw:blank')
data[0].webSocketDebuggerUrl.assert_Is("ws://localhost:#{nodeWebKit.port_Debug}/devtools/page/#{data[0].id}")
done()

Reference error is not thrown from MongoDB callback

Introduction
All people know that if we call undefined.test we will receive the following error (same for both: NodeJS and Javascript):
$ node
> undefined.test
TypeError: Cannot read property 'test' of undefined
at repl:1:11
at REPLServer.self.eval (repl.js:110:21)
at Interface.<anonymous> (repl.js:239:12)
at Interface.EventEmitter.emit (events.js:95:17)
at Interface._onLine (readline.js:202:10)
at Interface._line (readline.js:531:8)
at Interface._ttyWrite (readline.js:760:14)
at ReadStream.onkeypress (readline.js:99:10)
at ReadStream.EventEmitter.emit (events.js:98:17)
at emitKey (readline.js:1095:12)
That's correct!
How did I find the problem?
Passed week I wasted about 30 minutes in debugging the following problem: A script was stopping accidentally and no error was thrown.
I had the urls variable that was supposed to be an object:
var urls = settings.urls || {};
Then in next lines I needed to get shop key of urls that was a string:
var shop = urls.shop || "/";
I started adding console.log to find the values of variables:
console.log(urls); // undefined
var shop = urls.shop || "/";
console.log("Passed"); // This isn't run.
The problem in my script was that I was redefining a new urls variable that was making the urls undefined, but the question is: why cannot read property "shop" of undefined didn't appear here? Because urls was really undefined.
We know that the following is happening in both: NodeJS and Javascript:
var a = 10;
foo(function () {
console.log(a); // undefined
var a = 10;
});
function foo(callback) { callback(); }
The question
After debugging the problem I found that this problem comes from Mongo: inside of Mongo callbacks if we call undefined.something we DON'T get the error.
I've created a small script that demonstrates this:
var mongo = require("mongodb");
// Mongo server
var server = mongo.Server("127.0.0.1", 27017);
var db = new mongo.Db("test", server, { safe: true });
console.log("> START");
// Open database
console.log("Opening database.");
db.open(function(err, db) {
if (err) { return console.log("Cannot open database."); }
// get collection
console.log("No error. Opening collection.");
db.collection("col_one", function(err, collection) {
if(err) { return console.log(err) }
// do something with the collection
console.log("No error. Finding all items from collection.");
collection.find().toArray(function(err, items) {
if(err) { return console.log(err) }
console.log("No error. Items: ", items);
console.log("The following line is: undefined.shop." +
"It will stop the process.");
console.log(undefined.test); // THE ERROR DOES NOT APPEAR!
console.log("> STOP"); // this message doesn't appear.
});
});
});
My questions are:
Why the error doesn't appear? Which is the reason? (It would be great to debug together the MongoDB source code to find it.)
Why the process is stopped when calling undefined.something?
How can be this solved?
I've created a Github repository where you can download my small application that demonstrates the issue.
Interesting:
If we add a try {} catch (e) {} statement we find the error and the process continue showing the STOP message.
try {
console.log(undefined.test);
} catch (e) {
console.log(e);
}
LOGS:
> START
Opening database.
No error. Opening collection.
No error. Finding all items from collection.
No error. Items: []
The following line is: undefined.shop. It will stop the process.
[TypeError: Cannot read property 'test' of undefined]
> STOP

Looking on github.com at node-mongodb-native driver issues, you will notice that issue is solved in 1.3.18 version. But, I tested it and it does not work as expected.