We Keep Coding

Why am I getting "Cannot access 'server' before initialization" error in NodeJS?

I am getting the dreaded Cannot access 'server' before initialization error in code that is identical to code that's running in production.
The only things that have changed are my OS version (macOS 10.11->10.14) my NodeJS version (10->12) and my VSCode launch.json, but I cannot see anything in either that would cause an issue. My Node version went from 10 to 12, but in production it went from 8 to 15 without issue. I routinely keep launch.json pretty sparse, and the same error happens using node server in Terminal.
Here is the offending code. The issue occurs because I have shutdown() defined before server and it references server. It's written to add an event-handler and then cause the event. Yes, it could be refactored but it already works. It works, really. In 21 instances spread over 7 servers.
I have tried changing the declaraion/init of server from const to var but that does not fix it. As mentioned, this is code that's running in prod! What's wrong with my environment?
Maybe a better question is: why did this ever work?
'use strict'
const fs = require('fs');
const https = require('https');
const cyp = require('crypto').constants;
const stoppable = require('./common/stoppable.js');
const hu = require('./common/hostutil');
process.on('uncaughtException', err => {
wslog.error(`Uncaught Exception: ${err} ${err.stack}`);
shutdown();
});
process.on('unhandledRejection', (reason, p) => {
wslog.error(`Unhandled Promise Rejection: ${reason} - ${p}`);
});
// 'shutdown' is a known static string sent from node-windows wrapper.js if the service is stopped
process.on('message', m => {
if (m == 'shutdown') {
wslog.info(`${wsconfig.appName} has received shutdown message`);
shutdown();
}
});
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
process.on('SIGHUP', shutdown);
function shutdown() {
httpStatus = 503; // Unavailable
wslog.info(`${wsconfig.appName} httpStatus now ${httpStatus} - stopping server...`);
// Error happens on this next line; It should not execute till after server is running already
server.on('close', function () {
wslog.info(`${wsconfig.appName} HTTP server has stopped, now exiting process.`);
process.exit(0)
});
server.stop();
}
// Init and start the web server/listener
var combiCertFile = fs.readFileSync(wsconfig.keyFile, 'utf8');
var certAuthorityFile = fs.readFileSync(wsconfig.caFile, 'utf8');
var serverOptions = {
key: combiCertFile,
cert: combiCertFile,
ca: certAuthorityFile,
passphrase: wsconfig.certPass,
secureOptions: cyp.SSL_OP_NO_TLSv1 | cyp.SSL_OP_NO_TLSv1_1
};
var server = https.createServer(serverOptions, global.app)
.listen(wsconfig.port, function () {
wslog.info(`listening on port ${wsconfig.port}.`);
});
server.on('clientError', (err, socket) => {
if (err.code === 'ECONNRESET' || !socket.writable) { return; }
// ECONNRESET was already logged in socket.on.error. Here, we log others.
wslog.warn(`Client error: ${err} ${err.stack}`);
socket.end('HTTP/1.1 400 Bad Request\r\n\r\n');
});
server.on('error', (err)=>{
if ( err.code === 'EADDRINUSE' ) {
wslog.error(`${err.code} FATAL - Another ${wsconfig.appName} or app is using my port! ${wsconfig.port}`);
} else {
wslog.error(`${err.code} FATAL - Server error: ${err.stack}`);
}
shutdown();
})
combiCertFile = null;
certAuthorityFile = null;
// Post-instantiation configuration required (may differ between apps: need an indirect way to plug in app-specific behavior)
stoppable(server, wsconfig.stopTimeout);
// Load all RESTful endpoints
const routes = require('./routes/');

This is a runtime error, which happens only in a very specific situation. But actually this exact error shouldn't happen with var server = ... but only with const server = ... or let server = .... With var server = ... the error message should say "Cannot read properties of undefined"
What happens
You have an error handler for uncaughtException which is calling shutdown() and in shutdown() you are referencing your server. But consider what happens if your code throws an exception before you initialized your server. For instance if your cert or key cannot be read from the disk, cert or key are invalid ... So nothing will be assigned to server, and an exception will be raised.
Then the handler for your uncaught exception will fire and call the shutdown() function, which then tries to access the server, which of course hasn't been initialized yet.
How to fix
Check what the unhandled exception is, that is thrown before your server is initialized and fix it. In your production environment, there is probably no exception, because the configuration and environment is properly set up. But there is at least one issue in your develepment environment, which causes an exception.
Difference between var and const
And the difference between var server = ... and const server = ... is quite a subtle one. For both, the declaration of the variable is hoisted up to the top of their respective scope. In your case it's always global, also for const. But variables declared as var are assigned a value of undefined whereas variables declared as let/const are not initialized at all.
You can easily reproduce this error if you uncomment either error1 or error2 in the following code. But error3 alone won't produce this ReferenceError because bar will already be initialized. You can also replace const bar = with var bar = and you will see, that you get a different error message.
process.on("uncaughtException", err => {
console.log("uncaught exception");
console.log(err);
foo();
});
function foo() {
console.log("foo");
console.log(bar.name);
}
function init() {
// throw new Error("error1");
return { name: "foobar"}
}
// throw new Error("error2");
const bar = init();
//throw new Error("error3");

Intercept WebSocket messages

With ajax requests it can be done with this code:
let oldXHROpen = window.XMLHttpRequest.prototype.open;
window.lastXhr = '';
window.XMLHttpRequest.prototype.open = function(method, url, async, user, password) {
this.addEventListener('load', function() {
window.lastXhr = this.responseText;
});
return oldXHROpen.apply(this, arguments);
};
lastXhr variable will hold the last response.
But how can this be achieved for websockets too?

you would need to make this wrapper as soon as possible
#brunoff you're correct in that you can always use your functions before a server's by puppet window logic, or you could just hijack the data from the MessageEvent itself:
function listen(fn){
fn = fn || console.log;
let property = Object.getOwnPropertyDescriptor(MessageEvent.prototype, "data");
const data = property.get;
// wrapper that replaces getter
function lookAtMessage() {
let socket = this.currentTarget instanceof WebSocket;
if (!socket) {
return data.call(this);
}
let msg = data.call(this);
Object.defineProperty(this, "data", { value: msg } ); //anti-loop
fn({ data: msg, socket:this.currentTarget, event:this });
return msg;
}
property.get = lookAtMessage;
Object.defineProperty(MessageEvent.prototype, "data", property);
}
listen( ({data}) => console.log(data))
You can try putting in the code and running it in the console on this page and then running their WebSocket example.

To intercept the messages, you will have to spy on the onmessage = fn and addEventListener("message", fn) calls.
To be able to modify the onmessage we have to override the global WebSocket in the first place. The below is intercepting the incoming messages, but in a similar way you can spy on the send method to intercept the outgoing messages (the ones sent by the client to the server).
I tested this on a page using Firebase and it works nicely, but you have to initialize it before the other scripts making sure that the websocket library (it can be socket.io, ws, etc) is using the overridden WebSocket constructor.
Spy the Incoming Messages and modify the data
Eventually you can override the data before calling the real message listener – this becomes handy if you do not have control over the page functionality and want to inject your own data in the message listener.
const OriginalWebsocket = window.WebSocket
const ProxiedWebSocket = function() {
console.log("Intercepting web socket creation")
const ws = new OriginalWebsocket(...arguments)
const originalAddEventListener = ws.addEventListener
const proxiedAddEventListener = function() {
if (arguments[0] === "message") {
const cb = arguments[1]
arguments[1] = function() {
// Here you can get the actual data from the incoming messages
// Here you can even change the data before calling the real message listener
Object.defineProperty(e, "data", { value: 'your injected data' })
console.log("intercepted", arguments[0].data)
return cb.apply(this, arguments)
}
}
return originalAddEventListener.apply(this, arguments)
}
ws.addEventListener = proxiedAddEventListener
Object.defineProperty(ws, "onmessage", {
set(func) {
return proxiedAddEventListener.apply(this, [
"message",
func,
false
]);
}
});
return ws;
};
window.WebSocket = ProxiedWebSocket;
If you do not need to modify the data, you can follow the second part of the answer.
Spy the Incoming messages without modifying the data
If you want to listen for messages only, without overriding the data, things are simpler:
const OriginalWebsocket = window.WebSocket
const ProxiedWebSocket = function() {
const ws = new OriginalWebsocket(...arguments)
ws.addEventListener("message", function (e) {
// Only intercept
console.log(e.data)
})
return ws;
};
window.WebSocket = ProxiedWebSocket;
Spy the Outgoing Messages
In a very similar way, you can proxy the send method which is used to send data to the server.
const OriginalWebsocket = window.WebSocket
const ProxiedWebSocket = function() {
const ws = new OriginalWebsocket(...arguments)
const originalSend = ws.send
const proxiedSend = function() {
console.log("Intercepted outgoing ws message", arguments)
// Eventually change the sent data
// arguments[0] = ...
// arguments[1] = ...
return originalSend.apply(this, arguments)
}
ws.send = proxiedSend
return ws;
};
window.WebSocket = ProxiedWebSocket;
Feel free to ask any questions if anything is unclear.

In a solution similar to yours, where the window.XMLHttpRequest was replaced with a wrapped version that feeds window.lastXhr, we replace window.WebSockets with a wrapped version that feeds window.WebSocketMessages with all messages and timestamps received from all websockets created after this script.
window.watchedWebSockets = [];
window.WebSocketMessages = [];
function WebSocketAttachWatcher(websocket) {
websocket.addEventListener("message", (event)=>window.WebSocketMessages.push([event.data,Date.now()]));
window.watchedWebSockets.push(websocket);
}
// here we replace WebSocket with a wrapped one, that attach listeners on
window.WebSocketUnchanged = window.WebSocket;
window.WebSocket = function(...args) {
const websocket = new window.WebSocketUnchanged(...args);
WebSocketAttachWatcher(websocket);
return websocket;
}
Differently from your XMLRequest case, the websocket may already exist. If you need garanties that all websockets would be catched then you would need to make this wrapper as soon as possible. If you just can't, there's an not so good trick to capture already existing websockets once they send a message:
// here we detect existing websockets on send event... not so trustable
window.WebSocketSendUnchanged = window.WebSocketUnchanged.prototype.send;
window.WebSocket.prototype.send = function(...args) {
console.log("firstsend");
if (!(this in window.watchedWebSockets))
WebSocketAttachWatcher(this);
this.send = window.WebSocketSendUnchanged; // avoid passing here again on next send
window.WebSocketSendUnchanged.call(this, ...args);
}
It is not so trustable since if they don't send but receive they will stay unnoticed.

Intro
The question/bounty/op is specifically asking for a reputable source.
Instead of rolling a custom solution, my proposal is that a known proven library should be used - that has been used, audited, forked, and in general used by the community and that is hosted on github.
The second option is to roll your own (though not recommended) and there are many exccelent answers on how to do it involving the addEventListener
wshook
Wshook is a library (hosted on github) that allows to easily intercept and modify WebSocket requests and message events. It has been starred and forked multiple times.
Disclaimer: I don't have any relationship with the specific project.strong text
Example:
wsHook.before = function(data, url, wsObject) {
console.log("Sending message to " + url + " : " + data);
}
// Make sure your program calls `wsClient.onmessage` event handler somewhere.
wsHook.after = function(messageEvent, url, wsObject) {
console.log("Received message from " + url + " : " + messageEvent.data);
return messageEvent;
}
From the documentation, you will find:
wsHook.before - function(data, url, wsObject):
Invoked just before
calling the actual WebSocket's send() method.
This method must return data which can be modified as well.
wsHook.after - function(event, url, wsObject):
Invoked just after
receiving the MessageEvent from the WebSocket server and before
calling the WebSocket's onmessage Event Handler.
Websocket addEventListener
The WebSocket object supports .addEventListener().
Please see: Multiple Handlers for Websocket Javascript

if you are using nodejs then you can use socket.io
yarn add socket.io
after installation, you can use the middleware of socket.io
io.use(async (socket, next) => {
try {
const user = await fetchUser(socket);
socket.user = user;
} catch (e) {
next(new Error("unknown user"));
}
});

Node Js: Using a WebWorker to resize images

I am making a node.js application that can resize images. I am able to do this successfully with jimp. However resizing an image is not asynchronous, and freezes the UI while resizing. I want to avoid this, so I tried using a webworker.
// main.js
var worker = new Worker(__dirname + '\\worker.js');
worker.addEventListener('message', function(e) {
if (e.data == 'done') { worker.terminate() } // Done
}, false);
worker.postMessage({'buff': buf, 'filename': filename}); // Start the worker
|
// worker.js
const Jimp = require('jimp'); // Oops, this doesn't work
self.addEventListener('message', function(e) {
resize(e.data.buf, e.data.filename);
}, false);
function resize(buf, filename) {
Jimp.read(buf).then(image => {
image.resize(1920, Jimp.AUTO);
image.writeAsync(filename).then(cb => { self.postMessage('done') });
});
}
What I found is that I cannot use node.js functions like require() in a webworker. How can I use Jimp in a webworker or resize an image in a different asynchronous way?
Edit: I am trying to use webworkify, copying the answer from #RubyJunk. When I try to create the worker I get the error Uncaught TypeError: Cannot convert undefined or null to object at Function.keys (<anonymous>). Does anyone know how to fix that?
Edit 2: I am using electron to create my Node.js app. They have a window property (nodeIntegrationInWorker) that makes it seem like I can run Node.js functions in a Web Worker, but when I try to use require() it still tells me it is not a function.

Try using webworkify, and create your worker like so instead
var work = require('webworkify');
var worker = work(require('./worker.js');
and put your worker method into module.exports like so
const Jimp = require('jimp');
module.exports = function(self) {
self.addEventListener('message', function(e) {
resize(e.data.buf, e.data.filename);
}, false);
function resize(buf, filename) {
Jimp.read(buf).then(image => {
image.resize(1920, Jimp.AUTO);
image.writeAsync(filename).then(cb => { self.postMessage('done') });
});
}
}

How to use HTML DOM using npm test (CLI)

I am trying to find a way to run npm test using mocha over a HTML DOM. In this case, I am using the global document to retrieve a table out of the DOM. However, when I run npm test I get something like the error:
ReferenceError: document is not defined
at /home/luiz/Projects/linguist-unknown/src/scripts/ling-loader.js:92:61
at extFunc (/home/luiz/Projects/linguist-unknown/src/scripts/ling-loader.js:49:11)
at Array.every (native)
at Utilities.tryMatchUrlExtension (/home/luiz/Projects/linguist-unknown/src/scripts/ling-loader.js:60:25)
at Utilities.<anonymous> (/home/luiz/Projects/linguist-unknown/src/scripts/ling-loader.js:90:16)
at xhr.onload (/home/luiz/Projects/linguist-unknown/src/scripts/ling-loader.js:24:11)
at dispatchEvent (/home/luiz/Projects/linguist-unknown/node_modules/xmlhttprequest/lib/XMLHttpRequest.js:591:25)
at setState (/home/luiz/Projects/linguist-unknown/node_modules/xmlhttprequest/lib/XMLHttpRequest.js:614:14)
at IncomingMessage.<anonymous> (/home/luiz/Projects/linguist-unknown/node_modules/xmlhttprequest/lib/XMLHttpRequest.js:447:13)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickCallback (internal/process/next_tick.js:104:9)
1) should refresh table
16 passing (3s)
1 failing
1) Loader Utilities should refresh table:
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
I understand that the document is undefined and that I need to, somehow, create one myself, however, I believe that my main problems are:
My first time using npm and mocha and I cannot find anything related to it in their documentation.
Mostly, all problems people have regarding that are related to webbrowsers // I am using CLI, it will be tested with Travis on Github
In my code below you'll see that I solved a similar problem with XMLHttpRequest. However, I just can't figure out the best approach for including the document variable properly into my tests.
Thus, pardon me asking that shall this answer be already there on stackoverflow
My code is the following:
test-utilities.js
...
global.XMLHttpRequest = require('xmlhttprequest').XMLHttpRequest;
global.jsyaml = require('../src/scripts-min/js-yaml.min.js');
global.LinguistHighlighter = require('../src/scripts/ling-highlighter.js').LinguistHighlighter;
var LinguistLoader = require('../src/scripts/ling-loader.js').LinguistLoader;
describe('Loader', function () {
var utilities = new LinguistLoader.Utilities();
it('should refresh table', function(done) {
var location = {
hostname: "github.com",
href: "https://github.com/github-aux/linguist-unknown/blob/chrome/examples/Brain/human_jump.brain",
pathname: "/github-aux/linguist-unknown/blob/chrome/examples/Brain/human_jump.brain"
};
// check if it is not breaking
utilities.refresh(location, function(langObj, table){
done();
});
});
});
...
utilities.js:
...
Utilities.prototype.refresh = function(location, callback) {
var new_url = location.href;
if (new_url === current_url || !this.isGithub(location)) {
return;
}
current_url = new_url;
if (linguistObj === null) {
linguistObj = {
path: this.getPossibleFilepath(location)
};
}
setTimeout(function() {
var downloadHelper = new DownloadHelper();
downloadHelper.load(linguistObj.path, function(objs){
this.tryMatchUrlExtension(current_url, objs, function(langObj){
var table = document.getElementsByClassName("blob-wrapper")[0]
.getElementsByTagName("table")[0];
new LinguistHighlighter.Highlighter(langObj).draw(table);
// callback for tests purposes only
if (callback) {
callback(langObj, table);
}
});
}.bind(this));
}.bind(this), 100);
};
...
Any help is appreciated. Thank you!

I found a very good tool: JSDOM. Its goal is to emulate a subset of a web browser, such as the DOM. With that, I could implement my test-utilities.js file without even touching my utilities.js file, which is pretty much what I wanted.
Here goes the resolution of the file test-utilities.js
const jsdom = require("jsdom");
const { JSDOM } = jsdom;
global.XMLHttpRequest = require('xmlhttprequest').XMLHttpRequest;
global.jsyaml = require('../src/scripts-min/js-yaml.min.js');
global.LinguistHighlighter = require('../src/scripts/ling-highlighter.js').LinguistHighlighter;
var LinguistLoader = require('../src/scripts/ling-loader.js').LinguistLoader;
describe('Loader', function () {
var utilities = new LinguistLoader.Utilities();
it('should refresh the code table', function(done) {
// Download the HTML string and parse it to JSDOM
JSDOM.fromURL("https://github.com/github-aux/linguist-unknown/blob/chrome/examples/Brain/human_jump.brain").then(dom => {
// JSDOM does not support 'innerText' and that is why I am creating this property for all objects.
var o = Object.prototype;
Object.defineProperty(o, "innerText", {
get: function jaca() {
if (this.innerHTML === undefined)
return "";
return this.innerHTML;
}
});
var location = {
hostname: "github.com",
href: "https://github.com/github-aux/linguist-unknown/blob/chrome/examples/Brain/human_jump.brain",
pathname: "/github-aux/linguist-unknown/blob/chrome/examples/Brain/human_jump.brain"
};
// check if it is not breaking
utilities.refresh(location, function(langObj, table) {
done();
});
});
});
That is working properly now! I hope it helps anyone! :D

Service Worker Respond To Fetch after getting data from another worker

I am using service workers to intercept requests for me and provide the responses to the fetch requests by communicating with a Web worker (also created from the same parent page).
I have used message channels for direct communication between the worker and service worker. Here is a simple POC I have written:
var otherPort, parentPort;
var dummyObj;
var DummyHandler = function()
{
this.onmessage = null;
var selfRef = this;
this.callHandler = function(arg)
{
if (typeof selfRef.onmessage === "function")
{
selfRef.onmessage(arg);
}
else
{
console.error("Message Handler not set");
}
};
};
function msgFromW(evt)
{
console.log(evt.data);
dummyObj.callHandler(evt);
}
self.addEventListener("message", function(evt) {
var data = evt.data;
if(data.msg === "connect")
{
otherPort = evt.ports[1];
otherPort.onmessage = msgFromW;
parentPort = evt.ports[0];
parentPort.postMessage({"msg": "connect"});
}
});
self.addEventListener("fetch", function(event)
{
var url = event.request.url;
var urlObj = new URL(url);
if(!isToBeIntercepted(url))
{
return fetch(event.request);
}
url = decodeURI(url);
var key = processURL(url).toLowerCase();
console.log("Fetch For: " + key);
event.respondWith(new Promise(function(resolve, reject){
dummyObj = new DummyHandler();
dummyObj.onmessage = function(e)
{
if(e.data.error)
{
reject(e.data.error);
}
else
{
var content = e.data.data;
var blob = new Blob([content]);
resolve(new Response(blob));
}
};
otherPort.postMessage({"msg": "content", param: key});
}));
});
Roles of the ports:
otherPort: Communication with worker
parentPort: Communication with parent page
In the worker, I have a database say this:
var dataBase = {
"file1.txt": "This is File1",
"file2.txt": "This is File2"
};
The worker just serves the correct data according to the key sent by the service worker. In reality these will be very large files.
The problem I am facing with this is the following:
Since I am using a global dummyObj, the older dummyObj and hence the older onmessage is lost and only the latest resource is responded with the received data.
In fact, file2 gets This is File1, because the latest dummyObj is for file2.txt but the worker first sends data for file1.txt.
I tried by creating an iframe directly and all the requests inside it are intercepted:
<html>
<head></head>
<body><iframe src="tointercept/file1.txt" ></iframe><iframe src="tointercept/file2.txt"></iframe>
</body>
</html>
Here is what I get as output:
One approach could be to write all the files that could be fetched into IndexedDB in the worker before creating the iframe. Then in the Service Worker fetch those from indexed DB. But I don't want to save all the resources in IDB. So this approach is not what I want.
Does anybody know a way to accomplish what I am trying to do in some other way? Or is there a fix to what I am doing.
Please Help!
UPDATE
I have got this to work by queuing the dummyObjs in a global queue instead of having a global object. And on receiving the response from the worker in msgFromW I pop an element from the queue and call its callHandler function.
But I am not sure if this is a reliable solution. As it assumes that everything will occur in order. Is this assumption correct?

I'd recommend wrapping your message passing between the service worker and the web worker in promises, and then pass a promise that resolves with the data from the web worker to fetchEvent.respondWith().
The promise-worker library can automate this promise-wrapping for you, or you could do it by hand, using this example as a guide.
If you were using promise-worker, your code would look something like:
var promiseWorker = new PromiseWorker(/* your web worker */);
self.addEventListener('fetch', function(fetchEvent) {
if (/* some optional check to see if you want to handle this event */) {
fetchEvent.respondWith(promiseWorker.postMessage(/* file name */));
}
});