We Keep Coding

Read environment variables and then replace them in client-side JS when using gulp for building prod or dev code

So lets say I have some code in js
const myApiKey = 'id_0001'
But instead of harcoding it I want to put it in some bash script with other env vars and read from it and then replace it in the JS
So lets say for prod I would read from prod-env.sh or for dev I would read them from dev-env.sh and then gulp or some other tool does the magic and replaces MY_API_KEY based on whatever is established inside of prod-env.sh or dev-env.sh.
const myApiKey = MY_API_KEY
Update: I want to add I only care about unix OS, not concerned about windows. In golang there is way to read for example envVars.get('MY_API_KEY'), I'm looking for something similar but for JS in the client side.

If you're using gulp, it sounds like you could use any gulp string replacer, like gulp-replace.
As for writing the gulp task(s). If you are willing to import the environment into your shell first, before running node, you can access the environment via process.env
gulp.task('build', function(){
gulp.src(['example.js'])
.pipe(replace('MY_API_KEY', process.env.MY_API_KEY))
.pipe(gulp.dest('build/'));
});
If you don't want to import the environment files before running node, you can use a library like env2 to read shell environment files.
Another option would be to use js/json to define those environment files, and load them with require.
prod-env.js
{
"MY_API_KEY": "api_key"
}
gulpfile.js
const myEnv = require('./prod-env')
gulp.task('build', function(){
gulp.src(['example.js'])
.pipe(replace('MY_API_KEY', myEnv.MY_API_KEY))
.pipe(gulp.dest('build/'));
});
Also, for a more generic, loopy version of the replace you can do:
gulp.task('build', function () {
stream = gulp.src(['example.js']);
for (const key in process.env) {
stream.pipe('${' + key + '}', process.env[key]);
}
stream.pipe(gulp.dest('build/'));
});
In that last example I added ${} around the environment variable name to make it less prone to accidents. So the source file becomes:
const myApiKey = ${MY_API_KEY}

This answer is an easy way to do this for someone who doesn't want to touch the code they are managing. For example you are on the ops team but not the dev team and need to do what you are describing.
The environment variable NODE_OPTIONS can control many things about the node.js runtime - see https://nodejs.org/api/cli.html#cli_node_options_options
One such option we can set is --require which allows us to run code before anything else is even loaded.
So using this you can create a overwrite.js file to perform this replacement on any non-node_modules script files:
const fs = require('fs');
const original = fs.readFileSync;
// set some custom env variables
// API_KEY_ENV_VAR - the value to set
// API_KEY_TEMPLATE_TOKEN - the token to replace with the value
if (!process.env.API_KEY_TEMPLATE_TOKEN) {
console.error('Please set API_KEY_TEMPLATE_TOKEN');
process.exit(1);
}
if (!process.env.API_KEY_ENV_VAR) {
console.error('Please set API_KEY_ENV_VAR');
process.exit(1);
}
fs.readFileSync = (file, ...args) => {
if (file.includes('node_modules')) {
return original(file, ...args);
}
const fileContents = original(file, ...args).toString(
/* set encoding here, or let it default to utf-8 */
);
return fileContents
.split(process.env.API_KEY_TEMPLATE_TOKEN)
.join(process.env.API_KEY_ENV_VAR);
};
Then use it with a command like this:
export API_KEY_ENV_VAR=123;
export API_KEY_TEMPLATE_TOKEN=TOKEN;
NODE_OPTIONS="--require ./overwrite.js" node target.js
Supposing you had a script target.js
console.log('TOKEN');
It would log 123. You can use this pretty much universally with node, so it should work fine with gulp, grunt, or any others.

Override require to start from root and check node modules without changing it's name

I'm trying to share my client and server code and I want a tidy way of requiring my dependencies.
I'm currently using webpack as my build tool and I've set it to resolve to my assets/js folder so my dependencies for the client look something like this:
require('validators/login.js');
I'd like to share the exact same code across the server, but I don't want to put if statements in to check if it's the server or client that's doing the requiring so I can do something like
if server
rootRequire..
else
require
What i'd like is for the server to use require like the client does and start from the root directory, but I also don't want to strip it of its functionality to search the node_modules folder if it doesn't return a result.
I also want this done without placing all my code in the node_modules folder.
Does anyone have some good suggestions for how to override require properly? I've read most of the hacky solutions like using symlinks or placing things in node_modules and I'm not looking for those answers, I'd like something that is clean and actually overrides require properly.

I think I've come up with a solution I'm reasonably happy with, figured I'd share it with anyone who else needed an answer.
var module = require('module');
var baseRequire = module.prototype.require;
module.prototype.require = function()
{
// Add any extra paths here...
var extraPaths = [
__dirname
];
for (var i = 0, len = extraPaths.length; i < len; i++) {
if (this.paths.indexOf(extraPaths[i]) == -1) {
this.paths.push(extraPaths[i])
}
}
return baseRequire.apply(this, arguments);
};

Are JavaScript classes acceptable Node modules?

Bear with me as I lead you through the process that elicited my question.
I'm working on a CLI app in node and I'm using objects to encapsulate my business logic using this pattern:
// my-project/lib/widget/myobject.js
var MyObject = function(x) {
this.x = x;
};
MyObject.prototype.getX = function() {
return this.x;
};
module.exports = MyObject;
I'm also testing these objects:
// my-project/test/lib/widget/myobject.spec.js
var MyObject = require('../../../lib/widget/myobject.js');
describe('MyObject', function() {
...
});
At one point I was unhappy with the naming and directory structure I had chosen. I found myself tediously counting those parent directory references (..) in several spec files when rewriting the relative paths. I figured there must be an easier way to reference a root directory containing these object definitions.
One of the recommendations I found here suggested "putting application-specific modules into node_modules".
Now, as I understand modules, they are the packages I download from npm and use in my project. They contain libraries of useful things with a single API exported to me when I call require. This is not how I view the simple single-purpose classes built specifically for the internal use of my application.
If you've stuck with me this far, thank you! Here is my question:
How do I make the internals of my application more "modular" so it properly follows the intent of the Node module system while remaining object oriented?

I'm not sure how suitable this is for production or for modules you plan on distributing but in your main file you could add this:
process.env.NODE_PATH = __dirname;
require('module').Module._initPaths();
which would let you always require modules relative to the folder containing your main file. I.e. if you had a file in:
library/some_file.js
then in tests/some_other_file.js you could just do:
require('library/some_file');
Or as an alternative you could add this in your main file:
global.__base = __dirname + '/';
and then in your other modules require using:
var MyObject = require(__base + 'my-project/lib/widget/myobject');

ES6 dynamic imports and instanciation of classes

I'm trying to figure out how to perform dynamic import of classes in ES6 one the server side (node.js with Babel).
I would like to have some functionalities similar to what reflection offers in Java. The idea is to import all the classes in a specific folder and instanciate them dynamically.
So for example I could have multiple classes declared in a folder like the one below :
export default class MyClass {
constructor(somevar) {
this._somevar = somevar
}
//...
//some more instance level functions here
}
and then somewhere else in my app's code I could have a function that finds out all the classes in a specific folder and tries to instanciate them :
//somewhere else in my app
instanciationFunction(){
//find all the classes in a specific folder
var classFiles = glob.sync(p + '/path_to_classes/**/*.js', {
nodir: true
});
_.each(classFiles, async function (file) {
console.log(file);
var TheClass = import(file);
var instance = new TheClass();
//and then do whatever I want with that new instance
});
}
I've tried doing it with require but I get errors. Apparently the constructor cant be found.
Any idea would be greatly appreciated.
Thanks

ES module definitions are declarative, and the current direction tools are taking is the path where dependencies are determined during parse (via static analysis), waaay before any of the code is executed. This means dynamic and conditional imports go against the said path. It's not like in Node where imports are determined on execution, upon executing require.
If you want dynamic, runtime imports, consider taking a look at SystemJS. If you're familiar with RequireJS, it takes the same concept, but expands it to multiple module formats, including ES6. It has SystemJS.import which appears to do what you want, plus handles the path resolution that you're currently doing.
Alternatively, if your intention is to shed off excess code, consider using Rollup. It will analyze code for you and only include code that's actually used. That way, you don't need to manually do conditional loading.

You need to preprocess with babel, because they are not yet a part of node (for that matter, neither are static imports - node uses require).
https://github.com/airbnb/babel-plugin-dynamic-import-node
steps:
pre
npm i -D babel-cli or npm i -D babel
1
npm i -D babel-plugin-dynamic-import-node
2
.babelrc
{
"plugins": ["dynamic-import-node"]
}
ready, go!
babel-node test_import.js for babel-cli, or for raw babel:
a
(edit) package.json
"scripts": {
"pretest": "babel test_imports.js -o dist/test_imports.js",
"test": "node dist/test_imports.js"
//...
b
node test

I had the same usecase and i managed to dynamically load and instantiate default exported classes using:
const c = import("theClass.js")
const i = new c.default();
using node v16.4.0

Accessing global variables with node.js and CoffeeScript

I'm trying to declare global variables for my applications that I'm writing using Node.js and CoffeeScript. So I'm declaring it in a common file that is concatenated to both applications after compilation. In that file I have for example:
root = exports ? this
root.myVariable = 300
So my first application is a HTML one. When I try to access this variable, for example by
console.log myVariable
There is no problem with it. But my other application is a server application lauched by node command and I cannot access that variable in that application. I tried:
console.log root.myVariable
console.log myVariable
With first line I'm getting 'undefined' printed (so it looks that root is defined) and with the second one, I'm getting ReferenceError - myVariable is undefined.
So how can I access this variable?
Here is an output code in Javascript that I get, I guess it might be helpful:
(function() {
var root, _ref;
root = (_ref = typeof module !== "undefined" && module !== null ? module.exports : void 0) != null ? _ref : this;
root.myVariable = 300;
}).call(this);
(function() {
console.log(root.myVariable);
console.log(myVariable);
}).call(this);

You're close, but you need to change things just a little bit
# config.coffee
module.exports =
foo: "bar"
hello: "world"
db:
user: alice
pass: password1
# lib/a.coffee
config = require "../config"
# lib/b.coffee
config = require "../config"
# lib/db.coffee
dbconfig = require("../config").db

Client and server JavaScript (or CoffeeScript) works differently. So, its a really difficult to write a module that'll work in both applications.
There is a lot of libraries to solve this problem, like RequireJS and Browserify.
But I have two simpler suggestions for your problem.
First one is to use JSON to store your global constants. On the server side you can simply require you JSON file:
root = require './config.json'
On the client side you may either parse it manually or serve it as pjson.
My second suggestion is to write really simple module that'll be compatible with both your applications. It'll look something like this:
root =
myVariable: 300
myOtherVariable: 400
modulte.exports = root if module?.parent?
This code should be compatible with both node.js require function and browser <script> tag.
Update:
I just reread you question and realized, that you've did almost as I suggested. But your code looks fine to me. You may try to use module.export instead of its alias exports, it may help:
root = modulte?.exports ? this
root.myVariable = 300
But, as I said, your code looks fine to me as well.