We Keep Coding

Node.JS Module/Object that spans several files?

I am working on supporting a REST API that literally has thousands of functions/objects/stats/etc., and placing all those calls into one file does not strike me as very maintainable. What I want to do is have a 'base' file that has the main constructor function, a few utility and very common functions, and then files for each section of API calls.
The Problem: How do you attach functions from other files to the 'base' Object so that referencing the main object allows for access from the subsections you have added to your program??
Let me try and illustrate what I am looking to do:
1) 'base' file has the main constructor:
var IPAddr = "";
var Token = "";
exports.Main = function(opts) {
IPAddr = opts.IPAddr;
Token = opts.Token;
}
2) 'file1' has some subfunctions that I want to define:
Main.prototype.Function1 = function(callback) {
// stuff done here
callback(error, data);
}
Main.prototype.Function2 = function(callback) {
// stuff done here
callback(error,data);
}
3) Program file brings it all together:
var Main = require('main.js');
var Main?!? = require('file1.js');
Main.Function1(function(err,out) {
if(err) {
// error stuff here
}
// main stuff here
}
Is there a way to combine an Object from multiple code files?? A 120,000 line Node.JS file just doesn't seem to be the way to go to me....not to mention it takes too long to load! Thanks :)
SOLUTION: For those who may stumble upon this in the future... I took the source code for Object.assign and back ported it to my v0.12 version of Node and got it working.
I used the code from here: https://github.com/sindresorhus/object-assign/blob/master/index.js and put it in a separate file that I just require('./object-assign.js') without assigning it to a var. Then my code looks something like this:
require('./object-assign.js');
var Main = require('./Main.js');
Object.assign(Main.prototype, require('./file1.js'));
Object.assign(Main.prototype, require('./file2.js'));
And all my functions from the two files show up under the Main() Object...too cool :)

At first each file works in its own scope, so all local variables are not shared.
However, as hacky approach, you may just add Main into global scope available everywhere by writing global.Main = Main right after you define it, please make sure that you require main file first in list of requires.
The better(who said?) approach is to extend prototype of Main later, but in this case you may need to update a lot of code. Just mix-in additional functionality into base class
file1.js
module.exports = {
x: function() {/*****/}
}
index.js
var Main = require('main.js');
Object.assign(Main.prototype, require('file1.js'));

Shure.
constructor.js
module.exports = function(){
//whatever
};
prototype.js
module.exports = {
someMethod(){ return "test";}
};
main.js
const Main = require("./constructor.js");
Object.assign( Main.prototype, require("./prototype.js"));

How to realy include a js file in Node.js

I did't found any realy working solution of this issue on Google, so...
There is a module called vm, but people say it is very heavy. I need some simple function, like PHP's include, which works like the code of an including file inserts right into the place in code you need and then executes.
I tryed to create such function
function include(path) {
eval( fs.readFileSync(path) + "" );
}
but it is not as simple... it is better if I'll show you why in example.
Let's say I need to include file.js file with content
var a = 1;
The relative file look like this
include("file.js");
console.log(a); // undefined
As you already realized a is undefined because it is not inherits from a function.
Seems the only way to do that is to type this long creepy code
eval( fs.readFileSync("file.js") + "" );
console.log(a); // 1
every time with no wrapper function in order to get all the functionality from a file as it is.
Using require with module.exports for each file is also a bad idea...
Any other solutions?

Using require with module.exports for each file is also a bad idea...
No, require is the way you do this in NodeJS:
var stuff = require("stuff");
// Or
var stuff = require("./stuff"); // If it's within the same directory, part of a package
Break your big vm into small, maintainable pieces, and if they need to be gathered together into one big thing rather than being used directly, have a vm.js that does that.
So for example
stuff.js:
exports.nifty = nifty;
function nifty() {
console.log("I do nifty stuff");
}
morestuff.js:
// This is to address your variables question
exports.unavoidable = "I'm something that absolutely has to be exposed outside the module.";
exports.spiffy = spiffy;
function spiffy() {
console.log("I do spiffy stuff");
}
vm.js:
var stuff = require("./stuff"),
morestuff = require("./morestuff");
exports.cool = cool;
function cool() {
console.log("I do cool stuff, using the nifty function and the spiffy function");
stuff.nifty();
morestuff.spiffy();
console.log("I also use this from morestuff: " + morestuff.unavoidable);
}
app.js (the app using vm):
var vm = require("./vm");
vm.cool();
Output:
I do cool stuff, using the nifty function and the spiffy function
I do nifty stuff
I do spiffy stuff
I also use this from morestuff: I'm something that absolutely has to be exposed outside the module.

What you are trying to do breaks modularity and goes against Node.js best practices.
Lets say you have this module (sync-read.js):
var fs = require('fs');
module.exports = {
a: fs.readFileSync('./a.json'),
b: fs.readFileSync('./b.json'),
c: fs.readFileSync('./c.json')
}
When you call the module the first time ...
var data = require('./sync-read');
... it will be cached and you wont read those files from disk again. With your approach you'll be reading from disk on every include call. No bueno.
You don't need to append each of your vars to module.exports (as in comment above):
var constants = {
lebowski: 'Jeff Bridges',
neo: 'Keanu Reeves',
bourne: 'Matt Damon'
};
function theDude() { return constants.lebowski; };
function theOne() { return constants.neo; };
function theOnly() { return constants.bourne; };
module.exports = {
names: constants,
theDude : theDude,
theOne : theOne
// bourne is not exposed
}
Then:
var characters = require('./characters');
console.log(characters.names);
characters.theDude();
characters.theOne();

Passing in window to this in browserify

One of my dependencies uses the following to pass in window to its closure
(function (window) {
//
})(this)
For the time being I can just change it to something more sensible so that it doesn't break browserify, but is there some method whereby I can force a value for this in a browserified module?

I wrote a browserify transform called "moduleify" that should generally do what you want, i.e. wrap the offending code in an IIFE that looks kinda like this:
(function () {
// this === window
}.call(window));
In fact, my implementation is not much more sophisticated than that.
The original idea was to export a globals-polluting "module" as if it were a CommonJS module (e.g. have AngularJS export window.angular), but because it contains that wrapper, it should do the trick.
For instructions, see the README. If the offending script doesn't actually have anything it could reasonably export, just have it export window (which will result in module.exports = window['window']) or an arbitrary name that doesn't exist (resulting in undefined).
If you want to access the window object in your own browserify code, also check out the global module, which provides a nice wrapper to access browser globals safely in CommonJS modules.

To solve this specific problem, a simple transform, that wraps the code in a self calling function, will do the job.
CoffeeScript
through = require('through')
fenestrate = (file) ->
data = ';(function() {\n';
write = (buf) ->
data += buf
end = ->
data += '\n}).call(window);'
this.queue(data)
this.queue(null)
through(write, end)
JavaScript
var through = require('through');
var fenestrate = function(file) {
var data, end, write;
data = ';(function() {\n';
write = function(buf) {
return data += buf;
};
end = function() {
data += '\n}).call(window);';
this.queue(data);
return this.queue(null);
};
return through(write, end);
};
Writing Transforms: https://github.com/substack/browserify-handbook#transforms

How does require() in node.js work?

I tried this:
// mod.js
var a = 1;
this.b = 2;
exports.c = 3;
// test.js
var mod = require('./mod.js');
console.log(mod.a); // undefined
console.log(mod.b); // 2
console.log(mod.c); // 3, so this === exports?
So I image that require() may be implement like this:
var require = function (file) {
var exports = {};
var run = function (file) {
// include "file" here and run
};
run.apply(exports, [file]);
return exports;
}
Is that right? Please help me to understand require(), or where can I find the source code. Thanks!

Source code is here. exports/require are not keywords, but global variables. Your main script is wrapped before start in a function which has all the globals like require, process etc in its context.
Note that while module.js itself is using require(), that's a different require function, and it is defined in the file called "node.js"
Side effect of above: it's perfectly fine to have "return" statement in the middle of your module (not belonging to any function), effectively "commenting out" rest of the code

var mod = require('./mod.js');
The require is a function that takes one argument called path, in this case the path is ./mod.js
when the require is invoked, a sequences of tasks are happened:
call Module.prototype.require function declared in lib/module.js which assert that the path exists and was a string
call Module._load which is a function in lib/module.js that resolve the file through Module._resolveFilename(request, parent, isMain),
the Module._resolveFilename function is called and checks if the module is native (The native modules are returned by NativeModule function defined in lib/internal/bootstrap_node.js),
if yes it will return the module else it checks the number of characters of the parh (Must 2 character at least) and some characters (the path must started by ./)
via Module._resolveLookupPaths function defined in defined in lib/internal/bootstrap_node.js
check the directory that contains the file
If the path contains an extension (in our example yes: mod.js), the basename function defined in lib/path.js checks that the extension is "js"
then it will create a new module for the file given in argument var module = new Module(filename, parent);
the content will be compiled via v8 through the function NativeModule.prototype.compile defined in lib/internal/bootstrap_node.js
the NativeModule.wrap defined in lib/internal/bootstrap_node.js takes the javascript content compiled of mod.js and wraps it : It wraps it in some other code that makes all this work.
So the code you've written in mod.js is wrapped in a function expression. that means everything you write in node is run in V8
a module.exports is what's returned

Andrey showed the source code, but if you also wonder how to use it, the easy and simple explanation is here (http://nodejs.org/api/modules.html).
These were two good examples for me.
//foo.js, multiple methods
var circle = require('./circle.js');
console.log( 'The area of a circle of radius 4 is ' + circle.area(4));
//circle.js
var PI = Math.PI;
exports.area = function (r) {
return PI * r * r;
};
exports.circumference = function (r) {
return 2 * PI * r;
};
//bar.js
var square = require('./square.js');
var mySquare = square(2);
console.log('The area of my square is ' + mySquare.area());
//square.js, single method
module.exports = function(width) {
return {
area: function() {
return width * width;
}
};
}
My favourite pattern is
(function (controller) {
controller.init = function (app) {
app.get("/", function (req, res) {
res.render("index", {});
});
};
})(module.exports);

I dig a little more of nodejs source code/2/ and make a sequence diagram/1/, hope this could give you a intuitive overview. There is another article http://fredkschott.com/post/2014/06/require-and-the-module-system/ which also explain the require() mechanism in a easy way, go through this article first could help you to understand the diagram quickly.
Ref:
/1/ diagram source repo: https://github.com/z1yuan/nodejs.git
/2/ https://github.com/nodejs/node-v0.x-archive.git

Try This.
This is a snippet of what I used to create the same functionality as Node.js
/*
FILE: require.js
*/
/*
This is the file used
*/
window.require = function(src, ret) {
if (src === 'jsmediatags') {
src = 'https://cdnjs.cloudflare.com/ajax/libs/jsmediatags/3.9.5/jsmediatags.js';
};
var d = document.createElement('script');
d.src = src;
document.head.appendChild(d);
var fullURL = src.split('://');
var neededURL = fullURL[1];
var nameParts = neededURL.split('/');
var nameNUM = nameParts.length - 1;
var fileName = nameParts[nameNUM];
var g = fileName.split('.');
var global = g[0];
if (ret === true) {
return window[global]
};
};
See if this works, and to add more files to its library, just type more in. (if (src===yourfilenamehere) { src = "path/to/your/file" }

The module loading mechanism in Node.js is caching the modules on the first require call. It means that every time you use require('xyz-module') you will get the same instance of xyz-module, which ensures that the modules are singleton-like and have the same state across your application.
You can load native modules and path references from your file system or installed modules. If the identifier passed to the require function is not a native module or a file reference (beginning with /, ../, ./ or similar), then Node.js will look for installed modules. It will walk your file system looking for the referenced module in the node_modules folder. It starts from the parent directory of your current module and then moves to the parent directory until it finds the right module or until the root of the file system is reached.

The source is available here next to the downloads : http://nodejs.org/ exports/require are keywords, I don't think they are coded in javascript directly. Node is coded in C++ , javascript is just a scripting shell around the C++ core.

Load and execute external js file in node.js with access to local variables?

Is it easy/possible to do a simple include('./path/to/file') type of command in node.js?
All I want to do is have access to local variables and run a script. How do people typically organize node.js projects that are bigger than a simple hello world? (A fully functional dynamic website)
For example I'd like to have directories like:
/models
/views
... etc

Just do a require('./yourfile.js');
Declare all the variables that you want outside access as global variables.
So instead of
var a = "hello" it will be
GLOBAL.a="hello" or just
a = "hello"
This is obviously bad. You don't want to be polluting the global scope.
Instead the suggest method is to export your functions/variables.
If you want the MVC pattern take a look at Geddy.

You need to understand CommonJS, which is a pattern to define modules. You shouldn't abuse GLOBAL scope that's always a bad thing to do, instead you can use the 'exports' token, like this:
// circle.js
var PI = 3.14; // PI will not be accessible from outside this module
exports.area = function (r) {
return PI * r * r;
};
exports.circumference = function (r) {
return 2 * PI * r;
};
And the client code that will use our module:
// client.js
var circle = require('./circle');
console.log( 'The area of a circle of radius 4 is '
+ circle.area(4));
This code was extracted from node.js documentation API:
http://nodejs.org/docs/v0.3.2/api/modules.html
Also, if you want to use something like Rails or Sinatra, I recommend Express (I couldn't post the URL, shame on Stack Overflow!)

If you are writing code for Node, using Node modules as described by Ivan is without a doubt the way to go.
However, if you need to load JavaScript that has already been written and isn't aware of node, the vm module is the way to go (and definitely preferable to eval).
For example, here is my execfile module, which evaluates the script at path in either context or the global context:
var vm = require("vm");
var fs = require("fs");
module.exports = function(path, context) {
var data = fs.readFileSync(path);
vm.runInNewContext(data, context, path);
}
Also note: modules loaded with require(…) don't have access to the global context.

If you are planning to load an external javascript file's functions or objects, load on this context using the following code – note the runInThisContext method:
var vm = require("vm");
var fs = require("fs");
var data = fs.readFileSync('./externalfile.js');
const script = new vm.Script(data);
script.runInThisContext();
// here you can use externalfile's functions or objects as if they were instantiated here. They have been added to this context.

Expanding on #Shripad's and #Ivan's answer, I would recommend that you use Node.js's standard module.export functionality.
In your file for constants (e.g. constants.js), you'd write constants like this:
const CONST1 = 1;
module.exports.CONST1 = CONST1;
const CONST2 = 2;
module.exports.CONST2 = CONST2;
Then in the file in which you want to use those constants, write the following code:
const {CONST1 , CONST2} = require('./constants.js');
If you've never seen the const { ... } syntax before: that's destructuring assignment.

Sorry for resurrection. You could use child_process module to execute external js files in node.js
var child_process = require('child_process');
//EXECUTE yourExternalJsFile.js
child_process.exec('node yourExternalJsFile.js', (error, stdout, stderr) => {
console.log(`${stdout}`);
console.log(`${stderr}`);
if (error !== null) {
console.log(`exec error: ${error}`);
}
});