I have a parser for a language I'm implementing and I would like to allow the user to issue a request to use outside implementations of their own functions in a .js file they create. So, for example:
functions.js
var functions = {
var func1 = function(){/*...*/}
}
module.exports = functions;
toparse.mylanguage
#bind "functions.js"
x = func1();
And in my node.js server I would use var functions = require('./functions') and have the parser direct the execution to the relevant function. I assume that they upload the .js file to the same folder they uploade the .mylanguage file.
What would be a good way to do this?
EDIT: to clarify, I don't know the name of the .js file before #bind is used, so I can't require it specifically beforehand.
Related
Content
I have a large module that I am assembling in Javascript, which is problematic because JS currently has poor native module support.
Since my module is large, I personally do not like having one massive file with my module object e.g.
var my_module = {
func_1: function(param) {console.log(param)},
...,
func_n: function(param_1, param_2) {console.log(param_1 - param_2)}
}
where func_n ends around line number 3000. I would much rather store each of my functions (or several related functions) in separate files. I personally find this easier to manage.
This poses a problem, however, as although one could use synchronous calls to load the functions - the javascript will still be parsed asynchronously (to my understanding). Thus several independent synchronous calls to loading files is insufficient - as the mth file might call something related to the nth file (n < m) which has not yet been parsed causing an error.
Thus the solution a solution is apparent: recursively - synchronously - load files in the callback of the previous file.
Consider the code at the bottom of this post.
Now this isn't perfect. It has several assumptions e.g. that each file contains one function and that function is the same as the filename after striping the extension (a() is in a.js; do_something(a, b, c) is in do_something.js). It also doesn't encapsulate private variables. However, this could be worked around by adding a JSON file with these variables. Adding this JSON to the module as module.config and then passing the config object to each of the functions in the module.
In addition this still pollutes the namespace.
Question
My question is as follows:
what is a native JS way (nor do I not want a library that does this for me - jQuery included) to load functions stored across many files into a cohesive module without polluting the namespace, and ensuring that all the files are parsed before any function calls?
Code to consider (my solution)
Code directory structure:
- directory
---- index.html
---- bundle.js
---- test_module/
-------- a.js
-------- b.js
-------- log_num.js
-------- many_parameters.js
index.html
<head>
<script src="bundle.js"></script>
</head>
bundle.js
// Give JS arrays the .empty() function prototype
if (!Array.prototype.empty){
Array.prototype.empty = function(){
return this.length == 0;
};
};
function bundle(module_object, list_of_files, directory="") {
if (!list_of_files.empty()) {
var current_file = list_of_files.pop()
var [function_name, extension] = current_file.split(".")
var new_script = document.createElement("script")
document.head.appendChild(new_script)
new_script.src = directory + current_file
new_script.onload = function() {
module_object[function_name] = eval(function_name)
bundle(module_object, list_of_files, directory)
/*
nullify the function in the global namespace as - assumed - last
reference to this function garbage collection will remove it. Thus modules
assembled by this function - bundle(obj, files, dir) - must be called
FIRST, else one risks overwritting a funciton in the global namespace and
then deleting it
*/
eval(function_name + "= undefined")
}
}
}
var test_module = {}
bundle(test_module, ["a.js", "b.js", "log_num.js", "many_parameters.js"], "test_module/")
a.js
function a() {
console.log("a")
}
b.js
function b() {
console.log("b")
}
log_num.js
// it works with parameters too
function log_num(num) {
console.log(num)
}
many_parameters.js
function many_parameters(a, b, c) {
var calc = a - b * c
console.log(calc)
}
If we restrict our tools to the "native JS way", there is an import() proposal, currently at Stage 3 on the TC39 proposal process:
https://github.com/tc39/proposal-dynamic-import
System.js offers a similar approach to dynamically load modules.
Have you looked at RequireJS. From the home page:
RequireJS is a JavaScript file and module loader. It is optimized for in-browser use, but it can be used in other JavaScript environments, like Rhino and Node. Using a modular script loader like RequireJS will improve the speed and quality of your code.
It has support for Definition Functions with Dependencies
If the module has dependencies, the first argument should be an array of dependency names, and the second argument should be a definition function. The function will be called to define the module once all dependencies have loaded. The function should return an object that defines the module. The dependencies will be passed to the definition function as function arguments, listed in the same order as the order in the dependency array:
That would allow you to "split up" your module into what ever arbitrary pieces you decide and "assemble" them at load time.
I am using PhantomJS for page automation. For my script, I need to load another script which is also used on the actual client-side of my webpage. This script contains some parts where global variables - which are assumed to be set before the script is loaded - are used.
Now my problem is that I can't figure out how to set these variables before I require them within PhantomJS.
This (obviously) didn't do the trick:
variableX = 1024;
var moduleX = require('myScripts.js');
Now what's the proper and intended way ( if there is one) to do this?
If you prepare the testing script as a CommonJS module, you can require it and use its methods and variables in PhantomJS.
From the docs: http://phantomjs.org/release-1.7.html
As an example, supposed there is a script universe.js which contains the following code:
exports.answer = 42;
exports.start = function () {
console.log('Starting the universe....');
}
This module can be used in another script like the following:
var universe = require('./universe');
universe.start();
console.log('The answer is', universe.answer);
And if you want to assign global variables from such a module you can use the global object as in node.js:
exports.start = function () {
global.success = true;
console.log('Starting the universe....');
}
I am reading some source files and see that functions have been written and called in two different patterns. I am going to describe the two patterns first and then how they are called in the main file.
First pattern :
File module1.js:
function a(req,res,next){
//do somethin
}
module.exports.a = a;
seconds pattern :
File module2.js
module.exports = function(){
return function(req,res,next){
/* do something here */
}
}
File main.js
var mod1 = require('module1');
var mod2 = require('module2');
server.use(mod1.a);
server.use(mod2());
What confuses me is why mod1 and mod2 have been written so differently. What is the right way of writing these
The second pattern is useful if you need to pass in a configuration object or other information (e.g. a database instance) to the middleware that it will use for requests.
Either way is acceptable, it just depends on your needs.
There are some third party Javascript libraries that have some functionality I would like to use in a Node.js server. (Specifically I want to use a QuadTree javascript library that I found.) But these libraries are just straightforward .js files and not "Node.js libraries".
As such, these libraries don't follow the exports.var_name syntax that Node.js expects for its modules. As far as I understand that means when you do module = require('module_name'); or module = require('./path/to/file.js'); you'll end up with a module with no publicly accessible functions, etc.
My question then is "How do I load an arbitrary javascript file into Node.js such that I can utilize its functionality without having to rewrite it so that it does do exports?"
I'm very new to Node.js so please let me know if there is some glaring hole in my understanding of how it works.
EDIT: Researching into things more and I now see that the module loading pattern that Node.js uses is actually part of a recently developed standard for loading Javascript libraries called CommonJS. It says this right on the module doc page for Node.js, but I missed that until now.
It may end up being that the answer to my question is "wait until your library's authors get around to writing a CommonJS interface or do it your damn self."
Here's what I think is the 'rightest' answer for this situation.
Say you have a script file called quadtree.js.
You should build a custom node_module that has this sort of directory structure...
./node_modules/quadtree/quadtree-lib/
./node_modules/quadtree/quadtree-lib/quadtree.js
./node_modules/quadtree/quadtree-lib/README
./node_modules/quadtree/quadtree-lib/some-other-crap.js
./node_modules/quadtree/index.js
Everything in your ./node_modules/quadtree/quadtree-lib/ directory are files from your 3rd party library.
Then your ./node_modules/quadtree/index.js file will just load that library from the filesystem and do the work of exporting things properly.
var fs = require('fs');
// Read and eval library
filedata = fs.readFileSync('./node_modules/quadtree/quadtree-lib/quadtree.js','utf8');
eval(filedata);
/* The quadtree.js file defines a class 'QuadTree' which is all we want to export */
exports.QuadTree = QuadTree
Now you can use your quadtree module like any other node module...
var qt = require('quadtree');
qt.QuadTree();
I like this method because there's no need to go changing any of the source code of your 3rd party library--so it's easier to maintain. All you need to do on upgrade is look at their source code and ensure that you are still exporting the proper objects.
There is a much better method than using eval: the vm module.
For example, here is my execfile module, which evaluates the script at path in either context or the global context:
var vm = require("vm");
var fs = require("fs");
module.exports = function(path, context) {
context = context || {};
var data = fs.readFileSync(path);
vm.runInNewContext(data, context, path);
return context;
}
And it can be used like this:
> var execfile = require("execfile");
> // `someGlobal` will be a global variable while the script runs
> var context = execfile("example.js", { someGlobal: 42 });
> // And `getSomeGlobal` defined in the script is available on `context`:
> context.getSomeGlobal()
42
> context.someGlobal = 16
> context.getSomeGlobal()
16
Where example.js contains:
function getSomeGlobal() {
return someGlobal;
}
The big advantage of this method is that you've got complete control over the global variables in the executed script: you can pass in custom globals (via context), and all the globals created by the script will be added to context. Debugging is also easier because syntax errors and the like will be reported with the correct file name.
The simplest way is: eval(require('fs').readFileSync('./path/to/file.js', 'utf8'));
This works great for testing in the interactive shell.
AFAIK, that is indeed how modules must be loaded.
However, instead of tacking all exported functions onto the exports object, you can also tack them onto this (what would otherwise be the global object).
So, if you want to keep the other libraries compatible, you can do this:
this.quadTree = function () {
// the function's code
};
or, when the external library already has its own namespace, e.g. jQuery (not that you can use that in a server-side environment):
this.jQuery = jQuery;
In a non-Node environment, this would resolve to the global object, thus making it a global variable... which it already was. So it shouldn't break anything.
Edit:
James Herdman has a nice writeup about node.js for beginners, which also mentions this.
I'm not sure if I'll actually end up using this because it's a rather hacky solution, but one way around this is to build a little mini-module importer like this...
In the file ./node_modules/vanilla.js:
var fs = require('fs');
exports.require = function(path,names_to_export) {
filedata = fs.readFileSync(path,'utf8');
eval(filedata);
exported_obj = {};
for (i in names_to_export) {
to_eval = 'exported_obj[names_to_export[i]] = '
+ names_to_export[i] + ';'
eval(to_eval);
}
return exported_obj;
}
Then when you want to use your library's functionality you'll need to manually choose which names to export.
So for a library like the file ./lib/mylibrary.js...
function Foo() { //Do something... }
biz = "Blah blah";
var bar = {'baz':'filler'};
When you want to use its functionality in your Node.js code...
var vanilla = require('vanilla');
var mylibrary = vanilla.require('./lib/mylibrary.js',['biz','Foo'])
mylibrary.Foo // <-- this is Foo()
mylibrary.biz // <-- this is "Blah blah"
mylibrary.bar // <-- this is undefined (because we didn't export it)
Don't know how well this would all work in practice though.
I was able to make it work by updating their script, very easily, simply adding module.exports = where appropriate...
For example, I took their file and I copied to './libs/apprise.js'. Then where it starts with
function apprise(string, args, callback){
I assigned the function to module.exports = thus:
module.exports = function(string, args, callback){
Thus I'm able to import the library into my code like this:
window.apprise = require('./libs/apprise.js');
And I was good to go. YMMV, this was with webpack.
A simple include(filename) function with better error messaging (stack, filename etc.) for eval, in case of errors:
var fs = require('fs');
// circumvent nodejs/v8 "bug":
// https://github.com/PythonJS/PythonJS/issues/111
// http://perfectionkills.com/global-eval-what-are-the-options/
// e.g. a "function test() {}" will be undefined, but "test = function() {}" will exist
var globalEval = (function() {
var isIndirectEvalGlobal = (function(original, Object) {
try {
// Does `Object` resolve to a local variable, or to a global, built-in `Object`,
// reference to which we passed as a first argument?
return (1, eval)('Object') === original;
} catch (err) {
// if indirect eval errors out (as allowed per ES3), then just bail out with `false`
return false;
}
})(Object, 123);
if (isIndirectEvalGlobal) {
// if indirect eval executes code globally, use it
return function(expression) {
return (1, eval)(expression);
};
} else if (typeof window.execScript !== 'undefined') {
// if `window.execScript exists`, use it
return function(expression) {
return window.execScript(expression);
};
}
// otherwise, globalEval is `undefined` since nothing is returned
})();
function include(filename) {
file_contents = fs.readFileSync(filename, "utf8");
try {
//console.log(file_contents);
globalEval(file_contents);
} catch (e) {
e.fileName = filename;
keys = ["columnNumber", "fileName", "lineNumber", "message", "name", "stack"]
for (key in keys) {
k = keys[key];
console.log(k, " = ", e[k])
}
fo = e;
//throw new Error("include failed");
}
}
But it even gets dirtier with nodejs: you need to specify this:
export NODE_MODULE_CONTEXTS=1
nodejs tmp.js
Otherwise you cannot use global variables in files included with include(...).
Is it easy/possible to do a simple include('./path/to/file') type of command in node.js?
All I want to do is have access to local variables and run a script. How do people typically organize node.js projects that are bigger than a simple hello world? (A fully functional dynamic website)
For example I'd like to have directories like:
/models
/views
... etc
Just do a require('./yourfile.js');
Declare all the variables that you want outside access as global variables.
So instead of
var a = "hello" it will be
GLOBAL.a="hello" or just
a = "hello"
This is obviously bad. You don't want to be polluting the global scope.
Instead the suggest method is to export your functions/variables.
If you want the MVC pattern take a look at Geddy.
You need to understand CommonJS, which is a pattern to define modules. You shouldn't abuse GLOBAL scope that's always a bad thing to do, instead you can use the 'exports' token, like this:
// circle.js
var PI = 3.14; // PI will not be accessible from outside this module
exports.area = function (r) {
return PI * r * r;
};
exports.circumference = function (r) {
return 2 * PI * r;
};
And the client code that will use our module:
// client.js
var circle = require('./circle');
console.log( 'The area of a circle of radius 4 is '
+ circle.area(4));
This code was extracted from node.js documentation API:
http://nodejs.org/docs/v0.3.2/api/modules.html
Also, if you want to use something like Rails or Sinatra, I recommend Express (I couldn't post the URL, shame on Stack Overflow!)
If you are writing code for Node, using Node modules as described by Ivan is without a doubt the way to go.
However, if you need to load JavaScript that has already been written and isn't aware of node, the vm module is the way to go (and definitely preferable to eval).
For example, here is my execfile module, which evaluates the script at path in either context or the global context:
var vm = require("vm");
var fs = require("fs");
module.exports = function(path, context) {
var data = fs.readFileSync(path);
vm.runInNewContext(data, context, path);
}
Also note: modules loaded with require(…) don't have access to the global context.
If you are planning to load an external javascript file's functions or objects, load on this context using the following code – note the runInThisContext method:
var vm = require("vm");
var fs = require("fs");
var data = fs.readFileSync('./externalfile.js');
const script = new vm.Script(data);
script.runInThisContext();
// here you can use externalfile's functions or objects as if they were instantiated here. They have been added to this context.
Expanding on #Shripad's and #Ivan's answer, I would recommend that you use Node.js's standard module.export functionality.
In your file for constants (e.g. constants.js), you'd write constants like this:
const CONST1 = 1;
module.exports.CONST1 = CONST1;
const CONST2 = 2;
module.exports.CONST2 = CONST2;
Then in the file in which you want to use those constants, write the following code:
const {CONST1 , CONST2} = require('./constants.js');
If you've never seen the const { ... } syntax before: that's destructuring assignment.
Sorry for resurrection. You could use child_process module to execute external js files in node.js
var child_process = require('child_process');
//EXECUTE yourExternalJsFile.js
child_process.exec('node yourExternalJsFile.js', (error, stdout, stderr) => {
console.log(`${stdout}`);
console.log(`${stderr}`);
if (error !== null) {
console.log(`exec error: ${error}`);
}
});