I am making console utility which accepts a path to configuration file as a console argument.
F.e: utility -f /path/to/file.js
I need to require this file to read configuration. Is it possible to handle this with webpack? As I understand context can not help me in this situation.
Thanks.
P.S. I'm already using webpack.
P.S Solution is to use something like: eval('require')(dynamicPath)
Webpack can only do a dynamic require like this if the file to be required is available at compile time. For example, if you require a "dynamic" file path, like
require('./assets/images/' + someVariable + '.png')
Then under the hood, Webpack will include all images matching that pattern in the bundled require code. It basically will include all files matching the regex:
/.\/assets\/images\/*.png/
I would probably try putting the config file in a specific folder, and using require on that folder:
require('./configs/' + process.env.CONFIG_NAME);
This way Webpack will only include all files in the configs folder.
The Webpack documentation is horrible but there is more information on the dynamic requires page.
If you are passing in a config file as an argument to a node process, it will be accessible in the process.argv array of command line arguments. I don't know if you are using some other framework (like the excellent commander) to help with making command line programs, but we can just slice the array to find what we need.
To resolve a path from the directory the script is launched in, you can use process.cwd() - this returns an absolute path to the working directory of the node process.
Finally you can use path.resolve(processPath, configPath) (docs) to generate a path that is always guaranteed to resolve to the config. You can then require this path.
You probably need to do this first. The top of your file could look something like this:
/* relevant require() calls for necessary modules */
var path = require('path');
// first two arguments are node process and executed file
var args = process.argv.slice(2);
var configIndex = args.findIndex('-f') + 1;
var configPath = path.resolve(process.cwd(), args[configIndex]);
var config = require(configPath);
/* the rest of your code */
Related
I have a question regarding Nodejs's path-handling.
I know that they have a page about that in the docs but it didnt contain what I needed.
So basically, I have a file that includes a relative path referencing a file (png in this case).
Now, based on where I call the file from, the picture is either found or not found (as the point in the fileSystem from where its called changes).
I am using the 'sharp' framework, 'sharp('./picture.png')' is similar to require.
Example:
File 'render.js' :
const pic = sharp('./picture.png')
Calling:
cmd\examplePath> node render.js //picture is found
cmd> node ./examplePath/render.js //picture is not found
The location of the picture relative to the file stays the same at all times!
My question now is if what I have described is to be expected from Nodejs or if there is something wrong. What would I need to do to be able to call the file from anywhere and have it still work?
Any tips are appreciated.
Thanks :)
Normally file handling in nodejs such a fs.open() just resolves a relative path versus the current working directory in nodejs. The current working directory will be whatever the OS working directory was when you started your nodejs app.
The current working directory is not necessarily the same as the directory where your script is located because the current working directory might be different than where your script is located when your nodejs program was started.
So, in your two command line examples, each is starting with a different current working directory, thus one works and one doesn't.
In general in nodejs, it is not advisable to rely on what the current working directory is because this lessens the ability to reuse your code in other projects where it might be loaded from a different directory. So, if the file you are trying to reference is in a known location relative to your script's file system location, then you would typically build a full path, using either __dirname (in CommonJS modules) or import.meta.url or import.meta.resolve() (in ESM modules).
For example, if the image is in the same directory as your script, then you could do this in a CommonJS module:
const path = require('path');
const fullPath = path.join(__dirname, "picture.png");
const pic = sharp(fullPath);
In an ESM module, you could do:
import path from 'path';
const __dirname = new URL('.', import.meta.url).pathname;
const fullPath = path.join(__dirname, "picture.png");
const pic = sharp(fullPath);
My question now is if what I have described is to be expected from Nodejs or if there is something wrong.
Yes, that is to be expected. Your are starting your program with different current working directories and have code that depends upon the current working directory being a certain value (because of your usage of a relative path).
What would I need to do to be able to call the file from anywhere and have it still work?
Build an absolute path using your script's directory as the anchor location as shown above.
Note that require() or import use completely different file searching logic and have some built-in behavior that is relative to the module's location that is running them. But, they are the exception, not the rule. Operations in the fs module (or other modules that use the fs module) use the current working directory as the base path if you supply a relative path.
I have a NodeJS module that is supposed to translate an XML to HTML via an XSL file. The problem is I need to bundle the XSL with the rest of the module.
Right now it's inside the source folder and I use it via:
function createHtml(xml) {
const fs = require('fs');
const xsltProcessor = require('xslt-processor');
var xsl = fs.readFileSync('./src/xsl/html-export.xsl', 'utf-8');
return xsltProcessor.xsltProcess(xsltProcessor.xmlParse(xml), xsltProcessor.xmlParse(xsl));
}
Of course I made sure to have this folder in the files property of the package.json:
"files": [
"src"
],
This setup works inside the module, but not when I publish the module and require it from a second module. Then I need to copy the XSL file into the second module's path as well.
That's not really working for me. I want to access the file relative to the JavaScript source file, not the current NodeJS application.
I tried using a different kind of path (html-export.xsl, ./html-export.xsl, ./xsl/html-export.xsl) but nothing else works even inside the module. So I can't publish it to try it out elsewhere.
The good news is: the XSL file is actually included inside the module during the publishing; when I install it the module in the node_modules/ folder contains the XSL. So it's "just" the above code that's not working, because the path is resolved relative to the second module (i.e. the one with the require), not relative to the original one (i.e. the one inside the node_modules/).
How do I handle assets in NodeJS so they are bundled with the module they belong to?
This question seems related, but the answer is exactly what I did, and it doesn't work for me for some reason.
It appears you only want to use the .xsl file internally in your package, which is something that I originally misunderstood..
In order to accomplish this, you can use the path package to resolve the path of your .xsl file.
Like this:
function createHtml(xml) {
const fs = require('fs');
const path = require('path');
const xsltProcessor = require('xslt-processor');
var xsl = fs.readFileSync(path.resolve(__dirname, './src/xsl/html-export.xsl'), 'utf-8');
return xsltProcessor.xsltProcess(xsltProcessor.xmlParse(xml), xsltProcessor.xmlParse(xsl));
}
In order to demonstrate how to do this, I have built an example NPM package for you to review. While this package doesn't use an .xsl file, it uses a .txt file - the logic remains the same.
The readme file within the package contains instructions on how to use it..
You can find the package on NPM here
You can find the source code to the package here
I have some nodejs code running in a module somewhere. From this module, I would like to load a module located in a completely different place in the file system -- for example, given the path "/another/dir" and the module name "foo", I would like Node to act as if a module running in /another/dir had called require("foo"), rather than my own module.
My code is running here
/some/folder/node_modules/mine/my_module.js
I have the path "/another/dir/", the string "foo",
and want to load this module
/another/dir/node_modules/foo/index.js
In other words, the module documentation refers to the process "require(X) from module at path Y", and I would like to specify my own value for Y
Can this be accomplished? If so, how? If not, why not?
The simplest, is just to resolve the path into an absolute path, this will be the recommended approach for most if not all cases.
var path = require('path');
var basedir = '/another/dir';
var filename = 'foo'; // renamed from dirname
var filepath = path.join(basedir, 'node_modules', filename);
var imports = require(filepath);
If you really need to make require act as if it is in a different directory,
you can push the base directory to module.paths
module.paths.unshift('/another/dir/node_modules');
var imports = require('foo');
module.paths.shift();
module.paths can also be modified externally via the environment variable NODE_PATH, tho that would be the least recommended approach but this does apply it globally across all modules.
A symlink with npm link
To avoid problems or modify source code I would use npm link, from your example:
First:
cd /another/dir/node_modules/foo # go into the package directory
npm link # creates global link
This will create a global link for the foo module, on Linux you need root permissions to do this.
Then:
cd /some/folder/ # go into some other package directory.
npm link foo # link-install the package
/some/folder/package.json should contain foo as a dep, is not mandatory, without it you get an extraneous warning with npm ls:
"dependencies": {
[...]
"foo": "*"
}
No symlink with local NODE_PATH
You don't like symlinks ? You can still use NODE_PATH but locally instead setting a global variable as #rocketspacer suggested, because as he rightly stated, it's not recommended to use it globally.
Note: In any case I would use a User variable and not a System-wide variable, a co-worker could log with a different username on the same machine and still get a modified NODE_PATH.
But to do this locally for just one invocation on Linux you can simply call:
NODE_PATH=$NODE_PATH:/another/dir/node_modules npm start
It will use that NODE_PATH only for that invocation.
Same one time invocation on Windows:
#ECHO OFF
SET BASE_NODE_PATH=%NODE_PATH%
SET NODE_PATH=%BASE_NODE_PATH%;C:\another\dir\node_modules\
node index.js
SET NODE_PATH=%BASE_NODE_PATH%
And...
You could also use a local dep like:
"dependencies": {
"foo": "file:/another/dir/node_modules/foo"
}
But would require an npm install and it would copy the content of foo in the current package node_modules folder.
It's quite easy to achieve, just add absolute paths to module object
in your current script /some/folder/node_modules/mine/my_module.js add this
module.paths.push('/another/dir/node_modules');
//this must be absolute, like /Users/John/another/dir/node_modules
var foo = require('foo');
For demo, open node in terminal, and type module.paths, it will show all the path node will search for require, you just add your path
"require(X) from module at path Y"
Means calling require(X) from within a file located at path Y. So you practically can't change Y.
Although the above solutions work (modifying module.paths & requiring absolute path), you'll have to add those snippets to every file in your project where you need requiring the foreign modules.
Even more, modifying module.paths is not officially supported. So it's not guaranteed to work in future updates of node
Introducing NODE_PATH
NODE_PATH is an environment variable that is set to a colon-delimited list of absolute paths.
Within those absolute paths, Node.js will search for a module that matches your require statement when all else has failed (Having indexed node_modules up to File System Root and still no match was found)
It is officially supported, although not recommended as it goes against convention (Your co-worker may not be aware of the NODE_PATH usage as there is no descriptive way of telling that in your project itself)
Notes:
On Windows, NODE_PATH is delimited by semicolons instead of
colons.
Node doesn't look for node_modules within those paths like
its default behavior, so your paths should be exactly where you
contain needed modules. For example: "/another/dir/node_modules"
Detailed information can be found on NodeJs official document:
https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders
For those of you who are unaware of what environment variables is and how to set them, I have a windows screenshot that will get you started
In simple lines, u can call your own folder as module :
For that we need: global and app-module-path module
here "App-module-path" is the module ,it enables you to add additional directories to the Node.js module search path And "global" is, anything that you attach to this object will b available everywhere in your app.
Now take a look at this snippet:
global.appBasePath = __dirname;
require('app-module-path').addPath(appBasePath);
__dirname is current running directory of node.You can give your own path here to search the path for module.
I'm writing a javascript library that contains a core module and several
optional submodules which extend the core module. My target is the browser
environment (using Browserify), where I expect a user of my module will only
want to use some of my optional submodules and not have to download the rest to
the client--much like custom builds work in lodash.
The way I imagine this working:
// Require the core library
var Tasks = require('mymodule');
// We need yaks
require('mymodule/yaks');
// We need razors
require('mymodule/razors');
var tasks = new Tasks(); // Core mymodule functionality
var yak = tasks.find_yak(); // Provided by mymodule/yaks
tasks.shave(yak); // Provided by mymodule/razors
Now, imagine that the mymodule/* namespace has tens of these submodules. The
user of the mymodule library only needs to incur the bandwidth cost of the
submodules that she uses, but there's no need for an offline build process like
lodash uses: a tool like Browserify solves the dependency graph for us and
only includes the required code.
Is it possible to package something this way using Node/npm? Am I delusional?
Update: An answer over here seems to suggest that this is possible, but I can't figure out from the npm documentation how to actually structure the files and package.json.
Say that I have these files:
./lib/mymodule.js
./lib/yaks.js
./lib/razors.js
./lib/sharks.js
./lib/jets.js
In my package.json, I'll have:
"main": "./lib/mymodule.js"
But how will node know about the other files under ./lib/?
It's simpler than it seems -- when you require a package by it's name, it gets the "main" file. So require('mymodule') returns "./lib/mymodule.js" (per your package.json "main" prop). To require optional submodules directly, simply require them via their file path.
So to get the yaks submodule: require('mymodule/lib/yaks'). If you wanted to do require('mymodule/yaks') you would need to either change your file structure to match that (move yaks.js to the root folder) or do something tricky where there's a yaks.js at the root and it just does something like: module.exports = require('./lib/yaks');.
Good luck with this yak lib. Sounds hairy :)
This should be a very simple question. I think it might be as simple as just being the convention, but I would like to check as I have no idea if there is anything else behind it or what phrases to even search regarding it.
var hello = require('./hello');
hello.world();
Imagine the above code. The require path is prefixed by a ./ this is always present for files in the same folder. Why not just the file name?
Comparitively the common use
var http = require('http');
Is not prefixed by a ./ I am currently assuming this is due to the http file being a "native" module. Therefore would I be right in saying that anything without ./ is looking in the native Node namespace and anything with a ./ is looking for a local file?
Also would a file in a higher directory like in PHP it would be ../
In Node would it be .././ or ./../
Yeah, this is a simple convention used in node. Please see the module.require docs
And for what it's worth, you won't always be using require("./hello"). Sometimes you'll be using require("../../foo") or require("../").
Simply put,
You use a path for requiring files within your module
You use a string identifier for including other modules