How to get node package consumer directory from node_modules? - javascript

I am trying to create a simple node module that creates a set of folders in the app that consumes it. I exported a simple createLayout function that creates the folders. I pushed my changes to git and did an npm i from another folder. Lets call the modules creator and consumer for the sake of explanation. When I try to call createLayout in consumer I am running in to several issues. I am in E:\ drive.
Below is the index.js in creator:
import {sync} from 'mkdirp';
export function createLayout(config) {
sync('./folder1');
}
And index.js in consumer:
var createLayout = require('creator').createLayout;
createLayout();
// with config createLayout({path: __dirname})
This results in creating a folder in E:\ not relative to consumer. So I tried including __dirname:
sync(__dirname + '/folder1');
Once again, this also creates a folder in E:\ not relative to consumer. I searched for bit like in various modules to see how they are doing when they are reading the config file, for instance webpack uses process.cwd. So I tried that too.
sync(process.cwd() + '/folder1');
Same, results in creating a folder in E:\ not relative to consumer. Then I tried to pass the __dirname or cwd through a config object.
// get __dirname from the `consumer` in config.path
sync(config.path + '/folder1');
But it ends up in following error:
Error: EPERM: operation not permitted, mkdir 'E:\'
I tried logging all the values in both creator and consumer:
console.log(__dirname, process.cwd(), config.path)
// creator: / / E:\projects\consumer
// consumer: E:\projects\consumer E:\projects\consumer E:\projects\consumer
I am using webpack with babel to pack the creator, plain js in consumer. I do not know what am I doing wrong. I am pretty new to nodejs ways of working.
Update
I am noticing that this is occurring only when I use webpack to build the creator. A simple module.exports works normally as anyone would expect. So I am including my webpack config file:
module.exports = {
entry: [
'./index.js'
],
output: {
filename: 'creator.js',
path: __dirname + '/dist',
library: 'creator',
libraryTarget: 'umd'
},
module: {
loaders: [
{
test: /\.js$/,
exclude: /node_modules/,
loader: 'babel'
}
]
},
externals: {
fs: 'fs'
}
};

Correct solution is adding this line in config:
target: 'node'
this will make webpack to ignore modules like fs and mkdirp and some other.
Now no longer need to specify externals.
Incorrect solution given before:
Just add mkdirp to externals and it will resolve you problem:
externals: {
fs: 'fs',
mkdirp: 'mkdirp'
}

Related

Why is require not defined when using require('fs') in Vue instance inside a beforeCreate hook? [duplicate]

I'm using node.js and webpack to create a bundle. From what I've read, node.js should contain fs module for managing files. However when I call require("fs") I get an Cannot find module "fs" error. What should I do?
I came across this problem myself when bundling with webpack and found the answer on this thread.
The way to solve it for me was to use the following config:
module.exports = {
entry: "./app",
output: {
path: __dirname,
filename: "bundle.js"
},
module: {
loaders: [
{
test: /\.js$/,
exclude: 'node_modules',
loader: 'babel',
query: {presets: ['es2015']},
}
]
},
target: 'node'
};
By setting target to node webpack will make the necessary changes to bundle your node application
Edit: This answer targeted webpack 1.x which has now been superseded.
If you are running your webpack bundle in nodejs environment then target: 'node' is required in webpack.config.js file otherwise webpack takes default value as web for target check here.
You can resolve the issue in two ways
Add below configuration to your webpack.config.js
node: {
fs: "empty"
}
OR
Add below configuration to your package.json
"browser": {
"fs": false
}
Edit:
promising fix is
"browser": {
"fs": false
}
I had the same issue when bundling a NWjs application using webworkers (which in turn had node enabled).
The solution I found was to include each native module I used in externals with the prefix commonjs to the name of the module. For example:
...
target: "webworker", // or 'node' or 'node-webkit'
externals:{
fs: "commonjs fs",
path: "commonjs path"
}
...
I've done the same for targets "webworker" and "node-webkit" in different projects to solve the same issue.
webpack nwjs webworker nodejs node
Add below configuration to your webpack.config.js
resolve: {
fallback: {
fs: false
}
}
I needed to build a class that would use fetch if executed in a browser, or fs if executed in node. For other reasons, it was impractical to produce separate bundles, so I produced a single browser-targeted bundle.
The solution I used was to use eval('require("fs")') if the script was running in node.
const fs = eval('require("fs")')
Browser-safe (fs is null in the browser):
const fs = typeof window === 'object'
? null
: eval('require("fs")')
After trying everything I found on the internet (target, externals, node configs), the only solution that actually worked for me was replacing:
const filesystem = require("fs")
or
import fs from "fs"
by the special webpack version
const fs = __non_webpack_require__("fs")
This generates a require function that is not parsed by webpack.
In addition to the answer of PDG
I'm used to this short copy/paste candy.
Using path and fs :
var nodeModules = {};
fs.readdirSync(path.resolve(__dirname, 'node_modules'))
.filter(x => ['.bin'].indexOf(x) === -1)
.forEach(mod => { nodeModules[mod] = `commonjs ${mod}`; });
// Before your webpack configuration
module.exports = {
...
}
Then inside your configuration file, include the nodeModules variable in the externals
...
externals: nodeModules,
...
It would be more elegant to use pre-defined solution as:
Adding target: 'node' to webpack config file.
More info on: official documentation
For the solution we are building we had to force an older version of webpack:
npm install --save --force webpack#webpack-3

How to avoid webpack bundle all files in one? [duplicate]

So right now I'm working with a prototype where we're using a combination between webpack (for building .tsx files and copying .html files) and webpack-dev-server for development serving. As you can assume we are also using React and ReactDOM as a couple of library dependencies as well. Our current build output is the following structure:
dist
-favicon.ico
-index.html
-main.js
-main.js.map // for source-mapping between tsx / js files
This places ALL of the modules (including library dependencies into on big bundled file). I want the end result to look like this:
dist
-favicon.ico
-index.html
-appName.js
-appName.min.js
-react.js
-react.min.js
-reactDOM.js
-reactDOM.min.js
I have references to each of the libraries in index.html and in import statements in the .tsx files. So my question is this...
How do I go from webpack producing this gigantic bundled .js file to individual .js files (libraries included, without having to specify each individually)? **Bonus: I know how to do prod/dev environment flags, so how do I just minify those individual files (again without bundling them)?
current webpack.config:
var webpack = require("webpack"); // Assigning node package of webpack dependency to var for later utilization
var path = require("path"); // // Assigning node package of path dependency to var for later utilization
module.exports = {
entry: [
"./wwwroot/app/appName.tsx", // Starting point of linking/compiling Typescript and dependencies, will need to add separate entry points in case of not deving SPA
"./wwwroot/index.html", // Starting point of including HTML and dependencies, will need to add separate entry points in case of not deving SPA
"./wwwroot/favicon.ico" // Input location for favicon
],
output: {
path: "./dist/", // Where we want to host files in local file directory structure
publicPath: "/", // Where we want files to appear in hosting (eventual resolution to: https://localhost:4444/)
filename: "appName.js" // What we want end compiled app JS file to be called
},
// Enable sourcemaps for debugging webpack's output.
devtool: "source-map",
devServer: {
contentBase: './dist', // Copy and serve files from dist folder
port: 4444, // Host on localhost port 4444
// https: true, // Enable self-signed https/ssl cert debugging
colors: true // Enable color-coding for debugging (VS Code does not currently emit colors, so none will be present there)
},
resolve: {
// Add '.ts' and '.tsx' as resolvable extensions.
extensions: [
"",
".ico",
".js",
".ts",
".tsx",
".web.js",
".webpack.js"
]
},
module: {
loaders: [
// This loader copies the index.html file & favicon.ico to the output directory.
{
test: /\.(html|ico)$/,
loader: 'file?name=[name].[ext]'
},
// All files with a '.ts' or '.tsx' extension will be handled by 'ts-loader'.
{
test: /\.tsx?$/,
loaders: ["ts-loader"]
}
],
preLoaders: [
// All output '.js' files will have any sourcemaps re-processed by 'source-map-loader'.
{
test: /\.js$/,
loader: "source-map-loader"
}
]
},
// When importing a module whose path matches one of the following, just
// assume a corresponding global variable exists and use that instead.
// This is important because it allows us to avoid bundling all of our
// dependencies, which allows browsers to cache those libraries between builds.
// externals: {
// "react": "React",
// "react-dom": "ReactDOM",
// "redux": "Redux"
// }
};
Change the output setting to be name driven e.g.
entry: {
dash: 'app/dash.ts',
home: 'app/home.ts',
},
output: {
path: './public',
filename: 'build/[name].js',
sourceMapFilename: 'build/[name].js.map'
},
To expand upon #basarat's answer, you can use the glob package from node's standard library to build the "entry" config:
const glob = require("glob");
module.exports = [
{
target: "node",
entry: glob.sync("./src/**/*.test.{ts,tsx}").reduce((acc, file) => {
acc[file.replace(/^\.\/src\//, "")] = file;
return acc;
}, {}),
output: {
filename: "[name].js",
chunkFilename: "[name]-[id].js",
path: __dirname + "/dist"
},
//...
}
];
This builds files with the same name as their source, replacing .ts and .tsx with .js.
OPs answer copied out of the question
Ended up finding a solution that fit my needs, although, again, in that webpack-y way, requires some additional configuration. Still would like to make it a little more dynamic, but will perfect this at a later point in time. The resolution I was looking for was the ability to "chunk" common modules, but I stated it as filename given "entry"-points provided in webpack. I didn't mind some files being combined, where it made sense, but wanted overall files to be at a component-level given the project wasn't a SPA (single page application).
The additional code ended up being:
plugins: [
new webpack.optimize.CommonsChunkPlugin({ // This plugin is for extracting and created "chunks" (extracted parts of the code that are common and aren't page specific)
// One of these instances of plugins needs to be specified for EACH chunk file output desired
filename: "common.js", // Filename for this particular set of chunks to be stored
name: "common", // Entry point name given for where to pull all of the chunks
minChunks: 3 // Minimum number of chunks to be created
})
]
I also had to parameterize the entry points (see below for example), by variable name so that I could assign react, react-dom, and redux modules to common.js file.
entry: {
main: "./wwwroot/app/appName.tsx", // Starting point of linking/compiling Typescript and dependencies, will need to add separate entry points in case of not deving SPA
index: "./wwwroot/index.html", // Starting point of including HTML and dependencies, will need to add separate entry points in case of not deving SPA
favicon: "./wwwroot/favicon.ico", // Input location for favicon
common: [ "react", "react-dom", "redux" ] // All of the "chunks" to extract and place in common file for faster loading of common libraries between pages
},

creating javascript library with webpack

I don't understand why this is being so complicated I want my project to have 2 separate work spaces where one is a library that will be distributed and the other will be used for testing... this is how i have the file structure
project
--engine
---math
----vec2.js
---dist
----library.js
---main.js
--sandbox
---main.js
I want to build the "engine" project with webpack and es6 modules so I get a "library" file that can be used in "sandbox".
The "engine" main file would look something like this
import vec2 from './math/vec2';
export default class Library {
constructor() {
this.vec2 = vec2;
}
}
An then the sandbox main file would look something like this
import lib from '../engine/dist/library';
const game = new lib();
The problem is when I build the "library.js" file with webpack and import it in the "sandbox" main file I can't call any of the classes therein. I get this error.
Uncaught TypeError: o.default is not a constructor
at Object.<anonymous> (library.js:1)
at e (library.js:1)
at library.js:1
at library.js:1
My webpack.config.js file looks like this
var webpack = require('webpack');
module.exports = {
context: __dirname,
entry: __dirname+"/main.js",
output: {
path: __dirname+"/dist",
filename: "library.js"
},
module: {
loaders: [
{
test: /\.js$/,
exclude: /(node_modules)/,
loader: 'babel-loader',
query: {
presets: ['es2015']
}
}
]
},
plugins: [
new webpack.optimize.UglifyJsPlugin()
]
};
I must be missing some configuration webpack needs or some plugin that will make this work. I simply want to build the library with webpack using es6 modules so it can be used in another project but I have no idea how to configure it. I'm using babel for transpilling es6 to es5
You need to configure output.libraryTarget. In this case the target commonjs-module is appropriate. So your output would be:
output: {
path: __dirname+"/dist",
filename: "library.js",
libraryTarget: "commonjs-module"
},
The different targets are described in the docs. And you might also want to read Guides - Authoring Libraries.

How can i require an entry point in Webpack?

Im trying to export these two pieces of code cli.js and program.js, where cli depends on program and program has a bunch of other dependencies...
Webpack is doing a great job in bundling all dependencies of program.js (./a,./b,./c...) and correctly ignoring the ones that are externals like 'jquery', 'bluebird' ...
however when it comes to bundle the cli.js .. its not referencing the program.dist.js entry point, but bundling a copy of the entire program once again...
how could i fix this issue? is it a limitation with webpack? or is there any way around it? im currently using webpack 2.1.0-beta.27
this is my webpack.config.js
const path = require('path');
module.exports = {
entry: {
cli: './bin/cli.js',
program: './program.js',
},
target: 'node',
output: {
libraryTarget: 'umd',
filename: '[name].dist.js',
umdNamedDefine: true,
path: path.resolve(__dirname, 'distribution'),
},
externals: [
/^[a-z\-0-9]+$/
]
}
program.js
let a = require('./a'),
b = require('./b'),
c = require('./c');
bin/cli.js
const program = require('../program');
program.doSomething();
just a side node...
I cant split it into chunks with CommonsChunkPlugin because it would make my cli.dist.js unable to be executed by node.js like node cli.dist.js

How to copy static files to build directory with Webpack?

I'm trying to move from Gulp to Webpack. In Gulp I have task which copies all files and folders from /static/ folder to /build/ folder. How to do the same with Webpack? Do I need some plugin?
Requiring assets using the file-loader module is the way webpack is intended to be used (source). However, if you need greater flexibility or want a cleaner interface, you can also copy static files directly using my copy-webpack-plugin (npm, Github). For your static to build example:
const CopyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
context: path.join(__dirname, 'your-app'),
plugins: [
new CopyWebpackPlugin({
patterns: [
{ from: 'static' }
]
})
]
};
Compatibility note: If you're using an old version of webpack like webpack#4.x.x, use copy-webpack-plugin#6.x.x. Otherwise use latest.
You don't need to copy things around, webpack works different than gulp. Webpack is a module bundler and everything you reference in your files will be included. You just need to specify a loader for that.
So if you write:
var myImage = require("./static/myImage.jpg");
Webpack will first try to parse the referenced file as JavaScript (because that's the default). Of course, that will fail. That's why you need to specify a loader for that file type. The file- or url-loader for instance take the referenced file, put it into webpack's output folder (which should be build in your case) and return the hashed url for that file.
var myImage = require("./static/myImage.jpg");
console.log(myImage); // '/build/12as7f9asfasgasg.jpg'
Usually loaders are applied via the webpack config:
// webpack.config.js
module.exports = {
...
module: {
loaders: [
{ test: /\.(jpe?g|gif|png|svg|woff|ttf|wav|mp3)$/, loader: "file" }
]
}
};
Of course you need to install the file-loader first to make this work.
If you want to copy your static files you can use the file-loader in this way :
for html files :
in webpack.config.js :
module.exports = {
...
module: {
loaders: [
{ test: /\.(html)$/,
loader: "file?name=[path][name].[ext]&context=./app/static"
}
]
}
};
in your js file :
require.context("./static/", true, /^\.\/.*\.html/);
./static/ is relative to where your js file is.
You can do the same with images or whatever.
The context is a powerful method to explore !!
One advantage that the aforementioned copy-webpack-plugin brings that hasn't been explained before is that all the other methods mentioned here still bundle the resources into your bundle files (and require you to "require" or "import" them somewhere). If I just want to move some images around or some template partials, I don't want to clutter up my javascript bundle file with useless references to them, I just want the files emitted in the right place. I haven't found any other way to do this in webpack. Admittedly it's not what webpack originally was designed for, but it's definitely a current use case.
(#BreakDS I hope this answers your question - it's only a benefit if you want it)
Webpack 5 adds Asset Modules which are essentially replacements for common file loaders. I've copied a relevant portion of the documentation below:
asset/resource emits a separate file and exports the URL. Previously achievable by using file-loader.
asset/inline exports a data URI of the asset. Previously achievable by using url-loader.
asset/source exports the source code of the asset. Previously achievable by using raw-loader.
asset automatically chooses between exporting a data URI and emitting a separate file. Previously achievable by using url-loader with asset size limit.
To add one in you can make your config look like so:
// webpack.config.js
module.exports = {
...
module: {
rules: [
{
test: /\.(jpe?g|gif|png|svg|woff|ttf|wav|mp3)$/,
type: "asset/resource"
}
]
}
};
To control how the files get output, you can use templated paths.
In the config you can set the global template here:
// webpack.config.js
module.exports = {
...
output: {
...
assetModuleFilename: '[path][name].[hash][ext][query]'
}
}
To override for a specific set of assets, you can do this:
// webpack.config.js
module.exports = {
...
module: {
rules: [
{
test: /\.(jpe?g|gif|png|svg|woff|ttf|wav|mp3)$/,
type: "asset/resource"
generator: {
filename: '[path][name].[hash][ext][query]'
}
}
]
}
};
The provided templating will result in filenames that look like build/images/img.151cfcfa1bd74779aadb.png. The hash can be useful for cache busting etc. You should modify to your needs.
Above suggestions are good. But to try to answer your question directly I'd suggest using cpy-cli in a script defined in your package.json.
This example expects node to somewhere on your path. Install cpy-cli as a development dependency:
npm install --save-dev cpy-cli
Then create a couple of nodejs files. One to do the copy and the other to display a checkmark and message.
copy.js
#!/usr/bin/env node
var shelljs = require('shelljs');
var addCheckMark = require('./helpers/checkmark');
var path = require('path');
var cpy = path.join(__dirname, '../node_modules/cpy-cli/cli.js');
shelljs.exec(cpy + ' /static/* /build/', addCheckMark.bind(null, callback));
function callback() {
process.stdout.write(' Copied /static/* to the /build/ directory\n\n');
}
checkmark.js
var chalk = require('chalk');
/**
* Adds mark check symbol
*/
function addCheckMark(callback) {
process.stdout.write(chalk.green(' ✓'));
callback();
}
module.exports = addCheckMark;
Add the script in package.json. Assuming scripts are in <project-root>/scripts/
...
"scripts": {
"copy": "node scripts/copy.js",
...
To run the sript:
npm run copy
The way I load static images and fonts:
module: {
rules: [
....
{
test: /\.(jpe?g|png|gif|svg)$/i,
/* Exclude fonts while working with images, e.g. .svg can be both image or font. */
exclude: path.resolve(__dirname, '../src/assets/fonts'),
use: [{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: 'images/'
}
}]
},
{
test: /\.(woff(2)?|ttf|eot|svg|otf)(\?v=\d+\.\d+\.\d+)?$/,
/* Exclude images while working with fonts, e.g. .svg can be both image or font. */
exclude: path.resolve(__dirname, '../src/assets/images'),
use: [{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: 'fonts/'
},
}
]
}
Don't forget to install file-loader to have that working.
You can write bash in your package.json:
# package.json
{
"name": ...,
"version": ...,
"scripts": {
"build": "NODE_ENV=production npm run webpack && cp -v <this> <that> && echo ok",
...
}
}
Most likely you should use CopyWebpackPlugin which was mentioned in kevlened answer. Alternativly for some kind of files like .html or .json you can also use raw-loader or json-loader. Install it via npm install -D raw-loader and then what you only need to do is to add another loader to our webpack.config.js file.
Like:
{
test: /\.html/,
loader: 'raw'
}
Note: Restart the webpack-dev-server for any config changes to take effect.
And now you can require html files using relative paths, this makes it much easier to move folders around.
template: require('./nav.html')
I was stuck here too. copy-webpack-plugin worked for me.
However, 'copy-webpack-plugin' was not necessary in my case (i learned later).
webpack ignores root paths
example
<img src="/images/logo.png'>
Hence, to make this work without using 'copy-webpack-plugin'
use '~' in paths
<img src="~images/logo.png'>
'~' tells webpack to consider 'images' as a module
note:
you might have to add the parent directory of images directory in
resolve: {
modules: [
'parent-directory of images',
'node_modules'
]
}
Visit https://vuejs-templates.github.io/webpack/static.html
The webpack config file (in webpack 2) allows you to export a promise chain, so long as the last step returns a webpack config object. See promise configuration docs. From there:
webpack now supports returning a Promise from the configuration file. This allows to do async processing in you configuration file.
You could create a simple recursive copy function that copies your file, and only after that triggers webpack. E.g.:
module.exports = function(){
return copyTheFiles( inpath, outpath).then( result => {
return { entry: "..." } // Etc etc
} )
}
lets say all your static assets are in a folder "static" at the root level and you want copy them to the build folder maintaining the structure of subfolder, then
in your entry file) just put
//index.js or index.jsx
require.context("!!file?name=[path][name].[ext]&context=./static!../static/", true, /^\.\/.*\.*/);
In my case I used webpack for a wordpress plugin to compress js files, where the plugin files are already compressed and need to skip from the process.
optimization: {
minimize: false,
},
externals: {
"jquery": "jQuery",
},
entry: glob.sync('./js/plugin/**.js').reduce(function (obj, el) {
obj[path.parse(el).name] = el;
return obj
}, {}),
output: {
path: path.resolve(__dirname, './js/dist/plugin'),
filename: "[name].js",
clean: true,
},
That used to copy the js file as it is to the build folder. Using any other methods like file-loader and copy-webpack create issues with that.
Hope it will help someone.

Categories