Node.js require webpacked modules - javascript

I'm trying to get rid of the thousand files you get once you npm install various modules having their own dependencies.
Thus I was thinking of compiling only the libraries using webpack into one javascript file (and other required resources), then loading it to the Node.js project this way :
Entry point, that will be compiled to bundle by webpack.
module.exports = {
lodash : require('lodash'),
colors : require('colors'),
test : require('test'),
abc : require('abc')
} ;
Main
var { lodash, colors, test, abc } = require('./lib/bundle') ;
The problem I got is that some modules require system (or uncompilable) modules, such as fs, and webpack tries to bundle them to.
You just have to specify in the webpack.config.js file :
node: {
fs : "empty",
electron : "empty"
}
However, once packed into bundle, it seems that every require('fs') is replaced by Object.freeze({}) because of this setting, and then the modules fail using fs.
Would anyone have a solution for using packed modules in a Node.js project ?
P.S.: I tried using yarn with yarn autoclean --force to remove all unnecessary files, but it only removed 5% to 10% of the total.

The problem using the current node config object and set fs: 'empty' is that it will provide an empty object for those modules. More info about Webpack Node here.
You can set the Webpack target property to 'node'
Compile for usage in a Node.js-like environment (uses Node.js require to load chunks and not touch any built in modules like fs or path)
module.exports = {
target: 'node'
};
Read more about Webpack Targets
Also, to import a built-in module, use __non_webpack_require__
Generates a require function that is not parsed by webpack

Related

Cannot find module an external js whith fullpath after build a bundle whith Webpack and Babel in NodeJS

I build a bundle with all the needed resources with Webpack including node_modules because Im going to run this bundle in another place where the package.json and node_modules not exist, thats the reason why Im building the bundle including node_modules.
In some moment, the bundle needs to require an external js that is downloaded with a dynamic name, if I move the bundle to the final location and I run it with Node, when it try to require('dynamic_fullpath.js'), the log tells Error: Cannot find module dynamic_fullpath.js, the file(in this case: dynamic_fullpath.js) exists in the right path.
I think the problem is: Webpack changes the require js methods to require webpack methods and when builds the bundle: the dynamic_fullpath.js does not exist and dont add to the bundle.
Any idea how to resolve this dynamic require?
Finally the problem as supposed: Webpack changes the require js methods to require webpack methods and when builds the bundle: the dynamic_fullpath.js does not exist and dont add to the bundle, so, to avoid that on an specific require I found this solution, and as the post said:
"
A simpler way to pull this off without resorting to eval is:
const requireFunc = typeof __webpack_require__ === "function" ? __non_webpack_require__ : require;
const foo = requireFunc(moduleName);
In the bundled output, this will become
const requireFunc = true ? require : require;
const foo = requireFunc(moduleName);
"
The solution was found here

webpack & using node modules in an isomorphic package

I am building an isomorphic package where I am using a flag in various files to detect if we are in the browser or in node. If in node, I require an internal package ie if (isNode) { require("/.nodeStuff) } that has as one of its dependencies the fs module. However, webpack does not like this for obvious reasons. Is there any type of module-based webpack config file that I can configure to ignore the node-based requires entirely so that this does not happen?
First option
As stated in the docs, in order to solve this isomorphic problem you could simply run two builds, one for each environment (node and web). The guide can be found here. Keep in mind you should probably mock any built ins in the clientConfig by adding this block
node: { fs: 'empty',//any other node lib used }. That way webpack will not complain and since your client code will be under the !IS_NODE condition the empty fs will never be used.
Although this is a solid solution you end up with 2 bundles and you need of course a way to distribute them to the correct platform each time.
Second way
This solution is based on the not very well known __non_webpack_require__ function. This is a webpack specific function that will instruct the parser to avoid bundling this module that is being requested and assume that a global require function is available. This is exactly what happens while running in node instead of a browser.
//webpack.config.js
{
mode: "development",
entry: './index.js',
output: {
path: path.resolve(__dirname, 'dist'),
filename: '[name].js'
},
node: false
}
// nodeStuff.js
const fs = __non_webpack_require__('fs'); //this will be transformed to require('fs')
fs.writeFileSync('some','thing)
That way since nodeStuff.js will only be required under the IS_NODE condition, the native require will be available.
I would suggest to use __non_webpack_require__ on native libraries only, that you are sure that will be available!

Accessing module.exports with a browserified node app

We are trying to Browserify our node app
Sample File (index.js)
module.exports = {
index: () => 'test',
};
Browserify command
browserify src/index.js > dist/bundle.js --node
If we use a file to require and console
console.log(require('src/index')); // { index: [Function: index] }
console.log(require('dist/bundle')); // { }
Our expectation is that bundle.js would export the same as index.js.
Can anyone point us at what we are doing wrong or missing?
Additional Info
~This is not our app, this is a sample to demonstrate the issue
We are currently sending our entire app zipped to AWS Lambda with the entry point src/index.index and the objective is to just send the bundle.js file and be able to to have the entry point bundle.index
bundle.js
(function(){function r(e,n,t){function o(i,f){if(!n[i]){if(!e[i]){var c="function"==typeof require&&require;if(!f&&c)return c(i,!0);if(u)return u(i,!0);var a=new Error("Cannot find module '"+i+"'");throw a.code="MODULE_NOT_FOUND",a}var p=n[i]={exports:{}};e[i][0].call(p.exports,function(r){var n=e[i][1][r];return o(n||r)},p,p.exports,r,e,n,t)}return n[i].exports}for(var u="function"==typeof require&&require,i=0;i<t.length;i++)o(t[i]);return o}return r})()({1:[function(require,module,exports){
module.exports = {
index: () => 'test',
};
},{}]},{},[1]);
You need to use the --standalone flag. If I reproduce the setup you describe in your question and execute:
$ browserify src/index.js --standalone mylib > dist/bundle.js
Then I can run an interactive Node session on it and use the library in the way you expect it to be used:
$ node
> require("./dist/bundle").index()
'test'
The --standalone flag tells Browserify to wrap your code in a UMD stub which allows loading the bundle as CommonJS module, an AMD module, or as a plain script (i.e. does not use a module system). The argument you pass with --standalone indicates what name your library will take in the "plain script" case. So in the example above, if you were to load the library in a browser without any module system, you'd be able to run index as mylib.index().
You can use serverless for this, pretty easy to configure. No need to use browserify cli for this.
Keep following following official documentations to setup serverless cli.
Installation
AWS - Credentials
Quick Start
Once everything is setup and you are able to deploy your lambda functions to AWS using serverless cli. Follow following steps to setup browserify.
Install browserify as a dev dependecy.
Install serverless-plugin-browserifier as a dev dependency.
Add the plugin to your serverless.yml file and set package.individually to true. (Ref)
plugins:
- serverless-plugin-browserifier
package:
individually: true
Note: Personally tried this and is working.

Webpack resolve.root and TypeScript loader

Our project is using the webpack resolve.root option to import modules with absolute paths. (avoiding something like ../../../module)
In its current state the project is using babel-loader which works perfectly fine.
My task is to migrate the app to Angular 2.
Therefor I am currently in the process of transitioning to TypeScript.
Somehow it seems like the ts-loader does not work in combination with the resolve.root option of the webpack config.
Example of the webpack.config.js
resolve: {
root: [
path.resolve('./node_modules'),
path.resolve('./app'),
path.resolve('./app/lib'),
]
},
Example of a module import
import AbstractListState from 'states/abstract_list_state';
The states directory is inside the app/lib directory.
Error when executing webpack
ERROR in ./app/mainViews/panel/panel.controller.ts
Module not found: Error: Cannot resolve module 'states/abstract_list_state' in C:\Users\...\Project\app\mainViews\panel
# ./app/mainViews/panel/panel.controller.ts 4:28-65
Pre version 2.0 TypeScript will try to load modules with an absolute path from the node_modules directory. This is because TypeScript's module resultion is per default set to "node". Which means it works like node's require method. So, even if you're using webpack to build your app, TypeScript (and its compiler) will still want to load the files.
In order to let webpack import your modules with absolute path you have to go back and use the require method. This way TypeScript will let webpack import stuff. But of course you will not get any type-inference, autocomplete, ...
Or, you update to the TypeScript 2.0 beta and give this a try: https://github.com/Microsoft/TypeScript/wiki/What%27s-new-in-TypeScript#module-resolution-enhancements-baseurl-path-mapping-rootdirs-and-tracing

Webpack and Nodejs isomorphic require with absolute path

GOAL: I am trying to set up a project in nodejs and webpack such that the require function can use the project directory as root, so I can require with absolute path relative to project root in both environments (isomorphic uses i.e. React server+client render).
SITUATION: In webpack you can set the config.resolve.root to make it work, but in nodejs, its best practice not to override/modify the global.require.
PROPOSITION 1: I can make a new global function
global.p_require
so it works in node; however, I cannot find a way to let webpack parse "p_require" into __webpack_require__ without changing the webpack source code.
PROPOSITION 2: I can make a new global variable
global.ROOT_DIR = process.cwd()
so it works in node by
require(ROOT_DIR + <relative path to root>);
however, webpack would recognize this as dynamic require. Is there a way such that webpack would parse ROOT_DIR? I have already tried the Define Plugin, but it seems to load after require is parsed by webpack.
QUESTION
Anyone has a solution or faces the same issue?
I'm addressing this by letting webpack do more; instead of "node and webpack", it's "webpack: client and server". I have webpack do a build for the client and a build for the server (the latter uses 'node' as its target property in config). It's easy to customize the directories webpack uses to require, so you let it do its work and create a build for node.
When rendering on the server, you just require the compiled server build. If you need to pass some stuff in from the server to the application that webpack built, wire that up in the entry point that you use for the server build -- webpack will build it as a commonJs module, so your entry point can export whatever is the most convenient interface when the server needs to render.

Categories