I have somewhere in my code following construction:
var getMenu = function () {
return window.fetch("portal/content/json/menu.json").then(function (data) {
return data.json();
});
};
I tried in my webpack.config.js this:
module: {
loaders: [
...
{
test: /\.json$/,
exclude: /node_modules/,
use: [
'file-loader?name=[name].[ext]&outputPath=portal/content/json'
]
},
...
]
}
Project structure:
dist
content
json
menu.json <- this is missing
src
content
json
menu.json <- source file
Question:
How can webpack copy src/content/json/menu.json to dist/content/json/menu.json ?
You're using fetch to request a JSON file and that will only happen at runtime. Furthermore, webpack only processes anything that is imported. You expected it to handle an argument to a function, but if webpack did that, every argument to a function would be considered a module and that breaks any other use for that function.
If you want your loaders to kick in, you can import the file.
import './portal/content/json/menu.json';
You can also import the JSON and use it directly instead of fetching it a runtime. Webpack 2 uses json-loader by default for all .json files. You should remove the .json rule and you would import the JSON as follows.
import menu from './portal/content/json/menu.json';
menu is the same JavaScript object that you would get from your getMenu function.
if you'd like your json to be loaded in runtime/deferred you can use awesome webpack's dynamic imports feature:
import(
/* webpackChunkName: "json_menu" */
'./portal/content/json/menu.json'
);
it will return a Promise which resolves to the module object, with "default" field containing your data. So you might want something like this (with es6 it looks really nice):
import(
/* webpackChunkName: "json_menu" */
'./portal/content/json/menu.json'
).then(({default: jsonMenu}) => {
// do whatever you like with your "jsonMenu" variable
console.log('my menu: ', jsonMenu);
});
Notice that dynamic imports require a babel plugin syntax-dynamic-import, install it with npm:
npm i babel-plugin-syntax-dynamic-import -D
Have a nice day
Related
I'm working on an extension system for my web app. Third-party developers should be able to extend the app by providing named AMD modules exporting constants and functions following a predefined spec and bundled into a single .js JavaScript file.
Example JavaScript bundle:
define('module1', ['exports', 'module3'], (function (exports, module3) {
exports.spec = 'http://example.com/spec/extension/v1'
exports.onRequest = function (request) { return module3.respond('Hello, World.') }
}));
define('module2', ['exports', 'module3'], (function (exports, module3) {
exports.spec = 'http://example.com/spec/extension/v1'
exports.onRequest = function (request) { return module3.respond('Foo. Bar.') }
}));
define('module3', ['exports'], (function (exports) {
exports.respond = function (message) { return { type: 'message', message: message } }
}));
In the above example module1 and module2 are extension modules (identified by the spec export) and module3 is a shared dependency (e.g. coming from an NPM package). Extension bundles will be loaded in a worker within a sandboxed iframe to seal of the untrusted code in the browser.
Example TypeScript source:
// module1.ts
import respond from 'module3'
export const spec = 'http://example.com/spec/extension/v1'
export const onRequest = (request: Request): Response => respond('Hello, World.')
// module2.ts
import respond from 'module3'
export const spec = 'http://example.com/spec/extension/v1'
export const onRequest = (request: Request): Response => respond('Foo. Bar.')
// module3.ts
import dep from 'some-npm-package'
export respond = (message: string) => dep.createMessageObject(message)
Here is my list of requirements to bundling:
All necessary dependencies (e.g. shared module, NPM package logic) must be included in the bundle
The source code needs to be transpiled to browser compatible code if necessary
The AMD format is required by the custom extension loader implementation
The AMD modules must not be anonymous as the module file names are lost while bundling
No relative paths must be used among dependencies (e.g. ./path/to/module3 instead of module3)
The result should be one JavaScript bundle, thus ONE JavaScript file and ONE sourcemaps file
What's the easiest way to do this?
This is the closest solution I found using rollup and the following rollup.config.js:
import { nodeResolve } from '#rollup/plugin-node-resolve'
import { terser } from 'rollup-plugin-terser'
import typescript from '#rollup/plugin-typescript'
export default {
input: [
'src/module1.ts',
'src/module2.ts'
],
output: {
dir: 'dist',
format: 'amd',
sourcemap: true,
amd: {
autoId: true
}
},
plugins: [
typescript(),
nodeResolve(),
terser()
]
}
From this I get the desired named AMD modules (one for each entry point and chunk) in separate .js files. Problems:
Some dependencies are referenced by ./module3 while being named module3.
The modules appear in separate JavaScript and Sourcemap files instead of being concatenated into a single bundle.
Questions:
Is there an easy fix to the above rollup.config.js config to solve the problem?
I tried to write a small rollup plugin but I failed to get the final AMD module code within it to concatenate it to a bundle. Only the transpiled code is available to me. In addition I don't know how to handle sourcemaps during concatenation.
Is there an alternative to rollup better suited to this bundling scenario?
The big picture: Am I completely on the wrong track when it comes to building an extension system? Is AMD the wrong choice?
I found a way to extend the rollup.config.js mentioned in the question with a custom concatChunks rollup plugin to bundle multiple AMD chunks within a single file and having the source maps rendered, too. The only issue I didn't find an answer to was the relative module names that kept popping up. However, this may be resolved in the AMD loader.
Here's the full rollup.config.js that worked for me:
import Concat from 'concat-with-sourcemaps'
import glob from 'glob'
import typescript from '#rollup/plugin-typescript'
import { nodeResolve } from '#rollup/plugin-node-resolve'
import { terser } from 'rollup-plugin-terser'
const concatChunks = (
fileName = 'bundle.js',
sourceMapFileName = 'bundle.js.map'
) => {
return {
name: 'rollup-plugin-concat-chunks',
generateBundle: function (options, bundle, isWrite) {
const concat = new Concat(true, fileName, '\n')
// Go through each chunk in the bundle
let hasSourceMaps = false
Object.keys(bundle).forEach(fileId => {
const fileInfo = bundle[fileId]
if (fileInfo.type === 'chunk') {
let hasSourceMap = fileInfo.map !== null
hasSourceMaps = hasSourceMaps || hasSourceMap
// Concat file content and source maps with bundle
concat.add(
fileInfo.fileName,
fileInfo.code,
hasSourceMap ? JSON.stringify(fileInfo.map) : null
)
// Prevent single chunks from being emitted
delete bundle[fileId]
}
})
// Emit concatenated chunks
this.emitFile({
type: 'asset',
name: fileName,
fileName: fileName,
source: concat.content
})
// Emit concatenated source maps, if any
if (hasSourceMaps) {
this.emitFile({
type: 'asset',
name: sourceMapFileName,
fileName: sourceMapFileName,
source: concat.sourceMap
})
}
}
}
}
export default {
input: glob.sync('./src/*.{ts,js}'),
output: {
dir: 'dist',
format: 'amd',
sourcemap: true,
amd: {
autoId: true
}
},
plugins: [
typescript(),
nodeResolve(),
terser(),
concatChunks()
]
}
Please make sure you npm install the dependencies referenced in the import statements to make this work.
Considering the big picture, i.e. the extension system itself, I am moving away from a "one AMD module equals one extension/contribution" approach, as current developer tools and JavaScript bundlers are not ready for that (as this question shows). I'll go with an approach similar to the Visual Studio Code Extension API and will use a single "default" module with an activate export to register contributions a bundle has to offer. I hope that this will make extension bundling an easy task no matter what tools or languages are being used.
So I have resources that I hash with copy webpack plugin.
{
from: "data/json/*.json",
to: "data/json/[name].[hash:6].json",
},
Now during runtime I need to get the access to the actual url of these json files.
What I would ideally like is to be able to fetch this url during runtime so that I can do something like
let name = "tiles";
const tileDataUrl = requireUrl(`data/json/${name}.json`);
fetch(tileDataUrl) // tileData Url here would data/json/tiles.abc34f.json
...
What I need is a method requireUrl (or whatever it might be called) which returns the actual url of the static resources with hashes during runtime.
( For anyone wondering, the hashes are used to do cache busting here)
Please and thank you :)
Assuming you're on version 5, Webpack asset modules will provide what you want without the need for copy-webpack-plugin. Webpack can recognize a require statement with expressions. Webpack will automatically included all possibly matching files for you without additional configuration. In this case you may want to watch out for optimizations where Webpack knows that name is set to "tiles". Here's the required addition to your config:
module.exports = {
module: {
rules: [
{
test: /data\/json\/.+\.json$/
type: 'asset/resource',
generator: {
// Look at https://webpack.js.org/configuration/output/#template-strings to see additional template strings.
filename: '[path][name].[hash:6][ext]'
}
}
]
}
}
Alternatively for Webpack 4 you can add file-loader as a dependency and use it with this equivalent config addition:
module.exports = {
module: {
rules: [
{
test: /data\/json\/.+\.json$/
loader: 'file-loader',
options: {
name: '[path][name].[hash:6][ext]'
}
}
]
}
}
Either way your code will now be able to work simply as follows:
let name = "tiles";
const tileDataUrl = require(`data/json/${name}.json`); // tileDataUrl will display the interpolated filename.
fetch(tileDataUrl);
I have a Javascript file utils.js which contains some utility functions. Here is an example:
export function RemoveHTML(Str) {
return Str.replace(/<[^>]*(?:>|$)/gi,'');
}
I can use these functions by importing the utils.js file like such:
import {RemoveHTML} from '../js/utils.js';
I also have a model.js file for some data queries like this (pseudo-code for brevity):
async function getStuff() {
await DBConnection.connect();
return results
}
module.exports = {
getStuff
}
For 'consistency' I thought I'd change model.js to just this:
export async function getStuff() {
await DBConnection.connect();
return results
}
If I do that then I get an app crash error stating:
SyntaxError: Unexpected token 'export'
What is the difference between exporting a function using export function() and module.exports?
UPDATE:
I am using Babel in Webpack like below so why would I get an error?:
{
test: /\.js$/,
include: [ srcPath ],
exclude: ['/node_modules/'],
use: {
loader: 'babel-loader',
options: {
presets: ["#babel/preset-env"] //Preset used for env setup
}
}
},
export function() is an ES6 syntax used for exporting while module.exports is or exports is a special object which is included in every JS file in the Node.js application by default. You can also use the ES6 syntax for exporting/importing a module in Node.js but for using that you will have to transpile the new ES6 code to Node.js supported ES5 format, You can easily do that using babel (which is a tool for transpiling JavaScript).
#JonasWilms is definitely correct about the point he made. I see you are using commonjs on the server code and es6 on the client-side.
There is no difference between module.export or export. In your project, your server code is using commonjs modules, so you should use module.exports. In your client code, keep on using export (es6) syntax of JavaScript.
But if you want to write your javascript globally with es6, you will have to install some dependencies and configure your babel.
Check out this link https://www.codementor.io/#iykyvic/writing-your-nodejs-apps-using-es6-6dh0edw2o
I am quite new to Webpack, so bear with me if thats a stupid question.
My goal is to transform my old, AMD based codebase to a ES6 Module based solution. What I am struggling with is handling dynamic import()s. So my app router works on a module basis, i.e. each route is mapped to a module path and then required. Since I know what modules will be included, I just add those dynamically imported modules to my r.js configuration and am able to build everything in a single file, with all require calls still working.
Now, I am trying to do the same with ES6 modules and Webpack. With my devmode this is no problem as I can just replace require() with import(). However I cannot get this to work with bundling. Either Webpack splits my code (and still fails to load the dynamic module anyways), or - if I use the Array format for the entry config, the dynamic module is included in the bundle but loading still fails: Error: Cannot find module '/src/app/DynClass.js'
This is how my Webpack config looks like:
const webpack = require('webpack');
const path = require('path');
module.exports = {
mode: "development",
entry: ['./main.js', './app/DynClass.js'],
output: {
filename: 'main.js',
path: path.resolve(__dirname, "../client/")
},
resolve: {
alias: {
"/src": path.resolve(__dirname, '')
}
},
module: {
rules: [
{
test: /\.tpl$/i,
use: 'raw-loader',
},
]
}
};
So basically I want to tell Webpack: "hey, there is another module (or more) that is to be loaded dynamically and I want it to be included in the bundle"
How can I do this?
So yeah, after much fiddling there seems to be a light at the end of the tunnel. Still, this is not a 100% solution and it is surely not for the faint of heart, as it is quite ugly and fragile. But still I want to share my approach with you:
1) manual parsing of my routes config
My router uses a config file looking like this:
import StaticClass from "/src/app/StaticClass.js";
export default {
StaticClass: {
match: /^\//,
module: StaticClass
},
DynClass: {
match: /^\//,
module: "/src/app/DynClass.js"
}
};
So as you can see the export is an object, with keys acting as the route id, and an object that contains the matches (regex based) and the module which should be executed by the router if the route matches. I can feed my router with both a Constructor function (or an object) for modules which are available immediatly (i.e. contained in the main chunk) or if the module value is a string, this means that the router has to load this module dynamically by using the path specified in the string.
So as I know what modules could be potentially loaded (but not if and when) I can now parse this file within my build process and transform the route config to something webpack can understand:
const path = require("path");
const fs = require("fs");
let routesSource = fs.readFileSync(path.resolve(__dirname, "app/routes.js"), "utf8");
routesSource = routesSource.substr(routesSource.indexOf("export default"));
routesSource = routesSource.replace(/module:\s*((?!".*").)*$/gm, "module: undefined,");
routesSource = routesSource.replace(/\r?\n|\r/g, "").replace("export default", "var routes = ");
eval(routesSource);
let dummySource = Object.entries(routes).reduce((acc, [routeName, routeConfig]) => {
if (typeof routeConfig.module === "string") {
return acc + `import(/* webpackChunkName: "${routeName}" */"${routeConfig.module}");`;
}
return acc;
}, "") + "export default ''";
(Yeah I know this is quite ugly and also a bit brittle so this surely could be done better)
Essentially I create a new, virtual module where every route entry which demands a dynamic import is translated, so:
DynClass: {
match: /^\//,
module: "/src/app/DynClass.js"
}
becomes:
import(/* webpackChunkName: "DynClass" */"/src/app/DynClass.js");
So the route id simply becomes the name of the chunk!
2) including the virtual module in the build
For this I use the virtual-module-webpack-plugin:
plugins: [
new VirtualModulePlugin({
moduleName: "./app/dummy.js",
contents: dummySource
})
],
Where dummySource is just a string containing the sourcecode of my virtual module I just have generated. Now, this module is pulled in and the "virtual imports" can be processed by webpack. But wait, I still need to import the dummy module, but I do not have any in my development mode (where I use everything natively, so no loaders).
So in my main code I do the following:
let isDev = false;
/** #remove */
isDev = true;
/** #endremove */
if (isDev) { import('./app/dummy.js'); }
Where "dummy.js" is just an empty stub module while I am in development mode. The parts between that special comments are removed while building (using the webpack-loader-clean-pragma loader), so while webpack "sees" the import for dummy.js, this code will not be executed in the build itself since then isDev evaluates to false. And since we already defined a virtual module with the same path, the virtual module is included while building just like I want, and of course all dependencies are resolved as well.
3) Handling the actual loading
For development, this is quite easy:
import routes from './app/routes.js';
Object.entries(routes).forEach(async ([routeId, route]) => {
if (typeof route.module === "function") {
new route.module;
} else {
const result = await import(route.module);
new result.default;
}
});
(Note that this is not the actual router code, just enough to help me with my PoC)
Well, but for the build I need something else, so I added some code specific to the build environment:
/** #remove */
const result = await import(route.module);
new result.default;
/** #endremove */
if (!isDev) {
if (typeof route.module === "string") { await __webpack_require__.e(routeId); }
const result = __webpack_require__(route.module.replace("/src", "."));
new result.default;
}
Now, the loading code for the dev environment is just stripped out, and there is another loading code that uses webpack internally. I also check if the module value is a function or string, and if it is the latter I invoke the internal require.ensure function to load the correct chunk: await __webpack_require__.e(routeId);. Remember that I named my chunks when generating the virtual module? Now thats why I still can find them now!
4) more needs to be done
Another thing I encountered is when several dynamically loaded modules have the same dependencies, webpack tries to generate more chunks with names like module1~module2.bundle.js, breaking my build. To counter this, I needed to make sure that all those shared modules go into a specific named bundle I called "shared":
optimization: {
splitChunks: {
chunks: "all",
name: "shared"
}
}
And when in production mode, I simply load this chunk manually before any dynamic modules depending on it are requested:
if (!isDev) {
await __webpack_require__.e("shared");
}
Again, this code only runs in production mode!
Finally, I have to prevent webpack renaming my modules (and chunks) to something like "1", "2" etc, but rather keep the names I just have defined:
optimization: {
namedChunks: true,
namedModules: true
}
Se yeah, there you have it! As I said this wasn't pretty but seems to work, at least with my simplified test setup. I really hope there aren't any blockers ahead of me when I do all the rest (like ESLint, SCSS etc)!
I'm writing some frontend code with ECMAScript 6 (transpiled with BabelJS, and then browserified with Browserify) so that I can have a class in one file, export it and import it in another file.
The way I'm doing this is:
export class Game {
constructor(settings) {
...
}
}
And then on the file that imports the class I do:
import {Game} from "../../lib/pentagine_browserified.js";
var myGame = new Game(settings);
I then compile it with grunt, this is my Gruntfile:
module.exports = function(grunt) {
"use strict";
grunt.loadNpmTasks('grunt-babel');
grunt.loadNpmTasks('grunt-browserify');
grunt.initConfig({
"babel": {
options: {
sourceMap: false
},
dist: {
files: {
"lib/pentagine_babel.js": "lib/pentagine.js",
"demos/helicopter_game/PlayState_babel.js": "demos/helicopter_game/PlayState.js"
}
}
},
"browserify": {
dist: {
files: {
"lib/pentagine_browserified.js": "lib/pentagine_babel.js",
"demos/helicopter_game/PlayState_browserified.js": "demos/helicopter_game/PlayState_babel.js"
}
}
}
});
grunt.registerTask("default", ["babel", "browserify"]);
};
However, on the new Game( call, I get the following error:
Uncaught TypeError: undefined is not a function
As so, what I did was analyse the generated code by Babel and Browserify and I found this line on PlayState_browserified.js:
var Game = require("../../lib/pentagine_browserified.js").Game;
I decided to print the require output:
console.log(require("../../lib/pentagine_browserified.js"));
And it is nothing but an empty object. I decided to check out the pentagine_browserified.js file:
var Game = exports.Game = (function () {
It seems like it is correctly exporting the class, but for some other reason it is not being required on the other file.
Also, I'm sure the file is being required properly because changing the string "../../lib/pentagine_browserified.js" spits out a Not Found error, so it is going for the right file, that I'm sure about.
Browserify is meant to be fed a single "entry point" file, through which it recursively traverses all of your require statements, importing the code from other modules. So you should be require'ing the _babel.js versions of modules, not _browserified.js ones.
From the looks of it, you intend for your app's "entry point" to be demos/helicopter_game/PlayState_browserified.js, yeah? If that's the case:
In PlayState.js, change it to import {Game} from "../../lib/pentagine_babel.js";.
In Gruntfile.js, remove "lib/pentagine_browserified.js": "lib/pentagine_babel.js".
Works for me. Let me know if that suffices or I am misunderstanding your requirements here.
P.S. You can use babelify to avoid having separate Grunt tasks for Babel and Browserify. See my answer here for an example.
I had a slightly different file configuration, that gave me some difficulty to get the "require" syntax to work in Node, but this post gave me the hint on how to used the babel-ified version of the file name.
I am using WebStorm with the FileWatcher option set to Babel, and I have the FileWatcher configured to watch all files with suffix .jsx, and rename the compiled output file from {my_file}.jsx to {my_file}-compiled.js.
So in my test case, I have 2 files:
Person.jsx:
class Person { ... }
export { Person as default}
and another file that wants to import it:
Test.jsx:
var Person = require('./Person-compiled.js');
I couldn't get the "require" statement to find the module until I started the file path with './' and also add '-compiled.js' to properly specify the file name so that Node es5 could find the module.
I was also able to use the "import" syntax:
import Person from './Person-compiled.js';
Since I have set up my WebStorm project as a Node ES5 project, I have to run 'Test-compiled.js' (not 'Test.jsx').