We Keep Coding

Bundle multiple named AMD modules with dependencies into one JS file (building a web app extension system)

I'm working on an extension system for my web app. Third-party developers should be able to extend the app by providing named AMD modules exporting constants and functions following a predefined spec and bundled into a single .js JavaScript file.
Example JavaScript bundle:
define('module1', ['exports', 'module3'], (function (exports, module3) {
exports.spec = 'http://example.com/spec/extension/v1'
exports.onRequest = function (request) { return module3.respond('Hello, World.') }
}));
define('module2', ['exports', 'module3'], (function (exports, module3) {
exports.spec = 'http://example.com/spec/extension/v1'
exports.onRequest = function (request) { return module3.respond('Foo. Bar.') }
}));
define('module3', ['exports'], (function (exports) {
exports.respond = function (message) { return { type: 'message', message: message } }
}));
In the above example module1 and module2 are extension modules (identified by the spec export) and module3 is a shared dependency (e.g. coming from an NPM package). Extension bundles will be loaded in a worker within a sandboxed iframe to seal of the untrusted code in the browser.
Example TypeScript source:
// module1.ts
import respond from 'module3'
export const spec = 'http://example.com/spec/extension/v1'
export const onRequest = (request: Request): Response => respond('Hello, World.')
// module2.ts
import respond from 'module3'
export const spec = 'http://example.com/spec/extension/v1'
export const onRequest = (request: Request): Response => respond('Foo. Bar.')
// module3.ts
import dep from 'some-npm-package'
export respond = (message: string) => dep.createMessageObject(message)
Here is my list of requirements to bundling:
All necessary dependencies (e.g. shared module, NPM package logic) must be included in the bundle
The source code needs to be transpiled to browser compatible code if necessary
The AMD format is required by the custom extension loader implementation
The AMD modules must not be anonymous as the module file names are lost while bundling
No relative paths must be used among dependencies (e.g. ./path/to/module3 instead of module3)
The result should be one JavaScript bundle, thus ONE JavaScript file and ONE sourcemaps file
What's the easiest way to do this?
This is the closest solution I found using rollup and the following rollup.config.js:
import { nodeResolve } from '#rollup/plugin-node-resolve'
import { terser } from 'rollup-plugin-terser'
import typescript from '#rollup/plugin-typescript'
export default {
input: [
'src/module1.ts',
'src/module2.ts'
],
output: {
dir: 'dist',
format: 'amd',
sourcemap: true,
amd: {
autoId: true
}
},
plugins: [
typescript(),
nodeResolve(),
terser()
]
}
From this I get the desired named AMD modules (one for each entry point and chunk) in separate .js files. Problems:
Some dependencies are referenced by ./module3 while being named module3.
The modules appear in separate JavaScript and Sourcemap files instead of being concatenated into a single bundle.
Questions:
Is there an easy fix to the above rollup.config.js config to solve the problem?
I tried to write a small rollup plugin but I failed to get the final AMD module code within it to concatenate it to a bundle. Only the transpiled code is available to me. In addition I don't know how to handle sourcemaps during concatenation.
Is there an alternative to rollup better suited to this bundling scenario?
The big picture: Am I completely on the wrong track when it comes to building an extension system? Is AMD the wrong choice?

I found a way to extend the rollup.config.js mentioned in the question with a custom concatChunks rollup plugin to bundle multiple AMD chunks within a single file and having the source maps rendered, too. The only issue I didn't find an answer to was the relative module names that kept popping up. However, this may be resolved in the AMD loader.
Here's the full rollup.config.js that worked for me:
import Concat from 'concat-with-sourcemaps'
import glob from 'glob'
import typescript from '#rollup/plugin-typescript'
import { nodeResolve } from '#rollup/plugin-node-resolve'
import { terser } from 'rollup-plugin-terser'
const concatChunks = (
fileName = 'bundle.js',
sourceMapFileName = 'bundle.js.map'
) => {
return {
name: 'rollup-plugin-concat-chunks',
generateBundle: function (options, bundle, isWrite) {
const concat = new Concat(true, fileName, '\n')
// Go through each chunk in the bundle
let hasSourceMaps = false
Object.keys(bundle).forEach(fileId => {
const fileInfo = bundle[fileId]
if (fileInfo.type === 'chunk') {
let hasSourceMap = fileInfo.map !== null
hasSourceMaps = hasSourceMaps || hasSourceMap
// Concat file content and source maps with bundle
concat.add(
fileInfo.fileName,
fileInfo.code,
hasSourceMap ? JSON.stringify(fileInfo.map) : null
)
// Prevent single chunks from being emitted
delete bundle[fileId]
}
})
// Emit concatenated chunks
this.emitFile({
type: 'asset',
name: fileName,
fileName: fileName,
source: concat.content
})
// Emit concatenated source maps, if any
if (hasSourceMaps) {
this.emitFile({
type: 'asset',
name: sourceMapFileName,
fileName: sourceMapFileName,
source: concat.sourceMap
})
}
}
}
}
export default {
input: glob.sync('./src/*.{ts,js}'),
output: {
dir: 'dist',
format: 'amd',
sourcemap: true,
amd: {
autoId: true
}
},
plugins: [
typescript(),
nodeResolve(),
terser(),
concatChunks()
]
}
Please make sure you npm install the dependencies referenced in the import statements to make this work.
Considering the big picture, i.e. the extension system itself, I am moving away from a "one AMD module equals one extension/contribution" approach, as current developer tools and JavaScript bundlers are not ready for that (as this question shows). I'll go with an approach similar to the Visual Studio Code Extension API and will use a single "default" module with an activate export to register contributions a bundle has to offer. I hope that this will make extension bundling an easy task no matter what tools or languages are being used.

next-images Error: Module parse failed: Unexpected character '�'

I'm trying to load an image with next-images:
when i type in the image name it works fine:
//Working
<Image src={require(`../../images/exampleImage.jpg` )}/>
but i dont want that i want dynamic url like this:
//Not working
<img src={require(`../../images/${image}.jpg` )}/>
i get this error:
Error: Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)
my next.config.js file:
const withImages = require("next-images");
module.exports = withImages();
i also tried this config:
module.exports = {
webpack: (config, options) => {
config.module.rules.push(
{
test: /\.(jpe?g|png|gif|woff|woff2|eot|ttf|svg)(\?[a-z0-9=.]+)?$/,
loader: 'url-loader?limit=100000'
}
)
return config
},
}
I tried many methods but none seems to work please help, thanks

If you're open to using the file-loader library to handle images on the project. You could install the library and set the rules like this on webpack:
...
config.module.rules.push(
{
test: /\.(png|jpeg|jpg|gif|svg)$/,
use: {
loader: "file-loader"
},
}
),
You can read more about file-loader from its documentation on webpack

Webpack is most likely trying to find & include your images at the build time. This cannot work with reading the name from a variable. You have 2 options:
manage images differently
if you have a finite (or rather short) list of images, just import all & use some kind of switch to control which image is displayed.

I had this issue too.
delete all your code in the next.config.js
add the below codes instead:
/** #type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
images: {
dangerouslyAllowSVG: true,
contentSecurityPolicy: "default-src 'self'; script-src 'none'; sandbox;",
},
};
module.exports = nextConfig;
It resolved my problem.

Prevent Webpack's default loaders from running

Webpack has a JSON loader built into it. How can I write a custom loader that doesn't try to run the built-in JSON loader on the result?
Basically, I want my loader to take in a config object (stored in a JSON file) and generate source code from that configuration, which is no longer valid JSON, it's JavaScript (which could subsequently be fed through babel-loader).
Here's a really stupid, contrived example. The source file is a JSON file, but I want the output of my loader to instead be some JS.
Loader
function compile(doc) {
return `alert(${JSON.stringify(doc.title)})`
}
function myLoader(source) {
return compile(JSON.parse(source))
}
Webpack config
rules: [
{
test: /\.json$/,
use: [
'babel-loader',
'my-loader',
],
},
],
Instead, I end up getting this error:
ERROR in ./config.json
Module parse failed: Unexpected token ' in JSON at position 0
You may need an appropriate loader to handle this file type.
SyntaxError: Unexpected token ' in JSON at position 0
at JSON.parse (<anonymous>)
at JsonParser.parse (.../node_modules/webpack/lib/JsonParser.js:15:21)
As you can see from the stack trace, it's coming from webpack/lib/JsonParser.js. How can I tell Webpack not to run its built-in JSON parser in the first place, and instead delegate the processing of JSON files that match this rule to my loader?

I think I figured it out, although it appears to be an undocumented feature (at least, I couldn't find it on the configuration docs.
It looks like one of the properties you can pass in the Rule object is type. By setting it to javascript/auto, you can override the default JSON parser and have it parse the source of the file as JavaScript.
This property is in the JSON schema used by Webpack to validate the config object.
rules: [
{
test: /\.json$/,
use: [
'babel-loader',
'my-loader',
],
type: 'javascript/auto',
},
],

Webpack Loaders JSX string to JSX

I put together a small loader so that when I require html files I will get JSX in return making use of this Htmltojsx converter
Unfortunately, since the loader just returns a string, my loader is crashing. I can verify from the loader that the string I am getting is what's expected:
import bodyHtml from './landing-body.html';
const Landing = () => (
<React.Fragment>
<h3> Landing Page </h3>
bodyHtml
...
And then the webpack build is crashing with this kind of error:
ERROR in ./Landing/landing-body.html
Module parse failed: Unexpected token (1:0)
You may need an appropriate loader to handle this file type.
| <div>
| <h5 className="product__title"> Your Product Name </h5>
Maybe I need to put in another loader for getting this from a string to raw JSX output?

Figured it out!
Since the first loader just returns a JSX string, there's still more transpiling to be done.
What I needed to use was webpack's 'loader chaining'
So in the end my html rule looks like this:
{
test: /\.html$/,
exclude: /node_modules/,
use: [
{
loader: 'babel-loader'
},
{
loader: path.resolve(__dirname, 'loaders/html-loader.js')
}
]
}
I guess the chained loaders are done in 'reverse order' so to speak, so the html loader first converts it to a JSX string, and then the babel-loader treats that as JSX code.

Load static JSON file in Webpack

I have somewhere in my code following construction:
var getMenu = function () {
return window.fetch("portal/content/json/menu.json").then(function (data) {
return data.json();
});
};
I tried in my webpack.config.js this:
module: {
loaders: [
...
{
test: /\.json$/,
exclude: /node_modules/,
use: [
'file-loader?name=[name].[ext]&outputPath=portal/content/json'
]
},
...
]
}
Project structure:
dist
content
json
menu.json <- this is missing
src
content
json
menu.json <- source file
Question:
How can webpack copy src/content/json/menu.json to dist/content/json/menu.json ?

You're using fetch to request a JSON file and that will only happen at runtime. Furthermore, webpack only processes anything that is imported. You expected it to handle an argument to a function, but if webpack did that, every argument to a function would be considered a module and that breaks any other use for that function.
If you want your loaders to kick in, you can import the file.
import './portal/content/json/menu.json';
You can also import the JSON and use it directly instead of fetching it a runtime. Webpack 2 uses json-loader by default for all .json files. You should remove the .json rule and you would import the JSON as follows.
import menu from './portal/content/json/menu.json';
menu is the same JavaScript object that you would get from your getMenu function.

if you'd like your json to be loaded in runtime/deferred you can use awesome webpack's dynamic imports feature:
import(
/* webpackChunkName: "json_menu" */
'./portal/content/json/menu.json'
);
it will return a Promise which resolves to the module object, with "default" field containing your data. So you might want something like this (with es6 it looks really nice):
import(
/* webpackChunkName: "json_menu" */
'./portal/content/json/menu.json'
).then(({default: jsonMenu}) => {
// do whatever you like with your "jsonMenu" variable
console.log('my menu: ', jsonMenu);
});
Notice that dynamic imports require a babel plugin syntax-dynamic-import, install it with npm:
npm i babel-plugin-syntax-dynamic-import -D
Have a nice day