We Keep Coding

Webpack: Infinite watch loop and auto-generated files

I have a script which generates a file, call it auto.js. This file contains some dynamically generated imports and is being used within a VueJS project.
// auto.js
import { apple, strawberry, orange } from 'delicious-fruits';
import { carrot, cucumber, celery } from 'delicious-vegetables';
While using Webpacks dev server, should any project file change, my goal is to have this script re-generate my auto.js file, and then have that included in the re-compiled project.
I have turned this script into a Webpack plugin, whereby I'm listening for the watchRun compiler hook. This seems like the ideal hook, per its description:
Executes a plugin during watch mode after a new compilation is triggered but before the compilation is actually started.
class AutoGenerate {
constructor(options) {
this.options = options;
}
apply(compiler) {
compiler.hooks.watchRun.tap('AutoGenerate', () => {
generateFile()
})
}
}
function generateFile () {
// generate auto.js and write to disk
}
I always wind up with an infinite loop situation. I have tried approaching the problem by using various life cycles events (hooks), as well ignoring the auto-generated file. Of course, by ignoring it, those changes are not included in the re-compiled project.
const webpack = require('webpack');
const AutoGenerate = require("./auto.plugin");
module.exports = {
configureWebpack: {
plugins: [
new webpack.WatchIgnorePlugin([/.*auto\.js/]),
new AutoGenerate()
]
}
}
I've also tried tapping into the compilation, and adding a new asset to the compilation. While the process does not error out, the generated asset is not a part of the final compilation.
// auto.plugin.js
class AutoGenerate {
static defaultOptions = {
outputFile: 'auto.js',
};
constructor(options = {}) {
this.options = { ...AutoGenerate.defaultOptions, ...options };
}
apply(compiler) {
compiler.hooks.thisCompilation.tap('AutoGenerate', (compilation) => {
const path = require("path")
const filePath = path.resolve(__dirname, `src/plugins/${this.options.outputFile}`)
const { RawSource } = require('webpack-sources')
const fileContent = new RawSource(generateFile())
compilation.emitAsset(
filePath,
fileContent
)
});
}
}
function generateFile() {
// generate file content & return as string
}
module.exports = { AutoGenerate };
// vue.config.js
const AutoGenerate = require("./auto.plugin");
module.exports = {
configureWebpack: {
plugins: [
new AutoGenerate()
]
}
}
How can I trigger my logic for auto-generating this file, while having this file be included as part of any re-compilation, while at the same time avoiding an infinite loop?

I have not been able to identify a direct solution to the problem posed above. However, for anyone reading, I've come to discover that this can be accomplished by utilizing a package called before-build-webpack, notably by including the watch-run trigger.
// vue.config.js
const WebpackBeforeBuildPlugin = require('before-build-webpack')
module.exports = {
configureWebpack: {
plugins: [
new WebpackBeforeBuildPlugin(function(stats, callback) {
// ...
}, ['run', 'watch-run'])
]
}
}

How can I run one command to bundle libraries and output separately in vue-cli3.0?

I have read the document of build library in VUE-CLI3.0.
My directory:
--src
--components
--componentA.vue
--componentB.vue
....
--componentZ.vue
--build
--libs.js
I want to run one command with my one entry "libs.js" (Maybe there is a loop to create multiple entries in libs.js) to bundle my components separately. The destination folder maybe like the following:
--dist
--componentA.css
--componentA.command.js
--componentA.umd.js
--componentA.umd.min.js
...
--componentZ.css
--componentZ.command.js
--componentZ.umd.js
--componentZ.umd.min.js
Can anyone give me some suggetions?

I add a script file. In which, I get the list of components and using 'child_process' to execute each command.
The following is an example:
lib.js
const { execSync } = require('child_process')
const glob = require('glob')
// console font color
const chalk = require('chalk')
// loading
const ora = require('ora')
// 获取所有的moduleList
const components = glob.sync('./src/components/*.vue')
// const buildFile = path.join(__dirname, 'build.js')
// const webpack = require('vuec')
const spinner = ora('Packaging the components...\n').start()
setTimeout(() => {
spinner.stop()
}, 2000)
for (const component of components) {
// const file = path.join(__dirname, module);
const name = component.substring(component.lastIndexOf('/') + 1).slice(0, -4)
const cmd = `vue build -t lib -n ${name} ${component} -d lib/components/${name}`
execSync(cmd)
console.log(chalk.blue(`Component ${name} is packaged.`))
}
console.log(`[${new Date()}]` + chalk.green('Compeleted !'))
What's more, add a script command in package.json:
"build-all": "node ./src/build/lib.js"
You just enter npm run build-all. That's all~

Mock require function in Jest

I am trying to mock require using Jest because I am working in a plugin system for electron that will also use NPM for resolving dependencies.
I need to then be able to mock require inside nodejs to be able to test something similar to the next logic.
const load = (installPath, pluginsName, extensionPoint) => {
require.main.paths.push(`${installPath}/node_modules`)
pluginsName.forEach(pluginName => {
let plugin = require(pluginName)
plugin.onLoad(extensionPoint)
});
}
module.exports = { load }
Is there any simple way of doing this?. As require is not a global, is there is any other way than wrapping and injecting it to test the logic?

As explained in this answer, it's possible to replace require in particular module by evaluating it with custom require, similarly to how Node does this internally:
const Module = require('module');
const vm = require('vm');
const childModuleAbsPath = path.resolve('./foo/bar.js');
const childModuleBody = fs.readFileSync(childModuleAbsPath);
const childModuleObj = { exports: {} };
const { dir: childModuleDirname, base: childModuleFilename } = path.parse(childModuleAbsPath);
const childRequire = jest.fn().mockReturnValue(...);
vm.runInThisContext(Module.wrap(childModuleBody))(
childModuleObj.exports,
childRequire,
childModuleObj,
childModuleDirname,
childModuleFilename
);

How to read text files with Elm and Webpack

I have written a simple quizz application with Elm. As it was explained in the tutorial, the only way in Elm to access external files is to use ports with Javascript. So I have included ports in my Elm file, and I now have to add them in the index.js file that I use as an entry point. I use webpack to build the complete app.
However, I don't get the webpack logic. This is my file tree:
resources
|---- images
|---- questions
|---- question_1.txt
|---- question_2.txt
|---- ...
|---- scores
|---- scores_table.json
src
|---- MyElm.elm
|---- Questions.elm
|---- index.js
|---- index.html
webpack.config.js
My JS component needs to read all possible questions in the questions folder to both determine the total number of questions and provide them to Elm through ports.
In the same way, the JS component needs to parse the scores_table.json file to send results to the Elm app.
What can I use in my index.js app to read these files? I tried with require, but I think I didn't use it correctly.
It's my first question on Stack Overflow, so if there's anything missing, please tell me.
Minimal example
This is a simplified version of what I have:
webpack.config.js
var path = require("path");
module.exports = {
entry: {
app: [
'./src/index.js'
]
},
output: {
path: path.resolve(__dirname + '/dist'),
filename: '[name].js',
},
module: {
rules: [
{
test: /\.txt$/,
use: 'raw-loader'
},
{
test: /\.(css|scss)$/,
loaders: [
'style-loader',
'css-loader',
]
},
{
test: /\.html$/,
exclude: /node_modules/,
loader: 'file-loader?name=[name].[ext]',
},
{
test: /\.elm$/,
exclude: [/elm-stuff/, /node_modules/],
loader: 'elm-webpack-loader?verbose=true&warn=true',
},
{
test: /\.woff(2)?(\?v=[0-9]\.[0-9]\.[0-9])?$/,
loader: 'url-loader?limit=10000&mimetype=application/font-woff',
},
{
test: /\.(ttf|eot|svg)(\?v=[0-9]\.[0-9]\.[0-9])?$/,
loader: 'file-loader',
},
],
noParse: /\.elm$/,
},
devServer: {
inline: true,
stats: { colors: true },
},
};
index.js
const RESOURCES_DIR = "../resources/";
const IMAGES_DIR = RESOURCES_DIR + "images/"
const QUESTIONS_DIR = RESOURCES_DIR + "questions/"
const SCORES_DIR = RESOURCES_DIR + "scores/"
require("./index.html");
const scores_table =
require(SCORES_DIR + "scores_table.json");
var total_question_nb = 0;
var questions_table = [];
var end = false;
while (!end) {
try {
data = require(
"raw!" +
QUESTIONS_DIR +
"question_${total_question_nb}.txt");
questions_table.push(data);
total_question_nb += 1;
} catch (e) {
end = true;
}
}
console.log(questions_table[0]);
console.log(total_question_nb);
var Elm = require("./MyElm.elm");
var mountNode = document.getElementById("elm-app");
var app = Elm.MyElm.embed(mountNode);
// Need to add port gestion there
MyElm.elm
...
import Questions
...
Questions.elm
...
-- (current_question_no, ans_id)
port ask_question_score : (Int, Int) -> Cmd msg
-- next_question_no
port ask_next_question : Int -> Cmd msg
-- question_score
port receive_question_score : (List Int -> msg) -> Sub msg
-- (next_question_no, total_question_nb, next_question_text)
port receive_next_question : ((Int, Int, String) -> msg) -> Sub msg
-- ()
port receive_end_question : (() -> msg) -> Sub msg
...
And this is what I get when I load the page using Webpack:
Uncaught Error: Cannot find module "../resources/scores/scores_table.json".
at r (app.js:1)
at Object.<anonymous> (app.js:1)
at r (app.js:1)
at Object.<anonymous> (app.js:1)
at r (app.js:1)
at app.js:1
at app.js:1

TLDR You'll need to setup Webpack's code splitting with dynamic imports to enable dynamic require's
Webpack is designed to compress all of your source files into one 'build' file. This, of course, relies on identifying which files you're importing. When you pass an expression rather than a plain string to require, webpack may not correctly identify the files you want.
In order to explicitly tell webpack what to include, you can use a "dynamic" import, ie code splitting. I'd say code splitting is rather advanced, if you want to avoid it, just hardcode the files you want to import. That should be fine if the files don't change often.
Examples
If you know the filenames:
const scores = require('../resources/scores/scores_table.json')
// or
import scores from '../resources/scores/scores_table.json'
// The `import()` function could be used here, but it returns a Promise
// I believe `require()` is blocking, which is easier to deal with here
// (though not something I'd want in a production application!)
const questions = [
require('../resources/questions/question_1.txt'),
require('../resources/questions/question_2.txt'),
]
If you want to dynamically import files (like you would probably do with questions):
// Utility function
// Example:
// arrayOfLength(4) -> [0, 1, 2, 3]
const arrayOfLength = length =>
(new Array(length)).fill(0).map((val, i) => i)
const questionsCount = 100 // <- You need to know this beforehand
// This would import questions "question_0.txt" through "question_99.txt"
const questionPromises = arrayOfLength(questionsCount)
.map(i =>
import(`../resources/questions/question_${i}.txt`)
.catch(error => {
// This allows other questions to load if one fails
console.error('Failed to load question ' + i, error)
return null
})
)
Promise.all(questionPromises)
.then(questions => { ... })
With the dynamic imports, you need to handle promises. You could also use async / await to make it look a bit nicer (that isn't support in all browsers -- needs transpiling set up)
If the files do change often, that means you are frequently modifying questions and/or the score table, and you should probably be using a database instead of dynamic imports.

This I use in React project. It will allow you to change files without rebuilding with webpack:
in config/index.js
const CONFIG_FILE = '/config.json';
let loadedConfig = {};
export const loadConfig = () => {
const headers = new Headers();
headers.append('Content-type', 'application/json');
try {
return fetch(CONFIG_FILE, { headers })
.then(response => response.json())
.then((json) => { loadedConfig = json; });
} catch (error) {
console.log(error); // eslint-disable-line no-console
}
};
export default () => ({ ...loadedConfig });
In index.jsx I load it:
import React from 'react';
import ReactDOM from 'react-dom';
import { loadConfig } from 'config';
loadConfig().then(() => {
require.ensure([], (require) => {
const App = require('App').default;
ReactDOM.render(
<App />,
document.getElementById('root'),
);
});
});
And then I import it and use
import config from 'config';
console.log(config().SOME_VAR_FROM_JSON);

Systemjs-Builder - Cannot configure properly - Bundling Typescript into a package

I want to build a quick nodejs script to package a Typescript app as SystemJS modules, a lot like what Angular2 bundles look like.
I tried different configurations but I can't seem to put my finger on it, and haven't found clear enough documentation as of yet.
Note that for this "test", I am not using Gulp or Jspm at all, just systemjs-builder for the time being (and don't plan on using jspm at all either)
Here's what my "project" looks like:
---- Project's Root
-------- index.ts // export * from './modules/index' and eventually more
-------- modules
------------ index.ts // export * from './menu/index'
------------ menu
---------------- menu.component.ts // export class
---------------- menu.service.ts // export class
I want to package this under a single file, where I will have multiple SystemRegister modules that can be consumed in an app thereafter
I tried the following without success:
var Builder = require('systemjs-builder');
// optional constructor options
// sets the baseURL and loads the configuration file
var builder = new Builder('./modules');
builder.bundle('./modules/index.ts', {
/* SystemJS Configuration Here */
baseURL: './modules',
transpiler: 'typescript',
typescriptOptions: {
"module": "system",
"emitDecoratorMetadata": true,
"experimentalDecorators": true
},
defaultExtension: 'ts',
packages: {
'modules': {
defaultExtension: 'ts'
}
}
}, 'infrastructure.js')
.then(function() {
console.log('Build complete');
})
.catch(function(err) {
console.error(err);
})
First of all, the defaultExtension options doesn't seem to work at all
So when I do import {something} from 'filePath'; (without extension), it tries to load filePath, instead of filePath.ts;
Second, if I try adding the .ts extension in my imports (which I don't want to do), it complains that the code is invalid (unexpected token #, unexpected token menuItem and so forth)
Anyone have a good example or some explanations on how this is supposed to work?
Thank you

here you have an example: angular typescript skeleton
build task looks like this:
const path = require('path');
const Builder = require('jspm').Builder;
const builder = new Builder();
const packageJson = require(path.join(config.projectDir, 'package.json'));
return beginBuild()
.then(buildSFX)
.catch((err) => console.log('Build Failed', err));
function beginBuild() {
builder.reset();
return builder.loadConfig(path.join(config.projectDir, packageJson.jspm.configFile))
}
function buildSFX() {
const appName = packageJson.name;
const distFileName = `${appName}.min.js`;
const outFile = path.join(config.distDir, distFileName);
const moduleName = 'app';
const buildConfig = {
format: 'global',
minify: true,
sourceMaps: true
};
return builder.buildStatic(moduleName, outFile, buildConfig);
}
and jspm conf looks like this:
System.config({
defaultJSExtensions: true,
transpiler: "typescript",
typescriptOptions: {
"tsconfig": "src/tsconfig.json"
},
paths: {
"github:*": "vendor/jspm_packages/github/*",
"npm:*": "vendor/jspm_packages/npm/*",
"app": "src/index"
}
/// ...
}

Why do you want to bundle typescript? Bundling is a method used for optimizing the delivery of source code to the browser. The browser doesn't know typescript, it only knows javascript (unless you do on the fly transpiling).