Deno is super cool. I saw it in the morning and want to migrate to deno now. I was trying to move my existing nodejs script to deno. Can any one help me on how to use npm modules in deno. I need esprima module. This one has the package https://github.com/denoland/deno_third_party/tree/master/node_modules but i am not able to figure out how to use that.
Deno provides a Node Compatibility Library, that allows using some NPM packages that do not use non-polyfilled Node.js APIs.
As of Deno 1.25 there's an experimental NPM support by using npm: specifier
npm:<package-name>[#<version-requirement>][/<sub-path>]
import express from "npm:express";
const app = express();
app.get("/", function (req, res) {
res.send("Hello World");
});
app.listen(3000);
console.log("listening on http://localhost:3000/");
The --unstable flag is required.
When doing this, no npm install is necessary and no node_modules folder is created
You can also require and installed npm package by using https://deno.land/std/node/module.ts
The following works on deno >= 1.0.0
npm install esprima
import { createRequire } from "https://deno.land/std/node/module.ts";
const require = createRequire(import.meta.url);
const esprima = require("esprima");
const program = 'const answer = 42';
console.log(esprima.tokenize(program))
The above code will use esprima from node_modules/.
To run it, you'll need --allow-read && --allow-env flag
# you can also use --allow-all
deno run --allow-read --allow-env esprima.js
You can restrict it only to node_modules
deno run --allow-read=node_modules esprima.js
Which outputs:
[
{ type: "Keyword", value: "const" },
{ type: "Identifier", value: "answer" },
{ type: "Punctuator", value: "=" },
{ type: "Numeric", value: "42" }
]
Note: many APIs used by std/ are still unstable, so you may need to run it with --unstable flag.
Although since that whole project is written in TypeScript already, and it's not using any dependencies, it will be very easy for them to adapt it to Deno. All they need to do is use .ts extension on their imports.
You can also fork the project and do the changes.
// import { CommentHandler } from './comment-handler';
import { CommentHandler } from './comment-handler.ts';
// ...
Once they do, you'll be able to just do:
// Ideally they would issue a tagged release and you'll use that instead of master
import esprima from 'https://raw.githubusercontent.com/jquery/esprima/master/src/esprima.ts';
const program = 'const answer = 42';
console.log(esprima.tokenize(program))
Alternative
You can also use https://jspm.io/ which will convert NPM modules to ES Modules
All modules on npm are converted into ES modules handling full
CommonJS compatibility including strict mode conversions.
import esprima from "https://dev.jspm.io/esprima";
const program = 'const answer = 42';
console.log(esprima.tokenize(program))
For packages that use Node.js modules not supported by jspm it will throw an error:
Uncaught Error: Node.js fs module is not supported by jspm core.
Deno support here is tracking in
https://github.com/jspm/jspm-core/issues/4, +1's are appreciated!
To polyfill those Node.js APIs you'll have to include std/node.
// import so polyfilled Buffer is exposed
import "https://deno.land/std/node/module.ts";
import BJSON from 'https://dev.jspm.io/buffer-json';
const str = BJSON.stringify({ buf: Buffer.from('hello') })
console.log(str);
Issue
In general, there are two issues with npm packages in Deno:
ES Module (ESM) conformity is not given.
Bare imports like import _ from "lodash" don't work - no "magic" node_modules resolution
All import specifiers need to include the file extension - .ts,.js etc.
CommonJS module system is not usable in Deno
The npm package uses native Node.js builtins like fs or path.
Solutions to issue 1
1.1: Third party modules
The Third Party Modules section is the quickest way to discover compatible packages.
1.2: ESM CDN providers
Also take a look at CDN providers, that can auto-convert npm packages to ES Modules (ESM):
Skypack CDN
jspm.io
unpkg.com with ?module query parameter
Skypack CDN can deliver auto-converted packages, that e.g. have set a "module" entrypoint in package.json. For TypeScript users: It fetches .d.ts type definitions along with .js files (via X-TypeScript-Types HTTP headers used by Deno).
unpkg.com describes its ?module flag as follows: "Expands all 'bare' import specifiers in JavaScript modules to unpkg URLs. This feature is very experimental".
Esprima does not depend on Node.js builtins, so we can simplify its import by a CDN URL:
import esprima from "https://cdn.skypack.dev/esprima#^4.0.1"; // Option 1: Skypack
import esprima from "https://dev.jspm.io/esprima"; // Option 2: jspm
// your program
const tokens = esprima.tokenize("const foo = 'bar'"); // works
jspm would be a good choice here - Skypack TS types didn't work for me in this particular case.
1.3: Other approaches
You might also try to import an ESM compatible version directly from repository sources (e.g. an ESM branch). Though for Esprima it won't work because of missing file extensions in code.
Snowpack and jspm stand in for a more manual approach to convert CommonJS → ESM. The rollup plugin #rollup/plugin-commonjs (internally used by Snowpack) is even a more low-level tool.
Solution to issue 2
Deno provides a Node compatibility layer, see Marcos Casagrande's answer. However, not all native Node.js built-ins are fully supported.
As Esprima doesn't rely on Node builtins, you can go with the simpler CDN option.
As of version Deno 1.25 (released today) deno is now included with experimental npm support.
// main.ts
import express from "npm:express";
const app = express();
app.get("/", function (req, res) {
res.send("Hello World");
});
app.listen(3000);
console.log("listening on http://localhost:3000/");
You can now run deno run --unstable --A main.ts and express will be downloaded.
Starting with v1.15 Deno provides Node compatibility mode that makes it possible to run a subset of programs authored for Node.js directly in Deno. Compatibility mode can be activated by passing --compat flag in CLI.
deno run --compat --unstable --allow-read test.js
Currently, not all node.js built-in modules are supported and many are partially supported.
The following modules are not yet implemented:
cluster, dgram, http2, https, repl, tls, vm, lib
Related
I wrote an npm tool (similar to the one in the node_modules./bin directory).
The tool is written in ts. The import syntax is used to import modules into the code. So I wrote type: "module" in package. json.
However, in the . bin/cli. js file, the require syntax is used. Only type: "commonjs" can use the following code.
#!/ usr/bin/env node
require('../dist/index.js')
How should we deal with this situation?
Based on the Creating ESM-based shell scripts for Unix and Windows with Node.js blog post by Dr. Axel Rauschmayer, you should be able to rename your bin file from cli.js to cli.mjs:
#!/ usr/bin/env node
import '../dist/index.js';
I'd expect that nodejs should be then use the package.json "type": "module" value to treat your module as ESM after that.
You can then just run the bin file as usual:
$ ./bin/cli.mjs
I'm creating a program to analyze security camera streams and got stuck on the very first line. At the moment my .js file has nothing but the import of node-fetch and it gives me an error message. What am I doing wrong?
Running Ubuntu 20.04.2 LTS in Windows Subsystem for Linux.
Node version:
user#MYLLYTIN:~/CAMSERVER$ node -v
v14.17.6
node-fetch package version:
user#MYLLYTIN:~/CAMSERVER$ npm v node-fetch
node-fetch#3.0.0 | MIT | deps: 2 | versions: 63
A light-weight module that brings Fetch API to node.js
https://github.com/node-fetch/node-fetch
keywords: fetch, http, promise, request, curl, wget, xhr, whatwg
dist
.tarball: https://registry.npmjs.org/node-fetch/-/node-fetch-3.0.0.tgz
.shasum: 79da7146a520036f2c5f644e4a26095f17e411ea
.integrity: sha512-bKMI+C7/T/SPU1lKnbQbwxptpCrG9ashG+VkytmXCPZyuM9jB6VU+hY0oi4lC8LxTtAeWdckNCTa3nrGsAdA3Q==
.unpackedSize: 75.9 kB
dependencies:
data-uri-to-buffer: ^3.0.1 fetch-blob: ^3.1.2
maintainers:
- endless <jimmy#warting.se>
- bitinn <bitinn#gmail.com>
- timothygu <timothygu99#gmail.com>
- akepinski <npm#kepinski.ch>
dist-tags:
latest: 3.0.0 next: 3.0.0-beta.10
published 3 days ago by endless <jimmy#warting.se>
esm package version:
user#MYLLYTIN:~/CAMSERVER$ npm v esm
esm#3.2.25 | MIT | deps: none | versions: 140
Tomorrow's ECMAScript modules today!
https://github.com/standard-things/esm#readme
keywords: commonjs, ecmascript, export, import, modules, node, require
dist
.tarball: https://registry.npmjs.org/esm/-/esm-3.2.25.tgz
.shasum: 342c18c29d56157688ba5ce31f8431fbb795cc10
.integrity: sha512-U1suiZ2oDVWv4zPO56S0NcR5QriEahGtdN2OR6FiOG4WJvcjBVFB0qI4+eKoWFH483PKGuLuu6V8Z4T5g63UVA==
.unpackedSize: 308.6 kB
maintainers:
- jdalton <john.david.dalton#gmail.com>
dist-tags:
latest: 3.2.25
published over a year ago by jdalton <john.david.dalton#gmail.com>
Contents of the .js file (literally nothing but the import):
user#MYLLYTIN:~/CAMSERVER$ cat server.js
import fetch from "node-fetch";
Result:
user#MYLLYTIN:~/CAMSERVER$ node -r esm server.js
/home/user/CAMSERVER/node_modules/node-fetch/src/index.js:1
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/user/CAMSERVER/node_modules/node-fetch/src/index.js
require() of ES modules is not supported.
require() of /home/user/CAMSERVER/node_modules/node-fetch/src/index.js from /home/user/CAMSERVER/server.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/user/CAMSERVER/node_modules/node-fetch/package.json.
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1089:13) {
code: 'ERR_REQUIRE_ESM'
}
user#MYLLYTIN:~/CAMSERVER$
From the Upgrade Guide
node-fetch was converted to be a ESM only package in version 3.0.0-beta.10. node-fetch is an ESM-only module - you are not able to import it with require.
Alternatively, you can use the async import() function from CommonJS to load node-fetch asynchronously:
// mod.cjs
const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));
or you stay on v2 as they say in the README
node-fetch is an ESM-only module - you are not able to import it with require. We recommend you stay on v2 which is built with CommonJS unless you use ESM yourself. We will continue to publish critical bug fixes for it.
EDIT
I saw that in the doc, used it, and the script crashes with :
error TS2556: A spread argument must either have a tuple type or be passed to a rest parameter.
Since you are using typescript you should do something like this
import { RequestInfo, RequestInit } from "node-fetch";
const fetch = (url: RequestInfo, init?: RequestInit) => import("node-fetch").then(({ default: fetch }) => fetch(url, init));
To fix this, I downgraded node-fetch to the latest version 2, which right now is 2.6.6. This is the script I ran:
yarn add node-fetch#^2.6.6
or
npm install node-fetch#^2.6.6
I also added these compiler options:
{
"compilerOptions": { "allowJs": true, "outDir": "./dist" },
}
node-fetch v3 is ESM-only: https://github.com/node-fetch/node-fetch#loading-and-configuring-the-module. The esm module you’re adding is for adding ESM compatibility, but it’s unnecessary now that Node 12+ supports ESM natively; and it doesn’t work with ESM-only packages like node-fetch 3+.
To fix your issue:
Remove the esm package.
Add "type": "module" to your package.json.
And that’s it. Then when you run node server.js it should work.
I always had errors with previous solutions, this is what worked every single time for me
npm i -D node-fetch
const _importDynamic = new Function('modulePath', 'return import(modulePath)');
export const fetch = async function (...args: any) {
const {default: fetch} = await _importDynamic('node-fetch');
return fetch(...args);
}
Use ESM syntax, also use one of these methods before running the file.
specify "type":"module" in package.json
Or use this flag --input-type=module when running the file
Or use .mjs file extension
Many times that error happens through the current version of fetch what we can do to solve it, is to install a previous version I tried it and it worked for me.
to install the previous version:
npm i node-fetch#2.6.1 or whatever you prefer.
Since the question was really about requiring node-fetch and there are definitely reasons that a developer might need to use require vs import, the answer to import the module using import() is somewhat accurate but isn't complete, because it ignores the fact that using import is async and if you're using node-fetch in your code in multiple places you're going to have a bit of an async mess all over the place whenever you want to use node-fetch, so the answer is very incomplete IMHO. A more complete answer would include a better how-to. Something like the following:
You need to use import() rather than require because this is an ES module. To avoid an async mess, you'll want to await the importing of this module once and then be able to use it wherever else you need it. To do that, create a module that imports any of your ES-only modules something like this:
"use strict";
let got;
let fetch;
module.exports.load = async function() {
queueMicrotask(function() {
// any ESM-only module that needs to be loaded can be loaded here, just add import for each using specific structure of each
import("got").then(({default: Got}) => got = Got );
fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));
});
while(
// check each module to see if it's been loaded here
!got || !got.get || !fetch || typeof fetch !== "function"
) {
// waiting for modules to load
console.log("Waiting for ES-Only modules to load...");
await new Promise((resolve)=>setTimeout(resolve, 1000));
}
module.exports.got = got;
module.exports.fetch = fetch;
console.log("ES-Only modules finished loading!");
}
This then allows you to call the await the loading of the ES-Only modules once and then use the module later by grabbing it from your intermediary module like this:
"use strict";
const esmModules = require("esm_modules"); // or whatever you called the intermiary module
async function doMyFetching(url, options) {
const fetch = esmModules.fetch;
const result = await fetch(url, options)
}
The nice part of doing it this way is that you know that your ES-Only modules are loaded and you know that once they're loaded you can use them at will throughout the rest of your code in a simple and clean way that doesn't force you to add additional async logic all over the place whenever you need the module.
Here is in Typescript, this will install to the global scope, just import the file and fetch ready to use:
/* filename: fetch.ts */
if (!globalThis.fetch) {
import('node-fetch').then(({ default: fetch, Headers, Request, Response }) => {
Object.assign(globalThis, { fetch, Headers, Request, Response })
})
}
In case someone is solving this issue in 2022:
Recent versions of nodejs already includes global fetch api. It is still an experimental feature and may be not ready to be used in production, but if you are writing some basic script or doing some sandbox project, you should definitely try it. Works without issues for me.
This is an official announcement and some examples here and here
I think this link is a comprehensive explanation about Error [ERR_REQUIRE_ESM]: require() of ES [node-fetch Module ] not supported.
node-fetch Error [ERR_REQUIRE_ESM]: require() of ES Module not supported
I'm bundling a JS library using Rollup. This lib has a dependency on #tensorflow/tfjs-core.
On tfjs's code, there's a function that fetches a URL. If it's in the browser environment, it uses the global fetch function; if it's not, it tries to import node-fetch.
Something among these lines:
fetch(path: string, requestInits?: RequestInit): Promise<Response> {
if (env().global.fetch != null) {
return env().global.fetch(path, requestInits);
}
if (systemFetch == null) {
systemFetch = require('node-fetch');
}
return systemFetch(path, requestInits);
}
My library is made to run in the browser, so it always uses the global fetch function. However, Rollup still bundles node-fetch's require in my lib's assets.
It should not be an issue, but some consumers are reporting errors when using the library in a React project that uses webpack:
Failed to compile.
./node_modules/[my lib]/index.js
Cannot find module: 'node-fetch'. Make sure this package is installed.
You can install this package by running: npm install node-fetch.
Question is: is there some way I can tell Rollup not no bundle this?
I thought about replacing the require('node-fetch') by undefined after the bundle is generated, but it feels like a dirty hack. Any other sugestions?
PS: I believe marking node-fetch as external on consumer projects would fix the issue, but since I do not use node-fetch in my lib, it would be nice to remove it from final output.
Other package managers can include or exclude files based on the environment, test, development, production, etc.
There is any number of ways of implementing this, even going so far as
# Makefile
ENVIRONMENT ?= test
ROLLUP = $(which rollup)
ENVSUBST = $(which envsubst)
rollup.config.js: src/$(ENVIRONMENT)
${ENVSUBST} < $# > $^
${ROLLUP} $^ -o $(ENVIRONMENT).js
If you created files named after your environments, you could compile them using
make -e environment=browser
I don't expect my code to work, only to express ideas.
There is this loc which is used to exclude node-fetch from the bundle. You could consider a similar approach in your rollup configuration. (I think) If you add that, node-fetch will/should not be a part of your minified library.
I'm creating a program to analyze security camera streams and got stuck on the very first line. At the moment my .js file has nothing but the import of node-fetch and it gives me an error message. What am I doing wrong?
Running Ubuntu 20.04.2 LTS in Windows Subsystem for Linux.
Node version:
user#MYLLYTIN:~/CAMSERVER$ node -v
v14.17.6
node-fetch package version:
user#MYLLYTIN:~/CAMSERVER$ npm v node-fetch
node-fetch#3.0.0 | MIT | deps: 2 | versions: 63
A light-weight module that brings Fetch API to node.js
https://github.com/node-fetch/node-fetch
keywords: fetch, http, promise, request, curl, wget, xhr, whatwg
dist
.tarball: https://registry.npmjs.org/node-fetch/-/node-fetch-3.0.0.tgz
.shasum: 79da7146a520036f2c5f644e4a26095f17e411ea
.integrity: sha512-bKMI+C7/T/SPU1lKnbQbwxptpCrG9ashG+VkytmXCPZyuM9jB6VU+hY0oi4lC8LxTtAeWdckNCTa3nrGsAdA3Q==
.unpackedSize: 75.9 kB
dependencies:
data-uri-to-buffer: ^3.0.1 fetch-blob: ^3.1.2
maintainers:
- endless <jimmy#warting.se>
- bitinn <bitinn#gmail.com>
- timothygu <timothygu99#gmail.com>
- akepinski <npm#kepinski.ch>
dist-tags:
latest: 3.0.0 next: 3.0.0-beta.10
published 3 days ago by endless <jimmy#warting.se>
esm package version:
user#MYLLYTIN:~/CAMSERVER$ npm v esm
esm#3.2.25 | MIT | deps: none | versions: 140
Tomorrow's ECMAScript modules today!
https://github.com/standard-things/esm#readme
keywords: commonjs, ecmascript, export, import, modules, node, require
dist
.tarball: https://registry.npmjs.org/esm/-/esm-3.2.25.tgz
.shasum: 342c18c29d56157688ba5ce31f8431fbb795cc10
.integrity: sha512-U1suiZ2oDVWv4zPO56S0NcR5QriEahGtdN2OR6FiOG4WJvcjBVFB0qI4+eKoWFH483PKGuLuu6V8Z4T5g63UVA==
.unpackedSize: 308.6 kB
maintainers:
- jdalton <john.david.dalton#gmail.com>
dist-tags:
latest: 3.2.25
published over a year ago by jdalton <john.david.dalton#gmail.com>
Contents of the .js file (literally nothing but the import):
user#MYLLYTIN:~/CAMSERVER$ cat server.js
import fetch from "node-fetch";
Result:
user#MYLLYTIN:~/CAMSERVER$ node -r esm server.js
/home/user/CAMSERVER/node_modules/node-fetch/src/index.js:1
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/user/CAMSERVER/node_modules/node-fetch/src/index.js
require() of ES modules is not supported.
require() of /home/user/CAMSERVER/node_modules/node-fetch/src/index.js from /home/user/CAMSERVER/server.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/user/CAMSERVER/node_modules/node-fetch/package.json.
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1089:13) {
code: 'ERR_REQUIRE_ESM'
}
user#MYLLYTIN:~/CAMSERVER$
From the Upgrade Guide
node-fetch was converted to be a ESM only package in version 3.0.0-beta.10. node-fetch is an ESM-only module - you are not able to import it with require.
Alternatively, you can use the async import() function from CommonJS to load node-fetch asynchronously:
// mod.cjs
const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));
or you stay on v2 as they say in the README
node-fetch is an ESM-only module - you are not able to import it with require. We recommend you stay on v2 which is built with CommonJS unless you use ESM yourself. We will continue to publish critical bug fixes for it.
EDIT
I saw that in the doc, used it, and the script crashes with :
error TS2556: A spread argument must either have a tuple type or be passed to a rest parameter.
Since you are using typescript you should do something like this
import { RequestInfo, RequestInit } from "node-fetch";
const fetch = (url: RequestInfo, init?: RequestInit) => import("node-fetch").then(({ default: fetch }) => fetch(url, init));
To fix this, I downgraded node-fetch to the latest version 2, which right now is 2.6.6. This is the script I ran:
yarn add node-fetch#^2.6.6
or
npm install node-fetch#^2.6.6
I also added these compiler options:
{
"compilerOptions": { "allowJs": true, "outDir": "./dist" },
}
node-fetch v3 is ESM-only: https://github.com/node-fetch/node-fetch#loading-and-configuring-the-module. The esm module you’re adding is for adding ESM compatibility, but it’s unnecessary now that Node 12+ supports ESM natively; and it doesn’t work with ESM-only packages like node-fetch 3+.
To fix your issue:
Remove the esm package.
Add "type": "module" to your package.json.
And that’s it. Then when you run node server.js it should work.
I always had errors with previous solutions, this is what worked every single time for me
npm i -D node-fetch
const _importDynamic = new Function('modulePath', 'return import(modulePath)');
export const fetch = async function (...args: any) {
const {default: fetch} = await _importDynamic('node-fetch');
return fetch(...args);
}
Use ESM syntax, also use one of these methods before running the file.
specify "type":"module" in package.json
Or use this flag --input-type=module when running the file
Or use .mjs file extension
Many times that error happens through the current version of fetch what we can do to solve it, is to install a previous version I tried it and it worked for me.
to install the previous version:
npm i node-fetch#2.6.1 or whatever you prefer.
Since the question was really about requiring node-fetch and there are definitely reasons that a developer might need to use require vs import, the answer to import the module using import() is somewhat accurate but isn't complete, because it ignores the fact that using import is async and if you're using node-fetch in your code in multiple places you're going to have a bit of an async mess all over the place whenever you want to use node-fetch, so the answer is very incomplete IMHO. A more complete answer would include a better how-to. Something like the following:
You need to use import() rather than require because this is an ES module. To avoid an async mess, you'll want to await the importing of this module once and then be able to use it wherever else you need it. To do that, create a module that imports any of your ES-only modules something like this:
"use strict";
let got;
let fetch;
module.exports.load = async function() {
queueMicrotask(function() {
// any ESM-only module that needs to be loaded can be loaded here, just add import for each using specific structure of each
import("got").then(({default: Got}) => got = Got );
fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));
});
while(
// check each module to see if it's been loaded here
!got || !got.get || !fetch || typeof fetch !== "function"
) {
// waiting for modules to load
console.log("Waiting for ES-Only modules to load...");
await new Promise((resolve)=>setTimeout(resolve, 1000));
}
module.exports.got = got;
module.exports.fetch = fetch;
console.log("ES-Only modules finished loading!");
}
This then allows you to call the await the loading of the ES-Only modules once and then use the module later by grabbing it from your intermediary module like this:
"use strict";
const esmModules = require("esm_modules"); // or whatever you called the intermiary module
async function doMyFetching(url, options) {
const fetch = esmModules.fetch;
const result = await fetch(url, options)
}
The nice part of doing it this way is that you know that your ES-Only modules are loaded and you know that once they're loaded you can use them at will throughout the rest of your code in a simple and clean way that doesn't force you to add additional async logic all over the place whenever you need the module.
Here is in Typescript, this will install to the global scope, just import the file and fetch ready to use:
/* filename: fetch.ts */
if (!globalThis.fetch) {
import('node-fetch').then(({ default: fetch, Headers, Request, Response }) => {
Object.assign(globalThis, { fetch, Headers, Request, Response })
})
}
In case someone is solving this issue in 2022:
Recent versions of nodejs already includes global fetch api. It is still an experimental feature and may be not ready to be used in production, but if you are writing some basic script or doing some sandbox project, you should definitely try it. Works without issues for me.
This is an official announcement and some examples here and here
I think this link is a comprehensive explanation about Error [ERR_REQUIRE_ESM]: require() of ES [node-fetch Module ] not supported.
node-fetch Error [ERR_REQUIRE_ESM]: require() of ES Module not supported
I was about to publish a module to NPM, when I thought about rewriting it in ES6, to both future-proof it, and learn ES6. I've used Babel to transpile to ES5, and run tests. But I'm not sure how to proceed:
Do I transpile, and publish the resulting /out folder to NPM?
Do I include the result folder in my Github repo?
Or do I maintain 2 repos, one with the ES6 code + gulp script for Github, and one with the transpiled results + tests for NPM?
In short: what steps do I need to take to publish a module written in ES6 to NPM, while still allowing people to browse/fork the original code?
The pattern I have seen so far is to keep the es6 files in a src directory and build your stuff in npm's prepublish to the lib directory.
You will need an .npmignore file, similar to .gitignore but ignoring src instead of lib.
I like José's answer. I've noticed several modules follow that pattern already. Here's how you can easily implement it with Babel6. I install babel-cli locally so the build doesn't break if I ever change my global babel version.
.npmignore
/src/
.gitignore
/lib/
/node_modules/
Install Babel
npm install --save-dev babel-core babel-cli babel-preset-es2015
package.json
{
"main": "lib/index.js",
"scripts": {
"prepublish": "babel src --out-dir lib"
},
"babel": {
"presets": ["es2015"]
}
}
TL;DR - Don't, until ~October 2019. The Node.js Modules Team has asked:
Please do not publish any ES module packages intended for use by Node.js until [October 2019]
2019 May update
Since 2015 when this question was asked, JavaScript support for modules has matured significantly, and is hopefully going to be officially stable in October 2019. All other answers are now obsolete or overly complicated. Here is the current situation and best practice.
ES6 support
99% of ES6 (aka 2015) has been supported by Node since version 6. The current version of Node is 12. All evergreen browsers support the vast majority of ES6 features. ECMAScript is now at version 2019, and the versioning scheme now favors using years.
ES Modules (aka ECMAScript modules) in browsers
All evergreen browsers have been supporting import-ing ES6 modules since 2017. Dynamic imports are supported by Chrome (+ forks like Opera and Samsung Internet) and Safari. Firefox support is slated for the next version, 67.
You no longer need Webpack/rollup/Parcel etc. to load modules. They may be still useful for other purposes, but are not required to load your code. You can directly import URLs pointing to ES modules code.
ES modules in Node
ES modules (.mjs files with import/export) have been supported since Node v8.5.0 by calling node with the --experimental-modules flag. Node v12, released in April 2019, rewrote the experimental modules support. The most visible change is that the file extension needs to be specified by default when importing:
// lib.mjs
export const hello = 'Hello world!';
// index.mjs:
import { hello } from './lib.mjs';
console.log(hello);
Note the mandatory .mjs extensions throughout. Run as:
node --experimental-modules index.mjs
The Node 12 release is also when the Modules Team asked developers to not publish ES module packages intended for use by Node.js until a solution is found for using packages via both require('pkg') and import 'pkg'. You can still publish native ES modules intended for browsers.
Ecosystem support of native ES modules
As of May 2019, ecosystem support for ES Modules is immature. For example, test frameworks like Jest and Ava don't support --experimental-modules. You need to use a transpiler, and must then decide between using the named import (import { symbol }) syntax (which won't work with most npm packages yet), and the default import syntax (import Package from 'package'), which does work, but not when Babel parses it for packages authored in TypeScript (graphql-tools, node-influx, faast etc.) There is however a workaround that works both with --experimental-modules and if Babel transpiles your code so you can test it with Jest/Ava/Mocha etc:
import * as ApolloServerM from 'apollo-server'; const ApolloServer = ApolloServerM.default || ApolloServerM;
Arguably ugly, but this way you can write your own ES modules code with import/export and run it with node --experimental-modules, without transpilers. If you have dependencies that aren't ESM-ready yet, import them as above, and you'll be able to use test frameworks and other tooling via Babel.
Previous answer to the question - remember, don't do this until Node solves the require/import issue, hopefully around October 2019.
Publishing ES6 modules to npm, with backwards compatibility
To publish an ES module to npmjs.org so that it can be imported directly, without Babel or other transpilers, simply point the main field in your package.json to the .mjs file, but omit the extension:
{
"name": "mjs-example",
"main": "index"
}
That's the only change. By omitting the extension, Node will look first for an mjs file if run with --experimental-modules. Otherwise it will fall back to the .js file, so your existing transpilation process to support older Node versions will work as before — just make sure to point Babel to the .mjs file(s).
Here's the source for a native ES module with backwards compatibility for Node < 8.5.0 that I published to NPM. You can use it right now, without Babel or anything else.
Install the module:
npm install local-iso-dt
# or, yarn add local-iso-dt
Create a test file test.mjs:
import { localISOdt } from 'local-iso-dt/index.mjs';
console.log(localISOdt(), 'Starting job...');
Run node (v8.5.0+) with the --experimental-modules flag:
node --experimental-modules test.mjs
TypeScript
If you develop in TypeScript, you can generate ES6 code and use ES6 modules:
tsc index.js --target es6 --modules es2015
Then, you need to rename *.js output to .mjs, a known issue that will hopefully get fixed soon so tsc can output .mjs files directly.
#Jose is right. There's nothing wrong with publishing ES6/ES2015 to NPM but that may cause trouble, specially if the person using your package is using Webpack, for instance, because normally people ignore the node_modules folder while preprocessing with babel for performance reasons.
So, just use gulp, grunt or simply Node.js to build a lib folder that is ES5.
Here's my build-lib.js script, which I keep in ./tools/ (no gulpor grunt here):
var rimraf = require('rimraf-promise');
var colors = require('colors');
var exec = require('child-process-promise').exec;
console.log('building lib'.green);
rimraf('./lib')
.then(function (error) {
let babelCli = 'babel --optional es7.objectRestSpread ./src --out-dir ./lib';
return exec(babelCli).fail(function (error) {
console.log(colors.red(error))
});
}).then(() => console.log('lib built'.green));
Here's a last advice: You need to add a .npmignore to your project. If npm publish doesn't find this file, it will use .gitignore instead, which will cause you trouble because normally your .gitignore file will exclude ./lib and include ./src, which is exactly the opposite of what you want when you are publishing to NPM. The .npmignore file has basically the same syntax of .gitignore (AFAIK).
Following José and Marius's approach, (with update of Babel's latest version in 2019): Keep the latest JavaScript files in a src directory, and build with npm's prepublish script and output to the lib directory.
.npmignore
/src
.gitignore
/lib
/node_modules
Install Babel (version 7.5.5 in my case)
$ npm install #babel/core #babel/cli #babel/preset-env --save-dev
package.json
{
"name": "latest-js-to-npm",
"version": "1.0.0",
"description": "Keep the latest JavaScript files in a src directory and build with npm's prepublish script and output to the lib directory.",
"main": "lib/index.js",
"scripts": {
"prepublish": "babel src -d lib"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"#babel/cli": "^7.5.5",
"#babel/core": "^7.5.5",
"#babel/preset-env": "^7.5.5"
},
"babel": {
"presets": [
"#babel/preset-env"
]
}
}
And I have src/index.js which uses the arrow function:
"use strict";
let NewOneWithParameters = (a, b) => {
console.log(a + b); // 30
};
NewOneWithParameters(10, 20);
Here is the repo on GitHub.
Now you can publish the package:
$ npm publish
...
> latest-js-to-npm#1.0.0 prepublish .
> babel src -d lib
Successfully compiled 1 file with Babel.
...
Before the package is published to npm, you will see that lib/index.js has been generated, which is transpiled to es5:
"use strict";
var NewOneWithParameters = function NewOneWithParameters(a, b) {
console.log(a + b); // 30
};
NewOneWithParameters(10, 20);
[Update for Rollup bundler]
As asked by #kyw, how would you integrate Rollup bundler?
First, install rollup and rollup-plugin-babel
npm install -D rollup rollup-plugin-babel
Second, create rollup.config.js in the project root directory
import babel from "rollup-plugin-babel";
export default {
input: "./src/index.js",
output: {
file: "./lib/index.js",
format: "cjs",
name: "bundle"
},
plugins: [
babel({
exclude: "node_modules/**"
})
]
};
Lastly, update prepublish in package.json
{
...
"scripts": {
"prepublish": "rollup -c"
},
...
}
Now you can run npm publish, and before the package is published to npm, you will see that lib/index.js has been generated, which is transpiled to es5:
'use strict';
var NewOneWithParameters = function NewOneWithParameters(a, b) {
console.log(a + b); // 30
};
NewOneWithParameters(10, 20);
Note: by the way, you no longer need #babel/cli if you are using the Rollup bundler. You can safely uninstall it:
npm uninstall #babel/cli
If you want to see this in action in a very simple small open source Node module then take a look at nth-day (which I started - also other contributors). Look in the package.json file and at the prepublish step which will lead you to where and how to do this. If you clone that module you can run it locally and use it as a template for yous.
Node.js 13.2.0+ supports ESM without the experimental flag and there're a few options to publish hybrid (ESM and CommonJS) NPM packages (depending on the level of backward compatibility needed): https://2ality.com/2019/10/hybrid-npm-packages.html
I recommend going the full backward compatibility way to make the usage of your package easier. This could look as follows:
The hybrid package has the following files:
mypkg/
package.json
esm/
entry.js
commonjs/
package.json
entry.js
mypkg/package.json
{
"type": "module",
"main": "./commonjs/entry.js",
"exports": {
"./esm": "./esm/entry.js"
},
"module": "./esm/entry.js",
···
}
mypkg/commonjs/package.json
{
"type": "commonjs"
}
Importing from CommonJS:
const {x} = require('mypkg');
Importing from ESM:
import {x} from 'mypkg/esm';
We did an investigation into ESM support in 05.2019 and found that a lot of libraries were lacking support (hence the recommendation for backward compatibility):
esm package's support doesn't align with Node's which causes issues
"Builtin require cannot sideload .mjs files." https://github.com/standard-things/esm#loading, https://github.com/standard-things/esm/issues/498#issuecomment-403496745
"The .mjs file extension should not be the thing developers reach for if they want interop or ease of use. It's available since it's in --experimental-modules but since it's not fully baked I can't commit to any enhancements to it." https://github.com/standard-things/esm/issues/498#issuecomment-403655466
mocha doesn't have native support for .mjs files
Update 2020-01-13: Mocha released experimental support in mocha#7.0.0-esm1
Many high-profile projects had issues with .mjs files:
create-react-app
react-apollo
graphql-js
inferno
The main key in package.json decides the entry point to the package once it's published. So you can put your Babel's output wherever you want and just have to mention the right path in main key.
"main": "./lib/index.js",
Here's a well written article on how to publish an npm package
https://codeburst.io/publish-your-own-npm-package-ff918698d450
Here's a sample repo you can use for reference
https://github.com/flexdinesh/npm-module-boilerplate
The two criteria of an NPM package is that it is usable with nothing more than a require( 'package' ) and does something software-ish.
If you fulfill those two requirements, you can do whatever you wish.
Even if the module is written in ES6, if the end user doesn't need to know that, I would transpile it for now to get maximum support.
However, if like koa, your module requires compatibility with users using ES6 features, then perhaps the two package solution would be a better idea.
Takeaway
Only publish as much code as you need to make require( 'your-package' ) work.
Unless the between ES5 & 6 matters to the user, only publish 1 package. Transpile it if you must.
A few extra notes for anyone, using own modules directly from github, not going through published modules:
The (widely used) "prepublish" hook is not doing anything for you.
Best thing one can do (if plans to rely on github repos, not published stuff):
unlist src from .npmignore (in other words: allow it). If you don't have an .npmignore, remember: A copy of .gitignore will be used instead in the installed location, as ls node_modules/yourProject will show you.
make sure, babel-cli is a depenency in your module, not just a devDepenceny since you are indeed building on the consuming machine aka at the App developers computer, who is using your module
do the build thing, in the install hook i.e.:
"install": "babel src -d lib -s"
(no added value in trying anything "preinstall", i.e. babel-cli might be missing)
Deppending on the anatomy of your module, this solution may not work, but if your module is contained inside a single file, and has no dependencies (does not make use of import), using the following pattern you can release your code as it is, and will be able to be imported with import (Browser ES6 Modules) and require (Node CommonJS Modules)
As a bonus, it will be suittable to be imported using a SCRIPT HTML Element.
main.js :
(function(){
'use strict';
const myModule = {
helloWorld : function(){ console.log('Hello World!' )}
};
// if running in NODE export module using NODEJS syntax
if(typeof module !== 'undefined') module.exports = myModule ;
// if running in Browser, set as a global variable.
else window.myModule = myModule ;
})()
my-module.js :
// import main.js (it will declare your Object in the global scope)
import './main.js';
// get a copy of your module object reference
let _myModule = window.myModule;
// delete the the reference from the global object
delete window.myModule;
// export it!
export {_myModule as myModule};
package.json :`
{
"name" : "my-module", // set module name
"main": "main.js", // set entry point
/* ...other package.json stuff here */
}
To use your module, you can now use the regular syntax ...
When imported in NODE ...
let myModule = require('my-module');
myModule.helloWorld();
// outputs 'Hello World!'
When imported in BROWSER ...
import {myModule} from './my-module.js';
myModule.helloWorld();
// outputs 'Hello World!'
Or even when included using an HTML Script Element...
<script src="./main.js"></script>
<script>
myModule.helloWorld();
// outputs 'Hello World!'
</script>