I'm operating a bot on Wikipedia using npm mwbot, and planning to migrate to npm mwn. This is because you need a "token" to edit pages on Wikipedia, and this can expire after a while so you need to prepare your own countermeasures against this if you use mwbot, but it seems like mwn handles this issue on its own.
When you use mwn, you first need to initialize a bot instance as documented on the turotial:
const bot = await mwn.init(myUserInfo);
Then your token is stored in the bot instance and you can for example edit a page using:
const result = await bot.edit('Page title', () => {text: 'text'});
So, basically you want to share the initialized bot instance across modules. I believe it'd be easiest to declare a global variable like so:
// bot.js (main script)
const {mwn} = require('mwn');
const {my} = require('./modules/my');
(async() => {
global.mw = await mwn.init(my.userinfo);
const {t} = require('./modules/test');
t();
})();
// modules/test.js
/* global mw */
exports.t = async () => {
const content = await mw.read('Main page');
console.log(content);
return content;
};
I'm currently using JavaScript, but will hopefully migrate to TypeScript (although I'm new to it) because I feel like it'd be useful in developing some of my modules. But I'm stuck with how I should use the initialized bot instance across modules in TypeScript.
-+-- dist (<= where ts files are compiled into)
+-- src
+-- types
+-- global.d.ts
+-- bot.ts
+-- my.ts
// bot.ts
import {mwn} from 'mwn';
import {my} from './my';
(async() => {
global.mw = await mwn.init(my.userinfo);
})();
// global.d.ts
import {mwn} from 'mwn';
declare global {
// eslint-disable-next-line no-var
var mw: mwn;
}
This doesn't work and returns "Element implicitly has an 'any' type because type 'typeof globalThis' has no index signature. (at mw in global.mw)".
This is probably a naive question but any help would be appreciated.
Edit:
Thanks #CertainPerformance, that's a simple and easy approach. Actually, I once tried the same kind of an approach:
export const init = async () => {
if (typeof mw === 'undefined') {
return Promise.resolve(mw);
} else {
return mwn.init(my.userinfo).then((res) => {
mw = res;
return mw;
});
}
}
But I was like "init().then() in every module?"... don't know why I didn't come up with just exporting the initialized mwn instance.
Anyway, is it like the entry point file should be a .js file? I've been trying with a .ts file and this is one thing that's been giving me a headache. I'm using ts-node or nodemon to auto-compile .ts files, but without "type: module", "Cannot use import statement outside a module" error occurs and with that included, "TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts"" occurs. How do you tell a given file should be a .js or .ts file?
Edit2:
Just making a note: The error I mentioned above was caused by not having "module": "CommonJS" in my tsconfig.json, as I commented to CertainPerformance's answer below.
One of the main benefits of modules is the ability to drop dependencies on global variables. Rather than going back on that and assigning a global anyway, a better approach that happens to solve your problem as well would be to have a module that exports two functions:
One that initializes the asynchronous mwn
One that returns mwn when called
// mw.ts
import {mwn} from 'mwn';
import {my} from './my';
let mw: mwn;
export const init = async () => {
mw = await mwn.init(my.userinfo);
};
export const getMw = () => mw;
Then it can be consumed by other modules quite naturally, barely requiring any typing at all:
// modules/index.ts
// Entry point
import { init } from './mw';
import { test } from './test';
init()
.then(() => {
test();
})
// .catch(handleErrors);
// modules/test.ts
import { getMw } from './bot';
export const test = () => {
const mw = getMw();
// anything else that depends on mw
};
The above could be a reasonable approach if mw is used in many places across your app and you don't think that passing it around everywhere would be maintainable.
If you could pass it around everywhere, that would have even less of a code smell, though.
// initMw.ts
import {mwn} from 'mwn';
import {my} from './my';
export const initMw = () => mwn.init(my.userinfo);
// modules/index.ts
// Entry point
import { initMw } from './initMw';
import { test } from './test';
initMw()
.then((mw) => {
test(mw);
})
// .catch(handleErrors);
// modules/test.ts
import { mwn } from 'mwn';
export const test = (mw: mwn) => {
// anything that depends on mw
};
Initialize it once, then pass it down (synchronously) everywhere it's needed - that's the approach I'd prefer.
You could put the mwn type in a global d.ts file to avoid having to add import { mwn } from 'mwn'; to every module if you wanted. (Yes, it's somewhat of a global variable, but it's a type rather than a value, so it's arguably less of a problem.)
I have written a javascript file that provides configuration, and separately a proxy that lets me provide a fallback value if the property access was unsuccessfull
// stackUrlMap.js
export const stacks = {
'stackoverflow.com': 'StackOverflow',
'cooking.stackexchange.com': 'SeasonedAdvice',
}
// stackUrlConfig.js
import { stacks } from './stackUrlMap';
const getOrParse = parser => ({
get: (target, prop) => target?.[prop] ?? parser(prop);
})
const urlToName = url => url.split('.')[0];
export const stackUrlToNameMap = new Proxy(stacks, getOrParse(urlToName));
This was working perfectly, and I had a test written for it that passed.
import { expect } from 'chai';
import sinon from 'sinon';
import * as urlMap from './stackUrlMap';
describe('stackUrlConfig.js', function() {
let sandbox;
let stackUrlToNameMap = {};
beforeEach(async function() {
sandbox = sinon.createSandbox();
sandbox.stub(urlMap, 'stack').value({'madeup.url.com': 'Made Up Stack'});
// import here so we get the stubbed values passed through.
({ stackUrlToNameMap } = await import('./stackUrlConfig.js'))
});
it('can get an existing prop from the proxied map', function() {
const field = 'madeup.url.com';
// stub working correctly when accessed from test file
expect(urlMap.stacks[field]).to.exist();
// tests that if the proxied map can find the prop it uses that value
expect(stackUrlToNameMap[field]).to.equal('Made Up Stack');
});
})
However, now I've written code that uses stackUrlConfig.js elsewhere, that is imported by my test setup elsewhere, and the dynamically imported proxied map stackUrlToNameMap in the beforeEach of my test is not being 'respected', and the real stacks object is being used for the lookup, meaning I get this AssertionError:
AssertionError: expected 'madeup' to equal 'Made Up Stack'
If I put a console.trace in the stackUrlConfig.js, I can see it's first called before my test runs as it's imported into another file that isn't run by mocha yet.
I don't want to have to use decache (I'm not even sure it would work, I'm not using require) as seen in Re-importing modules between mocha tests if I can avoid it.
I took the code and tried to add a cache busting, like so:
const absolutePath = path.resolve(__dirname, './stackUrlConfig.js');
({ stackUrlToNameMap } = await import(`${absolutePath}?update=${Date.now()}`));
But my mocha setup did not find the resulting module.
I can't control whether this file is imported elsewhere first in my test, so can I reset mocha's reference to the module somehow?
I have a custom NextJS webpack config where I call a script that gets executed at build-time in the Node runtime (not in the browser):
module.exports = {
webpack: (config, options) => {
if (options.isServer) {
require("./some-script.js")
}
return config
},
}
What the script does doesn't really matter, but I would like to import some code from my React application (that gets executed in the browser, except for SSR, but that's beside the point).
const fs = require("fs");
const getImages = require("../utils/img");
(() => {
// writes something to disk
})();
When I do this I get this error message: SyntaxError: Cannot use import statement outside a module. This makes sense because I use require for const getImages = require("../utils/img"); but this file is part of the React client-side codebase which uses ES6 import/export modules:
const getImages = () => {
// return something
};
export default getImages;
How can I achieve this? Meaning, how can I call client-side import/export code from a Node.js require script?
Try and convert the required files to use require and module.exports to import and export respectively
const getImages = () => {
// return something
};
module.exports = getImages;
I'd like to export a different package from a custom module based on an expression
const settings = {}
const init = (sentry) => {
sentry.init(settings)
return sentry
}
const pkg = async () => {
let mod
if (__CLIENT__) {
mod = await import('#sentry/browser').then(init)
} else {
mod = await import('#sentry/node').then(init)
}
return mod
}
const Sentry = pkg()
console.log({ Sentry })
export default Sentry
However, when i import this file later i receive a pending promise
import Sentry from 'config/sentry'
console.log(Sentry) // -> promise pending
Is it possible to default export the dynamically imported module at the top level?
update
I ended up going with require instead of dynamic imports, as supported by webpack globals setup on my system
const Sentry = __CLIENT__ ? require('#sentry/browser') : require('#sentry/node')
Async functions always return a promise. That's by definition. Since dynamic imports also return promises (and are async), you can't export them, since exports have to be at the top level of a script. The best you can do is export the promise and include a .then handler in whichever script imports it; since you're returning mod from your async function, that will be passed to the .then handler's parameter for you to use there.
This is a trivial example that illustrates the crux of my problem:
var innerLib = require('./path/to/innerLib');
function underTest() {
return innerLib.doComplexStuff();
}
module.exports = underTest;
I am trying to write a unit test for this code. How can I mock out the requirement for the innerLib without mocking out the require function entirely?
So this is me trying to mock out the global require and finding out that it won’t work even to do that:
var path = require('path'),
vm = require('vm'),
fs = require('fs'),
indexPath = path.join(__dirname, './underTest');
var globalRequire = require;
require = function(name) {
console.log('require: ' + name);
switch(name) {
case 'connect':
case indexPath:
return globalRequire(name);
break;
}
};
The problem is that the require function inside the underTest.js file has actually not been mocked out. It still points to the global require function. So it seems that I can only mock out the require function within the same file I’m doing the mocking in. If I use the global require to include anything, even after I’ve overridden the local copy, the files being required will still have the global require reference.
You can now!
I published proxyquire which will take care of overriding the global require inside your module while you are testing it.
This means you need no changes to your code in order to inject mocks for required modules.
Proxyquire has a very simple api which allows resolving the module you are trying to test and pass along mocks/stubs for its required modules in one simple step.
#Raynos is right that traditionally you had to resort to not very ideal solutions in order to achieve that or do bottom-up development instead
Which is the main reason why I created proxyquire - to allow top-down test driven development without any hassle.
Have a look at the documentation and the examples in order to gauge if it will fit your needs.
A better option in this case is to mock methods of the module that gets returned.
For better or worse, most node.js modules are singletons; two pieces of code that require() the same module get the same reference to that module.
You can leverage this and use something like sinon to mock out items that are required. mocha test follows:
// in your testfile
var innerLib = require('./path/to/innerLib');
var underTest = require('./path/to/underTest');
var sinon = require('sinon');
describe("underTest", function() {
it("does something", function() {
sinon.stub(innerLib, 'toCrazyCrap').callsFake(function() {
// whatever you would like innerLib.toCrazyCrap to do under test
});
underTest();
sinon.assert.calledOnce(innerLib.toCrazyCrap); // sinon assertion
innerLib.toCrazyCrap.restore(); // restore original functionality
});
});
Sinon has good integration with chai for making assertions, and I wrote a module to integrate sinon with mocha to allow for easier spy/stub cleanup (to avoid test pollution.)
Note that underTest cannot be mocked in the same way, as underTest returns only a function.
Another option is to use Jest mocks. Follow up on their page
I use mock-require. Make sure you define your mocks before you require the module to be tested.
Simple code to mock modules for the curious
Notice the parts where you manipulate the require.cache and note require.resolve method as this is the secret sauce.
class MockModules {
constructor() {
this._resolvedPaths = {}
}
add({ path, mock }) {
const resolvedPath = require.resolve(path)
this._resolvedPaths[resolvedPath] = true
require.cache[resolvedPath] = {
id: resolvedPath,
file: resolvedPath,
loaded: true,
exports: mock
}
}
clear(path) {
const resolvedPath = require.resolve(path)
delete this._resolvedPaths[resolvedPath]
delete require.cache[resolvedPath]
}
clearAll() {
Object.keys(this._resolvedPaths).forEach(resolvedPath =>
delete require.cache[resolvedPath]
)
this._resolvedPaths = {}
}
}
Use like:
describe('#someModuleUsingTheThing', () => {
const mockModules = new MockModules()
beforeAll(() => {
mockModules.add({
// use the same require path as you normally would
path: '../theThing',
// mock return an object with "theThingMethod"
mock: {
theThingMethod: () => true
}
})
})
afterAll(() => {
mockModules.clearAll()
})
it('should do the thing', async () => {
const someModuleUsingTheThing = require('./someModuleUsingTheThing')
expect(someModuleUsingTheThing.theThingMethod()).to.equal(true)
})
})
BUT... jest has this functionality built in and I recommend that testing framework over rolling your own for testing purposes.
Mocking require feels like a nasty hack to me. I would personally try to avoid it and refactor the code to make it more testable.
There are various approaches to handle dependencies.
1) pass dependencies as arguments
function underTest(innerLib) {
return innerLib.doComplexStuff();
}
This will make the code universally testable. The downside is that you need to pass dependencies around, which can make the code look more complicated.
2) implement the module as a class, then use class methods/ properties to obtain dependencies
(This is a contrived example, where class usage is not reasonable, but it conveys the idea)
(ES6 example)
const innerLib = require('./path/to/innerLib')
class underTestClass {
getInnerLib () {
return innerLib
}
underTestMethod () {
return this.getInnerLib().doComplexStuff()
}
}
Now you can easily stub getInnerLib method to test your code.
The code becomes more verbose, but also easier to test.
If you've ever used jest, then you're probably familiar with jest's mock feature.
Using "jest.mock(...)" you can simply specify the string that would occur in a require-statement in your code somewhere and whenever a module is required using that string a mock-object would be returned instead.
For example
jest.mock("firebase-admin", () => {
const a = require("mocked-version-of-firebase-admin");
a.someAdditionalMockedMethod = () => {}
return a;
})
would completely replace all imports/requires of "firebase-admin" with the object you returned from that "factory"-function.
Well, you can do that when using jest because jest creates a runtime around every module it runs and injects a "hooked" version of require into the module, but you wouldn't be able to do this without jest.
I have tried to achieve this with mock-require but for me it didn't work for nested levels in my source. Have a look at the following issue on github: mock-require not always called with Mocha.
To address this I have created two npm-modules you can use to achieve what you want.
You need one babel-plugin and a module mocker.
babel-plugin-mock-require
jestlike-mock
In your .babelrc use the babel-plugin-mock-require plugin with following options:
...
"plugins": [
["babel-plugin-mock-require", { "moduleMocker": "jestlike-mock" }],
...
]
...
and in your test file use the jestlike-mock module like so:
import {jestMocker} from "jestlike-mock";
...
jestMocker.mock("firebase-admin", () => {
const firebase = new (require("firebase-mock").MockFirebaseSdk)();
...
return firebase;
});
...
The jestlike-mock module is still very rudimental and does not have a lot of documentation but there's not much code either. I appreciate any PRs for a more complete feature set. The goal would be to recreate the whole "jest.mock" feature.
In order to see how jest implements that one can look up the code in the "jest-runtime" package. See https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/index.js#L734 for example, here they generate an "automock" of a module.
Hope that helps ;)
You can't. You have to build up your unit test suite so that the lowest modules are tested first and that the higher level modules that require modules are tested afterwards.
You also have to assume that any 3rd party code and node.js itself is well tested.
I presume you'll see mocking frameworks arrive in the near future that overwrite global.require
If you really must inject a mock you can change your code to expose modular scope.
// underTest.js
var innerLib = require('./path/to/innerLib');
function underTest() {
return innerLib.toCrazyCrap();
}
module.exports = underTest;
module.exports.__module = module;
// test.js
function test() {
var underTest = require("underTest");
underTest.__module.innerLib = {
toCrazyCrap: function() { return true; }
};
assert.ok(underTest());
}
Be warned this exposes .__module into your API and any code can access modular scope at their own danger.
You can use mockery library:
describe 'UnderTest', ->
before ->
mockery.enable( warnOnUnregistered: false )
mockery.registerMock('./path/to/innerLib', { doComplexStuff: -> 'Complex result' })
#underTest = require('./path/to/underTest')
it 'should compute complex value', ->
expect(#underTest()).to.eq 'Complex result'
I use a simple factory the returns a function that calls a function with all of its dependencies:
/**
* fnFactory
* Returns a function that calls a function with all of its dependencies.
*/
"use strict";
const fnFactory = ({ target, dependencies }) => () => target(...dependencies);
module.exports = fnFactory;
Wanting to test the following function:
/*
* underTest
*/
"use strict";
const underTest = ( innerLib, millions ) => innerLib.doComplexStuff(millions);
module.exports = underTest;
I would setup my test (I use Jest) as follows:
"use strict";
const fnFactory = require("./fnFactory");
const _underTest = require("./underTest");
test("fnFactory can mock a function by returng a function that calls a function with all its dependencies", () => {
const fake = millions => `Didn't do anything with ${millions} million dollars!`;
const underTest = fnFactory({ target: _underTest, dependencies: [{ doComplexStuff: fake }, 10] });
expect(underTest()).toBe("Didn't do anything with 10 million dollars!");
});
See results of test
In production code I would manually inject the callee's dependencies as below:
/**
* main
* Entry point for the real application.
*/
"use strict";
const underTest = require("./underTest");
const innerLib = require("./innerLib");
underTest(innerLib, 10);
I tend to limit the scope of most of the modules that I write to one thing, which reduces the number of dependencies that have to be accounted for when testing and integrating them into the project.
And that's my approach to dealing with dependencies.