Exclude node_modules recursively - javascript

I came across a specific issue when setting up Firebase in my project directory.
Initially, project's hierarchy could be summed up by the following :
.
./src
./src/my_code.js
./src/my_code.test.js
./node_modules
./node_modules/...
./package.json
To run my tests, I'm using mocha.
On this very website, I found a really useful regexp able to match all *.test.js files in the whole project hierarchy, except the node_modules "branch" :
./{,!(node_modules)/**}/*test.js
Next step for me was to initialize Firebase Functions & Hosting in the directory.
A design choice was made to set up the functions folder in ./src.
After running firebase init and answering all the questions, my folder looked like that :
.
./src
./src/my_code.js
./src/my_code.test.js
// ===== NEW =====
./src/functions
./src/functions/index.js
./src/functions/index.test.js
./src/functions/node_modules
./src/functions/node_modules/...
./src/functions/package.json
// ===============
./node_modules
.node_modules/...
./package.json
The problem lies in that the regexp appears not to exclude the ./src/functions/node_modules folder.
I'm not really at ease to extend this cool regexp, not knowing if I have to add ** or a !(src/functions/node_modules) somewhere.

I took a deep look into this mochajs' issue.
First of all, it appears that, as #Wiktor mentioned in my question's comments, ./{,!(node_modules)/**}/*test.js isn't a regex, but a glob.
I didn't investigated this concept further for now, because mochajs now implement an --exclude flag feature.
My solution is the following : mocha \"./**/*test.js\" --exclude \"./**/node_modules/**\".

Related

VSCode: trigger organizeImports when git staging

I like the automated organize feature in VSCode, but having it doing it on every save has given me some trouble.
...
"editor.codeActionsOnSave": {
"source.organizeImports": true
},
...
Is it possible to set up some git hooks that organize the imports (either via vscode or some other script/lib) when I stage them?
Issues
If I hit save too early (do it to kick off prettier all the time) before I have used the imported methods - then it removes it and I have to write the import again.
If I break the code (.jsx) and something appears to not be used and I hit save (to trigger prettier), then it removes the imports. I then have to import them again.
There is some form of hook that can be applied when running git add : filters defined in gitconfig and .gitattributes.
See this section of git book for detailed explanations.
Here are the sketches from the documentation (it illustrates how you can configure a filter to run on *.txt files) :
when running git add :
when running git checkout :
You can define in your gitconfig a filter, which consists of two commands to "clean" and "smudge" :
$ git config --global filter.jsximports.clean fiximports
$ git config --global filter.jsximports.smudge cat
and edit the .gitattributes file to apply this filter on jsx files
*.jsx filter=jsximports
The script to apply may be tslint --fix, with the ordered-imports rule.
Actually : tslint's rule seem to have its own implementation, but it does something similar (see https://github.com/palantir/tslint/pull/4064)
In this answer : https://stackoverflow.com/a/57458656/86072
user thorn points to this npm package :
https://www.npmjs.com/package/organize-imports-cli
which calls organizeImports from cli

Measure Babel compilation performance (per file or module)

I'm trying to figure out how to extract some information from babel compilation process.
More specifically, when I run babel (no matter if using Webpack's babel-loader, test frameworks' transformers, Babel's CLI, etc) I'd need to extract some information for each compiled file. Like:
file path
time taken to compile
any other meta data?
What I've tried so far
Speed Measure Plugin for Webpack (link)
Works fine but it provides only Webpack's loaders running time. No info about single compiled files.
Hook into Webpack's compiler/compilation instance
I considered writing a Webpack plugin to hook into the compilation process as described here, but I couldn't find the proper hooks to recognize a file being processed by babel.
Updates
I guess #kidroca pointed out the right direction. More specifically I understand that Babel’s wrapPluginVisitorMethod option is the key for hooking into Babel compilation process.
See babel-minify’s timing plugin.
Related threads:

https://github.com/babel/babel/issues/5340
https://github.com/babel/babel/issues/2206
https://github.com/babel/babel/pull/3659
https://github.com/babel/minify/pull/93
https://github.com/babel/babel/pull/3659
Updates 28/04/18
I eventually tried to wrap a solution into a tool I called babel-timing.
You can use #babel/core and babel.transformSync(code) which will return
Abstract Syntax Tree (AST) information along with some other data. And you can also add some logic to measure the performance of this method
I've setup a minimal repo and played with it myself a little bit: https://github.com/kidroca/babel-meta
Basically you can run npm run analyze-file ./es6-src/es6-module.js or npm run analyze-dir ./es6-src/es6-module.js and checkout the results
This will return:
{
"filename": "/full/path/to/src/file.js",
"cwd": "current/dir",
"ast": "ast information json - lines, comments, and other info",
"executionTime": "execution time in ms",
/* a lot of other info */
}
You can modify the analyze.js file to filter out the info you need
You can modify the .babelrc file to control the transformation and add/remove plugins

How can I select all folders and subfolders any layers deep with custom file ending?

I made a React app using Create-React-App. I have a testing script in my React app's package.json like this:
"test": "node -r #babel/register -r #babel/polyfill **/*.test.js | tap-color",
This catches and executes the files in src/ like src/App.test.js, but not header.test.js in src/layout/header/header.test.js. And what if I add even more layers of folders? Is there a regex that will catch all folders in src/ and their subfolders no matter how nested with a file ending of .test.js?
Edit: I found this question on StackOverflow. According to that answer you would have to write:
"test": "node -r #babel/register -r #babel/polyfill 'src/**/*.test.js' | tap-color",
which unfortunately matches nothing for me. I'm on Mac.
The question you refer to is not especially useful for your case. My answer there works because Mocha has been designed to pass the patterns you give to it to the glob library for interpretation. So when you do mocha 'src/app/**/*.tests.js' the quotes prevent the shell from interpreting the pattern, Mocha gets src/app/**/*.tests.js as the first pattern given to it, which it gives to glob to get a list of files to actually run. Your case is different at least one crucial way: glob is not involved so there is nothing that can correctly interpret **.
Your first attempt is consistent with what happens when you are using a shell that does not understand **. It is interpreted exactly the same as *. So the shell interprets **/*.test.js as */*.test.js, expands this pattern and passes the result to node.
In your second attempt, you quote the pattern but that does not help you because node does not do pattern interpretation. It tries to load a file at path src/**/*.test.js, interpreted literally. This is not what you want.
I'm not sure what the compatibility implication with Windows are, but you could replace 'src/**/*.test.js' with $(find src -type f -name '*.test.js'). (See the man page for details.) At run-time, the shell will replace this with the result of the find command.
Or for greater simplicity, and less risk of platform issues creeping up, you could use babel-tap like this:
babel-tap 'src/**/*.test.js' | tap-color
If you use babel-tap, there's actually no need for using find because internally babel-tap calls on facilities that use the glob library to interpret the file names passed to it.
I've focused on the file pattern issue but I'm not seeing how what you're trying to do would work, even without the pattern issue. Consider this command, and assume that the files exist:
node -r #babel/register -r #babel/polyfill src/a.test.js src/b.test.js
This is not telling Node to run src/a.test.js and src/b.test.js. Rather, it tells node "run the script src/a.test.js and pass to it the parameter src/b.test.js". I've not used tap very much but I don't recall it working this way. Using babel-tap like I show above also avoids the problem here.

WebPack sourcemaps confusing (duplicated files)

I decided to try out WebPack on a new project I'm spinning up today and I'm getting really strange behavior from the sourcemaps. I can't find anything about it in the documentation, nor can I find anyone else having this issue when skimming StackOverflow.
I'm currently looking at the HelloWorld app produced by Vue-CLI's WebPack template -- no changes have been made to the code, the build environment, or anything.
I installed everything and ran it like so:
vue init webpack test && cd test && npm install && npm run dev
Looking at my sourcemaps, I see the following:
This is a hot mess. Why are there three version of HelloWorld.vue and App.vue? Worse yet, each version has a slightly different version of the code and none of them match the original source. The HellowWorld.vue sitting in the root directory does match the original source, but what's it doing down there instead of in the ./src/components folder? Finally, why isn't there a fourth App.vue that has the original source for it?
As far as I can tell this may have something to do with the WebPack loaders. I've never gotten these kinds of issues with any other bundler, though. Below is an example of the exact same steps using the Browserify Vue-CLI template:
No webpack:// schema, only one copy of every file, the files actually contain the original source code (kind of important for source maps), no unexpected (webpack)/buildin or (webpack)-hot-middleware, no . subdirectory,.... just the source code.
I haven't worked with Vue so can't really describe how exactly this is happening but it seems to be related to Vue Loader. Looking at the documentation I did not really find anything that clarifies why it would create three different files for one component. But it does seem logical considering that a .vue file might contain three types of top-level language blocks: <template>, <script>, and <style>.
Also, looking at two of those files you do see a comment at end of each file that suggests it was modified in some way by a Vue loader. Either this
//////////////////
// WEBPACK FOOTER
// ./node_modules/vue-loader/lib/template-compiler
or
//////////////////
// WEBPACK FOOTER
// ./node_modules/vue-style-loader!./node_modules/css-loader
The third file is different but it still does have code that identifies it as being modified by Vue loader. Here is some of that code
function injectStyle (ssrContext) {
if (disposed) return
require("!!vue-style-loader...")
}
/* script */
import __vue_script__ from "!!babel-loader!../../node_modules/vue-loader/..."
/* template */
import __vue_template__ from "!!../../node_modules/vue-loader/..."
/* styles */
var __vue_styles__ = injectStyle
The document also says this:
vue-loader is a loader for Webpack that can transform Vue components written in the following format into a plain JavaScript module:
Which explains why you might not see the same type of behaviour with other bundlers.
Now, This might not be the answer you were looking for but just wanted to share what I had found.
This is actually a feature of webpack.
webpack has HMR (Hot Module Reloading). If you look in your network tab, go ahead and make an update to your HelloWorld.vue file. You'll see a js chunk come thru as well as an updated JSON manifest. Both of these will have a unique hash at the end for each time you make a change to the application. It does this so the browser does not have to do a full reload.
For a better explanation of this I would highly recommend reading through https://webpack.js.org/concepts/hot-module-replacement/

how to bust #providesModule cache

I'm building a REST API that will live in AWS Lambda. I'm using apex to deploy it, and since I'm implementing it in javascript, I'm using webpack and babel to get my ES6+ goodies. I've discovered a problem with respect to #providesModule names, and I haven't been able to work around it.
I just created a new lambda function, and when I compile it, I can see from the output that one module couldn't be found -- but the other one can. Here's a sketch of the filesystem:
functions
├─ database.js #providesModule Database -- this works
├─ configuration.js #providesModule MyConfig -- this does not work
├── entity-post
├── index.js imports both 'Database' & 'MyConfig'
When I compile the lambda, the output indicates that the MyConfig dependency couldn't be resolved, but that Database could be. I've tried replacing the content of configuration.js with dead-simple code, in case resolution is failing on a basic parse error; no luck. I should add that I've got 5 other lambda functions in the same project that use similar imports, and they all work great.
I assume this is some kind of caching problem: the module resolution system hasn't noticed the #providesModule inside configuration.js.
How do I clear that cache? For that matter, which piece of software is even handling these resolutions? I've tried looking around online, but it's not clear whether recognition of #providesModule is a webpack thing, or a babel thing, or a webpack-babel-loader thing, or maybe just a node v5 thing.
I've done a lot of digging, both online and within my project's codebase. Here's what I've learned:
Almost all of the links you'll see online about #providesModule are about react or react-native projects. That's because Facebook's tooling -- specifically HasteMap (I think), part of the react-native packager suite -- explicitly adds support for #providesModule. My project is neither react nor react-native, and it doesn't use any Facebook tools in the build process. My project uses webpack v1.14 and babel v6.22 to build the source.
I've found the error message within webpack's source, at ./node_modules/webpack/lib/dependencies/WebpackMissingModule.js:
exports.moduleCode = function(request) {
return "var e = new Error(" + JSON.stringify("Cannot find module \"" + request + "\"") + "); " +
"e.code = 'MODULE_NOT_FOUND'; " +
"throw e;";
};
Webpack seems to have a term for scripts that declare a module name in this way: 'Labeled Modules'. There are several scripts within webpack's source that deal with labeled modules, but I haven't been able to pin down the code that recognizes a label within a module. My assumption is that the problem exists in the part of the system that populates the module registry, and the labeled module code I've found all seems to be related to accessing those modules within the (already populated) registry.
It's hard to trace this much further within webpack's source code, which is understandably very meta, and distributed across a bajillion files.
Any help troubleshooting this, or links to relevant documentation or framework code that shows how #providesModule is handled, is appreciated. I want to understand why my new module isn't resolving, so I can fix that problem. If my caching theory is off-base, fine.
This may not come as a surprise to most, but webpack doesn't (currently) honor #providesModule directives.
The reason it appeared as though it did is that webpack seems to flatten case when doing filename comparisons, and my filenames generally match the labels that I give them:
database.js == #providesModule Database
assemble.js == #providesModule Assemble
The reason I ran into trouble is that configuration.js != #providesModule MyConfig.
This also means the question I asked originally was off-base, in that I assumed resolution was failing because labels were being cached, when in fact labels were being ignored. Facebook's HasteMap may indeed use a cache, but it wasn't the culprit here.

Categories