How to exclude a subfolder of compiled resources from a Sonar analysis? - javascript

I am trying to integrate Sonarqube analysis into the JavaScript sources of my project. It is a project using Spring components for the back-end, and as a first step, we did the integration of Java sources, without problem at that point.
We are using Sonarqube v5.6.3
The problem I am finding comes with the sonar.exclusions property. Apparently, that property can't exclude a folder that has already been added as sources (see question and answer explaining that exact issue).
I have the following lines in my pom.xml, which are not working properly; and that's understandable according to the aforelinked question:
<sonar.sources>src/main/java,src/main/docker,js-sources</sonar.sources>
<sonar.tests>src/test</sonar.tests>
<sonar.exclusions>**/target/*</sonar.exclusions>
The problem is: the front-end is made of several modules which are compiled one by one under their own /target sub-folder before being deployed all together into src/main/webapp. (They work as regular target folders: when a new compilation is launched, those folders get deleted/recreated.)
Those js-sources/moduleA/target, js-sources/moduleB/target, js-sources/moduleC/target folders are being automatically included as sources, and thus ignored by the exclusions directive. Those target folder still contain a /src subfolder, which makes it hard to use the limited Sonar patterns (full xpath-like selectors are not allowed) to include or exclude only certain paths.
As I don't think that the Sonarqube team was expecting everyone to add each little subfolder one by one (that's why they made patterns in the first term), I am looking for help: How do I exclude those per-module target folders living down the folder-tree inside my sources?
Another possibility would be that it is kind of a bug forcing us to store this config at a Jenkinsfile or even directly in the Jenkins config (at a job level), but I remain unsure and still think that something can be fixed in the way I am declaring the sources and exclusions.

Try
<sonar.exclusions>**/target/**/*</sonar.exclusions>

EDIT : while inclusions are useful in other cases, the accepted answer above is the correct one. I'm leaving mine, which follows, for the record and just as an example of using inclusions.
Try using inclusions rather than exclusions, I've setup a project as close to yours as I could guess from your description and I was able to ignore the target folders of the js-sources modules :
<properties>
<sonar.sources>src/main/java,js-sources</sonar.sources>
<sonar.inclusions>**/*.java, **/src/**/*.js</sonar.inclusions>
</properties>
You can read this as : 'scan all java files no matter where they are, scan only the javascript files that are found within the src of a subfolder of root'

Related

Why does a Cordova project have identical copies of index.js?

I included a lot of background information to help you answer this question, however you can skip down to the heading called 'Questions' to skip to the main point.
Background
I'm new to using Cordova, and I'm new to an existing Cordova project I want to further develop. As a result, when I look at the project files, I am not sure what are choices made by the previous developers and what are choices made automatically by Cordova. I suspect that Cordova generates a lot of files that are not created by the application developers because in my case there are over 7900 files including source code and README's, and the application was previously (to my knowledge at least) developed by only one person.
While many questions could be asked from that perspective, I would like to narrow in on a specific question to avoid being too broad. I've noted that are many files within the path structure called index.js.
$ find . -name "index.js"
./platforms/android/app/build/intermediates/assets/debug/www/js/index.js
./platforms/android/app/src/main/assets/www/js/index.js
./platforms/android/cordova/node_modules/balanced-match/index.js
./platforms/android/cordova/node_modules/os-tmpdir/index.js
./platforms/android/cordova/node_modules/concat-map/index.js
./platforms/android/cordova/node_modules/properties-parser/index.js
./platforms/android/cordova/node_modules/elementtree/lib/parsers/index.js
./platforms/android/cordova/node_modules/os-homedir/index.js
./platforms/android/cordova/node_modules/xmlbuilder/lib/index.js
./platforms/android/cordova/node_modules/ansi/examples/beep/index.js
./platforms/android/cordova/node_modules/ansi/examples/progress/index.js
./platforms/android/cordova/node_modules/ansi/examples/clear/index.js
./platforms/android/cordova/node_modules/path-is-absolute/index.js
./platforms/android/cordova/node_modules/brace-expansion/index.js
./platforms/android/cordova/node_modules/sax/test/index.js
./platforms/android/cordova/node_modules/android-versions/index.js
./platforms/android/cordova/node_modules/lodash/index.js
./platforms/android/cordova/node_modules/cordova-registry-mapper/index.js
./www/js/index.js
./node_modules/objectorarray/index.js
./node_modules/es-abstract/index.js
./node_modules/es-abstract/test/index.js
./node_modules/isexe/index.js
./node_modules/balanced-match/index.js
./node_modules/object-keys/index.js
./node_modules/object-keys/test/index.js
./node_modules/has/src/index.js
./node_modules/has/test/index.js
./node_modules/semver/test/index.js
./node_modules/string.prototype.trim/index.js
./node_modules/string.prototype.trim/test/index.js
./node_modules/cordova-plugin-file/src/blackberry10/index.js
./node_modules/jsonfile/index.js
./node_modules/fs-extra/lib/ensure/index.js
./node_modules/fs-extra/lib/remove/index.js
./node_modules/fs-extra/lib/move/index.js
./node_modules/fs-extra/lib/output/index.js
./node_modules/fs-extra/lib/copy-sync/index.js
./node_modules/fs-extra/lib/fs/index.js
./node_modules/fs-extra/lib/empty/index.js
./node_modules/fs-extra/lib/index.js
./node_modules/fs-extra/lib/path-exists/index.js
./node_modules/fs-extra/lib/copy/index.js
./node_modules/fs-extra/lib/move-sync/index.js
./node_modules/fs-extra/lib/json/index.js
./node_modules/fs-extra/lib/mkdirs/index.js
./node_modules/concat-map/index.js
./node_modules/elementtree/lib/parsers/index.js
./node_modules/fs.realpath/index.js
./node_modules/plist/index.js
./node_modules/path-key/index.js
./node_modules/xmlbuilder/lib/index.js
./node_modules/ansi/examples/beep/index.js
./node_modules/ansi/examples/progress/index.js
./node_modules/ansi/examples/clear/index.js
./node_modules/cordova-android/node_modules/balanced-match/index.js
./node_modules/cordova-android/node_modules/os-tmpdir/index.js
./node_modules/cordova-android/node_modules/concat-map/index.js
./node_modules/cordova-android/node_modules/properties-parser/index.js
./node_modules/cordova-android/node_modules/elementtree/lib/parsers/index.js
./node_modules/cordova-android/node_modules/os-homedir/index.js
./node_modules/cordova-android/node_modules/xmlbuilder/lib/index.js
./node_modules/cordova-android/node_modules/ansi/examples/beep/index.js
./node_modules/cordova-android/node_modules/ansi/examples/progress/index.js
./node_modules/cordova-android/node_modules/ansi/examples/clear/index.js
./node_modules/cordova-android/node_modules/path-is-absolute/index.js
./node_modules/cordova-android/node_modules/brace-expansion/index.js
./node_modules/cordova-android/node_modules/sax/test/index.js
./node_modules/cordova-android/node_modules/android-versions/index.js
./node_modules/cordova-android/node_modules/lodash/index.js
./node_modules/cordova-android/node_modules/cordova-registry-mapper/index.js
./node_modules/cordova-android/bin/templates/project/assets/www/js/index.js
./node_modules/for-each/index.js
./node_modules/tape/index.js
./node_modules/path-is-absolute/index.js
./node_modules/brace-expansion/index.js
./node_modules/deep-equal/index.js
./node_modules/has-symbols/index.js
./node_modules/has-symbols/test/index.js
./node_modules/shebang-regex/index.js
./node_modules/is-callable/index.js
./node_modules/through/index.js
./node_modules/through/test/index.js
./node_modules/resumer/index.js
./node_modules/resolve/index.js
./node_modules/resolve/test/resolver/dot_main/index.js
./node_modules/resolve/test/resolver/multirepo/packages/package-b/index.js
./node_modules/resolve/test/resolver/multirepo/packages/package-a/index.js
./node_modules/resolve/test/resolver/quux/foo/index.js
./node_modules/resolve/test/resolver/incorrect_main/index.js
./node_modules/resolve/test/resolver/same_names/foo/index.js
./node_modules/resolve/test/resolver/dot_slash_main/index.js
./node_modules/resolve/test/shadowed_core/node_modules/util/index.js
./node_modules/resolve/test/node_path/x/ccc/index.js
./node_modules/resolve/test/node_path/x/aaa/index.js
./node_modules/resolve/test/node_path/y/ccc/index.js
./node_modules/resolve/test/node_path/y/bbb/index.js
./node_modules/resolve/test/module_dir/ymodules/aaa/index.js
./node_modules/resolve/test/module_dir/xmodules/aaa/index.js
./node_modules/resolve/test/precedence/aaa/index.js
./node_modules/resolve/test/dotdot/abc/index.js
./node_modules/resolve/test/dotdot/index.js
./node_modules/function-bind/index.js
./node_modules/function-bind/test/index.js
./node_modules/shebang-command/index.js
./node_modules/minimist/index.js
./node_modules/cordova-plugin-network-information/src/blackberry10/index.js
./node_modules/cross-spawn/index.js
./node_modules/es-to-primitive/index.js
./node_modules/es-to-primitive/test/index.js
./node_modules/strip-bom/index.js
./node_modules/is-date-object/index.js
./node_modules/path-parse/index.js
./node_modules/is-symbol/index.js
./node_modules/is-symbol/test/index.js
./node_modules/base64-js/index.js
./node_modules/node-version-compare/index.js
./node_modules/node-version-compare/test/index.js
./node_modules/universalify/index.js
./node_modules/define-properties/index.js
./node_modules/define-properties/test/index.js
./node_modules/nice-try/src/index.js
./node_modules/defined/index.js
./node_modules/object-inspect/index.js
./node_modules/is-regex/index.js
./plugins/cordova-plugin-file/src/blackberry10/index.js
./plugins/cordova-plugin-network-information/src/blackberry10/index.js
It would be extremely broad and impractical to question what each and every one of these identically-named files are for, and some may be inanswerable if their existence depends on design choices of the previous developers of this project. Fortunately, I can be more specific than that. I have an issue to debug, and with a specific search string related to the bug, I was able to find that specific files named index.js contained that string.
$ grep -ril "<search_string>" .
./platforms/android/app/build/intermediates/assets/debug/www/js/index.js
./platforms/android/app/src/main/assets/www/js/index.js
./www/js/index.js
And to clarify further, while not being the only files returned by grep, these three index.js files were the only files that appeared to be relevant to my bug. I then used cmp to compare these three files, and I found that they're mutually identical (not shown since cmp by default only prints output when there is a difference).
There seems to be some confusion about the importance of ./www/js/index.js, but it also seems to be the case that I should prefer to edit ./www/js/index.js rather than ./platforms/android/app/src/main/assets/www/js/index.js because ./platforms/android/app/src/main/assets/www/js/index.js is automatically generated by Cordova according to this post. This suggest that ./www/js/index.js is not automatically generated, although this still leaves open whether ./platforms/android/app/build/intermediates/assets/debug/www/js/index.js is automatically generated. Given that build is in the path of ./platforms/android/app/build/intermediates/assets/debug/www/js/index.js, I suspect that this is a file automatically generated when the command cordova build android is run. This leads me to the suspicion that I should edit ./www/js/index.js and not the other two of these three files.
Questions
I'm really asking 3 questions in one post, however since their scope and background are nearly identical I would prefer to keep them together for context.
Which of the three index.js should be edited in developing an application with Cordova?
./platforms/android/app/build/intermediates/assets/debug/www/js/index.js
./platforms/android/app/src/main/assets/www/js/index.js
./www/js/index.js
The three index.js files mentioned in Question 1 are identical. Why do each of these identical copies exist?
There are non-identical files named index.js in a Cordova project. Why are they all called the same name if they are non-identical in content?
You should edit /www/js/index.js.
The other two files are created during the build process. A built Cordova app will have all www folder contents inside an android app structure, that's why they are inside /platforms/android/app/src/main/
The other index.js files are there because it's a Node.js pattern

priv/static/js/app.js updates randomly when changes Vue files - Phoenix/Elixir/Vue.js

I have a vue.js/Phoenix app. I'm trying to understand how to properly configure the frontend assets. I'm having trouble understanding why my priv/static/js/app.js file keeps updating whenever I Change something in other files. I'm trying to research this behavior but I can't seem to find out any information.
app.html.eex
<body>
<%= render #view_module, #view_template, assigns %>
<script src="<%= static_path(#conn, "/js/app.js") %>"></script>
</body>
My basic question is how to structure a vue.js app? The fact that I change something in asset/src dynamically changes something in static/js/app.js seems really strange. Does anybody have resources or answers on what might be happening here or places I can go to learn more?
In addition to what Pawel said, this behaviour might be intentionally configured. There is the watcher specified in config/dev.exs:
watchers: [
node: ["node_modules/brunch/bin/brunch", "watch", "--stdin",
cd: Path.expand("../assets", __DIR__)]]
That would be used in development mode to allow so-called “hot reload”: one does not need to reload the application when some changes in assets are made, app.js will be rebuilt and reloaded automagically.
There is also assets/brunch-config.js file, where one might specify rules of how the resulting app.js is being produced. By default is just compiles everything found in assets to the single javascript file, but this behaviour might be easily changed (e.g. one might exclude anything from being built into app.js and specify their own rules to provide an access to these excluded files.)
As contrary as this might sound, this is exactly the behaviour Phoenix (with Brunch) provides.
The main idea is to implement your JS functionality in assets/js/app.js, then Brunch (http://brunch.io/) as a build tool will take the content, compile/transpile and output to priv/static/js/app.js.
This means, with default configuration that comes with Phoenix, you can use ES6 in your code in assets/js/app.js, but this will be "translated" to executable form (that's understood by browsers), and located in priv/. priv/static is exposed publicly, and this will be the content available by:
<script src="<%= static_path(#conn, "/js/app.js") %>"></script>
To wrap up.
Code in priv/static is not meant to be changed by code, it gets there automatically by changes you put under your source control in assets/.
If that's any help, you can take a look at one of old blog posts about assets in Phoenix here.
Good luck!
I have been happy using webpack with Vue as of now. It uses a similar, configurable, watcher as the one mentioned by mudasobwa. In Webpack if you touch a file that is in part of the bundle it will recompile the needed files only (which can still be many depending on the dependency graph), probably brunch recompiles all.
I also use Yarn to manage npm, and I always include vuex unless it's really something just basic (although not related to file organisation it does help a lot organising vue on any non-trivial apps). Then
/assets
js
entry point files that I use for webpack output into its own individual bundles/apps
folders to organise these, usually /components-views-related, /store-related, /shared-utilities
css
.scss files, divided so that they can be split into "global" styles and individual styles that then are required in each "entry point". Then I use a "general" scss stylesheet on "all pages" and each page the corresponding css bundle where they're needed.
Then on the templates side, I wrote a small, overly complex, brittle, system to just automate the "bundle" that gets loaded on the template (in the html document head) but you can just load each bundle/s where you need them.

Create separate JavaScript bundles with a shared common library using Browserify and Gulp

For my team at work, I'm trying to set up a semi-automated JavaScript script and dependency management system with the help of Gulp and Browserify.
I'm not even sure if what I'm trying to achieve is possible with the currently available set of tools (and my limited JavaScript knowledge). I believe what I'm trying to achieve is a pretty common scenario, but I haven't been able to find the information I've been looking for.
Consider the following diagram:
The lines indicate depedencies. For shared modules, such as Module-v and Module-y, I don't want the scripts to be duplicated by being included in each of their respective bundles.
I know that using Browserify I can manually ignore or exclude modules, which is fine when the project is young - but as the project grows managing which dependencies need to be included where is going to become very cumbersome.
A similar Q&A here I think has the same essence of what I'm trying to ask, but to me, it isn't quite clear. It also references gulp-browserify which has since been blacklisted.
In my diagram, I can see that I have three Browserify entry points, but my lack of Gulp/Node/Browserify experience means I'm struggling to wrap my head around how I can try to achieve what I want to.
I'm happy to do the work to try and piece it together, as I already have been trying - however project managers are breathing down my neck so I'm having to hack together a temporary "solution" until I can implement something a little more automated and robust.
Thanks in advance.
Edit
It seems from Browserify's plugin documentation that this might be able to be achieved by using factor-bundle which substack pointed out to me; however again due to my lack of Node/Browserify/Gulp experience I am struggling to pull all the pieces together.
Related Questions
How can I use factor-bundle with browserify programmatically?
Figured it out, sharing the learns:
Code example:
var gulp = require('gulp'),
source = require('vinyl-source-stream'),
browserify = require('browserify'),
factor = require('factor-bundle');
gulp.task('browserify', function(){
return browserify({
entries: ['blog.js', 'page.js']
})
.plugin(factor, {
// File output order must match entry order
o: ['bundle/blog.js', 'bundle/page.js']
})
.bundle({
debug: true
})
.pipe(source('common.js'))
.pipe(gulp.dest('bundle/'));
});
The key difference between this output and the diagram, is that the common.js file is automatically generated based on common depenedencies between blog.js and page.js. This is described in the factor-bundle documentation.
Notes:
I found that Node would throw an error if the output files didn't already exist. I'm unsure why as I would have assumed that factor-bundle would simply write a stream to the outputting file regardless of whether it was there or not. As a workaround, after 'cleaning' the output directory, I simply created 'placeholder' files to keep it happy.
I haven't figured out how to access the factor-bundle stream event when using it as a browserify plugin (it may not even be possible), so any further work on the output files (such as uglifying etc) would likely need to be done somewhere else in the pipeline, possibly with another browserify plugin, or even after the fact.

Best way to import JavaScript files into one file?

I have a background in coding in languages that have a concept of "classes". Now that I am coding JavaScript, I would like to code in a similar way so that each object oriented "class" I create is its own separate file.
see Accessing "Public" methods from "Private" methods in javascript class
see http://phrogz.net/JS/classes/OOPinJS.html
In other languages, I would create import statements at the top of the class file to ensure other custom classes that were used within a class file so that the other custom classes were compiled into the final binary.
Of course JavaScript is not a compiled language; however, I would still like to be able to be include some kind of "import" statement at the top of custom class files so I could ensure the imported JS "class" file was available for the user's browser to download.
It would be ideal if there were a 3rd party tool that combined all of my separate class files into one JS file so the browser only had to make one HTTP request for a single JS file instead of many calls for each indicidual JS "class". Does anyone know if such a tool exists where it would do the following:
allowed me to choose which JS files that I wanted to include in a single JS file
crawled thru the files I selected in step 1 and found all the "import" statements at the top of each custom "class" file. These "import" statements could simply be specially formatted comments in the code that the 3rd party recognizes as import statements.
The 3rd party would then create the single JS file with all of the files that were selected from step 1 and from all of the imported files that were found in step 2.
Some popular JavaScript frameworks seem to do just that. For example, jQueryUI allows you to customize the download of a single jQueryUI source file by allowing the user to check off which objects you want to use. If you uncheck an element that is needed for an item that you checked off, then the form tells you that there is a dependency you need to rectify before being able to proceed to download the file.
see http://jqueryui.com/download/
So is there a 3rd party tool that allows a developer to use some kind of "import" statement comment to ensure that many dependent JS files (and only the ones that the developer needs) to be combined into a single JS file?
RequireJS was built for exactly this purpose.
Have a look at Require.js. It lets you import various javascript files in a modularized fashion and add the required dependencies between them. Also at the end you can minify them all into one single JS file using r.js
A trivial batch file can do this for you:
#for %i in (classes/*.js) type %i >> build.js
This works best if your JS source files are all in one folder, and this example assumes that folder is named classes. It gets a bit more complicated if you have subfolders, but a similar principle can be applied.
Have a look at GruntJS, JQuery uses it for building. If you don't care for HTTP requests, you can use already mentioned RequireJS, which also has nice async methods to load files, which can improve perfomance in some situations.
Check out this class https://www.youtube.com/watch?v=KnQfGXrRoPM
This allows for importing on the fly within classes. also it allows
for importing all classes within an folder and all of its sub folders.
and its really simple because it is just a prototype function added to String.
just by adding the importer class you will call in classes like "com.project.Classfile.js".import();
or "com.project.*".import() to get all sub-classes.
fork on - https://github.com/jleelove/Utils

Issue with concatenating a few javascript files

This is a complete noob question, but I gotta ask it anyway
I started playing with backbone.js a few days ago and I was really fascinated. As I got over the "ToDo", I started working on a project of my own. Coming from the world of Java, I prefer keeping everything in as many separate files as possible. Therefore, I split my models views, and routers into separate files, into separate folders.
The problem came when I tried to combine those fiels into one single applciation.js file. Again, coming from the Java world, I love when I can automate stuff, and even more, when I can use familiar tools like ant, to setup build processes for my javascript projects.
I got a sample ant build template which concatenates and minifies all the files in an arbitrary order. When it finished, I tried to run my JS app, and not surprisingly, it failed with a bunch of errors. Many of my models and views try to extend each other, others depende on them as components. If they are not defined in a proper order, the app just reaches a point where it is trying to execute extend of an undefined
I know from before that for JavaScript the order is very important, but somehow I was left with the impression that if all the scripts are in one single file, the JS parser will load all the stuff first and then will try to execute whatever is to be executed. Well, my assumption was wrong.
It is possible to list all the files in the specific order I want them, but do I really need to go for such a primitive step? Unfortunately after spending a few hours researching, I couldn't find anything better.
Is it really possible to concatenate JS files, which depend on each other, in an arbitrary order, without them clashing? I guess, the biggest problem is the fact that the extend function is actually being called, rather than each script simply defining and object literal
So, what's the solution?
UPDATE: I just saw that Sproutcore has its own builder. If SC is roughly similar to BB, in the way one creates and extends entities, how does the SC builder work without clashing?
There are many ways to do this, but here's my recipe. I prefix my development files with a number, starting from the one with no dependencies (base "classes", models that will be depended upon from other models, then views using these models, then routers calling those views, etc.).
Then I use uglify-js (available as a node.js library, that you install using npm install uglify-js) to minify all my js in one file (don't know from your question if you use node.js server-side, though). Then I cat *.js | uglifyjs -o min/myfile.min.js. This will send the content of all my .js files (respecting the order of dependencies because of my prefix) to uglify, which will minify it and save it to a single file.
Since I, too, like automation, I have this set up in a Makefile, though I guess it could be done using Ant (not too familiar with it). The relevant part of the Makefile look like this:
TARGET_MIN_FILE = public/js/min/myfile.min.js
JS = $(shell echo public/js/*.js)
public/js/min/myfile.min.js: $(JS)
cat $(JS) | uglifyjs -o $(TARGET_MIN_FILE)
clean:
rm -f $(TARGET_MIN_FILE)
.PHONY: clean
On the other hand, if you go for the asynchronous module definition (AMD) format, you can require() your modules and it will manage for you the dependency loading in the correct order (see Require.js for more info), as mentioned by TheShelfishMeme.
Your "assumption" is only true for var statements and functions of the form function name(a,b) {}. Those two get hoisted to the top of the script (or function block they are in) and are evaluated first.
If your files depend on other files being loaded first, it stands to reason that when you concatenate them they must be in that order in the final file.
Have a look at requirejs. It takes some time to set up but it should help you with your problem.
This article should help with the implementation.

Categories