We Keep Coding

how to specify local source path for example like angular in node_modules in total.js

So, I would like to have a simple chart in web page (with websockets), but it must be available offline.
First thing I did, I've downloaded empty project, then npm install total.js, then npm install n3-charts. Now I need to specify path to local file and here I bumped into two questions:
1) Can I have "index.html" with headers of its own, or all must be put inside "layout.html" (will they combine)? How would index.htm then look like?
2) Wherever I put, path seems to not to work because (I guess) path is wrong. I've tried <script src="node_modules/angular/angular.min.js"> and to place that to layout.htm, but no success.
UPDATE: there is a folder "public/js" where you can put js files, but that would imply that node_modules folder is just not to be used and that all of the files you need are to be copied to "public/js". Is this the way it was intended?

sorry for delay.
you can use section, documentation
of course, you have to create another node_modules directory in public folder: /app/public/node_modules/angular/angular.min.js and then it will work.
YOUR UPDATE: you can rename it as you wish

Grunt: have package.json and Gruntfile.js on different folders

Im having problems trying to implement grunt on diferent folders, in my root i have:
<root>/package.json
<root>/node_modules
And inside another folder, my gruntfile with diferent subfolders and files wich i work:
<root>/apps/static/Gruntfile.js
If i go to root and execute
grunt --gruntfile /apps/static/Gruntfile.js MyTaskName
I get:
Local Npm module "grunt-contrib-concat" not found. Is it installed?
Local Npm module "grunt-contrib-cssmin" not found. Is it installed?
Local Npm module "grunt-contrib-clean" not found. Is it installed?
Local Npm module "grunt-contrib-watch" not found. Is it installed?
Local Npm module "grunt-contrib-uglify" not found. Is it installed?
And i run several times npm install.
On my gruntfile.js y have
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-uglify');
I triple check and folders are ok (in fact, originally gruntfile and package where in the same folder and everything was working perfect, run several task and everything is ok). I really need to have a common package.json and node_modules on root and the Gruntfile.js on a specific project folder
Any idea whats going on? thanks in advance

Grunt makes certain assumptions regarding the location of gruntfile.js.
When you specify the location of gruntfile.js using the --gruntfile option, Grunt sets the current directory to the directory containing the specified file:
// Change working directory so that all paths are relative to the
// Gruntfile's location (or the --base option, if specified).
process.chdir(grunt.option('base') || path.dirname(gruntfile));
And when Grunt loads NPM tasks, it does so relative to the current directory:
var root = path.resolve('node_modules');
var pkgfile = path.join(root, name, 'package.json');
There is a --base option with which the current directory can be specifed, but whether or not that will solve your problem (without introducing other problems) I do not know. The simplest solution is likely to locate gruntfile.js where it wants and expects to be located.

Sometimes, it may be the need of project to have Gruntfile.js in a different folder than package.json.
I had a very similar use-case where there were multiple submodules each with its own build process, one of them was Grunt. But at the same time I wanted to have a common package.json just to avoid multiple node_modules folders being created, so that common dependencies (including transitive) use to install once. It helped in reducing install time as well as disk usage.
I was expecting a solution in Grunt itself. But as #cartant mentioned, Grunt has made certain assumptions.
So, here is what I did:
In Gruntfile.js,
Define a function:
function loadExternalNpmTasks(grunt, name) {
const tasksdir = path.join(root, 'node_modules', name, 'tasks');
if (grunt.file.exists(tasksdir)) {
grunt.loadTasks(tasksdir);
} else {
grunt.log.error('Npm module "' + name + '" not found. Is it installed?');
}
}
And instead of
grunt.loadNpmTasks('grunt-contrib-concat');
do:
loadExternalNpmTasks(grunt, 'grunt-contrib-concat');
Reference: https://github.com/gruntjs/grunt/blob/master/lib/grunt/task.js#L396

Assemble every module into a single .js file

I want to minimize the number of HTTP requests from the client to load scripts in the browser. This is going to be a pretty general question but I still hope I can get some answers because module management in javascript has been a pain so far.
Current situation
Right now, in development, each module is requested individually from the main html template, like this:
<script src="/libraries/jquery.js"></script>
<script src="/controllers/controllername.js"></script>
...
The server runs on Node.js and sends the scripts as they are requested.
Obviously this is the least optimal way of doing so, since all the models, collections, etc. are also separated into their own files which translates into numerous different requests.
As far as research goes
The libraries I have come across (RequireJS using AMD and CommonJS) can request modules from within the main .js file sent to the client, but require a lot of additional work to make each module compliant with each library:
;(function(factory){
if (typeof define === 'function' && define.amd) define([], factory);
else factory();
}(function(){
// Module code
exports = moduleName;
}));
My goal
I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect. Then I can simply serve that single file to the client when it is requested.
Is this possible?
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?

What you are asking to do, from what I can tell, is concat your js files into one file and then in your main.html you would have this
<script src="/pathLocation/allMyJSFiles.js"></script>
If my assumption is correct, then the answer would be to use one of the two following items
GULP link or GRUNT link
I use GULP.
You can either use gulp on a case by case basis, which means calling gulp from the command line to execute gulp code, or use a watch to do it automatically on save.
Besides getting gulp to work and including the gulp files you need to do what you need, I will only provide a little of what I use to get your answer.
In my gulp file I would have something like this
var gulp = require('gulp');
var concat = require('gulp-concat');
...maybe more.
Then I have the file paths I need to be reduced into one file.
var onlyProductionJS = [
'public/application.js',
'public/directives/**/*.js',
'public/controllers/**/*.js',
'public/factories/**/*.js',
'public/filters/**/*.js',
'public/services/**/*.js',
'public/routes.js'
];
and I use this info in a gulp task like the one below
gulp.task('makeOneFileToRuleThemAll', function(){
return gulp.src(onlyProductionJS)
.pipe(concat('weHaveTheRing.js'))
.pipe(gulp.dest('public/'));
});
I then run the task in my command line by calling
gulp makeOneFileToRuleThemAll
This call runs the associated gulp task which uses 'gulp-concat' to get all the files together into one new file called 'weHaveTheRing.js' and creates that file in the destination 'public/'
Then just include that new file into your main.html
<script src="/pathLocation/weHaveTheRing.js"></script>
As for including all your files into one file, including your vendor files, just make sure that your vendor code runs first. It's probably best to keep those separate unless you have a sure fire way of getting your vendor code to load first without any issues.
UPDATE
Here is my gulp watch task.
gulp.task('startTheWatchingEye', function () {
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll']);
});
Then I start up my server like this (yours may differ)
npm start
// in a different terminal window I then type
gulp startTheWatchfuleye
NOTE: you can use ANY movie or show reference you wish! :)
Now just code it up, every time you make a change in the specified files GULP will run your task(s).
If you want to say run Karma for your test runner...
add the following to your gulp file
var karma = require('karma').server;
gulp.task('karma', function(done){
karma.start({
configFile: __dirname + '/karma.conf.js'
}, done);
});
Then add this task karma to your watch I stated above like this...
gulp.task('startTheWatchingEye', function(){
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll', 'karma']);
});
ALSO
Your specific settings may require a few more gulp modules. Usually, you install Gulp globally, as well as each module. Then use them in your various projects. Just make sure that your project's package.json has the gulp modules you need in dev or whatever.
There are different articles on whether to use Gulp or Grunt. Gulp was made after Grunt with a few additions that Grunt was lacking. I don't know if Grunt lacks them anymore. I like Gulp a lot though and find it very useful with a lot of documentation.
Good luck!

I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect.
Sure you can. You can use Grunt or Gulp to do that, more specifically grunt-contrib-concat or gulp-concat
Here's an example of a Gruntfile.js configuration to concat every file under a js directory:
grunt.initConfig({
concat: {
dist: {
files: {
'dist/built.js': ['js/**/**.js'],
},
},
},
});
Also, you can minify everything after concatenating, using grunt-contrib-minify.
Both libraries support source maps so, in the case a bug gets to production, you can easily debug.
You can also minify your HTML files using grunt-contrib-htmlmin.
There's also an extremely useful library called grunt-usemin. Usemin let's you use HTML comments to "control" which files get minified (so you don't have to manually add them).
The drawback is that you have to explicitely include them in your HTML via script tags, so no async loading via javascript (with RequireJS for instance).
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?
That's debatable. Both have pros and cons. Concatenating vendors assures that, if for some reason, the CDN isn't available, your page works as intended. However the file served is bigger so you consume more bandwidth.
In my personal experience, I tend to include vendor libraries that are absolutely essential for the page to run such as AngularJS for instance.

If I understand you correctly, you could use a task runner such as Grunt to concatenate the files for you.
Have a look at the Grunt Concat plugin.
Example configuration from the docs:
// Project configuration.
grunt.initConfig({
concat: {
dist: {
src: ['src/intro.js', 'src/project.js', 'src/outro.js'],
dest: 'dist/built.js',
}
}
});
Otherwise, as you have stated, a 'module loader' system such as Require JS or Browserify may be the way to go.

Gulp - Target all files in a folder and its subfolders

I'd like to be able to add a watch task in gulp to all of the js files in the frontend/js and any other js files below
gulp.watch('./frontend/js/**/*.js', ['browserify']);
This will only target js files one folder deep

It's supposed to match any number of subdirectories:
** If a "globstar" is alone in a path portion, then it matches zero or more directories and subdirectories searching for matches. It does not crawl symlinked directories.
https://github.com/isaacs/node-glob
Do you have symlinked directories in there?
Symlinks
I don't think you'll get gulp to natively traverse your symlinked directories. I recommend you take a look at node.js fs.readdir recursive directory search and see if any of those solutions can be applied to your use case. Nothing in the question or answers specifically addresses symlinks, so I don't know if there's a solution for you there or not. If you can get an array of dereferenced pathnames using one of those solutions, then you can just pass the array to gulp.src().

I just did some testing - and this actually works just fine for me.
I currently have the following structure -
--apps
--scripts
----test.js
----test-folder
------test2.js
------test-folder-deep
--------test3.js
--myApp
----scripts-symlinked (symlinked to apps/scripts)
----gulpfile.js
I set up my symlink folder (on Mac - from 'myApp' folder) using:
ln -s /Users/kandrews/apps/scripts ./scripts-symlinked
In my gulpfile.js I have the following:
var gulp = require('gulp'),
jshint = require('gulp-jshint');
gulp.task('jshint', function () {
gulp.src('./scripts-symlinked/**/*.js')
.pipe(jshint())
.pipe(jshint.reporter('default'));
});
gulp.task('watch', function () {
gulp.watch('./scripts-symlinked/**/*.js', ['jshint']);
});
Works perfectly. I also tried this in a sub directory as well ('scripts/symlinked-scripts') and was successful as well.

I think it’s not worth doing difficult:
gulp.watch('./frontend/js/', ['browserify']);

Working project structure that uses grunt.js to combine JavaScript files using RequireJS?

I have some projects that use RequireJS to load individual JavaScript modules in the browser, but I haven't optimized them yet. In both development and production, the app makes a separate request for each JavaScript file, and now I would like to fix that using Grunt.
I have tried to put together a simple project structure to no avail, so I'm wondering if someone can provide a working example for me. My goals are the following:
In development mode, everything works in the browser by issuing a separate request for each required module. No grunt tasks or concatenation are required in development mode.
When I'm ready, I can run a grunt task to optimize (combine) all of the JavaScript files using r.js and test that out locally. Once I'm convinced the optimized application runs correctly, I can deploy it.
Here's a sample structure for the sake of this conversation:
grunt-requirejs-example/
grunt.js
main.js (application entry point)
index.html (references main.js)
lib/ (stuff that main.js depends on)
a.js
b.js
requirejs/
require.js
text.js
build/ (optimized app goes here)
node_modules/ (necessary grunt tasks live here)
Specifically, I'm looking for a working project structure that I can start from. My main questions are:
If this project structure is flawed, what do you recommend?
What exactly needs to be in my grunt.js file, especially to get the r.js optimizer working?
If all of this isn't worth the work and there's a way to use the grunt watch task to automatically build everything in development mode every time I save a file, then I'm all ears. I want to avoid anything that slows down the loop from making a change to seeing it in the browser.

I use the grunt-contrib-requirejs task to build project based on require.js. Install it inside your project directory with:
npm install grunt-contrib-requirejs --save-dev
BTW: --save-dev will add the package to your development dependencies in your package.json. If you're not using a package.json in your project, ignore it.
Load the task in your grunt file with:
grunt.loadNpmTasks('grunt-contrib-requirejs');
And add the configuration to your grunt.initConfig
requirejs: {
production: {
options: {
baseUrl: "path/to/base",
mainConfigFile: "path/to/config.js",
out: "path/to/optimized.js"
}
}
}
Now you're able to build your require.js stuff into a single file that will be minimized with uglifyjs by running grunt requirejs
You can bundle a set of different tasks into some sort of main task, by adding this to your grunt file
grunt.registerTask('default', ['lint', 'requirejs']);
With this, you can simply type grunt and grunt will automatically run the default task with the two 'subtasks': lint and requirejs.
If you need a special production task: define it like the above
grunt.registerTask('production', ['lint', 'requirejs', 'less', 'copy']);
and run it with
grunt production
If you need different behaviors for 'production' and 'development' inside i.e. the requirejs task, you can use so called targets. In the configuration example above it's already defined as production. You can add another target if you need (BTW, you can define a global config for all targets by adding a options object on the same level)
requirejs: {
// global config
options: {
baseUrl: "path/to/base",
mainConfigFile: "path/to/config.js"
},
production: {
// overwrites the default config above
options: {
out: "path/to/production.js"
}
},
development: {
// overwrites the default config above
options: {
out: "path/to/development.js",
optimize: none // no minification
}
}
}
Now you can run them both at the same time with grunt requirejs or individually with grunt requirejs:production, or you define them in the different tasks with:
grunt.registerTask('production', ['lint', 'requirejs:production']);
grunt.registerTask('development', ['lint', 'requirejs:development']);
Now to answer your questions:
I would definitely use a subfolder in your project. In my case I use a 'src' folder for development that is build into a 'htdocs' folder for production. The project layout I prefere is:
project/
src/
js/
libs/
jquery.js
...
appname/
a.js
b.js
...
main.js // require.js starter
index.html
...
build/
... //some tmp folder for the build process
htdocs/
... // production build
node_modules/
...
.gitignore
grunt.js
package.json
see above
You can do so, but I wouldn't recommend to add requirejs to the watch task, it's a resource hungry task and it will slow down your machine noticeable.
Last but not least: Be very cautious when playing around with r.js. Especially when you want to optimize the whole project with r.js by adding a modules directive to your config. R.js will delete the output directory without asking. If it happens that it is accidentally configured to be your system root, r.js will erase your HDD. Be warned, I erased my whole htdocs folder permanently some time ago while setting up my grunt task... Always add keepBuildDir:true to your options when playing around with the r.js config.