We Keep Coding

Javascript bundling and module loading

I've recently been thrown in to clean up a project which has like 45-50 individual .js javascript files. I wonder what the best approach would be to decrease the loading size of them all. Just concatenate all files into one with npm or gulp? Install some module loader? webpack?

If you're already concatenating, minifying, and uglifying and you don't want all the files to be loaded on all the pages due to a monolithic bundle, you might be looking for something like Webpack's Commons Chunk Plugin.
This plugin walks down the tree of dependencies for each endpoint defined in your Webpack.config file and determines which modules are required across all pages. It then breaks the code into two bundles, a "common" bundle containing the modules that every page requires, which you must load with a script tag on each page:
<script src="commons.js" charset="utf-8"></script>
And an endpoint bundle for each individual page that you reference normally in a script tag placed after the commons script tag:
<script src="specificpage.bundle.js" charset="utf-8"></script>
The result is that an individual page will not have to load modules that will only ever be used on other pages.
Again, this is a Webpack plugin. I don't know if this functionality is available as a Gulp plugin, because it must have knowledge of all endpoints in order to determine which dependencies are common to them all.

I redirect you to the very good https://github.com/thedaviddias/Front-End-Checklist
In particular the following advises:
JavaScript Inline: High You don't have any JavaScript code inline
(mixed with your HTML code).
Concatenation: High JavaScript files
are concatenated.
Minification: High JavaScript files are minified (you can add the .min suffix).
You can accomplish this with a package manager such as gulp, grunt or webpack (for the most famous ones). You just need to choose what you prefer to use.
If you consider webpack, You can start with my very simple (but understanding) starter: https://github.com/dfa1234/snippets-starter

There's no much thing that you can do, basically is:
Concatenation - https://www.npmjs.com/package/gulp-concat
Minification - https://www.npmjs.com/package/gulp-minify
Instead of creating all those scripts, you can get something to re-use on yeoman, f.e. the Fountain, so it will reduce a lot of time just typing procedural code for doing the concatenation/minification.
Also if you can use some lazy load (like RequireJS or some frameworks have support to lazy load the module, like Angular) that will improve the performance of your aplication
EDIT:
If you want even more performance, you can install some compression tool in your server, for example this one for NodeJS https://www.npmjs.com/package/compression

I'm my personal opinion, if you have time, the best approach would be to read and understand the purpose of the project. Then plan a proper refactor. You are not fixing anything with concatenating, this is just a deployment step.
You should analyze which technologies are being used and if you want to maintain this code, in the long run, make a proper refactor into a much more modern stack, maybe you can take a seed project with ES6, webpack, Babel... and create a proper repository well maintained with proper modularity and dependencies resolution.
Once you have that, decreasing the load its just about adding proper tools in build time (babel, webpack, etc).
You would like to add some unit tests and continue working properly :)

Why bundle a React server.js for Node?

In the React Starter Kit they bundle with Webpack the server-side code into a server.js which I assumed can be handy but then I started wondering why bundling up the server-side code in the first place, after all Node comes with its own dependency management and doesn't need that. Second, wouldn't that even make it more inefficient due to parsing one whole file?

React code usually got written in ES6 however the Node doesn't support the es6 code yet, it can only understand es5. so if you want to use it without bundling and all the code will get complied first on the go. which will affect the performance eventually so you can use it for the development purpose but in production it is advised to use the bundled code.
So the major reason is Performance

Is there still a need for a module loader with applications that use JavaScript ES6 modules and classes?

If you code an application using ES6 modules and classes, is there any need to use a module loader framework, or is the best practice to just use a build tool to concatenate all the code into a file (or files) and include those using a normal script tag?

Yes. Somebody, somewhere along the line has to load the module.
I think you're conflating compiling modules ahead of time vs loading them individually. Webpack is a module loader that outputs a single file for the browsers to use later, while the System API and requirejs et al load a number of individual files.
There are performance factors on both sides, particularly longer build time (when precompiling) vs longer load time (with multiple files).
Webpack, Browserify, and most other module loaders (with the notable exception of the System API) allow you to define some loaders for certain file types and automagically compile your (S)CSS or templates on the way through, as well as running other tools to uglify or obfuscate your code. The ES6 System API does not provide these features, but is a more robust runtime loader than most.
This boils down to two trade-offs:
support for non-JS modules (styles, templates) vs build time
single request and longer build vs many requests and short/no build
Evaluate them for your users (high-bandwidth vs mobile), environment (if you have two dozen CI agents, who cares if the build takes an extra 3s?), and stack (if you have a lot of template files, compiling them AOT could be important).

how to create an environment agnostic javascript library

I'm creating a javascript library, and i want it to be environment agnostic (It will not use DOM, AJAX, or NodeJS api. It will be vanilla javascript). So, it's supposed to run in any javascript environment (browsers, npm, meteor smart packages, V8 C bindings...).
My currently approach is creating git repo with the library, with all the library inside a single global variable, without thinking about patterns like CommonJS or AMD.
Later, i'll create another git repo, using my library as a git submodule, and create what is needed to release it as a npm module. I'm concerned if it's a good approach, i didn't found anyone doing this way.
Pros: code will be vanilla javascript, without awareness of environment patterns. It will not bind itself to CommonJS. It will be repackable (copy and paste or git submodule) to any javascript environment. It will be as small as needed to be sent to browsers.
Cons: I'll have to maintain as many git as environments i want to support. At least a second git repo to deliver on npm.
Taking jQuery as example, it runs in both browser and nodejs, with just one git repo. There is some code to be aware of the "exports" variable to run on nodejs or other CommonJS compatible enviroment.
Pros: Just one git repo to mantain.
Cons: It will be binded to CommonJS pattern (to achieve npm compatibility)
My question is: Am i following a correct (or acceptable) approach? Or should i follow jquery's path, and try to create a single git repo?
Update 1:
Browserify and other require() libraries are not valid answers. My question is not how to use require() on the browser, instead, it's about the architecture pattern to achieve enviroment agnosticism.
Update 2:
Create a browser/nodejs module is not the question, it's known. The question is: can make a real enviroment agnostic library? This example is binded to CommonJS pattern, used in NodeJS.

If you are looking for design recommendation for your future library work then in my opinion you can think-future and just use usual Object Oriented Practices well proven in other languages, systems and libraries.
Mainly concentrate on the UML view of your design.
Forget the "one variable" requirement.
Use features proposed in the planned next version of JavaScript.
http://wiki.ecmascript.org/doku.php?id=strawman:maximally_minimal_classes
http://wiki.ecmascript.org/doku.php?id=harmony:modules_rationale
There is an experimental compiler available that allows you to write ES6-style code even today (see https://www.npmjs.org/package/es6-module-transpiler-rewrite).
Node.js has a --harmony command line switch that allows for the same (see What does `node --harmony` do?)
So in my opinion correct approach is to follow best practices and "think future"

"Use a build tool" is the answer for this question. With a build tool, you can develop with the best code pratices, without accopling your code to some enviroment standard of today (AMD, commonjs...) and still publish your code to these kind of enviroments.
For example, I'm using Grunt.js to run some tasks, like build, lint, test, etc.
It perform tedious operations (minification, compilation...) like Make, Maven, Gulp.js, and various others.
The build task can handle standards (like commonjs) for the compiled code. So, the library can be totally enviroment agnostic, and the build process handle enviroment adaptations.
Note that i'm not talking about compiling to binaries. It's compiling source to another source, like CoffeScript to JavaScript. In my case, it's compilation of JavaScript without enviroment standard to JavaScript with commonjs standard (to run as a Node.js module).
The final result is that i can compile my project to various standards without messing with my code.
Aditionally, with a build phase i can "think-future", like xmojmr answered and use the EcmaScript 6 features on my JavaScript code, using Grunt plugins like grunt-es6-transpiler or grunt-traceur to compile JavaScript code from ES 6 to 5 (so it can run on enviroments of today)

According to modular your library (if you need modules). Read this question Relation between CommonJS, AMD and RequireJS?
Take bootstrap for example, it uses npm to manage project dependencies and use bower to publish as static content for other web apps.
Take a look at browserify as reference, it's a little heavy because it provides the capability to bundle dependent npm modules as resource for browsers.

How can I use npm for front-end dependencies?

I want to ask if it is possible (and generally a good idea) to use npm to handle front-end dependencies (Backbone, jQuery).
I have found that Backbone, jQuery and so on are all available through npm but I would have to set another extraction point (the default is node_modules) or symlink or something else...
Has somebody done this before?
Is it possible?
What do I have to change in package.json?

+1 for using Browserify. We use it here at diy.org and love it. The best introduction and reasoning behind Browserify can be found in the Browserify Handbook. Topics like CommonJS & AMD solutions, build pipelines and testing are covered there.
The main reason Browserify works so well is it transparently works w/ NPM. As long as a module can be required it can be Browserified (though not all modules are made to work in the browser).
Basics:
npm install jquery-browserify
main.js
var $ = require('jquery-browserify');
$("img[attr$='png']").hide();
Then run:
browserify main.js > bundle.js
Then include bundle.js in your HTML doc and the code in main.js will execute.

Short answer: sort of.
It is largely up to the module author to support this, but it isn't common. Socket.io is an example of such a supporting module, as demonstrated on their landing page. There are other solutions however. These are the two I actually know anything about:
http://ender.no.de/ - Ender JS, self-described NPM analogue for client modules. A bit too involved for my tastes.
https://github.com/substack/node-browserify - Browserify, a utility that will walk your dependencies and allow you to output a single script by emulating the node.js module pattern. You can use a jake|cake|rake|make build script to spit out your application.js, and even automate it if you want to get fancy. I used this briefly, but decided it was a bit clunky, and became annoying to debug. Also, not all dual-environment npm modules like to be run through browserify.
Personally, I am currently opting for using RequireJS ( http://requirejs.org/ ) and manually managing my modules, similar to how Mozilla does with their BrowserQuest sample application ( https://github.com/mozilla/BrowserQuest ). Note that this comes with the challenge of having to potentially shim modules like backbone or underscore which removed support for AMD style module loaders. You can find an example of what is involved in shimming here: http://tbranyen.com/post/amdrequirejs-shim-plugin-for-loading-incompatible-javascript
Really it seems like it is going to hurt no matter what, which is why native module support is such a hot topic.

Our team maintains a tool called Lineman for building front-end projects. The tool is node-based, so a project relies on a lot of npm modules that operate server-side to build your assets, but out-of-the-box it expects to find your client-side dependencies in copied and committed to vendor/js.
However, a bunch of folks (myself included) have tried integrating with browserify, and we've run into a lot of complexity and problems, ranging from (a) npm modules being maintained by a third party which are either out of date or add unwanted changes, to (b) actual libraries that start failing when loaded traditionally whenever a top-level function named require is even defined, due to AMD/Require.js baggage.
My short-term recommendation is to hold off and stick with good ol' fashioned script concatenation until the dust settles. Until you have problems big enough or complex enough to warrant it, I suspect you'll spend more time debugging and remediating your build than you otherwise would. And I think most of us agree the best use of your time is focusing on your application code, not its build tools.

You might want to take a look at http://jspm.io/ which is a browser package manager. Has nice ES6 support too.

I personally use webmake for my small projects. It is an alternative to browserify in the way it brings npm dependencies into your browser, and it's apparently lighter.
I didn't have the opportunity to compare in details browserify and webmake, but I noticed webmake doesn't work well with modules internally using global variables such as socket.io (which is full of bloat anyway IMO).
I would be cautious about RequireJS, which has been recommended above. Because it is an AMD loader, your browser will load your JS files asynchronously. It will induces more exchanges between your client and server and may degrade the UX of people browsing from mobile networks / under bad WiFi. Moreover, if you succeed to keep your JS code simple and tiny, asynchronous loading is absolutely not needed !