We Keep Coding

Javascript bundling and module loading

I've recently been thrown in to clean up a project which has like 45-50 individual .js javascript files. I wonder what the best approach would be to decrease the loading size of them all. Just concatenate all files into one with npm or gulp? Install some module loader? webpack?

If you're already concatenating, minifying, and uglifying and you don't want all the files to be loaded on all the pages due to a monolithic bundle, you might be looking for something like Webpack's Commons Chunk Plugin.
This plugin walks down the tree of dependencies for each endpoint defined in your Webpack.config file and determines which modules are required across all pages. It then breaks the code into two bundles, a "common" bundle containing the modules that every page requires, which you must load with a script tag on each page:
<script src="commons.js" charset="utf-8"></script>
And an endpoint bundle for each individual page that you reference normally in a script tag placed after the commons script tag:
<script src="specificpage.bundle.js" charset="utf-8"></script>
The result is that an individual page will not have to load modules that will only ever be used on other pages.
Again, this is a Webpack plugin. I don't know if this functionality is available as a Gulp plugin, because it must have knowledge of all endpoints in order to determine which dependencies are common to them all.

I redirect you to the very good https://github.com/thedaviddias/Front-End-Checklist
In particular the following advises:
JavaScript Inline: High You don't have any JavaScript code inline
(mixed with your HTML code).
Concatenation: High JavaScript files
are concatenated.
Minification: High JavaScript files are minified (you can add the .min suffix).
You can accomplish this with a package manager such as gulp, grunt or webpack (for the most famous ones). You just need to choose what you prefer to use.
If you consider webpack, You can start with my very simple (but understanding) starter: https://github.com/dfa1234/snippets-starter

There's no much thing that you can do, basically is:
Concatenation - https://www.npmjs.com/package/gulp-concat
Minification - https://www.npmjs.com/package/gulp-minify
Instead of creating all those scripts, you can get something to re-use on yeoman, f.e. the Fountain, so it will reduce a lot of time just typing procedural code for doing the concatenation/minification.
Also if you can use some lazy load (like RequireJS or some frameworks have support to lazy load the module, like Angular) that will improve the performance of your aplication
EDIT:
If you want even more performance, you can install some compression tool in your server, for example this one for NodeJS https://www.npmjs.com/package/compression

I'm my personal opinion, if you have time, the best approach would be to read and understand the purpose of the project. Then plan a proper refactor. You are not fixing anything with concatenating, this is just a deployment step.
You should analyze which technologies are being used and if you want to maintain this code, in the long run, make a proper refactor into a much more modern stack, maybe you can take a seed project with ES6, webpack, Babel... and create a proper repository well maintained with proper modularity and dependencies resolution.
Once you have that, decreasing the load its just about adding proper tools in build time (babel, webpack, etc).
You would like to add some unit tests and continue working properly :)

How to integrate Webpack in a multi-page PHP application

I'm working on an old Codeigniter website where the Javascript codebase is messy and poorly structured and I would like to use Webback to manage the scripts. My goal would be start using it to bundle the code I have the way it is, and gradually refactoring it to make use of modules and imports.
At the moment I'm using Gulp on development (but mostly to minify the files) and Carabiner (a Codeigniter library) to insert the scripts in the views.
The scripts, which are all written as IIFEs are not bundled, so in every controller function I have an array of the scripts needed in that page. For example:
public function homepage()
{
$this->carabiner->js([
['libraryThatIOnlyNeedHere.min.js'],
['myscript1.js'],
['myscript2.js'],
['myscript3.js'],
]);
I would like to use Webpack to create a series of bundles so that I end up loading maximum two files on every page: one for the libraries and one for my scripts.
All the practical examples I've seen with Webpack, though, are for Single Page Applications where it's quite easy to bundle everything together.
What would be the best approach in my case? Considering that the code is still not ready to properly use modules and imports, shall I create many entry points in the Webpack configuration file, possibly one for every page and list every script needed in that page?

Main idea behind webpack is to build modules graph from provided sources and then to combine it into bundles. If your code doesn't have explicit dependencies - it may be good idea to start with creating a module from every file using any of available approaches (AMD, CommonJS, etc). You will need to create module identifiers and define dependencies for every module. It may be worth to read this article for example.
As intermediate step you may want to use some loader like Require.JS to load your modularized code.
Until this step will be done - there is not much use of Webpack.

gulp-sourcemaps Keep Intermediate Files

When I use the gulp-sourcemaps plugin, I typically have both the comment and content removed so the sourcemap header is used instead. This makes it much easier to show the original files in the browser debugger in development and staging, then remove it in production. We also try to load sourcemaps wherever possible, mostly for vendor libraries.
This works fine for the most part. However, intermediate files seem to be lost. I am guessing the loadMaps option to sourcemaps.init(), which allows for loading existing maps, removes the intermediate files when it gets their maps. This is likely fine for most cases, as the original files are there, but too often libraries only include the output file with maps added as a comment, but don't include those original files. This means we don't have access to the original files, and the browser does not have access to the built library file.
For example, angular-ui-router is written with TypeScript, then output to JavaScript when built. This means the end developer does not have to compile with TypeScript in his/her build process, as this is already done in the ui-router build process. Now, the output JavaScript file is accessible at <server root>/node_modules/#uirouter/angularjs/release/angular-ui-router.js, and the original TypeScript files were, according to the built file's sourcemap comment, in the <server root>/node_modules/#uirouter/angularjs/node_modules/#uirouter/core/src and <server root>node_modules/#uirouter/angularjs/src folders.
When I include the angular-ui-router file into a vendor.js or vendor.min.js file using my favorite tool, it bundles the angular-ui-router maps into the resulting .map file, but does not include the <server root>/node_modules/#uirouter/angularjs/release/angular-ui-router.js file as a source. This means we can't see either that file, or the TypeScript files, which makes debugging pretty much impossible using only the browser. To make matters worse, any errors we get are sourcemapped all the way back to the TypeScript files. We don't have the TypeScript files, so we can't look there, and now we don't know where in the JavaScript the error is, either! Thankfully, angular-ui-router doesn't have many complex errors, and the ones it does have are nicely documented, but other libraries are not so helpful.
Is there a way to include intermediate files as sources in the output sourcemap? Otherwise, is there an easy way to load some sourcemaps with something like loadMaps, but not others?

Compiling Modular Client-side Javascript

In Node.js, you can dynamically "require()" any javascript file likewise to PHP's require. I'd like to use this in my client-side code just for ease of development but not actually call a javascript function, but have a compiler replace the line with the contents of the respective file; effectively concatenating the files, not one after another, but inline within the code of one of the files. The closest thing I have found to this is smash. Are there any compilers, minifiers, etc that can do this?

Browserify might not be exactly what you want but it does definitely help with the ease of development issue. When you use Browserify, your code is your build tool. Browserify gives you all the benefits of writing code in node (no anon functions to avoid globals, npm, simple requires, imports instead of namespaced globals) and it allows you to package that code to run on the client with one command and only load one file.
You can checkout my open source js framework Luc JS for an example. It runs on node and IE6. I'm able keep the code modular and build the single browser file with a one line command.

what are the advantages of using an AMD like requirejs or commonjs modules in javascript?

I've read a lot of articles on AMD solutions like RequireJS or module loaders that follow CommonJS style in Javascript.
Let's say I have an app splitted in this parts:
App definition that rely on the framework i use
Model 1 that rely on App definition and framework
Model 2 that rely on App definition, Model 1 and my framework
I may write each part as a RequireJS module or a common JS module and split my project in how many files I want but what's the advantage of writing each part as a module or splitting them in many files and then load them in the right order (to avoid dependency problems) maybe concatenatening all the files in a big one to reduce HTTP requets (as done by r.js optimizer)?

In my opinion, there are three rather important reasons:
You can create and re-use modules without polluting the global namespace. The more polluted your global namespace is, the bigger the chance of a function/variable collision. That means you define a function called "foo" and another developer defines the function "foo" = one of the functions gets overwritten.
You can structure your code into separate folders and files and requirejs will load them asynchronously when needed, so everything just works.
You can build for production. RequireJS comes with its own build tool called R.JS that will concat and uglify your javascript modules into a single (or multiple) packages. This will improve your page speed as the user will have to make less script calls and load less content (as your JS is uglified).
You can take a look at this simple demo project: https://c9.io/peeter-tomberg/requirejs (in cloud9ide).
To build your modules into a single app, all you have to do is have requirejs npm package installed and run the command: r.js -o build/build.properties.js
If there are any questions, ask away.
Edit:
In development, having all modules in separate files is just a good way to structure and manage your code. It also helps you in debugging (e.g. error on "Module.js line 17" instead of "scripts.js line 5373").
For production, you should use the build tool to concat and uglify the javascript into a single file. This will help the page load quicker as you are making less requests. Every request you make to load something slows down your page. The slower your page, the less points Google gives you. The slower the page, the more frustrated your users will be. The slower your page, the less sales you will get.
If you wish to read more about web page performance, look at http://developer.yahoo.com/performance/rules.html