I'm developing a JavaScript project targeting browsers using the closure compiler, but lately I've been thinking to start using browserify and follow a node-like development approach: splitting the project in a number of smaller node modules, building everything using something like grunt, using node's require system (thanks to browserify) etc.
In order to build the whole project after slicing it into some modules, I need a way to get from every module its scripts, assets, some optional configuration, and to recursively do the same on its dependencies.
Is there any standardized way to do this? Any module already taking care of this? Anything providing assistance to this work?
As requested, here an example of what i need. I'll try to pick the most complex one.
The project heavily rely on HTML5 Canvas. It's supporting FlashCanvas, but I want it to be in a separate module because I also want to build the project without FlashCanvas support, and because I'd like to release the flashcanvas module on NPM so that it can be easily integrated in other projects.
When building with the FlashCanvas support, I need to:
put flashcanvas/flashcanvas.js and flashcanvas/flashcanvas.swf in the data directory that will be shipped with the project.
Include <!--[if lt IE 9]><script src="flashcanvas/flashcanvas.js"></script><![endif]--> in the HTML page.
In case I'm compiling with closure-compiler, the FlashCanvas externs file must be passed to the compiler.
Pass an option to browserify that replaces require('canvas') with require('flashcanvas'), in order to override the functions that create the canvases.
In case FlashCanvas was depending on other modules (this is not the case for this module), take assets/configuration/whatever from the dependencies.
I believe that my problem should be quite common among who's developing with browserify: it sounds pretty normal that some modules offer assets (CSS files, images etc) that you need to include. Still I couldn't find anything to solve the issue. Is my approach the right way to structure a node/browserify project? How am I supposed to get the data listed above from my main module's Gruntfile?
Related
I am running on a huge project with thousands of .js files that are written in es5, and for many reasons and benefits of TS, we've decided to start migrating to TS, after a couple of days and many pieces of research, I am going to elaborate a couple of points:
To start migrating from es5 to ts we can start either by two ways:
First way:
1- Installing ts, creating tsconfig with allowJS set to true then start changing the extension of files to .ts and everything will work by default
2- As we want to migrate gradually we don't want at once to replace the global scripts to native modules, in other words, we don't want immediately to type import and export, but instead we want to keep the old way global scripts and use /// <reference path=""> to load dependencies
3- after previous step we can gradually start turning the files into native modules esm
Second way:
1- As I've read about UMD it will work on both borwser(client) and server, which means support all type of modules AMD, CommonJS, SystemJS and Native ES modules
2- after re-writing the scripts in UMD fashion, we can gradually start moving our scripts to ESM
last but not least, and regarding the intelligence we'll start writing .d.ts files accordingly or we can rely on ts-loader to generate the files
lastly, we either go with ts-loader or babel but we are not sure if there are couple of limitations for each of which
Any idea is really appreciated on what is the best way to start migrating
Maybe this tool can help you:
https://github.com/airbnb/ts-migrate
ts-migrate is a tool for helping migrate code to TypeScript. It takes a JavaScript, or a partial TypeScript, project in and gives a compiling TypeScript project out.
ts-migrate is intended to accelerate the TypeScript migration process. The resulting code will pass the build, but a followup is required to improve type safety.
If possible, start fresh and write migration script which does copy/paste task of code. In which Project setup will be error free in term of compile/build/run/lint. So even in future there won't be any issue you just need to add modules. You can migrate module by module.
Now if you are migrating to TS means not just by extension or package type, because it won't benefit more, you will need to change in code like types, interfaces, configs etc.
So starting a fresh project and then converting code to TS and then gradually moving should be better.
I would suggest you should write new modules in typescript and slowly convert existing files and modules to typescript. But keep the output in js only. Convert each ts file into js, For e.g, if you have a module Utils create two folders inside this ts and js. Write a build script which will convert ts to js and move inside the js folder. This way you will be converting all files into ts. For import/export you can start it for the new module. I have done the same in one project.
Regards,
omi
I appreciate all the answers but for my case it is so complicated because ts modules will affect the scope, what I found useful is using what so-called shimmer modules in webpack which allows a feature called imports-loader which I can use to manipulate the scope of js, hence nothing is broken, then I can move file by file to start migrating
I've recently been thrown in to clean up a project which has like 45-50 individual .js javascript files. I wonder what the best approach would be to decrease the loading size of them all. Just concatenate all files into one with npm or gulp? Install some module loader? webpack?
If you're already concatenating, minifying, and uglifying and you don't want all the files to be loaded on all the pages due to a monolithic bundle, you might be looking for something like Webpack's Commons Chunk Plugin.
This plugin walks down the tree of dependencies for each endpoint defined in your Webpack.config file and determines which modules are required across all pages. It then breaks the code into two bundles, a "common" bundle containing the modules that every page requires, which you must load with a script tag on each page:
<script src="commons.js" charset="utf-8"></script>
And an endpoint bundle for each individual page that you reference normally in a script tag placed after the commons script tag:
<script src="specificpage.bundle.js" charset="utf-8"></script>
The result is that an individual page will not have to load modules that will only ever be used on other pages.
Again, this is a Webpack plugin. I don't know if this functionality is available as a Gulp plugin, because it must have knowledge of all endpoints in order to determine which dependencies are common to them all.
I redirect you to the very good https://github.com/thedaviddias/Front-End-Checklist
In particular the following advises:
JavaScript Inline: High You don't have any JavaScript code inline
(mixed with your HTML code).
Concatenation: High JavaScript files
are concatenated.
Minification: High JavaScript files are minified (you can add the .min suffix).
You can accomplish this with a package manager such as gulp, grunt or webpack (for the most famous ones). You just need to choose what you prefer to use.
If you consider webpack, You can start with my very simple (but understanding) starter: https://github.com/dfa1234/snippets-starter
There's no much thing that you can do, basically is:
Concatenation - https://www.npmjs.com/package/gulp-concat
Minification - https://www.npmjs.com/package/gulp-minify
Instead of creating all those scripts, you can get something to re-use on yeoman, f.e. the Fountain, so it will reduce a lot of time just typing procedural code for doing the concatenation/minification.
Also if you can use some lazy load (like RequireJS or some frameworks have support to lazy load the module, like Angular) that will improve the performance of your aplication
EDIT:
If you want even more performance, you can install some compression tool in your server, for example this one for NodeJS https://www.npmjs.com/package/compression
I'm my personal opinion, if you have time, the best approach would be to read and understand the purpose of the project. Then plan a proper refactor. You are not fixing anything with concatenating, this is just a deployment step.
You should analyze which technologies are being used and if you want to maintain this code, in the long run, make a proper refactor into a much more modern stack, maybe you can take a seed project with ES6, webpack, Babel... and create a proper repository well maintained with proper modularity and dependencies resolution.
Once you have that, decreasing the load its just about adding proper tools in build time (babel, webpack, etc).
You would like to add some unit tests and continue working properly :)
Existing App:
My application is using require.js.
Benefit of my application is that other people can extend my application via writing 3rd party plugins using require.js
For example: (a 3rd party plugin registration)
// registering a new plugin
{
url:'#new-page-url',
js:'plugin-folder/new-page-url.js'
}
So when #new-page-url is hit anywhere, requirejs consumes js file from plugin-folder/new-page-url.js
Please note that after I compile my application, it does not include third party source as they can be fetched on the fly with requriejs
Question:
I have been looking into webpack and since it compiles everything before distribution (bundle.js as starting file).How can a third party plugin work on the fly like above example ?
Please note that after I compile my application, it does not include third party source as they can be fetched on the fly with requriejs
Yes, that's the problem.
I'm in a situation similar to yours where a large application of mine written as a collection of AMD modules can load editing modes at runtime, which are also AMD modules. The modes are not generally bundled with the application.
AFAIK it is not possible to replicate only with Webpack the possibility of doing the equivalent of a RequireJS' require([a]) where a is a variable whose value cannot be known at build time but is determined at run time. (And for the benefit of readers who may not be familiar with RequireJS, I'll add that yes, I do mean the first argument to be [a] and not just a. RequireJS makes a distinction between the two.)
Webpack needs to know at build time which modules it is going to bundle together. ("Needs to know" means it needs to know the name and find the module's source.) So it does not support dynamically loading modules that only get to be known at runtime. If you read the documentation, or tutorial, you'll run into discussions of dynamic loading with Webpack but these do not allow doing the equivalent of the require([a]) case. Webpack can split a bundle into chunks and load chunks as needed but for this to work, Webpack still needs to know ahead of time the whole set of modules it is ever going to need. This does not allow loading at runtime a module that was unknown at build time. There's also require.context, but while it allows you to determine at run time which specific module you want, the set from which the module comes is determined at build time. If at build time you know that you're going to be using one of A, B, C, that's fine. But if you don't know at build time the name of your module, or cannot feed its source to Webpack, you're out of luck.
This being said, it should be possible to build your application's core with Webpack and have this core perform direct calls to a module loader like RequireJS or SystemJS. This is the direction I'm moving into with my own application but I've not crossed that bridge yet.
If you code an application using ES6 modules and classes, is there any need to use a module loader framework, or is the best practice to just use a build tool to concatenate all the code into a file (or files) and include those using a normal script tag?
Yes. Somebody, somewhere along the line has to load the module.
I think you're conflating compiling modules ahead of time vs loading them individually. Webpack is a module loader that outputs a single file for the browsers to use later, while the System API and requirejs et al load a number of individual files.
There are performance factors on both sides, particularly longer build time (when precompiling) vs longer load time (with multiple files).
Webpack, Browserify, and most other module loaders (with the notable exception of the System API) allow you to define some loaders for certain file types and automagically compile your (S)CSS or templates on the way through, as well as running other tools to uglify or obfuscate your code. The ES6 System API does not provide these features, but is a more robust runtime loader than most.
This boils down to two trade-offs:
support for non-JS modules (styles, templates) vs build time
single request and longer build vs many requests and short/no build
Evaluate them for your users (high-bandwidth vs mobile), environment (if you have two dozen CI agents, who cares if the build takes an extra 3s?), and stack (if you have a lot of template files, compiling them AOT could be important).
I have written a self contained angular js module using browserify in order to make use of the commonJS/Node style syntax.
The module works fantastic when tested by itself, so I then use gulp to minify and host that on GitHub.
I've then imported that into another app that is also using browserify. When I run browserify it seems to try and rebrowserify the module and causes no end of problems.
I believe this is because the module requires angular and jquery and qtip2. So it's obviously trying to re parse these.
Is there a standard to not parse modules, or is there a way to exclude the browserifying of the modules? Or is it best to not include things like angular and jquery within your modules? I was trying to make them perfectly stand alone, maybe that's unwise?
Many thanks!
I would suggest providing both options, if it is important for you to have a standalone version that includes angular. This will provide people using your code with a total of three ways of using your code: Using the standalone version, the version that only includes the module, and cloning the repository directly and including the source files as part their build process.
I generally use the third option, but people who don't have build processes will likely prefer the first or second.