I have used node to manage dependencies on React apps and the like, in those you use package.json to keep track of libs and use them in your scripts using ES6 import module syntax.
But now I'm working on a legacy code base that uses a bunch of jQuery plugins (downloaded manually and placed in a "libs" folder) and links them directly in the markup using script tags.
I want to use npm to manage these dependencies. Is my only option:
run npm init
install all plugins through npm and have them in package.json
link to the scripts in the node_modules folder directly from the markup:
<script src="./node_modules/lodash/lodash.js"></script>
or is there a better way?
Check out this tutorial for going from using script tags to bundling with Webpack. You will want to do the following: (Do steps 1 and 2 as you mentioned in your question then your step 3 will change to the following 3 steps)
Download webpack with npm: npm install webpack --save-dev
Create a webpack.config.js file specifying your entry file and output file. Your entry file will contain any custom JS components your app is using. You will also need to specify to include your node_modules within your generated Javascript bundle. Your output file will be the resulting Javascript bundle that Webpack will create for you and it will contain all the necessary Javascript your app needs to run. A simple example webpack.config.js would be the following:
const path = require('path');
module.exports = {
entry: './path/to/my/entry/file.js',
output: {
path: path.resolve(__dirname, 'dist'),
filename: 'my-first-webpack.bundle.js'
},
resolve: {
alias: {
'node_modules': path.join(__dirname, 'node_modules'),
}
}
};
Lastly, add a <script> tag within your main HTML page pointing to your newly generated Javascript bundle:
<script src="dist/my-first-webpack.bundle.js"></script>
Now your web application should work the same as before your refactoring journey.
Cheers
I recommend Parcel js.
Then you only need:
Run npm init
Install dependency, for example npm install jquery
Import with ES6 syntax: import $ from "jquery";
And run with parcel
I've created a standalone routable engine with ember-engines 0.4.0, ember-cli 2.10.0.
I get this error if I call the engines index route (/thingy/):
Assertion Failed: Asset manifest does not list any available bundles.
Consuming App router.js:
this.mount('thingy-frontend', { as: 'thingy', path: 'thingy' });
Engine App routes.js:
this.route('index', { path: '/' });
The engine is 'installed' via a symlink in the node_modules/ dir of the consuming ember-cli app. (See here why).
Just for fun I've tried to change the routes to test if that works ...
Consuming App router.js:
this.mount('thingy-frontend', { as: 'thingy' });
Engine App routes.js:
this.route('index', { path: 'new' });
I've called /thingy/new and got an UnrecognizedURLError. Alternative, if I call the root path, I get an Assertion Failed: Asset manifest does not list any available bundles.
Also if I place a console.log('...'); in the engines index.js, I can't see any output. Seems like it isn't loaded at all.
The setup was inspired by the official README and the official example repos.
Any idea how to fix this Ember Engines setup?
You can find the repos on GitHub:
Engine: https://github.com/phortx/ember-engines-engine
Consuming App with README: https://github.com/phortx/ember-engines-app
We could solve the issue. There have been several problems and I'll share with you what we did:
1. Add ember-engines as dependency (not just dev-dependency)
You have to addember-engines as a dependency in the package.json both for the app and the engine. So we change
the package.json to:
"dependencies": {
"ember-cli-htmlbars": "^1.0.10",
"ember-cli-babel": "^5.1.7",
"ember-data": "^2.10.0",
"ember-engines": "0.4.0"
}
Don't forget to npm install.
2. Add the actual engine to the package.json
Even if it's not public and symlinked in node_modules like in our case, you have to add the engine to the package.json.
In our case this was "thingy-frontend": "*".
Don't forget to npm install.
3. Check symlink name
In our case the symlink had the name of the engine repo instead of the actual engine name. That won't work. We changed
the symlink name to thingy-frontend (that's the name from the engines index.js).
4. Use the right resolver
You have to ensure, that both in the addon/engine.js and the app/resolver.js use the ember-resolver.
5. Failed to load asset manifest.
This is probably a bug in ember-engines. See the issue for more details: https://github.com/ember-engines/ember-engines/issues/282#issuecomment-268834293
You can workaround that issue by manually adding a <meta />-Tag to the <head> (see the GitHub issue link above)
Many thanks to Michael Donaldson!
I can't find reference to your Engine app from Consuming app package.json. I think you should add to Consuming package.json Engine app. For in-repo-addons - engines I can find in ember-engines-demo that in package.json they have:
"ember-addon": {
"paths": [
"lib/ember-chat-engine"
]
}
For not in-repo-addon, but for normal modules they have:
"dependencies": {
"ember-data": "^2.6.0",
"ember-engines": "dgeb/ember-engines#v0.2",
"ember-blog-engine": "dgeb/ember-blog-engine"
},
Notice ember-blog-engine. Here's full reference to their package.json.
However in your Consuming ember-engines-app app package.json does not list
ember-engines-engine name.
Ember processes addons from package.json lists so you have to reference your engine addon there. Otherwise you will never get any line of code from such package executed in Ember CLI environment.
Please add your ember-engines-engine to consuming app package.json.
I'd add that incompatibility could also be an issue...
As Ember Engines is experimental and being developed against the master branches of Ember and Ember-CLI, be sure that you are using compatible versions.
I wanted to start a simple hello world app for Angular.
When I followed the instructions in the official quickstart the installation created 32,000 files in my project.
I figured this is some mistake or I missed something, so I decided to use angular-cli, but after setting up the project I counted 41,000 files.
Where did I go wrong? Am I missing something really really obvious?
There is nothing wrong with your configuration.
Angular (since version 2.0) uses npm modules and dependencies for development. That's the sole reason you are seeing such a huge number of files.
A basic setup of Angular contains transpiler, typings dependencies which are essential for development purposes only.
Once you are done with development, all you will need to do is to bundle this application.
After bundling your application, there will be only one bundle.js file which you can then deploy on your server.
'transpiler' is just a compiler, thanks #omninonsense for adding that.
Typical Angular2 Project
NPM Package Files (Development) Real World Files (Deployment)
#angular 3,236 1
rxJS 1,349 1*
core-js 1,341 2
typings 1,488 0
gulp 1,218 0
gulp-typescript 1,243 0
lite-server 5,654 0
systemjs-builder 6,470 0
__________________________________________________________________
Total 21,999 3
*: bundled with #angular
[ see this for bundling process ⇗ ]
There is nothing wrong with your development configuration.
Something wrong with your production configuration.
When you develop a "Angular 2 Project" or "Any Project Which is based on JS" you can use all files, you can try all files, you can import all files. But if you want to serve this project you need to COMBINE all structured files and get rid of useless files.
There are a lot of options for combine these files together:
YUI Compressor
Google Closure Compiler
For server side (I think it is best) GULP
As several people already mentioned: All files in your node_modules directory (NPM location for packages) are part of your project dependencies (So-called direct dependencies). As an addition to that, your dependencies can also have their own dependencies and so on, etc. (So-called transitive dependencies). Several ten thousand files are nothing special.
Because you are only allowed to upload 10'000 files (See comments), I would go with a bundler engine. This engine will bundle all your JavaScript, CSS, HTML, etc. and create a single bundle (or more if you specify them). Your index.html will load this bundle and that's it.
I am a fan of webpack, so my webpack solution will create an application bundle and a vendor bundle (For the full working application see here https://github.com/swaechter/project-collection/tree/master/web-angular2-example):
index.html
<!DOCTYPE html>
<html>
<head>
<base href="/">
<title>Webcms</title>
</head>
<body>
<webcms-application>Applikation wird geladen, bitte warten...</webcms-application>
<script type="text/javascript" src="vendor.bundle.js"></script>
<script type="text/javascript" src="main.bundle.js"></script>
</body>
</html>
webpack.config.js
var webpack = require("webpack");
var path = require('path');
var ProvidePlugin = require('webpack/lib/ProvidePlugin');
var CommonsChunkPlugin = require('webpack/lib/optimize/CommonsChunkPlugin');
var UglifyJsPlugin = require('webpack/lib/optimize/UglifyJsPlugin');
/*
* Configuration
*/
module.exports = {
devtool: 'source-map',
debug: true,
entry: {
'main': './app/main.ts'
},
// Bundle configuration
output: {
path: root('dist'),
filename: '[name].bundle.js',
sourceMapFilename: '[name].map',
chunkFilename: '[id].chunk.js'
},
// Include configuration
resolve: {
extensions: ['', '.ts', '.js', '.css', '.html']
},
// Module configuration
module: {
preLoaders: [
// Lint all TypeScript files
{test: /\.ts$/, loader: 'tslint-loader'}
],
loaders: [
// Include all TypeScript files
{test: /\.ts$/, loader: 'ts-loader'},
// Include all HTML files
{test: /\.html$/, loader: 'raw-loader'},
// Include all CSS files
{test: /\.css$/, loader: 'raw-loader'},
]
},
// Plugin configuration
plugins: [
// Bundle all third party libraries
new CommonsChunkPlugin({name: 'vendor', filename: 'vendor.bundle.js', minChunks: Infinity}),
// Uglify all bundles
new UglifyJsPlugin({compress: {warnings: false}}),
],
// Linter configuration
tslint: {
emitErrors: false,
failOnHint: false
}
};
// Helper functions
function root(args) {
args = Array.prototype.slice.call(arguments, 0);
return path.join.apply(path, [__dirname].concat(args));
}
Advantages:
Full build line (TS linting, compiling, minification, etc.)
3 files for deployment --> Only a few Http requests
Disadvantages:
Higher build time
Not the best solution for Http 2 projects (See disclaimer)
Disclaimer: This is a good solution for Http 1.*, because it minimizes the overhead for each Http request. You only have a request for your index.html and each bundle - but not for 100 - 200 files. At the moment, this is the way to go.
Http 2, on the other hand, tries to minimize the Http overhead, so it's based on a stream protocol. This stream is able to communicate in both direction (Client <--> Server) and as a reason of that, a more intelligent resource loading is possible (You only load the required files). The stream eliminates much of the Http overhead (Less Http round trips).
But it's the same as with IPv6: It will take a few years until people will really use Http 2
You need to ensure that you're just deploying the dist (short for distributable) folder from your project generated by the Angular CLI. This allows the tool to take your source code and it's dependencies and only give you what you need in order to run your application.
That being said there is/was an issue with the Angular CLI in regards to production builds via `ng build --prod
Yesterday (August 2, 2016) a release was done which switched the build mechanism from broccoli + systemjs to webpack which successfully handles production builds.
Based upon these steps:
ng new test-project
ng build --prod
I am seeing a dist folder size of 1.1 MB across the 14 files listed here:
./app/index.js
./app/size-check.component.css
./app/size-check.component.html
./favicon.ico
./index.html
./main.js
./system-config.js
./tsconfig.json
./vendor/es6-shim/es6-shim.js
./vendor/reflect-metadata/Reflect.js
./vendor/systemjs/dist/system.src.js
./vendor/zone.js/dist/zone.js
Note Currently to install the webpack version of the angular cli, you must run... npm install angular-cli#webpack -g
Creating a new project with angular cli recently and the node_modules folder was 270 mb, so yes this is normal but I'm sure most new devs to the angular world question this and is valid. For a simple new project it would make sense to pare the dependencies down maybe a bit;) Not knowing what all the packages depend on can be a bit unnerving especially to new devs trying the cli out for the first time. Add to the fact most basic tutorials don't discuss the deployment settings to get the exported files only needed. I don't believe even the tutorial offered on the angular official website talks about how to deploy the simple project.
Angular itself has lots of dependencies, and the beta version of CLI downloads four times more files.
This is how to create a simple project will less files ("only" 10K files)
https://yakovfain.com/2016/05/06/starting-an-angular-2-rc-1-project/
Seems like nobody have mentioned Ahead-of-Time Compilation as described here: https://angular.io/docs/ts/latest/cookbook/aot-compiler.html
My experience with Angular so far is that AoT creates the smallest builds with almost no loading time. And most important as the question here is about - you only need to ship a few files to production.
This seems to be because the Angular compiler will not be shipped with the production builds as the templates are compiled "Ahead of Time". It's also very cool to see your HTML template markup transformed to javascript instructions that would be very hard to reverse engineer into the original HTML.
I've made a simple video where I demonstrate download size, number of files etc. for an Angular app in dev vs AoT build - which you can see here:
https://youtu.be/ZoZDCgQwnmQ
You'll find the source code for the demo here:
https://github.com/fintechneo/angular2-templates
And - as all the others said here - there's nothing wrong when there are many files in your development environment. That's how it is with all the dependencies that comes with Angular, and many other modern frameworks. But the difference here is that when shipping to production you should be able to pack it into a few files. Also you don't want all of these dependency files in your git repository.
This is actually not Angular specific, it happens with almost any project that uses the NodeJs / npm ecosystem for its tooling.
Those project are inside your node_modules folders, and are the transititve dependencies that your direct dependencies need to run.
In the node ecosystem modules are usually small, meaning that instead of developing things ourselves we tend to import most of what we need under the form of a module. This can include such small things like the famous left-pad function, why write it ourselves if not as an exercise ?
So having a lot of files its actually a good thing, it means everything is very modular and module authors frequently reused other modules. This ease of modularity is probably one of the main reasons why the node ecosystem grew so fast.
In principle this should not cause any issue, but it seems you run into a google app engine file count limit. In that I case I suggest to not upload node_modules to app engine.
instead build the application locally and upload to google app engine only the bundled filesn but don't to the build in app engine itself.
If you are using angular cli's newer version use ng build --prod
It will create dist folder which have less files and speed of project will increased.
Also for testing in local with best performance of angular cli you can use ng serve --prod
if you use Angular CLI you can always use --minimal flag when you create a project
ng new name --minimal
I've just run it with the flag and it creates 24 600 files and ng build --prod produces 212 KB dist folder
So if you don't need water fountains in your project or just want to quickly test something out this I think is pretty useful
If your file system supports symbolic links, then you can at least relegate all of these files to a hidden folder -- so that a smart tool like tree won't display them by default.
mv node_modules .blergyblerp && ln -s .blergyblerp node_modules
Using a hidden folder for this may also encourage the understanding that these are build-related intermediate files that don't need to be saved to revision control -- or used directly in your deployment.
There is nothing wrong. These are all the node dependencies that you have mentioned in the package.json.
Just be careful if you have download some of the git hub project, it might have lot of other dependencies that are not actually require for angular 2 first hello world app :)
make sure you have angular dependencies
-rxjs
-gulp
-typescript
-tslint
-docker
There is nothing wrong with your development configuration.
if you use Angular CLI you can always use --minimal flag when you create a project
ng new name --minimal
I would like to be able to use compass to pre-process my SASS into CSS in an ember-cli project.
Doing this with broccoli-sass is trivial, as bower install broccoli-sass is all that is required, as the support for it is already built in.
Doing this with broccoli-compass however, has turned out to be rather tricky... how?
Details:
This question has been asked previously, for ember-cli v0.0.23;
and it's answer appears to be outdated -
The main issue appears to be that ember-cli abstracts a lot of the stuff inBrocfile.js, and puts it into another file, preprocessor.js, using a Registry, and thus the solution would be different, to a standard looking Brocfile.js
Update:
This question has been asnwered by #saygun, and the solution allows one to use broccoli-compass to compile SCSS --> CSS. However there are a couple of caveats:
Minor issue: The existing minifyCss preprocessor in meber-cli will not work. You will need to configure compass to minify its own CSS.
Major issue: If the SCSS files reference images, the generated CSS files contain links to images where the paths are within the temporary tree folders created by Broccoli. I am not sure how to work around this, and have asked a follow up question: How to generate image sprites in ember-cli using compass?
I have recently published ember-cli-compass-compiler which is a ember-cli addon for newer ember-cli apps(>= 0.0.37).
Just install using the command:
npm install --save-dev ember-cli-compass-compiler
No other configuration is needed, it converts app/styles/appname.scss to appname.css correctly. As the name indicates, it allows you to use Compass in addition to sass as well.
you need to install broccoli-compass:
npm install broccoli-compass --save-dev
and you need to install ruby gem sass-css-importer:
gem install sass-css-importer --pre
then in your brocfile do this:
var compileCompass = require('broccoli-compass');
app.styles = function() {
var vendor = this._processedVendorTree();
var styles = pickFiles(this.trees.styles, {
srcDir: '/',
destDir: '/app/styles'
});
var stylesAndVendor = mergeTrees([vendor, styles, 'public']);
return compileCompass(stylesAndVendor, 'app' + '/styles/app.scss', {
outputStyle: 'expanded',
require: 'sass-css-importer',
sassDir: 'app' + '/styles',
imagesDir: 'images',
fontsDir: 'fonts',
cssDir: '/assets'
});
};
I have some projects that use RequireJS to load individual JavaScript modules in the browser, but I haven't optimized them yet. In both development and production, the app makes a separate request for each JavaScript file, and now I would like to fix that using Grunt.
I have tried to put together a simple project structure to no avail, so I'm wondering if someone can provide a working example for me. My goals are the following:
In development mode, everything works in the browser by issuing a separate request for each required module. No grunt tasks or concatenation are required in development mode.
When I'm ready, I can run a grunt task to optimize (combine) all of the JavaScript files using r.js and test that out locally. Once I'm convinced the optimized application runs correctly, I can deploy it.
Here's a sample structure for the sake of this conversation:
grunt-requirejs-example/
grunt.js
main.js (application entry point)
index.html (references main.js)
lib/ (stuff that main.js depends on)
a.js
b.js
requirejs/
require.js
text.js
build/ (optimized app goes here)
node_modules/ (necessary grunt tasks live here)
Specifically, I'm looking for a working project structure that I can start from. My main questions are:
If this project structure is flawed, what do you recommend?
What exactly needs to be in my grunt.js file, especially to get the r.js optimizer working?
If all of this isn't worth the work and there's a way to use the grunt watch task to automatically build everything in development mode every time I save a file, then I'm all ears. I want to avoid anything that slows down the loop from making a change to seeing it in the browser.
I use the grunt-contrib-requirejs task to build project based on require.js. Install it inside your project directory with:
npm install grunt-contrib-requirejs --save-dev
BTW: --save-dev will add the package to your development dependencies in your package.json. If you're not using a package.json in your project, ignore it.
Load the task in your grunt file with:
grunt.loadNpmTasks('grunt-contrib-requirejs');
And add the configuration to your grunt.initConfig
requirejs: {
production: {
options: {
baseUrl: "path/to/base",
mainConfigFile: "path/to/config.js",
out: "path/to/optimized.js"
}
}
}
Now you're able to build your require.js stuff into a single file that will be minimized with uglifyjs by running grunt requirejs
You can bundle a set of different tasks into some sort of main task, by adding this to your grunt file
grunt.registerTask('default', ['lint', 'requirejs']);
With this, you can simply type grunt and grunt will automatically run the default task with the two 'subtasks': lint and requirejs.
If you need a special production task: define it like the above
grunt.registerTask('production', ['lint', 'requirejs', 'less', 'copy']);
and run it with
grunt production
If you need different behaviors for 'production' and 'development' inside i.e. the requirejs task, you can use so called targets. In the configuration example above it's already defined as production. You can add another target if you need (BTW, you can define a global config for all targets by adding a options object on the same level)
requirejs: {
// global config
options: {
baseUrl: "path/to/base",
mainConfigFile: "path/to/config.js"
},
production: {
// overwrites the default config above
options: {
out: "path/to/production.js"
}
},
development: {
// overwrites the default config above
options: {
out: "path/to/development.js",
optimize: none // no minification
}
}
}
Now you can run them both at the same time with grunt requirejs or individually with grunt requirejs:production, or you define them in the different tasks with:
grunt.registerTask('production', ['lint', 'requirejs:production']);
grunt.registerTask('development', ['lint', 'requirejs:development']);
Now to answer your questions:
I would definitely use a subfolder in your project. In my case I use a 'src' folder for development that is build into a 'htdocs' folder for production. The project layout I prefere is:
project/
src/
js/
libs/
jquery.js
...
appname/
a.js
b.js
...
main.js // require.js starter
index.html
...
build/
... //some tmp folder for the build process
htdocs/
... // production build
node_modules/
...
.gitignore
grunt.js
package.json
see above
You can do so, but I wouldn't recommend to add requirejs to the watch task, it's a resource hungry task and it will slow down your machine noticeable.
Last but not least: Be very cautious when playing around with r.js. Especially when you want to optimize the whole project with r.js by adding a modules directive to your config. R.js will delete the output directory without asking. If it happens that it is accidentally configured to be your system root, r.js will erase your HDD. Be warned, I erased my whole htdocs folder permanently some time ago while setting up my grunt task... Always add keepBuildDir:true to your options when playing around with the r.js config.