We Keep Coding

Installing gulp plugins locally vs globally

I am pretty new to Gulp and I've got a question: should I install gulp plugins (such as gulp-sass or gulp-imagemin) locally or globally? In examples around the web people mostly do it locally (with --save-dev option). As I understand, by doing so the modules are stored in the local node_modules folder, added in the local package.json as devDependencies and can be referenced to in the local gulpfile.js via require(). So, if I need to install the same modules for another project I work on, it may be accomplished by copying package.json to a new project's folder and typing in npm install in a command line tool (after getting to the project's folder). Fine.
But what if I, as a regular gulp user, don't have plans to upload and share my stuff on the npm space and not interested in maintaining devDependencies, can I in this case just install gulp plugins globally like npm install -g gulp-sass? Will they be found through a global path in my system? Is it an option on the whole, if I don't want to bother myself to copy package.json, run npm install every time I create a new project or have multiple copies of the same modules scattered around on my disk?

For plugins that you are using via gulp, what you have to do is install them locally to the project. Although you have to install gulp itself globally and locally, to run the file and then for the project to pick up the gulp based commands and functions.
The reason you should install the plugins via npm locally is so that it is specific to the project, for example if you then went to upload this project to your server or host on github then you would have to then go and have to globally install all of your packages again. If they are saved, they exist in your packages.json, this is so that when you go to run npm install to install all the packages for said project npm knows what to install.
If I can further clarify anything let me know.

A kind of "global install" for the typical case OP explained, could be to have a folder where you put your package.json with every dependencies you need for each of your projects, and run npm install from there.
And put here also your gulpfiles for each project (you could named them as you want), like gulpfile-project1.js, conf-project2.js... whatever.
Of course adjust your paths in those files accordingly.
And from this directory, run gulp using the -f flag:
gulp -f gulpfile-project1.js
So you end with this kind of scaffolding :
my_dev_directory
- project1
- dev
- main.less
- main.js
- www
- main.css
- main.js
- gulp
- node_modules
- package.json
- package-lock.json
- gulpfile-project1.js
In your gulpfiles, start to declare a const with the base path of your project to ease all path declarations, like:
const path_dev = '../project1/dev';
const path_www = '../project1/www';
This way you have only one node_modules directory installed on your disk and your projects directories contain only public files you want to upload, no dev files.
Of course one of the main drawback is that you share the same module version with all your projects, but it answered to OP's need and even if it's not recommended, not the "proper way to do", you are free to do as you want with your dev tools :)

Install node extensions without commands

I am doing an internship in a company.
I need to create a node server.
I installed node on the computer (Windows) and I should install some plugins like:
- nodejs-webpack
- colors
- uglify
Normally I need to enter a command like : npm install "theModule"
But the software can not access the internet (due to company restrictions) and support service can not authorize the software (or do not want).
Can I install modules in any other way ? (download from Google and slide archives in the correct folder for example).
If the answer is no, do you know how can i get around this security?

You need a private npm repository.
Check out this answer:
can you host a private repository for your organization to use with npm?

I found it !
Just for exemple, we will install 'nodejs-websocket' :
1) You just have to download it here.
2) Put files into your Node's directory (for me it's "C:\Program Files\nodejs\node_modules\npm\node_modules")
3) in your .js file just add this line : var ws = require("C:/Program Files/nodejs/node_modules/npm/node_modules/nodejs-websocket/")
Done ! Thanks for all :D

I added this as a comment on your own answer, but I figured I should add a real answer with a better explanation.
Normally when you run npm install package-name npm installs the package to a node_modules directory in the directory you are in at that moment. So if your app was located at C:\code\my-app then you would cd into that directory and run npm install package-name. This would create a node_modules directory at C:\code\my-app\node_modules if it didn't already exist. Then it would install package-name into that directory at C:\code\my-app\node_modules\package-name.
As long as the module is in the node_modules directory for your app, you can require the module in your code without entering a big long file path.
var ws = require('nodejs-websocket');
The place you manually installed your module is the global node_modules directory. It's where npm would install a module if you did npm install -g package-name. The global node_modules directory is added to your system path when you install npm. In other words, any modules you install in there would be accessible from the command line like any other command. For example:
npm install -g bower
That would install the "bower" package to the global npm module directory. Bower would then be accessible as a command line tool for you to use. For example:
bower install angularjs
The global directory is more for tools like that and not really for modules that you intend to use in your code. Technically you can require a module from anywhere by including the full path in the require call like you did, but the standard practice would be to place it in the node_modules directory in the root of your application, and then require it with just its name and not a full path.
Edit: Here's another tip that you might like to take advantage of as well.
When you normally install a module with npm install package-name, your application usually has a package.json file at the root of it. If it does, you can do npm install package-name --save and npm will add the package-name module to a list in your app's package.json file. Once a module is listed in your app's package.json file it's called a "dependency" of your app because it basically says your app depends on package-name.
Normally, when you have dependencies listed in package.json, you can completely delete your app's node_modules directory and then simply run npm install from within your app's root directory and npm will automatically install all dependencies it finds listed in your app's package.json file. Since your corporate firewall won't allow this automatic downloading of modules, you won't get that benefit. However it is still good practice to follow the same conventions.
A trick you can do to create your package.json file is to manually install your dependencies into your app's node_modules directory. Once you have the modules your app needs, you can instruct npm to create a package.json file for your app by simply running npm init. It will walk you through a few little prompts to fill out your package.json file with details about your app. It will then peek inside your node_modules directory to see if you've already installed any modules before having a package.json file. If it finds any, it will automatically add them to the dependencies field in the package.json file it creates :D
With a proper package.json in place you'll always have a nice tidy list of what dependencies your application needs, even if your node_modules directory gets deleted. Normally people add their app's node_modules directory to their .gitignore file, only checking in the package.json file. This way they don't store their dependencies in source control but they can still be easily installed on new machines that clone it by simply running npm install from inside the app's directory. In your case though you may want to just add node_modules to your source control since you can't let npm install dependencies automatically.

How do you uninstall all your bower packages?

Sometimes it's useful to rebuild an entire site and force bower to reinstall new versions of all the packages in bower.json.
However, there doesn't seem to be any way of doing that:
Attempt #1:
$ bower uninstall
bower not-installed 0
Nope, that only works on a package-by-package basis, even though a clean 'bower install' uses bower.json.
Attempt #2:
$ bower install -f -l 0
$
Nope, despite '-f', this does absolutely nothing if the dependencies are met.
Attempt #3:
$ rm -r bower_components
$
! Ah victory! ... wait, what's this?
rm: bower_components: No such file or directory
Oh darn, there's a .bowrc in this project that sets the directory to install things to.
My current terrible solution:
Run custom script that:
- Parse .bowerrc if one exists
- Load the directory if one is specified in the json block
- If the directory currently exists...
- ...recursively delete the directory.
It works, I suppose, but it's pretty annoying to have to setup repeatedly.
Am I missing something?
Is there not just a simple bower command to delete the local installed modules?
Seems like really basic functionality I would expect bower uninstall to do.
(This isn't really a very javascript question, but I'll happily accept something that hooks into the bower module somehow to make this happen in a simple node script)
Context
Edit: If you want 'motivation' for such a task, it's this: We have a jenkins server that builds our projects and runs tests. However, periodically it fails for no obvious reason; investigating, it's almost always because jenkins is using a previous copy of the repository with just a git-pull to update to the most recent version before building and running tests; as a result, the previous bower_components directory is there, and it is full of cached copies of the various components.
Here a few example of things which are ##$##$'d and require bower to be run again as a forced install:
1) Some idiot (>_> fitvids) deletes the previous tagged release of a project.
2) Some project has dropped off of bower / moved its github page
3) Some project (>_> jquery) has changed the way the files are laid out in a non-major version revision.
I realize that the 'correct' solution to this problem is: fix jenkins so it creates a new temporary directory for each build. ...but that's not in my control.
So, as a build step, I need to automate a way to delete the bower components and force them to all be reinstalled; either as a grunt task (part of the build) or a jenkins build step. However, remember from (3) above, that our projects use .bowerrc, so it's not as simple as simply deleting a folder.
It would be great if I could uninstall all the existing bower components as a pre-build step to make this work.
So... back to the question: Can this be done with bower?

Updated Answer
If you're trying to update all of your packages, use
$ bower update
Original Answer
Go to your bower.json file and remove all of the components, or libraries, that you want to uninstall from devDependencies.
After you have removed the ones you want gone, execute -
$ bower prune
start with -
"devDependencies": {
"angular": "~1.2.15",
"angular-ui-router": "~0.2.10",
"moment": "~2.5.1"
}
remove angular references from file -
"devDependencies": {
"moment": "~2.5.1"
}
execute
$ bower prune
watch your angular dependencies get uninstalled

how about
edit the bower.json
'rm -Rf bower_components/*'
bower install
I was trying to upgrade to polymer 0.2.4 from 0.2.3. I can't seem to find a quick way to uninstall a set of dependencies. So I just manually removed those polymer* dir under bower_components. But for some reason bower kept remembering I had 0.2.3 installed event with bower.json modified. A 'rm -Rf bower_component/*' seems to do the tricks.

Actually I do something a little bit tricky but it works for me:
for package in $(ls your_bower_components_folder); do bower uninstall "$package"; done;
bower install

Uninstalling Packages
To remove a package you can use the uninstall command followed by the name of the package you wish to remove.
bower uninstall
It’s possible to remove multiple packages at once by listing the package names.
bower uninstall jquery modernizr sass-bootstrap

Adapting Jumar Polanco's answer to use it in Powershell, it is possible to programmatically uninstall bower components in the following way:
In the Powershell interface, navigate to the location where bower.json and the bower_components folder is located. Usually is the root app folder.
Then you can run:
foreach($package in ls bower_components){bower uninstall $package}
Depending on what the packages dependencies are, it may be required to pay extra attention to the process, as some prompts which require extra input (Y/n) to continue the process may arise (such as dependency conflicts).

I don't know what build tools you use, but if it includes Grunt with grunt-bowercopy, you could use the clean option. It removes the bower_components folder (or whatever you've configured it to use) after copying out the required files.
Ideally, I'd prefer something that didn't require me to re-download all the dependencies with each build, but just the ones where doing a fresh install would find a newer version.
I'm looking for a better solution to this as well, so I'll update if I find one.

I've been using nombom to do this (as a bonus, it also re-installs your npm packages from scratch):
https://www.npmjs.com/package/nombom

This is what ended up working for me via Windows cmd prompt:
forfiles /p .\bower_components /c "cmd /c cd .. && bower uninstall #fname"

Difference between Grunt, NPM, and Bower (package.json vs bower.json)

When I want to add a package (and check in the dependency into git), where does it belong - into package.json or into bower.json?
From what I gather,
running bower install will fetch the package and put it in /vendor directory,
running npm install it will fetch it and put it into /node_modules directory.
This SO answer says bower is for front-end and npm is for backend stuff.
Ember-app-kit seems to adhere to this distinction from the first glance... But instructions in gruntfile for enabling some functionality give two explicit commands, so I'm totally confused here.
Intuitively I would guess that
npm install --save-dev package-name would be equivalent to adding the package-name to my package.json
bower install --save package-name might be the same as adding the package to my bower.json and running bower install?
If that is the case, when should I ever install packages explicitly like that without adding them to the file that manages dependencies (apart from installing command line tools globally)?

Npm and Bower are both dependency management tools. But the main difference between both is npm is used for installing Node js modules but bower js is used for managing front end components like html, css, js etc.
A fact that makes this more confusing is that npm provides some packages which can be used in front-end development as well, like grunt and jshint.
These lines add more meaning
Bower, unlike npm, can have multiple files (e.g. .js, .css, .html, .png, .ttf) which are considered the main file(s). Bower semantically considers these main files, when packaged together, a component.
Edit: Grunt is quite different from Npm and Bower. Grunt is a javascript task runner tool. You can do a lot of things using grunt which you had to do manually otherwise. Highlighting some of the uses of Grunt:
Zipping some files (e.g. zipup plugin)
Linting on js files (jshint)
Compiling less files (grunt-contrib-less)
There are grunt plugins for sass compilation, uglifying your javascript, copy files/folders, minifying javascript etc.
Please Note that grunt plugin is also an npm package.
Question-1
When I want to add a package (and check in the dependency into git), where does it belong - into package.json or into bower.json
It really depends where does this package belong to. If it is a node module(like grunt,request) then it will go in package.json otherwise into bower json.
Question-2
When should I ever install packages explicitly like that without adding them to the file that manages dependencies
It does not matter whether you are installing packages explicitly or mentioning the dependency in .json file. Suppose you are in the middle of working on a node project and you need another project, say request, then you have two options:
Edit the package.json file and add a dependency on 'request'
npm install
OR
Use commandline: npm install --save request
--save options adds the dependency to package.json file as well. If you don't specify --save option, it will only download the package but the json file will be unaffected.
You can do this either way, there will not be a substantial difference.

Update for mid 2016:
The things are changing so fast that if it's late 2017 this answer might not be up to date anymore!
Beginners can quickly get lost in choice of build tools and workflows, but what's most up to date in 2016 is not using Bower, Grunt or Gulp at all! With help of Webpack you can do everything directly in NPM!
Google "npm as build tool" result:
https://medium.com/#dabit3/introduction-to-using-npm-as-a-build-tool-b41076f488b0#.c33e74tsa
Webpack: https://webpack.github.io/docs/installation.html
Don't get me wrong people use other workflows and I still use GULP in my legacy project(but slowly moving out of it), but this is how it's done in the best companies and developers working in this workflow make a LOT of money!
Look at this template it's a very up-to-date setup consisting of a mixture of the best and the latest technologies:
https://github.com/coryhouse/react-slingshot
Webpack
NPM as a build tool (no Gulp, Grunt or Bower)
React with Redux
ESLint
the list is long. Go and explore!
Your questions:
When I want to add a package (and check in the dependency into git),
where does it belong - into package.json or into bower.json
Everything belongs in package.json now
Dependencies required for build are in "devDependencies" i.e. npm install require-dir --save-dev (--save-dev updates your package.json by adding an entry to devDependencies)
Dependencies required for your application during runtime are in "dependencies" i.e. npm install lodash --save (--save updates your package.json by adding an entry to dependencies)
If that is the case, when should I ever install packages explicitly like that without adding them to the file that manages dependencies (apart from installing command line tools globally)?
Always. Just because of comfort. When you add a flag (--save-dev or --save) the file that manages deps (package.json) gets updated automatically. Don't waste time by editing dependencies in it manually. Shortcut for npm install --save-dev package-name is npm i -D package-name and shortcut for npm install --save package-name is npm i -S package-name

NPM Pack / Nodejitsu ignores buried Node_modules folders

It seems that NPM pack, and by extension, nodejitsu, do not like my node_modules folders. :*(
I currently am building a web app.
My web app's project folder structure is as follows:
Root
Engine(folder) Server(folder) readme.md package.json
(Multiple Folders) (folder)(folder) node_modules(folder)
easyimage,mongodb,mysql(folders) socket.io (folder)
node_modules(folder, NPM Pack ignores this) node_modules(folder, NPM Pack ignores this)
Socket.io-client (folder, NPM Pack ignores this)
I hope everyone can see this structure alright!
The problem I am having is that when I run NPM Pack at the root directory, the entire directory structure is packed correctly, except for all of the node_modules folders below the first node_modules folder.
It is as if NPM pack completely ignores those node_modules folders. (The one below socket.io for instance).
Due to the fact NPM pack ignores these npm folders, jitsu is also ignoring them and I can't get my web app started.
How can I get NPM pack/nodejitsu to correctly package all of the node_modules folders correctly?
My current package.json file, at the root directory, looks like follows:
http://pastebin.com/SAU6rwb5
As you can see, I have attempted to use bundleDependencies to tell NPM Pack that I am trying to include some node_modules folders (modules?), but pack still ignores all of them... Also, if i include anything under "dependencies", NPM start creates a new (??) node_modules folder at the root directory... but at the root directory nothing needs node_modules... as you can see node_modules are used inside of the server folder.
How can I get NPM Pack to recognize the files and folders inside of all of the node_modules folders and pack them correctly?

(jump to last paragraph if you want a really easy solution)
I'm having a hard time understanding your app structure. I think I get what your trying to do though.
From https://npmjs.org/doc/folders.html , it actually goes into detail about when and why sub modules will or wont show up.
When installing locally, npm first tries to find an appropriate prefix folder. This is so that npm install foo#1.2.3 will install to the sensible root of your package, even if you happen to have cded into some other folder.
Starting at the $PWD, npm will walk up the folder tree checking for a folder that contains either a package.json file, or a node_modules folder. If such a thing is found, then that is treated as the effective "current directory" for the purpose of running npm commands. (This behavior is inspired by and similar to git's .git-folder seeking logic when running git commands in a working dir.)
If no package root is found, then the current folder is used.
When you run npm install foo#1.2.3, then the package is loaded into the cache, and then unpacked into ./node_modules/foo. Then, any of foo's dependencies are similarly unpacked into ./node_modules/foo/node_modules/....
Bear With me for another quote...
Cycles are handled using the property of node's module system that it walks up the directories looking for node_modules folders. So, at every stage, if a package is already installed in an ancestor node_modules folder, then it is not installed at the current location.
Consider the case above, where foo -> bar -> baz. Imagine if, in addition to that, baz depended on bar, so you'd have: foo -> bar -> baz -> bar -> baz .... However, since the folder structure is: foo/node_modules/bar/node_modules/baz, there's no need to put another copy of bar into .../baz/node_modules, since when it calls require("bar"), it will get the copy that is installed in foo/node_modules/bar.
This shortcut is only used if the exact same version would be installed in multiple nested node_modules folders. It is still possible to have a/node_modules/b/node_modules/a if the two "a" packages are different versions. However, without repeating the exact same package multiple times, an infinite regress will always be prevented.
As far as the bundledDependencies:
Upon publishing [this also applies to packing], npm will look in the node_modules folder. If any of the items there are not in the bundledDependencies array, then they will not be included in the package tarball.
I take this to mean that it will only bundle the listed modules in ./node_modules/ and specific sub-modules for a ./package.json . Then of course as written above, it recursively walks down the directory tree again... so if it sees another package.json file in this directory npm will look and see if that has any bundled deps to include in the pack.
So as I understand it right now, since you have no packages in your base directory, your bundled dependencies in package.json doesn't do anything, and actually having items in your bundledDependencies field does more harm than good.
To fix you need to edit the package.json files to include these bundles at each level.
I have had this issue before when trying to get a packed then unpacked meteor app working on nodejitsu. I solved it in a different way. In the root folder of my app I included all of the top level node modules and explicitly set their version in my package.json file.
From what I understand your file structure is such:
app
+-- Engine
+-- Server
| +-- socket.io
| | `-- package.json
| | +-- node_modules
`-- readme.md
`-- package.json
If this is so then you need to be editing the package.json under socket.io to include the bundled deps you want. Generally though you can trust the package maintainers to keep valid versions.(but in this case you can't?)
As for Socket.io-client not being packed that's a result of it being a dependency of socket.io.
If I was to suggest a way for you to make this easier for yourself, i'd suggest you to, in your MAIN top level package.json file, to include the dependencies you need for your app # the specific version you need. If you need them bundled for some reason, add them into the bundled section, if you need sub modules at a different version than what the author intended. Consider making a folder called package or vendor and then placing the modules in there, in those you can edit the package.json and bundles their dependencies to your hearts content. Be sure though that you do not ignore any files or folders under you vendor or packages directory with .npmignore or .gitignore files.
Alternatively, if this is too difficult (editing all theses files and specifying certain versions can be a pain) I'd suggest hosting your vendor packages some where where you could download them with a script, and then execute this script in the postinstall part of your package.json (take a read at https://npmjs.org/doc/scripts.html ... you would add this in the same section you have your "start" script.
I hope that clarifies things.