We Keep Coding

Must i specify dependencies in package json file

I just recently heard of the package.json file but all my small projects have always worked without it.
I've installed many npm modules globally and always use var x = require("express");"just for example" without even initializing the package.json and putting express as a dependency.
Is it really important

First of all, I strongly doubt require("express") will work out of the box without having the express package installed in your project's node_modules folder, but just globally. There are ways to accomplish this, though.
But even if you accomplished to require packages from the global installation folder, what happens if you have multiple packages requiring different versions of the same package? There may be breaking changes between major versions of packages. So if you install a new version of package xy globally, older projects of yours expecting a different version of package xy may stop working.
On just your local machine, you can achieve all that, still without a package.json though.
The main purpose of the package.json comes clear, when you want to share your project with other people. Aside from some basic information (like a package name and some description), it will also list the dependencies which need to be installed for the project to be runable. While the necessary dependencies may exist on your machine, how will you make sure, they are also installed on a different machine without having them listed somewhere? And the place for listing the dependencies of a package is -- surprise surprise --- the package.json file ...

They are global, not in your project so when you do the deploy, you will must have to install all global for each server.
Yuu can install packages-cli global, but project dependencies ( also dev on dev dependencies) is better have its own package.json so you can deploy.
Also if you share your project, how someone will know what packages is needed.
The better is to have for each project its own package.json on its root folder, even if you always use the same libs.

What's the point of having a "compatible version" (^version) declared in package.json if package-lock.json locks it?

I know the main advantages of package-lock.json and I agree with that. It not only locks the downloaded version in the last install, but also the uri... and that's required on most cases for being possible to replicate the most similar project as possible.
But one thing that seems weird to me is that package.json has the feature of declaring a dependency like dependency: ^1.0.0, that should make npm to download the most recent and compatible version of that package in each installation.
I'm working at a project that I actually need this. Otherwise every time my dependency releases a patch, it will be required to make a new commit updating package.json only changing the version, so my pipeline can also overwrite package-lock.json.
In short, it seems that while package.json uses a feature... package-lock.json prevents that one.
Am I missing something?

The point of package-lock.json is to accurately represent the tree as it actually exists at a point in time, so that someone cloning the project gets exactly the same tree you had.
If you want to upgrade that dependency to a newer version, just use npm update and then commit the updated package-lock.json. Other members of your team will get that update as part of the normal process of picking up the latest.
More in the npmjs.com page on package locks.
Let's consider as scenario where you and I are on a team and our project uses nifty-lib, with package.json saying "nifty-lib": "^0.4.0", and we don't share package-lock.json. Perhaps I've been working on the project a couple of months longer than you have and I got nifty-lib v0.4.0 when I installed it. But when you picked it up and installed, you got v0.4.1 (a bugfix update which, sadly, introduced a new bug). At some point, you notice what seems like a bug in our project, but I can't replicate it. We spin in place for a while trying to figure out why it happens to you and not to me. In the end, we realize it's because it's actually a bug in nifty-lib that they introduced in v0.4.1. Hopefully we then get 0.4.2 or something (or if there isn't one, we fix the bug and do a PR, meanwhile rolling back to 0.4.0 across the project).
If we'd been sharing package-lock.json, we wouldn't have spun in place wondering why the problem happened to you and not to me, because you would have had the same version of nifty-lib as me. As part of our normal cycle, we'd do npm update periodically, and if a new bug showed up in our tests, we'd know from the commit history that it was because of a bug in a dependency.
Now, for "me" and "you" read "dev" and "production". :-)
Which is why package-lock.json locks the version, but package.json lets you say "this or better". package-lock.json keeps your team unified on versions, but you can intentionally update with npm update, which shows up in the commit history so you can track down regressions to it.

As I mentioned in a comment above, the short answer is it makes updateing your dependencies easier.
However, another way I like to think about the two files is: package.json is the file the human reads, while package-lock.json is the file the computer reads.
NPM is a package / dependency manager. So, in your package.json file, you write out "these libraries are needed for my library to work." As a feature, you have a range of versions you can list a dependency at. This helps when you run npm update on a specific package. It'll look to see what is the latest version that matches within your *package.json**, and updates you lockfile.
The package-lock.json lockfile is useful because it verbosely describes what your node_modules/ folder looks like so it can be accurately recreated when someone else installs your library. Additionally, since this file is generated automatically, you don't have to worry about maintaining it.
Of course, all of this just happens to be how NPM (and conversely how most package managers) handle this. That is, there isn't a technical reason why we couldn't have one file to describe both the version range that would be allowed when running updates, and a verbose lockfile portion that pins versions to allow for a recreatable dependency tree.
Basically, it is just a convenience. You have one file to succinctly list what dependencies your projects needs. It is readable and easily updatable. The other file, the lockfile, is automatically generated and ensures each npm install gives you the exact same node_modules/ folder as before.

The best way to resolve vulnerabilities in package-lock.json?

I am warned about vulnerabilities in the packages listed in the package-lock.json file of my Node.Js project.
I can follow the advice here and reinstall all the packages with npm install <package-name>, however, I also use other npm projects that use the older versions of those packages, which will not get reinstalled with a simple npm install.
Does it mean I have to go to package-lock.json and manually change all the dependencies to the latest version?
What if they break?
Isn't there a proper way of doing the updates that ensures you won't break the other packages dependent on the old versions?

If the issue is on a package you directly depend upon, you should update it directly and save it to the package.json + lock its version in package-lock.json in the process by doing something like npm install your-dependency#latest --save[-dev]. But beware: there might be breaking changes that will break your code (for example in case the dependency had a major version update inbetween with some deprecations and breaking changes).
But if the issue is from a dependency of one of your dependencies, the very very best way to solve it is to raise an issue (potentially with a PR to help them) with the maintainer of the parent package, then when they provide an update, update the dependency itself in your project.
You can use npm audit to resolve some issues as well (probably not all, and if a sub-dependency version is specifically required by a dependency, it will not update it because it could break things), but the single best way to solve the issue for you and for everybody else is to get the maintainer of the module you want to update its dependencies, when/if they can.
Reinstalling everything will not solve the issue if the dependency is still vulnerable. Installing does not magically fix stuff, people do :-) However, what you may want to do is use npm outdated to list all the packages that have newer versions available and try to update them, one by one, and see if your vulnerabilities are resolved after that (npm audit).
One more thing: it's usually a bad practice to go and change stuff around manually in package-lock.json. This file should be only auto-generated by one of your npm install (or similar) scripts. This file is what is used by npm to resolve the list of exact dependency/subdependency versions on a fresh install, and it is really the single best way to ensure all the people who use or work on this project have the exact same version of all their dependencies, so it better be correct. Always commit your package-lock.json!

Is bad practice to run "bower install" when deploying in production?

I am working on an application that uses bower.js; it's the first time I use bower, so please correct me if you see anything evidently wrong in my problem description.
------------------------------------------------
Coming from a Ruby background, I expect a package manager to have a .lock file, tracked under git, that tells me exactly which are the versions currently in use. This doesn't seem to happen with bower (am I right?).
A couple of days ago I deleted and re-cloned my repository, and ran bower install, thinking that such command would just install the required versions of the js components.
Then, today I did a one-line fix in a javascript file, compiled application.js using grunt watch, and realised that application.js was automatically filled up with tons of new code from bower component updates I wasn't aware of.
I found out that our bower components were under .gitignore, and that bower install, that I had run a few days ago, had actually updated two components without me noticing it.
When I realised what was happening, I immediately looked into our deployment procedure, which I paste here:
bundle install --path ${SNAP_CACHE_DIR}/.bundle
npm install -g bower grunt-cli
bower cache clean && bower install && bower list
bundle exec cap [our application name] deploy
Is this dangerous? Will bower install update all the components, that are likely not updated in my local version and are not tracked by git, ending up having completely different js code in production?

Is this dangerous? Will bower install update all the components, that are likely not updated in my local version and are not tracked by git, ending up having completely different js code in production?
Yes, this may happen and can cause problems. Although the impact will be limited as long as your dependency versions are specified as e.g. "~1.2.3", which will lock the major/minor version and only allow patch level updates.
In contrast to bower, the package manager normally used in node.js environments - npm - has a feature/command called npm shrinkwrap, which creates an npm-shrinkwrap.json file which locks down your dependency versions so that it is safe to run npm install afterwards.
This is probably what you would want.
However, bower as it stands does not have this feature yet - there is a discussion about it going on on Github e.g. here.
I think there currently are the following options to solve this problem in your situation:
Un-ignore and commit your bower_components (very ugly because of the huge amount of noise this produces in git).
Specify your dependency versions down to the patch level, e.g. "1.2.3" instead of "~1.2.3".
Culprit: If your dependencies have sub-dependencies, they might still be specified on the minor-version level, which means that even if your direct dependencies have a predictable version, your transitive dependencies may not.
Stop using bower and use npm instead (interface-/usability-wise, they are almost identical imho) and use npm shrinkwrap to lock down your dependencies.
Cheers, Alex

package.json generation / npm unused packages

I'm introducing unit testing in my project and for this, I need to make myself a package.json file.
First question is, which unit testing suite are you using? I'm looking forward mocha which seem to be pretty much standard for Node.js projects.
Second question is: Is there any magical way of generating a package.json file? (for dependencies and versions)
Third question is: I've been testing a lot of npm packages while developing my project and now I'm stuck with a lot of probably unused packages. Is there any way to tell which one are useless? (I saw npm list installed which is useful though)

I am using Mocha.
npm init
npm ls will list "extraneous" next to ones that are not in your package.json. But, it sounds like you don't have a package.json yet.
Basically, your workflow is very backward. Here is how it is intended to work:
Start a new project with npm init. It has no dependencies.
Oh, I want to start using a package, say express? Add it to package.json under dependencies, then run npm install.
Oh, I want to start using a package for development, say mocha? Add it to package.json under devDependencies, then run npm install.
You seem to have some existing code with manually-installed packages (via npm install <packageName>), which is a mess. I suggest starting over and following the above workflow.

To answer the third question:
npm prune
will remove all installed modules that are no longer mentioned in your package.json.
And you should really have asked 3 separate questions.

I am also using Mocha. It has code coverage, BDD, TDD, runs in browser. It is pretty complete and also heavily maintained by I think one of the most brilliant javascript/node.js programmers named TJ.
It is almost impossible to guess which version(s) to use. Because npm does not know which version breaks which dependencies. You could probably install all dependencies using something like node-detective. Then you can just install them using npm.js from within javascript. Maybe I would like to tackle this in the future.
I would also probably delete all dependencies , next install needed dependencies back using step(2). But also disc-space is not such a big case anymore with the current HDs.
P.S: I think I also agree with Domenic

I am using vows. It's pretty good, but not perfect. I have found unit testing in node to often be challenging because of async callbacks to dbs & such, and have mostly been testing top level functionality.
Here's your magic: Managing Node.js Dependencies with Shrinkwrap.
The only way to know what packages you are using is to know. You can't generate this programmatically. My advice would be to remove packages aggressively, then retest all functionality - if it breaks, you'll know you need to reinstall one of your packages.

Answering your third question, you can use Sweeper to list unused dependencies, and them remove them from your package.json. Just npm install -g sweeper then on your project directory call sweeper on the command line.