npm script, copy package.json to dist when bundling - javascript

I am trying to add a second part to my npm bundle script. The first part runs great, however I am trying to copy in 3 files along with the bundle.
So right now I have :
"bundle": "NODE_ENV=production webpack --output-file bundledFile.js && cp package.json dist/",
The NODE_ENV=production webpack --output-file bundledFile.js works great by itself. The part that is not working is the && cp package.json dist/, I would like the script to copy my package.json (along with 2 other files actually, but just starting with this one) to the dist folder. Brand new to these scripts, any idea how to fix? Appreciate any advice, thanks!

The syntax should work (and seems to, looking at your comments). I would suggest splitting your npm scripts across multiple points, though:
{
"bundle": "NODE_ENV=production webpack --output-file bundledFile.js",
"copy": "cp package.json dist/ && cp README.md dist/ && cp .npmrc dist/",
"build": "npm run bundle && npm run copy"
}
In order to be cross-platform compatible (cp is not typically available on windows), I would also suggest adding a build file somewhere such as ./tools/copy-distrubution-files.js which would make use of fs to copy the necessary files, then call it in the npm scripts with node ./tools/copy-distribution-files.js. That will be (mostly) platform independent (you still have to assume that node is available as the nodejs executable, but that seems fairly reasonable to me).

The quickest way for me was to reference powershell in a package.json script like this:
"copyFile": "#powershell copy './source/package.json' './deploy'",

If you're running on windows use the following command:
"copy": "copy \"package.json\" \"dist\" && copy \"README.md\" \"dist\" && copy \".npmrc\" \"dist\""
copy instead of cp.
Don't forget to use "" for each path (escape them with \ within the quoted command). and if you need to define a long path, don't use / (slashes) but \ (backslashes)
like:
copy "devices\\VS-88UT\\index.html" "devices\\VS-88UT\\dist"
Also, if you prefere ther is a nice plugin to run bash command before and after each build

To copy folders and files in windows just use
xcopy git\\* dist\\ /e /i /h
I think this might help someone.

Related

How to bundle a nodejs server application and prepare it for distribution?

First of all, I'm not after a technique to package my application into a single executable.
I'm curious to find out what is the approach to create a copy of the app within the current project that is ready to be moved to server-enabled location.
Right now my server app is a git repo and it has all the usual files in there, together with the source folder:
./src/server/index.js
Do we simply copy everything that is in the ./src/server folder to ./dist/ then also copy the package.json into ./dist?
Then we copy the contents of the ./dist folder to a location that will be able to serve the application, like /www/app2/ and inside that location, we make sure that we have NODE_ENV=production in the environment and run npm install to pull the production dependencies?
But then, our package.json file would still have the development related scripts and other things we don't need in production?
What is a best-practices way to deploy a NodeJs app?
--- UPDATE ---
This is what I have prototyped so far and it is working:
"scripts": {
"clean:dist": "./node_modules/.bin/rimraf dist",
"prep:dist": "./node_modules/.bin/mkdirp ./dist",
"copy:server": "./node_modules/.bin/ycopy ./src/server/ ./dist/ -r '^((?!tests$).)*$' -i",
"copy:package": "./node_modules/.bin/copyfiles package-production.json ./dist/",
"build": "npm run clean:dist && npm run prep:dist && npm run copy:server",
"start:dev": "nodemon src/server/index.js",
"start:server": "node dist/server/index.js",
"prompt": "echo 'No prompt functionality available'",
"greet": "echo 'Welcome to my project.'"
},
So the idea is to selectively move bits from the dev/src folder to a production ready dist folder. The idea behind having a simple package.json file is that we will not be needing the dev dependencies in there also we will not be needing most of the dev scripts as well. So probably something like the following will be enough:
"scripts": {
"setup:server": "NODE_ENV='production' && npm install"
"start:server": "pm2 start index.js"
}
... or maybe we would like to have some csh/bash scripts inside ./dist/bin that will streamline the start process.
"scripts": {
"start:server": "./bin/launcher"
}
I can definitely see a need for a custom project tree structure existing within the the ./dist folder and totally different to the ./src structure.
I am not sure why the "development" contents of your package.json are a problem, so perhaps I am not getting the crux of your problem. However, for our environment we deploy all of our node apps (primarily microservices) with ansible. The deployment package is just a tarball (could be a zip file). The ansible package includes templates for a config.js file and a pm2 startup.json file that are both customized based on the environment of the target (staging/test vs production).
Let me know if you want a few more details if you are interested in this approach.

Can i move my node_modules folder to another project?

I had some issues with a current iOS React Native project that i spent a long time working on. I've decided to start afresh. However i did spend a lot of time ensuring i got the correct versions of various react native modules and configuring these to work. These sit in my old 'node_modules' folder.
Can i copy these modules/folders (from the old 'node_modules') over to my new project? Do i need to update 'package.json' or link using 'react-native link xxxxx'? It it as simple as that in theory?
You don't need to copy over your node_modules directory. You can if you'd like, but it is not considered best practice. In any case, you shouldn't make any modifications to the files inside node_modules.
Preferably you need to only copy over your package.json file and optionally, your package-lock.json (or yarn.lock if you're using Yarn) file so that your project will be easily installable and upgradeable on other computers.
When you have a package.json or package-lock.json file, you can run npm install (or yarn install) to install the packages to your node_modules directory.
Copying your package-lock.json file over as well would ensure that the exact same versions of all of the packages that you have installed in your previous project will be installed in the new project as well. See this for more information on the package-lock.json file.
Unfortunately, I don't know much about react-native and linking react-native dependencies, but from this answer it seems that you would have to link any dependencies that contain native code again after you've run npm install.
No, I wouldn't recommend you to copy specific module from node_modules folder as once installed it has entries in .bin folder and files which you will miss while copying and it will be of no help in new project as they will be downloaded and installed again due to missing indexes.
Solutions :
You can use same package.json and package-lock in another project if you are sure that the dependencies version in it are exactly compatible or one you want, and install those dependencies in new project. The package-lock.json will ensure the version you choose.
Yes you can copy whole node_modules (have done it multiple times) from one project to another and use same package.json and package-lock (Will only save time in dependencies installation/download)
As you are starting fresh what i will recommend is create a new package.json and copy only those dependencies which are important for the project initialization, and as you progress add specific dependencies you need while developing. This will save you from huge garbage of unused dependencies which are hard to keep track of once the project inflates.
Maybe you can move it but you can't copy it, at least not with a naive cp -r! The node_modules/.bin directory ought to be full of symlinks to scripts in their respective directories in node_moudles, but after a cp -r these will be copies, which will now resolve relative module paths incorrectly.
For example, npm install typescript installs a executable script with the contents
#!/usr/bin/env node
require('../lib/tsc.js')
in two locations. Before a copy, that looks like this:
$ ls -l node_modules/.bin/tsc node_modules/typescript/bin/tsc
-rwxr-xr-x 45 node_modules/.bin/tsc
-rwxr-xr-x 45 node_modules/typescript/bin/tsc
and after a cp -r copy, like this:
$ ls -l node_modules/.bin/tsc node_modules/typescript/bin/tsc
lrwxr-xr-x 21 node_modules/.bin/tsc -> ../typescript/bin/tsc
-rwxr-xr-x 45 node_modules/typescript/bin/tsc
which makes the ../typescript/bin/tsc no longer exist, resulting in
$ tsc -b
...
Error: Cannot find module '../lib/tsc.js'
...

Why to use "rimraf dist" command in build script

What is the reason for running "rimraf dist" command in the build script in package.json file?
"scripts": {
"build": "rimraf dist ..."
},
The rimraf package which you will probably find in the devDependencies section of the package.json you have, is used to safely remove files and folders on all platforms (Unix and Windows).
When you have a build system, you want to also make sure you remove the output files before re-emitting the build output content (especially in case you have removed some files which are not needed anymore). You could go ahead and do:
"scripts": {
"clean": "rm ./dist/* -Recurse -Force"
}
But that will work on Windows only, and sometimes will also give you problems due to issues around the use of *. On Unix the command would be different. In order to make things better, rimraf does the removal for you so you can simply invoke it in your scripts/clean section:
"scripts": {
"clean": "rimraf dist"
}
This will ensure your package.json (your build system) to be cross-platform.
rimraf
A rm -rf util for nodejs
$ rimraf dist removes the dist file or folder.
I guess the build script puts stuff inside the dist directory and wants to remove the old stuff from the last time you build it.
To be sure that all old files are deleted from the folder before the script will compile the files.
What actually needs to be known is what the script rimraf dist is doing, and that's simple it's cleaning out the dist folder to give you a clean start on what is assumed to be a web app or TypeScript app that requires some level compilation.
But, it isn't limited to that alone. Rimraf is a package that allows you the power of rm -rf ./dist which is specific to Linux OS. As suggested in the above comment this command will give you problems on Windows, especially when there is a directory that isn't empty or the file path becomes too long (very common with nested npm dependencies). Rimraf will bypass those exceptions as if you're running on Linux and this becomes a very powerful tool.

Development server of create-react-app does not auto refresh

I am following a tutorial on React using create-react-app.
The application is created by create-react-app v1.3.0
create-react-app my-app
The dev server is run by
npm start
After changing the code several times, the browser is not updated live / hot reload with the changes. Refreshing the browser does not help. Only stopping the dev server and starting it over again capture the new changes to the code.
I was reading a lot of unnecesary stuff.
I am using React 17.
And my problem was that the page just add new components but the browser was not refreshing the page.
If your terminal is Compiling... and then you don't see changes on the browser, you should try adding a .env file in the root path of the project and add FAST_REFRESH=false.
Hot refresh was replaced by Fast refresh by default.
Have you seen the “Troubleshooting” section of the User Guide?
It describes a few common causes of this problem:
When you save a file while npm start is running, the browser should refresh with the updated code.
If this doesn’t happen, try one of the following workarounds:
If your project is in a Dropbox folder, try moving it out.
If the watcher doesn’t see a file called index.js and you’re referencing it by the folder name, you need to restart the watcher due to a Webpack bug.
Some editors like Vim and IntelliJ have a “safe write” feature that currently breaks the watcher. You will need to disable it. Follow the instructions in “Disabling swap files creation in vim”.
If your project path contains parentheses, try moving the project to a path without them. This is caused by a Webpack watcher bug.
On Linux and macOS, you might need to tweak system settings to allow more watchers.
If the project runs inside a virtual machine such as (a Vagrant provisioned) VirtualBox, create an .env file in your project directory if it doesn’t exist, and add CHOKIDAR_USEPOLLING=true to it. This ensures that the next time you run npm start, the watcher uses the polling mode, as necessary inside a VM.
If none of these solutions help please leave a comment in this thread.
I hope this helps!
Adding a .env file in the base path of the project and inside add FAST_REFRESH=false.
This disables fast refresh and returns to hot reload.
If you don't want to add a .env file to your base path you can choose these options:
"start": "FAST_REFRESH=false react-scripts start", in the package.json.
FAST_REFRESH=false npm run start, on the command line.
FAST_REFRESH=false, exporting it as an environment variable.
React 17
React-scrits 4
Have you tried npm start with super user permissions? I had the issue with my project and I solved it like this.
$sudo bash
#npm start
In WSL2 work for me, "If the project runs inside a virtual machine such as (a Vagrant provisioned) VirtualBox, create an .env file in your project directory if it doesn’t exist, and add CHOKIDAR_USEPOLLING=true to it. This ensures that the next time you run npm start, the watcher uses the polling mode, as necessary inside a VM."
Or just run:
$ CHOKIDAR_USEPOLLING=true npm start
Spent few hours fixing this:
1 . Create a file .env (beside package.json file) add below contents:
File: .env
FAST_REFRESH=false
2 . Now, stop & start the server
(Ctrl + C to start, if in CMD, on Windows OS)
npm start
4 . Now, change some text in the App.js
File: App.js
from "Learn React"
to "Learn React And it's working"
NOTE:
1 . Server restart is important.
2 . Refresh browser tab if you dont see changes.
In my case, it was there are not enough number of file watchers. I have to change the configurations manually.
See active file watchers limit using below command on terminal.
cat /proc/sys/fs/inotify/max_user_watches
Add below line to the /etc/sysctl.conf file.
fs.inotify.max_user_watches = 524288
Apply changes using the command below.
sudo sysctl -p
I had this problem while running npm within WSL. I had the project folder in my windows Desktop folder from which npm cannot recompile automatically in WSL.
After moving the project folder to user home directory of WSL solved the issue.
just create .env file in the root of your app
and add the following to it
.env
FAST_REFRESH=false
On win10 in WSL2, I had to create the .env in the root folder, and include both
FAST_REFRESH = false
CHOKIDAR_USEPOLLING=true
My setup doesn't include any virtual machine (unless WSL2 is considered a VM?). Nevertheless the .env with the above two entries got it up and running.
Find your index.js and change something in this file, for example add a white space, then save.
It should show "compiling..." in your console.
Then you can modify other files and react will refresh on save.
It seems that npm is looking for changes in the index.js at the very first time, if you refactor your folder structure the index.js could be missed. Force an update in index.js file get the problem solved.
At least this has worked to me
For users of WSL2, be mindful that if your project is in your Windows System(ie C: or D:) then it won't work.
Solution 1:
access these files through the WSL share, \wsl$\DISTRO_NAME from Windows.
Solutions 2:
Inside your package.json
find
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
}
and replace it with
"scripts": {
"start": "CHOKIDAR_USEPOLLING=true react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
}
Credits to https://github.com/facebook/create-react-app/issues/10253
As of react-scripts version 5.x.x CHOKIDAR_USEPOLLING=true no longer works in a .env file. You now need to use WATCHPACK_POLLING=true in the .env file in the root of your project. Per this thread.
If you are using visual studio code, you can create a new file and save it as .env. Inside, .env write FAST_REFRESH=false and save. Then, run npm start and this worked for me.
Edit: This might not be a recommended solution. The solution worked for Docker.
If using docker with a volume mount, you need to add an .env file in the src folder with the command CHOKIDAR_USEPOLLING=true in it. However, for me this threw an error
/app/src/App.js: Cannot find module '#babel/parser'
. To resolve this new error, changing the "react-scripts": "3.4.3" to "react-scripts": "3.4.0" in the package.json file worked. So you depending on your situation you may need to add the .env file and also change the react-scripts version.
Note: To put a little more context, I was working with docker and the react app files were mounted as a volume in the docker image (so that making changes in the app are directly reflected in the app without rebuilding a docker image). The above solution is based on other solutions posted in the community for docker where people had suggested changing the react scripts version. I don't think this should be a recommended solution. However, since I was doing a tutorial series I wanted to save time and focus on other things.
In case people come here looking for a better solution, my issue was resolved by moving my files inside the WSL2 filesystem. Hot reloading worked straight away with no need to add an .env file.
If you are running your app behind a reverse proxy / nginx (e.g. to enable https locally) you also need to enable websockets so it can detect the refresh:
location /sockjs-node {
proxy_pass http://dockerhost:5000/sockjs-node;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
In package.json, use "react-scripts": "3.4.4"
Delete package-lock.json
Run "rm -rf node_modules && npm install"
npm start
I was confused about how to create a .env file and even when i did it still didn't work. I was able to fix the issue by running this command on my terminal.
npm run start -FAST_REFRESH=false
Hopes it helps someone.
I ended up here seeking answers to a similar issue. Although mine was specific to one page that wasn't hot reloading. I do want to point out this is using Nextjs and it was a page under /pages named subscribe.js (/pages/subscribe.js)
It was very frustrating and admittedly took a while to figure out.
A lot of technical answers here, but interestingly, my issue was that my functional component did not start with a capital letter.
import React from 'react'
export default function subscribe() {
return (
<div>
</div>
)
}
Where the function name needed to start with a capital like so
import React from 'react'
export default function Subscribe() {
return (
<div>
</div>
)
}
As they say, you learn something new every day. In this game it's more like 10 things, the trouble is remembering what you learn ;)
push your commited changes to the branch and then delete the local repo folder then clone the repo again and run npm install or yarn install whichever you prefer.
this workaround solved my issue
Try using this command
echo fs.inotify.max_user_watches=524288
sudo tee -a /etc/sysctl.conf && sudo sysctl -p
If still, the error is there then you need to remove your node modules and again
npm install
and then
npm start
You might wanna add -w to {"start": "react-scripts start -w"}.
I had the same issue, fixed by adding --watch.
After creating a new project using create-react-app toolchain
Make sure to run
npm install, then
npm start
"dependencies": {
"react": "^17.0.1",
"react-scripts": "4.0.1",
}
if you are using these version then create .env file on the root directory of your project and type FAST_REFRESH=false and save it. Then yarn start OR npm start.
for linux first check the number of the files allowed using:
cat /proc/sys/fs/inotify/max_user_watches
in my, it was 8192 so I just change it to 524288 and it worked perfectly.
commands use to change is:
cd /proc/sys/fs/inotify
sudo nano max_user_watches
and then change no to 524288
save it and then use this command to apply the changes
sudo sysctl -p
reference from https://blog.jetbrains.com/idea/2010/04/native-file-system-watcher-for-linux/
Watch out where you add your .env file to. When you call create-react-app app-name, React will add following folder structure:
./root
./app-name
./node_modules
./public
./src
package.json
README.md
My problem was, I added .env file (with FAST_REFRESH=false inside) under the ./root directory. Placing the file in my ./app-name folder solved the issue.
I tried all the above suggestions, but still my react app does not refresh on code changes. Then I copied the previously working react project folder (only frontend) pasted it into the new project I am starting. Removed all the code related to old project and started using it. This solved my problem.
If this is feasible for you, you too can adopt the same method. If anyone discover a new simple solution, please post it here.
I had to run npm build everytime I wanted to refresh the app. Reinstalling node and npm didn't help. The project used a build tool called Parcel. Deleting the auto generated files such as .parcel-cache and dist resolved the issue for me.
My solution was to delete auto generated untracked/ignored files. The following git command might help:
git status --ignored
If you are on linux, check if it works with root access. If it does, stop your server and disable enforcement (for more details, man selinux).
sudo setenforce 0
Start your server again (without root), it might work.

Running npm-scripts on node

I am trying to get away from using Grunt or Gulp in my projects. I thought that a good way to replace them is by using npm-scripts.
I know that npm-scripts leverages package.json, but I noticed that to run more advanced build processes, you need to include command line functions. However, this is not a cross-platform solution since Windows doesn't support the wide variety of commands that an OS like Linux supports.
So, I was wondering if you could just run npm-scripts on Node, and just reference any npm package you want to with a require statement.
Is this possible? If not, are there any good cross-platform solutions that exist for npm-scripts excluding Grunt and Gulp?
Yes you could run "npm-scripts on node". For instance I have this in my package.json (irrelevant parts are removed for clarity), and both rimraf and webpack are implemented in pure JS and interpreted by node.js. In fact rimraf is a good example of cross platform rm -Rf. This solution runs fine on windows, mac or linux boxes by just issuing npm run-script build.
{
"scripts": {
"build": "rimraf dist && webpack --config ./blah.js"
},
"devDependencies": {
"rimraf": "^2.5.0",
"webpack": "^1.12.10"
}
}
Or you could do something like:
"scripts": {
"hello": "node hello"
}
And implement everything you want in hello.js in the same dir as your package.json, and include whatever you need in that script like:
const hello = require("debug")("hello"); // require whatever module you need
console.log("hello world");
It would run just fine with npm run-script hello
> your-module#1.0.0 hello D:\dev\tmp
> node hello
hello world
I would stick with webpack and npm only. Using webpack is much simpler than writing custom grunt or gulp tasks, dev server will give a stable local enviornment to work, and it is just a config setup and you are golden. I would also move away from using global installs especially if you work team environment or are running continuous delivery. As you won't be able to control the global installation or installed version on any of these environments.

Categories