We Keep Coding

Gitlab CI - server gets 'killed' before Cypress tests can run

I am running a CI pipeline in Gitlab which runs some Cypress integration tests as part of the testing stage. The tests are working absolutely fine on my machine locally but when I try and run them in Gitlab CI it appears that the Gitlab runner is killing my local server before I can run my Cypress tests against it. Here is my Gitlab config:
variables:
API_BASE_URL: https://t.local.um.io/api
CYPRESS_API_BASE_URL: https://t.local.um.io/api
npm_config_cache: '$CI_PROJECT_DIR/.npm'
CYPRESS_CACHE_FOLDER: '$CI_PROJECT_DIR/cache/Cypress'
cache:
paths:
- node_modules/
- cache/Cypress
stages:
- install
- build
- tests
install:
image: cypress/browsers:node14.15.0-chrome86-ff82
stage: install
cache:
key: 'e2eDeps'
paths:
- node_modules/
- cache/Cypress/
script:
- npm ci
build:
stage: build
dependencies:
- install
script:
- npm run build
artifacts:
expire_in: 1 days
when: on_success
tests:
image: cypress/browsers:node14.15.0-chrome86-ff82
stage: tests
script:
- npm ci
- npm run test:ci
And here are the relevant package.json scripts that the above config runs in CI:
"scripts": {
"build": "webpack --config webpack.prod.js",
"dev": "webpack serve --config webpack.dev.js",
"start:ci": "export NODE_OPTIONS=--max_old_space_size=4096 serve dist --no-clipboard --listen ${PORT:-3000}",
"test": "cross-env NODE_ENV=test && npm run test:cypress && npm run test:jest",
"test:ci": "cross-env NODE_ENV=test && start-server-and-test start:ci http-get://localhost:3000 test",
"test:cypress": "cypress run --headless --browser chrome",
"test:jest": "jest",
},
It is the final stage tests that is currently failing. Here is the console output from the Gitlab runner, you can see where it says 'killed' then 'err no 137' it appears that it just stops the start:ci process which is what runs my local server so the integration tests can run against them.
Finally here is a small snippet of my test, I use the cy.visit command which never responds as the server is killed:
describe('Code entry page - API responses are managed correctly', () => {
beforeEach(() => {
cy.visit(routes.APP.HOME); // this just times out
});
...
EDIT
I have tried running the test:ci script inside of the exact same docker container that it uses (cypress/browsers:node14.15.0-chrome86-ff82) locally (not in gitlabci) and it works no problem. The issue must lay with Gitlab surely?

Error 137 typically means that your Docker container was killed due to not having sufficient resources. As I mentioned in the comments, your current container is running with 4GB of memory. Since you are defining no tag keys within your CI/CD, you are likely running on GitLab's Linux runner cloud, which runs using a n1-standard-1 instance on GCP, which is limited to 3.75 GB of ram. Essentially, as soon as your test container starts, it instantly consumes all the memory available on the runner and your container is killed.
To get around the memory limitation, you must run your own gitlab-runner. There is no way to run using a higher amount of memory on the shared runner cloud. You can test this fairly easily by spinning up a gitlab-runner on your local machine (see instructions here for installing a gitlab runner). Once you've installed your runner, register your runner to the high-memory tag, then update your CI/CD to use that tag using the following syntax on that last job:
tests:
image: cypress/browsers:node14.15.0-chrome86-ff82
stage: tests
tags:
- high-memory
script:
- npm ci
- npm run test:ci
Your jobs are allowed to use as much memory as your runner has allocated. If your machine has 8Gb of memory, the jobs will be allowed to use up to 8Gb of memory.
If your machine doesn't have enough memory by itself, you could always temporarily spin up a cloud instance with sufficient memory. You could try a digital ocean droplet with 16GB of memory for 0.11c / hour, for example. This would allow you to run an instance for a couple of hours to test out the solution before you determine what's viable long-term.

Deploying app to google app engine and want to start api and client

Following this tutorial to get an api and client on google cloud platform:
https://www.freecodecamp.org/news/create-a-react-frontend-a-node-express-backend-and-connect-them-together-c5798926047c/
I have a root dir with an /api and /client inside it
the /api package.json has the following script
"scripts": {
"start": "node ./bin/www"
},
the /client package.json has the following script
"scripts": {
"client-install": "npm install --prefix client",
"start": "node server.js",
"server": "nodemon server.js",
"client": "npm start --prefix client",
"dev": "concurrently \"npm run server\" \"npm run client\""
},
When I try to deploy it says:
Step #0: Application detection failed: Error: node.js checker: Neither
"start" in the "scripts" section of "package.json" nor the "server.js"
file were found. Finished Step #0
I'm thinking it can't find the scripts? What is the best approach to start my api and client at the same time when I deploy?

My supposition is that the reported problem is generated by the fact that in root directory there is no package.json file. You could try to add one in root directory to handle stuff in app and client directories, but...
... but a taken a look to the tutorial you used and I found some things I don't like, so you want to give you some suggestions. I recently developed a simple tool to visualize charts about covid-19 diffusion in Italy, it uses same structure as your app with different approaches. The main difference is I deploy it in a VPS and I use external tool to launch it so in my package.json file there is not the script command to launch it in prod. It is something like:
cd client && npm install && npm run build && cd .. && npm install && node server.js
You can take a look to github repos to get ideas, but I'm going to explain the main differences.
server stuff (with package.json) is in root directory (simply this could solve your problem)
as per other answers, you need to add two lines to your express app to serve the built client as static files:
I suggest to use proxy features rather than cors package.
In your express app:
// At the begenning, with other middlewares
app.use(express.static(path.join(__dirname, "client", "build")));
// As last, to substitute any 404 with your client home page
app.get("*", (req, res) => res.sendFile(path.join(__dirname, "client", "build", "index.html")));
I don't know which kind of data you are going to treat, but CORS is a sort of protection you should never disable (by cors package), at least as it is possible. More than this, once you'll be able to solve the reported problem, I'm afraid that following part from the tutorial you used will not work, as it will try to fetch APIs from your app users's localhost:9000.
callAPI() {
fetch("http://localhost:9000/testAPI")
.then(res => res.text())
.then(res => this.setState({ apiResponse: res }));
}
To both enable CORS and solve fetching from localhost problem you should use proxy feature. In your client package.json file just add:
"proxy": "http://localhost:9000/",
This makes your webpack dev server to proxy calls it can't handle to your dev server, than changing your client fetching function to:
callAPI() {
fetch("/testAPI")
.then(res => res.text())
.then(res => this.setState({ apiResponse: res }));
}
it will automagically works both in dev and prod envs and let you to enable back CORS.
Hope this helps.

Well in that tutorial that guy creates the react app using create-react-app and because of that I don't know why you have in the 'start' command in your second package.json the following command 'node server.js'. You should have "start": "react-scripts start", that won't fix your problem but I'm not sure if server.js another server or what.
Anyway I'll try to help you, first 'create-react-app' creates an app which internally uses webpack-dev-server which is cool for developing, but in order to deploy your app you need to run this command
npm run build
After that you will have a folder called 'build' with your react app. Then you need to copy that folder and go to your server project and paste the
'build' folder there. Once you finish that you need to add the following code to tell your server that you want to serve static files
app.use(express.static(path.join(__dirname, 'build')));
And you also need to add the route
app.get('/', function (req, res) {
res.sendFile(path.join(__dirname, 'build', 'index.html'));
});
Now you can test it locally running only your server and the '/' path should take you to the react app. Once you know that is working, you can deploy ONLY that server with the build folder and just one package.json to google app engine.
I hope helped you!

How can I deploy Next js app on cpanel hosting

I am trying to deploy next js app on cPanel. I have installed node and npm on it.
How can I deploy the next js app on this set up?
I am getting the following error while trying to build the app on cpanel terminal:
Unhandled rejection TypeError: child.send is not a function

You'll need root privileges. If you've got them you can remote into your server via SSH and run the npm commands there. Then it should work. I.e, from console:
ssh user#serverip
cd /path/to/appdirectory
npm run start

There are some options to deploy NextJS app in CPanel:
You just deploy serverless app (without NodeJS) in CPanel. NextJS has provided a syntax like next export to producing optimized frontend files.
New version of CPanel already has entry point for your nodejs application:
Specify your server file on that field.

You should run next export to make static html and js files in out you can directly upload it to Cpanel as normal html website
you can run npm run export or yarn export by adding
"scripts": {
"dev": "next",
"build": "next build",
"start": "next start",
"export": "next export"
},
these in package.json

Next.js applications should run fine on a web hosting provider using cPanel, but running the default next.js server tends to exceed process count and memory quotas applied by those types of providers.
However you can make hosting next.js apps work if you build a production version of your app elsewhere and then use a custom server to call your next.js app to handle incoming requests.
I describe the process to deploy a very basic application as a subsite on my website here - https://www.watfordconsulting.com/2021/02/08/deploy-next-js-applications-to-a-cpanel-web-host/

All you really need to do is run next export and put the contents of the out folder in the public_html folder or /var/www/html/, /sites-available/, etc, depending on setup.

Run many nodejs servers in one command

I have few Nodejs servers, they are small servers and each one of them is stored in a separate folder. And all the folders are stored in one root folder. Every time i want to run the servers i have to go through each one of them and type
nodemon *name*.
This is becoming tiresome, especially that the number of servers is growing. is there any tool or a script i could use to run all the servers in one command??
Basically, how can i run all the servers in one command or a script?

With NPM. Write this in package.json :
{
"name": "project-name",
"version": "1.0.0",
"scripts": {
"start": "nodemon server1.js | nodemon server2.js | nodemon server3.js"
}
}
Then you only need to execute npm start.
Also see this post

PM2 is a great answer for this.
pm2 start app.js -i 4 (or max will take up all available cores)
You also get great benefits such as automatic restarts, log aggregation and load balancing.

Use pm2
If you use Linux
#!/bin/bash
pm2 start << Path to User Server>>
pm2 start << Path to User Server>>
pm2 logs
You can save
pm2 save
pm2 list
pm2 stop

Harp and browser-sync invoked by package.json with npm

I'd like to facilitate a way to start a harp.js server and run a browser-sync process at the same time. It's working like a charm on Linux. This is the content of my package.json
{
"scripts": {
"dev": "bash ./serve.sh"
}
}
And this is the serve.sh
#!/bin/bash
harp server &
browser-sync start --proxy 'localhost:9000' --files '*.jade, *.less'
When doing this on windows, I am supposed to do a bat file I presume, but isn't there a way to translate harp server & browser-sync etc etc to a corresponding command on windows?

for a package.json file, we would do something like this instead.
{
"scripts": {
"dev": "harp server & browser-sync start --proxy 'localhost:9000' --files '*.jade, *.less'"
}
}
give that a spin.