My problem is that I need to increase max_payload value that NATS receive but I have no idea where I can do it.
The project is using Moleculer and NATS is created as a container with docker.
When I try to make a request which is bigger than 1MB NATS returns:
ERROR - NATS error. 'Maximum Payload Violation
Inside dockstation logs NATS returns:
cid:1 - maximum payload exceeded: 1341972 vs 1048576
I tried the following items:
Changing tranporter inside Moleculer Broker configs (https://moleculer.services/docs/0.12/transporters.html);
Add an config file for NATS to modify some options (https://hub.docker.com/_/nats);
Code example of Moleculer Broker configs:
const brokerConfig: BrokerOptions = {
...,
transporter: "NATS",
transit: {
maxQueueSize: 100000,
disableReconnect: false,
disableVersionCheck: false,
},
...
}
Code example of nats config file:
{
max_payload: 1000000
}
Error when I run docker with NATS config file:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/home/matheus/nats-server.conf\\\" to rootfs \\\"/var/lib/docker/overlay2/08959b2fce0deb2abea27e103f7f4426b7ed6f3ef64b214f713ebb993c2373e6/merged\\\" at \\\"/var/lib/docker/overlay2/08959b2fce0deb2abea27e103f7f4426b7ed6f3ef64b214f713ebb993c2373e6/merged/nats-server.conf\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type. error Command failed with exit code 125.
You should create a configuration file for NATS. And push it to the container as a Docker volume and set the command as -c nats-server.conf
nats-server.conf
max_payload: 4Mb
Start container
docker run -d -p 4222:4222 -v ~/nats-server.conf:/nats-server.conf nats -c /nats-server.conf
Related
Using the following tracing enabling script from OpenTelemetry docs:
const opentelemetry = require("#opentelemetry/sdk-node");
const { getNodeAutoInstrumentations } = require("#opentelemetry/auto-instrumentations-node");
const { diag, DiagConsoleLogger, DiagLogLevel } = require('#opentelemetry/api');
// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
const sdk = new opentelemetry.NodeSDK({
traceExporter: new opentelemetry.tracing.ConsoleSpanExporter(),
instrumentations: [getNodeAutoInstrumentations()]
});
sdk.start()
running my Next.js server as I thought is required, I get an error:
$ node --require './tracing/opentelemetry.js' ./node_modules/next/dist/bin/next start -p 3000
No modules instrumentation has been defined, nothing will be patched
#opentelemetry/instrumentation-grpc Module #grpc/grpc-js has been loaded before #opentelemetry/instrumentation-grpc so it might not work, please initialize it before requiring #grpc/grpc-js
Exporter "otlp" requested through environment variable is unavailable.
/mnt/vol/.local/share/pnpm/global/5/.pnpm/next#12.1.5_zpnidt7m3osuk7shl3s4oenomq/node_modules/next/dist/lib/get-project-dir.js:40
const realDir = _fs.default.realpathSync.native(resolvedDir);
^
TypeError: _fs.default.realpathSync.native is not a function
at Object.getProjectDir (/mnt/vol/.local/share/pnpm/global/5/.pnpm/next#12.1.5_zpnidt7m3osuk7shl3s4oenomq/node_modules/next/dist/lib/get-project-dir.js:40:50)
at nextStart (/mnt/vol/.local/share/pnpm/global/5/.pnpm/next#12.1.5_zpnidt7m3osuk7shl3s4oenomq/node_modules/next/dist/cli/next-start.js:80:37)
at /mnt/vol/.local/share/pnpm/global/5/.pnpm/next#12.1.5_zpnidt7m3osuk7shl3s4oenomq/node_modules/next/dist/bin/next:141:34
at processTicksAndRejections (node:internal/process/task_queues:96:5)
Node.js v17.8.0
Now this can be simplified to a minimal reproduction as follows. This has the fs.realpathSync.native function:
$ node -e 'console.log(require("fs").realpathSync)'
[Function: realpathSync] { native: [Function (anonymous)] }
This doesn't have fs.realpathSync.native:
$ node --require ./tracing/opentelemetry.js -e 'console.log(require("fs").realpathSync)'
No modules instrumentation has been defined, nothing will be patched
#opentelemetry/instrumentation-grpc Module #grpc/grpc-js has been loaded before #opentelemetry/instrumentation-grpc so it might not work, please initialize it before requiring #grpc/grpc-js
[Function (anonymous)]
Exporter "otlp" requested through environment variable is unavailable.
My Node's --require is working correctly (noop.js is an empty file):
$ node --require ./tracing/noop.js -e 'console.log(require("fs").realpathSync)'
[Function: realpathSync] { native: [Function (anonymous)] }
Why would the OpenTelemetry setup script break the fs module?
$ node --version
v17.8.0
//package.json dependencies
"#opentelemetry/api": "^1.3.0",
"#opentelemetry/auto-instrumentations-node": "^0.35.0",
"#opentelemetry/sdk-node": "^0.34.0",
$ uname -a
Linux code-server 5.15.0-1025-oracle #31~20.04.2-Ubuntu SMP Tue Nov 29 13:01:56 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
Does my ARM machine have something to do with it?
I can reproduce the same on x86_64 on https://replit.com/#JakubKoralewski/opentelemetry-repro with the same behavior.
The reason this error occurs is due to a bug in #opentelemetry/instrumentation-fs introduced as a new dependency to #opentelemetry/auto-instrumentations-node in PR #981 which got released with version 0.34.0. The issue was reported but at the time of writing is still open. However, as also already linked above a PR to address the issue is being reviewed.
Fow now, I see three ways to address the problem:
As suggested in a comment above downgrade #opentelemetry/auto-instrumentations-node to next lower version 0.33.1.
Disable the file system instrumentation when configuring the node instrumentation. For that simply replace getNodeAutoInstrumentations() with getNodeAutoInstrumentations({ '#opentelemetry/instrumentation-fs': { enabled: false } }) in your code. Given that your project is in Next.js and you likely have little file system activity aside from maybe public files this is likely the best option for now.
Remove #opentelemetry/auto-instrumentations-node altogether and simply instrument the libraries you actually use. Using the auto instrumentation for Node.js pulls in a lot of transitive dependencies. Say you have a Next.js app, connect to a Postgres database and use winston for logging your instrumentation setup could look something like this:
const sdk = new opentelemetry.NodeSDK({
traceExporter: new opentelemetry.tracing.ConsoleSpanExporter(),
instrumentations: [
// new FsInstrumentation(), TODO: re-enable once bug is fixed
new HttpInstrumentation(),
new PgInstrumentation(),
new WinstonInstrumentation(),
]
});
I have a node.js application running inside a docker container with base image node:16-alpine3.11.
I want to get the name of the container on which it is running on.
ex:
docker ps:
CONTAINER ID NAMES
xyz node
test.js:
const c_name= //get container name
From outside the container
You can use the docker cli to do this. In the example we filter for ancestor bitnami/redis (in your case this would be node:16-alpine3.11)
$ docker container ls --filter "ancestor=bitnami/redis" --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"
This returns the id, name and image of the container as a table. To retrieve the results as a json object, update the format flag like below:
docker container ls --filter "ancestor=bitnami/redis" --format 'json { "Id":"{{.ID}}", "Image": "{{.Image}}", "Names": "{{.Names}}" }'
See the handy cli reference docs provided by docker.
From Inside the Container
Pass the unix socket to the container and invoke a script to get the container name:
$ docker run -it -v "/var/run/docker.sock:/var/run/docker.sock" origami-duckling:latest
The Dockerfile for `origami-duckling looks like this:
FROM node:16-alpine3.11
WORKDIR /usr/app
RUN apk add curl jq
ENV DOCKER_HOST="unix:///run/docker.sock"
COPY get-container-name.sh /usr/app/get-container-name.sh
CMD /usr/app/get-name.sh
You would probably run the get-container-name before you use cmd to run the node app in your container.
The get-container-name.sh would look like this:
#!/usr/bin/env sh
export CONTAINER_NAME="$(curl -s --unix-socket /run/docker.sock http://docker/containers/$HOSTNAME/json | jq '.Name')"
echo $CONTAINER_NAME
Update: You can do this from within your node.js app if you need to:
I'm using got#11.8.3, got 12.x is pure ESM.
const got = require('got');
async function getHostName() {
const metadata = await got(`http://unix:/var/run/docker.sock:/containers/${process.env.HOSTNAME}/json`).json();
console.log('Container name', metadata.Name);
return metadata.Name
}
I have docker-compose.yml like following
version: '3'
services:
api-server:
build: ./api
links:
- 'db'
ports:
- '3000:3000'
volumes:
- ./api:/src
- ./src/node_modules
tty: true
container_name: api-server
db:
build:
context: .
dockerfile: ./db/Dockerfile
restart: always
hostname: db
environment:
MYSQL_ROOT_PASSWORD: test
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_DATABASE: test
volumes:
- './db:/config'
ports:
- 3306:3306
container_name: db
When I tried to docker-compose up
$ docker-compose up -d
I suffered some error like following one.
It seems like to set them absolute path. but before this message didn't appeared.
I'd like to know
①What cause this error.
③How to fix it without rewrite to absolute path.
db is up-to-date
Recreating bf6187ceff2f_api-server ... error
ERROR: for bf6187ceff2f_api-server Cannot create container for service api-server: invalid volume specification: '9131f8ce856163b2935bd8f09d5e6a2e67509fd5adee9b2d1903cd760639beaa:src/node_modules:rw': invalid mount config for type "volume": invalid mount path: 'src/node_modules' mount path must be absolute
ERROR: for api-server Cannot create container for service api-server: invalid volume specification: '9131f8ce856163b2935bd8f09d5e6a2e67509fd5adee9b2d1903cd760639beaa:src/node_modules:rw': invalid mount config for type "volume": invalid mount path: 'src/node_modules' mount path must be absolute
ERROR: Encountered errors while bringing up the project.
If someone has opinion,please let me know.
Thanks
Just saw that you're trying to do this.
The error you're describing is happening because you need to specify the absolute path inside the container, so it mounts over a volume. Something like
this: /src/node_modules
I used to start a nodejs script with this command
node cli/myscript.js -c configs/some-config.json
I tried to start the same script using pm2. I found here a post handling the same theme.
I tried this :
pm2 start cli/myscript.js --node-args="-c configs/some-config.json"
I get a syntax error on the config file which I don't get when I don't use pm2.
SyntaxError: Unexpected token :
0|myscript | at checkScriptSyntax
I also tried this and get the same error:
pm2 start cli/myscript.js -- -c configs/some-config.json
It seems like that pm2 tries to execute the config as a js file.. because the config file is a valid json.
You can pass the file name in the same-config.json.
{
"apps" : [{
"name" : "myscript",
"script" : "cli/myscript.js",
"watch" : true,
"env": {
"NODE_ENV": "development"
}
}]
}
Then you can run the node server by following command -
pm2 start same-config.json
For more details please refer PM2 docs
I can't understand why I'm getting this "Error: ENOENT" error. Here is my Meteor server method:
createImage: function(coords) {
console.log('createImage')
console.log(coords.area)
console.log(coords.x)
console.log(coords.y)
console.log(coords.x2)
console.log(coords.y2)
console.log(coords.w)
console.log(coords.h)
var gd = Meteor.npmRequire('node-gd');
var path = Meteor.npmRequire('path');
var fs = Meteor.npmRequire('fs');
var source = 'forrest.png';
var target = 'compimages';
if (path.exists(target)) fs.unlink(target);
gd.openPng(source, function(png, path) {
if(png) {
console.log(png)
console.log(path)
}
}
);
}
Here is the output I get from it on the terminal:
=> Meteor server restarted
I20140827-15:30:18.451(-7)? createImage
I20140827-15:30:18.455(-7)? 27888
I20140827-15:30:18.456(-7)? 242
I20140827-15:30:18.459(-7)? 164
I20140827-15:30:18.459(-7)? 410
I20140827-15:30:18.459(-7)? 330
I20140827-15:30:18.459(-7)? 168
I20140827-15:30:18.460(-7)? 166
W20140827-15:30:18.527(-7)? (STDERR) path.exists is now called `fs.exists`.
I20140827-15:30:18.547(-7)? { [Error: ENOENT, open 'forrest.png'] errno: 34, code: 'ENOENT', path: 'forrest.png' }
I20140827-15:30:18.548(-7)? undefined
These are the directories within ~/myapp/server/
me#ubuntu:~/myapp/server$ ls
compimages forrest.png privateimages server.js user-setup.js
I want to access forrest.png and use node-gd to cut a section of it out based on the coords passed in.
As far as I understand it this error means that there is a directory missing. The png file I'm trying to access is in the same directory as the .js file calling it (they are both in ~/myapp/server/ so to me it doesn't look like that's the problem. The only thing I can see is that the path is undefined. I've got that installed though:
me#ubuntu:~/myapp/packages/npm/npm/node_modules$ ls
node-gd path
Can anyone see what's going on with this? I'm using Meteor 0.9.0
If you console.log(process.cwd()) in your server code you will notice that the current working directory of your Meteor app is project/.meteor/local/build/programs/server however your code assumes that the CWD is project/server.
What you can do is prefixing your paths with the project root server folder which is obtained by going 5 level ups in the filesystem hierarchy.
var projectRootServer="../../../../../server";
var source=projectRootServer+"forrest.png";
This is not very elegant but it works.
Unrelated but it seems that path.exists has been deprecated in favor of fs.exists, you should fix this too.