I'm building a Node.js application and can't figure out how to configure the winston logger the right way.
What I would like to acchieve:
My application should have 6 custom logging levels (fatal, error, warning, status, info & debug). For those levels, there should be methods such as logger.status(message, ...details).
I would like to be able to pass additional JS objects (details) after the message for debugging/evaluation purposes.
Logs should be printed to the console
All console outputs should look the following:
[ISO Date String] [Level in uppercase]: [Log Message]
fatal & error logs should be printed in red
debug & info logs should only be printed if a constant (e.g. this.debug related to my class) is set to true. debug logs should be printed in yellow.
Logs of all levels should be written to a DailyRotateFile, in the same format as above (except for the colors of course).
Logs should be stored in a MongoDB with the following properties: date, level, message, details (containing all objects passed), label (specifies from which part of the log was sent, to be set manually).
What I've done so far:
As recommended, I've extended the default winston class using TypeScript in order to add methods for my specific logging levels:
import winston = require('winston');
interface Logger extends winston.Logger {
fatal: winston.LeveledLogMethod;
error: winston.LeveledLogMethod;
warning: winston.LeveledLogMethod;
status: winston.LeveledLogMethod;
info: winston.LeveledLogMethod;
debug: winston.LeveledLogMethod;
}
From this interface, I've created an instance like so:
this.logger = <Logger>winston.createLogger({
levels: {
fatal: 0,
error: 1,
warning: 2,
status: 3,
info: 4,
debug: 5
},
level: 'debug',
transports: [
new winston.transports.Console({
}),
new winston.transports.DailyRotateFile({
filename: 'cplus-%DATE%.log',
dirname: this.logsDir,
datePattern: 'YYYY-MM-DD-HH-mm'
}),
new winston.transports.MongoDB({
db: this.dbClient
})
]
});
Basic logging to the console & file using the new methods works fine, for example
this.logger.fatal('Connection to database ' + this.dbUrl + ' failed, cannot start', err);
creates the following output:
{"message":"Connection to database mongodb://localhost:27017 failed, cannot startAuthentication failed.","name":"MongooseServerSelectionError","reason": ...
Now, how do I acchieve the rest, such as formatting and correct storing in MongoDB? I've tried a few different things using the preset Winston formats, with no success.
Best regards!
Related
I am trying to deploy my Cloud Function using Cloud Firestore as Trigger. The Cloud function simply listens to any new document creation on my firestore path and logs the new data to the console. However, the function deployment is failing and there is no clear error message. Could you please help me identify what could be the issue?
Cloud Function Code:
const functions = require('firebase-functions');
exports.createUser = functions.firestore
.document('test_restaurant/{id}/reviews/{id}')
.onCreate((snap, context) => {
console.log(snap.data());
});
Error Log:
2020-06-28 18:51:03.110 IST
Cloud Functions
UpdateFunction
asia-east2:function-test-2
abc#gmail.com
Function failed on loading user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs
Expand all | Collapse all
{
insertId: "5u231ccch0"
logName: "projects/fs-22/logs/cloudaudit.googleapis.com%2Factivity"
operation: {
id: "operations/ZmlyZXN0b3JlLTI0OTcwNS9hc2lhLWVhc3QyL2Z1bmN0aW9uLXRlc3QtMi9xOVJCbHpESzdjSQ"
last: true
producer: "cloudfunctions.googleapis.com"
}
protoPayload: {
#type: "type.googleapis.com/google.cloud.audit.AuditLog"
authenticationInfo: {
principalEmail: "abc#gmail.com"
}
methodName: "google.cloud.functions.v1.CloudFunctionsService.UpdateFunction"
resourceName: "projects/fs-22/locations/asia-east2/functions/function-test-2"
serviceName: "cloudfunctions.googleapis.com"
status: {
code: 3
message: "Function failed on loading user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs"
}
}
receiveTimestamp: "2020-06-28T13:21:03.364975479Z"
resource: {
labels: {
function_name: "function-test-2"
project_id: "fs-22"
region: "asia-east2"
}
type: "cloud_function"
}
severity: "ERROR"
timestamp: "2020-06-28T13:21:03.110Z"
}
The issue resolved. Basically the error was using the same wildcard twice in the document path. The reference causing the problem was {id}, I just changed one of those references. The new path is: .document('test_restaurant/{id}/reviews/{reviewsId}') and the deployment succeeded.
Check if you have some dependencies installed on your root folder. You should have all your modules installed inside the functions folder. Only firebase dependencies should be in your root folder. Try to do npm uninstall yourDependencie inside your root folder and then npm i yourDependencie inside root/functions folder.
If you are using lint, you need to make sure your code is passing eslint . command, other wise youl'll get that error deploying to firebase
{
"#type":"type.googleapis.com/google.cloud.audit.AuditLog",
"status":{
"code":3,
"message":"Function failed on loading user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging"
},
"authenticationInfo":{
"principalEmail":"your_email"
},
"serviceName":"cloudfunctions.googleapis.com",
"methodName":"google.cloud.functions.v1.CloudFunctionsService.UpdateFunction",
"resourceName":"projects/your_project/locations/us-central1/functions/your_func"
}
For my case I fix it by dissabling prettier on index, also, if you have more files you have to disable it for each one.
/* eslint-disable prettier/prettier */
I am sending logs directly to Elasticsearch from a Node.js app using the winston and winston-elasticsearch packages. Elasticsearch 7.5.1, Logstash & Kibana 7.5.1 were deployed on a remote server using Docker Compose.
Problem 1: After running the node.js file that sends 2 log messages to Elasticsearch, the program does not automatically exit to return to the terminal. Using Node.js v12.6.0 on Mac OS X Mojave 10.14.6.
Problem 2: After these 2 log messages were sent to Elasticsearch, they can be viewed using a web browser at http://<example.com>:9200/logs-2020.02.01/_search.
{"took":5,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":2,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"logs-2020.02.01","_type":"_doc","_id":"85GgA3ABiaPPk4as1pEc","_score":1.0,"_source":{"#timestamp":"2020-02-02T02:00:35.789Z","message":"a debug message","severity":"debug","fields":{}}},{"_index":"logs-2020.02.01","_type":"_doc","_id":"9JGgA3ABiaPPk4as1pEc","_score":1.0,"_source":{"#timestamp":"2020-02-02T02:00:35.791Z","message":"an info log","severity":"info","fields":{}}}]}}
However, these logs do not show up on Kibana, such as the Logs section at https://<example.com>/app/infra#/logs/stream?_g=().
Any idea how to get the logs to also show up on Kibana? Also, why is the Node.js app not exiting after sending the log messages?
Thank you!
Node.js App
const winston = require('winston');
const ElasticsearchWinston = require('winston-elasticsearch');
const options = {
console: {
level: 'debug',
handleExceptions: true,
json: false,
colorize: true
},
elasticsearch: {
level: 'debug',
clientOpts: {
node: 'http://user:pass#example.com:9200',
log: 'debug',
maxRetries: 2,
requestTimeout: 10000,
sniffOnStart: false,
}
}
}
var logger = winston.createLogger({
exitOnError: false,
transports: [
new winston.transports.Console(options.console),
new ElasticsearchWinston(options.elasticsearch)
]
});
logger.debug('a debug message');
logger.info('an info log');
I'm not a node.js expert so I will only focus on the kibana issue. The Logs app is not meant to be for "custom" logs/indices like yours.
As stated in the documentation (https://www.elastic.co/guide/en/kibana/current/xpack-logs.html):
The Logs app in Kibana enables you to explore logs for common servers, containers, and services.
The logs app is for monitoring your infrastructure and ELK-Services, e.g. through certain Beats-modules (e.g. the Elasticsearch-, Kibana- and Logstash-Module of Filebeat).
Also from the docs (https://www.elastic.co/guide/en/kibana/current/xpack-logs-configuring.html):
The default source configuration for logs is specified in the Logs app settings in the Kibana configuration file. The default configuration uses the filebeat-* index pattern to query the data.
This explains why you dont see any data in the logs app since your indices use the 'logs-*' index pattern.
Long story short:
To view the documents in your log-* indices, you need to open the Discovery (first icon on the left sidebar in Kibana) and select the index pattern you already have set up. This is the appropriate way of searching your application data in Kibana.
I hope I could help you.
I am creating full mean stack app with
NodeJs , Angular 6 , ExpressJs and MongoDB
I have managed to create a server and its working perfectly, instead of using console.log when logging errors in my app I have decided to use Winston Logger here is what I have now
Server side
var appRoot = require('app-root-path');
var winston = require('winston');
// define the custom settings for each transport (file, console)
var options = {
file: {
level: 'info',
filename: `${appRoot}/logs/app.log`,
handleExceptions: true,
json: true,
maxsize: 5242880, // 5MB
maxFiles: 5,
colorize: false,
},
console: {
level: 'debug',
handleExceptions: true,
json: false,
colorize: true,
},
};
// instantiate a new Winston Logger with the settings defined above
const logger = winston.createLogger({
transports: [
new winston.transports.File(options.file),
new winston.transports.Console(options.console)
],
exitOnError: false, // do not exit on handled exceptions
});
// create a stream object with a 'write' function that will be used by `morgan`
logger.stream = {
write: function (message, encoding) {
// use the 'info' log level so the output will be picked up by both transports (file and console)
logger.info(message);
},
};
module.exports = logger;
Note: Winston in server side works perfectly
Client-Side
I want to use winston in my client side angular 6 app .
Example: in one of my components i have this.
import * as logger from "winston";
.........
this.activeRouter.params.subscribe((params) => {
// tslint:disable-next-line:prefer-const
let id = params['id'];
this.moviesService.getReview(id)
.subscribe(review => {
console.log(review);
this.review = review;
});
});
As you can see I am using console.log(review) , Instead of console log I would like to use Winston .
How to use Winston logger in client-side ? am newbie to all this stuff any help will be apreciated.
Yeah it is possible, however default transport for browser is very limited. I recommend to use https://www.npmjs.com/package/winston-transport-browserconsole
npm install winston-transport-browserconsole -S
It is easy to use and supports logging json objects:
import * as winston from "winston";
import BrowserConsole from 'winston-transport-browserconsole';
const level = "debug";
winston.configure({
transports: [
new BrowserConsole(
{
format: winston.format.simple(),
level,
},
),
],
});
winston.debug("DEBUG ", {a: 1, b: "two"});
Yes - it can (technically) be used in the browser. Should it be? Almost definitely not (sadly). Winston is a fantastic logger for node. But, emphasis on "for node". If you want to use it on the client, you would need to add a bunch of node polyfills in addition to winston itself, which is very large relative to other client loggers. Between winston and those polyfills you are going to significantly increase the size of your artifact. Also, just fyi webpack 5 removed those node polyfills, so you would need to add them back manually.
According to this ticket: https://github.com/winstonjs/winston/issues/287 it's almost ready for browser use? Or mostly ready? It sounds like they recently started supporting logging in a browser environment.
I am trying to integrate the stackdriver-error-js library into my Vue project as a module.
The code and the setup:
in package.json
"stackdriver-errors-js": "^0.2.0"
in bootstrap.js
import errorHandler from './error/error-reporting';
in error-reporting.js
import { StackdriverErrorReporter } from 'stackdriver-errors-js';
let errorHandler;
errorHandler = new StackdriverErrorReporter();
errorHandler.start({
key: "{{.Config.StackDriverApiKey}}",
projectId: "{{.Config.StackDriverProject}}",
service: "{{.Config.GoogleCloudProjectID}}",
version: "{{.Copacknfig.GaeEnv}}",
disabled: false
});
export default errorHandler;
The actual error
The error I got now is (console output and test)
[vue-devtools] Ready. Detected Vue v2.4.2
(function testErrorReporting() {window.onerror(null, null, null, null, new Error('Test: Something broke!'));})();
stackdriver-errors.js:109 Uncaught ReferenceError: StackTrace is not defined
at StackdriverErrorReporter.webpackJsonp.556.StackdriverErrorReporter.report (stackdriver-errors.js:109)
at window.onerror (stackdriver-errors.js:67)
at testErrorReporting (<anonymous>:1:40)
at <anonymous>:1:111
and line (stackdriver-errors.js:109)
...
StackTrace.fromError(err).then(function(stack){
...
If you do not load the stackdriver-errors-concat.min.js file, you also manually need to also the stacktrace-js module.
stackdriver-errors expects a StackTrace object to be present.
Since the library you want to use is experimental, and therefore cannot be used in a production environment, it would be better to use a different library which has been tested and validated for production use.
I suggest using this other library instead, which includes features related to Stackdriver error reporting for Node.js and JavaScript.
First of all, install the dependency by running this command:
npm install --save #google-cloud/error-reporting
This will add the dependency automatically to package.json.
In error-reporting.js, you can add the dependencyby adding this to your code (All the parameters are optional):
var errors = require('#google-cloud/error-reporting')({
projectId: 'my-project-id',
keyFilename: '/path/to/keyfile.json',
credentials: require('./path/to/keyfile.json'),
// if true library will attempt to report errors to the service regardless
// of the value of NODE_ENV
// defaults to false
ignoreEnvironmentCheck: false,
// determines the logging level internal to the library; levels range 0-5
// where 0 indicates no logs should be reported and 5 indicates all logs
// should be reported
// defaults to 2 (warnings)
logLevel: 2,
// determines whether or not unhandled rejections are reported to the
// error-reporting console
reportUnhandledRejections: true,
serviceContext: {
service: 'my-service',
version: 'my-service-version'
}
});
After that, use this code to test if the error is properly reported by Stackdriver:
errors.report(new Error('Something broke!'));
Please be aware that this library is currently on a beta stage, so there might be some changes to it in the future.
This is my winston logger -
var winston = require('winston')
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({
// log which takes care of all the MAS scoring - node clusters and servers
filename: './test.log',
level: 'info',
json: true,
eol: 'rn'
})
]
})
And when I run the actual logger:
logger.info('hi', 'it meeeeee')
logger.info('hi', 'it youuuuuu')
It prints out like this in the test.log file -
{"level":"info","message":"hi it meeeeee","timestamp":"2016-08-18T08:42:01.768Z"}rn{"level":"info","message":"hi it youuuuuu","timestamp":"2016-08-18T08:42:01.770Z"}rn
And I want it to be like this:
{"level":"info","message":"hi it meeeeee","timestamp":"2016-08-18T08:42:01.768Z"}
{"level":"info","message":"hi it youuuuuu","timestamp":"2016-08-18T08:42:01.770Z"}
How is this possible? I have read this question Node.js winston logger; How to start from a newline when insert log into log file? - however it does not solve the issue in my case.
I.e. if I open in Notepad ++, command line or Sublime text I still get the same issue.
Try this eol: \r\n
Look official documentation