We Keep Coding

Using the Datadog logs and RUM SDKs sends multiple requests when errors are being thrown

I have the following issue:
Trigger
An uncaught exception is being thrown and datadog logs SDK sends request to log the incident.
Expected outcome
One request is sent/incident and the incident is logged only once in Datadog UI.
Actual outcome
Datadog logs SDK sends many requests/incident (between 1k-2.5k) and the incident is logged many times in Datadog UI.
Additional Information
When disabling the Datadog RUM SDK, then the Datadog logs SDK behaves as expected. However, I want to run them both, so this is not an option at the moment.
I am using version 3.1.3 for both #datadog/browser-logs and #datadog/browser-rum packages.
Here's a screenshot to illustrate the issue:
Many requests being sent for one uncaught exception example
This is the code I am using to initialise both logs and RUM SDKs:
import { datadogLogs } from '#datadog/browser-logs';
import { datadogRum } from '#datadog/browser-rum';
if (process.env.NODE_ENV === 'production' && process.env.DATADOG_CLIENT_TOKEN) {
const environment = getEnvironment();
const config = {
site: 'datadoghq.eu',
clientToken: process.env.DATADOG_CLIENT_TOKEN,
service: typeof DATADOG_SERVICE !== 'undefined' ? DATADOG_SERVICE : undefined,
env: environment ? `${environment}` : undefined,
proxyHost: process.env.PROXY_HOST
};
datadogLogs.init(config);
if (process.env.DATADOG_APPLICATION_ID) {
datadogRum.init({
...config,
trackInteractions: true,
applicationId: process.env.DATADOG_APPLICATION_ID
});
datadogRum.setUser({
name: service.getName(),
email: service.getEmail()
});
}
}

I had the same issue, Upgrading both Datadog Rum and logging to 3.6.13 fixed this for me:
"#datadog/browser-logs": "^3.6.13",
"#datadog/browser-rum": "^3.6.13",

For anyone else:
Use localhost, not custom domain names. For some reason, DD_RUM.getInternalContext() didn't work for custom domain names.
Install versions of both RUM and logs - latest and possibly the same version. Incase you want specific varying versions, check the dependency packages for the datadog packages on the package.lock. It can be different which can throw errors.

How can I see what EXACTLY is Node.js passing as command line arguments?

Running commands via Node's execFile on windows is a nightmare. Many a time, when I get a "Command failed" error if I copy the "failed" command from the error it executes just fine. Sometimes, trying windowsVerbatimArguments: true helps but usually, it does not.
For example, right now I am running Visual Studio's MSBuild.exe, with the following parameters in Node.js - I have printed the exact array passed to execFile:
Running MSBuild with parameters: [
'D:\\PROJECT\\vcxproj\\HelperProject.vcxproj',
'-t:OutputBuildMacro',
'-p:ProjectToImport=D:\\PROJECT\\vcxproj\\HelperProject.vcxproj;PropertyToGet="OutputType,BlueProjectsVersionPropsGlobalIncluded"'
]
Executable path: C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\amd64\MSBuild.exe
If I copy these items from the array and paste them into the command line it will run just fine. Clearly, Node.js is messing up when passing the command. Because this has happened to me many times, just a few weeks ago with Irfan View, I am not looking for a solution for this specific problem - I would just be asking a similar question again very soon.
I am looking for a guide how to see what is Node.js actually passing so that I can make a guess how to edit the parameters so that they are accepted by the command I call.
How to see what is Node.js exactly sunning why I call execFile?

You can use NODE_DEBUG environment variable to get extra info out of NodeJS.
You can find docs here.
For example this index.js:
const { execFile } = require('child_process');
execFile('/bin/ls', ['/Volumes'], {}, (err, stdout, stderr) => {
if (err) {
console.error('Command failed');
} else {
console.log('Command succeded');
}
});
And to enable debug logging for child_process module execute:
NODE_DEBUG=CHILD_PROCESS node index.js
(On Windows the environment variable seems to be set differently)
set NODE_DEBUG=CHILD_PROCESS & node index.js
in my case it produces this kind of output:
CHILD_PROCESS 85208: spawn {
cwd: null,
env: null,
gid: undefined,
uid: undefined,
shell: false,
windowsHide: false,
windowsVerbatimArguments: false,
args: [ '/bin/ls', '/Volumes' ],
detached: false,
envPairs: [
'COMMAND_MODE=unix2003',
...
'TERM=xterm-256color',
'NODE_DEBUG=CHILD_PROCESS',
],
file: '/bin/ls'
}
P.S.
Setting NODE_DEBUG=* will produce all debug log information that is available in NodeJS.

Configuring Winston Logger

I'm building a Node.js application and can't figure out how to configure the winston logger the right way.
What I would like to acchieve:
My application should have 6 custom logging levels (fatal, error, warning, status, info & debug). For those levels, there should be methods such as logger.status(message, ...details).
I would like to be able to pass additional JS objects (details) after the message for debugging/evaluation purposes.
Logs should be printed to the console
All console outputs should look the following:
[ISO Date String] [Level in uppercase]: [Log Message]
fatal & error logs should be printed in red
debug & info logs should only be printed if a constant (e.g. this.debug related to my class) is set to true. debug logs should be printed in yellow.
Logs of all levels should be written to a DailyRotateFile, in the same format as above (except for the colors of course).
Logs should be stored in a MongoDB with the following properties: date, level, message, details (containing all objects passed), label (specifies from which part of the log was sent, to be set manually).
What I've done so far:
As recommended, I've extended the default winston class using TypeScript in order to add methods for my specific logging levels:
import winston = require('winston');
interface Logger extends winston.Logger {
fatal: winston.LeveledLogMethod;
error: winston.LeveledLogMethod;
warning: winston.LeveledLogMethod;
status: winston.LeveledLogMethod;
info: winston.LeveledLogMethod;
debug: winston.LeveledLogMethod;
}
From this interface, I've created an instance like so:
this.logger = <Logger>winston.createLogger({
levels: {
fatal: 0,
error: 1,
warning: 2,
status: 3,
info: 4,
debug: 5
},
level: 'debug',
transports: [
new winston.transports.Console({
}),
new winston.transports.DailyRotateFile({
filename: 'cplus-%DATE%.log',
dirname: this.logsDir,
datePattern: 'YYYY-MM-DD-HH-mm'
}),
new winston.transports.MongoDB({
db: this.dbClient
})
]
});
Basic logging to the console & file using the new methods works fine, for example
this.logger.fatal('Connection to database ' + this.dbUrl + ' failed, cannot start', err);
creates the following output:
{"message":"Connection to database mongodb://localhost:27017 failed, cannot startAuthentication failed.","name":"MongooseServerSelectionError","reason": ...
Now, how do I acchieve the rest, such as formatting and correct storing in MongoDB? I've tried a few different things using the preset Winston formats, with no success.
Best regards!

Can Winston Logger be used on the front-end for logging?

I am creating full mean stack app with
NodeJs , Angular 6 , ExpressJs and MongoDB
I have managed to create a server and its working perfectly, instead of using console.log when logging errors in my app I have decided to use Winston Logger here is what I have now
Server side
var appRoot = require('app-root-path');
var winston = require('winston');
// define the custom settings for each transport (file, console)
var options = {
file: {
level: 'info',
filename: `${appRoot}/logs/app.log`,
handleExceptions: true,
json: true,
maxsize: 5242880, // 5MB
maxFiles: 5,
colorize: false,
},
console: {
level: 'debug',
handleExceptions: true,
json: false,
colorize: true,
},
};
// instantiate a new Winston Logger with the settings defined above
const logger = winston.createLogger({
transports: [
new winston.transports.File(options.file),
new winston.transports.Console(options.console)
],
exitOnError: false, // do not exit on handled exceptions
});
// create a stream object with a 'write' function that will be used by `morgan`
logger.stream = {
write: function (message, encoding) {
// use the 'info' log level so the output will be picked up by both transports (file and console)
logger.info(message);
},
};
module.exports = logger;
Note: Winston in server side works perfectly
Client-Side
I want to use winston in my client side angular 6 app .
Example: in one of my components i have this.
import * as logger from "winston";
.........
this.activeRouter.params.subscribe((params) => {
// tslint:disable-next-line:prefer-const
let id = params['id'];
this.moviesService.getReview(id)
.subscribe(review => {
console.log(review);
this.review = review;
});
});
As you can see I am using console.log(review) , Instead of console log I would like to use Winston .
How to use Winston logger in client-side ? am newbie to all this stuff any help will be apreciated.

Yeah it is possible, however default transport for browser is very limited. I recommend to use https://www.npmjs.com/package/winston-transport-browserconsole
npm install winston-transport-browserconsole -S
It is easy to use and supports logging json objects:
import * as winston from "winston";
import BrowserConsole from 'winston-transport-browserconsole';
const level = "debug";
winston.configure({
transports: [
new BrowserConsole(
{
format: winston.format.simple(),
level,
},
),
],
});
winston.debug("DEBUG ", {a: 1, b: "two"});

Yes - it can (technically) be used in the browser. Should it be? Almost definitely not (sadly). Winston is a fantastic logger for node. But, emphasis on "for node". If you want to use it on the client, you would need to add a bunch of node polyfills in addition to winston itself, which is very large relative to other client loggers. Between winston and those polyfills you are going to significantly increase the size of your artifact. Also, just fyi webpack 5 removed those node polyfills, so you would need to add them back manually.

According to this ticket: https://github.com/winstonjs/winston/issues/287 it's almost ready for browser use? Or mostly ready? It sounds like they recently started supporting logging in a browser environment.

Loggly: Unable to receive logs from NodeJs with Winston

I configured everything regarding the documentation's page.
I get no error from my Node app, however I don't receive any log on loggy from my app.
I can't figure out where does the problem come from.
My code in config.js:
var winston = require('winston');
require('winston-loggly-bulk');
winston.add(winston.transports.Loggly, {
inputToken: "aToken",
subdomain: "aSubDomain",
tags: ["Winston-NodeJS"],
json:true
});
winston.log('info',"Hello World from Node.js!");

Your config looks fine. Did you check you are using the right token from https://YourSubdomain.loggly.com/tokens not from https://YourSubdomain.loggly.com/account/users/api/tokens? The use of both the tokens are different so this could be a reason that you are not seeing any log in your account.