I have TDD knowledge, and I've been trying to start a project in javascript applying the same principles.
I am building an API that, once hit, fires a request to an external service in order to gather some data, and once retrieved, parses it and returns the response.
So far, I've been unlucky with my saga, I've searched a lot and the most similar issue on SO I've found is this one. But I've had no success in order to apply the same solution.
My implementation on the implementation side is as follows:
//... other handlers
weather(request, response) {
//... some setup and other calls
this.externalService.get(externalURL)
.then(serviceResponse => {
this.externalResponseParser.parse(serviceResponse)
});
//... some more code
}
And on the test side:
let requester;
let mockParser;
let handler;
let promiseResolve;
let promiseReject;
beforeEach(function () {
requester = new externalRequesterService();
mockParser = sinon.createStubInstance(...);
handler = new someHandler({
externalService: requester,
externalResponseParser: mockParser
});
});
it('returns data', function () {
sinon.stub(requester, 'get').callsFake((url) => {
return new Promise((resolve, reject) => {
// so I can be able to handle when the promise gets resolved
promiseResolve = resolve;
promiseReject = reject;
});
});
handler.weather(request, response);
// assertions of what happens before externalService.get gets called - all green
promiseResolve(expectedServiceResponse);
assert.ok(mockExternalResponseParser.parse.calledOnce, 'Expected externalResponseParser.parse to have been called once');
});
In the last line of the test, it fails, even though I am calling what I am supposed to.
At some point I've added some logging, and I was able to see that the code of the then block, seems to get executed after the assertion in the test, which might be source of the problem.
I've tried to find out if there is some sort of eventually that could be used, so my assertion after resolving the promise would be something like:
assert.eventually(mockExternalResponseParser.parse.calledOnce, 'Expected externalResponseParser.parse to eventually be called once');
but no luck.
Does anyone have some clear explanation of what is missing? Many thanks in advance
P.S.- As per request, please find a stripped down version of my code here. Just run npm install, followed by npm test in order to get the same output.
Thank you all for the time spent.
I ended up finding this very good article on Medium which allowed me to solve my issue. It has a very nice explanation moving from callbacks to promises, which was exactly the scenario I had in hands.
I have updated the github repository I created on how to fix it.
If you want to TL;DR here goes just the highlight of the changes. Implementation side:
async weather(request, response) {
//...
let serviceResponse = await this.requesterService.get(weatherURL);
//...
};
And on the test side:
it('returns weather on success', async function () {
sinon.stub(requester, 'get').callsFake((_) => {
return new Promise((resolve, _) => {
resolve(expectedServiceResponse);
});
});
await handler.weather(request, response);
//...
assert.ok(mockWeatherParser.parseWeather.calledOnce, 'Expected WeatherParser.parseWeather to have been called once'); // no longer fails here
//...
});
Now keep in mind in this example, it is still synchronous. However, I evolved my API already step by step, and after migrating to this synchronous version using Promises, it was way more easier to migrate to an async version. Both in terms of testing and implementation.
If you have the same issue and need help or have questions, let me know. I'll be happy to help.
I just started trying out node.js a few days ago. I've realized that the Node is terminated whenever I have an unhandled exception in my program. This is different than the normal server container that I have been exposed to where only the Worker Thread dies when unhandled exceptions occur and the container would still be able to receive the request. This raises a few questions:
Is process.on('uncaughtException') the only effective way to guard against it?
Will process.on('uncaughtException') catch the unhandled exception during execution of asynchronous processes as well?
Is there a module that is already built (such as sending email or writing to a file) that I could leverage in the case of uncaught exceptions?
I would appreciate any pointer/article that would show me the common best practices for handling uncaught exceptions in node.js
Update: Joyent now has their own guide. The following information is more of a summary:
Safely "throwing" errors
Ideally we'd like to avoid uncaught errors as much as possible, as such, instead of literally throwing the error, we can instead safely "throw" the error using one of the following methods depending on our code architecture:
For synchronous code, if an error happens, return the error:
// Define divider as a syncrhonous function
var divideSync = function(x,y) {
// if error condition?
if ( y === 0 ) {
// "throw" the error safely by returning it
return new Error("Can't divide by zero")
}
else {
// no error occured, continue on
return x/y
}
}
// Divide 4/2
var result = divideSync(4,2)
// did an error occur?
if ( result instanceof Error ) {
// handle the error safely
console.log('4/2=err', result)
}
else {
// no error occured, continue on
console.log('4/2='+result)
}
// Divide 4/0
result = divideSync(4,0)
// did an error occur?
if ( result instanceof Error ) {
// handle the error safely
console.log('4/0=err', result)
}
else {
// no error occured, continue on
console.log('4/0='+result)
}
For callback-based (ie. asynchronous) code, the first argument of the callback is err, if an error happens err is the error, if an error doesn't happen then err is null. Any other arguments follow the err argument:
var divide = function(x,y,next) {
// if error condition?
if ( y === 0 ) {
// "throw" the error safely by calling the completion callback
// with the first argument being the error
next(new Error("Can't divide by zero"))
}
else {
// no error occured, continue on
next(null, x/y)
}
}
divide(4,2,function(err,result){
// did an error occur?
if ( err ) {
// handle the error safely
console.log('4/2=err', err)
}
else {
// no error occured, continue on
console.log('4/2='+result)
}
})
divide(4,0,function(err,result){
// did an error occur?
if ( err ) {
// handle the error safely
console.log('4/0=err', err)
}
else {
// no error occured, continue on
console.log('4/0='+result)
}
})
For eventful code, where the error may happen anywhere, instead of throwing the error, fire the error event instead:
// Definite our Divider Event Emitter
var events = require('events')
var Divider = function(){
events.EventEmitter.call(this)
}
require('util').inherits(Divider, events.EventEmitter)
// Add the divide function
Divider.prototype.divide = function(x,y){
// if error condition?
if ( y === 0 ) {
// "throw" the error safely by emitting it
var err = new Error("Can't divide by zero")
this.emit('error', err)
}
else {
// no error occured, continue on
this.emit('divided', x, y, x/y)
}
// Chain
return this;
}
// Create our divider and listen for errors
var divider = new Divider()
divider.on('error', function(err){
// handle the error safely
console.log(err)
})
divider.on('divided', function(x,y,result){
console.log(x+'/'+y+'='+result)
})
// Divide
divider.divide(4,2).divide(4,0)
Safely "catching" errors
Sometimes though, there may still be code that throws an error somewhere which can lead to an uncaught exception and a potential crash of our application if we don't catch it safely. Depending on our code architecture we can use one of the following methods to catch it:
When we know where the error is occurring, we can wrap that section in a node.js domain
var d = require('domain').create()
d.on('error', function(err){
// handle the error safely
console.log(err)
})
// catch the uncaught errors in this asynchronous or synchronous code block
d.run(function(){
// the asynchronous or synchronous code that we want to catch thrown errors on
var err = new Error('example')
throw err
})
If we know where the error is occurring is synchronous code, and for whatever reason can't use domains (perhaps old version of node), we can use the try catch statement:
// catch the uncaught errors in this synchronous code block
// try catch statements only work on synchronous code
try {
// the synchronous code that we want to catch thrown errors on
var err = new Error('example')
throw err
} catch (err) {
// handle the error safely
console.log(err)
}
However, be careful not to use try...catch in asynchronous code, as an asynchronously thrown error will not be caught:
try {
setTimeout(function(){
var err = new Error('example')
throw err
}, 1000)
}
catch (err) {
// Example error won't be caught here... crashing our app
// hence the need for domains
}
If you do want to work with try..catch in conjunction with asynchronous code, when running Node 7.4 or higher you can use async/await natively to write your asynchronous functions.
Another thing to be careful about with try...catch is the risk of wrapping your completion callback inside the try statement like so:
var divide = function(x,y,next) {
// if error condition?
if ( y === 0 ) {
// "throw" the error safely by calling the completion callback
// with the first argument being the error
next(new Error("Can't divide by zero"))
}
else {
// no error occured, continue on
next(null, x/y)
}
}
var continueElsewhere = function(err, result){
throw new Error('elsewhere has failed')
}
try {
divide(4, 2, continueElsewhere)
// ^ the execution of divide, and the execution of
// continueElsewhere will be inside the try statement
}
catch (err) {
console.log(err.stack)
// ^ will output the "unexpected" result of: elsewhere has failed
}
This gotcha is very easy to do as your code becomes more complex. As such, it is best to either use domains or to return errors to avoid (1) uncaught exceptions in asynchronous code (2) the try catch catching execution that you don't want it to. In languages that allow for proper threading instead of JavaScript's asynchronous event-machine style, this is less of an issue.
Finally, in the case where an uncaught error happens in a place that wasn't wrapped in a domain or a try catch statement, we can make our application not crash by using the uncaughtException listener (however doing so can put the application in an unknown state):
// catch the uncaught errors that weren't wrapped in a domain or try catch statement
// do not use this in modules, but only in applications, as otherwise we could have multiple of these bound
process.on('uncaughtException', function(err) {
// handle the error safely
console.log(err)
})
// the asynchronous or synchronous code that emits the otherwise uncaught error
var err = new Error('example')
throw err
Following is a summarization and curation from many different sources on this topic including code example and quotes from selected blog posts. The complete list of best practices can be found here
Best practices of Node.JS error handling
Number1: Use promises for async error handling
TL;DR: Handling async errors in callback style is probably the fastest way to hell (a.k.a the pyramid of doom). The best gift you can give to your code is using instead a reputable promise library which provides much compact and familiar code syntax like try-catch
Otherwise: Node.JS callback style, function(err, response), is a promising way to un-maintainable code due to the mix of error handling with casual code, excessive nesting and awkward coding patterns
Code example - good
doWork()
.then(doWork)
.then(doError)
.then(doWork)
.catch(errorHandler)
.then(verify);
code example anti pattern – callback style error handling
getData(someParameter, function(err, result){
if(err != null)
//do something like calling the given callback function and pass the error
getMoreData(a, function(err, result){
if(err != null)
//do something like calling the given callback function and pass the error
getMoreData(b, function(c){
getMoreData(d, function(e){
...
});
});
});
});
});
Blog quote: "We have a problem with promises"
(From the blog pouchdb, ranked 11 for the keywords "Node Promises")
"…And in fact, callbacks do something even more sinister: they deprive us of the stack, which is something we usually take for granted in programming languages. Writing code without a stack is a lot like driving a car without a brake pedal: you don’t realize how badly you need it, until you reach for it and it’s not there. The whole point of promises is to give us back the language fundamentals we lost when we went async: return, throw, and the stack. But you have to know how to use promises correctly in order to take advantage of them."
Number2: Use only the built-in Error object
TL;DR: It pretty common to see code that throws errors as string or as a custom type – this complicates the error handling logic and the interoperability between modules. Whether you reject a promise, throw exception or emit error – using Node.JS built-in Error object increases uniformity and prevents loss of error information
Otherwise: When executing some module, being uncertain which type of errors come in return – makes it much harder to reason about the coming exception and handle it. Even worth, using custom types to describe errors might lead to loss of critical error information like the stack trace!
Code example - doing it right
//throwing an Error from typical function, whether sync or async
if(!productToAdd)
throw new Error("How can I add new product when no value provided?");
//'throwing' an Error from EventEmitter
const myEmitter = new MyEmitter();
myEmitter.emit('error', new Error('whoops!'));
//'throwing' an Error from a Promise
return new promise(function (resolve, reject) {
DAL.getProduct(productToAdd.id).then((existingProduct) =>{
if(existingProduct != null)
return reject(new Error("Why fooling us and trying to add an existing product?"));
code example anti pattern
//throwing a String lacks any stack trace information and other important properties
if(!productToAdd)
throw ("How can I add new product when no value provided?");
Blog quote: "A string is not an error"
(From the blog devthought, ranked 6 for the keywords “Node.JS error object”)
"…passing a string instead of an error results in reduced interoperability between modules. It breaks contracts with APIs that might be performing instanceof Error checks, or that want to know more about the error. Error objects, as we’ll see, have very interesting properties in modern JavaScript engines besides holding the message passed to the constructor.."
Number3: Distinguish operational vs programmer errors
TL;DR: Operations errors (e.g. API received an invalid input) refer to known cases where the error impact is fully understood and can be handled thoughtfully. On the other hand, programmer error (e.g. trying to read undefined variable) refers to unknown code failures that dictate to gracefully restart the application
Otherwise: You may always restart the application when an error appear, but why letting ~5000 online users down because of a minor and predicted error (operational error)? the opposite is also not ideal – keeping the application up when unknown issue (programmer error) occurred might lead unpredicted behavior. Differentiating the two allows acting tactfully and applying a balanced approach based on the given context
Code example - doing it right
//throwing an Error from typical function, whether sync or async
if(!productToAdd)
throw new Error("How can I add new product when no value provided?");
//'throwing' an Error from EventEmitter
const myEmitter = new MyEmitter();
myEmitter.emit('error', new Error('whoops!'));
//'throwing' an Error from a Promise
return new promise(function (resolve, reject) {
DAL.getProduct(productToAdd.id).then((existingProduct) =>{
if(existingProduct != null)
return reject(new Error("Why fooling us and trying to add an existing product?"));
code example - marking an error as operational (trusted)
//marking an error object as operational
var myError = new Error("How can I add new product when no value provided?");
myError.isOperational = true;
//or if you're using some centralized error factory (see other examples at the bullet "Use only the built-in Error object")
function appError(commonType, description, isOperational) {
Error.call(this);
Error.captureStackTrace(this);
this.commonType = commonType;
this.description = description;
this.isOperational = isOperational;
};
throw new appError(errorManagement.commonErrors.InvalidInput, "Describe here what happened", true);
//error handling code within middleware
process.on('uncaughtException', function(error) {
if(!error.isOperational)
process.exit(1);
});
Blog Quote: "Otherwise you risk the state"
(From the blog debugable, ranked 3 for the keywords "Node.JS uncaught exception")
"…By the very nature of how throw works in JavaScript, there is almost never any way to safely “pick up where you left off”, without leaking references, or creating some other sort of undefined brittle state. The safest way to respond to a thrown error is to shut down the process. Of course, in a normal web server, you might have many connections open, and it is not reasonable to abruptly shut those down because an error was triggered by someone else. The better approach is to send an error response to the request that triggered the error, while letting the others finish in their normal time, and stop listening for new requests in that worker"
Number4: Handle errors centrally, through but not within middleware
TL;DR: Error handling logic such as mail to admin and logging should be encapsulated in a dedicated and centralized object that all end-points (e.g. Express middleware, cron jobs, unit-testing) call when an error comes in.
Otherwise: Not handling errors within a single place will lead to code duplication and probably to errors that are handled improperly
Code example - a typical error flow
//DAL layer, we don't handle errors here
DB.addDocument(newCustomer, (error, result) => {
if (error)
throw new Error("Great error explanation comes here", other useful parameters)
});
//API route code, we catch both sync and async errors and forward to the middleware
try {
customerService.addNew(req.body).then(function (result) {
res.status(200).json(result);
}).catch((error) => {
next(error)
});
}
catch (error) {
next(error);
}
//Error handling middleware, we delegate the handling to the centrzlied error handler
app.use(function (err, req, res, next) {
errorHandler.handleError(err).then((isOperationalError) => {
if (!isOperationalError)
next(err);
});
});
Blog quote: "Sometimes lower levels can’t do anything useful except propagate the error to their caller"
(From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”)
"…You may end up handling the same error at several levels of the stack. This happens when lower levels can’t do anything useful except propagate the error to their caller, which propagates the error to its caller, and so on. Often, only the top-level caller knows what the appropriate response is, whether that’s to retry the operation, report an error to the user, or something else. But that doesn’t mean you should try to report all errors to a single top-level callback, because that callback itself can’t know in what context the error occurred"
Number5: Document API errors using Swagger
TL;DR: Let your API callers know which errors might come in return so they can handle these thoughtfully without crashing. This is usually done with REST API documentation frameworks like Swagger
Otherwise: An API client might decide to crash and restart only because he received back an error he couldn’t understand. Note: the caller of your API might be you (very typical in a microservices environment)
Blog quote: "You have to tell your callers what errors can happen"
(From the blog Joyent, ranked 1 for the keywords “Node.JS logging”)
…We’ve talked about how to handle errors, but when you’re writing a new function, how do you deliver errors to the code that called your function? …If you don’t know what errors can happen or don’t know what they mean, then your program cannot be correct except by accident. So if you’re writing a new function, you have to tell your callers what errors can happen and what they mea
Number6: Shut the process gracefully when a stranger comes to town
TL;DR: When an unknown error occurs (a developer error, see best practice number #3)- there is uncertainty about the application healthiness. A common practice suggests restarting the process carefully using a ‘restarter’ tool like Forever and PM2
Otherwise: When an unfamiliar exception is caught, some object might be in a faulty state (e.g an event emitter which is used globally and not firing events anymore due to some internal failure) and all future requests might fail or behave crazily
Code example - deciding whether to crash
//deciding whether to crash when an uncaught exception arrives
//Assuming developers mark known operational errors with error.isOperational=true, read best practice #3
process.on('uncaughtException', function(error) {
errorManagement.handler.handleError(error);
if(!errorManagement.handler.isTrustedError(error))
process.exit(1)
});
//centralized error handler encapsulates error-handling related logic
function errorHandler(){
this.handleError = function (error) {
return logger.logError(err).then(sendMailToAdminIfCritical).then(saveInOpsQueueIfCritical).then(determineIfOperationalError);
}
this.isTrustedError = function(error)
{
return error.isOperational;
}
Blog quote: "There are three schools of thoughts on error handling"
(From the blog jsrecipes)
…There are primarily three schools of thoughts on error handling: 1. Let the application crash and restart it. 2. Handle all possible errors and never crash. 3. Balanced approach between the two
Number7: Use a mature logger to increase errors visibility
TL;DR: A set of mature logging tools like Winston, Bunyan or Log4J, will speed-up error discovery and understanding. So forget about console.log.
Otherwise: Skimming through console.logs or manually through messy text file without querying tools or a decent log viewer might keep you busy at work until late
Code example - Winston logger in action
//your centralized logger object
var logger = new winston.Logger({
level: 'info',
transports: [
new (winston.transports.Console)(),
new (winston.transports.File)({ filename: 'somefile.log' })
]
});
//custom code somewhere using the logger
logger.log('info', 'Test Log Message with some parameter %s', 'some parameter', { anything: 'This is metadata' });
Blog quote: "Lets identify a few requirements (for a logger):"
(From the blog strongblog)
…Lets identify a few requirements (for a logger):
1. Time stamp each log line. This one is pretty self explanatory – you should be able to tell when each log entry occured.
2. Logging format should be easily digestible by humans as well as machines.
3. Allows for multiple configurable destination streams. For example, you might be writing trace logs to one file but when an error is encountered, write to the same file, then into error file and send an email at the same time…
Number8: Discover errors and downtime using APM products
TL;DR: Monitoring and performance products (a.k.a APM) proactively gauge your codebase or API so they can auto-magically highlight errors, crashes and slow parts that you were missing
Otherwise: You might spend great effort on measuring API performance and downtimes, probably you’ll never be aware which are your slowest code parts under real world scenario and how these affects the UX
Blog quote: "APM products segments"
(From the blog Yoni Goldberg)
"…APM products constitutes 3 major segments:1. Website or API monitoring – external services that constantly monitor uptime and performance via HTTP requests. Can be setup in few minutes. Following are few selected contenders: Pingdom, Uptime Robot, and New Relic
2. Code instrumentation – products family which require to embed an agent within the application to benefit feature slow code detection, exceptions statistics, performance monitoring and many more. Following are few selected contenders: New Relic, App Dynamics
3. Operational intelligence dashboard – these line of products are focused on facilitating the ops team with metrics and curated content that helps to easily stay on top of application performance. This is usually involves aggregating multiple sources of information (application logs, DB logs, servers log, etc) and upfront dashboard design work. Following are few selected contenders: Datadog, Splunk"
The above is a shortened version - see here more best practices and examples
You can catch uncaught exceptions, but it's of limited use. See http://debuggable.com/posts/node-js-dealing-with-uncaught-exceptions:4c933d54-1428-443c-928d-4e1ecbdd56cb
monit, forever or upstart can be used to restart node process when it crashes. A graceful shutdown is best you can hope for (e.g. save all in-memory data in uncaught exception handler).
nodejs domains is the most up to date way of handling errors in nodejs. Domains can capture both error/other events as well as traditionally thrown objects. Domains also provide functionality for handling callbacks with an error passed as the first argument via the intercept method.
As with normal try/catch-style error handling, is is usually best to throw errors when they occur, and block out areas where you want to isolate errors from affecting the rest of the code. The way to "block out" these areas are to call domain.run with a function as a block of isolated code.
In synchronous code, the above is enough - when an error happens you either let it be thrown through, or you catch it and handle there, reverting any data you need to revert.
try {
//something
} catch(e) {
// handle data reversion
// probably log too
}
When the error happens in an asynchronous callback, you either need to be able to fully handle the rollback of data (shared state, external data like databases, etc). OR you have to set something to indicate that an exception has happened - where ever you care about that flag, you have to wait for the callback to complete.
var err = null;
var d = require('domain').create();
d.on('error', function(e) {
err = e;
// any additional error handling
}
d.run(function() { Fiber(function() {
// do stuff
var future = somethingAsynchronous();
// more stuff
future.wait(); // here we care about the error
if(err != null) {
// handle data reversion
// probably log too
}
})});
Some of that above code is ugly, but you can create patterns for yourself to make it prettier, eg:
var specialDomain = specialDomain(function() {
// do stuff
var future = somethingAsynchronous();
// more stuff
future.wait(); // here we care about the error
if(specialDomain.error()) {
// handle data reversion
// probably log too
}
}, function() { // "catch"
// any additional error handling
});
UPDATE (2013-09):
Above, I use a future that implies fibers semantics, which allow you to wait on futures in-line. This actually allows you to use traditional try-catch blocks for everything - which I find to be the best way to go. However, you can't always do this (ie in the browser)...
There are also futures that don't require fibers semantics (which then work with normal, browsery JavaScript). These can be called futures, promises, or deferreds (I'll just refer to futures from here on). Plain-old-JavaScript futures libraries allow errors to be propagated between futures. Only some of these libraries allow any thrown future to be correctly handled, so beware.
An example:
returnsAFuture().then(function() {
console.log('1')
return doSomething() // also returns a future
}).then(function() {
console.log('2')
throw Error("oops an error was thrown")
}).then(function() {
console.log('3')
}).catch(function(exception) {
console.log('handler')
// handle the exception
}).done()
This mimics a normal try-catch, even though the pieces are asynchronous. It would print:
1
2
handler
Note that it doesn't print '3' because an exception was thrown that interrupts that flow.
Take a look at bluebird promises:
https://github.com/petkaantonov/bluebird
Note that I haven't found many other libraries other than these that properly handle thrown exceptions. jQuery's deferred, for example, don't - the "fail" handler would never get the exception thrown an a 'then' handler, which in my opinion is a deal breaker.
I wrote about this recently at http://snmaynard.com/2012/12/21/node-error-handling/. A new feature of node in version 0.8 is domains and allow you to combine all the forms of error handling into one easier manage form. You can read about them in my post.
You can also use something like Bugsnag to track your uncaught exceptions and be notified via email, chatroom or have a ticket created for an uncaught exception (I am the co-founder of Bugsnag).
One instance where using a try-catch might be appropriate is when using a forEach loop. It is synchronous but at the same time you cannot just use a return statement in the inner scope. Instead a try and catch approach can be used to return an Error object in the appropriate scope. Consider:
function processArray() {
try {
[1, 2, 3].forEach(function() { throw new Error('exception'); });
} catch (e) {
return e;
}
}
It is a combination of the approaches described by #balupton above.
I would just like to add that Step.js library helps you handle exceptions by always passing it to the next step function. Therefore you can have as a last step a function that check for any errors in any of the previous steps. This approach can greatly simplify your error handling.
Below is a quote from the github page:
any exceptions thrown are caught and passed as the first argument to
the next function. As long as you don't nest callback functions inline
your main functions this prevents there from ever being any uncaught
exceptions. This is very important for long running node.JS servers
since a single uncaught exception can bring the whole server down.
Furthermore, you can use Step to control execution of scripts to have a clean up section as the last step. For example if you want to write a build script in Node and report how long it took to write, the last step can do that (rather than trying to dig out the last callback).
Catching errors has been very well discussed here, but it's worth remembering to log the errors out somewhere so you can view them and fix stuff up.
Bunyan is a popular logging framework for NodeJS - it supporst writing out to a bunch of different output places which makes it useful for local debugging, as long as you avoid console.log.
In your domain's error handler you could spit the error out to a log file.
var log = bunyan.createLogger({
name: 'myapp',
streams: [
{
level: 'error',
path: '/var/tmp/myapp-error.log' // log ERROR to this file
}
]
});
This can get time consuming if you have lots of errors and/or servers to check, so it could be worth looking into a tool like Raygun (disclaimer, I work at Raygun) to group errors together - or use them both together.
If you decided to use Raygun as a tool, it's pretty easy to setup too
var raygunClient = new raygun.Client().init({ apiKey: 'your API key' });
raygunClient.send(theError);
Crossed with using a tool like PM2 or forever, your app should be able to crash, log out what happened and reboot without any major issues.
After reading this post some time ago I was wondering if it was safe to use domains for exception handling on an api / function level. I wanted to use them to simplify exception handling code in each async function I wrote. My concern was that using a new domain for each function would introduce significant overhead. My homework seems to indicate that there is minimal overhead and that performance is actually better with domains than with try catch in some situations.
http://www.lighthouselogic.com/#/using-a-new-domain-for-each-async-function-in-node/
If you want use Services in Ubuntu(Upstart): Node as a service in Ubuntu 11.04 with upstart, monit and forever.js
getCountryRegionData: (countryName, stateName) => {
let countryData, stateData
try {
countryData = countries.find(
country => country.countryName === countryName
)
} catch (error) {
console.log(error.message)
return error.message
}
try {
stateData = countryData.regions.find(state => state.name === stateName)
} catch (error) {
console.log(error.message)
return error.message
}
return {
countryName: countryData.countryName,
countryCode: countryData.countryShortCode,
stateName: stateData.name,
stateCode: stateData.shortCode,
}
},
I am working with Node and I have a "class" that takes a directory as a parameter. It tries to create that directory and if it fails, then it throws an error:
function Config(dir) {
fs.mkdir(dir, function(err) {
if(err) throw new Error('Error', err);
}
}
My question is, is this an approved way of doing this? If I were to use a callback, then the rest of my program would have to reside in that callback, which seems odd to me.
This issue manifested itself when I tried to write a test using mocha which won't work since the exception is thrown in an async call:
it('should throw an error on a bad directory', function() {
var fn = function() {
var badConfig = new Config('/asdf');
};
assert.throws(fn, Error);
});
I've investigated domains as a way to solve the unit test issue, but that didn't seem to solve my problem (or I didn't implement them correctly).
var d = domain.create().on('error', function(err) { throw err; }
d.run(function() {
function Config(dir) {
fs.mkdir(dir, function(err) {
if(err) throw err;
}
}
});
Ultimately, I'm looking for a best practice that allows me to indicate to the application that something bad happened, and allows me to create tests for that solution.
You have three possibilities:
Using a synchronous call. As AsolBerg explained, your case suits exactly why some fs functions have their synchronous equivalent. It's ok because in your case, all your application depends on one Config instance to be loaded. but there are cases
Using a callback as constructor argument.
If constructor callback sounds really too odd for you, put your initialization code into an init() method, that takes a callback. It's a matter of personnal preference, but rather use this technic.
Last option, you can returns a Future in your init() method. There are several future libraries in NodeJS, that are an elegant alternative to callback parameter. But you can't use it in your constructor... as the constructor's return is the created object.
It sounds like in this case you might actually want to make a synchronous call (e.g. the rest of your application depends on this call being finished before proceeding). So although its normally not the way you want to think about building your node apps you could use the synchronous version mkdirSync().
http://nodejs.org/api/fs.html#fs_fs_mkdirsync_path_mode
Then if the call fails you can catch the error and return it and (probably) exit the app.
I have developed a small lib for the Dynamics CRM REST/ODATA webservice (CrmRestKit). The lib dependes on jQuery and utilizes the promise-pattern, repectivly the promise-like-pattern of jQuery.
Now I like to port this lib to bluebird and remove the jQuery dependency. But I am facing a problem because bluebird does not support the synchronous resolution of promise-objects.
Some context information:
The API of the CrmRestKit excepts an optional parameter that defines if the web-service call should be performed in sync or async mode:
CrmRestKit.Create( 'Account', { Name: "foobar" }, false ).then( function ( data ) {
....
} );
When you pass "true" or omit the last parameter, will the method created the record in sync. mode.
Sometimes it is necessary to perform a operation in sync-mode, for instance you can write JavaScript code for Dynamics CRM that is involed for the save-event of an form and in this event-handler you need to perform sync-operation for validation (e.g. validate that a certain number of child-records exist, in case the right number of records exist, cancel the save-operation and show an error message).
My problem now is the following: bluebird does not support the resolution in sync-mode. For instance when I do the following, the "then" handler is invoked in async fashion:
function print( text ){
console.log( 'print -> %s', text );
return text;
}
///
/// 'Promise.cast' cast the given value to a trusted promise.
///
function getSomeTextSimpleCast( opt_text ){
var text = opt_text || 'Some fancy text-value';
return Promise.cast( text );
}
getSomeTextSimpleCast('first').then(print);
print('second');
The output is the following:
print -> second
print -> first
I would expect that the "second" appears after the "first" because the promise is already resolved with an value. So I would assume that an then-event-handler is immediately invoked when applied on an already resolved promise-object.
When I do the same (use then on an already resolved promise) with jQuery I will have my expected result:
function jQueryResolved( opt_text ){
var text = opt_text || 'jQuery-Test Value',
dfd = new $.Deferred();
dfd.resolve(text);
// return an already resolved promise
return dfd.promise();
}
jQueryResolved('third').then(print);
print('fourth');
This will generate the following output:
print -> third
print -> fourth
Is there a way to make bluebird work in the same fashion?
Update:
The provided code was just to illustrate the problem. The idea of the lib is: Regardless of the execution-mode (sync, async) the caller will always deal with an promise-object.
Regarding "... asking the user... doesn't seems to make any sense": When you provide two methods "CreateAsync" and "CreateSync" it is also up to the user to decide how the operation is executed.
Anyway with the current implementation the default behavior (last parameter is optional) is a async execution. So 99% of the code requires a promise-object, the optional parameter is only use for the 1% cases where you simply need a sync execution. Furthermore I developed to lib for myself and I use in 99,9999% of the case the async mode but I thought it is nice to have the option to go the sync-road as you like.
But I thinks I got the point an sync method should simply return the value. For the next release (3.0) I will implement "CreateSync" and "CreateAsync".
Thanks for your input.
Update-2
My intension for the optional parameter was to ensure a consistend behavior AND prevent logic error. Assume your as a consumer of my methode "GetCurrentUserRoles" that uses lib. So the method will alway return an promise, that means you have to use the "then" method to execute code that depends on the result. So when some writes code like this, I agree it is totally wrong:
var currentUserRoels = null;
GetCurrentUserRoles().then(function(roles){
currentUserRoels = roles;
});
if( currentUserRoels.indexOf('foobar') === -1 ){
// ...
}
I agree that this code will break when the method "GetCurrentUserRoles" changes from sync to async.
But I understand that this I not a good design, because the consumer should now that he deals with an async method.
Short version: I get why you want to do that, but the answer is no.
I think the underlying question being asked is whether a completed promise should immediately run a callback, if the promise has already completed. I can think of a lot of reasons that this might happen - for example, an asynchronous save procedure that only saves data if changes were made. It may be able to detect changes from the client side in a synchronous fashion without having to go through an external resource, but if changes are detected then and only then would an asynchronous operation be required.
In other environments that have asynchronous calls, the pattern seems to be that the developer is responsible for understanding that their work might complete immediately (for example, .NET framework's implementation of the async pattern accomodates this). This is not a design problem of the framework, it's the way it's implemented.
JavaScript's developers (and many of the commenters above) seem to have a different point of view on this, insisting that if something might be asynchronous, it must always be asynchronous. Whether this is "right" or not is immaterial - according to the specification I found at https://promisesaplus.com/, item 2.2.4 states that basically no callbacks can be called until you are out of what I'll refer to as "script code" or "user code"; that is, the specification says clearly that even if the promise is completed you can't invoke the callback immediately. I've checked a few other places and they either say nothing on the topic or agree with the original source. I don't know if https://promisesaplus.com/ could be considered a definitive source of information in this regard, but no other sources that I saw disagreed with it and it seems to be the most complete.
This limitation is somewhat arbitrary and I frankly prefer the .NET perspective on this one. I'll leave it up to others to decide if they consider it "bad code" to do something that might or might not be synchronous in a way that looks asynchronous.
Your actual question is whether or not Bluebird can be configured to do the non-JavaScript behavior. Performance-wise there may be a minor benefit to doing so, and in JavaScript anything's possible if you try hard enough, but as the Promise object becomes more ubiquitous across platforms you will see a shift to using it as a native component instead of custom written polyfills or libraries. As such, whatever the answer is today, reworking a promise in Bluebird is likely to cause you problems in the future, and your code should probably not be written to depend on or provide immediate resolution of a promise.
You might think this is a problem, because there's no way to have
getSomeText('first').then(print);
print('second');
and to have getSomeText "first" printed before "second" when the resolution is synchronous.
But I think you have a logic problem.
If your getSomeText function may be synchronous or asynchronous, depending on the context, then it shouldn't impact the order of execution. You use promises to ensure it's always the same. Having a variable order of execution would likely become a bug in your application.
Use
getSomeText('first') // may be synchronous using cast or asynchronous with ajax
.then(print)
.then(function(){ print('second') });
In both cases (synchronous with cast or asynchronous resolution), you'll have the correct execution order.
Note that having a function being sometimes synchronous and sometimes not isn't a weird or unlikely case (think about cache handling, or pooling). You just have to suppose it's asynchronous, and all will be always fine.
But asking the user of the API to precise with a boolean argument if he wants the operation to be asynchronous doesn't seem to make any sense if you don't leave the realm of JavaScript (i.e. if you don't use some native code).
The point of promises is to make asynchronous code easier, i.e. closer to what you feel when using synchronous code.
You're using synchronous code. Don't make it more complicated.
function print( text ){
console.log( 'print -> %s', text );
return text;
}
function getSomeTextSimpleCast( opt_text ){
var text = opt_text || 'Some fancy text-value';
return text;
}
print(getSomeTextSimpleCast('first'));
print('second');
And that should be the end of it.
If you want to keep the same asynchronous interface even though your code is synchronous, then you have to do it all the way.
getSomeTextSimpleCast('first')
.then(print)
.then(function() { print('second'); });
then gets your code out of the normal execution flow, because it's supposed to be asynchronous. Bluebird does it the right way there. A simple explanation of what it does:
function then(fn) {
setTimeout(fn, 0);
}
Note that bluebird doesn't really do that, it's just to give you a simple example.
Try it!
then(function() {
console.log('first');
});
console.log('second');
This will output the following:
second
first
There are some good answers here already, but to sum up the crux of the matter very succinctly:
Having a promise (or other async API) that is sometimes asynchronous and sometimes synchronous is a bad thing.
You may think it's fine because the initial call to your API takes a boolean to switch off between sync/async. But what if that's buried in some wrapper code and the person using that code doesn't know about these shenanigans? They've just wound up with some unpreditable behavior through no fault of their own.
The bottom line: Don't try to do this. If you want synchronous behavior, don't return a promise.
With that, I'll leave you with this quotation from You Don't Know JS:
Another trust issue is being called "too early." In application-specific terms, this may actually involve being called before some critical task is complete. But more generally, the problem is evident in utilities that can either invoke the callback you provide now (synchronously), or later (asynchronously).
This nondeterminism around the sync-or-async behavior is almost always going to lead to very difficult to track down bugs. In some circles, the fictional insanity-inducing monster named Zalgo is used to describe the sync/async nightmares. "Don't release Zalgo!" is a common cry, and it leads to very sound advice: always invoke callbacks asynchronously, even if that's "right away" on the next turn of the event loop, so that all callbacks are predictably async.
Note: For more information on Zalgo, see Oren Golan's "Don't Release Zalgo!" (https://github.com/oren/oren.github.io/blob/master/posts/zalgo.md) and Isaac Z. Schlueter's "Designing APIs for Asynchrony" (http://blog.izs.me/post/59142742143/designing-apis-for-asynchrony).
Consider:
function result(data) {
console.log( a );
}
var a = 0;
ajax( "..pre-cached-url..", result );
a++;`
Will this code print 0 (sync callback invocation) or 1 (async callback invocation)? Depends... on the conditions.
You can see just how quickly the unpredictability of Zalgo can threaten any JS program. So the silly-sounding "never release Zalgo" is actually incredibly common and solid advice. Always be asyncing.
What about this case, also CrmFetchKit related which in latest version uses Bluebird. I have upgraded from version 1.9 that was based on jQuery. Still the old app code that uses CrmFetchKit has methods the prototypes of which I can't or won't change.
Existing App Code
CrmFetchKit.FetchWithPaginationSortingFiltering(query.join('')).then(
function (results, totalRecordCount) {
queryResult = results;
opportunities.TotalRecords = totalRecordCount;
done();
},
function err(e) {
done.fail(e);
}
);
Old CrmFetchKit implementation (a custom version of fetch())
function fetchWithPaginationSortingFiltering(fetchxml) {
var performanceIndicator_StartTime = new Date();
var dfd = $.Deferred();
fetchMore(fetchxml, true)
.then(function (result) {
LogTimeIfNeeded(performanceIndicator_StartTime, fetchxml);
dfd.resolve(result.entities, result.totalRecordCount);
})
.fail(dfd.reject);
return dfd.promise();
}
New CrmFetchKit implementation
function fetch(fetchxml) {
return fetchMore(fetchxml).then(function (result) {
return result.entities;
});
}
My problem is that the old version had the dfd.resolve(...) where I was able to pass any number of params that I need.
The new implementation just returns, the parent seems to call the callback, I can't call it directly.
I went and made a custom version of the fetch() in the new implementation
function fetchWithPaginationSortingFiltering(fetchxml) {
var thePromise = fetchMore(fetchxml).then(function (result) {
thePromise._fulfillmentHandler0(result.entities, result.totalRecordCount);
return thePromise.cancel();
//thePromise.throw();
});
return thePromise;
}
But the problem is that the callback gets called two times, once when I do it explicitly and second by the framework but it passes it one parameter only. To trick it and "tell" not to call anything because I do it explicitly I try to call .cancel() but it is ignored. I understood why but still how do you do the "dfd.resolve(result.entities, result.totalRecordCount);" in the new version with out having to changes prototypes in the app that uses this library ?
You can in fact do this, yes.
Modify the bluebird.js file (for npm: node_modules/bluebird/js/release/bluebird.js), with the following change:
[...]
target._attachExtraTrace(value);
handler = didReject;
}
- async.invoke(settler, target, {
+ settler.call(target, {
handler: domain === null ? handler
: (typeof handler === "function" &&
[...]
For more info, see here: https://github.com/stacktracejs/stacktrace.js/issues/188