In one of my previous posts I realised I needed to promisify (turn a callback into a new promise) a npm module to make clean error handling. Problem is, the more I started to think about the organisation of promisifing a library the more it dawned on me that it could easily become a big fat mess. How should I access the promisified function, in what object? Could I maybe replace the original library functions with the new promisified functions? Should I just create a new object for the promisified functions? Example:
const mongodb = require("mongodb");
const { MongoClient } = mongodb;
//Promisifing MongoClient.connect(), and storing it into connectPr into the original library
mongodb.MongoClient.connectPr = function (url) {
const options = { useNewUrlParser: true, useUnifiedTopology: true };
const promise = new Promise(function (resolve, reject) {
MongoClient.connect(url, options, (error, client) => {
if (error) reject(error);
else resolve(client);
});
});
return promise;
};
//Exporting whole module with added functions
module.exports = mongodb;
What I am doing here is just modifing the original library and adding the new function connectPr into it. Then I export it and use it like this in another file:
const mongodb = require("./helperMongoDB");
const { MongoClient } = mongodb;
const connectionURL = "mongodb://127.0.0.1:27017";
const databaseName = "tasker";
//Using new connectPr function
(async function () {
const client = await MongoClient.connectPr(connectionURL);
const db = client.db(databaseName);
db.collection("tasks").insertOne(
{
description: "Clean the house",
completed: true,
},
(error, result) => {
if (error) {
console.error(error);
return;
}
console.log(result.ops);
}
);
})();
As u can see I can still use all the original properties from the module and I can also use my connectPr function. This works but its nothing pretty. I am sure that if I worked in a team of people that most of them would be pretty confused on why I am not requiring the library normally and why it is hidden somwhere in a helper file. What is the standard on doing this kind of thing? Is promisifing a callback function even a practise?
Be sure to first check that the asynchronous API does not already provide a promise interface. In your question you give the example of mongodb. This node API's asynchronous methods will return a promise when the callback argument is not provided.
For instance, take the connect method. The documentation says:
Returns:
Promise if no callback passed
So in an async function body you can write:
const client = await MongoClient.connect(url, options);
In case of the Mongoose library (built on top of the Node MongoDb driver API), when using query methods, you may need to chain an .exec() call, as is discussed in the answers to What does the exec function do?.
Any asynchronous library that wants to remain on track, will have a promise API included. If really this is not the case, look at Node's util.promisify
Related
This question already has answers here:
How do I convert an existing callback API to promises?
(24 answers)
Closed 5 months ago.
I am trying to integrate into my code an async library function. It works fine as:
client.checkPermission(checkPermissionRequest, (err, response) => {
if (err || response?.permissionship !== HAS_PERMISSION) {
const e = err || new Error('You do not have access to event:', id);
res.code(404).send({ error: e });
}
});
I want to rewrite it in an "await form" and this does not work. I have been trying:
const response = await client.checkPermission(checkPermissionRequest);
When I run the await code, it fails with: Error: Incorrect arguments passed.
Why could this be happening and how could I fix this?
await doesn't magically make a callback-based API return a promise.
You can however, wrap the call in a promise:
const response = await new Promise((resolve, reject) => {
client.checkPermission(checkPermissionRequest, (err, response) => {
if (err) return reject(err);
return resolve(response);
});
});
This is called "promisification". If you're a little confused as to how resolve and reject work, maybe the examples in MDN could help with that.
In Node environments you can use the node utility util.promisify to convert the calling conventions of a node style function taking an (error, value) callback as their last argument, into a function that returns a promise without taking a callback argument:
const util = require('node:util'); // **OR**
import {promisify} from 'node:util';
client.checkPermission = util.promisify( client.checkPermission)
And then call with
const response = await client.checkPermission(checkPermissionRequest);
Core module imports
import {promisify} from 'node:util';
imports the promisify utility in a script module. From the docs
Core modules provide named exports of their public API. A default export is also provided which is the value of the CommonJS exports
I have a simple function which deletes a product entry from the database, and now I'm trying to delete the image file of that product as well. I checked the Node.js file system docs and found 2 functions which deal with that - fs.unlink(path, callback) and fs.unlinkSync(path). I understand that the first one is asynchronous and that the second one is synchronous, but I'm still not quite sure which one should I use and why.
module.exports.deleteProduct = async (req, res, next) => {
let productId = req.body.productId
try {
let product = await Product.destroy({
where: {
id: productId
}
})
res.status(200).json({
product: product
})
} catch (e) {
console.log(e)
res.status(500)
}
}
Some code and an idea for you:
As others have already said, async is better than sync, so you won't end up blocking, even though, unless your API volume is extremely high, it probably won't matter, as indicated in another answer.
You can use the fs promises API via
const fs = require('fs').promises; //es5 OR
import { promises as fs } from 'fs'; //es6
to use the async (non-blocking) API with a one-liner await.
Special note: you may not want your API request to fail if you failed to unlink the directory, as you did in fact delete the product from the database.
// make sure you are using the promise API from fs
const fs = require('fs').promises;
module.exports.deleteProduct = async (req, res, next) => {
let productId = req.body.productId
try {
let product = await Product.destroy({
where: {
id: productId
}
})
try {
await fs.unlink('the/path/to/the/product/image');
} catch {
// you may want to handle a failure to delete separately
}
res.status(200).json({product: product})
} catch (e) {
console.log(e)
res.status(500)
}
}
If your server OS is Linux or some other UNIX derivative with a local file system, both .unlinkSync() and .unlink() run quickly: the OS-level unlinking operation is designed to complete quickly and predictably. So, if you use the blocking .unlinkSync() version you won't do much harm, especially if your unlinking is infrequent.
That being said, if you can use the asynchronous version it's a good practice to do so.
It looks like you can; you can call res.status()... from within a callback or after an await.
Don't Block the event loop in Node Js
The synchronous methods blocks the event loop unnecessarily ,which affects your application performance .always use async methods ,wherever possible.
or if you want to use it with awaitoperation (pseudo sync) ,you can do something like below ,by wrapping it within promise
const fs=require("fs");
function unlinkPromise(file)
{
return new Promise((resolve,reject)=>{
fs.unlink(file,(err,data)=>{
if(err)
{
reject(err);
}
resolve(data);
})
})
}
async function data()
{
console.log(await unlinkPromise("file"));
}
I have a Meteor.method() that server side returns a promise from oracledb. Client side I have:
Meteor.call('myMethod', (error, result) => {
result.then() // err -> no .then() method?,
});
So what is result? It does not have a .then() method, so it is not a promise?
Meteor does not "send" the promise to the client.
The server returns a result value to the client (which triggers the callback) once the promise is resolved (or rejected) on the server, and not at the moment the promise is returned from the method itself (unless it is already settled when returned).
You can also use async/await to simplify the code.
Here is a blog post with more details about the using asynchronous code in methods.
Note:
The value sent from the server is serialized using EJSON. Object methods, getters, etc. are stripped from it, unless you create a custom serializer. In some cases, serialization might even fail (I think it happened with certain moment objects) and result in undefined being returned.
Meteor is not using promises by default, however, you can wrap your Meteor.calls into a promise function as below
const callWithPromise = (method, myParameters) => {
return new Promise((resolve, reject) => {
Meteor.call(method, myParameters, (err, res) => {
if (err) reject('Something went wrong');
resolve(res);
});
});
}
(async function() {
const myValue1 = await callWithPromise('myMethod1', someParameters);
const myValue2 = await callWithPromise('myMethod2', myValue1);
})();
Sample code has copied from Meteor forum.
Also, this topic gives you a better insight in taking advantages of Aysnc/Await syntax or Promises in Meteor calls.
As part of my learning of Node internals, I'm trying to add some basic functions to the Node response prototype without requiring some external libraries. It shouldn't be a difficult task, however, the response is never passed to the new functions and can never be retrieved through the this statement. This is an example to bind a template render function to the server response.
const http = require('http');
const newMethods = Object.create(http.ServerResponse.prototype);
newMethods.render = async (view, data) => {
const renderResult = await aTemplateLibray(view, data);
this.writeHead(200, {'Content-Type': 'text/html'});
this.end(renderResult);
}
// Other methods here...
// Then
Object.assign(http.ServerResponse.prototype, newMethods);
module.exports = http;
Once I use this http server, I can use the new render function, but the response is not passed to it, so an error message is thrown, something like this.writeHead is not a function.
I also tried with the Object.defineProperty method.
Object.defineProperty(http.ServerResponse.prototype, 'render',
{writable: true,
enumerable: true,
get: return this.socket.parser.incoming
});
I found some old library that returned that socket with the old __defineGetter__ method and I test it with the new form, but it doesn't work either.
The main issue is your use of the arrow function expression (=>), which has particular consequences with regards to what this points to inside the function (more on that here).
In your case, you want to use async function(...):
newMethods.render = async function(view, data) {
const renderResult = await aTemplateLibray(view, data);
this.writeHead(200, {'Content-Type': 'text/html'});
this.end(renderResult);
}
Recently I work on a new project and this project use JavaScript callbacks in nodejs. Now we use KOA but the problem happens when we try to use ES6 Generators and callbacks.
//Calback function
function load(callback){
result = null;
//Do something with xmla4js and ajax
callback(result);
return result;
}
Now in KOA I need to call load and response json to client so i use this code below :
router= require('koa-router');
app = koa();
app.use(router(app));
app.get('load',loadjson);
function *loadJson(){
var that = this;
load(function(result){
that.body = result;
});
}
but i get this error :
_http_outgoing.js:331
throw new Error('Can\'t set headers after they are sent.');
^
Error: Can't set headers after they are sent.
at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:331:11)
at Object.module.exports.set (G:\NAP\node_modules\koa\lib\response.js:396:16)
at Object.length (G:\NAP\node_modules\koa\lib\response.js:178:10)
at Object.body (G:\NAP\node_modules\koa\lib\response.js:149:19)
at Object.body (G:\NAP\node_modules\koa\node_modules\delegates\index.js:91:31)
at G:\NAP\Server\OlapServer\index.js:40:19
at G:\NAP\Server\OlapServer\OLAPSchemaProvider.js:1599:9
at _LoadCubes.xmlaRequest.success (G:\NAP\Server\OlapServer\OLAPSchemaProvider.js:1107:13)
at Object.Xmla._requestSuccess (G:\NAP\node_modules\xmla4js\src\Xmla.js:2113:50)
at Object.ajaxOptions.complete (G:\NAP\node_modules\xmla4js\src\Xmla.js:2024:34)
Just to clarify things, let's write your callback as
//Calback function
function load(callback){
setTimeout(function() {
var result = JSON.stringify({ 'my': 'json'});
callback(/* error: */ null, result);
}, 500);
}
in Koa world, this is called a thunk, meaning that it is an asynchronous function that takes only one argument: a callback with the prototype (err, res). you can check https://github.com/visionmedia/node-thunkify for a better explanation.
now you have to write your middleware with
function *loadJson(){
this.type = 'application/json';
this.body = yield load;
}
this is mainly because KOA is generator based, if your on the top of the middleware it does not support callbacks. so its not waiting for the function to finish. best solution would be to convert your function into a promise. promise works great with KOA.
I had a very similar problem using braintree (regular callbacks) and koa. Based on your code, the only change I needed to do was with the load function and how it was called.
router = require('koa-router');
app = koa();
app.use(router(app));
app.get('/load',loadjson);
function *loadJson(){
this.body = yield load;
}
// Callback function
function load(callback) {
// Prepare some data with xmla4js and ajax
whatever_inputs = {...};
final_method(whatever_inputs, callback);
}
The explanation by Jerome and Evan above is absolutely correct, and thunkify looks like a suitable process for automatically doing it.
While thunks were a nice idea, in my view a Promise is a better long-term approach. Many libraries are already moving to promises for async instead of the old node standard callback(err, data), and they're dead-simple to wrap around any async code to make a promise. Other devs will have experiences with Promises and naturally understand your code, while most would have to look up what a "thunk" is.
e.g. here I am wrapping the not-yet-promise-based jsdom up in a promise, so I can yield it in my koa generator.
const jsdom = require('node-jsdom');
const koa = require('koa');
const app = koa();
app.use(function *() {
this.body = yield new Promise((resolve, reject) => jsdom.env({
url: `http://example.org${this.url}`,
done(errors, { document }) {
if (errors) reject(errors.message);
resolve(`<html>${document.body.outerHTML}</html>`);
},
}));
});
app.listen(2112);
Semantically, promises and generators go hand-in-hand to really clarify async code. A generator can be re-entered many times and yield several values, while a promise means "I promise I'll have some data for you later". Combined, you get one of the most useful things about Koa: the ability to yield both promises and synchronous values.
edit: here's your original example wrapped with a Promise to return:
const router = require('koa-router');
const { load } = require('some-other-lib');
const app = koa();
app.use(router(app));
app.get('load', loadjson);
function* loadJson() {
this.body = yield new Promise(resolve => {
load(result => resolve(result));
});
}
To bypass Koa's built-in response handling, you may explicitly set this.respond = false;. Use this if you want to write to the raw res object instead of letting Koa handle the response for you.
Header is already written by built-in response handling before your callback is invoked.