We Keep Coding

async function on route.js file | nodejs

I have an application that it has nodejs as backend and some scripts in Python
The problem is to make the 'PythonShell' (function to access the scripts) as a async function. I do not know why but it is not working.
I'll put the code from my router.js file and inside of it I put three 'console.log('steps')' to check the sequences.
It should be Step01 > Step02 > Step03, but as it is not working, It always prints Step01 > Step03 > Step02
Everything is working fine, except for this async problem! For me it should work as it is.
How can I edit my functions to execute first the 'goToscript/PythonShell' and then execute 'res.json(responseScript)'?
Thanks
router.put("/uploads/script-03", async (req, res) => {
let options = {
scriptPath: "scripts",
args: JSON.stringify(req.body)
};
const goToScript = async () => {
await PythonShell.run("script-01.py", options, (err, res) => {
if (err) {
}
if (res) {
responseScript = JSON.parse(res)
console.log('Step 02')
}
});
}
console.log('Step 01')
goToScript()
console.log('Step 03')
res.json(responseScript)
});
module.exports = router

A couple things:
1. Your goToScript is not actually async/returning a Promise
From what I can tell, PythonShell doesn't support async, only callbacks, so you can rewrite your gotToScript like so:
const goToScript = () => {
return new Promise((resolve, reject) => {
PythonShell.run("script-01.py", options, (err, res) => {
if (err) {
reject(err)
}
if (res) {
responseScript = JSON.parse(res)
console.log('Step 02')
resolve(responseScript)
}
})
})
}
const scriptResult = await goToScript()
This code will work like a regular async function, where the promise will resolve to the parsed JSON, and reject with the error if it meets one.
2. You are not awaiting your call to goToScript
When you want to make an async call that finishes in sequence with everything else, you need to await it. Take these two examples:
In this first chunk of code, waitFn waits before 100ms before logging "Step 2!":
const waitFn = () => {
return new Promise((resolve) => {
setTimeout(() => {
console.log('Step 2!')
resolve()
}, 100)
})
}
console.log('Step 1!')
waitFn()
console.log('Step 3!')
Because you do not await the result of the Promise, your code doesn't care that is has not finished, and will print:
Step 1!
Step 3!
Step 2!
Instead, however, if you await the result of the Promise returned in waitFn, it will execute in order:
const waitFn = () => {
return new Promise((resolve) => {
setTimeout(() => {
console.log('Step 2!')
resolve()
}, 100)
})
}
console.log('Step 1!')
await waitFn() // Finishes before moving on
console.log('Step 3!')
You can read a bunch more about Promises and async/await here :)

To be able to await function - this function needs to return a promise. In the npm page of your lib there is a description, about what PythonShell.run returns. It does not return a promise. So it is asynchronous, but not awaitable, it is callback based.
All you need to do - to promisify this function. Additionaly - you need to await the call to goToScript();.
router.put("/uploads/script-03", async (req, res) => {
let options = {
scriptPath: "scripts",
args: JSON.stringify(req.body)
};
const goToScript = async () => {
return new Promise((resolve, reject) => {
PythonShell.run("script-01.py", options, (err, res) => {
console.log("Step 02");
if (err) return reject(err);
return resolve(JSON.parse(res));
});
});
};
console.log("Step 01");
const responseScript = await goToScript();
console.log("Step 03");
res.json(responseScript);
});
module.exports = router;

Upgrade .then .catch to async await and try catch

I'm tryng to upgrade this code for a better maintenance, this code uploads two images to a server, i know it's possible to get rid of those .catch, by applying async await functions, and try catch blocks, but it's pretty confusing for me, any help will be apreciated.
this._uploadService.makeFileRequest(Global.url + "/upload-image1/" + response.product._id, [], this.filesToUpload1, 'image')
.then((result: Product) => {
this.filesToUpload1 = null;
this._uploadService.makeFileRequest(Global.url + "/upload-image/" + response.product._id, [], this.filesToUpload, 'image')
.then((result: Product) => {
this.filesToUpload = null;
setTimeout( () => this._router.navigate(['/detail', this.saveProduct._id]), 800 );
})
.catch(err => {
console.log(err);
this._router.navigate(['/detail', this.saveProduct._id]);
})
})
.catch(err => {
console.log(err);
this._router.navigate(['/detail', this.saveProduct._id]);
})

I suggest using a pen and paper to draw a block diagram for the logic involved, i.e. which api gets called first, with what kind of data, then which api comes afterwards; also include any logical conditionals through branching.
After that, you should attempt to write something like
const aggregateFunction = async() => {
try {
const someResponse = await callFirstApi(); // return response
await callSecondApi(someResponse); // use the response of the first api for the second api
if (someConditional) {
await callThirdApi(); // response not returned (i.e. when not required)
}
} catch (error) { // catch all errors from all await if they're not within another try-catch
console.log(error);
}
}
This pattern should eliminate all then and catch blocks. If you need more specific error handling for calling say a specific api, wrap function call inside another try-catch block, but everything should still be within the outer try-catch so that all errors will be caught regardless.

this._uploadService.makeFileRequest = function(){
return new Promise(resolve => {
// do logic of file request
resolve(true);
})
}
var waitForTime = function() {
return new Promise(resolve => {
setTimeout( () => {
this._router.navigate(['/detail', this.saveProduct._id]),
resolve(true)
}, 800 );
})
}
var f = async function(){
try {
await this._uploadService.makeFileRequest(Global.url + "/upload-image1/" + response.product._id, [], this.filesToUpload1, 'image');
await this.fileToUpload1 = null;
await this._uploadService.makeFileRequest(Global.url + "/upload-image/" + response.product._id, [], this.filesToUpload, 'image')
await this.fileToUpload = null;
await waitForTime();
}
catch(e) {
// error logic
}
}
if (this.filesToUpload1 && this.filesToUpload) {
f()
}
this might be another cleaner approach with async,await and promise

Javascript how the better way to code nested callback?

I have 3 layer callbacks like this :
app.post('/', (req, res) => {
var filename = `outputs/${Date.now()}_output.json`;
let trainInput = req.files.trainInput;
let trainOutput = req.files.trainInput;
let testInput = req.files.trainInput;
//first
trainInput.mv(`inputs/${req.body.caseName}/train_input.csv`, function (err) {
if (err) return res.status(500).send(err);
//second
trainOutput.mv(`inputs/${req.body.caseName}/train_output.csv`, function (err) {
if (err) return res.status(500).send(err);
//third
testInput.mv(`inputs/${req.body.caseName}/test_input.csv`, function (err) {
if (err) return res.status(500).send(err);
res.send('success');
});
});
});
});
In this case, there are only 3 file uploads. In another case, I have more than 10 file uploads, and it makes 10 layer callbacks. I know it because of JavaScript asynchronous.
Is there any way, with this case, to make a beautiful code? This is because when it 10 layer callbacks, the code looks horizontally weird.
Thanks

You can use the following code to make you code look better and avoid callback hell
app.post('/', async (req, res) => {
var filename = `outputs/${Date.now()}_output.json`;
let trainInput = req.files.trainInput;
let trainOutput = req.files.trainInput;
let testInput = req.files.trainInput;
try {
var result1 = await trainInput.mv(`inputs/${req.body.caseName}/train_input.csv`);
var result2 = await trainInput.mv(`inputs/${req.body.caseName}/train_output.csv`);
var result2 = await testInput.mv(`inputs/${req.body.caseName}/test_input.csv`);
res.send('success');
}
catch (error) {
res.status(500).send(error);
}
});

You can make the functions return a Promise
I advice to make one function because you do the same thing 3 times. In this case I called the function 'save' but you can call it what ever you want. The first parameter is the file end the second the output filename.
function save(file, output) = return new Promise((resolve, reject) => {
file.mv(`inputs/${req.body.caseName}/${output}`, err =>
if (err) return reject(err)
resolve()
})
Promise.all([
save(req.files.trainInput, 'train_input.csv'),
save(req.files.trainInput, 'train_output.csv'),
save(req.files.trainInput, 'test_input.csv')
])
.then(_ => res.send(200))
.catch(err => res.send(400);

What version of Node you using? If async/await is available that cleans it up a bunch.
const moveCsv = (file, dest) => {
return new Promise((resolve, reject) => {
//third
file.mv(dest, function (err) {
if (err) reject(err);
resolve();
});
})
}
app.post('/', async(req, res) => {
try {
var filename = `outputs/${Date.now()}_output.json`;
const {
trainInput,
trainOutput,
testInput
} = req.files;
const prefix = `inputs/${req.body.caseName}`;
await moveCsv(trainInput, `${prefix}/train_input.csv`);
await moveCsv(trainOutput, `${prefix}/train_output.csv`);
await moveCsv(testInput, `${prefix}/test_input.csv`);
res.send('success');
} catch(err) {
res.status(500).send(err);
}
});
I'm also assuming here that your trainInput, trainOutput, testOutput weren't all meant to be req.files.trainInput.
Just be careful since the synchronous nature of the await calls are thread blocking. If that writer function takes ages you could also looking at putting those calls onto a worker thread. Won't really matter if your requests to that server endpoint are fast and non-frequent.

You can add RXJS to your project and use Observables.forkJoin()
Solution with Observables(assuming that trainInput.mv() returns Observable):
/* Without a selector */
var source = Rx.Observable.forkJoin(
trainInput.mv(`inputs/${req.body.caseName}/train_input.csv`),
trainInput.mv(`inputs/${req.body.caseName}/train_output.csv`),
trainInput.mv(`inputs/${req.body.caseName}/test_input.csv`)
);
var subscription = source.subscribe(
function (x) {
// On success callback
console.log('Success: %s', x);
},
function (err) {
// Error callback
console.log('Error');
},
function () {
// Completed - runs always
console.log('Completed');
});
// => Success: [result_1, result_2, result_3] or Error
// => Completed

Creating a promise for rendering templates in NodeJS [duplicate]

This question already has answers here:
How do I convert an existing callback API to promises?
(24 answers)
Closed 4 years ago.
i am trying to create a function for rendering views in NodeJS that will be called using await later on.
const getTemplate = async (template, context) => {
app.render(template, { data: context }, (error, html) => {
return html;
});
}
const data = require('./data.json');
app.get('/pdf', async (req, res) => {
const html = await getTemplate('invoice', data);
res.send(html);
});
Right now it gives me an empty response and that's probably because it exits the getTemplate function before the render callback fires, but i don't know how to modify it in an elegant way.

You need to return a promise from getTemplate. Since you want to wait for the app.render to finish, I hope the following code does what you intend to do :)
// `async` is not really necessary as you are not using `await`, but you may provide it, if you want
const getTemplate = /* async */ (template, context) => {
return new Promise((resolve, reject) => {
app.render(template, { data: context }, (error, html) => {
if (error) return reject(error);
return resolve(html);
});
});
}
const data = require('./data.json');
app.get('/pdf', async (req, res) => {
const html = await getTemplate('invoice', data);
res.send(html);
});

Is npm request asynchronous?

I can't understand Javascript asynchronous behavior.
I always thought that 'request' module would be synchronous. I used it , though, in my code, and something went absolutely wrong.
An example:
download_page = function(item) {
page = request.get( { url: 'http://test-fdfdfd.com/' + String(item) })
}
node = new App();
node.on('ready', () => {
console.log("Ready.");
Array.from(Array(3).keys()).forEach(item => download_page(item));
node.stop()
})
In that code, node should stop only after the three requests were completed. However, that didn't happen and I don't know why.
Could someone give me an explanation?

request is actually asynchronous.
You might want to return a Promise from your function, and then Promise.all all of them.
download_page = function(item) {
return new Promise((resolve, reject) => {
request.get( { url: 'http://test-fdfdfd.com/' + String(item) }, (err, data) => {
if (err) {
reject(err);
return;
}
resolve(data);
});
});
}
node = new App();
node.on('ready', () => {
console.log("Ready.");
Promise.all(Array.from(Array(3).keys()).map(item => download_page(item)));
node.stop()
})