We Keep Coding

Callback with recursive functions

I am using the google translate api to translate data from a json file to french locale and then write it back to a file. I am using a recursive function to iterate over the json file since it is deeply nested. However the execution is not waiting till the translation is completed before it writes to the file. I have tried using callback and promise approaches but i couldn't get it right.
Just for it to work as I required an output as an emergency I have set a timout before the write method is called. It work but I would like to learn the appropriate/correct approach to implement this.
const fs = require('fs')
const {Translate} = require('#google-cloud/translate').v2
require('dotenv').config()
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0
const credentials = JSON.parse(process.env.credentials)
const translate = new Translate({
credentials,
projectId: credentials.project_id,
})
let data = {}
// writeJSONTofile should be executed only after readJSONFile execution is completed
//read file
const readJSONFile = () => {
try {
data = JSON.parse(fs.readFileSync('...\\locale\\en.json'))
iterateAndTranslate(data)
setTimeout(() => {
writeJSONToFile()
}, 25000)
} catch (error) {
console.log(error)
}
}
// iterate, translate, reassign
const iterateAndTranslate = async (data) => {
for(key in data) {
if (typeof data[key] === 'object' && data[key] !== null) {
iterateAndTranslate(data[key])
} else{
data[key] = await translateText(data[key], 'fr')
}
}
}
//translate method
const translateText = async (text, targetLanguage) => {
try {
let [response] = await translate.translate(text, targetLanguage)
return response
} catch (error) {
console.log(error)
return 0
}
}
const writeJSONToFile = async () => {
var outputFileName = 'C:\\test\\test.json'
await fs.writeFileSync(outputFileName, JSON.stringify(data,null,4), (err) => {
if(err) {
console.log(err)
} else {
console.log('Done!')
}
})
}
// start from here
readJSONFile()

You have a few issues with your code.
Your functions use a global variable and mutate it instead of getting input and returning output.
timeout will cause unexpected behavior in your case.
you are using var
you have redundant async-await on the writeJSONToFile function
See my view of point about the possible solution.
const fs = require("fs");
const { Translate } = require("#google-cloud/translate").v2;
require("dotenv").config();
process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0;
const credentials = JSON.parse(process.env.credentials);
const translate = new Translate({
credentials,
projectId: credentials.project_id,
});
// writeJSONTofile should be executed only after readJSONFile execution is completed
//read file
const readJSONFile = async () => {
try {
const data = JSON.parse(fs.readFileSync("...\\locale\\en.json"));
return iterateAndTranslate(data);
} catch (error) {
console.log(error);
}
return {};
};
// iterate, translate, reassign
const iterateAndTranslate = async (data) => {
for (let key in data) {
if (typeof data[key] === "object" && data[key] !== null) {
await iterateAndTranslate(data[key]);
} else {
data[key] = await translateText(data[key], "fr");
}
}
return data;
};
//translate method
const translateText = async (text, targetLanguage) => {
try {
let [response] = await translate.translate(text, targetLanguage);
return response;
} catch (error) {
console.log(error);
}
return null;
};
const writeJSONToFile = (data) => {
let outputFileName = "C:\\test\\test.json";
fs.writeFileSync(outputFileName, JSON.stringify(data, null, 4), (err) => {
if (err) {
console.log(err);
} else {
console.log("Done!");
}
});
};
// start from here
const run = async () => {
const data = await readJSONFile();
writeJSONToFile(data);
};
run();
See more:
why not using global varible
why not using var

Factory function for mongoose transaction session on Typescript

I have a lot of function like these
currency.service.ts
async createCurrencyOnSession(createCurrencyDto: CreateCurrencyDto): Promise<ResponseCurrencyDto> {
const session = await this.currencyModel.startSession();
session.startTransaction();
try {
const currency = await this.createCurrency(createCurrencyDto, {session});
await session.commitTransaction();
return currency;
} finally {
await session.endSession();
}
}
user.service.ts
async createUserOnSession(createUserDto: CreateUserDto): Promise<ResponseUserDto> {
const session = await this.userModel.startSession();
session.startTransaction();
try {
const user = await this.createUser(createUserDto, {session});
await session.commitTransaction();
return user;
} finally {
await session.endSession();
}
}
I don't like many try catch on code so I try to edit
currency.service.ts
async createCurrencyOnSession(
createCurrencyDto: CreateCurrencyDto,
): Promise<CurrencyResponseDto> {
const session = await this.currencyModel.startSession();
session.startTransaction();
return handleSession(this.createCurrency(createCurrencyDto, { session }));
}
export const handleSession =
async (handler) => async (dto, options: ISessionOptions) => {
try {
return await handler(dto, options);
} finally {
await options.session.endSession();
}
};
I can see error on Typescript because the main function return a specific interface: CurrencyResponseDto, UserResponseDto. How can I add dynamic interface to return CurrencyResponseDto, UserResponseDto, ... on factory function.
Could you help me to make it clean or suggest a better version on problem. Thanks

can i make the async.retry method retry even on successfull queries but based on a condition

I'm studying the node.js module async,I want to find out if there is a way to change the async.retry method to retry even on successfull operations but stop based on some condition or response let's say its an api call.
According to its docs ,the function will continue trying the task on failures until it succeeds.if it succeeds it will only run only that time But how can i make it work the same on successfull operations and make it stop on some condition ?
const async = require('async');
const axios = require('axios');
const api = async () => {
const uri = 'https://jsonplaceholder.typicode.com/todos/1';
try {
const results = await axios.get(uri);
return results.data;
} catch (error) {
throw error;
}
};
const retryPolicy = async (apiMethod) => {
async.retry({ times: 3, interval: 200 }, apiMethod, function (err, result) {
// should retry untill the condition is met
if (result.data.userId == 5) {
// stop retring
}
});
};
retryPolicy(api);

Yes, You can just throw a custom error if condition is not met. Would be something like that:
const async = require('async');
const axios = require('axios');
const api = async () => {
const uri = 'https://jsonplaceholder.typicode.com/todos/1';
try {
const results = await axios.get(uri);
if(typeof result.data.userId != 'undefined' && result.data.userId == 5){ // change this condition to fit your needs
return results.data;
}else{
throw {name : "BadDataError", message : "I don't like the data I got"};
}
} catch (error) {
throw error;
}
};

I don't think this is possible.
On the async.retry documentation you can find this description:
Attempts to get a successful response from task no more than times
times before returning an error. If the task is successful, the
callback will be passed the result of the successful task. If all
attempts fail, the callback will be passed the error and result (if
any) of the final attempt.
However, using the delay function given here, you can do what you want another way:
const async = require('async');
const axios = require('axios');
const delay = (t, val) => {
return new Promise((resolve) => {
setTimeout(() => { resolve(val) }, t);
});
}
const api = async () => {
const uri = 'https://jsonplaceholder.typicode.com/todos/1';
try {
const results = await axios.get(uri);
return results.data;
} catch (error) {
throw error;
}
};
const retryPolicy = async (apiMethod) => {
const times = 3
const interval = 200
let data
for (count = 0; count < 3; count++) {
try {
data = await apiMethod()
catch(e) {
console.log(e)
await delay(interval)
continue
}
if (data.userId === 5) {
break;
}
await delay(interval)
}
// do something
};
retryPolicy(api);

How to fix "Function returned undefined, expected Promise or value"

What I am trying to accomplish in a Firebase Function:
Read from Firebase database email addresses to batch email.
Loop through each one and send out emails.
I am having an issue with closing promises I believe. These don't need to be ran in order, I just need to resolve all promises before ending.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const https = require('axios');
exports.sendEmail = functions.pubsub.topic('nightly_topic').onPublish(() => {
let emailsToSend = [];
async function getTests (){
admin.firestore()
.collection("tests")
.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
emailsToSend.push(doc.data())
});
})
.catch(function (error) {
console.error(error);
});
}
async function send (address){
let body = {
//MANDRILL INFO REMOVED
};
let endpoint = 'https://mandrillapp.com/api/1.0/messages/send-template.json';
https.post(endpoint, body)
.then((result) => {
console.log('SUCCESS');
})
.catch(function (error) {
console.error(error);
});
}
async function init() {
await getTests();
for (const email of emailsToSend) {
await send(email.address);
}
}
init();
});

You're missing return statements from your functions.
return https.post(... etc

Try this:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const https = require('axios');
exports.sendEmail = functions.pubsub.topic('nightly_topic').onPublish(() => {
let emailsToSend = [];
function getTests (){
return new Promise((resolve, reject) => {
admin.firestore()
.collection("tests")
.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
emailsToSend.push(doc.data())
});
resolve(emailsToSend);
})
.catch(function (error) {
reject(error)
});
});
}
async function send (address){
let body = {
//MANDRILL INFO REMOVED
};
let endpoint = 'https://mandrillapp.com/api/1.0/messages/send-template.json';
https.post(endpoint, body)
.then((result) => {
console.log('SUCCESS');
})
.catch(function (error) {
console.error(error);
});
}
async function init() {
const emailsToSend = await getTests();
for (const email of emailsToSend) {
await send(email.address);
}
}
init();
});
It might help you.

so you are almost there. The issue is that you are not returning anything. You should fix it by just doing:
Return Promise
// code ...
exports.sendEmail = functions.pubsub.topic('nightly_topic').onPublish(() => {
// code
return init();
}
Use async
// code ...
exports.sendEmail = functions.pubsub.topic('nightly_topic').onPublish(async () => {
// code
await init();
}
Note: an async function always returns a Promise.
Suggestion
In your code, you are sending emails one at a time. await send(email.address); This line of code waits until the email is sent, before sending the next one, which is not efficient.
My suggestion is sending all emails at the same time and return a promise that resolves when every email is sent. That should look something like this:
//change this
for (const email of emailsToSend) {
await send(email.address);
}
// --------------into this--------------------------------
//This is an array of promises
const promises = emailsToSend.map(email => {
return send(email.address);
});
await Promise.all(promises);
Hope that helps :)

Access all variables within a function even when error is thrown?

Below I have a main function. This function has many other variables declared in it and runs many functions. If an error occurs within the function I can catch this error but within my catch I do not have the value for the main function. I'm looking for a way to have access to all the variables even when an error is thrown.
export async function main ({alpha, beta, gamma}) {
let one = await doSomething(alpha)
let two = await doSomethingElse(one, beta)
return {one, two}
}
export async function store (db) {
await db.insert(data)
}
export async function usage (db, data) {
try {
let operation = await main(data)
await store (db, operation)
return operation
} catch (e) {
// here we don't have access to operation variables :(
await store(db, {}, e.message)
throw e
}
}
The only reasonable way i've found to do this is to create a class where each value in the main function.
import { get } from 'lodash'
export class Operation () {
constructor(db){
this.db = db
}
main({alpha, beta, gamma}) {
this.one = await doSomething(alpha)
this.two = await doSomethingElse(one, beta)
return {one: this.one, two: this.two}
}
init(data) {
try {
let main = await this.main(data)
await this.store(main)
return main
} catch (e) {
await this.store()
throw e
}
}
store(data = {}, errorMessage) {
if (!data.one) data.one = get(this, 'one') || null
if (!data.two) data.two = get(this, 'two') || null
if (errorMessage) data.errMessage = errorMessage
return await this.db.insert(data)
}
}
How can I have access to all the variables within a function even when an error is thrown?

The simple solution is to use var to declare a function-scoped variable instead of one that is scoped to the let block.
However, you really should just use two try/catch statements, for you are trying to catch two different errors and going to handle them differently:
try {
let operation = await main(data)
try {
await store (db, operation)
} catch(e) {
// here we do have access to the `operation` variable :)
}
return operation
} catch (e) { // failed to create operation (or the inner catch did rethrow)
await store(db, {}, e.message)
throw e
}

I created this class and wrapper function alwaysRunner. which takes two function arguments and will always provide the available variables to the secondary function.
export class AlwaysRunner {
constructor(operation, reguardless, optional = {}){
Object.assign(this, optional)
this.operation = operation.bind(this)
this.reguardless = reguardless.bind(this)
}
async init(data) {
let operation = this.operation
let reguardless = this.reguardless
let optional = this.optional
delete this.operation
delete this.reguardless
delete this.optional
try {
let main = await operation(data)
await reguardless(this)
return main
} catch (error) {
await reguardless({...this, error})
throw error
}
}
}
export async function alwaysRunner(operation, reguardless, optional) {
return await new AlwaysRunner(operation, reguardless, optional).init()
}
Here's some tests on how it works.
describe('alwaysRunner', () => {
it('should work', async () => {
let ran = [false, false]
await alwaysRunner(function () {
ran[0] = true
this.alpha = true
this.beta = true
return this
}, function (values) {
ran[1] = true
assert.equal(values.alpha, true)
assert.equal(values.beta, true)
assert.equal(values.error, undefined)
}).should.not.be.rejected
assert.deepEqual(ran, [true, true])
})
it('should work with thrown error', async () => {
let ran = [false, false]
await alwaysRunner(function () {
ran[0] = true
this.alpha = true
throw new Error('some error')
this.beta = true
}, function (values) {
ran[1] = true
assert.equal(values.alpha, true)
assert.equal(values.beta, undefined)
assert.equal(values.error, new Error('some error'))
}).should.be.rejected
assert.deepEqual(ran, [true, true])
})
})