We Keep Coding

Get all returning values from a transaction containing multiple queries - knex.js

I'm trying to update some rows at once.
I'm waiting for all queries to be written before executing them all in one query. However, I need some values to be returned by the query after the update. I'm using the returning() method for that.
I can't manage to get all the returned values at once as an array after the query has been executed. If I use a then() directly in the transaction function it returns the returned values one by one and my promise won't work. It also considerably increases the execution time.
How do I get all the returning values from my update request at once in a list.
return await knex.transaction(trx => {
const queries = [];
data.forEach(pair => {
const query = knex('pair')
.where('symbol', pair.s)
.update({
last_price: pair.c,
})
.returning(['id'])
.transacting(trx)
//.then(function (updated) {
// ... doing it one by one here
//})
queries.push(query);
});
return Promise.all(queries) // Once all queries are written
.then(() => {
trx.commit // We try to execute them all
.catch((e) => {
trx.rollback // And rollback in case any of them goes wrong
console.error(e)
});
})

I found a solution without using trx.commit():
async function updateDB(data) {
return await knex.transaction(async (trx) => {
const queries = [];
data.forEach(item => {
const query = knex('core_exchangepair')
.where('symbol_slug', item.s)
.transacting(trx)
.update({
last_price: item.c,
})
.returning(['id'])
queries.push(query);
});
try {
return await Promise.all(queries);
} catch (e) {
console.error(e);
return e;
}
})
}

The await after the return is unnecessary. You need to await your Promise.all before returning it, so your trx function should be async.

I query firestore for a document and try to add it to an array, but the array returns empty

I am trying to get a user from a document meant to hold user data. The document holds this data
First: "Josh"
Last: "Solders"
email: "example#gmail.com"
phone: "7865572525"
superAdmin: "on"
userID: "admin-1"
I run this function:
export const getAdmins = functions.https.onCall(async (data, context) => {
var admins: FirebaseFirestore.DocumentData[] = [];
admin.firestore().collection("admin").get().then((querySnapshot) => {
var c = 0;
querySnapshot.forEach((doc) => {
admins[c] = doc.data();
c++;
console.log(doc);
console.log(admins[c]);
});
});
return admins;
})
and it returns to here:
const getAdmins = firebase.functions().httpsCallable('getAdmins');
// Passing params to data object in Cloud functinon
getAdmins({}).then((results) => {
admins = results;
console.log("admins retrieved");
console.log(results);
});
The function returns and the log shows that it accessed the file, however, the array returns empty. I am not quite sure why this isn't returning properly unless the local variable admins can't be accessed once its inside get call. I don't think that makes sense, but that is the best explanation for it myself. I could use some help figuring out this issue.

The return statement may run even before your promise is resolved as it's asynchronous. Your function is async so try using await instead of Promise chaining:
export const getAdmins = functions.https.onCall(async (data, context) => {
const querySnapshot = await admin.firestore().collection("admin").get()
return querySnapshot.docs.map((doc) => doc.data())
})
Also you can use map method instead to simplify the code.

How to mutate error response from Promise.allSettled?

I'm currently using Promise.allSettled and am mutating a successful response. However, I cannot figure out how to mutate the error response. It will always return an object for each promise of:
{ status: error, reason: reasonObject }
I would like to add two properties to the object so I can track the IDs and other meta data.
This is how I'm currently adding two keys to a successful response:
const results = await Promise.allSettled(
batchedPayloads.map(async (payload, idx) => {
return { accountId: payload && payload?.accountId, tag: payload?.tagId, result: await mutation(payload) };
})
I would like to do the same to an error response, while still using async/await

Add a .catch onto the call of mutation so that if there's an error, it returns the desired object structure. Do this inside the allSettled loop. To be concise, define the object of properties to be added in advance.
const results = await Promise.allSettled(
batchedPayloads.map(payload => {
// properties to add
const obj = { accountId: payload && payload?.accountId, tag: payload?.tagId };
return mutation(payload)
.then(result => ({ result, ...obj }))
.catch((error) => {
Object.assign(error, obj);
// throw here so that allSettled shows this in the error section
throw error;
})
})
);

NodeJS with MariaDB, Cannot read property 'threadId' of undefined

I'm quite desperate, because of this annoying error. I can't find any information about that on the internet.
Actually my application works fine, but if I start multiple queries and wait for them with Promise.all(), the server crashes. But step by step:
In my index.js, I'm initializing the connection pool with 60 max. connections, and exporting the variable.
const pool = mariadb.createPool(config.mariaDB);
module.exports.pool = pool
Later I'm importing the index file as "app" and using app.pool.query(...) to query my database.
Next, I have a post request /getUsersGroups. A User can be a member of multiple groups, so I expect the response to be an array of objects. Each object contains information about the group. Moreover there is a members field inside this object. This field also is an array containing information about the members, so s.th. like that:
[
{
"groupId": 125758,
"title": "test",
"members": [
{userId: 5, name:Max, }, ...]
}, ...
]
Therefore I have a post request, which looks like that. I'm getting all group ids the user is a member. Then for each groupId, I call the method Group.groupInfo.
This returns a promise, therefore I'm collecting all promises in an array and wait for all of them, before sending the response.
router.post("/getUsersGroups", (req, res) => {
let userId = req.body.userId;
let result = []
let promises = []
Group.getGroupsByUser(userId) // this gets all group ids the user is a member
.then((item) => {
let groups = item.map(x => objectParser.parseUnderscoreToCamel(x))
for (let i = 0; i < item.length; i++) {
let groupId = item[i].groupId
promises.push( //adding promises
Group.groupInfo(groupId, userId)
.then((item) => {
result.push(item)
})
.catch((err)=>{console.log(err)}))
}
})
.then(() => {
// waiting for all promises to finish
return Promise.all(promises)
.then(() => {
res.status(200).json({groups: result})
})
})
.catch((err) => {
console.log(err)
}
})
}
So the Promise Group.groupInfo looks like this. It first selects the group information and then for each member inside the group it calls another Promise User.userInfo. The pattern is the same like in the method above. So I push the Promise to an array and wait for them to finish:
module.exports.groupInfo = (groupId, userId)=>{
let groupInfo = {}
let promises = []
return new Promise(((resolve, reject) => {
app.pool.query("SELECT* FROM Groups WHERE group_id=?", [groupId])
.then((item) =>{
groupInfo = item[0]
groupInfo["members"] = [];
return app.pool.query("SELECT user_id FROM Users_groups WHERE group_id=?", [groupId])
.then((item) =>{
let members = [] //contains all user ids
for(let i=0; i<item.length;i++){
members.push(item[i].userId)
}
members.forEach((memberId)=>{
// for each memberID call User.userInfo
promises.push(User.userInfo(userId,memberId)
.then((item)=>{
groupInfo.members.push(item)}))
}
})
})
})
.then(()=>{
// wait for all Promises
return Promise.all(promises)
})
.then(() =>{
return resolve(groupInfo)
})
.catch((err)=>{
return reject(err)
})
}))
}
User.userInfo itself, makes a query to the Database to collect the information about the user.
If I'm calling this, the server crashes and giving me this error:
activeConnections[conn.threadId] = conn;
TypeError: Cannot read property 'threadId' of undefined
at Pool.handleTaskQueue (****\node_modules\mariadb\lib\pool.js:431:30)
at _combinedTickCallback (internal/process/next_tick.js:132:7)
at process._tickCallback (internal/process/next_tick.js:181:9)
I guess the problem is somehow with the Promise.all() right?
But I cannot find any information about this error. I'm thankful for every hint!
My solution was not to use Promise.all but to call the Promises sequentially, but seems not right to me.
return groupsId.reduce((promise, groupId) => {
return promise.then(() => {
return Group.groupInfo(groupId, userId)
.then((item)=>{
result.push(item)
console.log(result)
})
.catch((err)=>{
throw err
})
})
}, Promise.resolve())
I cannot rely on this, as long as I don't understand the error.
Thanks!

How do I skip a callback when running .map in JavaScript/Node.js?

Kind of a sequel to this question, I need to accept multiple objects in a POST request and then for each object process it, save it, and then return the saved object to the frontend (so that the client can see which columns were successfully edited).
When I use .map, it does save to the database and I can confirm this. However, I have two problems:
It does not execute res.locals.retval.addData(dtoObject); correctly, and my returning payload has no data transfer objects inside of it.
My object validation cannot be done inside of the callback of map. I initially tried reduce, but that didn't work at all and just saved all the same values to each database object. How can I exclude invalid JSON objects while I'm mapping them?
var jsonObjects = req.body;
//for (var n in req.body) {
var promises = jsonObjects.map((jsonObject) => {
var transform = new Transform();
// VALIDATION OF jsonObject VARIABLE IS HERE
if (jsonObject.id == 0) {
var databaseObject = Database.getInstance().getModel(objectName).build(jsonObject);
transform.setNew(true);
transform.setJsonObject(jsonObject);
transform.setDatabaseObject(databaseObject);
transform.baseExtract()
.then(() => transform.extract())
.then(() => transform.clean())
.then(() => transform.getDatabaseObject().save())
.then(function(data) {
// PROCESSING DATA
}).catch((e) => {
// ERROR
});
} else {
var queryParameters = {
where: {id: jsonObject.id}
};
console.log("Query parameters: ");
console.log(queryParameters);
Database.getInstance().getModel(objectName).findOne(queryParameters).then((databaseObject) => {
transform.setJsonObject(jsonObject);
transform.setDatabaseObject(databaseObject);
})
.then(() => transform.baseExtract())
.then(() => transform.extract())
.then(() => transform.clean())
.then(() => transform.getDatabaseObject().save())
.then((data) => {
// PROCESSING DATA
}).catch((e) => {
// ERROR
});
}
});
Promise.all(promises)
.then((results) => {
return next();
}).catch((e) => {
throw e;
});
Here's the resulting payload:
{
"errors": [],
"warnings": [],
"data": []
}

As #KevinB said in the comments, you are missing the return calls inside of your arrow functions so the database saves are going through because they are part of the Promise chain, but pushes to the response are stalled waiting for the return, and then the Express.js call resolves before the Promises do. Add return Database.getInstance() and return transform.baseExtract() to your code to fix this.
Use Array.prototype.filter() to remove elements you want to ignore since you won't ever need to execute Promises on them, then call Array.prototype.map() on the resulting array. If you don't want to use the arrow functions, you can specify this as a parameter to filter and map:
jsonObjects.filter(function(jsonObject) {
}, this);
var promises = jsonObjects.map(function(jsonObject) {
}, this);