We Keep Coding

Get all returning values from a transaction containing multiple queries - knex.js

I'm trying to update some rows at once.
I'm waiting for all queries to be written before executing them all in one query. However, I need some values to be returned by the query after the update. I'm using the returning() method for that.
I can't manage to get all the returned values at once as an array after the query has been executed. If I use a then() directly in the transaction function it returns the returned values one by one and my promise won't work. It also considerably increases the execution time.
How do I get all the returning values from my update request at once in a list.
return await knex.transaction(trx => {
const queries = [];
data.forEach(pair => {
const query = knex('pair')
.where('symbol', pair.s)
.update({
last_price: pair.c,
})
.returning(['id'])
.transacting(trx)
//.then(function (updated) {
// ... doing it one by one here
//})
queries.push(query);
});
return Promise.all(queries) // Once all queries are written
.then(() => {
trx.commit // We try to execute them all
.catch((e) => {
trx.rollback // And rollback in case any of them goes wrong
console.error(e)
});
})

I found a solution without using trx.commit():
async function updateDB(data) {
return await knex.transaction(async (trx) => {
const queries = [];
data.forEach(item => {
const query = knex('core_exchangepair')
.where('symbol_slug', item.s)
.transacting(trx)
.update({
last_price: item.c,
})
.returning(['id'])
queries.push(query);
});
try {
return await Promise.all(queries);
} catch (e) {
console.error(e);
return e;
}
})
}

The await after the return is unnecessary. You need to await your Promise.all before returning it, so your trx function should be async.

JavaScript Promise inside async/await function resolve final array of responses

I'm quite a newbie in JavaScript and in Promises.
I'm trying to build an array of objects that I get from an API.
To do so, I've build two functions in a file MyFile.js.
The first one returns a promise when an axios promise is resolved. It's
function get_items (url) {
return new Promise((resolve, reject) => {
let options = {
baseURL: url,
method: 'get'
}
axios(options)
.then(response => {
resolve(response.data)
})
.catch(error => {
reject(error.stack)
})
})
}
The second one looks like this:
let output = []
let next_url = 'https://some_url.com/api/data'
async function get_data () {
try {
let promise = new Promise((resolve, reject) => {
if (next_url) {
get_items(next_url)
.then(response => {
output.push(...response.results)
if (response.next) {
next_url = response.next
console.log('NEXT_URL HERE', next_url)
get_data()
} else {
console.log('else')
next_url = false
get_data()
}
})
.catch(error => {
reject(error.stack)
})
} else {
console.log('before resolve')
resolve(output)
}
})
return await promise
} catch(e) {
console.log(e)
}
}
It's where I'm grinding my teeth.
What I think I understand of this function, is that:
it's returning the value of a promise (that's what I understand return await promise is doing)
it's a recursive function. So, if there is a next_url, the function continues on. But if there is not, it gets called one last time to go into the else part where it resolves the array output which contains the results (values not state) of all the promises. At least, when I execute it, and check for my sanity checks with the console.log I wrote, it works.
So, output is filled with data and that's great.
But, when I call this function from another file MyOtherFile.js, like this:
final_output = []
MyFile.get_data()
.then(result => {
console.log('getting data')
final_output.push(...result)
})
it never gets into the then part. And when I console.log MyFile.get_data(), it's a pending promise.
So, what I would like to do, is be able to make get_data() wait for all the promises result (without using Promise.all(), to have calls in serie, not in parallel, that would be great for performances, I guess?) and then be able to retrieve that response in the then part when calling this function from anywhere else.
Keep in mind that I'm really a newbie in promises and JavaScript in general (I'm more of a Python guy).
Let me know if my question isn't clear enough.
I've been scratching my head for two days now and it feels like I'm running in circle.
Thanks for being an awesome community!

This is a bit untested
const api_url = 'https://some_url.com/api/data';
get_data(api_url).then((results) => {
console.log(results);
}).catch((error) => {
// console.error(error);
});
function get_items (url) {
const options = {
baseURL: url,
method: 'get'
};
return axios(options).then((response) => response.data);
}
async function get_data(next_url) {
const output = [];
while (next_url) {
const { results, next } = await get_items(next_url);
output.push(...results);
next_url = next;
}
return output;
}
Basically it makes things a bit neater. I suggest to look at more examples with Promises and the advantage and when to ease await/async. One thing to keep in mind, if you return a Promise, it will follow the entire then chain, and it will always return a Promise with a value of the last then.. if that makes sense :)

There are a few problems. One is that you never resolve the initial Promise unless the else block is entered. Another is that you should return the recursive get_data call every time, so that it can be properly chained with the initial Promise. You may also consider avoiding the explicit promise construction antipattern - get_items already returns a Promise, so there's no need to construct another one (same for the inside of get_items, axios calls return Promises too).
You might consider a plain while loop, reassigning the next_url string until it's falsey:
function get_items (baseURL) {
const options = {
baseURL: url,
method: 'get'
}
// return the axios call, handle errors in the consumer instead:
return axios(options)
.then(res => res.data)
}
async function get_data() {
const output = []
let next_url = 'https://some_url.com/api/data'
try {
while (next_url) {
const response = await get_items(next_url);
output.push(...response.results)
next_url = response.next;
}
} catch (e) {
// handle errors *here*, perhaps
console.log(e)
}
return output;
}
Note that .catch will result in a Promise being converted from a rejected Promise to a resolved one - you don't want to .catch everywhere, because that will make it difficult for the caller to detect errors.

Another way of doing it is to not use async at all and just recursively return a promise:
const getItems = (url) =>
axios({
baseURL: url,
method: 'get',
}).then((response) => response.data);
const getData = (initialUrl) => {
const recur = (result, nextUrl) =>
!nextUrl
? Promise.resolve(result)
: getItems(nextUrl).then((data) =>
recur(result.concat([data.results]), data.next),
);
return recur([],initialUrl)
.catch(e=>Promise.reject(e.stack));//reject with error stack
};
As CertainPerformance noted; you don't need to catch at every level, if you want getData to reject with error.stack you only need to catch it once.
However; if you had 100 next urls and 99 of them were fine but only the last one failed would you like to reject in a way that keeps the results so far so you can try again?
If you do then the code could look something like this:
const getData = (initialUrl) => {
const recur = (result, nextUrl) =>
!nextUrl
? Promise.resolve(result)
: getItems(nextUrl)
.catch(e=>Promise.reject([e,result]))//reject with error and result so far
.then((data) =>
recur(result.concat([data.results]), data.next),
);
return recur([],initialUrl);//do not catch here, just let it reject with error and result
};

Method to stop an Await from executing if there's an empty Object from Promise but shouldn't return an error and execute other tasks

So what I want to achieve is that when a function returns an empty Object from the Promise, the Await function must not be executed and the rest of the Application must carry on executing other tasks. As the object that is returned maybe not always be available but should be returned when available.
function getData(Data) : Promise<Object> {
return new Promise((resolve, reject) => {
request({
// Method
}, (err, resp, file)=> {
if (err) {
reject(err);
} else {
resolve({
// Return object infomation
});
}
});
});
}
let someData = await Promise.all(data.map(getData));
// This should have a part that ignores if getData is empty and this await function ignored.
The rest of the application should be able to run as normal. I have tried to use:
.catch(error => { });
But didn't work the way I wanted it to work

There may be a better way, but what solved my issue was to pass an empty array to the data
if (isNullOrUndefined(data)) {
data = [];
}
In this way the await function now works the way I want it to work and does not throw error:
TypeError: Cannot read property 'map' of undefined

How to call a function inside bluebirdjs promise

I am trying to get a promise function with bluebirdjs. but all attempts falling my way because maybe I don't know what I am doing...?
I want to get files locations, then download the files one after the other then push to an array.
import * as Promise from 'bluebird';
fileFunction(files){
Promise.map(files, function(file) {
// Promise.map awaits for returned promises as well.
//Promise.delay(1000);
return this.getLocation(file);
},
).then((file) => {
console.log('done')
});
}
getLocation(file){
if(file){
return this._storage.ref(file).getDownloadURL().subscribe(url =>
this.img_array.push(url)
);
}
When I call the return this.getLocation(file)... I get the following error
bluebird.js:1545 Unhandled rejection TypeError: Cannot read property 'getLocation' of undefined ....bluebird.js
Edit part of the code am using now!
fileFunction(files){
return Promise.map(files, file => {
return this.getDownloadUrl(file);
}).then(locations => {
// add these locations onto this.img_array
this.img_array.push(...locations);
console.log('done');
return locations;
});
}
getFiles(e): Promise<any>{
this.outPutFiles = e;
this.fileFunction(this.outPutFiles).then(locations => {
locations.map((arr) => arr.subscribe((files) => this.downloadUrls.push(files)));
}).catch(err => {
console.log(err);
});
}
getDownloadUrl(file){
if(file){
return this._storage.ref(file).getDownloadURL();
} else {
return Promise.reject(new Error('No file passed to getLocation'));
}
}

this.getLocation(file) does not work because you've lost the value of this because you're inside a Promise.map() callback. Remember, that every normal function call in Javascript changes the value of this unless you specifically control the value of this.
You can fix that part of the issue with a simple arrow function for your callback like this:
fileFunction(files){
return Promise.map(files, file => {
return this.getLocation(file);
}).then(locations => {
// add these locations onto this.img_array
this.img_array.push(...locations);
console.log('done');
return locations;
});
}
This assumes that this.getLocation(file) returns a promise that resolves to the location value. Are you sure it does that? It looks like there may be more to your problem than just that first error you ran into.
And, after a side conversation, you also need to fix getLocation() to return a promise that resolves to the desired URL. Looking in the firebase Javascript doc, it appears that getDownloadURL() already returns a promise that resolves to the desired URL. So, you can just return that promise and let Promise.map() manage the results for you.
getLocation(file){
if(file){
return this._storage.ref(file).getDownloadURL();
} else {
return Promise.reject(new Error("No file passed to getLocation"));
}
}
And, then you would use it all like this:
obj.fileFunction(fileArray).then(locations => {
console.log(locations);
}).catch(err => {
console.log(err);
});

my graphql server mutation return null value

I am having challenges retrieving the results of my mutation. I need to create a db record and send an email notifying to user that the registration was successful. since both the sending of the email and the db update is server side I want to do both in the same mutation. If the email message fail the db must not be updated. So I have the following Mutation:
Mutation: {
createDealer(_, params) {
console.log("params: " + JSON.stringify(params));
bcrypt.genSalt(10, function(err, salt) {
bcrypt.hash(params.dealer.password, salt, function(err, hash) {
// Store hash in your password DB.
console.log("hashed password " + params.dealer.password)
params.dealer.password = hash;
console.log("hashed password " + params.dealer.password + " Hash: " + hash);
let session = driver.session();
let query = "CREATE (d:Dealer {email:$dealer.email}) SET d += $dealer RETURN d";
let here = "here".link("mymail#example.com");
let messageObj = {
to: params.dealer.email,
subject: 'Dealer Registration',
text: `Thank you for signing up. To complete and activate your registration please click ${here}.`
}
return (sendEmail(messageObj))
.then(data => {
console.log('SendMail data....' + JSON.stringify(data));
return session.run(query, params)
})
.then(result => {
console.log('SendNeo4j data....' + JSON.stringify(result));
return result.records[0].get("d").properties
})
.catch((err) => {
console.log(err);
});
//});
});
}); // genSalt
} // Create Dealer
}, // Mutation
Even thought both actions are successful I can't seem to retrieve the results. I get 'undefined' for:
console.log('SendMail data....' + JSON.stringify(data));
while
console.log('SendNeo4j data....' + JSON.stringify(result));
does display the correct data
but graphiql returns 'null' for the mutate.
this is the graphiql mutation:
mutation CreateDealer($dealer: DealerInput!) {
createDealer(dealer: $dealer) {
email
name
}
}
with the DealerInput variables of course.
I have read where you can retrieve multiple results from a query/mutation but I am not sure how it works. Here I need both the results of the sendEmail and the db update for my Angular/apollo front-end....I would imaging graphiql knows nothing of the sendEmail but I expected it to return the properties I requested.
SendEmail:
module.exports = (message) =>
new Promise((resolve, reject) => {
const data = {
from: 'mymail#example.com',
to: message.to,
subject: message.subject,
text: message.text
};
mailgun.messages().send(data, (error) => {
if (error) {
return reject(error);
}
return resolve();
});
});
Can someone with a little more experience than I help me out here...thanks

Couple of things to fix here. Returning a Promise (or any other value) inside a callback doesn't do anything, and doing so won't let you chain additional Promises like you want. Instead, your promise gets fired off inside the callback and isn't awaited.
As a general rule of thumb, don't mix Promises and callbacks. If you absolutely have to use callbacks, always wrap the callback in a Promise (like you did inside sendMail). Luckily, most popular libraries today support both callbacks and Promises. Here's how you could refactor the code above to correctly chain all your Promises:
createDealer(_, params) {
return bcrypt.hash(params.dealer.password, 10) // note the return here!
.then(hash => {
params.dealer.password = hash
const session = driver.session()
const query = "CREATE (d:Dealer {email:$dealer.email}) SET d += $dealer RETURN d"
const here = "here".link("mymail#example.com")
const messageObj = {
to: params.dealer.email,
subject: 'Dealer Registration',
text: `Thank you for signing up. To complete and activate your registration please click ${here}.`
}
return sendEmail(messageObj) // note the return here!
}).then(data => {
return session.run(query, params) // note the return here!
}).then(result => {
result.records[0].get("d").properties // note the return here!
})
bcrypt.hash will autogenerate the salt for you if you don't pass one in -- there's no need to call two separate functions
We kick off our Promise chain with bcrypt.hash, so we need to return the Promise it returns. A resolver must return a value or a Promise that will resolve to a value, otherwise it returns null.
Inside each then, we return a Promise. This way we "chain" our Promises, allowing the final value we return in the resolver to be the value the very last Promise in the chain resolves to.
We need to also fix your sendMail function to actually return the value. You're correctly returning the new Promise inside the function, but you also need to pass the returned data object to resolve. That tells the Promise to resolve to that value.
module.exports = (message) => new Promise((resolve, reject) => {
const data = // ...etc
mailgun.messages().send(data, (error) => {
if (error) reject(error) // no need to return, it's pointless
resolve(data) // pass data to resolve
})
})
Side note: looks like the official mailgun library supports Promises.
Additionally, I would strongly encourage you to look into using async/await, especially when dealing with a long Promise chain. It's less error prone and more readable:
createDealer async (_, params) {
const hash = await bcrypt.hash(params.dealer.password)
params.dealer.password = hash
const session = driver.session()
const query = "CREATE (d:Dealer {email:$dealer.email}) SET d += $dealer RETURN d"
const here = "here".link("mymail#example.com")
const messageObj = {
to: params.dealer.email,
subject: 'Dealer Registration',
text: `Thank you for signing up. To complete and activate your registration please click ${here}.`
}
const emailResult = await sendEmail(messageObj)
const result = await session.run(query, params)
return result.records[0].get("d").properties // still need to return!
}
EDIT: With regard to catching errors, GraphQL will catch any errors thrown by your resolver, which means you can often skip using catch yourself. For example, if your mailgun request fails, it'll generate some kind of error and your query will return null for data and the error details inside of the errors array.
That may be sufficient, although 1) you may want to log your error's stack elsewhere; and 2) in production, you probably don't want to expose internal error details to the public.
That means you'll probably want to use custom errors. As a bonus, you can add some custom properties to your errors to help the client deal with them eloquently. So your code may end up looking more like this:
class DeliveryFailureError extends Error {}
DeliveryFailureError.code = 'DELIVERY_FAILURE'
DeliveryFailureError.message = 'Sorry, we could not deliver the email to your account'
try {
await mailgun.messages.create()
} catch (err) {
logger.error('Mailgun request failed:', err.stack)
throw new DeliveryFailureError()
}