We Keep Coding

Trying to make async function get data from database

So basically I am trying to make a function in javascript that fetches data from my SQLite database. This function is eventually supposed to be exported and used in another script, which will run the function with different SQL querys and send out the results to my web page.
import sqlite3 from "sqlite3";
const db = new sqlite3.Database(
"path/to/my/db.db"
);
function getContent(sqlQuery, whenloaded) {
return new Promise((resolve) => {
resolve(
db.all(sqlQuery, [], (err, rows) => {
if (err) {
console.log("Something went wrong in the db");
} else {
rows.forEach((row) => {
whenloaded(row.page_content);
});
}
})
);
});
}
async function showContent(sqlQuery) {
await getContent(sqlQuery, (result) => {
return result;
});
}
console.log(
showContent(`SELECT page_content FROM fruit WHERE page_name = "apples"`)
);
Console log being: Promise { <pending> }
Since this function seems to need to be an async one, because of the database being involved, I have tried the code above, which didn´t work. I have had the code below as a reference since I don´t have any prior experience of async functions.
function square(a, b) {
return new Promise((resolve) => {
resolve(a + b);
});
}
async function output(a, b) {
const ans = await square(a, b);
console.log(ans);
}
output(10, 20);
I´ve seen a bunch of async function-related questions on here, even some with a db involved like in my case, but it´s just not exactly like mine and I´m not clever enough to implement their solutions in my own project. :)

Mock implementation of another module not working in jest

I am unit testing (using jest) a function named "getTripDetails" (inside file trip.js) that calls another file "getTrace.js" from different module (which exports a function as shown below).
I want to mock the call of function "getTrace" while testing "getTripDetails" function.
file: trips.js
const gpsTrace = require("./gpsTrace");
getTripDetails = async(req, res)=>{
let gpsTraceRes = await gpsTrace(req.body, req.adToken)
//more code...
return {status:200};
}
file: getTrace.js
module.exports = async(payload, token) =>{
try {
//code
} catch (e) {
error(e)
throw new Error(e)
}
}
This is what i tried after reading the docs.
file: test.js
let ctrl = require("./trips");
describe("API -- testing", function () {
it("Trip details", async function () {
jest.mock('./gpsTrace');
const gpsTrace = require('./gpsTrace');
gpsTrace.mockImplementation(() => {});
gpsTrace();
await ctrl.getTripDetails({},{});
expect(response.status).to.eql(200);
});
});
It did not get mocked, instead it was calling the original implementation.
Any suggesstions?

You were pretty close! Here are the updated files with comments describing the changes:
gpsTrace.js
Added a console.log message. We won't see this in the test if the mock works successfully.
module.exports = async (payload, token) => {
try {
//code
console.log("You won't see me in the Jest test because of the mock implementation")
} catch (e) {
error(e)
throw new Error(e)
}
}
trips.js
You needed to export your code to be used in other modules. Seeing as you're calling ctrl.getTripDetails() in the test, it makes sense to export your getTripDetails() on an object at the bottom of the file.
const gpsTrace = require("./gpsTrace");
const getTripDetails = async (req, res) =>{
let gpsTraceRes = await gpsTrace(req.body, req.adToken)
//more code...
return { status:200 };
}
module.exports = {
getTripDetails,
}
gpsTrace.test.js
Make sure to import your modules at the top of the file. Remember that ctrl.getTripDetails({}, {}) calls gpsTrace internally, so no need to call it twice in your test. You also needed to save the response returned from getTripDetails into a variable to be able to compare it: const response = await ctrl.getTripDetails({}, {});.
// make sure your require statements go at the top of the module
const gpsTrace = require('./gpsTrace');
let ctrl = require("./trips");
jest.mock('./gpsTrace');
gpsTrace.mockImplementation(() => {});
describe("API -- testing", function () {
it("Trip details", async function () {
// ctrl.getTripDeals() calls your gpsTrace function internally, so no need to call it twice
// gpsTrace(); <-- can be removed
// you needed to save the returned response into a variable to be able to test it.
const response = await ctrl.getTripDetails({}, {});
expect(response.status).toEqual(200);
});
});
Result
After running the test it now successfully passes. Notice that we DO NOT see the console.log message in the gpsTrace function, which indicates our mockedImplementation of the function is working in the test script. 👍

How do I call Airtable rest API from inside an AWS Lambda?

I am trying to call my rest api endpoint in AIRTABLE from inside an AWS Lambda with no success. I get no errors, no outputs.
If I call the same code using node - it works.
I am able to use Axios in my code.
Pure airtable code (works)
var Airtable = require('airtable');
var base = new Airtable({apiKey: 'keyoMYSECRETKEY'}).base('Mybaseid');
base('MyBase').select({maxRecords: 3,view: "MyView"}).eachPage(function page(records, fetchNextPage) {
// This function (`page`) will get called for each page of records.
records.forEach(function(record) {
console.log('Retrieved',JSON.stringify(record.get('Session Information')));
});
fetchNextPage();
}, function done(err) {
if (err) { console.error(err); return; }
});
If I put it inside a Lambda handler - I get nothing.
const axios = require('axios')
const url = 'https://checkip.amazonaws.com/';
var Airtable = require('airtable');
var base = new Airtable({apiKey: 'keySECRETKEY'}).base('MYBASEID');
let response;
exports.lambdaHandler = async (event, context) => {
try {
base('MyBase').select({maxRecords: 3,view: "MyView"}).eachPage(function page(records, fetchNextPage) {
records.forEach(function(record) { //HERE - NOTHING HAPPENS
console.log('Retrieved',JSON.stringify(record.get('Session Information')));
});
fetchNextPage();
}, function done(err) {
if (err) { console.error(err); return; }
});
const ret = await axios(url); //THIS WORKS
response = {
'statusCode': 200,
'body': JSON.stringify({
message: 'hello world - boo',
location: ret.data.trim()
})
}
} catch (err) {
console.log(err);
return err;
}
return response
};
What am I missing so I can call Airtable API from inside an AWS Lambda?

It seems that your lambda terminates before the API call execution your trying to perform.
I believe this will be solved using a synchronous lambda or with a correct usage of promises with await calls.

Best way to troubleshoot this is to go back to the basics.
See if you can at least get a meaningful console log by wrapping a simpler fetch request into a lambda handler:
const baseId = 'exampleAppId123';
const tableName = 'Table 1';
const api_key = 'keyExample123';
const url = `https://api.airtable.com/v0/${baseId}/${tableName}?api_key=${api_key}`;
exports.lambdaHandler = async () => {
const res = await fetch(url)
.then(res => res.json())
.then(data=>console.log(data))
.then(() => {
//do more stuff
})
}
Then report back if you can't. Or better yet, report back either way as that's bound to help more people in the future.
Worst case? The above code still doesn't do anything. If that happens, I suggest going with #Shoty's first instinct and turning this code into a synchronous fetch request by removing the async/await syntax and returning chained thenables. Not that blocking behavior of this sort is acceptable from a UX perspective, but it should at least help with debugging.

How to make nodejs wait for mongodb connection before continuing (calling async in blocking way)

I want my nodejs app to not continue unless it connects to mongodb.
I tried:
//Mongo
async function mongoConnect(mongoDB) {
var mongoConnectionSuccessful = false;
LOG("Connecting to mongodb at " + mongoDB + "...");
while (!mongoConnectionSuccessful) {
try {
await mongoose.connect(mongoDB, { useNewUrlParser: true, useUnifiedTopology: true });
LOG("connected!");
mongoConnectionSuccessful = true;
mongoose.connection.on('error', ()=>LOG('MongoDB connection error:'));
return;
} catch (error) {
LOG(error);
}
await utils.sleep(500);
}
}
mongoConnect(config.mongoUrl);
but in order to use await in mongoose.connect, I must make mongConnect async, but I then cannot call it in a blocking way from the code because in order to call like this, I must call with await, but await is only permitted inside async functions.

I must call with await, but await is only permitted inside async functions
That's correct. So do just that:
async function main () {
await mongoConnect(config.mongoUrl);
// rest of your code...
}
main();
For example if this is an Express server do something like:
const express = require('express');
const app = express();
async function main () {
await mongoConnect(config.montoUrl);
const routes = require('./my-routes');
app.use(routes);
app.listen(config.port);
}
main();

You want to attempt reconnect if mongoose fails to connect. Here is an example logic without helper lib.Props to the guy who posted this solution in a github issue for mongoose. Here
function createConnection (dbURL, options) {
var db = mongoose.createConnection(dbURL, options);
db.on('error', function (err) {
// If first connect fails because mongod is down, try again later.
// This is only needed for first connect, not for runtime reconnects.
// See: https://github.com/Automattic/mongoose/issues/5169
if (err.message && err.message.match(/failed to connect to server .* on first connect/)) {
console.log(new Date(), String(err));
// Wait for a bit, then try to connect again
setTimeout(function () {
console.log("Retrying first connect...");
db.openUri(dbURL).catch(() => {});
// Why the empty catch?
// Well, errors thrown by db.open() will also be passed to .on('error'),
// so we can handle them there, no need to log anything in the catch here.
// But we still need this empty catch to avoid unhandled rejections.
}, 20 * 1000);
} else {
// Some other error occurred. Log it.
console.error(new Date(), String(err));
}
});
db.once('open', function () {
console.log("Connection to db established.");
});
return db;
}
// Use it like
var db = createConnection('mongodb://...', options);
and with a lib promise-retry
const promiseRetry = require('promise-retry')
const options = {
useNewUrlParser: true,
reconnectTries: 60,
reconnectInterval: 1000,
poolSize: 10,
bufferMaxEntries: 0 // If not connected, return errors immediately rather than waiting for reconnect
}
const promiseRetryOptions = {
retries: options.reconnectTries,
factor: 2,
minTimeout: options.reconnectInterval,
maxTimeout: 5000
}
const connect = () => {
return promiseRetry((retry, number) => {
logger.info(`MongoClient connecting to ${url} - retry number: ${number}`)
return MongoClient.connect(url, options).catch(retry)
}, promiseRetryOptions)
}
module.exports = { connect }

I found the solution in this article
I beleive what you need is to promise your call of mongoConnect(config.mongoUrl),
than await promises until it is called back.
async function getConcurrently() {
let promises = [];
promises.push(mongoConnect(config.mongoUrl))
// promises.push(getUsers());
// promises.push(getCategories());
// promises.push(getProducts());
let mongo = await Promise.all(promises);
//let [users, categories, products] = await Promise.all(promises);
}
Please Note this warning in the article:
As the first example, first we create an array of Promises (each one of the get functions are a Promise). Then, we execute all of them concurrently and simultaneously, awaiting for all of them to finish (await Promise.all). Finally, we assign the results to the respective variables users, categories and products. Despite the fact that it works, it’s important to say that using Promises.all() for everything is a bad idea.

I assume that for some reason you have incoming events meanwhile you are attempting to connect to mongo. My approach would be preventing any incoming events (such as starting the server and setting listeners) before connection to mongo. If it's not possible, one way to do it is to use a self-invoking function.
//Mongo
async function mongoConnect(mongoDB) {
var mongoConnectionSuccessful = false;
LOG("Connecting to mongodb at " + mongoDB + "...");
while (!mongoConnectionSuccessful) {
try {
await mongoose.connect(mongoDB, { useNewUrlParser: true, useUnifiedTopology: true });
LOG("connected!");
mongoConnectionSuccessful = true;
mongoose.connection.on('error', ()=>LOG('MongoDB connection error:'));
return;
} catch (error) {
LOG(error);
}
await utils.sleep(500);
}
}
(async function() {
// do stuff
await mongoConnect(config.mongoUrl);
// do other stuff
})();

JestJS: Async test isn't stopped

I got two problems with this jest test:
Is it possible to define the Content collection only once instead of doing that inside of the test?
I do get this error:
Jest did not exit one second after the test run has completed.
This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with --detectOpenHandles to troubleshoot this issue.
I don't see why my async code weren't stopped...
import resolvers from 'resolvers/'
import Db from 'lib/db'
const db = new Db()
describe('Resolver', () => {
let token
beforeAll(async () => {
await db.connect()
})
beforeEach(async () => {
token = 'string'
await db.dropDB()
})
afterAll(async () => {
await db.connection.close()
})
describe('articleGetContent()', () => {
test('should return dataset', async () => {
// SETUP
const Content = db.connection.collection('content')
const docs = [{
// some content...
}]
await Content.insertMany(docs)
// EXECUTE
const result = await resolvers.Query.articleGetContent({}, {
id: '123,
language: 'en'
}, {
token
})
// VERIFY
expect.assertions(1)
expect(result).toBeDefined()
})
})
})
resolver
import { articleGetContent } from '../models/article'
export default {
Query: {
articleGetContent: async (obj, { id }, { token }) => articleGetContent(id, token)
}
}
This is how my db class looks like
db.js
export default class Db {
constructor (uri, callback) {
const mongo = process.env.MONGO || 'mongodb://localhost:27017'
this.mongodb = process.env.MONGO_DB || 'testing'
this.gfs = null
this.connection = MongoClient.connect(mongo, { useNewUrlParser: true })
this.connected = false
return this
}
async connect (msg) {
if (!this.connected) {
try {
this.connection = await this.connection
this.connection = this.connection.db(this.mongodb)
this.gfs = new mongo.GridFSBucket(this.connection)
this.connected = true
} catch (err) {
console.error('mongo connection error', err)
}
}
return this
}
async disconnect () {
if (this.connected) {
try {
this.connection = await this.connection.close()
this.connected = false
} catch (err) {
console.error('mongo disconnection error', err)
}
}
}
async dropDB () {
const Content = this.connection.collection('content')
await Content.deleteMany({})
}
}

Related to the second question I hope you've found some issues on github about it.
In general, the issue is described in the debug log.
Jest works with promises, as a result, you shouldn't leave any async operations in any status except resolved.
In your case, you have your DB connection opened so you need to implement another method disconnect for your DB class, this link to docs will help you, but I guess you have it already as it's not the full db.js file ( I see some custom method dropDB. Main idea here is to have it in afterAll hook:
afterAll(() => db.disconnect());
Great example at the bottom of the page
What about the first question, it really depends on what you are doing in your method dropDB. If you're running method for dropping collection, you could store the reference to this collection somewhere outside and use it as it will automatically create the new one, but it would be great to see this method.
Additionally, your async test was created in a wrong way, you could read more here for example in my Update. You need to run this function in the beginning of the test: expect.assertions(number)
expect.assertions(number) verifies that a certain number of assertions
are called during a test. This is often useful when testing
asynchronous code, in order to make sure that assertions in a callback
actually got called.