I am trying to take this code, split it into functions across my MVC website, and make it work with my existing database and mongoose implementation. I cloned the repo and that code all works.
So far I have adapted my database connection to this:
const connectDB = async () => {
try {
const conn = mongoose.connection;
// check connection
conn.on("error", (err) => {
console.error(`connection error: ${err.message}`);
});
conn.once("open", () => {
console.log("MongoDB Connected");
});
// init gfs
let gfs;
conn.once("open", () => {
// init stream
gfs = new mongoose.mongo.GridFSBucket(mongoose.connection.db, {
bucketName: "assets",
});
});
// connection
await mongoose.connect(process.env.MONGO_URI);
return gfs;
} catch (err) {
console.error(err.message);
process.exit(1);
}
};
I'm assuming it's okay to call mongoose.connect() multiple times as it returns a singleton, and won't create multiple connections. So I'm calling it in my controller to get access to my bucket, so I can call .find on it and get data out of it. Is there a better way?
The main issue is that my bucket exists in my controller file inside the gfs variable, since when I print it, it shows this:
[0] Promise {
[0] GridFSBucket {
[0] _events: [Object: null prototype] {},
[0] _eventsCount: 0,
[0] _maxListeners: 0,
[0] s: {
[0] db: [Db],
[0] options: [Object],
[0] _chunksCollection: [Collection],
[0] _filesCollection: [Collection],
[0] checkedIndexes: false,
[0] calledOpenUploadStream: false
[0] },
[0] [Symbol(kCapture)]: false
[0] }
[0] }
and yet when a few lines down, gfs.find() is called, an error is logged:
TypeError: gfs.find is not a function
I barely know what I'm doing, so any advice would be appreciated.
Thanks
------- update
My controller code is:
const gfs = require("../config/db").connectDB();
const uploadFile = (req, res) => {
return res.json({
message: "File uploaded successfully: " + req.file.filename,
});
};
const getMyFilenames = async (req, res) => {
console.log(gfs);
try {
const files = await gfs
.find({
//"metadata.uploader": req.user._id
})
.toArray();
if (!files || files.length === 0) {
return res.status(404).json({
message: "No files available",
});
}
const filenames = files.map((file) => file.filename.split("-")[1]);
console.log(filenames);
return res.json(filenames);
} catch (err) {
console.error(err);
}
};
u can put gfs to global
for ex
let gfs
const connectDB = async () => {
try {
const conn = mongoose.connection;
conn.on("error", (err) => {
console.error(`connection error: ${err.message}`);
});
conn.once("open", () => {
console.log("MongoDB Connected");
gfs = new mongoose.mongo.GridFSBucket(mongoose.connection.db, {
bucketName: "assets",
});
});
await mongoose.connect(process.env.MONGO_URI);
} catch (err) {
console.error(err.message);
process.exit(1);
}
};
const customF = () => {
if (!gfs) return;
console.log(gfs)
// gfs.find()
}
await connectDB()
setTimeout(()=>{
customf() // it was console.log(gfs) here
}, 10000) // i think 10s is enough to connect DB
As you saw when you printed gfs in the controller, connectDB is an async function which will return a promise, in this case resolving to the value of gfs defined in that function once it finishes executing. That TypeError is because find is not a method of a promise, so you need to await the promise first to unwrap it before calling .find.
There are multiple ways to do this, including:
Doing it inline - (await gfs).find… (and calling it something like gfsPromise could help make that clearer)
Importing the function instead, and calling it inline in place of the variable
Importing the function, and then initializing the variable in the controller with await, something like
const gfs = await connectDB();
…gfs.find(…
Not knowing the libraries, I’m not sure which you would need for your purpose around singletons and such.
Also to note, but connectDB could return/resolve the promise after mongoose.connect resolves but before the callback in your once open handler is run, and thus resolving the promise to undefined without waiting for gfs to actually be set. So you’ll likely want to actually construct the Promise to return, with the construction and resolve call lines starting something like:
const gfsPromise = new Promise(…
…resolve(new mongoose.mongo…
Hope that helps and gives you enough to go off of/make how you would want!
Related
I'm trying to export a variable from a file, but when I require it in another I get undefined. I suspect my functions are causing it, but I'm not sure how to fix it.
Index.js:
app.post("/", (req,res) =>{
console.log("Get from /");
module.exports.SimpleMessage = 'Hello world'; //exporting variable
exec('npx hardhat run scripts/deploy.js --network goerli',
(error, stdout, stderr) => {
if (error !== null) {
console.log(`exec error: ${error}`);
}
});
res.send("Server received the request");
});
// starting the server
app.listen(3000, () => {
console.log('listening on port 3000');
});
Deploy.js:
async function main() {
const HelloWorld = await ethers.getContractFactory("HelloWorld");
var msg = require('../src/index.js'); //requiring variable
console.log(msg.SimpleMessage);
const hello_world = await HelloWorld.deploy();
}
(Edit) This is the whole Deploy.js function:
async function main() {
const HelloWorld = await ethers.getContractFactory("HelloWorld");
var msg = require('../src/index.js'); //requiring variable
console.log(msg.SimpleMessage);
const hello_world = await HelloWorld.deploy();
}
main()
.then(() => process.exit(0))
.catch(error => {
console.error(error);
process.exit(1);
});
so there are a few things to unwrap over here:
your require() is probably invoked before module.exports.SimpleMessage = 'Hello world'; - an easy issue to encounter when you deal with async functions
require() is implemented in such a way that if you call it twice with the same file, the second time around only the cached value is returned and so, if the value of SimpleMessage changes, you will not be able to import it any more. Instead only the original value will be returned.
A quick and simple solution is to export a value that will not change instead, like a function
let SimpleMessage;
module.exports.getSimpleMessage = () => SimpleMessage;
app.post("/", (req,res) =>{
console.log("Get from /");
SimpleMessage = 'Hello world';
// and the rest of your logic
});
at this point you can reference getSimpleMessage that will be cached by require() but still will always return the updated version of SimpleMessage
I have been getting terribly confused with how to test my controller functions. I realize that I have to mock my dependencies, request, response, and the controller functions. Here's what I have so far:
OrdersController.js
const OrderService = require('../services/orderServices')
module.exports = class OrdersController {
static async apiGetOrders(req, res, next) {
try {
const orders = await OrderService.getOrders()
return res.status(200).json(orders)
} catch (error) {
return res.status(500).json({ error: 'Unable to get orders' }) // 500, Internal Service Error, generic
}
}
static async apiPostOrder(req, res, next) {
// All good, create an orderDocument
try {
const orderDocument = {
_id: null, // undefined at this point Mongo creates this _id for us
orderId: req.body.orderId,
cookies: req.body.cookies,
daySelected: req.body.daySelected,
timeSelected: req.body.timeSelected,
userInfo: req.body.userInfo,
createdAt: new Date(),
}
await OrderService.addOrder(orderDocument)
return res.status(201).send('success') // status OK, something was Created
} catch (error) {
return res.status(500).json({ error }) // 500, Internal Server Error
}
}
OrdersController.spec.js
import OrderService from '../services/orderServices'
import { mockOrder, mockOrders } from '../mocks/fixtures'
import OrdersController from '../controllers/ordersController'
jest.mock('../controllers/ordersController.js')
const mockRequest = () => {
return {}
}
const mockResponse = (mockOrders) => {
const res = {};
res.status = jest.fn().mockReturnValue(200);
res.json = jest.fn().mockReturnValue(mockOrders);
return res;
}
// #3 Test the OrdersControllers
// mock dependencies: req, res, and spyOn the controller functions
describe('Orders Controller', () => {
test('[Positive], should call OrderService.getOrders and receive status 200', async () => {
jest.spyOn(OrdersController, 'apiGetOrders')
const req = mockRequest()
const res = mockResponse(mockOrders)
await OrdersController.apiGetOrders(req, res)
expect(res.status).toHaveBeenCalledWith(200)
expect(res.json()).toEqual(mockOrders)
})
test('[Negative], error yields status 500', async () => {
jest.spyOn(OrdersController, 'apiGetOrders')
const req = mockRequest()
const res = mockResponse({status: 500, error: 'Unable to get orders'})
await OrdersController.apiGetOrders(req, res)
expect(res.status).toHaveBeenCalledWith(500)
expect(res.json()).toEqual(error)
})
})
I'm trying to test the happy path and the negative path on the get request. I followed this expample, https://codewithhugo.com/express-request-response-mocking/, and read all of the jest docs, https://jestjs.io/docs/mock-functions. The error that I receive is:
Questions:
Am I actually writing the tests correctly?
Am I also supposed to mock the OrderService?
Should I use Sinon or is Jest more than sufficient?
I am also new to Jest and am struggling with finding documentation that goes to enough detail to suggest to me what I'm doing wrong. But in your case, I think you might be spying on the wrong thing. The OrdersController is the subject of the test, so I don't believe that you should be mocking that. Rather you should spy on OrdersController's dependency, Orderservice and mock its methods.
Also, its not clear to me why you have next in
static async apiGetOrders(req, res, next)
You don't seem to use it in the body of the method anywhere, so hanging any testing off that value, probably won't work either.
I want my nodejs app to not continue unless it connects to mongodb.
I tried:
//Mongo
async function mongoConnect(mongoDB) {
var mongoConnectionSuccessful = false;
LOG("Connecting to mongodb at " + mongoDB + "...");
while (!mongoConnectionSuccessful) {
try {
await mongoose.connect(mongoDB, { useNewUrlParser: true, useUnifiedTopology: true });
LOG("connected!");
mongoConnectionSuccessful = true;
mongoose.connection.on('error', ()=>LOG('MongoDB connection error:'));
return;
} catch (error) {
LOG(error);
}
await utils.sleep(500);
}
}
mongoConnect(config.mongoUrl);
but in order to use await in mongoose.connect, I must make mongConnect async, but I then cannot call it in a blocking way from the code because in order to call like this, I must call with await, but await is only permitted inside async functions.
I must call with await, but await is only permitted inside async functions
That's correct. So do just that:
async function main () {
await mongoConnect(config.mongoUrl);
// rest of your code...
}
main();
For example if this is an Express server do something like:
const express = require('express');
const app = express();
async function main () {
await mongoConnect(config.montoUrl);
const routes = require('./my-routes');
app.use(routes);
app.listen(config.port);
}
main();
You want to attempt reconnect if mongoose fails to connect. Here is an example logic without helper lib.Props to the guy who posted this solution in a github issue for mongoose. Here
function createConnection (dbURL, options) {
var db = mongoose.createConnection(dbURL, options);
db.on('error', function (err) {
// If first connect fails because mongod is down, try again later.
// This is only needed for first connect, not for runtime reconnects.
// See: https://github.com/Automattic/mongoose/issues/5169
if (err.message && err.message.match(/failed to connect to server .* on first connect/)) {
console.log(new Date(), String(err));
// Wait for a bit, then try to connect again
setTimeout(function () {
console.log("Retrying first connect...");
db.openUri(dbURL).catch(() => {});
// Why the empty catch?
// Well, errors thrown by db.open() will also be passed to .on('error'),
// so we can handle them there, no need to log anything in the catch here.
// But we still need this empty catch to avoid unhandled rejections.
}, 20 * 1000);
} else {
// Some other error occurred. Log it.
console.error(new Date(), String(err));
}
});
db.once('open', function () {
console.log("Connection to db established.");
});
return db;
}
// Use it like
var db = createConnection('mongodb://...', options);
and with a lib promise-retry
const promiseRetry = require('promise-retry')
const options = {
useNewUrlParser: true,
reconnectTries: 60,
reconnectInterval: 1000,
poolSize: 10,
bufferMaxEntries: 0 // If not connected, return errors immediately rather than waiting for reconnect
}
const promiseRetryOptions = {
retries: options.reconnectTries,
factor: 2,
minTimeout: options.reconnectInterval,
maxTimeout: 5000
}
const connect = () => {
return promiseRetry((retry, number) => {
logger.info(`MongoClient connecting to ${url} - retry number: ${number}`)
return MongoClient.connect(url, options).catch(retry)
}, promiseRetryOptions)
}
module.exports = { connect }
I found the solution in this article
I beleive what you need is to promise your call of mongoConnect(config.mongoUrl),
than await promises until it is called back.
async function getConcurrently() {
let promises = [];
promises.push(mongoConnect(config.mongoUrl))
// promises.push(getUsers());
// promises.push(getCategories());
// promises.push(getProducts());
let mongo = await Promise.all(promises);
//let [users, categories, products] = await Promise.all(promises);
}
Please Note this warning in the article:
As the first example, first we create an array of Promises (each one of the get functions are a Promise). Then, we execute all of them concurrently and simultaneously, awaiting for all of them to finish (await Promise.all). Finally, we assign the results to the respective variables users, categories and products. Despite the fact that it works, it’s important to say that using Promises.all() for everything is a bad idea.
I assume that for some reason you have incoming events meanwhile you are attempting to connect to mongo. My approach would be preventing any incoming events (such as starting the server and setting listeners) before connection to mongo. If it's not possible, one way to do it is to use a self-invoking function.
//Mongo
async function mongoConnect(mongoDB) {
var mongoConnectionSuccessful = false;
LOG("Connecting to mongodb at " + mongoDB + "...");
while (!mongoConnectionSuccessful) {
try {
await mongoose.connect(mongoDB, { useNewUrlParser: true, useUnifiedTopology: true });
LOG("connected!");
mongoConnectionSuccessful = true;
mongoose.connection.on('error', ()=>LOG('MongoDB connection error:'));
return;
} catch (error) {
LOG(error);
}
await utils.sleep(500);
}
}
(async function() {
// do stuff
await mongoConnect(config.mongoUrl);
// do other stuff
})();
I got two problems with this jest test:
Is it possible to define the Content collection only once instead of doing that inside of the test?
I do get this error:
Jest did not exit one second after the test run has completed.
This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with --detectOpenHandles to troubleshoot this issue.
I don't see why my async code weren't stopped...
import resolvers from 'resolvers/'
import Db from 'lib/db'
const db = new Db()
describe('Resolver', () => {
let token
beforeAll(async () => {
await db.connect()
})
beforeEach(async () => {
token = 'string'
await db.dropDB()
})
afterAll(async () => {
await db.connection.close()
})
describe('articleGetContent()', () => {
test('should return dataset', async () => {
// SETUP
const Content = db.connection.collection('content')
const docs = [{
// some content...
}]
await Content.insertMany(docs)
// EXECUTE
const result = await resolvers.Query.articleGetContent({}, {
id: '123,
language: 'en'
}, {
token
})
// VERIFY
expect.assertions(1)
expect(result).toBeDefined()
})
})
})
resolver
import { articleGetContent } from '../models/article'
export default {
Query: {
articleGetContent: async (obj, { id }, { token }) => articleGetContent(id, token)
}
}
This is how my db class looks like
db.js
export default class Db {
constructor (uri, callback) {
const mongo = process.env.MONGO || 'mongodb://localhost:27017'
this.mongodb = process.env.MONGO_DB || 'testing'
this.gfs = null
this.connection = MongoClient.connect(mongo, { useNewUrlParser: true })
this.connected = false
return this
}
async connect (msg) {
if (!this.connected) {
try {
this.connection = await this.connection
this.connection = this.connection.db(this.mongodb)
this.gfs = new mongo.GridFSBucket(this.connection)
this.connected = true
} catch (err) {
console.error('mongo connection error', err)
}
}
return this
}
async disconnect () {
if (this.connected) {
try {
this.connection = await this.connection.close()
this.connected = false
} catch (err) {
console.error('mongo disconnection error', err)
}
}
}
async dropDB () {
const Content = this.connection.collection('content')
await Content.deleteMany({})
}
}
Related to the second question I hope you've found some issues on github about it.
In general, the issue is described in the debug log.
Jest works with promises, as a result, you shouldn't leave any async operations in any status except resolved.
In your case, you have your DB connection opened so you need to implement another method disconnect for your DB class, this link to docs will help you, but I guess you have it already as it's not the full db.js file ( I see some custom method dropDB. Main idea here is to have it in afterAll hook:
afterAll(() => db.disconnect());
Great example at the bottom of the page
What about the first question, it really depends on what you are doing in your method dropDB. If you're running method for dropping collection, you could store the reference to this collection somewhere outside and use it as it will automatically create the new one, but it would be great to see this method.
Additionally, your async test was created in a wrong way, you could read more here for example in my Update. You need to run this function in the beginning of the test: expect.assertions(number)
expect.assertions(number) verifies that a certain number of assertions
are called during a test. This is often useful when testing
asynchronous code, in order to make sure that assertions in a callback
actually got called.
I'm trying to build some test helper functions that will help me to run some hooks with Mocha to test my GraphQL queries(just that you understand the context).
I want to do the following steps each time before I run the tests:
Connect to mongoDB with mongoose
Start the server (node)
Add some test data directly into the database (via mongoose models)
// this function returns back the server instance because I need it in the 'after' hook to be able to stop it after the tests execution.
export const startServer = () => {
mongoose.Promise = global.Promise;
mongoose.connect(MONGO_URI_TEST);
return mongoose.connection.once('open', () => {
const app = express();
app.post('/graphql', bodyParser.json(), graphqlExpress(req => {
return { schema: executableSchema };
})
);
return app.listen(9000, () => {
console.log('Server started!');
});
});
};
// now here's the before hook where the server is started
let server;
before( done => {
server = startServer(done); // here I need to wait
// some other async code that inserts test data into mongo
const user = new User({email: test#test.com});
user.save().then(() => {
// saved successfully
done();
})
});
I'm quite new in the JS world, how do you manage to wait (without async await syntax) until all the promises from startServer function are resolved and you get back the server instance and just after start inserting data into the database? It's a dummy question but also some links with a better explanation of this concept that I have here would be appreciated.
LE: The entire question has reduced to the followings:
const createServer = () => {
const server = app.listen(port, () => {
//callback body here
});
};
How to convert the callback to a Promise and at the same time return a valid server reference such that in the end I can do this:
const someOtherFunction = () => {
createServer().then(myValidServerInstance => {
//do something with that instance
}
}
TLDR: Some of the things about JS Promises were not clear for me, like for example the returning of the result via resolve() method.
So the right answer looks like this:
export const startServer = () => {
mongoose.Promise = global.Promise;
// first mistake was the use of callbacks rather than Promise approach offered by mongoose.connect
return mongoose.connect(MONGO_URI_TEST).then(() => {
console.log('Connected to MongoLab TEST instance.');
return createServer();
},
err => {
console.log('Error connecting to MongoLab TEST instance:', err);
});
};
The second one was returning directly the result of listen() before the operation get finished. So I've moved the code that is starting the server in another method and wrap the result of listen() into a promise and resolve the promise only when the server actually started listening.
const createServer = () => {
const app = express();
app.post('/graphql', bodyParser.json(), graphqlExpress(req => {
return {
schema: executableSchema,
context: { headers: req.headers},
};
})
);
return new Promise((resolve, reject) => {
const server = app.listen(9000, () => {
if (server) {
console.log('Server started on port 9000');
resolve(server);
} else {
reject();
}
});
});
};