Asynchronously load configuration for Sequelize CLI - javascript

I am using Sequelize CLI to run migrations against out database and I have the following .sequelizerc file
// .sequelizerc
const path = require('path');
module.exports = {
"config": path.resolve('src', 'database', 'config.js'),
"models-path": path.resolve('src', 'models'),
"seeders-path": path.resolve('src', 'database', 'seeders'),
"migrations-path": path.resolve('src', 'database', 'migrations'),
};
The credentials for our database are kept in a secrets manager so they can be easily rotated and never stored within our repository or someone's computer. Thus, our config.js file looks like:
// src/database/config.js
const { AsyncObtainDBCredentials } = require("../utilities");
async function setupDBCreds() {
const dbConfig = await AsyncObtainDBCredentials();
return dbConfig;
}
module.export = setupDBCreds(); // exports a promise and does not work
Sequelize CLI is expecting a configuration JSON like
const dotenv = require('dotenv')
dotenv.config()
const dbConfig = {
host: process.env.DB_HOST ?? 'localhost',
username: process.env.DB_USER ?? 'postgres',
port: process.env.DB_PORT ?? '5432',
database: process.env.DB_NAME ?? 'myDBName',
password: process.env.DB_PASSWORD ?? 'SuperSecretPasswd',
dialect: "postgres"
};
module.exports = dbConfig;
Is there a way to make this work so we can avoid users having write credentials in to an .env file?

Related

ConnectionNotFoundError: Connection "default" was not found. Can someone help me? [duplicate]

I use TypeORM with NestJS and I am not able to save properly an entity.
The connection creation works, postgres is running on 5432 port. Credentials are OK too.
However when I need to save a resource with entity.save() I got :
Connection "default" was not found.
Error
at new ConnectionNotFoundError (/.../ConnectionNotFoundError.ts:11:22)
I checked the source file of TypeORM ConnectionManager (https://github.com/typeorm/typeorm/blob/master/src/connection/ConnectionManager.ts) but it seems that the first time TypeORM creates connection it attributes "default" name if we don't provide one, which is the case for me.
I setup TypeORM with TypeOrmModule as
TypeOrmModule.forRoot({
type: config.db.type,
host: config.db.host,
port: config.db.port,
username: config.db.user,
password: config.db.password,
database: config.db.database,
entities: [
__dirname + '/../../dtos/entities/*.entity.js',
]
})
Of course my constants are correct. Any ideas ?
You are trying to create a repository or manager without the connection being established.
Try doing this const shopkeeperRepository = getRepository(Shopkeeper); inside a function. it will work
the upvoted answer is not necessarily correct, if you not specify the connection name it will default to "default".
const manager = getConnectionManager().get('your_orm_name');
const repository = manager.getRepository<AModel>(Model);
If anyone else has this problem in the future, check this out just in case:
I accidentally did "user.save()" instead of "userRepo.save(user)".
(And of course above initializing the connection like this:
const userRepo = getConnection(process.env.NODE_ENV).getRepository(User)
We are using lerna and using code from library A in package B.
The problem was that both TypeOrm versions in each package differ.
Solution is to make sure that you have exactly the same version installed in each package.
To be on the safe side, delete your node_modules directory and reinstall everything again with yarn install or npm install
Check your yarn.lock for multiple entries of typeorm and make sure there is only one.
If anyone using Express Router with getRepository(), check the code below
const router = Router();
router.get("/", async function (req: Request, res: Response) {
// here we will have logic to return all users
const userRepository = getRepository(User);
const users = await userRepository.find();
res.json(users);
});
router.get("/:id", async function (req: Request, res: Response) {
// here we will have logic to return user by id
const userRepository = getRepository(User);
const results = await userRepository.findOne(req.params.id);
return res.send(results);
});
Just make sure to call getRepository() in every route just like Saras Arya said in the accepted answer.
I follow the below approach creating the Database class. If the connection doesn't exist then it creates the connection else return the existing connection.
import { Connection, ConnectionManager, ConnectionOptions, createConnection, getConnectionManager } from 'typeorm';
export class Database {
private connectionManager: ConnectionManager;
constructor() {
this.connectionManager = getConnectionManager();
}
public async getConnection(name: string): Promise<Connection> {
const CONNECTION_NAME: string = name;
let connection: Connection;
const hasConnection = this.connectionManager.has(CONNECTION_NAME);
if (hasConnection) {
connection = this.connectionManager.get(CONNECTION_NAME);
if (!connection.isConnected) {
connection = await connection.connect();
}
} else {
const connectionOptions: ConnectionOptions = {
name: 'default',
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'password',
database: 'DemoDb',
synchronize: false,
logging: true,
entities: ['src/entities/**/*.js'],
migrations: ['src/migration/**/*.js'],
subscribers: ['src/subscriber/**/*.js'],
};
connection = await createConnection(connectionOptions);
}
return connection;
}
}
If you are using webpack the make sure entities are imported specifically & returned in array.
import {User} from 'src/entities/User.ts';
import {Album} from 'src/entities/Album.ts';
import {Photos} from 'src/entities/Photos.ts';
const connectionOptions: ConnectionOptions = {
name: 'default',
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'password',
database: 'DemoDb',
synchronize: false,
logging: true,
entities: [User, Album, Photos],
migrations: ['src/migration/**/*.js'],
subscribers: ['src/subscriber/**/*.js'],
};
Finally
const connectionName = 'default';
const database = new Database();
const dbConn: Connection = await database.getConnection(connectionName);
const MspRepository = dbConn.getRepository(Msp);
await MspRepository.delete(mspId);
For those of you looking for another answer, check this out.
In my case, the issue was because I was passing name in my db config.
export const dbConfig = {
name: 'myDB',
...
}
await createConnection(dbConfig) // like this
As a result, the only connection server knows is myDB not default.
At the same time, in my service, repository was injected without name which will fallback to default. (Service will looking for default connection as a result)
#Service() // typedi
export class Service {
constructor(
// inject without name -> fallback to default
#InjectRepository() private readonly repository
) {}
}
As a fix, I removed name property in my db config.
Or you can pass myDB as a parameter for InjectRepository like #InjectRepository('myDB'), either way works.
In my own case, the actual problem was that my index file imports my router file which imports my controllers which then import my services (where the call to getRepository was made). So the imports were resolving (and thus the call to getRepository) before the connection was established.
I considered implementing Sarya's answer but it's going to leave my code more verbose.
What I did was create a function to connect to the DB in a db/index.ts file
import { createConnection } from "typeorm";
export const getDBConnection = async () => {
const dbConnection = await createConnection();
if (!dbConnection.isConnected) await dbConnection.connect();
return dbConnection;
}
Then create an async function to bootstrap my app. I wait on getDBConnection to resolve before instantiating my app then I import my router file after. That way the import resolution only happens after the connection has been established.
routers/index.ts
import { Router } from 'express';
const router = Router();
/* ... route configurations ... */
export default router;
app.ts
const bootstrap = async () => {
try {
// wait on connection to be established
await getDBConnection();
} catch (error) {
// log error then throw
throw error;
}
// create app
const app = express();
// some middleware configuration...
// now import and setup the router
const { default: router } = await import("./routers");
app.use("/api", router);
// some more middleware configuration...
const server = http.createServer(app);
server.listen(3000, () => console.log('app running at port: 3000'));
};
bootstrap();
I got this error while using getConnectionOptions for different environments. Using one database for development and another for testing. This is how I fixed it:
const connectionOptions = await getConnectionOptions(process.env.NODE_ENV);
await createConnection({...connectionOptions, name:"default"});
I usegetConnectionOptions to get the connection for the current environment, in order to do that successfully you have to change ormconfig.json to an array, with keys "name" containing the different environments you want, like so:
[
{
"name" : "development",
"type": "USER",
"host": "localhost",
"port": 5432,
"username": "postgres",
"password": "PASS",
"database": "YOURDB"
},
{
"name" : "test",
"type": "USERTEST",
"host": "localhost",
"port": 5432,
"username": "postgres",
"password": "PASSTEST",
"database": "YOURDBTEST"
}
]
Now connectionOptions will contain the connection parameters of the current environment, but loading it to createConnection threw the error you pointed. Changing connectionOptions name to "default" fixed the issue.
I know it is super weird but someone might need this:
Windows related reason.
I had the same error caused by the current location set with the drive letter in the lower case (d:/apps/app-name/etc).
The problem got fixed once I updated the directory change instruction to use capital D (D:/apps/app-name/etc).
After verifying TypeOrm versions is same in both the packages i.e- external package and consumer repository as mentioned by #InsOp still issue persist then issue could be-
Basically when we create an external package - TypeORM tries to get the "default" connection option, but If not found then throws an error:
ConnectionNotFoundError: Connection "default" was not found.
We can solve this issue by doing some kind of sanity check before establishing a connection - luckily we have .has() method on getConnectionManager().
import { Connection, getConnectionManager, getConnectionOptions,
createConnection, getConnection, QueryRunner } from 'typeorm';
async init() {
let connection: Connection;
let queryRunner: QueryRunner;
if (!getConnectionManager().has('default')) {
const connectionOptions = await getConnectionOptions();
connection = await createConnection(connectionOptions);
} else {
connection = getConnection();
}
queryRunner = connection.createQueryRunner();
}
Above is a quick code-snippet which was the actual root cause for this issue but If you are interested to see complete working repositories (different examples) -
External NPM Package :
Git Repo : git-unit-of-work (specific file- src/providers/typeorm/typeorm-uow.ts)
Published in NPM : npm-unit-of-work
Consumer of above package : nest-typeorm-postgre (specific files- package.json, src/countries/countries.service.ts & countries.module.ts)
In my case was that I have an array of multiple connections, instead of just one. You have 2 alternatives.
To have at least one default named connection, example:
createConnections([
{
name: 'default',
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'root',
database: 'users',
entities: [`${__dirname}/entity/*{.js,.ts}`],
synchronize: true,
logging: true
}
]);
To be specific when using the connection:
import {getConnection} from "typeorm";
const db1Connection = getConnection("db1Connection");
// you can work with "db1" database now...
I had this same problem with the following code:
import { HttpException, Inject, NotFoundException } from "#nestjs/common";
import { Not } from "typeorm";
import { Transactional } from "typeorm-transactional-cls-hooked";
import { TENANT_CONNECTION } from "../tenant/tenant.module";
import {Feriados} from './feriados.entity';
export class FeriadosService {
repository: any;
constructor(
#Inject(TENANT_CONNECTION) private connection)
{
this.repository = connection.getRepository(Feriados)
}
#Transactional()
async agregar(tablaNueva: Feriados): Promise<Number> {
const tablaAGuardar = await this.repository.create(tablaNueva)
return await this.guardar(tablaAGuardar)
}
#Transactional()
async actualizar(tablaActualizada: Feriados): Promise<Number>{
const tablaAGuardar = await this.repository.merge(tablaActualizada);
return await this.guardar(tablaAGuardar)
}
async guardar(tabla:Feriados){
await this.repository.save(tabla)
return tabla.id
}
I fixed it by removing the 2 #Transactional()
I hope someone helps.
In typeorm v0.3 the Connection API was replaced by the DataSource API. NestJS adapted this change as well, so if you relied on the old API (e.g. getConnection method) you might see the Connection "default" was not found error.
You can read about the changes and the new API in the release notes: https://github.com/typeorm/typeorm/releases/tag/0.3.0
If you used getConnection you can use app.get(DataSource) instead.
In the new version of Typeorm, 0.3.7, a solution to this problem is next:
In the app.module.ts, change the constructor of the AppModule class and create a method to return Datasource:
export class AppModule {
constructor(private dataSource: DataSource) {}
getDataSource() {
return this.dataSource;
}
}
Then, in the file you need to use add:
const repository = app
.get(AppModule)
.getDataSource()
.getRepository('Entity_name');
Although Saras Arya has provided the correct answer, I have encountered the same error
ConnectionNotFoundError: Connection "default" was not found.
due to the fact that my typeORM entity did have an #Entity() decorator as well as that it had extended BaseEntity.
The two can't live together.

Error Connecting to the database: Error: ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server

I've tried following the suggestion at this link: MySQL 8.0 - Client does not support authentication protocol requested by server; consider upgrading MySQL client
and am aware that it is using cahcing_sha2_password instead of mysql_native. However, I've tried following the suggestions and am still receiving the same errors, trying to connect to the database with node.
Schema name: 'my_db'
//DATABASE CONNECTION SETTINGS
const APIServerPort = 3001;
const database = {
host: "localhost",
port: 3306,
user: "root",
password: "xxx",
database: "my_db"
};
module.exports = {
database,
APIServerPort
};
app.js file:
const express = require("express");
const app = express();
const router = express.Router();
const settings = require("./settings");
const routes = require("./routes");
const mysql = require("mysql");
const connection = mysql.createConnection(settings.database);
router.get("/employees", routes.employees.listAllEmployees);
app.use("/apis", router);
connection.connect(error => {
if (error) {
console.error("Error Connecting to the database: " + error);
return process.exit();
}
app.listen(settings.APIServerPort, () =>
console.info(`Server is running at port ${settings.APIServerPort}...`)
);
});
SQL Queries Ran:
CREATE USER 'root'#'localhost:3301/apis/employees' IDENTIFIED WITH mysql_native_password BY 'xxx';
FLUSH PRIVILEGES;
I'm still receiving the same error however.
mysql users are in the form 'root'#'localhost'. Urls are a node.js concept only.
As that user already exists create a new one:
CREATE USER myapplication#localhost
IDENTIFIED WITH mysql_native_password BY 'xxx';
GRANT ALL ON my_db.* TO myapplication#localhost`;
You don't need FLUSH PRIVILEGES.

Javascript class method returning undefined

I'm trying to return a value from a class but it's coming up undefined.
index.js
import DB from "./db.js"
import express from "express";
import cors from "cors";
const database = new DB();
app.get("/", (req, res) => {
const data = database.selectAllFromProducts();
console.log(data); // Returns undefined
});
app.listen(process.env.PORT, () =>
console.log(`Listening on Port ${process.env.PORT}`)
);
db.js
class DB {
constructor() {
this.connection = this.initialize();
}
initialize() {
return mysql.createConnection({
host: process.env.DB_HOST,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME,
});
selectAllFromProducts() {
this.initialize();
this.connection.query(`select * from ${process.env.DB_PRODUCTS_TABLE};`,
(err, results, fields) => {return results});
}
}
I have a front end that is sending the GET request and that is successful so it's not a routing problem. Console.logging the results works from db.js so I know it's not a MYSQL problem but for whatever reason it comes up blank in index.js. Thanks in advance for any help!
EDIT - I have module.exports = DB I just forgot to include it because I only included partial bits of the file. Importing works just fine for me because I'm using babel and type: module in my package.json. I couldn't tag node because it requires 1500 rep.
You forgot to export your module. Thus database should be undefined in your code, resulting in your issues.
class DB {
constructor() {
this.connection = this.initialize();
}
initialize() {
return mysql.createConnection({
host: process.env.DB_HOST,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME,
});
selectAllFromProducts() {
this.initialize();
this.connection.query(`select * from ${process.env.DB_PRODUCTS_TABLE};`,
(err, results, fields) => {return results});
}
}
module.exports = DB;
Also importing is a little different in node.js
var DB = require("./db.js");
var express = require("express");
var cors = require("cors");
const database = new DB();
app.get("/", (req, res) => {
const data = database.selectAllFromProducts();
console.log(data); // Returns undefined
});
app.listen(process.env.PORT, () =>
console.log(`Listening on Port ${process.env.PORT}`)
);
Note: You should be able to use javascript modules in node.js versions greater than 13 (which is why your code may be working). Make sure your deployment server supports node.js 13 as well if you decide to use that syntax. More on javascript modules

How can i access the variable "knex1" from the below code from "knexfile.js" file into the the db file?

How can i access the variable "knex1" from the below code from "knexfile.js" file into the the "db.js" file to check that whether the connection within "db.js" is connected or not ,thereby saying console message as "DB Connected"? ... But it says "TypeError: Cannot read property 'connect' of undefined" error when i run?
knexfile.js
var knex1 = {
development: {
client: 'mysql',
connection: {
host: 'localhost',
user: 'root',
password: 'password',
database: 'user_data',
}
},
};
module.exports = knex1;
db.js file
const mysql = require('mysql');
const db = require('../knexfile');
const NODE_ENV = 'development' ;
const knex = require('knex')((db)[NODE_ENV]);
const bookshelf = require('bookshelf')(knex);
db.connection.connect(function(err){
if(!err) {
console.log("Database is connected ... ");
}
else {
console.log("Error connecting database ... ");
}
});
let registerdata = bookshelf.Model.extend({
tableName: 'RegisterData'
});
module.exports = {
registerdata
};
First check your directory listing if your db.js and knexfile.js are in same folder like this
-db.js
-knexfile.js
then use const db = require('./knexfile');
if the db.js is inside another folder like this
-folder
-db.js
-knexfile.js
then use const db = require('../knexfile');
The ../ or ./ or ../../ is how you find your files.
Check out this link for more on how to handle es6 modules.
https://learnersbucket.com/tutorials/es6/es6-modules/

How can I keep a constant MySQL connection alive in NodeJS

I'm currently developing Discord Bot, for data storing I'm using MySQL, but after some hours de connection dies. I was wondering if someone has a clue on how to do this. This the way I currently connect:
// Initalise Variables
var config;
var mysql, conn;
var fs;
try {
// External Packages
fs = require('fs');
config = require('./config.json');
mysql = require('mysql');
// Connection Setup
conn = mysql.createConnection({
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.password,
database: config.mysql.database
});
conn.connect();
} catch (e) {
console.error(e);
}
You can try changing the timeout setting else if would default to 10000ms or 10 seconds.
try {
// External Packages
fs = require('fs');
config = require('./config.json');
mysql = require('mysql');
// Connection Setup
conn = mysql.createConnection({
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.password,
database: config.mysql.database,
connectTimeout: config.mysql.timeout //1000000 some large number
});
conn.connect();
} catch (e) {
console.error(e);
}
You should probably use a connection pool instead of a single DB connection (this doesn't seem to have always keep alive type of setting, and you would have to give a large timeout like above)
try {
// External Packages
fs = require('fs');
config = require('./config.json');
mysql = require('mysql');
// Connection Setup
conn = mysql.createConnection({
host: config.mysql.host,
user: config.mysql.user,
password: config.mysql.password,
database: config.mysql.database,
connectionLimit: 10,
queueLimit: 20
});
var pool = mysql.createPool(config);
var queryDB = function(qry, cb) {
pool.getConnection(function(error, connection) {
if(error) {
cb(error, null);
}
else {
connection.query(qry, function (e, rows) {
connection.destroy();
cb(e, rows);
});
}
});
I think you need to install forever and start the service again. Forever is a simple CLI tool for ensuring that a given script runs continuously. Once you install forever and run the node js file then it will keeps the file alive continuously. Using npm you can install forever
npm install forever -g
Then just restart the js file
Ex:
forever start app.js
Hope now the js serves continously without breaking the connection.

Categories