Related
So I'm in the process of learning NestJs ways. I have a small NestJs backend with only a few routes. Some of them call postgreSQL. I don't want to use any ORM and directly use pg package.
So my next step is learning how to use ConfigService. I have successfully used it to configure all env vars in the backend, but I'm struggling to use it in a small file I use to configure postgreSQL. This is the configuration file (pgconnect.ts):
import { Pool } from 'pg';
import configJson from './config/database.json';
import dotenv from 'dotenv';
dotenv.config();
const config = configJson[process.env.NODE_ENV];
const poolConfig = {
user: config.username,
host: config.host,
database: config.database,
password: config.password,
port: config.port,
max: config.maxClients
};
export const pool = new Pool(poolConfig)
database.json is a json file where I have all connect values divided by environment. Then in service classes I just:
import { Injectable } from '#nestjs/common';
import { Response } from 'express';
import { pool } from 'src/database/pgconnect';
#Injectable()
export class MyService {
getDocumentByName(res: Response, name: string) {
pool.query(
<query, error treatment, etc>
});
}
<...> more queries for insert, update, other selects, etc
}
So how could I use ConfigService inside my configuration file ? I already tried to instance class like this:
let configService = new ConfigService();
and what I would like to do is:
const config = configJson[configService.get<string>('NODE_ENV')];
but it didn't work. You have to pass .env file path to new ConfigService(). And I need to use NODE_ENV var to get it, because it depends on environment. To get NODE_ENV without using ConfigService I would have to use dotenv, but if I'm going to use dotenv I don't need ConfigService in the first place.
So then I tried to create a class:
import { Injectable, HttpException, HttpStatus } from '#nestjs/common';
import { ConfigService } from '#nestjs/config'
const { Pool } = require('pg');
import configJson from './config/database.json';
#Injectable()
export class PgPool {
constructor(private configService: ConfigService) { };
config = configJson[this.configService.get<string>('NODE_ENV')];
poolConfig = {
user: this.config.username,
host: this.config.host,
database: this.config.database,
password: this.config.password,
port: this.config.port,
max: this.config.maxClients
};
static pool = new Pool(this.poolConfig);
}
export const PgPool.pool;
But this doesn't work in several ways. If I use non-static members, I can´t export pool member which is the only thing I need. If I use static members one can't access the other or at least I'm not understanding how one access the other.
So, the questions are: How do I use ConfigService outside of a class or how can I change pgconnect.ts file to do it's job ? If it's through a class the best would be to export only pool method.
Also if you think there's a better way to configure postgreSQL I would be glad to hear.
What I would do, if you're going to be using the pg package directly, is create a PgModule that exposes the Pool you create as a provider that can be injected. Then you can also create a provider for the options specifically for ease of swapping in test. Something like this:
#Module({
imports: [ConfigModule],
providers: [
{
provide: 'PG_OPTIONS',
inject: [ConfigService],
useFactory: (config) => ({
host: config.get('DB_HOST'),
port: config.get('DB_PORT'),
...etc
}),
},
{
provide: 'PG_POOL',
inject: ['PG_OPTIONS'],
useFactory: (options) => new Pool(options),
}
],
exports: ['PG_POOL'],
})
export class PgModule {}
Now, when you need to use the Pool in another service you add PgModule to that service's module's imports and you add #Inject('PG_POOL') private readonly pg: Pool to the service's constructor.
If you want to see an overly engineered solution, you can take a look at my old implementation here
I normally have my own pg module handling the pool with either an additional config file (json) or via processing a .env file:
node-pg-sql.js:
/* INFO: Require json config file */
const fileNameConfigPGSQL = require('./config/pgconfig.json');
/* INFO: Require file operations package */
const { Pool } = require('pg');
const pool = new Pool(fileNameConfigPGSQL);
module.exports = {
query: (text, params, callback) => {
const start = Date.now()
return pool.query(text, params, (err, res) => {
const duration = Date.now() - start
// console.log('executed query', { text, duration, rows: res.rowCount })
callback(err, res)
})
},
getClient: (callback) => {
pool.connect((err, client, done) => {
const query = client.query.bind(client)
// monkey patch for the query method to track last queries
client.query = () => {
client.lastQuery = arguments
client.query.apply(client, arguments)
}
// Timeout of 5 secs,then last query is logged
const timeout = setTimeout(() => {
// console.error('A client has been checked out for more than 5 seconds!')
// console.error(`The last executed query on this client was: ${client.lastQuery}`)
}, 5000)
const release = (err) => {
// calling 'done'-method to return client to pool
done(err)
// cleat timeout
clearTimeout(timeout)
// reset query-methode before the Monkey Patch
client.query = query
}
callback(err, client, done)
})
}
}
pgconfig.json:
{
"user":"postgres",
"host":"localhost",
"database":"mydb",
"password":"mypwd",
"port":"5432",
"ssl":true
}
If you prefer processing a .env file:
NODE_ENV=develepment
NODE_PORT=45500
HOST_POSTGRESQL='localhost'
PORT_POSTGRESQL='5432'
DB_POSTGRESQL='mydb'
USER_POSTGRESQL='postgres'
PWD_POSTGRESQL='mypwd'
and process the file and export vars:
var path = require('path');
const dotenvAbsolutePath = path.join(__dirname, '.env');
/* INFO: Require dotenv package for retieving and setting env-vars at runtime via absolute path due to pkg */
const dotenv = require('dotenv').config({
path: dotenvAbsolutePath
});
if (dotenv.error) {
console.log(`ERROR WHILE READING ENV-VARS:${dotenv.error}`);
throw dotenv.error;
}
module.exports = {
nodeEnv: process.env.NODE_ENV,
nodePort: process.env.NODE_PORT,
hostPostgresql: process.env.HOST_POSTGRESQL,
portPostgresql: process.env.PORT_POSTGRESQL,
dbPostgresql: process.env.DB_POSTGRESQL,
userPostgresql: process.env.USER_POSTGRESQL,
pwdPostgresql: process.env.PWD_POSTGRESQL,
};
We are having problem accessing the API-key from our .env file when trying to fetch in our server.js. If we add the API-key manually to the URL in server.js it works, so the problem seems to be the connection between server.js and .env-file.
We have npm installed dotenv.
In the .env file we have written the key like this: WEATHER_API_KEY = XXXXXXXXXXXX
Does anyone know what we have done wrong?
import express from "express";
import cors from "cors";
import mongoose from "mongoose";
import crypto from "crypto";
import bcrypt from "bcrypt";
import request from "request";
import dotenv from "dotenv";
// import { stringify } from "querystring";
const mongoUrl = process.env.MONGO_URL || "mongodb://localhost/project-mongo";
mongoose.connect(mongoUrl, { useNewUrlParser: true, useUnifiedTopology: true });
mongoose.Promise = Promise;
const port = process.env.PORT || 8080;
const app = express();
dotenv.config();
app.get("/home", (req, res) => {
let city = req.query.city;
// const request = require("request");
// const options = {
// url: `https://api.openweathermap.org/data/2.5/weather?q=${city}&appid=${process.env.WEATHER_API_KEY}`,
// method: "GET",
// headers: {
// Accept: "application/json",
// },
// };
const key = "*******************";
const requesturl = `https://api.openweathermap.org/data/2.5/weather?q=${city}&appid=${key}`;
request(requesturl, function (error, response, body) {
let data = JSON.parse(body);
console.log(response);
if (response.statusCode === 200) {
res.send(`The weather in ${city} is ${data.weather[0].description}`);
} else {
res.send(data.message);
}
});
console.log(process.env.WEATHER_API_KEY);
});
You may try this
import 'dotenv/config';
in place of import dotenv from "dotenv"; and remove the dotenv.config(); call.
Source and explanation: https://github.com/motdotla/dotenv#how-do-i-use-dotenv-with-import
And update the request URL (which you might have changed for testing purpose) to
const requesturl = `https://api.openweathermap.org/data/2.5/weather?q=${city}&appid=${process.env.key}`;
Also, try defining your key without any spaces, though this is less likely to be the root cause.
WEATHER_API_KEY="XXXXXXXXXXXX"
Although the question is not specifically about React, it might be helpful for those who use React. For a React App, Environment Variables have to start with the REACT_APP_ prefix otherwise it won't work.
REACT_APP_WEATHER_API_KEY="XXXXXXXXXXXX"
I use TypeORM with NestJS and I am not able to save properly an entity.
The connection creation works, postgres is running on 5432 port. Credentials are OK too.
However when I need to save a resource with entity.save() I got :
Connection "default" was not found.
Error
at new ConnectionNotFoundError (/.../ConnectionNotFoundError.ts:11:22)
I checked the source file of TypeORM ConnectionManager (https://github.com/typeorm/typeorm/blob/master/src/connection/ConnectionManager.ts) but it seems that the first time TypeORM creates connection it attributes "default" name if we don't provide one, which is the case for me.
I setup TypeORM with TypeOrmModule as
TypeOrmModule.forRoot({
type: config.db.type,
host: config.db.host,
port: config.db.port,
username: config.db.user,
password: config.db.password,
database: config.db.database,
entities: [
__dirname + '/../../dtos/entities/*.entity.js',
]
})
Of course my constants are correct. Any ideas ?
You are trying to create a repository or manager without the connection being established.
Try doing this const shopkeeperRepository = getRepository(Shopkeeper); inside a function. it will work
the upvoted answer is not necessarily correct, if you not specify the connection name it will default to "default".
const manager = getConnectionManager().get('your_orm_name');
const repository = manager.getRepository<AModel>(Model);
If anyone else has this problem in the future, check this out just in case:
I accidentally did "user.save()" instead of "userRepo.save(user)".
(And of course above initializing the connection like this:
const userRepo = getConnection(process.env.NODE_ENV).getRepository(User)
We are using lerna and using code from library A in package B.
The problem was that both TypeOrm versions in each package differ.
Solution is to make sure that you have exactly the same version installed in each package.
To be on the safe side, delete your node_modules directory and reinstall everything again with yarn install or npm install
Check your yarn.lock for multiple entries of typeorm and make sure there is only one.
If anyone using Express Router with getRepository(), check the code below
const router = Router();
router.get("/", async function (req: Request, res: Response) {
// here we will have logic to return all users
const userRepository = getRepository(User);
const users = await userRepository.find();
res.json(users);
});
router.get("/:id", async function (req: Request, res: Response) {
// here we will have logic to return user by id
const userRepository = getRepository(User);
const results = await userRepository.findOne(req.params.id);
return res.send(results);
});
Just make sure to call getRepository() in every route just like Saras Arya said in the accepted answer.
I follow the below approach creating the Database class. If the connection doesn't exist then it creates the connection else return the existing connection.
import { Connection, ConnectionManager, ConnectionOptions, createConnection, getConnectionManager } from 'typeorm';
export class Database {
private connectionManager: ConnectionManager;
constructor() {
this.connectionManager = getConnectionManager();
}
public async getConnection(name: string): Promise<Connection> {
const CONNECTION_NAME: string = name;
let connection: Connection;
const hasConnection = this.connectionManager.has(CONNECTION_NAME);
if (hasConnection) {
connection = this.connectionManager.get(CONNECTION_NAME);
if (!connection.isConnected) {
connection = await connection.connect();
}
} else {
const connectionOptions: ConnectionOptions = {
name: 'default',
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'password',
database: 'DemoDb',
synchronize: false,
logging: true,
entities: ['src/entities/**/*.js'],
migrations: ['src/migration/**/*.js'],
subscribers: ['src/subscriber/**/*.js'],
};
connection = await createConnection(connectionOptions);
}
return connection;
}
}
If you are using webpack the make sure entities are imported specifically & returned in array.
import {User} from 'src/entities/User.ts';
import {Album} from 'src/entities/Album.ts';
import {Photos} from 'src/entities/Photos.ts';
const connectionOptions: ConnectionOptions = {
name: 'default',
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'password',
database: 'DemoDb',
synchronize: false,
logging: true,
entities: [User, Album, Photos],
migrations: ['src/migration/**/*.js'],
subscribers: ['src/subscriber/**/*.js'],
};
Finally
const connectionName = 'default';
const database = new Database();
const dbConn: Connection = await database.getConnection(connectionName);
const MspRepository = dbConn.getRepository(Msp);
await MspRepository.delete(mspId);
For those of you looking for another answer, check this out.
In my case, the issue was because I was passing name in my db config.
export const dbConfig = {
name: 'myDB',
...
}
await createConnection(dbConfig) // like this
As a result, the only connection server knows is myDB not default.
At the same time, in my service, repository was injected without name which will fallback to default. (Service will looking for default connection as a result)
#Service() // typedi
export class Service {
constructor(
// inject without name -> fallback to default
#InjectRepository() private readonly repository
) {}
}
As a fix, I removed name property in my db config.
Or you can pass myDB as a parameter for InjectRepository like #InjectRepository('myDB'), either way works.
In my own case, the actual problem was that my index file imports my router file which imports my controllers which then import my services (where the call to getRepository was made). So the imports were resolving (and thus the call to getRepository) before the connection was established.
I considered implementing Sarya's answer but it's going to leave my code more verbose.
What I did was create a function to connect to the DB in a db/index.ts file
import { createConnection } from "typeorm";
export const getDBConnection = async () => {
const dbConnection = await createConnection();
if (!dbConnection.isConnected) await dbConnection.connect();
return dbConnection;
}
Then create an async function to bootstrap my app. I wait on getDBConnection to resolve before instantiating my app then I import my router file after. That way the import resolution only happens after the connection has been established.
routers/index.ts
import { Router } from 'express';
const router = Router();
/* ... route configurations ... */
export default router;
app.ts
const bootstrap = async () => {
try {
// wait on connection to be established
await getDBConnection();
} catch (error) {
// log error then throw
throw error;
}
// create app
const app = express();
// some middleware configuration...
// now import and setup the router
const { default: router } = await import("./routers");
app.use("/api", router);
// some more middleware configuration...
const server = http.createServer(app);
server.listen(3000, () => console.log('app running at port: 3000'));
};
bootstrap();
I got this error while using getConnectionOptions for different environments. Using one database for development and another for testing. This is how I fixed it:
const connectionOptions = await getConnectionOptions(process.env.NODE_ENV);
await createConnection({...connectionOptions, name:"default"});
I usegetConnectionOptions to get the connection for the current environment, in order to do that successfully you have to change ormconfig.json to an array, with keys "name" containing the different environments you want, like so:
[
{
"name" : "development",
"type": "USER",
"host": "localhost",
"port": 5432,
"username": "postgres",
"password": "PASS",
"database": "YOURDB"
},
{
"name" : "test",
"type": "USERTEST",
"host": "localhost",
"port": 5432,
"username": "postgres",
"password": "PASSTEST",
"database": "YOURDBTEST"
}
]
Now connectionOptions will contain the connection parameters of the current environment, but loading it to createConnection threw the error you pointed. Changing connectionOptions name to "default" fixed the issue.
I know it is super weird but someone might need this:
Windows related reason.
I had the same error caused by the current location set with the drive letter in the lower case (d:/apps/app-name/etc).
The problem got fixed once I updated the directory change instruction to use capital D (D:/apps/app-name/etc).
After verifying TypeOrm versions is same in both the packages i.e- external package and consumer repository as mentioned by #InsOp still issue persist then issue could be-
Basically when we create an external package - TypeORM tries to get the "default" connection option, but If not found then throws an error:
ConnectionNotFoundError: Connection "default" was not found.
We can solve this issue by doing some kind of sanity check before establishing a connection - luckily we have .has() method on getConnectionManager().
import { Connection, getConnectionManager, getConnectionOptions,
createConnection, getConnection, QueryRunner } from 'typeorm';
async init() {
let connection: Connection;
let queryRunner: QueryRunner;
if (!getConnectionManager().has('default')) {
const connectionOptions = await getConnectionOptions();
connection = await createConnection(connectionOptions);
} else {
connection = getConnection();
}
queryRunner = connection.createQueryRunner();
}
Above is a quick code-snippet which was the actual root cause for this issue but If you are interested to see complete working repositories (different examples) -
External NPM Package :
Git Repo : git-unit-of-work (specific file- src/providers/typeorm/typeorm-uow.ts)
Published in NPM : npm-unit-of-work
Consumer of above package : nest-typeorm-postgre (specific files- package.json, src/countries/countries.service.ts & countries.module.ts)
In my case was that I have an array of multiple connections, instead of just one. You have 2 alternatives.
To have at least one default named connection, example:
createConnections([
{
name: 'default',
type: 'mysql',
host: 'localhost',
port: 3306,
username: 'root',
password: 'root',
database: 'users',
entities: [`${__dirname}/entity/*{.js,.ts}`],
synchronize: true,
logging: true
}
]);
To be specific when using the connection:
import {getConnection} from "typeorm";
const db1Connection = getConnection("db1Connection");
// you can work with "db1" database now...
I had this same problem with the following code:
import { HttpException, Inject, NotFoundException } from "#nestjs/common";
import { Not } from "typeorm";
import { Transactional } from "typeorm-transactional-cls-hooked";
import { TENANT_CONNECTION } from "../tenant/tenant.module";
import {Feriados} from './feriados.entity';
export class FeriadosService {
repository: any;
constructor(
#Inject(TENANT_CONNECTION) private connection)
{
this.repository = connection.getRepository(Feriados)
}
#Transactional()
async agregar(tablaNueva: Feriados): Promise<Number> {
const tablaAGuardar = await this.repository.create(tablaNueva)
return await this.guardar(tablaAGuardar)
}
#Transactional()
async actualizar(tablaActualizada: Feriados): Promise<Number>{
const tablaAGuardar = await this.repository.merge(tablaActualizada);
return await this.guardar(tablaAGuardar)
}
async guardar(tabla:Feriados){
await this.repository.save(tabla)
return tabla.id
}
I fixed it by removing the 2 #Transactional()
I hope someone helps.
In typeorm v0.3 the Connection API was replaced by the DataSource API. NestJS adapted this change as well, so if you relied on the old API (e.g. getConnection method) you might see the Connection "default" was not found error.
You can read about the changes and the new API in the release notes: https://github.com/typeorm/typeorm/releases/tag/0.3.0
If you used getConnection you can use app.get(DataSource) instead.
In the new version of Typeorm, 0.3.7, a solution to this problem is next:
In the app.module.ts, change the constructor of the AppModule class and create a method to return Datasource:
export class AppModule {
constructor(private dataSource: DataSource) {}
getDataSource() {
return this.dataSource;
}
}
Then, in the file you need to use add:
const repository = app
.get(AppModule)
.getDataSource()
.getRepository('Entity_name');
Although Saras Arya has provided the correct answer, I have encountered the same error
ConnectionNotFoundError: Connection "default" was not found.
due to the fact that my typeORM entity did have an #Entity() decorator as well as that it had extended BaseEntity.
The two can't live together.
Before, when I made apps with create-react-app, I would have a setupProxy.js file that would route API requests similar to this
const proxy = require('http-proxy-middleware');
module.exports = function(app) {
app.use('/api',
proxy({
target: 'http://localhost:8000',
changeOrigin: true,
})
);
};
But that doesn't seem to work with next.js. When I do the same thing, I get various errors.
Googling a solution, a lot say to use a custom server of some kind. But how would I accomplish a proxy like above using the default nextjs dev server? (Equivalent of npm run dev when dev in my package.json is next dev.
There is now an official feature for this in the config: Rewrites
Besides normal path rewrites, they can also proxy requests to another webserver
next.config.js:
module.exports = {
async rewrites() {
return [
{
source: '/api/:path*',
destination: 'http://localhost:8000/:path*' // Proxy to Backend
}
]
}
}
See Next.js Docs Rewrites
My server.js set up, hope it helps:
import express from 'express';
import next from 'next';
import proxy from 'http-proxy-middleware';
const port = parseInt(process.env.PORT, 10) || 3000;
const dev = process.env.NODE_ENV !== 'production';
const app = next({ dev });
const handle = app.getRequestHandler();
app.prepare().then(() => {
const server = express();
server.use(
'/api',
proxy({
target: process.env.API_HOST,
changeOrigin: true,
}),
);
server.all('*', (req, res) => handle(req, res));
server.listen(port, err => {
if (err) throw err;
console.log(`> Ready on http://localhost:${port}`);
});
});
package.json:
"scripts": {
"dev": "NODE_ENV=development node -r esm server.js",
"build": "NODE_ENV=production next build",
"start": "NODE_ENV=production node -r esm server.js",
},
Another solution with catch-all routes + http-proxy-middleware:
// pages/api/proxy/[...slug].js
import { createProxyMiddleware } from "http-proxy-middleware"; // #2.0.6
const proxy = createProxyMiddleware({
target: process.env.BACKEND_URL,
secure: false,
pathRewrite: { "^/api/proxy": "" }, // remove `/api/proxy` prefix
});
export default function handler(req, res) {
proxy(req, res, (err) => {
if (err) {
throw err;
}
throw new Error(
`Request '${req.url}' is not proxied! We should never reach here!`
);
});
}
see: https://stackoverflow.com/a/72310680
Rewrites didn't work for me, and neither did using axios config.proxy.
I've resorted to a good old constant:
const SERVER_URL =
process.env.NODE_ENV === 'production' ? 'https://produrl.com : 'http://localhost:8000';
export async function getStaticProps() {
const data = axios.get(`${SERVER_URL}/api/my-route`)
// ...
}
I would much rather proxy requests and keep my requests cleaner, but I don't have a day to spend wrestling with this.
Maybe this very simple setup will help others.
I cannot work out why I would get a query undefined when I know my definitions are correct. graphiQL is picking up my schemes without problems:
Auto complete works fine:
After hitting ctrl+enter all the fields are entered, see above.
Then I execute the query and I'll get:
{
"errors": [
{
"message": "Cannot read property 'query' of undefined",
"locations": [
{
"line": 1,
"column": 3
}
],
"path": [
"allAwards"
]
}
],
"data": {
"allAwards": null
}
}
npm run graphql
"graphql": "nodemon -r dotenv/config --experimental-modules --inspect=[9222] graphql_server.js",
graphql_server.js
import express from 'express'
import pg from 'pg'
import graphqlHTTP from 'express-graphql'
import PAS from 'postgraphile-apollo-server'
import AP from 'apollo-server-express'
const { makeSchemaAndPlugin } = PAS
const { ApolloServer } = AP
const env = process.env.NODE_ENV || 'development'
const dbHost = process.env.DB_HOST
const dbPort = process.env.DB_PORT
const dbName = process.env.DB_NAME
const dbUser = process.env.DB_USER
const dbPwd = process.env.DB_PWD
const dbUrl = dbPwd
? `postgres://${dbUser}:${dbPwd}#${dbHost}:${dbPort}/${dbName}`
: `postgres://${dbHost}:${dbPort}/${dbName}`
const pgPool = new pg.Pool({
connectionString: dbUrl,
})
async function main() {
const { schema, plugin } = await makeSchemaAndPlugin(
pgPool,
'public', // PostgreSQL schema to use
{
// PostGraphile options, see:
// https://www.graphile.org/postgraphile/usage-library/
}
)
const server = new ApolloServer({
schema,
plugins: [plugin],
})
const app = express()
app.use(
'/graphql',
graphqlHTTP({
schema: schema,
graphiql: true,
})
)
server.applyMiddleware({ app })
app.listen({ port: 4000 }, () => console.log(`🚀 Server ready at http://localhost:4000${server.graphqlPath}`))
}
main().catch(e => {
console.error(e)
process.exit(1)
})
There are 2 rows currently in psql db for awards as well
You should not utilize middleware from both express-graphql and apollo-server in your express application. Because postgraphile-apollo-server works explicitly with ApolloServer, drop express-graphql altogether. Having both middleware is likely to cause unexpected issues since they listen on the same paths.
Apollo has abandoned GraphiQL in favor of GraphQL Playground. If you want to use GraphiQL with Apollo, you can use a package like express-graphiql-middleware.