We Keep Coding

How can I configure postgreSQL in the Nestjs way?

So I'm in the process of learning NestJs ways. I have a small NestJs backend with only a few routes. Some of them call postgreSQL. I don't want to use any ORM and directly use pg package.
So my next step is learning how to use ConfigService. I have successfully used it to configure all env vars in the backend, but I'm struggling to use it in a small file I use to configure postgreSQL. This is the configuration file (pgconnect.ts):
import { Pool } from 'pg';
import configJson from './config/database.json';
import dotenv from 'dotenv';
dotenv.config();
const config = configJson[process.env.NODE_ENV];
const poolConfig = {
user: config.username,
host: config.host,
database: config.database,
password: config.password,
port: config.port,
max: config.maxClients
};
export const pool = new Pool(poolConfig)
database.json is a json file where I have all connect values divided by environment. Then in service classes I just:
import { Injectable } from '#nestjs/common';
import { Response } from 'express';
import { pool } from 'src/database/pgconnect';
#Injectable()
export class MyService {
getDocumentByName(res: Response, name: string) {
pool.query(
<query, error treatment, etc>
});
}
<...> more queries for insert, update, other selects, etc
}
So how could I use ConfigService inside my configuration file ? I already tried to instance class like this:
let configService = new ConfigService();
and what I would like to do is:
const config = configJson[configService.get<string>('NODE_ENV')];
but it didn't work. You have to pass .env file path to new ConfigService(). And I need to use NODE_ENV var to get it, because it depends on environment. To get NODE_ENV without using ConfigService I would have to use dotenv, but if I'm going to use dotenv I don't need ConfigService in the first place.
So then I tried to create a class:
import { Injectable, HttpException, HttpStatus } from '#nestjs/common';
import { ConfigService } from '#nestjs/config'
const { Pool } = require('pg');
import configJson from './config/database.json';
#Injectable()
export class PgPool {
constructor(private configService: ConfigService) { };
config = configJson[this.configService.get<string>('NODE_ENV')];
poolConfig = {
user: this.config.username,
host: this.config.host,
database: this.config.database,
password: this.config.password,
port: this.config.port,
max: this.config.maxClients
};
static pool = new Pool(this.poolConfig);
}
export const PgPool.pool;
But this doesn't work in several ways. If I use non-static members, I can´t export pool member which is the only thing I need. If I use static members one can't access the other or at least I'm not understanding how one access the other.
So, the questions are: How do I use ConfigService outside of a class or how can I change pgconnect.ts file to do it's job ? If it's through a class the best would be to export only pool method.
Also if you think there's a better way to configure postgreSQL I would be glad to hear.

What I would do, if you're going to be using the pg package directly, is create a PgModule that exposes the Pool you create as a provider that can be injected. Then you can also create a provider for the options specifically for ease of swapping in test. Something like this:
#Module({
imports: [ConfigModule],
providers: [
{
provide: 'PG_OPTIONS',
inject: [ConfigService],
useFactory: (config) => ({
host: config.get('DB_HOST'),
port: config.get('DB_PORT'),
...etc
}),
},
{
provide: 'PG_POOL',
inject: ['PG_OPTIONS'],
useFactory: (options) => new Pool(options),
}
],
exports: ['PG_POOL'],
})
export class PgModule {}
Now, when you need to use the Pool in another service you add PgModule to that service's module's imports and you add #Inject('PG_POOL') private readonly pg: Pool to the service's constructor.
If you want to see an overly engineered solution, you can take a look at my old implementation here

I normally have my own pg module handling the pool with either an additional config file (json) or via processing a .env file:
node-pg-sql.js:
/* INFO: Require json config file */
const fileNameConfigPGSQL = require('./config/pgconfig.json');
/* INFO: Require file operations package */
const { Pool } = require('pg');
const pool = new Pool(fileNameConfigPGSQL);
module.exports = {
query: (text, params, callback) => {
const start = Date.now()
return pool.query(text, params, (err, res) => {
const duration = Date.now() - start
// console.log('executed query', { text, duration, rows: res.rowCount })
callback(err, res)
})
},
getClient: (callback) => {
pool.connect((err, client, done) => {
const query = client.query.bind(client)
// monkey patch for the query method to track last queries
client.query = () => {
client.lastQuery = arguments
client.query.apply(client, arguments)
}
// Timeout of 5 secs,then last query is logged
const timeout = setTimeout(() => {
// console.error('A client has been checked out for more than 5 seconds!')
// console.error(`The last executed query on this client was: ${client.lastQuery}`)
}, 5000)
const release = (err) => {
// calling 'done'-method to return client to pool
done(err)
// cleat timeout
clearTimeout(timeout)
// reset query-methode before the Monkey Patch
client.query = query
}
callback(err, client, done)
})
}
}
pgconfig.json:
{
"user":"postgres",
"host":"localhost",
"database":"mydb",
"password":"mypwd",
"port":"5432",
"ssl":true
}
If you prefer processing a .env file:
NODE_ENV=develepment
NODE_PORT=45500
HOST_POSTGRESQL='localhost'
PORT_POSTGRESQL='5432'
DB_POSTGRESQL='mydb'
USER_POSTGRESQL='postgres'
PWD_POSTGRESQL='mypwd'
and process the file and export vars:
var path = require('path');
const dotenvAbsolutePath = path.join(__dirname, '.env');
/* INFO: Require dotenv package for retieving and setting env-vars at runtime via absolute path due to pkg */
const dotenv = require('dotenv').config({
path: dotenvAbsolutePath
});
if (dotenv.error) {
console.log(`ERROR WHILE READING ENV-VARS:${dotenv.error}`);
throw dotenv.error;
}
module.exports = {
nodeEnv: process.env.NODE_ENV,
nodePort: process.env.NODE_PORT,
hostPostgresql: process.env.HOST_POSTGRESQL,
portPostgresql: process.env.PORT_POSTGRESQL,
dbPostgresql: process.env.DB_POSTGRESQL,
userPostgresql: process.env.USER_POSTGRESQL,
pwdPostgresql: process.env.PWD_POSTGRESQL,
};

How to send piped response in Next.js API using wkhtmltoimage?

I'm new to Next.js, and I'm trying to use wkhtmltoimage but I can't seem to send the generated image stream as a response in my Next.js API.
const fs = require('fs')
const wkhtmltoimage = require('wkhtmltoimage').setCommand(__dirname + '/bin/wkhtmltoimage');
export default async function handler(req, res) {
try {
await wkhtmltoimage.generate('<h1>Hello world</h1>').pipe(res);
res.status(200).send(res)
} catch (err) {
res.status(500).send({ error: 'failed to fetch data' })
}
}
I know I'm doing plenty of stuff wrong here, can anyone point me to the right direction?

Since you're concatenating __dirname and /bin/wkhtmltoimage together, that would mean you've installed the wkhtmltoimage executable to ./pages/api/bin which is probably not a good idea since the pages directory is special to Next.js.
We'll assume you've installed the executable in a different location on your filesystem/server instead (e.g., your home directory). It looks like the pipe function already sends the response, so the res.status(200).send(res) line will cause problems and can be removed. So the following should work:
// ./pages/api/hello.js
const homedir = require("os").homedir();
// Assumes the following installation path:
// - *nix: $HOME/bin/wkhtmltoimage
// - Windows: $env:USERPROFILE\bin\wkhtmltoimage.exe
const wkhtmltoimage = require("wkhtmltoimage").setCommand(
homedir + "/bin/wkhtmltoimage"
);
export default async function handler(req, res) {
try {
res.status(200);
await wkhtmltoimage.generate("<h1>Hello world</h1>").pipe(res);
} catch (err) {
res.status(500).send({ error: "failed to fetch data" });
}
}

Next.js middleware Module not found: Can't resolve 'fs'

Getting this error in Next.js _middleware file when I try to initialize Firebase admin V9. Anyone know how to solve this issue?
./node_modules/#google-cloud/storage/build/src/bucket.js:22:0
Module not found: Can't resolve 'fs'
../../firebase/auth-admin
import * as admin from "firebase-admin";
if (!admin.apps.length) {
admin.initializeApp({
credential: admin.credential.cert({
projectId: process.env.NEXT_PUBLIC_FIREBASE_PROJECT_ID,
clientEmail: process.env.FIREBASE_CLIENT_EMAIL,
privateKey: process.env.FIREBASE_ADMIN_PRIVATE_KEY,
}),
});
}
const firestore = admin.firestore();
const auth = admin.auth();
export { firestore, auth };
Calling it in my _middleware
import { NextFetchEvent, NextRequest, NextResponse } from "next/server";
import { auth } from "../../firebase/auth-admin";
export default async function authenticate(
req: NextRequest,
ev: NextFetchEvent
) {
const token = req.headers.get("token");
console.log("auth = ", auth);
// const decodeToken = await auth.verifyIdToken(token);
return NextResponse.next();
}
I saw a solution here by customizing webpack but this does not fix it.
/** #type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
webpack: (config, { isServer, node }) => {
node = {
...node,
fs: "empty",
child_process: "empty",
net: "empty",
tls: "empty",
};
return config;
},
};
module.exports = nextConfig;

The Edge Runtime, which is used by Next.js Middleware, does not support Node.js native APIs.
From the Edge Runtime documentation:
The Edge Runtime has some restrictions including:
Native Node.js APIs are not supported. For example, you can't read or write to the filesystem
Node Modules can be used, as long as they implement ES Modules and do not use any native Node.js APIs
You can't use Node.js libraries that use fs in Next.js Middleware. Try using a client-side library instead.

I wasted a lot of time tying to get this to work. The weird thing is that this will work in the api itself.
So instead of calling firebase-admin action in the _middleware file. Call it in the api itself like:
import type { NextApiRequest, NextApiResponse } from 'next'
import { auth } from "../../firebase/auth-admin";
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
const authorization = req.headers.authorization
console.log(`Handler auth header: ${authorization}`)
if (!authorization) {
return res.status(401).json({ message: 'Authorisation header not found.' })
}
const token = authorization.split(' ')[1]
if (!token) {
return res.status(401).json({ message: 'Bearer token not found.' })
}
console.log(`Token: ${token}`)
try {
const {uid} = await auth.verifyIdToken("sd" + token)
console.log(`User uid: ${uid}`)
res.status(200).json({ userId: uid })
} catch (error) {
console.log(`verifyIdToken error: ${error}`)
res.status(401).json({ message: `Error while verifying token. Error: ${error}` })
}
}
A workaround to make this reusable is to create a wrapper function.
If anyone knows how to make this work in a _middleware file, I would be really grateful.
Edit: Gist for the wrapper middleware function:
https://gist.github.com/jvgrootveld/ed1863f0beddc1cc2bf2d3593dedb6da

make sure you're not calling firebase-admin in the client
import * as admin from "firebase-admin";

I've recently released a library that aims to solve the problem: https://github.com/ensite-in/next-firebase-auth-edge
It allows to create and verify tokens inside Next.js middleware and Next.js 13 server components. Built entirely upon Web Crypto API.
Please note it does rely on Next.js ^13.0.5 experimental "appDir" and "allowMiddlewareResponseBody" features.

How can I localize routes with the nextJs and next-i18next like an URL alias?

I'm using NextJs 10.0.5 with next-i18next 8.1.0 to localize my application. As we all know nextJs 10 has subpath routing for internationalized routing. In addition, I need to change the page names by language. For example, I have a contact-us file inside the pages folder. When I change the language to Turkish, I have to use localhost:3000/tr/contact-us. However, I want to use localhost:3000/bize-ulasin to access the contact-us page when the language is Turkish. So there are two URLs and only one page file.
It works when I use custom routing with express js in the server.js file. However, when I want to access the "locale" variable within the getStaticProps function in the contact-us file, I cannot access it. The getStaticProps function returns undefined for "locale" variable when I use localhost:3000/bize-ulasin URL.
server.js
const { createServer } = require("http");
const { parse } = require("url");
const next = require("next");
const app = next({ dev: process.env.NODE_ENV !== "production" });
const handle = app.getRequestHandler(app);
app.prepare().then(() => {
createServer((req, res) => {
const parsedUrl = parse(req.url, true);
const { pathname, query } = parsedUrl;
if (pathname === "/bize-ulasin") {
app.render(req, res, "/contact-us", query);
}else{
handle(req, res, parsedUrl);
}
}).listen(3000, (err) => {
if (err) throw err;
console.log("> Ready on http://localhost:3000");
});
});
/pages/contact-us-file
import { Fragment } from "react";
import Head from "next/head";
import { useTranslation } from "next-i18next";
import { serverSideTranslations } from "next-i18next/serverSideTranslations";
const ContactUs = () => {
const { t } = useTranslation("common");
return (
<Fragment>
<Head>
<title>Contact-Us</title>
</Head>
</Fragment>
);
};
export const getStaticProps = async ({ locale }) => {
console.log(locale); // When I use the URL localhost: 3000/bize-ulasin, it returns undefined.
return {
props: {
...(await serverSideTranslations(locale, ["common"])),
},
};
};
export default ContactUs;
How can I access the "locale" variable with getStaticProps? Or, how can I use the following URLs with the same page file?
->localhost:3000/contact-us
->localhost:3000/bize-ulasin

I also faced the same problem today. That's how I solved the issue.
First of all, delete the server.js file. With Next.JS 10, using server.js will create conflict with the i18n routes and you won't be able to get the locale data in getStaticProps.
NextJS has a beautiful method named rewrites. We will use that instead of our server.js file. For example, if you have a page named contact-us-file, we can rewrite our next.config.js file as
const { i18n } = require('./next-i18next.config')
module.exports = {
i18n,
async rewrites() {
return [
{
source: '/contact-us',
destination: '/en/contact-us-file',
},
{
source: '/bize-ulasin',
destination: '/tr/contact-us-file',
},
]
},
}
As you are already using Next-i18next, I hope you are familiar with the file that I am importing.
Now If you try to navigate localhost:3000/contact-us and localhost:3000/bize-ulasin you should be able to access your contact us page.

NestJS Multer Amazon S3 issues uploading multiple files

I'm using NestJS, Node, and Express for my backend and Angular for my frontend. I have a stepper where the user steps through and enters in information about themselves as well as a profile photo and any photos of their art that they want to post (it's a rough draft). I'm sending the files to the backend with this code:
<h2>Upload Some Photos</h2>
<label for="singleFile">Upload file</label>
<input id="singleFile" type="file" [fileUploadInputFor]= "fileUploadQueue"/>
<br>
<mat-file-upload-queue #fileUploadQueue
[fileAlias]="'file'"
[httpUrl]="'http://localhost:3000/profile/artPhotos'">
<mat-file-upload [file]="file" [id]="i" *ngFor="let file of fileUploadQueue.files; let i = index"></mat-file-upload>
</mat-file-upload-queue>
The front-end sends the photos as an array of files; I tried to change it so that it just sent a single file but could not get it working. I'm less focused on that because the user may need to upload multiple files, so I want to figure it out regardless. On the backend, I'm using multer, multer-s3, and AWS-SDK to help upload the files however it isn't working. Here is the controller code:
#Post('/artPhotos')
#UseInterceptors(FilesInterceptor('file'))
async uploadArtPhotos(#Req() req, #Res() res): Promise<void> {
req.file = req.files[0];
delete req.files;
// tslint:disable-next-line:no-console
console.log(req);
await this._profileService.fileupload(req, res);
}
Here is ProfileService:
import { Profile } from './profile.entity';
import { InjectRepository } from '#nestjs/typeorm';
import { Repository } from 'typeorm';
import { ProfileDto } from './dto/profile.dto';
import { Req, Res, Injectable, UploadedFile } from '#nestjs/common';
import * as multer from 'multer';
import * as AWS from 'aws-sdk';
import * as multerS3 from 'multer-s3';
const AWS_S3_BUCKET_NAME = 'blah';
const s3 = new AWS.S3();
AWS.config.update({
accessKeyId: 'blah',
secretAccessKey: 'blah',
});
#Injectable()
export class ProfileService {
constructor(#InjectRepository(Profile)
private readonly profileRepository: Repository<Profile> ){}
async createProfile( profileDto: ProfileDto ): Promise<void> {
await this.profileRepository.save(profileDto);
}
async fileupload(#Req() req, #Res() res): Promise<void> {
try {
this.upload(req, res, error => {
if (error) {
// tslint:disable-next-line:no-console
console.log(error);
return res.status(404).json(`Failed to upload image file: ${error}`);
}
// tslint:disable-next-line:no-console
console.log('error');
return res.status(201).json(req.file);
});
} catch (error) {
// tslint:disable-next-line:no-console
console.log(error);
return res.status(500).json(`Failed to upload image file: ${error}`);
}
}
upload = multer({
storage: multerS3({
// tslint:disable-next-line:object-literal-shorthand
s3: s3,
bucket: AWS_S3_BUCKET_NAME,
acl: 'public-read',
// tslint:disable-next-line:object-literal-shorthand
key: (req, file, cb) => {
cb(null, `${Date.now().toString()} - ${file.originalname}`);
},
}),
}).array('upload', 1);
}
I haven't implemented any middleware extending multer, but I don't think I have to. You can see in the controller I erase the files property on req and replace it with the file where it's value is just the first member of the files array but that was just to see if it would work if I send it something it was expecting, but it did not work then. Does anyone have any ideas regarding how I can fix this? Or can anyone at least point me in the right direction with a link to a relevant tutorial or something?

My first guess would be that you are using the FileInterceptor and multer. I assume FileInterceptor adds multer in the controller which makes it available to the #UploadedFile decorator. Which could cause a conflict to your later use of multer. Try removing the interceptor and see if that fixes the issue.
Also I am attaching how I am doing file uploads. I am only uploading single images and I am using the AWS SDK so I don't have to work with multer directly, but here is how I am doing it, it might be helpful.
In the controller:
#Post(':id/uploadImage')
#UseInterceptors(FileInterceptor('file'))
public uploadImage(#Param() params: any, #UploadedFile() file: any): Promise<Property> {
return this.propertyService.addImage(params.id, file);
}
Then my service
/**
* Returns a promise with the URL string.
*
* #param file
*/
public uploadImage(file: any, urlKey: string): Promise<string> {
const params = {
Body: file.buffer,
Bucket: this.AWS_S3_BUCKET_NAME,
Key: urlKey
};
return this.s3
.putObject(params)
.promise()
.then(
data => {
return urlKey;
},
err => {
return err;
}
);
}

Thanks Jedediah, I like how simple your code is. I copied your code however it still wasn't working. Turns out you have to instantiate the s3 object after you update the config with your accesskey and secretID.