How can I load environment variables with ts-node? - javascript

I've tried a few implementations which none have been successful.
First Attempt
Using eval in package.json script "fetch:data": "eval $(cat .env) ts-node -O '{\"module\":\"commonjs\"}' ./bin/build-api-data.ts".
This results in a JSON parsing error because eval is removing my quotes for some reason.
undefined:1
{module:commonjs}
^
SyntaxError: Unexpected token m in JSON at position 1
Second Attempt
Using dotenv, the problem I encountered here was it was a race condition resulting in errors like this:
$ CANDID_ENV=local ts-node -O '{"module":"commonjs"}' ./bin/build-api-data.ts
/Users/lassiter.gregg/code/candidco-web/node_modules/contentful/dist/webpack:/contentful/contentful.js:49
throw new TypeError('Expected parameter accessToken')
^
TypeError: Expected parameter accessToken
Code Sample
import fs from 'fs';
import path from 'path';
import fetchApiData from '../lib/apiData';
import dotEnv from 'dotenv-safe';
const { CANDID_ENV } = process.env;
const isLocalBuild = CANDID_ENV === 'local';
console.log(dotEnv);
const API_DATA_FILENAME = 'api_data.json';
const ensureDirectoryExistence = filePath => {
var dirname = path.dirname(filePath);
if (fs.existsSync(dirname)) {
return true;
}
ensureDirectoryExistence(dirname);
fs.mkdirSync(dirname);
};
const writeData = (filename, data) => {
const filePath = path.join(__dirname, '..', '.data', filename);
ensureDirectoryExistence(filePath);
fs.writeFileSync(filePath, JSON.stringify(data));
console.log('API data stored', filePath);
};
const fetchAndStoreApiData = async () => {
console.log('Fetching all API data');
await dotEnv.config({
path: isLocalBuild ? './.env' : `./.env.${CANDID_ENV}`,
});
const newData = await fetchApiData();
writeData(API_DATA_FILENAME, newData);
};
const init = async () => {
fetchAndStoreApiData();
};
if (require.main === module) {
init();
}
In the case above, I've tried doing dotenv.config at the top of the file, in the init, in the function as you see. It always throws the same error about contentful not getting the env variable it needs. That said, if I log process.env and comment out the code relevant to fetchApiData then I see all my environment variables. That's why I think it's a race-time condition but haven't been able to find anything similar to my own issue.
Additionally, what makes this even more thorny is that this is a custom script that has to work in a node and esnext environment. So, I've had my fair share of thorny import/export issues using syntax I don't really prefer but haven't found away around it (e.g. export = someFunction).

Do I see it correctly, that you are trying to configure dotenv with a variable that you initialize with an env variable? I don't think that's going to work out.
Dotenv's work is to load the env variables to process.env. You have to config it as early as possible in your app.
More about it here: https://www.npmjs.com/package/dotenv

Related

How to get the current module's file path in a node module?

I want to print the correct filepath even if the function is imported in some other module inorder to handle the errors correctly. How can I do that? I am using serverless stack.
Please refer the following code,
class Logger {
filePath: string;
constructor(fp: string) {
filePath = fp;
}
printLog(info) {
const { timestamp, message } = info;
return `${timestamp} ${filePath}: ${message}`;
}
}
This is used in dbConnection.ts as,
const logger = new Logger(__filename);
export const connectToDB = () => {
try {
//DB connection code.
} catch(error) {
logger.print({ timestamp: new Date().toISOString(), message: error.message });
}
};
Now, I want to connect to db from some other module lets say, test.ts then I will use it as follows,
export const test = () => {
//some code here...
connectToDB();
}
When there occurs an error while connecting to DB, then It prints something like this,
2022-05-27T05:24:47.548Z src/test.ts: Error in connecting DB url is unreachable please check your internet connection.
In order to have proper debuggability, I want to print the filename from where the exception is actually thrown. That is src/dbConnection.ts and not src/test.ts.
Try using
__filename
__filename: This will return the path of the file executing
__dirname: This will return the path of the directory in which the file executing is located.
Check if it does what you need like
console.log(__filename);
Try to change filePath to this.filePath in your Logger Class

Sveltekit & Fleek (IPFS) import syntax problem?

I have managed to use fleek to update IPFS via straight javascript. I am now trying to add this functionality to a clean install of a svelteKit app. I think I am having trouble with the syntax around imports, but am not sure what I am doing wrong. When I click the button on the index.svelte I get the following error
Uncaught ReferenceError: require is not defined
uploadIPFS upload.js:3
listen index.mjs:412..........(I truncated the error here)
A few thoughts
I am wondering if it could be working in javascript because it is being called in node (running on the server) but running on the client in svelte?
More Details
The index.svelte file looks like this
<script>
import {uploadIPFS} from '../IPFS/upload'
</script>
<button on:click={uploadIPFS}>
upload to ipfs
</button>
the upload.js file looks like this
export const uploadIPFS = () => {
const fleek = require('#fleekhq/fleek-storage-js');
const apiKey = 'cZsQh9XV5+6Nd1+Bou4OuA==';
const apiSecret = '';
const data = 'pauls test load';
const testFunctionUpload = async (data) => {
const date = new Date();
const timestamp = date.getTime();
const input = {
apiKey,
apiSecret,
key: `file-${timestamp}`,
data
};
try {
const result = await fleek.upload(input);
console.log(result);
} catch (e) {
console.log('error', e);
}
};
testFunctionUpload(data);
};
I have also tried using the other import syntax and when I do I get the following error
500
global is not defined....
import with the other syntax is
import fleekStorage from '#fleekhq/fleek-storage-js';
function uploadIPFS() {
console.log('fleekStorage',fleekStorage)
};
export default uploadIPFS;
*I erased the api secret in the code above. In future I will store these in a .env file.
Even more details (if you need them)
The file below will update IPFS and runs via the command
npm run upload
That file is below. For my version that I used in svelte I simplified the file by removing all the file management and just loading a variable instead of a file (as in the example below)
const fs = require('fs');
const path = require('path');
const fleek = require('#fleekhq/fleek-storage-js');
require('dotenv').config()
const apiKey = process.env.FLEEK_API_KEY;
const apiSecret = process.env.FLEEK_API_SECRET;
const testFunctionUpload = async (data) => {
const date = new Date();
const timestamp = date.getTime();
const input = {
apiKey,
apiSecret,
key: `file-${timestamp}`,
data,
};
try {
const result = await fleek.upload(input);
console.log(result);
} catch(e) {
console.log('error', e);
}
}
// File management not used a my svelte version to keep it simple
const filePath = path.join(__dirname, 'README.md');
fs.readFile(filePath, (err, data) => {
if(!err) {
testFunctionUpload(data);
}
})

error sending request for url (https://deno.land/std/encoding/csv.ts) (os error 10013)

Unable to import the stdlib files from deno.land to local cache on running mod.ts.
error: error sending request for url (https://deno.land/std/encoding/csv.ts): error trying to connect: tcp connect error: An attempt was made to access
a socket in a way forbidden by its access permissions. (os error 10013)
Imported from "file:///C:/Current_Tasks/Deno/Kepler/mod.ts:3"
Is there anything additional that needs to be enabled to import these files?
import { join } from "https://deno.land/std/path/mod.ts";
import { BufReader } from "https://deno.land/std/io/bufio.ts";
import { parse } from "https://deno.land/std/encoding/csv.ts";
async function loadPlanetsData() {
const path = join(".", "test.csv");
const file = await Deno.open(path);
const bufReader = new BufReader(file);
const result = await parse(bufReader, {
header: true,
comment: "#",
});
Deno.close(file.rid);
console.log(result);
}
await loadPlanetsData();
Update: Used
deno run --allow-read mod.ts
import { join } from "https://deno.land/std/path/mod.ts";
import { BufReader } from "https://deno.land/std/io/bufio.ts";
import { parse } from "https://deno.land/std/encoding/csv.ts";
async function loadPlanetsData() {
const path = join(".", "test.csv");
const file = await Deno.open(path);
const bufReader = new BufReader(file);
const result = await parse(bufReader, {
header: true,
comment: "#",
});
Deno.close(file.rid);
console.log(result);
}
await loadPlanetsData();
While running this file you need to give read access to the Deno.
Deno is secure by default. Therefore, unless you specifically enable it, a deno module has no file, network, or environment access for example. Access to security sensitive areas or functions requires the use of permissions to be granted to a deno process on the command line.
For the following example, mod.ts has been granted read-only access to the file system. It cannot write to it, or perform any other security sensitive functions.
deno run --allow-read mod.ts

Add new extension to nodejs dynamic import

How do I add an new file extension to Nodejs dynamic import?
I want to add my own filetype, lets call it .jszip. (No, this is just an example and what I actually want has nothing to do with zip).
Say I have
package.json:
{
"name": "test",
"scripts": {
"zip": "node --experimental-modules test.js"
}
}
test.js:
const fs = require('fs');
const Module = require('module');
function loadJsZip(module, filename) {
console.log('In loadJsZip');
const content = fs.readFileSync(filename, 'utf8');
// Do something to content
module._compile(content, filename);
}
require.extensions['.jszip'] = loadJsZip;
Module._extensions['.jszip'] = loadJsZip;
function loadJs(relativePath) {
import(f).then((module) => {
console.log(`imported from ${filename}:${module}`);
}).catch((err) => {
console.log(`While importing:${err}`);
});
}
loadJs('./testfile.jszip');
I am getting:
(node:20412) ExperimentalWarning: The ESM module loader is experimental.
While importing:TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension: c:\...\testfile.jszip
It seems other file types are not supported: https://nodejs.org/api/esm.html#esm_import_statements
What worked for my case is getting the normal require and using that. So I'm importing .graphql files using:
import {createRequire} from 'module';
const require = createRequire(import.meta.url);
require('graphql-import-node/register');
const myQuery = require('./myquery.graphql');
The graphql-import-node package does a require.extension[] = behind the scenes:
require.extensions['.graphql'] = loadPlainFile
But this is starting to become madness. You are probably better off using Webpack or something.

Testcafe - Test command line argument outside test case

As I'm getting familiar with Testcafe, I'm trying to use a command line argument to give the user more information on how to run tests. For that reason, I'm using the minimist package.
However, I cannot print or use any variables outside the test cases. Please find below my code.
import { Selector } from 'testcafe';
import minimist from 'minimist';
const args = minimist(process.argv.slice(2));
const env = args.env;
console.log('*** A SAMPLE CONSOLE OUTPUT ***'); // does not print
fixture `Getting Started`
.page `http://devexpress.github.io/testcafe/example`;
test('My first test', async t => {
console.log('*** ANOTHER SAMPLE CONSOLE OUTPUT ***'); // prints
await t
.typeText('#developer-name', 'John Smith')
.wait(1000)
.click('#submit-button')
// Use the assertion to check if the actual header text is equal to the expected one
.expect(Selector('#article-header').innerText).eql('Thank you, John Smith!');
});
I want to write an if statement that checks if env === '' or use a default argument.
How can I accomplish this?
However, I cannot print or use any variables outside the test cases.
 
Please use a programming way to run TestCafe.
I've changed you code example (test.js) and created a file that runs TestCafe programmatically (run.js).
Put these files into a folder and perform command 'node run.js --env value' in your terminal.
Then you will see the following output:
'*** A SAMPLE CONSOLE OUTPUT ***'
Getting Started
value
test.js
import { Selector } from 'testcafe';
import minimist from 'minimist';
const args = minimist(process.argv.slice(2));
const env = args.env;
console.log('*** A SAMPLE CONSOLE OUTPUT ***');
fixture `Getting Started`
.page `http://devexpress.github.io/testcafe/example`;
test('My first test', async t => {
console.log(env); // prints
await t
.typeText('#developer-name', 'John Smith')
.wait(1000)
.click('#submit-button')
.expect(Selector('#article-header').innerText).eql('Thank you, John Smith!');
});
run.js
const createTestCafe = require('testcafe');
let runner = null;
createTestCafe('localhost', 1337, 1338, void 0, true)
.then(testcafe => {
runner = testcafe.createRunner();
})
.then(() => {
return runner
.src('test.js')
.browsers('chrome')
.run()
.then(failedCount => {
console.log(`Finished. Count failed tests:${failedCount}`);
process.exit(failedCount)
});
})
.catch(error => {
console.log(error);
process.exit(1);
});
A solution to accomplish this is:
1) Create a separate config.js file that will handle your custom command-line options:
import * as minimist from 'minimist';
const args = minimist(process.argv.slice(2));
// get the options --env=xxx --user=yyy from the command line
export const config = {
env: args.env,
user: args.user,
};
2) In you test file:
remove any code outside the fixture and the test methods.
import the config file and inject it in the TestController context
get the command args via the TestController context
import 'testcafe';
import { Selector } from 'testcafe';
import { config } from './config';
fixture('Getting Started')
.beforeEach(async (t) => {
// inject config in the test context
t.ctx.currentConfig = config;
});
test('My first test', async (t) => {
// retrieve cli args from the test context
const currentConfig = t.ctx.currentConfig;
console.log(`env=${currentConfig.env}`);
});

Categories