We Keep Coding

Pulumi, how to export values coming from async function?

In my Pulumi project, in the index.ts file I have to call
const awsIdentity = await aws.getCallerIdentity({ async: true });
So for this reason I have to wrapp all my code into async function.
My problem is with exported variables at the end of the file.
async function go() {
...
const awsIdentity = await aws.getCallerIdentity({ async: true });
const accountId = awsIdentity.accountId;
...
return {
dnsZoneName: DNSZone.name,
BucketID: Bucket.id,
dbHardURL: DBHost.publicDns,
devDbURL: publicDbAddress.fqdn,
};
}
I want to export these 4 values.
I can't understand how, but the code that follows exports( at least, pulumi up shows the values at the end of execution).
const result = go();
export const dnsZoneName = result.then((res) => res.dnsZoneName);
look at this
I think I can't use top-level-await.
What is the clear solution ?

For anyone who comes across this post, asynchronous entry points are now a first class citizen in Pulumi, as detailed Here
You can have code that looks similar to this and Pulumi will automatically resolve everything for you, no need to do anything more for exporting Outputs
export async function go() {
...
const awsIdentity = await aws.getCallerIdentity({ async: true });
const accountId = awsIdentity.accountId;
...
return {
dnsZoneName: DNSZone.name,
BucketID: Bucket.id,
dbHardURL: DBHost.publicDns,
devDbURL: publicDbAddress.fqdn,
};
}

From the issue you linked, it seems like my first suggestion in my answer to your previous question should work: Export a promise for each value. Based on the issue comments, it looks like Pulumi understands exported promises.
async function go() {
...
const awsIdentity = await aws.getCallerIdentity({ async: true });
const accountId = awsIdentity.accountId;
...
return {
dnsZoneName: DNSZone.name,
BucketID: Bucket.id,
dbHardURL: DBHost.publicDns,
devDbURL: publicDbAddress.fqdn,
};
}
const goPromise = go();
goPromise.catch(error => {
// Report the error. Note that since we don't chain on this, it doesn't
// prevent the exports below from rejecting (so Pulumi will see the error too,
// which seems best).
});
export const dnsZoneName = goPromise.then(res => res.DNSZone.name);
export const BucketID = goPromise.then(res => res.Bucket.id);
export const dbHardURL = goPromise.then(res => res.DBHost.publicDns);
export const devDbURL = goPromise.then(res => res.publicDbAddress.fqdn);
Otherwise:
You've said you don't think you can use top-level await, but you haven't said why.
In case it's just that you're having trouble figuring out how to use it, you'd do it like this provided aws.getCallerIdentity and whatever's in the "..." of your code example provide promises:
const awsIdentity = await aws.getCallerIdentity({ async: true });
const accountId = awsIdentity.accountId;
// ...
export const dnsZoneName = DNSZone.name;
export const BucketID = Bucket.id;
export const dbHardURL = DBHost.publicDns;
export const devDbURL = publicDbAddress.fqdn;
Or if you need to export an object with those as properties as a default export:
const awsIdentity = await aws.getCallerIdentity({ async: true });
const accountId = awsIdentity.accountId;
// ...
export default {
dnsZoneName: DNSZone.name
BucketID: Bucket.id
dbHardURL: DBHost.publicDns
devDbURL: publicDbAddress.fqdn
};
Note that in both cases, the code isn't inside any function, that's at the top-level of your module.
With Node.js v13 and v14 (so far) you need the --harmony-top-level-await runtime flag. My guess is that it won't be behind a flag in v15 (or possibly even just a later version of v14).

The most straight-forward way I found is to map getCallerIdentity to an Output this way
import * as pulumi from "#pulumi/pulumi";
import * as aws from "#pulumi/aws";
const account = aws.getCallerIdentity({/* async = true (by default) */});
const bucket = new aws.s3.Bucket("bucket", {
bucket: pulumi.output(account).apply(acc => `my-bucket-${acc.accountId}`),
});
This way you don't have to wrap all your code in a top-level async function, which is somewhat undesirable if you need it only to get the result of that function.

You are working with promises. Therefore, the answer can come at any time. What you should do is that when you call the function you do it with await to wait for the function response

Import data from async function

I'm dealing with a project that uses AWS Cognito. There are some configuration params that needs to be fetched from server with an API call. I keep the API call in a config.js file and use async/await to get response from server like this
const getCognitoConfigs = async () => {
const res = await axios.get(`${apiurl.apiurl}/logininfo`);
console.log(res.data);
return res.data;
};
export default getCognitoConfigs;
And in my index.js (where I set up Cognito), I import the function from config.js file
import getCognitoConfigs from "./config";
const configs = getCognitoConfigs();
Amplify.configure({
Auth: {
mandatorySignIn: true,
region: configs.cognito.region,
userPoolId: configs.cognito.user_pool,
userPoolWebClientId: configs.cognito.app_client_id
}
});
The problem is async await does not stop the program execution so I'm getting 'configs' as undefined. Are there anyways that I can make the app stop until the api call has resolved? Thanks.

If you want to use async/await, you have to wrap index.js in an asynchronous function and add
await getCognitoConfigs();
or you can use promise like
getCognitoConfigs().then(res => Amplify.configure({...}))

How do I mock the imports my tested class makes with jest?

I'm trying to setup tests for my database crawler program and I can't manage to replace what the class method I'm testing imports.
So as not to write down too much code I'll just lay out the general form of the problem. In my test function I have:
describe("test",()=>{
let result1;
beforeAll(async ()=>{
await createConnection();
})
afterAll(async ()=>{
getConnection().close();
})
test("setup test",async () => {
result1 = await WeatherController.startForecastAPI();
expect(result1.status).toBe(Status.SUCCESS);
})
})
The WeatherController.ts file (... where code was taken out):
...
import AccessTokenService from '../services/AccessTokenService';
export default class WeatherController{
...
static async startForecastAPI(){
...
const accessToken = AccessTokenService.getAccessToken();//get and validate token
...
}
}
Inside the WeatherController class, startForecastAPI is defined as a static async method. The class imports multiple other classes, among them the AccessTokenService class which is used to get valid access tokens. AccessTokenService.getAccessToken() should return an object with several properties that it gets through a http request.
I want to mock the results of calling AccessTokenService but I'm not calling it directly in my test function, I'm calling WeatherController and WeatherController is calling AccessTokenService. How can I replace what WeatherController calls when I test it but without touching the WeatherController code? I've tried going through the jest docs but I'm fairly new to all of this and they're confusing. I'm not entirely clear how scoping works here either (I tried defining a function in the test code and calling it in the tested function but it's out of scope).
The await WeatherController.startForecastAPI() call in the test function returns undefined but the code works fine when I hard-code accessToken to be a valid object, I just can't find a way to inject that object into the code through the test function.

Assuming AccessTokenService.getAccessToken returns a promise or is an async function, then you can use jest.spyOn(...).mockResolvedValue() to prevent calling the server
describe("test",()=>{
let result1;
beforeAll(async ()=>{
await createConnection();
})
afterAll(async ()=>{
getConnection().close();
})
test("setup test",async () => {
const expectedResultFromGetToken = {property: 'property 1'};
const getTokenSpy = jest.spyOn(AccessTokenService, 'getAccessToken')
.mockResolvedValue(expectedResultFromGetToken)
result1 = await WeatherController.startForecastAPI();
expect(result1.status).toBe(Status.SUCCESS);
expect(getTokenSpy).toHaveBeenCalled()
})
})
if the AccessTokenService.getAccessToken is not an async function then you have to use jest.spyOn(...).mockReturnValue()

If inside your class you have
const AccessToken = require('access-token');
you can mock it with
jest.mock('access-token', () => {
function getToken() {
return 'fakeToken'
}
);
const WeatherController = require('weather-controller');
describe("test",()=>{
let result1;
beforeAll(async ()=>{
await createConnection();
})
afterAll(async ()=>{
getConnection().close();
})
test("setup test",async () => {
result1 = await WeatherController.startForecastAPI();
expect(result1.status).toBe(Status.SUCCESS);
})
})

What is the state of the art for testing/mocking functions within a module in 2018?

I have a module, for the purposes of learning testing, that looks like this:
api.js
import axios from "axios";
const BASE_URL = "https://jsonplaceholder.typicode.com/";
const URI_USERS = 'users/';
export async function makeApiCall(uri) {
try {
const response = await axios(BASE_URL + uri);
return response.data;
} catch (err) {
throw err.message;
}
}
export async function fetchUsers() {
return makeApiCall(URI_USERS);
}
export async function fetchUser(id) {
return makeApiCall(URI_USERS + id);
}
export async function fetchUserStrings(...ids) {
const users = await Promise.all(ids.map(id => fetchUser(id)));
return users.map(user => parseUser(user));
}
export function parseUser(user) {
return `${user.name}:${user.username}`;
}
Pretty straight forward stuff.
Now I want to test that fetchUserStrings method, and to do that I want to mock/spy on both fetchUser and parseUser. At the same time - I don't want the behaviour of parseUser to stay mocked - for when I'm actually testing that.
I run in the problem that it seems that it is not possible to mock/spy on functions within the same module.
Here are the resources I've read about it:
How to mock a specific module function? Jest github issue. (100+ thumbs up).
where we're told:
Supporting the above by mocking a function after requiring a module is impossible in JavaScript – there is (almost) no way to retrieve the binding that foo refers to and modify it.
The way that jest-mock works is that it runs the module code in isolation and then retrieves the metadata of a module and creates mock functions. Again, in this case it won't have any way to modify the local binding of foo.
Refer to the functions via an object
The solution he proposes is ES5 - but the modern equivalent is described in this blog post:
https://luetkemj.github.io/170421/mocking-modules-in-jest/
Where, instead of calling my functions directly, I refer to them via an object like:
api.js
async function makeApiCall(uri) {
try {
const response = await axios(BASE_URL + uri);
return response.data;
} catch (err) {
throw err.message;
}
}
async function fetchUsers() {
return lib.makeApiCall(URI_USERS);
}
async function fetchUser(id) {
return lib.makeApiCall(URI_USERS + id);
}
async function fetchUserStrings(...ids) {
const users = await Promise.all(ids.map(id => lib.fetchUser(id)));
return users.map(user => lib.parseUser(user));
}
function parseUser(user) {
return `${user.name}:${user.username}`;
}
const lib = {
makeApiCall,
fetchUsers,
fetchUser,
fetchUserStrings,
parseUser
};
export default lib;
Other posts that suggest this solution:
https://groups.google.com/forum/#!topic/sinonjs/bPZYl6jjMdg
https://stackoverflow.com/a/45288360/1068446
And this one seems to be a variant of the same idea:
https://stackoverflow.com/a/47976589/1068446
Break the object into modules
An alternative, is that I would break my module up, such that I'm never calling functions directly within each other.
eg.
api.js
import axios from "axios";
const BASE_URL = "https://jsonplaceholder.typicode.com/";
export async function makeApiCall(uri) {
try {
const response = await axios(BASE_URL + uri);
return response.data;
} catch (err) {
throw err.message;
}
}
user-api.js
import {makeApiCall} from "./api";
export async function fetchUsers() {
return makeApiCall(URI_USERS);
}
export async function fetchUser(id) {
return makeApiCall(URI_USERS + id);
}
user-service.js
import {fetchUser} from "./user-api.js";
import {parseUser} from "./user-parser.js";
export async function fetchUserStrings(...ids) {
const users = await Promise.all(ids.map(id => lib.fetchUser(id)));
return ids.map(user => lib.parseUser(user));
}
user-parser.js
export function parseUser(user) {
return `${user.name}:${user.username}`;
}
And that way I can mock the dependency modules when I'm testing the dependant module, no worries.
But I'm not sure that breaking up the modules like this is even feasible - I imagine that there might be a circumstance where you have circular dependencies.
There are some alternatives:
Dependency injection in the function:
https://stackoverflow.com/a/47804180/1068446
This one looks ugly as though, imo.
Use babel-rewire plugin
https://stackoverflow.com/a/52725067/1068446
I have to admit - I haven't looked at this much.
Split your test into multiple files
Am investigating this one now.
My question: This is all quite a frustrating and fiddly way of testing - is there a standard, nice and easy, way people are writing unit tests in 2018, that specifically solve this issue?

As you've already discovered attempting to directly test an ES6 module is extremely painful. In your situation it sounds like you are transpiling the ES6 module rather than testing it directly, which would likely generate code that looks something like this:
async function makeApiCall(uri) {
...
}
module.exports.makeApiCall = makeApiCall;
Since the other methods are calling makeApiCall directly, rather than the export, even if you tried to mock the export nothing would happen. As it stands ES6 module exports are immutable, so even if you did not transpile the module you would likely still have issues.
Attaching everything to a "lib" object is probably the easiest way to get going, but it feels like a hack, not a solution. Alternatively using a library that can rewire the module is a potential solution, but its extremely hokey and in my opinion it smells. Usually when you're running into this type of code smell you have a design problem.
Splitting up the modules into tiny pieces feels like a poor mans dependency injection, and as you've stated you'll likely run into issues quickly. Real dependency injection is probably the most robust solution, but it's something you need to build from the ground up, it's not something that you can just plug into an existing project and expect to have things working immediately.
My suggestion? Create classes and use them for testing instead, then just make the module a thin wrapper over an instance of the class. Since you're using a class you'll always be referencing your method calls using a centralized object (the this object) which will allow you to mock out the things you need. Using a class will also give you an opportunity to inject data when you construct the class, giving you extremely fine grained control in your tests.
Let's refactor your api module to use a class:
import axios from 'axios';
export class ApiClient {
constructor({baseUrl, client}) {
this.baseUrl = baseUrl;
this.client = client;
}
async makeApiCall(uri) {
try {
const response = await this.client(`${this.baseUrl}${uri}`);
return response.data;
} catch (err) {
throw err.message;
}
}
async fetchUsers() {
return this.makeApiCall('/users');
}
async fetchUser(id) {
return this.makeApiCall(`/users/${id}`);
}
async fetchUserStrings(...ids) {
const users = await Promise.all(ids.map(id => this.fetchUser(id)));
return users.map(user => this.parseUser(user));
}
parseUser(user) {
return `${user.name}:${user.username}`;
}
}
export default new ApiClient({
url: "https://jsonplaceholder.typicode.com/",
client: axios
});
Now lets create some tests for the ApiClient class:
import {ApiClient} from './api';
describe('api tests', () => {
let api;
beforeEach(() => {
api = new ApiClient({
baseUrl: 'http://test.com',
client: jest.fn()
});
});
it('makeApiCall should use client', async () => {
const response = {data: []};
api.client.mockResolvedValue(response);
const value = await api.makeApiCall('/foo');
expect(api.client).toHaveBeenCalledWith('http://test.com/foo');
expect(value).toBe(response.data);
});
it('fetchUsers should call makeApiCall', async () => {
const value = [];
jest.spyOn(api, 'makeApiCall').mockResolvedValue(value);
const users = await api.fetchUsers();
expect(api.makeApiCall).toHaveBeenCalledWith('/users');
expect(users).toBe(value);
});
});
I should note that I have not tested if the provided code works, but hopefully the concept is clear enough.

exporting after promise finishes

I would like to export a class which initial state depends on a value returned from a Promise in another module i cannot modify.
Here's the code:
let e = true;
APromiseFromAnotherModule()
.then(value => return value;);
export default class E {
constructor() {
if (e) {
//...
} else {
//...
}
}
}
I also tried with async/await encapsulating the Promise into an async function like this:
let e = true;
getInitialValue = async () => {
return await APromiseFromAnotherModule()
.then(value => e = value;);
};
e = getInitialValue();
export default class E {
constructor() {
if (e) {
//...
} else {
//...
}
}
}
But it doesn't make sense because that one is an async function so obviously it doesn't work.
What am I missing?

module exports are done synchronously. So, they cannot depend upon the results of an asynchronous operation.
And, await only works inside a function. It doesn't actually block the containing function (the containing function returns a promise) so that won't help you make an async operation into a synchronous one either.
The usual ways to deal with a module that uses some async code in its setup is to either export a promise and have the calling code use .then() on the promise or to initialize the module with a constructor function that returns a promise.
The code is only pseudo code so it's hard to tell exactly what your real problem is to show you specific code for your situation.
As an example. If you want to export a class definition, but don't want the class definition used until some async code has completed, you can do something like this:
// do async initialization and keep promise
let e;
const p = APromiseFromAnotherModule().then(val => e = val);
class E {
constructor() {
if (e) {
//...
} else {
//...
}
}
};
// export constructor function
export default function() {
return p.then(e => {
// after async initialization is done, resolve with class E
return E;
});
}
The caller could then use it like this:
import init from 'myModule';
init().then(E => {
// put code here that uses E
}).catch(err => {
console.log(err);
// handle error here
});
This solves several issues:
Asynchronous initialization is started immediately upon module loading.
class E is not made available to the caller until the async initialization is done so it can't be used until its ready
The same class E is used for all users of the module
The async initialization is cached so it's only done once
Edit in 2023. Modern nodejs versions when using ESM modules have top level await so it is possible to await an asynchronous result before your exports.

I understand that #jfriend00's answer works fine, but in my case, there is not just one block of code that depends on the foreign async function's completion.
In my app, configuration loading is asynchronous, so the server must wait for the configs to load before starting. Since there are other files (like routes) that compose the server that also need access to the configs, I would have to reluctantly call .then() in each other file.
Here is how I was able to do it with require statements instead of export.
config.js
Module that can be required by other modules in order to gain access to the global configs.
module.exports.loadAllConfigs = async () => {
const appConfigs = await AsyncConfigLibrary.load();
module.exports.CONFIG = appConfigs;
};
server.js
Main file for Node application that makes use of other modules that require access to global configs.
const { loadAllConfigs } = require('./modules/config');
loadAllConfigs()
.then(() => {
const { CONFIG } = require('./modules/config');
/* import modules */
const auth = require('./modules/auth');
};
auth.js
Module used by server.js that requires access to configs.
const { CONFIG } = require('./config');
const cookieSecret = CONFIG.secretItem;
Therefore, as long as the CONFIG property is set in config.js before any of the other modules attempt to access it, that is, before the modules are require'd, then the single .then() in server.js is sufficient.