AWS EC2 IAM Role Credentials not passed into Node.js application - javascript

I am developing a Node.js application which is deployed on an EC2 instance. I am using the AWS SDK for JavaScript. I have tried both v2 and v3. My problem is that whenever I make a call to any AWS service in my application, I get an error saying that the credentials are missing. However, according to the documentation, assigning an IAM role to the EC2 instance should enable the SDK to automatically retrieve the credentials: AWS Documentation. I believe that I have correctly added the IAM role with sufficient permissions to the EC2 instance so I don't understand why the requests are not going through. I do not want to use Environment variables as I would have to manually use them in my code then. Any suggestions of how to debug this issue or thoughts what the problem might be, are greatly appreciated.
For example, a call is made as follows:
const client = new CloudFormationClient({ region: "eu-central-1" });
const params = {
StackStatusFilter: [
"CREATE_IN_PROGRESS"
]
};
const command = new ListStacksCommand(params);
client.send(command).then(
(data) => {
console.log(data);
},
(error) => {
console.log(error);
}
);
The error is simply: Error: Credential is missing.
This originates from the console.log(error).
In have tried with multiple roles but even with the AdministratorAccess the same error occurs. For reference, the permissions are:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}
]
}

Related

How to log in to cognito Google Oauth using Cypress?

I want to test an app that only has Google Oauth Login via AWS Cognito. Lots of guides on how to use cypress to programatically login to Cognito using AWS Amplify with a username and password, but cannot find anything on how to do it with Google Oauth.
Im trying to use cypress to click the buttons to authenticate but I think there is a click forgery protection on Google.
I have also been able to use this cypress documentation to login to Google directly and get a jwt into session storage but not sure if there is a way to pass this to Cognito.
If you're doing end to end testing, then the simplest way would be to have another non prod staging environment without the Google Oauth login in Cognito, and instead the username password login you mentioned that has working examples.
This is also a good idea, as you shouldn't be using your production user system in testing anyway.
I tried many approaches to this including getting oauth token from Google and trying to exchange that manually with Cognito for the token I needed for AWS API Gateway. I also tried cypress-social-logins without success (because of this bug)
Finally I just wrote my own cypress steps to log me into my app with a valid session. NOTE THIS WILL NOT WORK IN A CI/CD because it will probably trigger Google validations and so it is a semi-automated solution.
If in doubt I recommend Tom Roman's answer below about creating a pre-prod environment in cognito that allows username/password login instead of messing about with google login.
// /support/login.js
Cypress.Commands.add('loginByGoogle', () => {
cy.visit('http://localhost:3030')
cy.origin('https://somecognitouserpool.auth.eu-west-1.amazoncognito.com', () => {
cy.contains('button', 'Continue with Google')
.click({force: true})
})
cy.origin('https://accounts.google.com', () => {
const resizeObserverLoopError = /^[^(ResizeObserver loop limit exceeded)]/;
Cypress.on('uncaught:exception', (err) => {
/* returning false here prevents Cypress from failing the test */
if (resizeObserverLoopError.test(err.message)) {
return false;
}
});
cy.get('input#identifierId[type="email"]')
.type(Cypress.env('googleSocialLoginUsername'))
.get('button[type="button"]').contains('Next')
.click()
.get('div#password input[type="password"]')
.type(Cypress.env('googleSocialLoginPassword'))
.get('button[type="button"]').contains('Next')
.click();
});
});
// /e2e/sometest.cy.js
before(() => {
cy.loginByGoogle();
});
describe('E2E testing', () => {
it('should now have a session', () => {
})
});
You also need a .env file (because you don't want to be saving your google credentials into github)
GOOGLE_USERNAME = ''
GOOGLE_PASSWORD = ''
You also need two experimental flags (as of 14th Nov 2022)
// cypress.config.js
const { defineConfig } = require('cypress');
require('dotenv').config()
module.exports = defineConfig({
env: {
googleSocialLoginUsername: process.env.GOOGLE_USERNAME,
googleSocialLoginPassword: process.env.GOOGLE_PASSWORD
},
e2e: {
experimentalSessionAndOrigin: true,
experimentalModifyObstructiveThirdPartyCode: true
}
})
Here is my package.json so that you can see the exact packages I am using.
In particular I added the flags --headed --no-exit in order to complete 2 factor authentication manually as necessary. I have not yet figured out how to stop Google asking for this every time.
{
"name": "docs",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"start": "http-server . -p 3030",
"cy:run": "cypress run --headed --no-exit",
"test": "start-server-and-test start http://localhost:3030 cy:run"
},
"author": "",
"license": "ISC",
"devDependencies": {
"cypress": "^11.0.1",
"start-server-and-test": "^1.14.0"
},
"dependencies": {
"dot-env": "^0.0.1",
"dotenv": "^16.0.3"
}
}

Why is my Firestore/Firebase add document function working in production, but not local machine?

I'm following along a tutorial where I'm creating google cloud functions to add dummy data to google Firestore. When I test locally using Firebase Serve, I get an error "Could not load the default credentials" after sending a POST request to the localhost endpoint. However, when I test on production using Firebase Deploy, I get no error after sending a POST request to the production endpoint and the database entry is created successfully.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
let db = admin.firestore();
exports.createScream = functions.https.onRequest((req, res) => {
let scream = {
body: req.body.body,
userHandle: req.body.userHandle,
createdAt: admin.firestore.Timestamp.fromDate(new Date())
};
let addScream = db.collection('screams').add(scream)
.then(ref => {
res.json({message: `Successfully created scream at ${ref.id}`})})
.catch(err => {res.status(500).json({error: `${err}`})});
});
For you to access your firebase project you need to have credentials to your project. It works on the cloud because this configuration is already set up for you.
If you go to your firebase console project, you should go to settings => service accounts where you can download a json file with your credentials.
Now create an environment variable, that points to the file you just downloaded.
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"
This is in the official documentation

The incoming JSON object does not contain a client_email field

I'm trying to create a firebase cloud function. So I would to run my firebase cloud function locally.
But it do not work how to setup authentication.
I have installed firebase tools : https://firebase.google.com/docs/functions/local-emulator
I've runned the command firebase login, so now I'm logged.
Then I've created my json key with this tutorial : https://cloud.google.com/docs/authentication/getting-started
Now if I type echo $GOOGLE_APPLICATION_CREDENTIALS the result is /home/$USER/.google/****.json which contain
"project_id","private_key_id","private_key","client_email", "client_id", "auth_uri", "token_uri", "auth_provider_x509_cert_url", "client_x509_cert_url"
Also I've tried to install the full google cloud sdk and I runned : gcloud auth application-default login but no success.
Npm package versions :
"firebase-functions":"3.0.2"
"firebase-admin": "8.2.0"
I think I've provided enought information but feel free to ask me more if you want.
const functions = require("firebase-functions");
const admin = require("firebase-admin");
const express = require("express");
const app = express();
app.get("/", async (req, res) => {
admin.firestore().collection('something').get().then((collection) =>
return res.send({"count": collection.docs.length, "status": 200});
});
exports.exports = functions.https.onRequest(app);
the code is not important, the most important thing is that even I've done all theses steps, when I emulate my firebase locally with firebase serve and I trigger a function, I have this error :
Error: The incoming JSON object does not contain a client_email field
I can ensure you the json file contains client_email field.
Can you help me to authenticate with google ?
Thanks for your help.
I had a similar problem. It's likely a bug in version 7.0.2 of firebase-tools. I rolled back to version 7.0.0 and it works now.
So the temporary solution is:
npm i firebase-tools#7.0.0 -g
In short:
admin.initializeApp({ credential: admin.credential.applicationDefault() });
See docs for admin.credential.applicationDefault()
Update: Note that this is only recommended for testing/experimenting:
This strategy is useful when testing and experimenting, but can make
it hard to tell which credentials your application is using. We
recommend explicitly specifying which credentials the application
should use, ... Source
A little more info
I had the same when trying to call a firebase function locally which tries to update some documents in firestore database in batch. (Didn't test without batch).
To start calling firebase functions locally, I use:
firebase function:shell
As you probably know, this lists the available functions for your project.
I called my function and got the following error callstack:
Unhandled error Error: The incoming JSON object does not contain a client_email field
> at JWT.fromJSON (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\google-auth-library\build\src\auth\jwtclient.js:165:19)
> at GoogleAuth.fromJSON (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\google-auth-library\build\src\auth\googleauth.js:294:16)
> at GoogleAuth.getClient (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\google-auth-library\build\src\auth\googleauth.js:476:52)
> at GrpcClient._getCredentials (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\google-gax\build\src\grpc.js:107:40)
> at GrpcClient.createStub (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\google-gax\build\src\grpc.js:223:34)
> at new FirestoreClient (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\#google-cloud\firestore\build\src\v1\firestore_client.js:128:39)
> at ClientPool.Firestore._clientPool.pool_1.ClientPool [as clientFactory] (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\#google-cloud\firestore\build\src\index.js:315:26)
> at ClientPool.acquire (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\#google-cloud\firestore\build\src\pool.js:61:35)
> at ClientPool.run (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\#google-cloud\firestore\build\src\pool.js:114:29)
> at Firestore.readStream (D:\thdk\Projects\timesheets\functions\node_modules\firebase-admin\node_modules\#google-cloud\firestore\build\src\index.js:995:26)
RESPONSE RECEIVED FROM FUNCTION: 500, {
"error": {
"status": "INTERNAL",
"message": "INTERNAL"
}
}
I was running my function locally using the command line:
firebase functions:shell
I was using this code:
// Reference report in Firestore
const db = admin.firestore();
admin.initializeApp();
export const performMyCallableFirebaseFunction = (db, { from, to }) => {
return db.collection("collectionName").where("prop", "==", from).limit(500).get().then(snapshot => {
if (snapshot.empty) return new Promise(resolve => resolve(`No docs found with prop: ${from}`));
const batch = db.batch();
snapshot.forEach(doc => batch.update(doc.ref, { prop: to }));
return batch.commit();
});
};
exports.myCallableFirebaseFunction = functions.https.onCall(data => performMyCallableFirebaseFunction(db, data.from, data.to));
I changed the line
admin.initializeApp();
to
admin.initializeApp({ credential: admin.credential.applicationDefault() });
and now I was able to call my function locally using:
firebase functions:shell
firebase > myCallableFirebaseFunction({from: "foo", to: "bar"})
See docs for admin.credential.applicationDefault()
You probably need to set up the Firebase Admin SDK to use the Firebase emulator. You can do it by passing a credential property when calling the admin.initializeApp() method:
const serviceAccount = require('../serviceAccount.json');
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
});
You can download your service account JSON file in the Firebase console:
Click on the "settings" icon;
Go to "Users and permissions";
Click on the link where it says "N service accounts also have access to this project";
Click on the "Generate new private key" button.
Here is how I've solved the problem after struggling couple of hours:
Short answer:
Create Firebase-adminsdk key
How to do it:
Go to Google-cloud-platform > Service accounts https://console.cloud.google.com/iam-admin/serviceaccounts/
Select your project
Select your firebase-admin-sdk looks like firebase-adminsdk-u4k3i#example..
Enable edit mode
Create key and select JSON
You get the option to download a .json. Which has ProjectID, PrivateKey and ClientEmail in it
use the information like this where you initialize your app:
// Providing a service account object inline
admin.initializeApp({
credential: admin.credential.cert({
projectId: "<PROJECT_ID>",
clientEmail: "foo#<PROJECT_ID>.iam.gserviceaccount.com",
privateKey: "-----BEGIN PRIVATE KEY-----<KEY>-----END PRIVATE KEY-----\n"
})
});
Once you have created a Firebase project, you can initialize the SDK with an authorization strategy that combines your service account file together with Google Application Default Credentials.
To authenticate a service account and authorize it to access Firebase services, you must generate a private key file in JSON format.
To generate a private key file for your service account:
In the Firebase console, open Settings > Service Accounts.
Click Generate New Private Key, then confirm by clicking Generate Key.
Securely store the JSON file containing the key.
Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\service-account-file.json"
https://firebase.google.com/docs/admin/setup?authuser=3
I was getting this error when running firebase emulators:start.
As per the investigation from this bug: https://github.com/firebase/firebase-tools/issues/1451, it seems that this is an issue with referencing the app directly instead of via the admin module.
i.e. this causes the error:
const app = admin.initializeApp();
const firestore = app.firestore();
but this does not:
admin.initializeApp();
const firestore = admin.firestore();
However for the original question, you're using admin.firestore() so that wouldn't be the problem. It seems that admin.initializeApp() is never called. Perhaps that could be the cause of your issue?

How do i pass the user credential to access s3 bucket once user is authenticated by cognito?

I am using javascript sdk in node env and want to be able to assign a "key"/folder per user. I am able to authenticate and get a token with cognito. However with the following rules set I get a access denied. How do I actually pass the credentials on to access the users object?
Policy is as follows in IAM:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*",
"s3:Put*"
],
"Resource": [
"arn:aws:s3:::myBucket",
"arn:aws:s3:::myBucket/*"
],
"Condition": {
"StringLike": {
"s3:prefix": [
"${aws:username}/*"
]
}
}
}
]
}
Your policy is wrong.
Here is how to fix it:
Once you have cognito setup in AWS console, you will get two roles, auth cognito role and unauth. Go to the auth role in IAM and resource to look something like this...
"Resource": [
"arn:aws:s3:::mybucket/Users/${cognito-identity.amazonaws.com:sub}",]
The cognito identity: sub part is what ties the folder to the authenticated user.
Then in node, when you do put or get just use the same key but where this is -- ${cognito-identity.amazonaws.com:sub pass the identityID of the user. I am not sure how to get the identityID from AWS console, but you can definitely get it using the cli. Just check out the cognito api section to see how to get the identityID.
Hope that helps.

Firebase Cloud Functions - onCall not working

I'm testing a very simple implementation as described on FB docs (https://firebase.google.com/docs/functions/callable), and it's not working.
Here's my Firebase Function, deployed to cloud:
exports.getRecSkills = functions.https.onCall((data, context) => {
return {text: data.text};
});
...and my client call (after initializing FB):
var getRecSkills = firebase.functions().httpsCallable('getRecSkills');
getRecSkills({text: '123'}).then(function(result) {
console.log(result);
}).catch(function(error) {
console.log(error.code);
console.log(error.message);
});
I get a CORS header related issue but in the docs, it doesn't mention the need for CORS... am I missing something?
Some notes:
I've been able to execute other Firebase Functions (i.e. HTTPS,
Database) so I don't think it's me setting up Firebase wrong.
Updated to latest Firebase, so don't think that's an issue either.
Gives me an "internal" error, which the API docs aren't helpful, other than "something is seriously wrong".
I can't seem to get the function to work (it keeps giving me
400-errors) when testing locally via the shell, even though I got it
to work with any other database and https functions
Been struggling with this for quite some time... Please help!
To get rid of your CORS error, make sure your firebase.json has the following headers:
"hosting": [
{
"headers": [
{
"source": "**",
"headers": [
{
"key": "Access-Control-Allow-Origin",
"value": "*"
}
]
}
]
}
]
If you're running on Firebase Emulator on local device, make sure you have the following after initializing your Firebase Functions, otherwise your local device will still be calling the remote the Firebase Function and you'll hit the CORS error again:
if (window.location.hostname === "localhost") {
console.log("localhost detected!");
firebase.functions().useFunctionsEmulator('http://localhost:5001');
};
I had the same problem just recently but solved it after including my "projectId" in my config object. Below is a code snippet of the Firebase config object for Javascript. Make sure all fields have been filled in your config object and it should solve your undefined issue.
var config = {
apiKey: "<API_KEY>",
authDomain: "<PROJECT_ID>.firebaseapp.com",
databaseURL: "https://<DATABASE_NAME>.firebaseio.com",
projectId: "<PROJECT_ID>",
storageBucket: "<BUCKET>.appspot.com",
messagingSenderId: "<SENDER_ID>",
};
If you have CORS issues and you are using express in order to expose the API functions you have to allow cors:
import * as cors from 'cors';
import * as express from 'express';
const corsHandler = cors({origin: true});
const app = express();
app.use(corsHandler);
app.post('/createUser', async (request, response) => {
await createUser(request, response);
});
exports.api = functions.https.onRequest(app);

Categories