We Keep Coding

Send data securely NODEjs - REST API

I would like to send encrypted / secure data that I send to my REST API.
However, I encounter difficulties here and would be very happy about external help.
Please note that the code is simplified for better illustration!
Here is the basic structure of the REST API in order to be able to reveal errors / problems of understanding on my part:
Sending the data:
First, I encrypt the relevant data:
const crypto = require('crypto')
const algorithm = 'aes-256-ctr'
const secretKey = 'somesecret'
const iv = crypto.randomBytes(16)
const encrypt = (t) => {
const cipher = crypto.createCipheriv(algorithm, secretKey, iv)
const encrypted = Buffer.concat([cipher.update(t), cipher.final()])
return {
iv: iv.toString('hex'),
content: encrypted.toString('hex')
}
}
const regdata = crypto.encrypt(password)
//some stuff --> sending data
I then send the data, including the iv and the data to be encrypted, to my API.
function apicall(data, Method) {
data = Object.assign(data, {
apisecret: process.env.REACT_APP_APISECRET,
cryptokey: process.env.REACT_APP_CRYPTOKEY
})
const mydata = fetch(`https://somepublic.url:4400/defined/query/${JSON.stringify(data)}`, {
method: Method
})
const jdata = JSON.parse(tdata)
//some stuff
return jdata
}
My API looks something like this:
//require some stuff
app.use(express.json())
app.use(function(req, res, next) {
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Methods', 'GET,POST,PUT,DELETE,OPTIONS');
res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Access-Control-Allow-Headers');
next();
});
// ** Select SQL Data
app.get('/defined/query/:data', (req, res) => {
const data = JSON.parse(req.params.data)
if ((data.apisecret != process.env.NODE_APP_APISECRET) || (data.cryptokey != process.env.NODE_APP_CRYPTOKEY)) {
throw "Authentication Required"
}
//do some stuff with encrypted Data
.then(response => {
res.status(200).send(response);
})
.catch(error => {
res.status(500).send(error);
})
})
It all works, I just don't quite understand what encrypting the data does for me if it can be read out in the URL.
If someone uses my APP and reads the full URL with Whireshark & Co., they can
read the "process.env.NODE_APP_APISECRET" named by me and thus has access to my API
Although this does not have my "secret key", it can read out the "iv", which would also like to be prevented.
However, I have to send the "iv" in the URL, since the data is only decrypted afterwards.
Did I make a mistake in my setup, or generally have a problem understanding the API interface?

You need to consider who you are trying to stop from intercepting the data.
If the goal is to stop third parties intercepting the data between the client and the server then use HTTPS, only use HTTPS, do not roll your own encryption on top of it.
If your goal is to let the client store data on the server without people who have access to the server being able to decrypt it then encrypt it on the client, and do not send the keys to the server.
If your goal is to let multiple clients exchange data without the server being able to decrypt it, then generate keys on each client, share public keys between clients and keep private keys private. Then each client should encrypt the data with the recipient's public key (which can only be decrypted by the recipient's private key).

How to stop nestjs from sending 200 response before completing task in service when using google identity

I am using google-auth-library to verify the token received from gsi client on my Nestjs server. After the token has been verified, I want to send the returned object from verifyIdToken method back to the frontend as a response. I can verify the JWT token through verifyIdToken just fine but the nestjs response handler sends the 200 OK status response before I can send the returned object from verifyIdToken.
I want to send the object returned from verifyIdToken method before sending the 200 OK response.
Here's the code from google docs -
const {OAuth2Client} = require('google-auth-library');
const client = new OAuth2Client(CLIENT_ID);
async function verify() {
const ticket = await client.verifyIdToken({
idToken: token,
audience: CLIENT_ID, // Specify the CLIENT_ID of the app that accesses the backend
// Or, if multiple clients access the backend:
//[CLIENT_ID_1, CLIENT_ID_2, CLIENT_ID_3]
});
//^---------- Nestjs sends 200 OK response after this line and then continues afterwards
const payload = ticket.getPayload();
//^---------- I want to send this payload back to the frontend.
const userid = payload['sub'];
// If request specified a G Suite domain:
// const domain = payload['hd'];
}
verify().catch(console.error);
I want to send the 200 OK response along with payload variable as response body.
Also I don't really understand what verifyIdToken is trying to do other than decoding jwt. Is it trying to maintain some session? I can decode the jwt through other means and send the object back to the frontend but I don't think that's a very good idea. As it defeats the whole purpose of google sign in verification from google servers.
This is what verifyIdToken calls internally in oauth2client.js -
verifyIdToken(options, callback) {
// This function used to accept two arguments instead of an options object.
// Check the types to help users upgrade with less pain.
// This check can be removed after a 2.0 release.
if (callback && typeof callback !== 'function') {
throw new Error('This method accepts an options object as the first parameter, which includes the idToken, audience, and maxExpiry.');
}
if (callback) {
this.verifyIdTokenAsync(options).then(r => callback(null, r), callback);
}
else {
return this.verifyIdTokenAsync(options);
}
}
Which in return calls -
async verifyIdTokenAsync(options) {
if (!options.idToken) {
throw new Error('The verifyIdToken method requires an ID Token');
}
const response = await this.getFederatedSignonCertsAsync();
//^------- This is where the Nestjs response handler actually returns the 200 OK response prematurely
const login = await this.verifySignedJwtWithCertsAsync(options.idToken, response.certs, options.audience, OAuth2Client.ISSUERS_, options.maxExpiry);
return login;
}
I tried passing callback to verifyIdToken but the problem still remains.
Edit - This is what google docs say about verifyIdToken -
The verifyIdToken function verifies the JWT signature, the aud claim, the exp claim, and the iss claim.

How to sync pouchdb with remote Coudant, not able to connect?

How do sync PoucDB to Cloudant by using user login? My Cloudant is set up with IAM and Legacy.
How do I register users and use the credentials to login to use Cloudant and PouchDB
I have not found answers in Cloudant, PouchDB docs. And do not see my answer in Stackoverflow
I am using the PouchDB docs:
var db = new PouchDB('todos');
const remoteUrl = "https://some address"
const remoteOptions = {
'auth': { 'username': user , 'password': pass } }
const remoteDb = new PouchDB(remoteUrl, remoteOptions);
localDB.sync(remoteDB).on('complete', function () {
// yay, we're in sync!
}).on('error', function (err) {
// boo, we hit an error!
});
Replication does work from Cloudant to on-premise Couchdb. Therefore I can connect to Coudant remotely.

As you have Legacy Auth available, I won't worry about IAM authentication. We can simply generate an API key/password for your mobile client and use legacy authentication.
Creating an API key
In the Cloudant Dashboard, choose the "Permissions" menu and click the "Generate API Key" button. You should see a generated Key and Password - these become the Username & Password in the URL you feed to PouchDB.
If you're doing a sync (i.e data is to flow in both directions) then your API key needs both _reader & _writer permissions. If data is to flow from server to mobile only, then _reader & _replicator will suffice.
Client-side code
Your client side code is pretty much correct:
remember that Cloudant only accepts HTTPS connections
the Key from your API Key/Password pair becomes your username
the Password from your API Key/Password pair becomes your password
e.g.
const PouchDB = require('pouchdb')
const db = new PouchDB('todos')
const username = 'KEY'
const password = 'PASSWORD'
const host = 'mycloudservice.cloudant.com'
const databaseName = 'todos'
const remoteDB = `https://${username}:${password}#${host}/${databaseName}`
db.sync(remoteDB).on('complete', async function () {
console.log('done')
}).on('error', function (err) {
console.error(err)
});

url signature missing or invalid getstream

var client = stream.connect('my-client-id', null, '7723');
var user1 = client.feed('flat', 'my-client-id', 'NuAW6yHVQ2sr9RQvBE-cCuewUlo'); // What is this token param (3rd one)? How is this generated?
var acticity = {
actor: 'QUXdERFPto',
tweet: 'Hello world',
verb: 'tweet',
object: 1
}
user1.addActivity(acticity).then(null).catch(function(e) {
// Error object is
// code: null
// detail: "url signature missing or invalid"
// duration: "10ms"
// exception: "AuthenticationFailed"
// status_code: 403
});
What is the signature that I am missing?

Stream-JS client-side feed tokens
When using the stream-js library on the client you should initiate the connect without your secret key, to avoid sharing your private key with the world (its secret).
var client = stream.connect('api-key', null, 'app-id');
Initiating the client this way does not allow you to read or write from any feed created from this client, thus the following feed will return 403 errors when you try to read or write from it.
client.feed('flat', 'user-id');
But if you generate a read/write token on server side you can initiate a feed with this token and allow read/writes from the client-side:
client.feed('flat', 'user-id', 'read/write token');
To generate a read/write token on the server initiate a client with your secret key and call the following methods:
var client = stream.connect('api-key', 'api-secret', 'app-id');
var readToken = client.getReadOnlyToken('flat', 'user-id');
var readWriteToken = client.getReadWriteToken('flat', 'user-id');
Supply one of these tokens to your client and create a feed instance with this token.
When to use Stream-JS on the client
In most use-cases however you would want to use the stream-js client on your server side and retrieve/publish activities there, enrich these activities with data stored in your local database and send this to the client. One use-case for using stream-js on the client is for realtime notifications

SPA - Firebase and .Net WebApi 2 authentication

I'm having a Single Page Application written in AngularJs (The framework is irrelevant at this point) The application is hosted in IIS and it's compose of index.html plus a bunch of client assets.
On backend I have WebApi 2, hosted also in IIS as a separate application.
For authentication on client I'm using Firebase (simple login) with several social netoworks enabled, like Facebook, Twitter or Google.
So far so good. I like how easy it is to enable twitter authentication for example with firebase.
On login with social network i get back from firebase, the firebaseAuthToken and provider accesstoken.
Now I want to use firebaseAuthToken or provider access token to authenticate with my WebApi.
The question is: What is the best way to authenticate with WebApi in given conditions?
There is not an option to use only firebase to store my data and get rid of web api since I have in place complex business logic on server.
One silly idea that i have so far, is to pass the social provider access token to the server, validate the token against provider and then issue a security token using Owin -Katana.
I'm not using build in social providers support from katana due to lack of documentation, complexity and bad integration with single page apps. I found the visual studio template for SPA too mvc specific. But that's me :)

tl;dr - Demo Project on GitHub
The steps below may seem long, but it's actually really easy. I created my demo project in just an hour or so.
I agree with you about using Owin and Katana. I've been through that process before and it wasn't really a great experience. Using Firebase was a heck of a lot easier.
This can all be done with JWTs!
When you authenticate through Firebase and whatever social provider, you get back a JSON Web Token (JWT) - firebaseAuthToken.
Grab your Firebase Secret from the Dashboard
The way JWTs work is that we have a secret token and a client token. The client token is the firebaseAuthToken we receive after logging in. The secret token is generated for us in the Firebase Dashboard.
Store your Firebase Secret in the appSettings section of your Web.config
We need to store this secret key in the Web.config so it's easier to access later.
<add key="FirebaseSecret" value="<Firebase-Secret-Token-Goes-Here" />
Create an Action Filter to check the JWT from the Authorization Header
We can verify the request is valid by passing the client token in the Authorization header. On the server we can store our secret key that we get from our Firebase Dashboard. When the request is checked by Web API we can decode the JWT using a JWT Library (available from NuGet). If the decoding is successful then we can check to token to make sure it isn't expired.
public class DecodeJWT: ActionFilterAttribute
{
public override void OnActionExecuting(System.Web.Http.Controllers.HttpActionContext actionContext)
{
string firebaseAuthToken = string.Empty;
if (actionContext.Request.Headers.Authorization != null) {
firebaseAuthToken = actionContext.Request.Headers.Authorization.Scheme;
} else {
throw new HttpException((int) HttpStatusCode.Unauthorized, "Unauthorized");
}
string secretKey = WebConfigurationManager.AppSettings["FirebaseSecret"];
try {
string jsonPayload = JWT.JsonWebToken.Decode(firebaseAuthToken, secretKey);
DecodedToken decodedToken = JsonConvert.DeserializeObject < DecodedToken > (jsonPayload);
// TODO: Check expiry of decoded token
} catch (JWT.SignatureVerificationException jwtEx) {
throw new HttpException((int) HttpStatusCode.Unauthorized, "Unauthorized");
} catch (Exception ex) {
throw new HttpException((int) HttpStatusCode.Unauthorized, "Unauthorized");
}
base.OnActionExecuting(actionContext);
}
}
Create a $httpInterceptor add the firebaseAuthToken to the header for every request
On the client, the trick is that the token has to be passed every time. To make this easier we need to create a $httpInterceptor with Angular that checks for a firebaseAuthToken on sessionStorage.
.factory('authInterceptor', function ($rootScope, $q, $window) {
return {
request: function (config) {
config.headers = config.headers || {};
if ($window.sessionStorage.firebaseAuthToken) {
config.headers.Authorization = $window.sessionStorage.firebaseAuthToken;
}
return config;
},
response: function (response) {
if (response.status === 401) {
// TODO: User is not authed
}
return response || $q.when(response);
}
};
})
Set the firebaseAuthToken to sessionStorage on a successful login
Whenever a user logs in we can set the value to sessionStorage.
$rootScope.$on('$firebaseSimpleLogin:login',
function (e, user) {
// add a cookie for the auth token
if (user) {
$window.sessionStorage.firebaseAuthToken = user.firebaseAuthToken;
}
cb(e, user);
});
Register the DecodeJWT filter globally
Inside of the WebApiConfig.cs Register method we can set the DecodeJWT filter to apply for all of our ApiControllers.
config.Filters.Add(new DecodeJWT());
Now whenever we make a request to an ApiController it will reject it unless there is a valid JWT. So after a user logs in we can save their data to a ApiController if it already doesn't exist.
// globally uses DecodeJWT
public class UsersController: ApiController
{
// POST api/users
public void Post([FromBody] FbUser user) // See GitHub for this Model
{
// Save user if we do not already have it
}
}