I would like to send encrypted / secure data that I send to my REST API.
However, I encounter difficulties here and would be very happy about external help.
Please note that the code is simplified for better illustration!
Here is the basic structure of the REST API in order to be able to reveal errors / problems of understanding on my part:
Sending the data:
First, I encrypt the relevant data:
const crypto = require('crypto')
const algorithm = 'aes-256-ctr'
const secretKey = 'somesecret'
const iv = crypto.randomBytes(16)
const encrypt = (t) => {
const cipher = crypto.createCipheriv(algorithm, secretKey, iv)
const encrypted = Buffer.concat([cipher.update(t), cipher.final()])
return {
iv: iv.toString('hex'),
content: encrypted.toString('hex')
}
}
const regdata = crypto.encrypt(password)
//some stuff --> sending data
I then send the data, including the iv and the data to be encrypted, to my API.
function apicall(data, Method) {
data = Object.assign(data, {
apisecret: process.env.REACT_APP_APISECRET,
cryptokey: process.env.REACT_APP_CRYPTOKEY
})
const mydata = fetch(`https://somepublic.url:4400/defined/query/${JSON.stringify(data)}`, {
method: Method
})
const jdata = JSON.parse(tdata)
//some stuff
return jdata
}
My API looks something like this:
//require some stuff
app.use(express.json())
app.use(function(req, res, next) {
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Methods', 'GET,POST,PUT,DELETE,OPTIONS');
res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Access-Control-Allow-Headers');
next();
});
// ** Select SQL Data
app.get('/defined/query/:data', (req, res) => {
const data = JSON.parse(req.params.data)
if ((data.apisecret != process.env.NODE_APP_APISECRET) || (data.cryptokey != process.env.NODE_APP_CRYPTOKEY)) {
throw "Authentication Required"
}
//do some stuff with encrypted Data
.then(response => {
res.status(200).send(response);
})
.catch(error => {
res.status(500).send(error);
})
})
It all works, I just don't quite understand what encrypting the data does for me if it can be read out in the URL.
If someone uses my APP and reads the full URL with Whireshark & Co., they can
read the "process.env.NODE_APP_APISECRET" named by me and thus has access to my API
Although this does not have my "secret key", it can read out the "iv", which would also like to be prevented.
However, I have to send the "iv" in the URL, since the data is only decrypted afterwards.
Did I make a mistake in my setup, or generally have a problem understanding the API interface?
You need to consider who you are trying to stop from intercepting the data.
If the goal is to stop third parties intercepting the data between the client and the server then use HTTPS, only use HTTPS, do not roll your own encryption on top of it.
If your goal is to let the client store data on the server without people who have access to the server being able to decrypt it then encrypt it on the client, and do not send the keys to the server.
If your goal is to let multiple clients exchange data without the server being able to decrypt it, then generate keys on each client, share public keys between clients and keep private keys private. Then each client should encrypt the data with the recipient's public key (which can only be decrypted by the recipient's private key).
Related
I create server (node.js / express / boby-parser)
and I need to get array of objects 'users'.
its part of code from server.js file:
let users = [{
name: 'user1',
}];
app.get('/users/', (req, res) => {
const filePath = path.join(pth.dir, 'build', 'index.html');
res.json(users);
res.sendFile(filePath);
});
Its my code from frontend:
const handleResponse = (response) => {
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
};
const getAll = (baseUrl) => {
const requestOptions = {
method: 'GET'
};
return fetch(baseUrl, requestOptions).then(handleResponse);
};
Something wrong with my code at server. (I just dodnt know how to use express server).
when I use getAll function I got JSON text replace my page. Can anyone help? How should I write app.get() in server.js. Or do I need write in server part one app.get() to get page or another app.get() to get JSON data?
Why are you trying to send a file in the response?:
res.sendFile(filePath);
For starters, the response content can either be JSON or a file (or any of a variety of other things of course), but not both. With data like JSON or XML it's possible to combine multiple objects into one larger object for a single response, but this won't work if the content types are entirely different as it is with a file.
Looking at your client-side code, you're not even doing anything with that file. You only read the JSON data:
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
So the simplest approach here would just be to not try to send back the file:
app.get('/users/', (req, res) => {
res.json(users);
});
Edit: Based on comments below, you seem to be struggling with the different requests the client makes to the server. The browser loading the page is one request with one response. If that page includes JavaScript that needs to fetch data, that would be a separate AJAX request with its own response containing just that data.
It’s possible to use JSON (or any data) server-side to populate a page template and return a whole page with the data. For that you’d need to use (or build) some kind of templating engine in the server-side code to populate the page before returning it.
The res.json() represents the HTTP response that an Express app sends when it gets an HTTP request. On the other hand, res.sendFile() transfers the file at the given path.
In both cases, the flow is essentially transferred to client who might have made the request.
So no, you cannot use res.sendFile and res.json together.
var options = {
headers: {
'name': 'user1',
}
};
res.sendFile(path.join(__dirname, 'build', 'index.html'), options);
Thats really the closest you can do to achieve the desired task.
i was wandering what is the best way to avoid sending the user data on every request,
lets say i want to add product from user's account, i have to send the user. or i want to order something, i have to send the user.
i thought about something like this:
app.use(async (req, res, next) => {
if (!req.body.userId) {
return next();
}
const user = await enteties.User.findByPk(req.body.userId);
req.user = user;
next();
});
but it also requires me to send the user on evey request..
there must be a way to avoid sending the user data to the server on almost every request.
also, it will make all my requests of type "post" since i have to send the user, and even "get" requests are now become "posts", for sure this is not correct
If you implement your JWT token correctly you don't need to send the logged in user id.
JWT tokens contain a payload section that is basically any JSON data you want to set. This is basically your decentralized session stored in the user's machine. When creating a JWT token you'd normally do something like:
const jwt = require('jsonwebtoken');
const config = require('./config');
function generateToken(user) {
let payload = {
sub: user.id
};
return jwt.sign(payload, config.secret, {
algorithm: 'HS512', // choose algorithm appropriate for you
expiresIn: config.expires
})
}
That payload part allows you to send user identifying information. In the case above, the user id. To get that id from a request simply verify it:
app.use((req, res, next) => {
const token = req.get('Authorization');
jwt.verify(token, config.secret, (err, payload) => {
if (err) {
next(err);
}
else {
req.user = payload; // user.sub is the user id
next();
}
});
});
Or you can use a library such as express-jwt to do it for you:
const expressJwt = require('express-jwt');
const express = require('express');
const config = require('./config');
const app = express();
app.use(expressJwt({ secret: config.secret }); // use express-jwt like any
// middleware, you can even install
// it on specific routes.
Now in your controller/route you can simply extract the payload in the req.user object. Invalid tokens or requests without tokens will completely skip your handler and immediately return an error or unauthorized response:
app.get('/some/endpoint', (req, res) => {
console.log('user is', req.user.sub); // note: req.user is our payload
});
Additional tricks:
As I mentioned, the payload is basically user defined. If you need to keep track of other user information such as roles or permissions you can store them in the JWT token:
// Example payload
let payload = {
sub: user.id,
admin: user.role === 'admin',
gender: user.gender
};
This reduces the number of database requests needed to process the user. Making the authentication system completely decentralized. For example you may have a service that consumes this JWT token that is not connected to your user database but need to check if user is admin. With the right payload that service does not even need to have access to the user database.
Note however that the payload is not encrypted. It is just base64 encoded. This means that the information in the token can be easily read by anyone with access to it (normally the user but beware of 3rd party scripts). So ideally you shouldn't store dox-able information in the payload if you have 3rd party scripts on your website (then again, it is highly unusual these days for anyone to write the entire front-end from scratch without any libraries or frameworks)
Also note that the more you put in your payload the larger your token will be.
My use case is as follows:
Incoming http request to server to login
User token is generated. The token is a Json object build from various fields. Then convert to String and Base64.
const stringObject = {
elementA: stringA,
elementB: stringB
};
const bufferString = new Buffer(`${JSON.stringify(stringObject)}`);
const encodedAccessToken = bufferString.toString('base64');
The generated string can now be decoded anywhere.
Is there a way I can encode it in such as way that only my server will be able to decode it? Like encoding it with some sort of a key.
Thanks.
You can use JWT token Node Module : link
Encode data and generate token :
var jwt = require('jsonwebtoken');
var token = jwt.sign({ foo: 'bar' }, 'shhhhh');
{ foo: 'bar' } is your feilds that you encrypt
Decode by same key shhhhh
// verify a token symmetric
jwt.verify(token, 'shhhhh', function(err, decoded) {
console.log(decoded.foo) // bar
});
This does not directly answer your question, but I think your overall approach is wrong. What you are trying to achieve is to have session data. You don't need to send this data to the client and back. That is not a very good practice. Instead you should store this data on the server, preferably in a database.
What you would do is to create a unique key, something randomly generated. You would store the user data using this key, and send the key to the client to be used on requests. You do it by setting it as a cookie variable as well.
The user data can have additional fields variables for more secure access. Like the IP of the client, and possibly a expiry time for cleanup.
use jsonwebtoken instead. it can encrypt your object with a secret phrase. library like node-jsonwebtoken is very easy to use.
I've created a dashboard that aggregates order processing for multiple Shopify stores. It's a private app so I save each Shopify store's API credentials and webhook secret in my database. I've encrypted all this sensitive stuff with a key that's stored as an environment variable. For any given event, I direct all Shopify stores to the same callback URL. Hence for each url I'll have to verify the request body against all possible webhook secrets, which are stored in the database. I have 2 questions:
Is this an acceptable approach for securing the stores' API credentials and webhook secrets?
How do I synchronously verify the request body if my secrets are stored in the database? My verify function is below.
verify: (buffer, hmacHeader) => {
Brand.find().exec((err, brands) => {
if (!err && brands) {
const allWebhookSecrets = brands.map(brand => {
console.log(`encrypted webhook secret is: ${brand.shopify_webhook_secret_encrypted}`)
return encryption.decrypt(brand.shopify_webhook_secret_encrypted);
});
const webhookIsValid = allWebhookSecrets.some(secret => {
var hmac = crypto.createHmac('sha256', secret)
.update(buffer)
.digest('base64');
return hmac == hmacHeader;
});
console.log(`webhookIsValid: ${webhookIsValid}`);
return webhookIsValid;
}
return false;
});
}
Generally the only thing you should really be storing in your database is the stores access token for your app and any other verification data for the store login. Webhook signatures and generated differently every time a request is sent to your app using your apps shared secret and the data the webhook is about to send to you.
Webhooks are verified using your apps shared secret and the post request data to the HMAC signature sent in the header: X-Shopify-Hmac-SHA256.
This is my simplified nodejs implementation of a webhook verification middleware
function verify_webhook(postData, hmac) {
if (!hmac || !postData || typeof postData !== 'object') {
return false;
}
const calculatedSignature = crypto.createHmac('sha256', process.env.SHOPIFY_SHARED_SECRET)
.update(postData)
.digest('hex');
return calculatedSignature === hmac;
}
where postData and hmac are JSON.stringify(req.body) and req.get('X-Shopify-Hmac-SHA256')
There's generally no point storing this information in a database and just calculating it and verifying it on a fly using middleware. I hope this helps.
I am concerned about security of my react/redux application as my api url is exposed to the public inside bundled app.js file. I've been researching this and some developers proxy it somehow i.e. instead of using my api url I can use api/ whenever I perform calls with libraries like axios or superagent and it gets proxied to my api url, but this way users can only see api/ on their side.
I'm trying to figure this out, I assume this is set up within express config?
You have a valid concern.
Typically you would have your clientside code make calls to, say, /api, and in express (or whatever server you use) create a route for "/api" that proxies that request to the actual api url.
This way you can obscure any sensitive information from the client. For example authentication tokens, api keys, etc.
In express you could do something like this:
app.use('/api', (req, res) => {
const method = req.method.toLowerCase();
const headers = req.headers;
const url = 'your_actual_api_url';
// Proxy request
const proxyRequest = req.pipe(
request({
url
headers,
method,
})
);
const data = [];
proxyRequest.on('data', (chunk) => {
data.push(chunk);
});
proxyRequest.on('end', () => {
const { response } = proxyRequest;
const buf = Buffer.concat(data).toString();
res.status(response.statusCode).send(buf);
});
});
This example is a bit more elaborate that is has to be, but it will probably work for you.