The explanation is a bit long so please bear with me.
I am building a Facebook messenger bot which uses my sails.js/node.js server in the backend and a MongoDB database.
In my sails app, I have applied policies to the method of the controller which handles the operations to be performed after recieving a text from the user. In this policy, I am following the documentation(https://developers.facebook.com/docs/messenger-platform/webhook-reference - "Security" section) and comparing the x-hub-signature that comes in the request's header with the sha1 digest of the request payload(body).
So now whenever I am sending a message to the bot, it says in the policy that the signature from the request and the signature calculated by me is different and thus, doesnt go further. I double checked the app secret which I should use while calculating the digest and it seems to be correct. Another difference which I found was that, Facebook request also sends a "content-length" field in its header, which is different than character length of the body they sent in the same request. And this is what I think is the reason for different signatures but I am unable to resolve it and get to the root of the problem as to why is this happening.
Also another thing to note is that the same code that throws this mismatch error, runs perfectly at certain times(actually, most of the times).
So can somebody please help me this? I ll be forever grateful :)
Here is the code from the policy
var crypto = require('crypto');
if(req.headers['x-hub-signature']){
//console.log('req headers -----', JSON.stringify(req.headers));
//console.log('req body -----', JSON.stringify(req.body));
var hmac, calculatedSignature, payload = req.body;
hmac = crypto.createHmac('sha1', app_secret);
hmac.update(JSON.stringify(payload));
calculatedSignature = 'sha1='+hmac.digest('hex');
//console.log("signature calculatedSignature",calculatedSignature);
if(calculatedSignature === req.headers['x-hub-signature']){
return next();
}else{
res.forbidden('You shall not pass!');
}
}
This is a sample request header -
{"host":"e93d4245id.ngrok.io","accept":"*/*","accept-encoding":"deflate, gzip","content-type":"application/json","x-hub-signature":"sha1=d0cd8177add9b1ff367d411942603b0d08183964","content-length":"274","x-forwarded-proto":"https","x-forwarded-for":"127.0.0.1"}
And this is the body from the same request -
{"object":"page","entry":[{"id":"1778585282425767","time":1479476014038,"messaging":[{"sender":{"id":"userId"},"recipient":{"id":"recipientId"},"timestamp":1479468097895,"message":{"mid":"mid.1479468097895:efdc7d2c68","seq":2355,"text":"Hahahaha"}}]}]}
I think the problem was some specific characters such as # and % were needed to be converted to their unicode escape sequence as specified in their documentation and replaced in the original stringified JSON. I converted them and then calculated the hmac signature of the new string and it got matched.
Also the reason why it was working and why it was not in certain cases was I think because of the special characters being present in that string which was being stringified. If it did not have the characters # or % then it worked without any problem.
This is how I solved it -
inside if
var hmac, calculatedSignature, payload = JSON.stringify(req.body);
var resStr = payload.replace(/\#|\%/g,function(a, i){
hex = payload.charCodeAt(i).toString(16);
var s = "\\u" + ("000"+hex).slice(-4);
return s;
});
hmac = crypto.createHmac('sha1', app_secret);
hmac.update(resStr);
calculatedSignature = 'sha1='+hmac.digest('hex');
if(calculatedSignature === req.headers['x-hub-signature']){
return next();
}else{
res.forbidden('You shall not pass!');
}
Your bodyParserJSON should return rawBody (just stringifying will fail in many cases):
bodyParser.json({
verify(req, res, buf) {
req.rawBody = buf;
},
})
Here is a middleware that I've written. It uses crypto module to generate sha1
fbWebhookAuth: (req, res, next) => {
const hmac = crypto.createHmac('sha1', process.env.FB_APP_SECRET);
hmac.update(req.rawBody, 'utf-8');
if (req.headers['x-hub-signature'] === `sha1=${hmac.digest('hex')}`) next();
else res.status(400).send('Invalid signature');
}
and finally in your route you can use it as:
app.post('/webhook/facebook', middlewares.fbWebhookAuth, facebook.webhook);
Related
I am sending a token to user email like below and using bcrypt for this as an encrypt/decrypt mechanism.
const token = await bcrypt.hash(joinDate, 10);
When the user clicks on the link in email, I get the above token back as that token is a part of
/api/unsubscribe?userId="abcd"&token="token_that_was_generated_using_bcrypt_and_sent_to_user"
const {userId, token} = req.query;
In the api, I am comparing joinDate obtained from database vs token sent by req.query but it never matches.
const joinDate = user.joinDate.toString();
const tokenValidated = await bcrypt.compare(joinDate, token)//this is always false although us generated from same joinDate field
Why is tokenValidated always false although it was generated using the same field joinDate?
Your use of bcrypt is not secure. Anyone can brute-force a few hundred or a few thousand dates and cause any user to become unsubscribed.
I assume your motivation is to avoid having to create a new table in your database and store a random token for every email you've sent out.
If so, the proper tool to use is HMAC.
Your URL should be of the form: /api/unsubscribe?userId=abcd&mac=....
Come up with a secret key known only to your server. This secret key will be used to create and authenticate all unsubscribe requests. Only use this key for authenticating unsubscribe requests.
Perform HMAC-SHA512 on the user id, with the HMAC output truncated to 128 bits. Then base64-encode the 128 bits and set it as the mac parameter in the URL.
HMAC means to create a hash-based message authentication code, which will confirm to your server that it must have created and emailed out that 'mac'.
Now, your server can authenticate each response, because only someone with knowledge of the server's secret key can produce a valid unsubscribe link.
Your logic of using bcrypt for hash and verifying on the other end should work. The only possible thing that can prevent it from working is either:
1 - the joinDate here: const token = await bcrypt.hash(joinDate, 10);
is not ===
to joinDate here:const tokenValidated = await bcrypt.compare(joinDate, token)
( maybe differes in ", ' or different format orsomething )
2 - or the token is kinda different in ' or " when passing through / read from queryparam; (e.g. you token is going "'token_that_was_generated_using_bcrypt_and_sent_to_user'" for comparison.)
I have been dealing with cryptography craziness since yesterday, I 've literally lost my sleep over this.
I am implementing a node red solution to get webhooks from Xero to be written in a custom app. I have experienced a lot of issues with the payload and how it needs to be stringified and how needs to be hashed, but eventually figured it out thanks to a Github fellow that posted this code to get the body to its 'raw' state
let msgPayloadRaw = JSON.stringify(msg.payload).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"');
I then create a sha256 base64 hash to check against the header value using the following js code
var cryptojs = context.global.cryptojs;
const webhookKey = 'MyWebhookKeyHere';
let msgPayloadRaw = JSON.stringify(msg.payload).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"');
let bdata = new Buffer(msgPayloadRaw).toString();
let ciphertext = cryptojs.HmacSHA256(bdata, webhookKey );
let base64encoded = cryptojs.enc.Base64.stringify(ciphertext);
msg.payload = base64encoded;
return msg;
Now everything should work great, but I get a crazy result showcased in this recording, where the web hooks intent status turns to 'OK', and some seconds later returns to this error:
Retry
We haven’t received a successful response on the most recent delivery attempt and will retry sending with decreasing frequency for an overall period of 24 hours.
Response not 200. Learn more
Last sent at 2022-06-22 11:48:28 UTC
What's the problem ?
The problem relies in the http input, where node red parsed the body.
Body needs to be hashed. The body in the http request is like that
{"events":[],"firstEventSequence": 0,"lastEventSequence": 0, "entropy": "IVMMHNWPBAZYRZJRCUAQ"}
Notice the spaces after each :
Node Red converts that body to JSON object. When I do JSON.stringify(msg.payload); I will get the following
{"events":[],"firstEventSequence":0,"lastEventSequence":0, "entropy":"IVMMHNWPBAZYRZJRCUAQ"}
which is obviously the same, but technically it is not (due to spaces) and when hashed it generates a different hash value.
The GitHub fellow did that that walkround
JSON.stringify(msg.payload).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"');
So in order to solve this, I need to find a way to get the raw http input, instead of the parsed one that node red is providing.
Any ideas how to get the raw input ?
Sort answer: you don't.
The raw body is not available if the Content-Type header is set to application/json the bodyParser will kick in and generate the matching JSON object that is passed as the msg.payload.
the httpNodeMiddleware is attached after the bodyParser so the body has already been changed.
I have an email in my mailbox and I want the AppScript program to reply to it with just me and a special google group as the recipients. The purpose of this is communication of the program with me as the program replies to the message once it has processed it with necessary details about the processing in the reply body. There might also be other recipients apart from me in the original message and I don't want the program to send the reply to them.
So I need to reply with a changed set of recipients. When I do it in the Gmail GUI it works just fine, I hit reply, change the recipients, send the message and the reply ends up in the original thread. However when I do it in the script the reply always ends up in a new thread. Originally I thought Gmail decides based on the subject of the email but it seems there's more to it (perhaps it has recently changed as I think it used to work that way).
I tried multitude of slightly different approached, one of them being:
var messageBody = "foo";
var newRecipients = "me#gmail.com, my-group#gmail.com";
var messageToReplyTo = ...;
var advancedParams = {from : "my-alias#gmail.com"};
var replyDraft = messageToReplyTo.createDraftReply(messageBody);
var replySubject = replyDraft.getMessage().getSubject();
var replyBody = replyDraft.getMessage().getBody();
replyDraft.update(newRecipients, replySubject, replyBody, advancedParams);
replyDraft.send();
There are a couple fun things you need to do in order to achieve this, but you can do it without too much trouble. You should definitely review the guide to Drafts.
Per the API spec:
In order to be part of a thread, a message or draft must meet the following criteria:
The requested threadId must be specified on the Message or Draft.Message you supply with your request.
The References and In-Reply-To headers must be set in compliance with the RFC 2822 standard.
The Subject headers must match.
To start, you need to get a reference to the draft you want to update. This is probably simplest by using GmailApp:
const thread = /** get the thread somehow */;
const newBody = /** your plaintext here */;
const reply = thread.createDraftReply(newBody);
The primary issue with Gmail & Drafts is that a Draft is an immutable message to server resources. If you change any of it, you change all of it. Thus, to change a header value such as the recipient address, you need to completely rebuild the message. This is why using the GmailApp methods to update a draft fail to maintain the existing thread information - you can't specify it as one of the advanced options for building the new message. Thus, you must use the Gmail REST API for this task:
const rawMsg = Gmail.Users.Drafts.get("me", reply.getId(), {format: "raw"}).message;
To update a draft, you need to supply an RFC 2822 formatted message encoded in base64. If you are comfortable converting the rich format message parts into such a valid string, by all means work with the non-raw format, as you have direct access to the headers in the message.payload.
To work with the raw message, know that Apps Script casts the described base64 encoded string to a byte array in the above call. The leap is then to treat that byte array as string bytes, specifically, charCodes:
const msg_string = rawMsg.raw.reduce(function (acc, b) { return acc + String.fromCharCode(b); }, "");
console.log({message: "Converted byte[] to str", bytes: rawMsg.raw, str: msg_string});
Once you have the message as a string, you can use regular expressions to update your desired headers:
const pattern = /^To: .+$/m;
var new_msg_string = msg_string.replace(pattern, "To: <....>");
// new_msg_string += ....
Since the Gmail API endpoint to update a Draft expects a base64 web-safe encoded string, you can compute that:
const encoded_msg = Utilities.base64EncodeWebSafe(new_msg_string);
And the only remaining bit is to perform the call (and/or send the updated draft).
const resource = {
id: <draft id>, // e.g. reply.getId()
message: {
threadId: <thread id>, // e.g. thread.getId()
raw: encoded_msg
}
}
const resp = Gmail.Users.Drafts.update(resource, "me", reply.getId());
const sent_msg = Gmail.Users.Drafts.send({id: resp.id}, "me");
console.log({message: "Sent the draft", msg: sent_msg});
I don't claim that the handling of the Byte array returned from the Message.raw property is 100% correct, only that it seems correct and didn't result in any errors in the test message I sent. There may also be an easier approach, as the Apps Script service has a Drafts.update endpoint which accepts a Blob input and I have not investigated how one would use that.
I would like to forward all incoming messages to some other number with the number to which I got the reply on Twilio,
For example:
I have a Twilio number say "+14444444444" to which I got the reply from number say: '+15555555555' and I would like to forward all messages to number '+19999999999'.
So I would like to forward all messages on number '+19999999999' which should be received from '+15555555555', not from '+14444444444',
Will anyone please let us know if there is an API in Twilio that can do a trick using nodeJs.
Note: The SMS forwarding number can be dynamic and in that case, we cannot use Twiml, if we can use Twiml so please let us know how to set dynamic forwarding number.
Also got the following link that says how to forward SMS but is not relevant to approach we are trying to accomplish using nodeJS:
https://support.twilio.com/hc/en-us/articles/223134287-Forwarding-SMS-messages-to-another-phone-number
Thanks, any help will be appreciated.
Updated Answer
Thanks, #Alex and #Piyush for clarifying the question:
Really sorry about that! Thanks for clarifying. If I understand you correctly now, you want to forward a message to another number, but preservice the original number from the message. Unfortunately, there's not a way to do this. You can forward the message and include the original sender in the message body, but there's no way to replace the actual sender as the original.
Let me know if I understood that correctly this time and if there's anything else I can help with.
Old Answer (Message forwarding with own number)
You can use TwiML dynamically when using our helper libraries, so that should be something you can setup using Node. When your webhook sends your message to your Node application, you can check the body of the message, and make a conditional SMS request or conditionally point to different TwiML based on the content of the body. Here's an example of how to setup a conditional reply for your incoming messages based on the message body in Node:
https://www.twilio.com/docs/sms/tutorials/how-to-receive-and-reply-node-js
While this example is for replying to messages, it shows you the principles of how conditional TwiML can work.
You would just have add a "to" number you want to forward the message to in the message request.
Below is the example of conditional forward.
const http = require('http');
const express = require('express');
const MessagingResponse = require('twilio').twiml.MessagingResponse;
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.urlencoded({ extended: false }));
app.post('/', (req, res) => {
const twiml = new MessagingResponse();
twiml.to = "+1234567890 // Your number here.
if (req.body.Body == 'hello') {
twiml.message('Hi!');
} else if (req.body.Body == 'bye') {
twiml.message('Goodbye');
} else {
twiml.message(
'No Body param match, Twilio sends this in the request to your server.'
);
}
res.writeHead(200, { 'Content-Type': 'text/xml' });
res.end(twiml.toString());
});
http.createServer(app).listen(1337, () => {
console.log('Express server listening on port 1337');
});
Let me know if that helps and if you have anymore questions on how to set this up.
I'm handling an incoming Webhook from github, and wants to verify the x-hub-signature. I'm using hmacto hash the "secret", and then compares the two hashes. The problem is that they never match. This is my setup:
router.route("/auth")
.post((req, res) => {
var hmac = crypto.createHmac("sha1", process.env.WEBHOOK_SECRET);
var calculatedSignature = "sha1=" + hmac.update(JSON.stringify(req.body)).digest("hex");
console.log(req.headers["x-hub-signature"] === calculatedSignature); // Returns false
console.log(req.headers["x-hub-signature"]) // => sha1=blablabla
console.log(calculatedSignature) // => sha1=foofoofoo
res.end();
});
I've tried everything, but can't make it work. Wondering if the hmac.update() should hold another parameter than JSON.stringify(req.body). Does anyone know why they won't match?
So the problem was with the settings of the webhook. The content-format was set to application/x-www-form-urlencoded, which for some reason hashed the x-hub-signature differently. I just changed it to application/json, and then it worked!
If the webhook Content-Type is set to application/x-www-url-encoded then string you need to use to check the HMAC is
"payload=" + query_encoded_payload.
For example in golang
payloadForm := r.PostFormValue("payload")
escaped := url.QueryEscape(payloadForm) # ex. http://www.url-encode-decode.com/
checkMe := "payload=" + escaped