We Keep Coding

How to verify PayPal Webhooks in node.js?

I found some old answers dealing with PHP and this code example, but I am not sure whether this is outdated now since the repo is archived and I know that generally PayPal moved to an approach that just uses the REST API.
I would love if somebody could give an update here on whats the latest recommendation is and whether the code here from 2015 is outdated now.
/* Copyright 2015-2016 PayPal, Inc. */
"use strict";
var paypal = require('../../../');
require('../../configure');
// Sends the webhook event data to PayPal to verify the webhook event signature is correct and
// the event data came from PayPal.
// Note this sample is only for illustrative purposes. You must have a valid webhook configured with your
// client ID and secret. This sample may not work due to other tests deleting and creating webhooks.
// Normally, you would pass all the HTTP request headers sent in the Webhook Event, but creating a
// JSON object here for the sample.
var certURL = "https://api.sandbox.paypal.com/v1/notifications/certs/CERT-360caa42-fca2a594-a5cafa77";
var transmissionId = "103e3700-8b0c-11e6-8695-6b62a8a99ac4";
var transmissionSignature = "t8hlRk64rpEImZMKqgtp5dlWaT1W8ed/mf8Msos341QInVn3BMQubjAhM/cKiSJtW07VwJvSX7X4+YUmHBrm5BQ+CEkClke4Yf4ouhCK6GWsfs0J8cKkmjI0XxfJpPLgjROEWY3MXorwCtbvrEo5vrRI2+TyLkquBKAlM95LbNWG43lxMu0LHzsSRUBDdt5IP1b2CKqbcEJKGrC78iw+fJEQGagkJAiv3Qvpw8F/8q7FCQAZ3c81mzTvP4ZH3Xk2/nNznEA7eMi3u1EjSpTmLfAb423ytX37Ts0QpmPNgxJe8wnMB/+fvt4xjYH6KNe+bIcYU30hUIe9O8c9UFwKuQ==";
var transmissionTimestamp = "2016-10-05T14:57:40Z";
var headers = {
'paypal-auth-algo': 'SHA256withRSA',
'paypal-cert-url': certURL,
'paypal-transmission-id': transmissionId,
'paypal-transmission-sig': transmissionSignature,
'paypal-transmission-time': transmissionTimestamp
};
// The eventBody parameter is the entire webhook event body.
var eventBody = '{"id":"WH-82L71649W50323023-5WC64761VS637831A","event_version":"1.0","create_time":"2016-10-05T14:57:40Z","resource_type":"sale","event_type":"PAYMENT.SALE.COMPLETED","summary":"Payment completed for $ 6.01 USD","resource":{"id":"8RS6210148826604N","state":"completed","amount":{"total":"6.01","currency":"USD","details":{"subtotal":"3.00","tax":"0.01","shipping":"1.00","handling_fee":"2.00","shipping_discount":"3.00"}},"payment_mode":"INSTANT_TRANSFER","protection_eligibility":"ELIGIBLE","protection_eligibility_type":"ITEM_NOT_RECEIVED_ELIGIBLE,UNAUTHORIZED_PAYMENT_ELIGIBLE","transaction_fee":{"value":"0.47","currency":"USD"},"invoice_number":"","custom":"Hello World!","parent_payment":"PAY-11X29866PC6848407K72RIQA","create_time":"2016-10-05T14:57:18Z","update_time":"2016-10-05T14:57:26Z","links":[{"href":"https://api.sandbox.paypal.com/v1/payments/sale/8RS6210148826604N","rel":"self","method":"GET"},{"href":"https://api.sandbox.paypal.com/v1/payments/sale/8RS6210148826604N/refund","rel":"refund","method":"POST"},{"href":"https://api.sandbox.paypal.com/v1/payments/payment/PAY-11X29866PC6848407K72RIQA","rel":"parent_payment","method":"GET"}]},"links":[{"href":"https://api.sandbox.paypal.com/v1/notifications/webhooks-events/WH-82L71649W50323023-5WC64761VS637831A","rel":"self","method":"GET"},{"href":"https://api.sandbox.paypal.com/v1/notifications/webhooks-events/WH-82L71649W50323023-5WC64761VS637831A/resend","rel":"resend","method":"POST"}]}';
// The webhookId is the ID of the configured webhook (can find this in the PayPal Developer Dashboard or
// by doing a paypal.webhook.list()
var webhookId = "3TR748995U920805P";
paypal.notification.webhookEvent.verify(headers, eventBody, webhookId, function (error, response) {
if (error) {
console.log(error);
throw error;
} else {
console.log(response);
// Verification status must be SUCCESS
if (response.verification_status === "SUCCESS") {
console.log("It was a success.");
} else {
console.log("It was a failed verification");
}
}
});

Those SDKs are abstractions for the REST API but are no longer being maintained, so it is best not to use them.
There are two possible ways to verify Webhooks
Posting the message back to PayPal with the verify webhook sygnature REST API call. You'll need to use a client_id and secret get an access token first, same as all other REST API calls.
Verifying the cryptographic signature yourself (Java pseudocode here).
For either method, the "webhookId" -- as opposed to each webhook event id -- is 17 alphadigits and for security (anti-spoof) reasons not part of the Webhook message itself (you get it when registering for webhooks or reviewing existing subscribed hooks in the REST app config)
As it can sometimes be a point of confusion, it's worth mentioning that verifying webhooks is for your own information -- to confirm the message did in fact originate from PayPal, and not some other (malicious) actor.
But for PayPal itself to consider the webhook message successfully delivered (and not keep retrying), all that needs to happen is for the listener URL it's posted to to respond with an HTTP 200 OK status. That concludes the webhook message delivery.

Is there a way to get ethereal email message ID in node.js through SendGrid?

I'm currently using SendGrid as an email service in a web app, and I want to set up my dev environment to send emails to https://ethereal.email/ so I can make sure they look correct. The way I currently have it set up is I'll use nodemailer.createTestAccount() on startup, then log the credentials, then if I need to see the emails I have to manually go to the ethereal site to log in and view all the emails.
Since I'm using SendGrid to send the messages I can't just use nodemailer.getTestMessageUrl(info), but I would really like to be able to just log the URL every time an email is sent in dev then I could just go directly to the message.
I've already tried printing the SendGrid response, as well as connecting via IMAP and loading the most recent message but neither of those contain the message ID anywhere that I can see.
Is there any way to get the message ID of an email sent to ethereal.email and construct a URL directly to the message?
My environment is node.js and I'm using #sendgrid/mail to send messages.
Thanks!

Twilio SendGrid developer evangelist here.
The Nodemailer documentation here suggests that you use ethereal.email as your SMTP settings in development. Then you can use nodemailer.getTestMessageUrl(info) to get the right URL.
You say you are using #sendgrid/mail to send messages, which won't respond with the URL as it is not sent out through the ethereal.email SMTP server. If you do want that URL, your best bet is to replace SendGrid in your development/test environment with ethereal.email.
You could do this by creating your own email sending objects with the same interface, one for #sendgrid/mail and one for Nodemailer with ethereal.email credentials and load the object you want based on the environment.
For example:
class SendGridEmail {
constructor(sgApiKey) {
// construct SG mail object
}
sendEmail(options) {
// use SG mail object to send email
}
}
class NodemailerEmail {
constructor() {
}
sendEmail(options) {
}
}
function getMailer() {
if (process.env.NODE_ENV === "production") {
return new SendGridEmail(process.env.SG_API_KEY);
} else {
return new NodemailerEmail();
}
}
// later
getMailer().sendEmail(options);

Front-end Authorization is INSECURE, almost always. Don't you agree?

I’m in web dev since many years ago, but I still do not believe you can easily implement user authorization in front-end apps or browser extensions.
While front-end authentication can be achieved (cookies, jwt, etc.) and works fairly well, the same is not for authorization. The common example is when you want to restrict access to some content and/or functionalities to logged-in users only.
I am inspecting many browser extensions and web apps, and I usually find something like this pseudocode:
if (user.isLogged === true) {
// code to show ui components and actions
} else {
// code to show ui components and actions
}
which is highly insecure.
For instance, this is coming from an extension available on the chrome web store:
function initApp() {
firebase
.auth()
.onAuthStateChanged(function (user) {
if (user) {
// User is signed in. var uid = user.uid; window.location.href = "app.html";
const uid = user.uid;
const name = user.displayName;
getUserData(uid);
// Plus a lot of other code to show/hide ui components and actions
} else { }
document
.getElementById('quickstart-button')
.disabled = false;
});
document
.getElementById('quickstart-button')
.addEventListener('click', startSignIn, false);
}
The only secure way is to actually load HTML and JS chunks dynamically only when and if the user is authorized to access that page/functionality, where the server decides which chunks serve to the current user based on his role, and then those chunks are injected at runtime. But this is, on one hand, failing with the spa architecture itself because it's very near to serving the app server-side (it's never the client to decide what to show). It also requires injecting HTML markup dynamically using JS which is not ideal for security. Plus, this is something not trivial to implement, so I guess the majority of js apps out there are handling restricted content/areas using the paradigm shown above, based on the current in-memory/store (or in cookie/storage) state of the current user, which is highly insecure and can be easily manipulated by the end-user. Especially in browser extensions where you are not even allowed to obfuscate the code.
Am I missing something?

TL;DR you are right, don't rely on front-end security. There is none. Grab the secret content from a backend and require authentication.
You mentioned SPAs so let's analyze the situation there.
Assume you have a React App with a main component looking similar to this:
function App() {
const { loggedIn, isSpecialUser } = useContext(authContext);
if (loggedIn)
return <MainPage />;
if (isSpecialUser)
return <SuperSecretPage />;
return <LoginPage />;
}
The loggedIn and isSpecialUser state is dynamically set after the user logged in (or relogin by cookies, jwt, whatever).
Therefore the restricted areas for MainPage and SuperSecretPage will only show after the user logged in.
So what if the user looks at the source of the SuperSecretPage?
He should see something like this:
function App() {
const { token } = useContext(authContext);
const { data } = useApi(getSuperSecretContent, token);
return <div>{data}</div>;
}
Even if the super secret content is static for all user, it should be fetched with a separate call that requires a token. Access rights must be checked by the backend. If you just write your content in there, it can be seen by everyone.
Edit Response to OP's comment. He wrote
a malicious user has at least 2 options: 1) change the js content of the main App component, to let's say return ; when the user is not logged in at all; 2) manually change the ajax callback to set his status to Superuser regardless of the actual response.
For both ways, the user would just break the clients rendering but not the security. In any case, the client must make an API-call to receive the secret content:
const { data } = useApi(getSuperSecretContent, token);
useApi is an abstract hook that uses some sort of API (e.g. REST-API) to fetch the content during runtime (ajax). The second parameter is a token. This token must be some sort of authentication token the servers sends the user after login. This token here is the crucial part (see OAuth, JWT). For each ajax call to get the super secret content, a valid token must be supplied. The backend then has to check if the token is valid. If and only if the token is valid, the backend returns the super secret content.
Pseudocode useApi:
function useApi(apiCall, token) {
make ajax request {
endpoint: apiCall,
authorization header: token
}
if success:
return data
else if auth error:
return error
}
Pseudocode backend:
get request to super secret content:
if token is not valid:
return auth error;
else
return super secret content
The user cannot trick the backend into sending him the super secret content by manipulating the frontend state.
Edit 2 Response to OP's confusion about authentication and authorization.
You must use the token also for authorization. In case of a JWT for instance, the token itself can include infos about rights management, otherwise you could query access rights from a database for a given user.
This must happen during the request handling #backend side.
Pseudocode backend e.g.:
get request to super secret content:
get access rights for token
if access rights include super secret resources:
return super secret content;
else
return auth error;

Unable to query Google Search Console API using a Service Account

I need to retrieve some data from Google Search Console (Webmaster Tools) using a service account.
So far I've been able to retrieve an access_token for the service account which I need to append to the url of the request. The problem is that I can't find a way to do so, this is the code i'm using:
function retrieveSearchesByQuery(token)
{
gapi.client.webmasters.searchanalytics.query(
{
'access_token': token,
'siteUrl': 'http://www.WEBSITE.com',
'fields': 'responseAggregationType,rows',
'resource': {
'startDate': formatDate(cSDate),
'endDate': formatDate(cEDate),
'dimensions': [
'date'
]
}
})
.then(function(response) {
console.log(response);
})
.then(null, function(err) {
console.log(err);
});
}
This is the url called by the function:
https://content.googleapis.com/webmasters/v3/sites/http%3A%2F%2Fwww.WEBSITE.com/searchAnalytics/query?fields=responseAggregationType%2Crows&alt=json"
Instead it should be something like this:
https://content.googleapis.com/webmasters/v3/sites/http%3A%2F%2Fwww.WEBSITE.com/searchAnalytics/query?fields=responseAggregationType%2Crows&alt=json&access_token=XXX"
The gapi.client.webmasters.searchanalytics.query doesn't recognize 'access_token' as a valid key thus it doesn't append it to the url and that's why I get a 401 Unauthorized as response.
If I use 'key' instead of 'access_token' the parameter gets appended to the url but 'key' is used for OAuth2 authentication so the service account token I pass is not valid.
Does anyone have a solution or a workaround for this?

If your application requests private data, the request must be authorized by an authenticated user who has access to that data. As specified in the documentation of the Search Console API, your application must use OAuth 2.0 to authorize requests. No other authorization protocols are supported.
If you application is correctly configured, when using the Google API, an authenticated request looks exactly like an unauthenticated request. As stated in the documentation, if the application has received an OAuth 2.0 token, the JavaScript client library includes it in the request automatically.
You're mentioning that you have retrieved an access_token, if correctly received, the API client will automatically send this token for you, you don't have to append it yourself.
A very basic workflow to authenticate and once authenticated, send a request would looks like the following code. The Search Console API can use the following scopes: https://www.googleapis.com/auth/webmasters and https://www.googleapis.com/auth/webmasters.readonly.
var clientId = 'YOUR CLIENT ID';
var apiKey = 'YOUR API KEY';
var scopes = 'https://www.googleapis.com/auth/webmasters';
function auth() {
// Set the API key.
gapi.client.setApiKey(apiKey);
// Start the auth process using our client ID & the required scopes.
gapi.auth2.init({
client_id: clientId,
scope: scopes
})
.then(function () {
// We're authenticated, let's go...
// Load the webmasters API, then query the API
gapi.client.load('webmasters', 'v3')
.then(retrieveSearchesByQuery);
});
}
// Load the API client and auth library
gapi.load('client:auth2', auth);
At this point, your retrieveSearchesByQuery function will need to be modified since it doesn't need to get a token by argument anymore in order to pass it in the query. The JavaScript client library should include it in the request automatically.
You can also use the API Explorer to check what parameters are supported for a specific query and check the associated request.
If you need to use an externally generated access token, which should be the case with a Service Account, you need to use the gapi.auth.setToken method to sets the OAuth 2.0 token object yourself for the application:
gapi.auth.setToken(token_Object);

Problems with OAuth on node.js

I am trying to get OAuth to work on node.js. I found this in the documentation of node-oauth:
var OAuth= require('oauth').OAuth;
var oa = new OAuth(requestUrl,accessUrl,consumerKey,consumerSecret,"1.0A",responseUrl,"HMAC-SHA1");
The next step in the official tutorial says:
"Then get hold of a valid access token + access token secret as per the normal channels"
What are these "normal channels"?
I know that the user has to authenticate somehow on the "vendor" site and that by some way a response url is called, but I can't find a description how to implement this. Can someone enlighten me?

I'm not sure what OAuth service you are trying to connect to so I'll just use twitter as an example. After you create your OAuth object you need to first request an oauth token. When you get that token, then you need to redirect to, for twitter, their authenticate page which either prompts them to login, then asks if it's ok for the app to login.
oa.getOAuthRequestToken(function(error, oauth_token, oauth_token_secret, results){
if (error) new Error(error.data)
else {
req.session.oauth.token = oauth_token
req.session.oauth.token_secret = oauth_token_secret
res.redirect('https://twitter.com/oauth/authenticate?oauth_token='+oauth_token)
}
});
When you first created the OAuth object, you set a responseURL, or the callback url. It can be anything, for my app its just /oauth/callback. In that callback you receive the oauth verifier token. You then use both the oauth request token and oauth verifier token to request the access tokens. When you receive the access tokens you will also receive anything else they pass, like their username.
app.get('/oauth/callback', function(req, res, next){
if (req.session.oauth) {
req.session.oauth.verifier = req.query.oauth_verifier
var oauth = req.session.oauth
oa.getOAuthAccessToken(oauth.token,oauth.token_secret,oauth.verifier,
function(error, oauth_access_token, oauth_access_token_secret, results){
if (error) new Error(error)
console.log(results.screen_name)
}
);
} else
next(new Error('No OAuth information stored in the session. How did you get here?'))
});
Hope this helps! I had the same problems when I started on this.

The access token is issued to your application after walking the user through the "OAuth dance" (as its affectionately known). This means obtaining a request token and redirecting the user to the provider (Twitter, in this case) for authorization. If the user grants authorization, Twitter redirects the user back to your application with a code that can be exchanged for an access token.
node-oauth can be used to manage this process, but a higher-level library will make it much easier. Passport (which I'm the author of), is one such library. In this case, check out the guide to Twitter authentication, which simplifies the OAuth dance down to a few lines of code.
After that, you can save the access token in your database, and use it to access protected resources in the usual manner using node-oauth.

An update to post tweet to user timeline:
#mattmcmanus, Extending #mattmcmanus nice answer, I would like to post a tweet to timeline. For this, I am using the same code as mattcmanus given above.
Step 1:
oa.getOAuthRequestToken(function(error, oauth_token, oauth_token_secret, results){
if (error) new Error(error.data)
else {
req.session.oauth.token = oauth_token
req.session.oauth.token_secret = oauth_token_secret
res.redirect('https://twitter.com/oauth/authenticate?oauth_token='+oauth_token)
}
});
Step 2:
app.get('/oauth/callback', function(req, res, next){
if (req.session.oauth) {
req.session.oauth.verifier = req.query.oauth_verifier
var oauth = req.session.oauth
oa.getOAuthAccessToken(oauth.token,oauth.token_secret,oauth.verifier,
function(error, oauth_access_token, oauth_access_token_secret, results){
if (error) new Error(error){
console.log(results.screen_name)
}else{
// NEW CODE TO POST TWEET TO TWITTER
oa.post(
"https://api.twitter.com/1.1/statuses/update.json",
oauth_access_token, oauth_access_token_secret,
{"status":"Need somebody to love me! I love OSIpage, http://www.osipage.com"},
function(error, data) {
if(error) console.log(error)
else console.log(data)
}
);
// POST TWEET CODE ENDS HERE
}
}
);
} else
next(new Error('No OAuth information stored in the session. How did you get here?'))
});
I have added oauth_access_token & oauth_access_token_secret in commented code. This will post a tweet update to user's timeline. Happy tweeting!!!