Here's a security problem I've encountered a couple of times when building small web-based projects interacting with a REST API service. For example, let's say you're building a casual JavaScript-based game where you want a leaderboard of highscores, so you need to post the scores of users to a database.
The easiest solution would be to build a simple web service, e.g. using PHP, Node.js or Python, that accepts GET request and saves the results to a database. Let's imagine the API looks something like this:
GET https://www.example.com/api/highscore?name=SuperGoat31&score=500
Creating such an API for posting highscores has some obvious drawbacks. A malicious user could write a three-line piece of PHP code to spam the database full of false results, for example:
for ($i = 0; $i < 100; i++) {
file_get_contents("https://www.example.com/api/highscore?name=SuperGoat31&score=5000000");
}
So, I'm looking for a way to prevent that. This mostly relates to small hobby or hackathon projects that just need some kind of protection that will prevent the most obvious of attacks, not large enterprise applications that need strict security. A couple of things I could think of:
1. Some form of authentication
An obvious way to solve this would be to have user accounts and only allow requests from logged-in users. This unfortunately has the drawback of putting up a large barrier for users, who need to get an account first. It would also require building a whole authentication workflow with password recovery and properly encrypting passwords and the like.
2. One-time token based protection
Generate a token on the server side and serve that to the user on first load, then only allow requests that serve that specific token. Simple enough, but also very easy to circumvent by finding the requests in a browser web inspector and using that for the three-line PHP script.
3. Log IP address's and ban when malicious use happens
This could work, but I feel it's not very privacy friendly. Also, logging IP addresses would require GDPR consent from users in Europe. Also doesn't prevent the actual spamming itself so you might to first clean up the mess before you start banning IP addresses.
4. Use an external service
There are services that provide solutions to this problem. For example, in the past I've used Google's reCAPTCHA to prevent malicious use. But that also means integrating an external service, making sure you keep it up to date, concerns about the privacy aspects (esp. regarding a service like reCAPTCHA), etc. It feels a bit much for a weekend project.
5. Throttle requests
I feel this is probably the easiest solution that actually works for a bit. This does require some form of IP address logging (which might give the problems stated in 3), but at least you can delete those IP addresses pretty quickly afterwards.
But I'm sure there are other methods I've missed, so I would be curious to see other ways of tackling this problem.
Taking into account all mentioned limitations, I would recommend using a combination of methods:
Simple session authentication based on one-time token
Script obfuscation
Request encryption with integrity control
Example:
let req_obj = {
user: 'SuperGoat31',
score: 123456,
sessionId: '4d2NhIgMWDuzarfAY0qT3g8U2ax4HCo7',
};
req_obj.hash = someCustomHashFunc(JSON.stringify(req_obj));
// now, req_obj.hash = "y0UXBY0rYkxMrJJPdoSgypd"
let req_string = "https://www.example.com/api/cmd?name=" +
req_obj.user +
"&data=" +
Buffer.from(JSON.stringify(req_obj)).toString('base64');
// now, your requests will look like that:
"https://www.example.com/api/cmd?name=SuperGoat31&data=eyJ1c2VyIjoiU3VwZXJHb2F0MzEiLCJzY29yZSI6MTIzNDU2LCJzZXNzaW9uSWQiOiI0ZDJOaElnTVdEdXphcmZBWTBxVDNnOFUyYXg0SENvNyIsImhhc2giOiJ5MFVYQlkwcllreE1ySkpQZG9TZ3lwZCJ9"
For casual players, this allows start playing very quickly, as no explicit registration is required. Upon generation, token might be saved as cookie for repetitive use, but this is not necessary, single-time use would also suffice. No personal info gathered.
However, if short-term storage of some client information is an option, the token might be not just some random bytes, but an encrypted string, containing some parameters, such as random salt + IP address + nickname + agent id + etc. In this case you may start silently ignore certain requests from fraudulent clients upon detection.
Obviously, this would be very easy to crack for a professional, but this is not our goal. When such simple methods are mixed with several kilobytes of logic of the game and obfuscated, figuring out how to deal with it would require significant amount of knowledge and time, which might serve as a sufficient barrier.
As it is all about balance between convenience and protection, you may implement some additional scoring logic to detect cheating attempts, like final score cannot end with '0', or cannot be even, etc. This would allow you to count cheating attempts (in addition to counting forged requests) and then estimate efficiency of implemented combination of methods.
Your list of solutions are mostly mitigations, and they are good ideas if they are your only tools. The list seems pretty exhaustive.
2 major ways to actually solve this problem are:
Remove the incentive of cheating. There's no point submitting a fake score if you are the only person who can see the score. Think about the purpose of why you even want a global high-score list. Maybe there's another way you can reach your objective that makes it uninteresting (or undesirable) to cheat.
Have the server completely manage (or duplicate) the game state. You can't cheat if the server calculates the score. For example, if you're modelling a chess game the server can compute every valid move, preventing clients from submitting moves that wouldn't be possible.
It's possible that for your specific case neither are possible, but if you can't adopt either of these strategies you are stuck to imperfect detection mechanisms.
I suspect that a perfect solution will be elusive because two of
your wishes are, perhaps, contradictory:
"You need to post the scores of users to a database" but... "prevent
the most obvious of attacks" without "Some form of authentication."
The most obvious of attacks are those from users without some form
of authentication.
You wish this system to work without placing an undue burden on
your users. You wish to avoid the usual login and password
authentication which can be cumbersome for users.
I think there is a way to accomplish what you want by creating a
very simple form of authentication by the use of a one-time token
based protection. And I would also incorporate IP tracking against
abuse. In other words, let's combine your options 1 and 2 and 3 in
the following way.
You already have implied that you will maintain a database, and that
within the database, user names will be unique (otherwise you couldn't
record unique high scores). Let people sign up freely by submitting
their requested user name, which you'll accept if not already used
by someone prior. Track the sign-up requests by IP address to detect
and prevent abuse: too many sign-ups from one IP address within a given timeframe. So far, the burden is all at the server end, not on the user.
When you process a valid sign-up (i.e. new user name) into the
database, you will also generate, record into the database, and return to the user a shared secret (a token) that will be used by the
Time-based One-time Password (TOTP) algorithm.
Don't reinvent this.
See:
Time-based One-Time Password
FreeOTP
OneTimePass
When you return a token to the user, it will be in the form of a "QR Code"
QR code
which the user will scan and store with his "Google Authenticator" or
equivalent TOTP application.
When the user returns to your web site to update his high score, he
will authenticate himself using his Google Authenticator" or
equivalent TOTP application. These are often used for "second factor"
authentication, 2FA (Multi-factor authentication), but because
of your need for less strict security, you'll be using the TOTP
authentication as the primary and only form of authentication.
So we have combined a form of authentication which doesn't place a
very high burden on the user (apps already widely available and in
use), with one-time token based protection (provided by the TOTP
app) and a little bit of IP address-based abuse protection for the initial sign-ups.
On of the weaknesses of my proposal is that a user may share his
TOTP token with another person, who may then impersonate him. But this
is no different from the risk of password sharing. And there will
be no "recover my lost password" option.
I would tackle this in a slightly different way: usernames/gamertags. Depending on how frequently you find gamertags and usernames sharing the same IP. So if you only accept a maximum of, say, 5 gamertags per IP, and you also throttle the frequency of updates per gamertag, you have a fairly spam-resistant system.
I would recommend a mix of code obfuscation and using web sockets to request the score, rather than post the score. Something like socket.io (https://socket.io/) where the server sends a request with a code in it and your game responds with the score and that code changed in some way.
Obviously a hacker could look through your code for how your game responds to requests and rewrite it, which is where the obfuscation is important, but it does at least hide the obvious network traffic and prevents them posting scores whenever they feel like it.
I would suggest using reCAPTCHA V2.
Admittedly, v3 provides better protection, but it is hard to implement, so go with v2.
Come on, it is just a few lines of code.
How it should work (according to me):
You are at the main page willing to play the game
You solve the reCAPTCHA
Then the app sends a one-time token with a script tag which establishes a websocket request with your server (using socket.io) with the one-time token and then it is destroyed immediately (from the server as well as the client) after establishment of a connection
Your server validates the token and accepts the request of websocket and then it will send the HTML content
Just create a div and set the value using obj.innerHTML
You can use styles in body (I guess)
And the most important point is obfuscating your code.
Security
Websockets are harder to reverse engineer in a test environment
Even if they create a web socket, it won't respond, because they don't know the one-time token
It prevents script blocking (as the script loads everything on the page)
It provides real-time communication
The only way out is to somehow get your hands on Google's reCAPTCHA token which is impossible, because it means going against Google
You can’t reuse any token (however immediate it be), because it was destroyed from both the sides
One more last tip: set a timeout for the one-time token to about 15 seconds
How will it help? It will prevent someone (extremely malicious) from pausing the Chrome debugger and get the token and put it in their stuff as 15 seconds is ok for slow networks also, but not a human
I am currently working on a project whose backend I will be writing using Flask (cannot change that), and the frontend will be developed using React by someone else (cannot change that too). I am currently working on the login functionality and I have a few questions. I am really just beginning backend development, so please excuse me if I have some redundant questions or beginner-level questions.
I am curious as to how the login-system as a whole will be handled, not just in terms of actual login - but more so when it comes to restricting access from non-logged in users.
The tutorials and online courses I have taken on Flask have done one of 2 approaches.
The first approach was using Flask-Login to login a user and ensure that they are logged in as they navigate to different websites. This was pretty straightforward to understand and relatively easy to implement. However, (and please correct me if I'm wrong), this would require all the navigation and redirecting to be handled server-side. Therefore, we would not be utilizing React's ability to create single-page websites (instead relying on hyperlinks from each page to the next), which is far from ideal and something that we'd like to avoid if possible.
The second approach was using JWT tokens. Again correct me if I'm wrong, but we would only be able to check that the JWT token is correct if it is included in an HTTP request, which again makes creating single-page websites not feasible. Another issue with JWT is that the tokens expire fairly quickly (around 15 minutes as far as my research tells me), which is not suitable for our project because it would require users to be logged in for a longer duration. I have read about refreshing JWT tokens, but I would like to know more about the feasibility of such a process given the short expiration time (we would have to do a lot of refreshes).
My main question is:
How can we create single-page websites while preventing non-logged in users from accessing restricted pages?
But also please feel free to give me any feedback or concerns about what I have talked about.
Thank you in advance and sorry for the long post
Just use JWT, you can add the token to the header of the HTTP request and you can set the lifetime (JWT_EXPIRATION_DELTA) of the token - check this link https://pythonhosted.org/Flask-JWT
I have a JavaScript SPA which consumes a backend REST API built with Django (Django Rest Framework). It's a small academic project which is likely to attract only a few hundred users on a regular basis. Our initial design assumed that users might wish to save data, but recent consultation with potential clients has made us question whether we need to incorporate this feature. So this would completely remove any need for POST requests to the backend API, leaving only GETs. These GETs (via axios) contain only path parameters, no query params, and return small JSON payloads, used to render SVG components in the application (this data is read-only, static data, already stored in the backend db).
The assumed need to cater for user uploads led us to also add user login/logout and account management features. We even started to look at integrating Auth0. But if we remove user upload entirely do we even need to secure our API endpoints in this way?
From what's given in the OP, answer is, No, you do not need authentication.
Irrespective of the HTTP verb used, just ask yourself, whatever functionality you are exposing through your REST apis, at least, do you care to find out who did what using your APIs? if not you don't need Authentication
Basically,
Do you want to restrict access to any APIs?
Do you want to tie an action to any persona?
Do you want to audit, (who did what when)?
If answer to all these 3 is no, you do not need authentication.
I'm trying to create a webpage that can incorporate LinkedIn info's (profile,people,company, etc...).
The things that it can/would do are the following:
When the user enters a name that is registered in LinkedIn, he gets the following
*Name, Company, Email
*List of LinkedIn messages that are waiting for reply
The same process goes on everytime the user adds a profile, I'm planning to use the Profile API of LinkedIn to get the Name, Company and Email but I can't find a working example to be my basis.
As for the 2nd one I still don't know how to get the LinkedIn messages.
Here's my Layout and expected result.
How can I achieved this? Opinions and Suggestions are highly appreciated tnx
This is far to broad a question for me to invest the necessary time in to figure the answers (multiple) for you, but do let me give you some hints. First of all, from my experience with the linkedin API not all the data you wish to access is available (do double check this though, I used the API quite awhile back and stuff might have changed in the meantime). As this data is not available through the API the only alternative would be to somehow bypass the cross domain policy, which in conclusion would require the user to install a chrome extension/firefox plugin which will function as a proxy for your application or even 'better', make you entire application a browser plugin based web app. Not that I am a fan of those whatsoever but if you application is meant in any way whatsoever as a linkedin (dedicated) plugin (probably as part of a greater service you're developing) then it might make most sense.
The whole system you are describing is very long winded and requires a large amount of development time. Alot of the data is not accessible directly or indirectly too. You cannot get email address's out from the API as a security feature (bots could just harvest emails for marketing campaigns).
First of all, you will need to make an application that allows for oAuth2 connections with the linkedin API service. People will log onto your website, click to join their linkedin account with your website and your website will receive back an access token to do the calls.
You will then need to build the queries which will access the data you require. The linkedin API documentation (http://developer.linkedin.com/) isn't greatly indepth but it gives you a good understand and points you where you need to go. There are also a couple of pre-done php API's around such as https://code.google.com/p/simple-linkedinphp/.
I have worked with many API's from twitters, facebooks and LinkedIn's and they all require a lot of back-end work to make sure that they are secure and get the correct data.
It would take me hours to go through exactly how to do it and has taken me many hours to get a solid implementation in place and working with all the different calls available.
If you have minimal coding knowledge, it would be best to go to an external company with a large amount of resources and knowledge in the field who can do it for you. Otherwise it may take many months to get a working prototype.
I've written a bit of JavaScript that will fetch all of the posts on a Facebook Page. The URL with which I do that is this:
http://graph.facebook.com/cocacola/feed?limit=5&callback=facebookResponse
and this worked fine and dandy, right up until last week sometime, when I started seeing oauth errors.
I've searched for an hour or so on Stack, and seen plenty of other people are trying to do this, - but none have been asked after all of my requests to FB started returning these oauth errors.
It really doesn't make sense, - I'm trying to access publicly available data from company pages. I can still do so without oauth (albeit in a limited fashion) via RSS.
Anyhow, I'm hoping someone can clue me in as to how to get this PUBLICLY AVAILABLE information without having to go through the rigmarole of getting an application ID, an authorization token, etc.
This is a recent not-so-recent change, but you now need an access token to access /feed and /posts. Annoying, but at least it's navigable.
Edit: updated the link, which has broken in the many years since this post was relevant. Here's the relevant text from that post for future posterity:
Breaking change: Graph API PROFILE_ID/feed and PROFILE_ID/posts requires access_token
The Graph API PROFILE_ID/feed/ for a Page, Application, User or Group and PROFILE_ID/posts for a Page or User will now require a vaild [sic] access_token to access the wall or posts of the corresponding object (where previously no access_token was required). This will also affect direct FQL queries to the stream table, when querying for posts on a wall.
You will need to pass a valid app or user access_token to access this functionality. Please update your code if you are calling this API without an access token. This change will go live a week from today - Friday(June 3rd). We have updated the Roadmap to reflect this change.
Moving forward, you should always pass a valid app or user access_token with all API requests.