We Keep Coding

Prevent malicious users from abusing and spamming unauthenticated open APIs

Here's a security problem I've encountered a couple of times when building small web-based projects interacting with a REST API service. For example, let's say you're building a casual JavaScript-based game where you want a leaderboard of highscores, so you need to post the scores of users to a database.
The easiest solution would be to build a simple web service, e.g. using PHP, Node.js or Python, that accepts GET request and saves the results to a database. Let's imagine the API looks something like this:
GET https://www.example.com/api/highscore?name=SuperGoat31&score=500
Creating such an API for posting highscores has some obvious drawbacks. A malicious user could write a three-line piece of PHP code to spam the database full of false results, for example:
for ($i = 0; $i < 100; i++) {
file_get_contents("https://www.example.com/api/highscore?name=SuperGoat31&score=5000000");
}
So, I'm looking for a way to prevent that. This mostly relates to small hobby or hackathon projects that just need some kind of protection that will prevent the most obvious of attacks, not large enterprise applications that need strict security. A couple of things I could think of:
1. Some form of authentication
An obvious way to solve this would be to have user accounts and only allow requests from logged-in users. This unfortunately has the drawback of putting up a large barrier for users, who need to get an account first. It would also require building a whole authentication workflow with password recovery and properly encrypting passwords and the like.
2. One-time token based protection
Generate a token on the server side and serve that to the user on first load, then only allow requests that serve that specific token. Simple enough, but also very easy to circumvent by finding the requests in a browser web inspector and using that for the three-line PHP script.
3. Log IP address's and ban when malicious use happens
This could work, but I feel it's not very privacy friendly. Also, logging IP addresses would require GDPR consent from users in Europe. Also doesn't prevent the actual spamming itself so you might to first clean up the mess before you start banning IP addresses.
4. Use an external service
There are services that provide solutions to this problem. For example, in the past I've used Google's reCAPTCHA to prevent malicious use. But that also means integrating an external service, making sure you keep it up to date, concerns about the privacy aspects (esp. regarding a service like reCAPTCHA), etc. It feels a bit much for a weekend project.
5. Throttle requests
I feel this is probably the easiest solution that actually works for a bit. This does require some form of IP address logging (which might give the problems stated in 3), but at least you can delete those IP addresses pretty quickly afterwards.
But I'm sure there are other methods I've missed, so I would be curious to see other ways of tackling this problem.

Taking into account all mentioned limitations, I would recommend using a combination of methods:
Simple session authentication based on one-time token
Script obfuscation
Request encryption with integrity control
Example:
let req_obj = {
user: 'SuperGoat31',
score: 123456,
sessionId: '4d2NhIgMWDuzarfAY0qT3g8U2ax4HCo7',
};
req_obj.hash = someCustomHashFunc(JSON.stringify(req_obj));
// now, req_obj.hash = "y0UXBY0rYkxMrJJPdoSgypd"
let req_string = "https://www.example.com/api/cmd?name=" +
req_obj.user +
"&data=" +
Buffer.from(JSON.stringify(req_obj)).toString('base64');
// now, your requests will look like that:
"https://www.example.com/api/cmd?name=SuperGoat31&data=eyJ1c2VyIjoiU3VwZXJHb2F0MzEiLCJzY29yZSI6MTIzNDU2LCJzZXNzaW9uSWQiOiI0ZDJOaElnTVdEdXphcmZBWTBxVDNnOFUyYXg0SENvNyIsImhhc2giOiJ5MFVYQlkwcllreE1ySkpQZG9TZ3lwZCJ9"
For casual players, this allows start playing very quickly, as no explicit registration is required. Upon generation, token might be saved as cookie for repetitive use, but this is not necessary, single-time use would also suffice. No personal info gathered.
However, if short-term storage of some client information is an option, the token might be not just some random bytes, but an encrypted string, containing some parameters, such as random salt + IP address + nickname + agent id + etc. In this case you may start silently ignore certain requests from fraudulent clients upon detection.
Obviously, this would be very easy to crack for a professional, but this is not our goal. When such simple methods are mixed with several kilobytes of logic of the game and obfuscated, figuring out how to deal with it would require significant amount of knowledge and time, which might serve as a sufficient barrier.
As it is all about balance between convenience and protection, you may implement some additional scoring logic to detect cheating attempts, like final score cannot end with '0', or cannot be even, etc. This would allow you to count cheating attempts (in addition to counting forged requests) and then estimate efficiency of implemented combination of methods.

Your list of solutions are mostly mitigations, and they are good ideas if they are your only tools. The list seems pretty exhaustive.
2 major ways to actually solve this problem are:
Remove the incentive of cheating. There's no point submitting a fake score if you are the only person who can see the score. Think about the purpose of why you even want a global high-score list. Maybe there's another way you can reach your objective that makes it uninteresting (or undesirable) to cheat.
Have the server completely manage (or duplicate) the game state. You can't cheat if the server calculates the score. For example, if you're modelling a chess game the server can compute every valid move, preventing clients from submitting moves that wouldn't be possible.
It's possible that for your specific case neither are possible, but if you can't adopt either of these strategies you are stuck to imperfect detection mechanisms.

I suspect that a perfect solution will be elusive because two of
your wishes are, perhaps, contradictory:
"You need to post the scores of users to a database" but... "prevent
the most obvious of attacks" without "Some form of authentication."
The most obvious of attacks are those from users without some form
of authentication.
You wish this system to work without placing an undue burden on
your users. You wish to avoid the usual login and password
authentication which can be cumbersome for users.
I think there is a way to accomplish what you want by creating a
very simple form of authentication by the use of a one-time token
based protection. And I would also incorporate IP tracking against
abuse. In other words, let's combine your options 1 and 2 and 3 in
the following way.
You already have implied that you will maintain a database, and that
within the database, user names will be unique (otherwise you couldn't
record unique high scores). Let people sign up freely by submitting
their requested user name, which you'll accept if not already used
by someone prior. Track the sign-up requests by IP address to detect
and prevent abuse: too many sign-ups from one IP address within a given timeframe. So far, the burden is all at the server end, not on the user.
When you process a valid sign-up (i.e. new user name) into the
database, you will also generate, record into the database, and return to the user a shared secret (a token) that will be used by the
Time-based One-time Password (TOTP) algorithm.
Don't reinvent this.
See:
Time-based One-Time Password
FreeOTP
OneTimePass
When you return a token to the user, it will be in the form of a "QR Code"
QR code
which the user will scan and store with his "Google Authenticator" or
equivalent TOTP application.
When the user returns to your web site to update his high score, he
will authenticate himself using his Google Authenticator" or
equivalent TOTP application. These are often used for "second factor"
authentication, 2FA (Multi-factor authentication), but because
of your need for less strict security, you'll be using the TOTP
authentication as the primary and only form of authentication.
So we have combined a form of authentication which doesn't place a
very high burden on the user (apps already widely available and in
use), with one-time token based protection (provided by the TOTP
app) and a little bit of IP address-based abuse protection for the initial sign-ups.
On of the weaknesses of my proposal is that a user may share his
TOTP token with another person, who may then impersonate him. But this
is no different from the risk of password sharing. And there will
be no "recover my lost password" option.

I would tackle this in a slightly different way: usernames/gamertags. Depending on how frequently you find gamertags and usernames sharing the same IP. So if you only accept a maximum of, say, 5 gamertags per IP, and you also throttle the frequency of updates per gamertag, you have a fairly spam-resistant system.

I would recommend a mix of code obfuscation and using web sockets to request the score, rather than post the score. Something like socket.io (https://socket.io/) where the server sends a request with a code in it and your game responds with the score and that code changed in some way.
Obviously a hacker could look through your code for how your game responds to requests and rewrite it, which is where the obfuscation is important, but it does at least hide the obvious network traffic and prevents them posting scores whenever they feel like it.

I would suggest using reCAPTCHA V2.
Admittedly, v3 provides better protection, but it is hard to implement, so go with v2.
Come on, it is just a few lines of code.
How it should work (according to me):
You are at the main page willing to play the game
You solve the reCAPTCHA
Then the app sends a one-time token with a script tag which establishes a websocket request with your server (using socket.io) with the one-time token and then it is destroyed immediately (from the server as well as the client) after establishment of a connection
Your server validates the token and accepts the request of websocket and then it will send the HTML content
Just create a div and set the value using obj.innerHTML
You can use styles in body (I guess)
And the most important point is obfuscating your code.
Security
Websockets are harder to reverse engineer in a test environment
Even if they create a web socket, it won't respond, because they don't know the one-time token
It prevents script blocking (as the script loads everything on the page)
It provides real-time communication
The only way out is to somehow get your hands on Google's reCAPTCHA token which is impossible, because it means going against Google
You can’t reuse any token (however immediate it be), because it was destroyed from both the sides
One more last tip: set a timeout for the one-time token to about 15 seconds
How will it help? It will prevent someone (extremely malicious) from pausing the Chrome debugger and get the token and put it in their stuff as 15 seconds is ok for slow networks also, but not a human

3D S Payment callback issue

I'm working on a payment system with React-Native. I want to do research on how to apply 3D payment methods. I want to listen to an event happening on my server on the client side. actually i only need this for the following reason. I feel there is something I got wrong here.
3D Secure Steps
I send the product to be purchased to a service named iyzico with
credit card information.
If the information entered is correct, it gives me an answer as below.
There is a special field here and I need to decrypto it and show it to the user.
In fact, this crypoto information contains an html page.
By decoding this code, I show the user an html page.
The password sent to the user's phone via SMS is entered on that
screen and presses the confirmation button.
The part I don't understand starts here. The person is in a true asynchronous. I want to callback. Because he can enter or cancel the password sent by SMS whenever he wants.
I'm not wondering how the process turned out. How will we inform the client application only in this case?
Should we use push notifications or other push services for this?
The client has to get information about this process. According to the information he receives, I would like to suggest that your payment is successful or your balance is insufficient.
I know that I should avoid making circular calls.
In short, how should I listen to the client for an event that will take place on my server? Which would be the best method?
I am working on React-native and I do not want to include push services in the application just for this. It is costly for me to this. I believe it is a more beautiful solution.
What do you think about this subject?
Thanks.

You either need push notifications or long polling. https://ably.com/blog/websockets-vs-long-polling

How avoid multiple POST as complete request information comes in network tab of browsers

We have a form, if a person fills that form, we are hitting POST API.
When user submits the form, complete INFO of API (request url, headers) is visible, I was thinking if someone starts hitting this POST API through server side code, our tables would be filled with junk values?
How can we avoid this bulk data entering the system through API being directly hit by C# client, POSTMAN or any other server side clients?

That's a really open ended question, but I don't think there's a way to do exactly what you ask (detect with certainty if a request originates from a browser or not), nor should you if it were possible - users may have a legitimate reason to use something other than a browser, or someone might want to integrate your API into a larger process, etc.
You should handle this via authentication - require the user to be authenticated with your API, then if they misbehave you can disable their user accounts (either manually or automatically with some sort of abuse detection).

There are times when authentication is not possible or desirable for a web form. At those times a I have used a key generated on the server that I embed in the form.
The form sends back the key along with the rest of the data and you can then make any decisions you need to make based on the key. Like limiting the rate of submissions, allowing only one submission and then expiring the key, etc...

Use a captcha: a challenge is presented to the user to prove being a human, usually an image that contains some garbled text is shown to the person filling the form and (s)he is required to transcribe the text content of the image in the form. If the captcha solution is wrong or missing that POST request to the API will be discarded.
Many types of challenges are commonly used such as audio/visual/logic/mini-puzzles. You can also customize your challenge in a way that fits best with your form. For example Google provides ReCaptcha, a captcha service with js and server api.
This helps you prevent or mitigate bots, as the captcha is a challenge that is very hard for computers but easy for humans.
Using one-time captcha tokens also prevents the replay attacks you are worried about. Also, checking that the ip that received the challenge and the ip resolving it are same helps mitigate other tricks.
This still leaves room for a determined human to spam your form. So you should also keep track of the number of submissions and throttle them by ip.

How do I handle communication between multiple Node.js servers for each user?

I'm trying to make a user log in just once, and have his information on all the servers. Any changes made to the user's information will instantly be available for all servers. Is this possible to do without having each user "log in" separately for each server?
Sort of like the $_SESSION for php, but for Node.js
Design 1 -
What I think would be best to do, but don't know how to share socket data between servers, perhaps using something like PHP's $_SESSION?
Design 2 -
What I'm currently doing:
User uses socket.emit to main.js
main.js adds user information onto the emit
main.js emits to the appropriate server
Appropriate server emits back to main.js
main.js finally emits back to user
This seems awfully inefficient and feels wrong

If your information is primarily static, you can try something similar to JWT. These are cryptographically signed tokens that your authenticating server can provide and the user can carry around. This token may contain information about the user that you want each server to have available without having the user accessing it.
If it's not, you may be looking into sharing a database across all servers, and have that be the point of synchronization between them.
Updates based on our comments (so they can be removed later):
If you decide to use auto-contained JWT tokens, you don't need to be making trips to the database at all. These tokens will contain all the information required, but it will be transparent to the end user that won't have insight into their contents.
Also, once you understand the JWT standard, you don't necessarily have to work with JSON objects, since it is just the serialization approach that you can switch by another one.
You'd provide one of these tokens to your user on authentication (or whenever required), and then you'd require that user to provide that token to the other servers when requesting information or behavior from them. The token will become your synchronization approach.

SINGLE SIGN ON SECURITY THREAT! FACEBOOK access_token broadcast in the open/clear

02/20/2011:
It was confirmed by Facebook today that indeed there is one call in which the access_token is broadcast in the open . . . it just happens to be one call I use to make sure that the USER is still logged in before saving to my application database. Their recommendation was to use the SSL option provided as of last month for canvase and facebook as a whole. For the most part the Auth and Auth are secure.
Findings:
Subsequent to my posting there was a remark made that this was not really a question but I thought I did indeed postulate one. So that there is no ambiquity here is the question with a lead in:
Since there is no data sent from Facebook during the Canvas Load process that is not at some point divulged, including the access_token, session and other data that could uniquely identify a user, does any one see any other way other than adding one more layer, i.e., a password, sent over the wire via HTTPS along with the access_toekn, that will insure unique untampered with security by the user?
Using Wireshark I captured the local broadcast while loading my Canvas Application page. I was hugely surprised to see the access_token broadcast in the open, viewable for any one to see. This access_token is appended to any https call to the Facebook OpenGraph API.
Using facebook as a single click log on has now raised huge concerns for me. It is stored in a session object in memory and the cookie is cleared upon app termination and after reviewing the FB.Init calls I saw a lot of HTTPS calls so I assumed the access_token was always encrypted.
But last night I saw in the status bar a call from what was simply an http call that included the App ID so I felt I should sniff the Application Canvas load sequence.
Today I did sniff the broadcast and in the attached image you can see that there are http calls with the access_token being broadcast in the open and clear for anyone to gain access to.
Am I missing something, is what I am seeing and my interpretation really correct. If any one can sniff and get the access_token they can theorically make calls to the Graph API via https, even though the call back would still need to be the site established in Facebook's application set up.
But what is truly a security threat is anyone using the access_token for access to their own site. I do not see the value of a single sign on via Facebook if the only thing that was established as secure was the access_token - becuase for what I can see it clearly is not secure. Access tokens that never have an expire date do not change. Access_tokens are different for every user, to access to another site could be held tight to just a single user, but compromising even a single user's data is unacceptable.
http://www.creatingstory.com/images/InTheOpen.png
Went back and did more research on this:
FINDINGS:
Went back an re ran the canvas application to verify that it was not any of my code that was not broadcasting.
In this call: HTTP GET /connect.php/en_US/js/CacheData HTTP/1.1
The USER ID is clearly visible in the cookie. So USER_ID's are fully visible, but they are already. Anyone can go to pretty much any ones page and hover over the image and see the USER ID. So no big threat. APP_ID are also easily obtainable - but . . .
http://www.creatingstory.com/images/InTheOpen2.png
The above file clearly shows the FULL ACCESS TOKEN clearly in the OPEN via a Facebook initiated call.
Am I wrong. TELL ME I AM WRONG because I want to be wrong about this.
I have since reset my app secret so I am showing the real sniff of the Canvas Page being loaded.
Additional data 02/20/2011:
#ifaour - I appreciate the time you took to compile your response.
I am pretty familiar with the OAuth process and have a pretty solid understanding of the signed_request unpacking and utilization of the access_token. I perform a substantial amount of my processing on the server and my Facebook server side flows are all complete and function without any flaw that I know of. The application secret is secure and never passed to the front end application and is also changed regularly. I am being as fanatical about security as I can be, knowing there is so much I don’t know that could come back and bite me.
Two huge access_token issues:
The issues concern the possible utilization of the access_token from the USER AGENT (browser). During the FB.INIT() process of the Facebook JavaScript SDK, a cookie is created as well as an object in memory called a session object. This object, along with the cookie contain the access_token, session, a secret, and uid and status of the connection. The session object is structured such that is supports both the new OAuth and the legacy flows. With OAuth, the access_token and status are pretty much al that is used in the session object.
The first issue is that the access_token is used to make HTTPS calls to the GRAPH API. If you had the access_token, you could do this from any browser:
https://graph.facebook.com/220439?access_token=...
and it will return a ton of information about the user. So any one with the access token can gain access to a Facebook account. You can also make additional calls to any info the user has granted access to the application tied to the access_token. At first I thought that a call into the GRAPH had to have a Callback to the URL established in the App Setup, but I tested it as mentioned below and it will return info back right into the browser. Adding that callback feature would be a good idea I think, tightens things up a bit.
The second issue is utilization of some unique private secured data that identifies the user to the third party data base, i.e., like in my case, I would use a single sign on to populate user information into my database using this unique secured data item (i.e., access_token which contains the APP ID, the USER ID, and a hashed with secret sequence). None of this is a problem on the server side. You get a signed_request, you unpack it with secret, make HTTPS calls, get HTTPS responses back. When a user has information entered via the USER AGENT(browser) that must be stored via a POST, this unique secured data element would be sent via HTTPS such that they are validated prior to data base insertion.
However, If there is NO secured piece of unique data that is supplied via the single sign on process, then there is no way to guarantee unauthorized access. The access_token is the one piece of data that is utilized by Facebook to make the HTTPS calls into the GRAPH API. it is considered unique in regards to BOTH the USER and the APPLICATION and is initially secure via the signed_request packaging. If however, it is subsequently transmitted in the clear and if I can sniff the wire and obtain the access_token, then I can pretend to be the application and gain the information they have authorized the application to see. I tried the above example from a Safari and IE browser and it returned all of my information to me in the browser.
In conclusion, the access_token is part of the signed_request and that is how the application initially obtains it. After OAuth authentication and authorization, i.e., the USER has logged into Facebook and then runs your app, the access_token is stored as mentioned above and I have sniffed it such that I see it stored in a Cookie that is transmitted over the wire, resulting in there being NO UNIQUE SECURED IDENTIFIABLE piece of information that can be used to support interaction with the database, or in other words, unless there were one more piece of secure data sent along with the access_token to my database, i.e., a password, I would not be able to discern if it is a legitimate call. Luckily I utilized secure AJAX via POST and the call has to come from the same domain, but I am sure there is a way to hijack that.
I am totally open to any ideas on this topic on how to uniquely identify my USERS other than adding another layer (password) via this single sign on process or if someone would just share with me that I read and analyzed my data incorrectly and that the access_token is always secure over the wire.
Mahalo nui loa in advance.

I am not terribly familiar with Facebook's authentication/authorization methods, but I do believe that they implement oauth (or something close to it) for delegation, distributed authorization, and "single sign-on".
OAuth is described by RFC-5849
EDIT: Facebook Uses OAuth 2.0 which is still in working draft.
In OAuth, and similar systems, the "access_token" is only part of the picture. There is also typically a secret key, which is known only by the service provider (facebook) and the client application (your app). The secret key is the only part that is expected to stay secret - and that part is never sent over the wire (after it's initial issuance).
In the case of Facebook, I think the secret key is assigned to you when you register your application to use their API, and the 'access_token' is returned to you for a given user, whenever the user agrees to allow your app to access their info.
Messages are sent in the clear, including the user's username, and the relevant "access_token"; However, each message must also include a valid signature in order to be accepted by the server. The signature is a cryptographically computed string, that is created using a technique called HMAC.
Computing the HMAC signature requires both the token and the secret, and includes other key parts of the message as well. Each signature is unique for the given message contents; and each message uses a nonce to ensure that no two messages can ever be exactly identical.
When the server receives a signed message, it starts by extracting the access_token (clear-text), and determining which app the token was issued for. It then retrieves the matching secret from its own local database (the secret is not contained in the message). Finally, the server uses the clear-text message, the clear-text access_token, and the secret to compute the expected HMAC signature for the message. If the computed signature matches the signature on the received message, then the message must have been sent by someone who knows the same secret (i.e. your application).
Have a look at Section 3.1 of RFC-5849 for an OAuth specific example, and further elaboration on the details.
Incidentally, the same approach is used by Amazon to control access to S3 and EC2, as well as most other service providers that offer API access with long-term authorization. Suffice it to say - this approach is secure. It might be a little counter-intuitive at first, but it makes sense once you think it through.
Adding a few links and quotes from Facebook Documentation:
Facebook is indeed using the HMAC-SHA256 algorithm. Registration document (PHP Example reading signed_request section).
Always verify the signed_request:
If you are unable to validate the
signed_request because you can't embed
your application secret (e.g. in
javascript or a desktop application)
then you MUST only use one piece of
information from the payload, the
oauth_token.
The Authentication Document contains a lot of useful info about the different flows you may use to authenticate a user. Also read the Security Considerations section at the bottom of the page:
Cross site request forgery is an
attack in which an trusted
(authenticated and authorized) user
unknowingly performs an action on
website. To prevent this attack, you
should pass an identifier in the state
parameter, and then validate the state
parameter matches on the response. We
strongly recommend that any app
implementing Facebook user login
implement CSRF protection using this
mechanism.

It was confirmed by Facebook that indeed there is one call in which the access_token is broadcast in the open - it just happens to be one call I use to make sure that the user is still logged in before saving to my application database. Their recommendation was to use the SSL option provided as of last month for canvas and Facebook as a whole. For the most part the Auth and Auth are secure.
To ensure a secure interface between a third party application and a Facebook application or even any website that uses Facebook Single Sign on, an identity question would provide the extra layer when used in conjunction with the access_token.
Either that or require your users to use Facebook with the new SSL feature of Facebook and Facebook Canvas Applications. If the access_token is broadcast in the open it cannot be used to uniquely identify anyone in your third party database when needing to have a confirmed identity before database interactions.