We Keep Coding

Prevent malicious users from abusing and spamming unauthenticated open APIs

Here's a security problem I've encountered a couple of times when building small web-based projects interacting with a REST API service. For example, let's say you're building a casual JavaScript-based game where you want a leaderboard of highscores, so you need to post the scores of users to a database.
The easiest solution would be to build a simple web service, e.g. using PHP, Node.js or Python, that accepts GET request and saves the results to a database. Let's imagine the API looks something like this:
GET https://www.example.com/api/highscore?name=SuperGoat31&score=500
Creating such an API for posting highscores has some obvious drawbacks. A malicious user could write a three-line piece of PHP code to spam the database full of false results, for example:
for ($i = 0; $i < 100; i++) {
file_get_contents("https://www.example.com/api/highscore?name=SuperGoat31&score=5000000");
}
So, I'm looking for a way to prevent that. This mostly relates to small hobby or hackathon projects that just need some kind of protection that will prevent the most obvious of attacks, not large enterprise applications that need strict security. A couple of things I could think of:
1. Some form of authentication
An obvious way to solve this would be to have user accounts and only allow requests from logged-in users. This unfortunately has the drawback of putting up a large barrier for users, who need to get an account first. It would also require building a whole authentication workflow with password recovery and properly encrypting passwords and the like.
2. One-time token based protection
Generate a token on the server side and serve that to the user on first load, then only allow requests that serve that specific token. Simple enough, but also very easy to circumvent by finding the requests in a browser web inspector and using that for the three-line PHP script.
3. Log IP address's and ban when malicious use happens
This could work, but I feel it's not very privacy friendly. Also, logging IP addresses would require GDPR consent from users in Europe. Also doesn't prevent the actual spamming itself so you might to first clean up the mess before you start banning IP addresses.
4. Use an external service
There are services that provide solutions to this problem. For example, in the past I've used Google's reCAPTCHA to prevent malicious use. But that also means integrating an external service, making sure you keep it up to date, concerns about the privacy aspects (esp. regarding a service like reCAPTCHA), etc. It feels a bit much for a weekend project.
5. Throttle requests
I feel this is probably the easiest solution that actually works for a bit. This does require some form of IP address logging (which might give the problems stated in 3), but at least you can delete those IP addresses pretty quickly afterwards.
But I'm sure there are other methods I've missed, so I would be curious to see other ways of tackling this problem.

Taking into account all mentioned limitations, I would recommend using a combination of methods:
Simple session authentication based on one-time token
Script obfuscation
Request encryption with integrity control
Example:
let req_obj = {
user: 'SuperGoat31',
score: 123456,
sessionId: '4d2NhIgMWDuzarfAY0qT3g8U2ax4HCo7',
};
req_obj.hash = someCustomHashFunc(JSON.stringify(req_obj));
// now, req_obj.hash = "y0UXBY0rYkxMrJJPdoSgypd"
let req_string = "https://www.example.com/api/cmd?name=" +
req_obj.user +
"&data=" +
Buffer.from(JSON.stringify(req_obj)).toString('base64');
// now, your requests will look like that:
"https://www.example.com/api/cmd?name=SuperGoat31&data=eyJ1c2VyIjoiU3VwZXJHb2F0MzEiLCJzY29yZSI6MTIzNDU2LCJzZXNzaW9uSWQiOiI0ZDJOaElnTVdEdXphcmZBWTBxVDNnOFUyYXg0SENvNyIsImhhc2giOiJ5MFVYQlkwcllreE1ySkpQZG9TZ3lwZCJ9"
For casual players, this allows start playing very quickly, as no explicit registration is required. Upon generation, token might be saved as cookie for repetitive use, but this is not necessary, single-time use would also suffice. No personal info gathered.
However, if short-term storage of some client information is an option, the token might be not just some random bytes, but an encrypted string, containing some parameters, such as random salt + IP address + nickname + agent id + etc. In this case you may start silently ignore certain requests from fraudulent clients upon detection.
Obviously, this would be very easy to crack for a professional, but this is not our goal. When such simple methods are mixed with several kilobytes of logic of the game and obfuscated, figuring out how to deal with it would require significant amount of knowledge and time, which might serve as a sufficient barrier.
As it is all about balance between convenience and protection, you may implement some additional scoring logic to detect cheating attempts, like final score cannot end with '0', or cannot be even, etc. This would allow you to count cheating attempts (in addition to counting forged requests) and then estimate efficiency of implemented combination of methods.

Your list of solutions are mostly mitigations, and they are good ideas if they are your only tools. The list seems pretty exhaustive.
2 major ways to actually solve this problem are:
Remove the incentive of cheating. There's no point submitting a fake score if you are the only person who can see the score. Think about the purpose of why you even want a global high-score list. Maybe there's another way you can reach your objective that makes it uninteresting (or undesirable) to cheat.
Have the server completely manage (or duplicate) the game state. You can't cheat if the server calculates the score. For example, if you're modelling a chess game the server can compute every valid move, preventing clients from submitting moves that wouldn't be possible.
It's possible that for your specific case neither are possible, but if you can't adopt either of these strategies you are stuck to imperfect detection mechanisms.

I suspect that a perfect solution will be elusive because two of
your wishes are, perhaps, contradictory:
"You need to post the scores of users to a database" but... "prevent
the most obvious of attacks" without "Some form of authentication."
The most obvious of attacks are those from users without some form
of authentication.
You wish this system to work without placing an undue burden on
your users. You wish to avoid the usual login and password
authentication which can be cumbersome for users.
I think there is a way to accomplish what you want by creating a
very simple form of authentication by the use of a one-time token
based protection. And I would also incorporate IP tracking against
abuse. In other words, let's combine your options 1 and 2 and 3 in
the following way.
You already have implied that you will maintain a database, and that
within the database, user names will be unique (otherwise you couldn't
record unique high scores). Let people sign up freely by submitting
their requested user name, which you'll accept if not already used
by someone prior. Track the sign-up requests by IP address to detect
and prevent abuse: too many sign-ups from one IP address within a given timeframe. So far, the burden is all at the server end, not on the user.
When you process a valid sign-up (i.e. new user name) into the
database, you will also generate, record into the database, and return to the user a shared secret (a token) that will be used by the
Time-based One-time Password (TOTP) algorithm.
Don't reinvent this.
See:
Time-based One-Time Password
FreeOTP
OneTimePass
When you return a token to the user, it will be in the form of a "QR Code"
QR code
which the user will scan and store with his "Google Authenticator" or
equivalent TOTP application.
When the user returns to your web site to update his high score, he
will authenticate himself using his Google Authenticator" or
equivalent TOTP application. These are often used for "second factor"
authentication, 2FA (Multi-factor authentication), but because
of your need for less strict security, you'll be using the TOTP
authentication as the primary and only form of authentication.
So we have combined a form of authentication which doesn't place a
very high burden on the user (apps already widely available and in
use), with one-time token based protection (provided by the TOTP
app) and a little bit of IP address-based abuse protection for the initial sign-ups.
On of the weaknesses of my proposal is that a user may share his
TOTP token with another person, who may then impersonate him. But this
is no different from the risk of password sharing. And there will
be no "recover my lost password" option.

I would tackle this in a slightly different way: usernames/gamertags. Depending on how frequently you find gamertags and usernames sharing the same IP. So if you only accept a maximum of, say, 5 gamertags per IP, and you also throttle the frequency of updates per gamertag, you have a fairly spam-resistant system.

I would recommend a mix of code obfuscation and using web sockets to request the score, rather than post the score. Something like socket.io (https://socket.io/) where the server sends a request with a code in it and your game responds with the score and that code changed in some way.
Obviously a hacker could look through your code for how your game responds to requests and rewrite it, which is where the obfuscation is important, but it does at least hide the obvious network traffic and prevents them posting scores whenever they feel like it.

I would suggest using reCAPTCHA V2.
Admittedly, v3 provides better protection, but it is hard to implement, so go with v2.
Come on, it is just a few lines of code.
How it should work (according to me):
You are at the main page willing to play the game
You solve the reCAPTCHA
Then the app sends a one-time token with a script tag which establishes a websocket request with your server (using socket.io) with the one-time token and then it is destroyed immediately (from the server as well as the client) after establishment of a connection
Your server validates the token and accepts the request of websocket and then it will send the HTML content
Just create a div and set the value using obj.innerHTML
You can use styles in body (I guess)
And the most important point is obfuscating your code.
Security
Websockets are harder to reverse engineer in a test environment
Even if they create a web socket, it won't respond, because they don't know the one-time token
It prevents script blocking (as the script loads everything on the page)
It provides real-time communication
The only way out is to somehow get your hands on Google's reCAPTCHA token which is impossible, because it means going against Google
You can’t reuse any token (however immediate it be), because it was destroyed from both the sides
One more last tip: set a timeout for the one-time token to about 15 seconds
How will it help? It will prevent someone (extremely malicious) from pausing the Chrome debugger and get the token and put it in their stuff as 15 seconds is ok for slow networks also, but not a human

Firebase Authentication via "Magic-Link"

I want to send notification mails to users of my community platform coded with react and firebase. I was wondering is there is a way to authenticate the user via a magic link which is contained in his or her personal notification mails.
E.g. You've got 5 new comments on your post click HERE to read them
Clicking "here" should redirect the user to the page and automatically authenticate him without having to enter their individual passwords or usernames.

Due to the cancelation of the privacy shield framework and Schrems II, you might no longer be able to store customer details in the GCP.
A solution for it can be pseudonymization.
I am running an open-source project that can help you with that.
Check out the following article for more information:
https://databunker.org/use-case/schrems-ii-compliance/

I think your question can be divided into 2 different questions.
1. How do I send notification mails to my community platform?
I have never used notification mails in Firebase, I usually use google's SMTP server directly and send a request to the server to send an email on behalf of me. So, in your case, I would search up the library on your tech stack (for me it was Go's gomail), then you can use Google's SMTP server (host:smtp.gmail.com, port:587).
2. How do I make a link that automatically authenticate your users whenever they click the link?
I feel like this is the harder question. I think you need to consider a few things:
Your links need to be short lived. Why? to prevent unauthorized brute force login attempt (this also depends on the structure of your link). However, best practices suggests that links should be short lived (less than 24 hours).
Given that links are short lived, this depends on whether the user checked their email frequently for notification from you! There's a pretty good chance that they will miss that email in the 24 hour window.
I'm pretty much against sending time sensitive notification like that through emails. But if it is something that you still want to do, it is pretty easy to create the link, the simplest way I did it was:
Generate a random uuid for a link.
Associate it to the user who will login with the link. You should save this association in a persistent data storage.
Create an endpoint for magic link, for example /magic/:link_id that takes link_id then checks what user should be logged in.
Voila! You got yourself a magic link!
Finally, you can send the link through email like the first part of the problem!

How avoid multiple POST as complete request information comes in network tab of browsers

We have a form, if a person fills that form, we are hitting POST API.
When user submits the form, complete INFO of API (request url, headers) is visible, I was thinking if someone starts hitting this POST API through server side code, our tables would be filled with junk values?
How can we avoid this bulk data entering the system through API being directly hit by C# client, POSTMAN or any other server side clients?

That's a really open ended question, but I don't think there's a way to do exactly what you ask (detect with certainty if a request originates from a browser or not), nor should you if it were possible - users may have a legitimate reason to use something other than a browser, or someone might want to integrate your API into a larger process, etc.
You should handle this via authentication - require the user to be authenticated with your API, then if they misbehave you can disable their user accounts (either manually or automatically with some sort of abuse detection).

There are times when authentication is not possible or desirable for a web form. At those times a I have used a key generated on the server that I embed in the form.
The form sends back the key along with the rest of the data and you can then make any decisions you need to make based on the key. Like limiting the rate of submissions, allowing only one submission and then expiring the key, etc...

Use a captcha: a challenge is presented to the user to prove being a human, usually an image that contains some garbled text is shown to the person filling the form and (s)he is required to transcribe the text content of the image in the form. If the captcha solution is wrong or missing that POST request to the API will be discarded.
Many types of challenges are commonly used such as audio/visual/logic/mini-puzzles. You can also customize your challenge in a way that fits best with your form. For example Google provides ReCaptcha, a captcha service with js and server api.
This helps you prevent or mitigate bots, as the captcha is a challenge that is very hard for computers but easy for humans.
Using one-time captcha tokens also prevents the replay attacks you are worried about. Also, checking that the ip that received the challenge and the ip resolving it are same helps mitigate other tricks.
This still leaves room for a determined human to spam your form. So you should also keep track of the number of submissions and throttle them by ip.

What are the best ways to prevent fake registrations?

I would like to know more about the solutions to restrict registering for a website for humans only.
Captcha may seem a proper solution but as it turns out it's not as good as it sounds.
And it's not a problem if a solution is not an option for blind, deaf people etc..

My newest web app uses a process that makes this really easy for the user and secure for me.
User goes to login page, enters their email address and clicks an "I am signing up" checkbox.
The user clicks "register", their email address gets inserted to a temporary SQL table (called signups) and an email with a verification link is sent to the email address.
The user clicks the verification link which takes them to a "create password" page.
After the user creates his password, the email address and password are inserted into the users table–and the entry in the signups table is deleted.
This makes it easy and reliable.
The signups table also includes a random SHA1 hash (for the verification link), a timestamp set for 12 hours after the sign up, and IP Address.
All entries in the signups table that have an expired timestamp get removed at certain parts of the day.
Update
Since writing this answer, I have found that keeping a signup form secure from robots is a never-ending battle.
As time goes on, new technologies and methods are being developed to circumvent security measures. The best thing any dev team can do is periodically be checking the quality of signups, and thinking of ways to make their signup form both more secure and intuitive to use.
Yeah, it is a good bit of work and resources that go into it, but having confidence in your signup form and quality of signups is well worth the expense.

Depending on how targeted your site is, using a honeypot can be quite effective.
In short, you have a field on your form with a common name -- let's say email. Your actual email field has some other random name like larp.
Hide the email field using CSS, and include a text label instructing users to leave that field blank, should they happen to see it.
If any registrations come in with the email field filled in, send a success message back then drop it.

Verifying the e-mail address and allowing only users who have verified their e-mails is the easiest and quickest solution.

You could have users listen to an audio file, and enter the word into a text box. That's what I wanted to do for phpBB3, but alas, they do not allow HTML markup for their verification questions. Alternatively, you can do defense in depth, and have a CAPTCHA be only part of the process.

Instead of captcha (typing in words displayed in an image) I've seen websites that require you to answer a question, usually basic math. I haven't implemented these, but I've seen them several places so it must be something that's a plugin for various CMSes and soforth.
Bottom line is, any system will eventually be circumvented. You can minimize bot-registrations, but I can tell you from experience that there are people out there who will pay other humans (who can pass just about any test) money to sign up for web sites - the pay is generally really poor, but there are always people who will find that acceptable.
So along with whatever solution you use, you'll want to periodically re-evaluate that solution, AND you'll want to have a human being review new registrations (maybe once a day?) to weed out the few 'bots that do get through.

preventing iframe security issue

As described here http://blogs.gartner.com/avivah-litan/2010/04/22/watch-out-transaction-signing-is-being-socially-engineered/
The way the transaction signing that was compromised works is that
once a payment request has been initiated by an online banking user,
the bank asks the user to sign his/her transaction on a dedicated
out-of-band EMV CAP reader with the user’s EMV Chip card inserted. The
user is asked to enter certain codes and values on the reader
confirming the currency amount to be transferred and the account it is
to be transferred to.
Well the crooks knew when the transaction signing was being initiated
and simply put an iframe up on top of the user’s browser that changed
the values to be entered so that the users ended up typing in the
fraudster’s bank account as the destination account rather than the
one they intended.
Something to think about when it comes to transaction signatures –
demonstrating the need to keep the entire process off the PC (in this
case) and on another channel entirely.
Can you explain how it is possible to "put an iframe up on top of the user’s browser" and how to prevent that kind of fraud technically ?

It's not clear from the quote, but it sounds like they are talking about consumer endpoint compromise. The user has picked up a banker trojan so their PC has become an untrusted device which can display misleading information. In this case the trojan operators changed the displayed destination account number so that funds would flow to a different party than the one the user thought they were crediting.
The point is that a complete user interface for making secure decisions must reside on the trusted device. Typing information from the PC to the secure device gives the user a chance to check the information is correct, as does a device with an on-screen display of information being authorised.
But there is a missing piece in that the consumer doesn't usually know that the account number they are typing is genuinely that of the party they mean to credit. Unless they've done many transactions with that party before such that they can remember the account number and spot it when it's wrong (and even then that wouldn't necessarily raise flags).
To correct this, the account ID would have to be something recognisable, something like a domain name whose issuance is controlled, rather than an arbitrary number. That would cause problems for any standalone device where you had to type the information, though, as it would then need a full-size keyboard. You could do it with a display-only device, something like Germany's TAN generators that read information off the screen, or you could do it with a very long account number that decoded to something readable, signed to prevent unauthorised changes.
Once the entire decision takes place on the secure device, including amount to transfer and the user-verifiable destination, the problem of the untrusted intermediary (your PC as a man in the middle) is solved and transaction is safe.
Although that information doesn't include the purpose of the transaction, so you could still imagine an attack where a reseller changed the actual items you were purchasing at a particular store, without changing the cost!

This is a sample of an XSS (Cross-site script) attack.
Say for example, Friendster, which had a lot of XSS "holes" before, particularly in the profile page. I can inject a script in my profile page, and make my page look like a login page.
Once another user looks at my profile, it looks like a Friendster login page! Then the user types his username and password not knowing that it was a fraud page. The entered values would be routed to the my server for storage, while the user gets redirected to the actual page.
In reality, the user was never logged out. The user was made to believe that it was a legit page and was made to reveal the account details.
To prevent this, you should not allow arbitrary HTML as well as scripts to get through your site. There are a lot of entry points for attack, usually input boxes that don't have thorough validation. Event SSL'ed sites are not safe if an injected script exists on the page.