We Keep Coding

Prevent malicious users from abusing and spamming unauthenticated open APIs

Here's a security problem I've encountered a couple of times when building small web-based projects interacting with a REST API service. For example, let's say you're building a casual JavaScript-based game where you want a leaderboard of highscores, so you need to post the scores of users to a database.
The easiest solution would be to build a simple web service, e.g. using PHP, Node.js or Python, that accepts GET request and saves the results to a database. Let's imagine the API looks something like this:
GET https://www.example.com/api/highscore?name=SuperGoat31&score=500
Creating such an API for posting highscores has some obvious drawbacks. A malicious user could write a three-line piece of PHP code to spam the database full of false results, for example:
for ($i = 0; $i < 100; i++) {
file_get_contents("https://www.example.com/api/highscore?name=SuperGoat31&score=5000000");
}
So, I'm looking for a way to prevent that. This mostly relates to small hobby or hackathon projects that just need some kind of protection that will prevent the most obvious of attacks, not large enterprise applications that need strict security. A couple of things I could think of:
1. Some form of authentication
An obvious way to solve this would be to have user accounts and only allow requests from logged-in users. This unfortunately has the drawback of putting up a large barrier for users, who need to get an account first. It would also require building a whole authentication workflow with password recovery and properly encrypting passwords and the like.
2. One-time token based protection
Generate a token on the server side and serve that to the user on first load, then only allow requests that serve that specific token. Simple enough, but also very easy to circumvent by finding the requests in a browser web inspector and using that for the three-line PHP script.
3. Log IP address's and ban when malicious use happens
This could work, but I feel it's not very privacy friendly. Also, logging IP addresses would require GDPR consent from users in Europe. Also doesn't prevent the actual spamming itself so you might to first clean up the mess before you start banning IP addresses.
4. Use an external service
There are services that provide solutions to this problem. For example, in the past I've used Google's reCAPTCHA to prevent malicious use. But that also means integrating an external service, making sure you keep it up to date, concerns about the privacy aspects (esp. regarding a service like reCAPTCHA), etc. It feels a bit much for a weekend project.
5. Throttle requests
I feel this is probably the easiest solution that actually works for a bit. This does require some form of IP address logging (which might give the problems stated in 3), but at least you can delete those IP addresses pretty quickly afterwards.
But I'm sure there are other methods I've missed, so I would be curious to see other ways of tackling this problem.

Taking into account all mentioned limitations, I would recommend using a combination of methods:
Simple session authentication based on one-time token
Script obfuscation
Request encryption with integrity control
Example:
let req_obj = {
user: 'SuperGoat31',
score: 123456,
sessionId: '4d2NhIgMWDuzarfAY0qT3g8U2ax4HCo7',
};
req_obj.hash = someCustomHashFunc(JSON.stringify(req_obj));
// now, req_obj.hash = "y0UXBY0rYkxMrJJPdoSgypd"
let req_string = "https://www.example.com/api/cmd?name=" +
req_obj.user +
"&data=" +
Buffer.from(JSON.stringify(req_obj)).toString('base64');
// now, your requests will look like that:
"https://www.example.com/api/cmd?name=SuperGoat31&data=eyJ1c2VyIjoiU3VwZXJHb2F0MzEiLCJzY29yZSI6MTIzNDU2LCJzZXNzaW9uSWQiOiI0ZDJOaElnTVdEdXphcmZBWTBxVDNnOFUyYXg0SENvNyIsImhhc2giOiJ5MFVYQlkwcllreE1ySkpQZG9TZ3lwZCJ9"
For casual players, this allows start playing very quickly, as no explicit registration is required. Upon generation, token might be saved as cookie for repetitive use, but this is not necessary, single-time use would also suffice. No personal info gathered.
However, if short-term storage of some client information is an option, the token might be not just some random bytes, but an encrypted string, containing some parameters, such as random salt + IP address + nickname + agent id + etc. In this case you may start silently ignore certain requests from fraudulent clients upon detection.
Obviously, this would be very easy to crack for a professional, but this is not our goal. When such simple methods are mixed with several kilobytes of logic of the game and obfuscated, figuring out how to deal with it would require significant amount of knowledge and time, which might serve as a sufficient barrier.
As it is all about balance between convenience and protection, you may implement some additional scoring logic to detect cheating attempts, like final score cannot end with '0', or cannot be even, etc. This would allow you to count cheating attempts (in addition to counting forged requests) and then estimate efficiency of implemented combination of methods.

Your list of solutions are mostly mitigations, and they are good ideas if they are your only tools. The list seems pretty exhaustive.
2 major ways to actually solve this problem are:
Remove the incentive of cheating. There's no point submitting a fake score if you are the only person who can see the score. Think about the purpose of why you even want a global high-score list. Maybe there's another way you can reach your objective that makes it uninteresting (or undesirable) to cheat.
Have the server completely manage (or duplicate) the game state. You can't cheat if the server calculates the score. For example, if you're modelling a chess game the server can compute every valid move, preventing clients from submitting moves that wouldn't be possible.
It's possible that for your specific case neither are possible, but if you can't adopt either of these strategies you are stuck to imperfect detection mechanisms.

I suspect that a perfect solution will be elusive because two of
your wishes are, perhaps, contradictory:
"You need to post the scores of users to a database" but... "prevent
the most obvious of attacks" without "Some form of authentication."
The most obvious of attacks are those from users without some form
of authentication.
You wish this system to work without placing an undue burden on
your users. You wish to avoid the usual login and password
authentication which can be cumbersome for users.
I think there is a way to accomplish what you want by creating a
very simple form of authentication by the use of a one-time token
based protection. And I would also incorporate IP tracking against
abuse. In other words, let's combine your options 1 and 2 and 3 in
the following way.
You already have implied that you will maintain a database, and that
within the database, user names will be unique (otherwise you couldn't
record unique high scores). Let people sign up freely by submitting
their requested user name, which you'll accept if not already used
by someone prior. Track the sign-up requests by IP address to detect
and prevent abuse: too many sign-ups from one IP address within a given timeframe. So far, the burden is all at the server end, not on the user.
When you process a valid sign-up (i.e. new user name) into the
database, you will also generate, record into the database, and return to the user a shared secret (a token) that will be used by the
Time-based One-time Password (TOTP) algorithm.
Don't reinvent this.
See:
Time-based One-Time Password
FreeOTP
OneTimePass
When you return a token to the user, it will be in the form of a "QR Code"
QR code
which the user will scan and store with his "Google Authenticator" or
equivalent TOTP application.
When the user returns to your web site to update his high score, he
will authenticate himself using his Google Authenticator" or
equivalent TOTP application. These are often used for "second factor"
authentication, 2FA (Multi-factor authentication), but because
of your need for less strict security, you'll be using the TOTP
authentication as the primary and only form of authentication.
So we have combined a form of authentication which doesn't place a
very high burden on the user (apps already widely available and in
use), with one-time token based protection (provided by the TOTP
app) and a little bit of IP address-based abuse protection for the initial sign-ups.
On of the weaknesses of my proposal is that a user may share his
TOTP token with another person, who may then impersonate him. But this
is no different from the risk of password sharing. And there will
be no "recover my lost password" option.

I would tackle this in a slightly different way: usernames/gamertags. Depending on how frequently you find gamertags and usernames sharing the same IP. So if you only accept a maximum of, say, 5 gamertags per IP, and you also throttle the frequency of updates per gamertag, you have a fairly spam-resistant system.

I would recommend a mix of code obfuscation and using web sockets to request the score, rather than post the score. Something like socket.io (https://socket.io/) where the server sends a request with a code in it and your game responds with the score and that code changed in some way.
Obviously a hacker could look through your code for how your game responds to requests and rewrite it, which is where the obfuscation is important, but it does at least hide the obvious network traffic and prevents them posting scores whenever they feel like it.

I would suggest using reCAPTCHA V2.
Admittedly, v3 provides better protection, but it is hard to implement, so go with v2.
Come on, it is just a few lines of code.
How it should work (according to me):
You are at the main page willing to play the game
You solve the reCAPTCHA
Then the app sends a one-time token with a script tag which establishes a websocket request with your server (using socket.io) with the one-time token and then it is destroyed immediately (from the server as well as the client) after establishment of a connection
Your server validates the token and accepts the request of websocket and then it will send the HTML content
Just create a div and set the value using obj.innerHTML
You can use styles in body (I guess)
And the most important point is obfuscating your code.
Security
Websockets are harder to reverse engineer in a test environment
Even if they create a web socket, it won't respond, because they don't know the one-time token
It prevents script blocking (as the script loads everything on the page)
It provides real-time communication
The only way out is to somehow get your hands on Google's reCAPTCHA token which is impossible, because it means going against Google
You can’t reuse any token (however immediate it be), because it was destroyed from both the sides
One more last tip: set a timeout for the one-time token to about 15 seconds
How will it help? It will prevent someone (extremely malicious) from pausing the Chrome debugger and get the token and put it in their stuff as 15 seconds is ok for slow networks also, but not a human

3D S Payment callback issue

I'm working on a payment system with React-Native. I want to do research on how to apply 3D payment methods. I want to listen to an event happening on my server on the client side. actually i only need this for the following reason. I feel there is something I got wrong here.
3D Secure Steps
I send the product to be purchased to a service named iyzico with
credit card information.
If the information entered is correct, it gives me an answer as below.
There is a special field here and I need to decrypto it and show it to the user.
In fact, this crypoto information contains an html page.
By decoding this code, I show the user an html page.
The password sent to the user's phone via SMS is entered on that
screen and presses the confirmation button.
The part I don't understand starts here. The person is in a true asynchronous. I want to callback. Because he can enter or cancel the password sent by SMS whenever he wants.
I'm not wondering how the process turned out. How will we inform the client application only in this case?
Should we use push notifications or other push services for this?
The client has to get information about this process. According to the information he receives, I would like to suggest that your payment is successful or your balance is insufficient.
I know that I should avoid making circular calls.
In short, how should I listen to the client for an event that will take place on my server? Which would be the best method?
I am working on React-native and I do not want to include push services in the application just for this. It is costly for me to this. I believe it is a more beautiful solution.
What do you think about this subject?
Thanks.

You either need push notifications or long polling. https://ably.com/blog/websockets-vs-long-polling

How to send message to Slack every 'x' number of hours?

I have a configuration page where the user can select at what time to receive certain information from my API, for example they could select to get the information every day at 5PM or every Friday at 5PM. After this is set, the user should receive a Slack message at the time they defined, for example, every day at 5PM they would receive a message...
I save the user's timezone so that I can send the message to them in Slack at the correct time for them.
That being said, how can I schedule this message to send out from my node.js app? I would have a few users in the system, all who would have likely chosen different times so I would need a timer per user?
The only thing I can think of is scanning all users in the system, getting their selected time and then sending the message to them...but this doesn't seem scalable.
I'm not looking for a complete solution for this, just some pointers for how to design this sort of functionality.
I also looked into Slack scheduled messages but this isn't exactly what I'm looking for. Note: I can already send messages to Slack, I'm more interested in how to build the timer mechanism.
Thanks in advance!
EDIT:
Did a bit more research and it looks like node-schedule could be an option to schedule jobs: https://github.com/node-schedule/node-schedule#readme
With using this package, is the approach that I scan all users in my database at let's say midnight everyday and schedule jobs based on their settings...then those jobs execute at their scheduled time and the user receives the message in Slack. Is this a good approach?

For the core logic I would suggest something like this:
Store the timing of delivery (e.g. 5 AM ever Friday) for every user in your database
Then have a worker process that is running on a regular basis, e.g.
every 5 minutes
When it runs it checks if there are any due messages to be delivered
If yes it sends the message with your API information to the user and store the last time of sending for the user
This approach is resilient to downtime. It will just resume sending due messages once the worker process is running again after a downtime.
It is also scaleable: If needed you can run multiple worker processes (make sure to design your workers to support concurrent processing, e.g. with transactions)
Some additional things to consider:
Would limit the number of messages sent per run to avoid timeouts and having too many workers running in parallel
You need some error handling if sending message to Slack fails
To avoid timezone complexities I would suggest to convert all timings to UTC for processing in your app

Is this a regular task? In other words, it's executed every day at the same time for user X? If so, node-schedule seems fine, and it can run the same job regularly, you just have to set it up properly via a cron-like string (see the instructions in the README). If a user changes their setting, you then modify the previous job. The downside with node-schedule is that you need to set it up everytime your application is loaded, which can take a while and consume lots of resources if you have too many users.
Alternatively, if the number of users is big or you prefer to keep your application stateless, you can set a number of slots for sending these messages (and run that in a separate process from your main application). Let's say, a slot every 30 min. Then you set timers for those time slots (using node-schedule if you like, it will be just 48 timers), fetch the list of users for that time slot from the database, and send the messages.
Overall, NodeJS/JavaScript is pretty efficient with this sort of timer-based scheduling. If you want an in-depth dive into the reasons, see this: https://nodejs.org/de/docs/guides/event-loop-timers-and-nexttick/
You also need to consider what happens in case your application suffers from downtime. Should users be guaranteed to receive those messages, even if they are late? But that's another story :-)

User Pool allows two users with same email despite configuration

Background
I'm using aws-amplify to interact with Cognito. So when a user registers with my app, I call Auth.signUp(). I'm passing only username (email) and password to this function.
My user pool is configured to allow sign in by email only:
The Bug?
In my front end code, I accidentally registered an event listener twice, so Auth.signUp() was being called twice (concurrently, or at least in rapid succession) with the same parameters.
This resulted in two users being created in my User Pool, with the same email. My understanding of my user pool configuration suggests that this shouldn't be possible.
Race Condition?
My first thought was that since I'm sending two requests so close together, this may be some sort of unavoidable race condition. If I introduce an artificial pause between the calls (a breakpoint, or a setTimeout, say), everything works as expected.
However, even with the requests very tightly spaced, the second request does return the error response I'd expect:
{ code: 'InvalidParameterException',
name: 'InvalidParameterException',
message: 'Alias entry already exists for a different username'
}
Sadly, this response is misleading, because I do get a second (duplicate) user created in my pool with this request.
MCVE
This is easy to reproduce by exercising Auth.signUp twice concurrently, either in a node script or a browser. This repository contains examples of both.
The Question(s)
Is this a legitimate bug with Cognito?
Is a preSignUp Lambda trigger my only way to defend against this? If so, what would the broad strokes of that implementation look like?

I sent this to AWS support. They're aware of the issue but have no ETA.
Thanks for contacting AWS Premium Support. I understand that you
would like to know whether Cognito team is aware of the issue posted
here[1].
I checked with Cognito team on our end and YES, they are aware of this
issue/bug. Good news is, we already have trouble ticket open with
Cognito Team to fix the issue. However, I won't be able to provide an
ETA on when this fix will go live as I don't have any visibility into
their development/release plans. But, I would like to thank you for
your valued contribution in bringing this issue to our attention, I do
appreciate it.

I talked to AWS, still no fix and no time estimation.

Cognito limits usernames to one user only. However, yes multiple user can share an email.

Google analytics - difference between analytics data and the data from the core platform

We use Google analytics with Google Tag manager for our ecommerce platform based to track the conversion, etc.,
In the thank you page, its used to track the order value and the revenues.
Consistently we see a difference of about 15 to 20 % difference between GA data and the data from the core platform.
Tried to find a pattern among the missed orders but couldn´t ascertain one easily. The GA recorded orders include devices like Desktop, tablet and mobile and we see different browsers too.
Need inputs to analyze this better.
Note: Thank you page is loaded by a redirect from the payment gateway system

15-20% difference is not good at all. Google says you should expect better than 95% accuracy, and to keep at it if you're not getting those numbers.
Note: The more "techy" crowd that your website has, the more folks you'll run into "Do not track" or with Ad-Blocking tech in their browser. Normally, you'd want to try to baseline that difference using device category filters to see if the gap is bigger for desktop (most phones/tablets don't use ad-blocking).
First, question. If the user lands on the thankyou page, and hits the refresh button, does it send another "conversion" to GA? IF so, you want to make sure that you build in logic that prevents duplicate conversions to be sent if the user was not making duplicate purchases. A browser refresh is not a purchase, don't record it as such.
Second, if the page takes forever to load, or you have users that have bad internet, then that could increase the difference. They might be closing the browser or exiting site before GA client has a chance to send the final conversion to the server. So how is the performance of your thankyou page?
Are you sure you're looking at the correct business data? I've been told GA numbers are off by the business before and it turned out they messed up their own query in the transaction system (and they had been doing so for years!). It is a long shot, but if you feel super confident about your GA measurement setup, then run it by the folks giving you the transaction numbers.
Finally, if you can't get the difference down then move to the Measurement Protocol server-side implementation of GA. You simply need to record the IP address of the user and their GA client id, and then construct an HTTPS GET request using the Measurement Protocol fields for a valid hit. Server side measurement is the most accurate way to do this, but requires code updates in the ecommerce platform itself.