Here's a security problem I've encountered a couple of times when building small web-based projects interacting with a REST API service. For example, let's say you're building a casual JavaScript-based game where you want a leaderboard of highscores, so you need to post the scores of users to a database.
The easiest solution would be to build a simple web service, e.g. using PHP, Node.js or Python, that accepts GET request and saves the results to a database. Let's imagine the API looks something like this:
GET https://www.example.com/api/highscore?name=SuperGoat31&score=500
Creating such an API for posting highscores has some obvious drawbacks. A malicious user could write a three-line piece of PHP code to spam the database full of false results, for example:
for ($i = 0; $i < 100; i++) {
file_get_contents("https://www.example.com/api/highscore?name=SuperGoat31&score=5000000");
}
So, I'm looking for a way to prevent that. This mostly relates to small hobby or hackathon projects that just need some kind of protection that will prevent the most obvious of attacks, not large enterprise applications that need strict security. A couple of things I could think of:
1. Some form of authentication
An obvious way to solve this would be to have user accounts and only allow requests from logged-in users. This unfortunately has the drawback of putting up a large barrier for users, who need to get an account first. It would also require building a whole authentication workflow with password recovery and properly encrypting passwords and the like.
2. One-time token based protection
Generate a token on the server side and serve that to the user on first load, then only allow requests that serve that specific token. Simple enough, but also very easy to circumvent by finding the requests in a browser web inspector and using that for the three-line PHP script.
3. Log IP address's and ban when malicious use happens
This could work, but I feel it's not very privacy friendly. Also, logging IP addresses would require GDPR consent from users in Europe. Also doesn't prevent the actual spamming itself so you might to first clean up the mess before you start banning IP addresses.
4. Use an external service
There are services that provide solutions to this problem. For example, in the past I've used Google's reCAPTCHA to prevent malicious use. But that also means integrating an external service, making sure you keep it up to date, concerns about the privacy aspects (esp. regarding a service like reCAPTCHA), etc. It feels a bit much for a weekend project.
5. Throttle requests
I feel this is probably the easiest solution that actually works for a bit. This does require some form of IP address logging (which might give the problems stated in 3), but at least you can delete those IP addresses pretty quickly afterwards.
But I'm sure there are other methods I've missed, so I would be curious to see other ways of tackling this problem.
Taking into account all mentioned limitations, I would recommend using a combination of methods:
Simple session authentication based on one-time token
Script obfuscation
Request encryption with integrity control
Example:
let req_obj = {
user: 'SuperGoat31',
score: 123456,
sessionId: '4d2NhIgMWDuzarfAY0qT3g8U2ax4HCo7',
};
req_obj.hash = someCustomHashFunc(JSON.stringify(req_obj));
// now, req_obj.hash = "y0UXBY0rYkxMrJJPdoSgypd"
let req_string = "https://www.example.com/api/cmd?name=" +
req_obj.user +
"&data=" +
Buffer.from(JSON.stringify(req_obj)).toString('base64');
// now, your requests will look like that:
"https://www.example.com/api/cmd?name=SuperGoat31&data=eyJ1c2VyIjoiU3VwZXJHb2F0MzEiLCJzY29yZSI6MTIzNDU2LCJzZXNzaW9uSWQiOiI0ZDJOaElnTVdEdXphcmZBWTBxVDNnOFUyYXg0SENvNyIsImhhc2giOiJ5MFVYQlkwcllreE1ySkpQZG9TZ3lwZCJ9"
For casual players, this allows start playing very quickly, as no explicit registration is required. Upon generation, token might be saved as cookie for repetitive use, but this is not necessary, single-time use would also suffice. No personal info gathered.
However, if short-term storage of some client information is an option, the token might be not just some random bytes, but an encrypted string, containing some parameters, such as random salt + IP address + nickname + agent id + etc. In this case you may start silently ignore certain requests from fraudulent clients upon detection.
Obviously, this would be very easy to crack for a professional, but this is not our goal. When such simple methods are mixed with several kilobytes of logic of the game and obfuscated, figuring out how to deal with it would require significant amount of knowledge and time, which might serve as a sufficient barrier.
As it is all about balance between convenience and protection, you may implement some additional scoring logic to detect cheating attempts, like final score cannot end with '0', or cannot be even, etc. This would allow you to count cheating attempts (in addition to counting forged requests) and then estimate efficiency of implemented combination of methods.
Your list of solutions are mostly mitigations, and they are good ideas if they are your only tools. The list seems pretty exhaustive.
2 major ways to actually solve this problem are:
Remove the incentive of cheating. There's no point submitting a fake score if you are the only person who can see the score. Think about the purpose of why you even want a global high-score list. Maybe there's another way you can reach your objective that makes it uninteresting (or undesirable) to cheat.
Have the server completely manage (or duplicate) the game state. You can't cheat if the server calculates the score. For example, if you're modelling a chess game the server can compute every valid move, preventing clients from submitting moves that wouldn't be possible.
It's possible that for your specific case neither are possible, but if you can't adopt either of these strategies you are stuck to imperfect detection mechanisms.
I suspect that a perfect solution will be elusive because two of
your wishes are, perhaps, contradictory:
"You need to post the scores of users to a database" but... "prevent
the most obvious of attacks" without "Some form of authentication."
The most obvious of attacks are those from users without some form
of authentication.
You wish this system to work without placing an undue burden on
your users. You wish to avoid the usual login and password
authentication which can be cumbersome for users.
I think there is a way to accomplish what you want by creating a
very simple form of authentication by the use of a one-time token
based protection. And I would also incorporate IP tracking against
abuse. In other words, let's combine your options 1 and 2 and 3 in
the following way.
You already have implied that you will maintain a database, and that
within the database, user names will be unique (otherwise you couldn't
record unique high scores). Let people sign up freely by submitting
their requested user name, which you'll accept if not already used
by someone prior. Track the sign-up requests by IP address to detect
and prevent abuse: too many sign-ups from one IP address within a given timeframe. So far, the burden is all at the server end, not on the user.
When you process a valid sign-up (i.e. new user name) into the
database, you will also generate, record into the database, and return to the user a shared secret (a token) that will be used by the
Time-based One-time Password (TOTP) algorithm.
Don't reinvent this.
See:
Time-based One-Time Password
FreeOTP
OneTimePass
When you return a token to the user, it will be in the form of a "QR Code"
QR code
which the user will scan and store with his "Google Authenticator" or
equivalent TOTP application.
When the user returns to your web site to update his high score, he
will authenticate himself using his Google Authenticator" or
equivalent TOTP application. These are often used for "second factor"
authentication, 2FA (Multi-factor authentication), but because
of your need for less strict security, you'll be using the TOTP
authentication as the primary and only form of authentication.
So we have combined a form of authentication which doesn't place a
very high burden on the user (apps already widely available and in
use), with one-time token based protection (provided by the TOTP
app) and a little bit of IP address-based abuse protection for the initial sign-ups.
On of the weaknesses of my proposal is that a user may share his
TOTP token with another person, who may then impersonate him. But this
is no different from the risk of password sharing. And there will
be no "recover my lost password" option.
I would tackle this in a slightly different way: usernames/gamertags. Depending on how frequently you find gamertags and usernames sharing the same IP. So if you only accept a maximum of, say, 5 gamertags per IP, and you also throttle the frequency of updates per gamertag, you have a fairly spam-resistant system.
I would recommend a mix of code obfuscation and using web sockets to request the score, rather than post the score. Something like socket.io (https://socket.io/) where the server sends a request with a code in it and your game responds with the score and that code changed in some way.
Obviously a hacker could look through your code for how your game responds to requests and rewrite it, which is where the obfuscation is important, but it does at least hide the obvious network traffic and prevents them posting scores whenever they feel like it.
I would suggest using reCAPTCHA V2.
Admittedly, v3 provides better protection, but it is hard to implement, so go with v2.
Come on, it is just a few lines of code.
How it should work (according to me):
You are at the main page willing to play the game
You solve the reCAPTCHA
Then the app sends a one-time token with a script tag which establishes a websocket request with your server (using socket.io) with the one-time token and then it is destroyed immediately (from the server as well as the client) after establishment of a connection
Your server validates the token and accepts the request of websocket and then it will send the HTML content
Just create a div and set the value using obj.innerHTML
You can use styles in body (I guess)
And the most important point is obfuscating your code.
Security
Websockets are harder to reverse engineer in a test environment
Even if they create a web socket, it won't respond, because they don't know the one-time token
It prevents script blocking (as the script loads everything on the page)
It provides real-time communication
The only way out is to somehow get your hands on Google's reCAPTCHA token which is impossible, because it means going against Google
You can’t reuse any token (however immediate it be), because it was destroyed from both the sides
One more last tip: set a timeout for the one-time token to about 15 seconds
How will it help? It will prevent someone (extremely malicious) from pausing the Chrome debugger and get the token and put it in their stuff as 15 seconds is ok for slow networks also, but not a human
I'm working on a payment system with React-Native. I want to do research on how to apply 3D payment methods. I want to listen to an event happening on my server on the client side. actually i only need this for the following reason. I feel there is something I got wrong here.
3D Secure Steps
I send the product to be purchased to a service named iyzico with
credit card information.
If the information entered is correct, it gives me an answer as below.
There is a special field here and I need to decrypto it and show it to the user.
In fact, this crypoto information contains an html page.
By decoding this code, I show the user an html page.
The password sent to the user's phone via SMS is entered on that
screen and presses the confirmation button.
The part I don't understand starts here. The person is in a true asynchronous. I want to callback. Because he can enter or cancel the password sent by SMS whenever he wants.
I'm not wondering how the process turned out. How will we inform the client application only in this case?
Should we use push notifications or other push services for this?
The client has to get information about this process. According to the information he receives, I would like to suggest that your payment is successful or your balance is insufficient.
I know that I should avoid making circular calls.
In short, how should I listen to the client for an event that will take place on my server? Which would be the best method?
I am working on React-native and I do not want to include push services in the application just for this. It is costly for me to this. I believe it is a more beautiful solution.
What do you think about this subject?
Thanks.
You either need push notifications or long polling. https://ably.com/blog/websockets-vs-long-polling
So I've been messing around with node.js and twilio these past couple of days. I managed to get it running and create a nice little system of keyword checks and responses. The problem is I need a way to prevent a conversation from going beyond X amount of exchanged messages. Anything that can get the user to stop messaging or get twilio to stop accepting them would be great.
I've looked into things like blocking their number, or even trying to activate the STOP keyword for them but nothing came up. The best I thought of was creating a counter of inbound and outbound messages and sending a final messaging stating "You've reached the conversation limit. You'll be charged X amount per every new message sent and received" to scare them off.I'm not really sure if that option is ethical or even legal.
I don't want to wake up one day and find that I owe $5000 in SMS fees because my numbers were spammed even after users have already gone through every possible dialogue option. Any help is appreciated.
EDIT : I think I've found my answer so I'll post it here in case anyone else has the same thought. https://support.twilio.com/hc/en-us/articles/223181648-Is-there-a-way-to-block-incoming-SMS-on-my-Twilio-phone-number-
Twilio billing is prepaid (for me), so i don't think you will "owe $5000". They could burn thru your credit and if you are set up for auto top up, that could be an issue.
That being said there are different ways to keep on top of spending without coding. I would recommend taking a look at the following:
Account Triggers - Get notifications
Projects - Isolate your numbers/apps that are in production
Some Coding Ideas
I am guessing you are using a twilio number to bridge two users that gives anonymity. What you proposed should be possible.
Use a function to query your logs and rate limit with your own logic
Docs: Usage Records
Use a database, record transactions, and query the data base with your own logic
Note: if the incoming message is generated by an incoming sms/mms, an incoming fee will be charged regardless of if the message is relayed or not.
Additionally, there are rate limiting npm modules available (I have not explored them). ex// npm limiter
As mentioned in the comments, this specific question is off-topic. However, I'll state few things.
As of now, you can't control incoming messages. Twilio does that for you. For example, twilio charge you only for numbers validated as spam-free. You can get more information from their support agents.
From your side, you can try few other things.
Control the outgoing messages (For instance, detect spam, and do not send automatic replies if any)
Stick into a basic plan
For controlling the incoming load to your application, there is a library called queuep(npm install queuep). You can easily write the spam detection logic and avoid a load of messages. There are other benefits such as throttling and memoization.
But this does not guarantee that you are not charged for the load if any. That is because, charging happens even before the message is received by the NodeJs application. Only possible place to control this is through the twilio admin panel
I want to send notification mails to users of my community platform coded with react and firebase. I was wondering is there is a way to authenticate the user via a magic link which is contained in his or her personal notification mails.
E.g. You've got 5 new comments on your post click HERE to read them
Clicking "here" should redirect the user to the page and automatically authenticate him without having to enter their individual passwords or usernames.
Due to the cancelation of the privacy shield framework and Schrems II, you might no longer be able to store customer details in the GCP.
A solution for it can be pseudonymization.
I am running an open-source project that can help you with that.
Check out the following article for more information:
https://databunker.org/use-case/schrems-ii-compliance/
I think your question can be divided into 2 different questions.
1. How do I send notification mails to my community platform?
I have never used notification mails in Firebase, I usually use google's SMTP server directly and send a request to the server to send an email on behalf of me. So, in your case, I would search up the library on your tech stack (for me it was Go's gomail), then you can use Google's SMTP server (host:smtp.gmail.com, port:587).
2. How do I make a link that automatically authenticate your users whenever they click the link?
I feel like this is the harder question. I think you need to consider a few things:
Your links need to be short lived. Why? to prevent unauthorized brute force login attempt (this also depends on the structure of your link). However, best practices suggests that links should be short lived (less than 24 hours).
Given that links are short lived, this depends on whether the user checked their email frequently for notification from you! There's a pretty good chance that they will miss that email in the 24 hour window.
I'm pretty much against sending time sensitive notification like that through emails. But if it is something that you still want to do, it is pretty easy to create the link, the simplest way I did it was:
Generate a random uuid for a link.
Associate it to the user who will login with the link. You should save this association in a persistent data storage.
Create an endpoint for magic link, for example /magic/:link_id that takes link_id then checks what user should be logged in.
Voila! You got yourself a magic link!
Finally, you can send the link through email like the first part of the problem!
Our team has a production-level Meteor app. In the app we have a particular Meteor method which sends an email. Today it sent 127 emails from one click of the Submit button (over the course of about 20 minutes).
I cannot post the exact code but the basic flow is pretty straightforward:
We catch the submit event and send everything to the server via Meteor.call
The Meteor method sends a request to a service to render a PDF
The Meteor method sends a request to SendGrid with the attachments and other email data
The Meteor method returns and triggers the callback
We don't really have much basis for determining the exact problem and are researching but are suspecting it is partly due to the end user's connection timing out and Meteor re-sending requests for which it did not get any response.
There are two threads we found related to the problem: https://groups.google.com/forum/#!topic/meteor-talk/vu5kk3t0Lr4 and https://github.com/meteor/meteor/issues/1285
Which both answer that methods should be idempotent. Obviously sending an email directly from a method is not idempotent so we proposed that the Meteor method should add these emails to a queue and have a different service process the queue on a schedule. However, we do not want to start implementing solutions that might help solve the problem.
So, this leaves me with two questions:
What exactly would cause a Meteor method to be called 127 times? How do we prevent this from happening? Is this a bug in Meteor or a bug in our app?
If we update the method so it uses EmailQueue.insert(...) (and let something else process the queue) does that just mean we, in this case, would put 127 records into the queue instead? Is the only solution here having some sort of lock to ensure duplicate records are not processed/inserted?
Thank you for any insight.
I would recommend you post the code, the process may be simple but there's usually small subleties that can help determine what's going on, i.e is it async/is there anything that may be blocking.
What may cause it
If Meteor doesn't respond to your browser it thinks it has not yet fired the call and it re-calls it. At least this is what I can gather may be happening using the information you've provided.
To get passed this ensure that
When sending the email you either use this.unblock (http://docs.meteor.com/#method_unblock) or use asynchronous JS with any of the proccesses you're doing
Ensure that nothing happening is blocking the main thread for your app. (It's hard to tell what it could be without any code)
Errors. If meteor restarts and reconnects it will prompt the browser to re-send the Meteor.call
The big clue here is that it took 20 mins, so its very likely to any of these.
How to check. Check the Network tab in your browser to see what's being sent to the server. Is the method being called multiple times? Are there re-connection attempts?
To answer the second question on whether using an Email Queue would help, it may not, it depends more on what's causing the initial problem.
Additionally: I think SendGrid automatically queues the emails anyway so you could send them all the emails at once and they queue them so they send out depending on your limits with them. (Not sure on this since I used them quite a while back)