After searching around a lot, I still can't find the answer to this question: Is it possible to login to a 3rd party website using GWT when the website is password protected?
I'm asking this because I would like to write a Google application that combines information from different websites (My news account, my forums accounts, etc.) ... like a kind of dashboard.
I have no problem doing it for non-password protected websites.
But for websites where you have to login and handle cookies, I'm just lost.
I found this very interesting tutorial that explains how to do it for Java: http://www.mkyong.com/java/how-to-automate-login-a-website-java-example/
But I can't figure out how to do it with GWT.
Any help will be greatly apreciated.
I thought about using RequestBuilder to send the authentication requests, and follow the tutorial you pointed to.
But after more consideration I guess the Same-Origin-Policy is going to prevent you to do ajax calls to another website from the client side in some browser. You also have to make sure that your domain is in HTTPS before calling remote HTTPS url. And you might run into other problems if you follow that path...
Conclusion: I suggest making those calls from the server using plain old java code.
Related
I'm working on a project for scraping some data (odds) from different sites (bookmakers).
Due to their tracking sistem, sometime i need to change the ip (to be able again to scrape their site).
I know there are services that can help me to solve this problem, like some APIs with the capability of handling the requests for me (hiding me from the sites).
But the problem of those services is that the majority do not have italians IPs, and due to the italian regulation on online gambling, we can bet only on italian bookmakers, and we can access to them only with italian IPs).
But i know that there are services able to provide me tons of italian IPs, and my idea was to solve the problem by:
Creating my proxy server to call for handling my request (a server that change IPs when i need).
Implement a function able to change the ip of my request made by my scraping tool (the server with the scraping functions).
Honestly i don't know if those ideas could be a valid solution to solve my problem, soo my question is:
How can i implement those features to my code?
And if those options are not valid, is there a solution or a service that i can use? (even if is required a payment or a subscription).
Thanks in advance to all those who will be able to give me a hand!
P.S: I'm still improving my english. I hope I was as correct as possible :)
P.S.2: I'm using NodeJS with the Selenium JavaScript library.
I have a problem trying to figure it out for few days, still didn't get any rigid solution.
I have an angular app which will be publicly available so no login or credential is needed to open the website, and user can have an Id as query param. lets' say www.mysite.com/123
to retrieve data for 123, I have a backend, that needs authentication with clientId and clientSecret, to get access_token and then call the api with access_token to get information.
If I put clientId and clientSecret in the code a hacker can steal it and will be using it.
There were some suggestion to enable CORS on backend to make sure only requests from my site will be handled. which I already did, but with knowing the clientId and clientSecret even if CORS is enable a hacker can send requests from postaman by adding origin and referrer as header.
I tried Angular Universal for server side rendering hoping the api call will be run on serve side and will be hidden from user, but api call still happing in front end.
Really stuck in this problem any kind of help is appreciated.
Since the application is small and it's only hosting a web component developed by stencilJs, if other languages such as react or vue can support my scenario I can switch
You can go extensive ways of obfuscating statically inserted token in the frontend client build, yet it is still going to be there and the only thing it will add — extra steps for an attacker to reverse it, at least to my knowledge.
So to explore the solution to your problem, i guess you want to define it in the more narrow way, i.e what are you trying to protect:
resource requiring privilege (admin panel)
then your answer will be dynamic auth.
protect your api from external usage/ddos
CORS, ratelimiting, dynamic token issuing(take notice that strategy still will be available inside frontend client code)
also i think owasp guidelines will be a good read on the topic:
https://cheatsheetseries.owasp.org/cheatsheets/REST_Security_Cheat_Sheet.html
I think i should clarify that in my opinion the solution that you are going to implement in the end in the case of the "protection of the api" is going to be building fences to jump over, rather than sealing api completely from the 3rd party usage.
I am developing a little app that is supposed to utilize Twitter Search API. You can only search tweets if you're authenticated. So I tried a js library (I am using webpack) that is recommended by Twitter as a JavaScript library for their API. I thought it would work on the front end, but it doesn't, because it uses an OAuth implementation that doesn't appear to support JSONP.
Is there a way to work with the Twitter API from the client-side? Maybe a better library? Or should I just move tweet search functionality to the back end? I've heard you need to include an 'Authorization' header for twitter API requests, and so far I am unaware of a way to inject headers into a JSONP request.
You should start with Getting Started With the Twitter API and read it thoroughly. This will help you make an informed decision about which tools to use and how to architect your application.
All calls to the Twitter API must be authenticated.
Read How To Ask a Good Question, which will improve your overall experience on Stack Overflow.
Whatever you do, don't write your own OAuth code. There are plenty of OAuth libraries already available for free that work. Writing your own OAuth mechanism is pain.
I'm new to browser extensions. i'm trying to create a Safari extension that allows me to perform basic tasks in my Github account (Create a repo, for example). I'm doing it mostly to learn how to develop extensions while practicing my JS.
The issue is that in order to connect to Github I want to use OAuth, but I have no clue of how to do that. I have googled to see if I can find a library that I can use without any results. The main problem is that when trying to register my "application" with Github, I have no idea of what URL to provide for the main application and the callback, since it is not a web application what I'm developing; it's an extension.
Does anyone have any experience with this and can give me a tip on how to proceed? What URL should I use to register with Github? Is it possible to use OAuth from an extension at all? Is there a library, SDK, anything that can help do this job?
If this question can be broaden to any browser, just tell me and I will change the title to help others in the same situation.
Thanks to all for helping.
I have used OAuth in a Safari extension and you're welcome to look at my code for an example (see link below), but this JS library for Github might have all you need. I haven't used it, but it looks nice, and it supports "personal access tokens", which I take to be a kind of OAuth token for Github that doesn't require specifying a redirect URI.
My Safari extension (Github link) uses OAuth 2.0 to talk to the Pocket API. It doesn't use a library. For the redirect URI, I use the same URL as that of the page that initiates the whole authorization process, but with "?status=done" appended to it. The page's script checks its URL, and if the URL ends in that string, then it knows that the page was loaded by the authorization window and so it can proceed with getting the access token.
One thing you should be mindful of when using OAuth in any kind of client-side application, such as a browser extension, is that your OAuth app token will be exposed to the users of your app. Even if you encrypt it in the app or retrieve it from a server over https, a user can use the browser's built-in JS debugger to get the token, because you have to decrypt it at some point in order to use it. If someone steals your token, they can use to trick people into authorizing their nefarious app that pretends to be yours. It's up to you to decide whether this risk is tolerable. The only way to avoid this risk is to have your own server-side app that communicates with the third-party API, and your extension communicates with that. But then someone could learn from your extension's code how to talk to your server-side app, so I'm not sure whether that really buys you anything.
I would really enjoy avoiding the use of PHP for authenticating users on a website I'm developing, and I was wondering if I could use a SQL database, Apache sans PHP, HTML, Javascript/JQuery, and CSS to accomplish this? I know it's a far-fetched idea probably... but I would be happy if I could.
PHP is a single web development language, there are tons of options.
Ruby, .Net(C#/VB), Java, Python are some common alternatives.
You could work something out with JavaScript, but it would likely be sacrificing quite a bit of functionality/security.
Apache does have some modules like mod_auth_mysql that let you do authentication, but it doesn't look like it's been updated in awhile. The login box will be a pop-up box from the browser that requests username and password. You won't be able to put a Username/Password box on your page like other sites do.
If your webserver can't handle the authentication, you need some sort of "glue" that goes between your server and the browser to handle the authentication piece. PHP does a good job of that, but any of the languages Ryathal mentions would work as well.
If you do use javascript, remember that all of the source code used to secure your site is visible to whoever has the knowledge to look at it.
Yes you can use authentication methods built in to .htaccess if the right modules are enabled in your Apache installation.
This is only basic http authentication though so you would just get a popup box and you cannot log out unless you close the browser I don't think.
Authentication, Authorization and Access Control - Apache HTTP Server
You would need to use a command line utility called htpasswd to generate the user files.