I'm not an english native speaker, I'm sorry if what I say make you confused. Thanks.
I'm crawing something from a website recently.The normal operation steps should be as follows:
I log on to this website
I jump to a subpage
I click a button (actually it executes a JS function)
The website return something
But this is inconvenient, because I always need to open the browser to complete these steps. So I found the XHR of step3 in F12 and copied it to a curl command to loop through.
This is much simpler than before, but I still need to log in with a broswer and copy the latest cookie to update my curl command.
Supposing that Step1 and Step3 are actually XHRs, and I know what the args mean.
My question is, how can I get the latest cookies without using a browser, I mean, a real browser software (which I think is the only problem to be solved)? Or is there anything else I don't know that needs to be done?
A typical login consists of a post-request that is sent to the server. The payload typically contains the username and password in plain text like it was sent from a html-form. The answer from the server then contains the session-cookie (sometimes called PHPSESSID or similar) in its header.
The session-cookie can then be used to place further requests.
Sometimes the session-cookie is already set before you login, but only becomes active upon logging in. Generally you will get a new session-cookie every time you connect.
Some websites deploy technologies like google-nocaptcha-recaptcha that prevent bots from logging in at all, just so you know.
Related
I want to make a secret URL that will authenticate a user to do something, but which won't be there in the browser history.
e.g. I thought something like this would work
Special link, e.g. https://example.org/page#specialHashGivesAccess is sent to the right people by email. Email is considered secure enough for this purpose.
Javascript running on the page sends the secret to a server via ajax which checks it and returns a temporary hash instead.
Javascript does a window.location.replace to the page with the temporary hash.
The app then checks the temporary hash and if OK shows the content.
the temporary hash expires in a short while, or can be manually expired.
The original special link (1) is not in browser history.
Further more, the script that does this work; the app script and the ajax server, are remote. i.e. siteA that generates example.org/page has <script src="https://siteb.com/app.js"> and ajax is done using CORS.
However it does not seem reliable in my testing, I'm finding specialHashGivesAccess in my history.
Is there a way to achieve this?
It can't be done (at the time of posting this answer).
While there are many methods for manipulating the session history, it is not possible to prevent an entry being created in the global history, or to overwrite the first page load request stored there.
Therefore the solution I'm going to go with is along the lines of
Hi,
To access the service you can use the following link and password:
https://example.org/myApp
password: aabbccddeeff00112233
If nobody else uses your browser you can use the following shortcut link to save you entering the password, however note that this will give access to the service to anyone that has access to your browsing history.
https://example.org/myApp#aabbccddeeff00112233
I need to do logging in JMeter. In Firefox, i go to the website, enter username and passwort and press the login button. Then i check in console which POSTs are shown. I take login data from xml and give it to HTTPRequest as a parameteres in JMeter. And right now i have a problem. This website, before going to the first page after logging, shows JS PopUp with warning (yes or no to choose). JS code to this popup is shown in console as a Response in this POST method. This popup blocked going further - a content from the URL from this POST is "Acces Denied".
How to fix it? I was thinking about some If Controller but I don't have any specific idea.
I would be greatful ;-)
Forget about JavaScript. JMeter is not a browser.
As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages.
So JMeter won't be able to "see" the popup or to "type" anything in it. However JMeter can mimic the relevant request just like it is originated from the browser. So instead of building your request manually I would suggest to record it. Even record 2 times so if there are any differences caused by dynamic parameters you could identify and work them around.
Also as JMeter is not a browser it requires some extra configuration in order to be more browser-like, i.e.:
HTTP Cookie Manager - to represent browser cookies and deal with cookie-based authentication
HTTP Header Manager - to represent browser headers e.g. User-Agent
HTTP Cache Manager - to mimic browser cache so in case of subsequent requests to the same URL JMeter wouldn't re-download all associated images
See How To Make JMeter Behave More Like A Real Browser article for detailed explanation of the aforementioned configuration suggestions and few more tips.
Thanks ! I recorded it and it helps me a lot. But now a have further issue:
This popup is optional and it is shown when more than 1 user is logged in with the same username. If this happens - I get as a POST response a js code like "new Popup(....etc... ". But when the PopUp won't appear - i get a piece of URL(POST response) which is sticked to a normal URL when there is only 1 user logged. Now I want to consider this two cases. How to do it in JMeter? I made a Response Assertion and as a Pattern to Test I put this piece of URL. If this second condition appers - everything is fine but when this PopUp apers - I would like to stick this piece of URL to the main address and go further.
I have tried to record it but it is just a JavaScript operation and I can't find this moment in JMeter recording. As You said - i have to forget about JavaScript ;)
So, is there any possibility to do it?
I am running a LAMP server and when users access the server homepage I am using the Apache basic authentication to log them in. Is there a way to force the user to logout (or be logged out)? I tried the mod_session & mod_auth_form to no avail. The page I'm using is a mix of php/html. I am self taught on all of my programming skills so I lack the depth to get a workable solution. Thanks.
The short answer is you can't.
Check these sources:
No correct way exists, not even one that's consistent across browsers.
https://stackoverflow.com/a/449914/3960852
Consequently, the answer to this question is, you can't. Sorry.
https://stackoverflow.com/a/1163876/3960852
However, there are certain ways for CERTAIN browsers. If you plan to only support a specific browser, which is a valid course of action, there are hackish ways of accomplishing it.
https://stackoverflow.com/a/36176917/3960852
However, the jury is out. If you want to log out HTTP authenticated users with extensive browser support, you're out of luck.
After looking at these suggestions I worked out a solution, albeit not a clean and elegant one. I followed the idea of the protected folder with an htaccess file. When the user clicks on my logout link they get a JavaScript alert to click on cancel when prompted for username and password. In my htaccess file I put in an override for a custom 401 error document which I then made as a logout page.
The user clicks cancel and gets redirected to the page telling them the logout was successful. I put a link on that page to log back in as well. With Firefox so far the user has always had to submit credentials to log back in. Dirty but it works.
I have a happy javascript based client that talks to my Django server-side code. I use several XMLHttpRequests for interactions and everything is fine there. Now there is one user interaction where I'd like to receive user-sent info at the server, but I don't need/want a response.
As I understand Django all views MUST respond with an HttpResponse item. I can send back a blank one, or a HttpResponseNotModified response. But anything I receive back seems to prompt the browser side to change to a blank screen.
I don't want to change pages, reload or anything. Just leave the browser session alone. I don't need a confirmation, and don't mind if some of the POST sends are even lost among a large population of responses.
I COULD do an XMLHttpResponse and catch the response and ignore it. But I'm wondering if there is any way around that.
You should return a Http204/No Content, which tells the browser that your server acknowledged and processed your request but it didn't return anything and handle that appropriately in your application (however you want to).
If you're using jQuery (for example), then this will automatically go to your success handler, and if you have no control over that, you've done the best you could.
Read the full list of status codes here
I want force users to first click on some links and then go to a specific address. How can I terminate traffic sources that are direct in the second page? Or is there any way to only accept requests that are from specific a url?
Note: if there is something wrong, I want the page not to be viewable.
Thank you
What you could do is use the document.referrer property, and check weather or not the url matches the first page. Something like this, perhaps:
if (document.referrer !== 'http://www.google.com')
{
window.location.href = 'http://www.google.com';//redirect to first url
}
As Bergi pointed out in his answer: this kind of functionality is really better left to the server side. Though suggesting cookies is, as far as I'm concerned, not the ideal way of doing this, since cookies can be turned off client side, or WCS, tempered with.
Depending on which server side technology you're using, it might pay off to read up on all tools you have at your disposal to play with the request the user sends to your application, and take appropriate actions accordingly.
You should set a cookie in the first page that the user is allowed to view the second one.
Then your serverside application would check the cookies before delivering the second page, and otherwise redirect or show an error message.
You should not do such things via JavaScript (which includes the jQuery library), elsewhile everybody who disables JS could view the page.