Visitors on my page have the option to save their prefered settings as a cookie (I know some are against it, but this is not the point of this discussion).
If the user does not have a cookie, the user is asked if he/she wants to set up settings and then if yes redirect with javascript.
Can I detect non human trafic and not ask the "question" to them?
I have noticed google speed analytics are always beeing redirected to my settingspage which gives me wrong data in the analytics page.
So can I detect the non human trafic, by php or javascript?
EDIT: I would prefer to detect them in php as I have plans to phase out the javascript as much as possible
Use a honeypot - an empty, non-visible (but not hidden) field that bots will likely fill in. Also you can try and catch the click event, since bots like Google are not likely to emulate it crawling your page. Overall your best option though is using your .htaccess file (or robots.txt) to disable crawling of unwanted pages - check this out: Block all bots/crawlers/spiders for a special directory with htaccess
It is quite easy to do this, even so, there are many options, depending on your specific needs.
Here is a simple solution:
on each page, make the first link styled to be "invisible" (opacity:0), which points to some place that either triggers some javascript, or points to some place you want for robots; also place it off-screen (top:-999px)
set a timeout (like 500ms) on page load to give a robot some time to "click" the link
after the timeout, it should be a human user -if the "trap" was not triggered
optionally you can also check for mouse activity, but the above should suffice
This should work well, because a "human user" cannot click the link, but a bot can because it reads the HTML. Beware not to: "display:none", else the bot may skip this.
I'd recommend using honeypots to detect them.
Here's an interesting Article about this.
Related
I find myself having to interact with a web page that hides state in various places so that one cannot easily share it as a URL, for example this page which allows users to look up information from city zoning applications:
https://aca.cityofberkeley.info/community/Default.aspx
You can interact with the page all you want, but the URL in the location bar will remain the same as the above.
Currently, city staff provide users with instructions like "Load this URL, click on the 'Zoning' tab, enter DRCP2020-0010 under the 'Permit Number' field, click 'Search', then when the records come up, click 'Record Info' and then select 'Attachments' from the dropdown menu, then click on the PDF document that says '2020-10-21_DRCP_APP_PCKT_2801 Adeline.pdf'". I would like to be able to replace these instructions with a URL.
Another example is the website where video from city council meetings is archived:
http://berkeley.granicus.com/MediaPlayer.php?publish_id=cbebb4e6-5b83-11eb-920e-0050569183fa
It would be nice to be able to produce a link which brings up one of the meeting videos, and seeks to a certain timestamp like 53:40, so that I can refer to something specific that was said at a meeting.
Looking at the pages that are loaded when I follow the instructions in each case, I can see that there are some POST forms, cookies, hidden input fields, and so on.
Is there some kind of tool that I can use to create "deep links" to pages like these, that were generated using non-URL hidden state, which will allow me to quickly share what I'm looking at with another user?
What I'm seeking is similar to the frmget "bookmarklet", which changes the forms on a page to use GET instead of POST. Sometimes this succeeds in producing a URL which captures form submission query parameters. However, it doesn't work for these applications, for whatever reason.
This question is possibly related to the idea of capturing a web page's DOM state using "browser screenshots" and a script called html2canvas. A possible solution might involve getting and setting cookies in a bookmarklet. Ideally something that produces a normal "https://" URL would be ideal, but if it is impossible to solve the problem except by outputting a "javascript:" URL (bookmarklet) then that is acceptable to me (in spite of the security implications). Thanks.
That seems like not a programming matter. It seems like the site has some security issues as well.
QUESTION A: About Zoning
Here are some links you can use
Direct link to Zoning (I've found it via Advanced search from the site):
https://aca.cityofberkeley.info/CitizenAccess/Cap/CapHome.aspx?module=Planning&TabName=Planning&TabList=Home%7C0%7CBuilding%7C1%7CHousing%7C2%7CPlanning%7C3%7CFire%7C4%7CLicenses%7C5%7CPublicWorks%7C6%7CCurrentTabIndex%7C3
A strange link to the list of files (I've found it via downloading a file, then going to chrome://downloads, then right-clicking the file I've download. The link has been the following):
https://aca.cityofberkeley.info/CitizenAccess/FileUpload/AttachmentsList.aspx?iframeid=ctl00_PlaceHolderMain_attachmentEdit&module=Planning&isInConfirm=False&isdetail=True&isaccountmanager=False&isAdmin=True&isPeopleDocument=&agencyCode=BERKELEY&isForConditionDocument=N
It still doesn't give the direct link to the file, but it it gives the list of attachement of the previously opened Zoning record.
Currently I have no idea what file is triggered by javascipt:__doPostBack('attachmentList$gdvAttachmentList$ctl02$lnkFileName','').
In any case, based on what we have, step one, and then step two seems like minimize the path to download the file. I guess there could be a way to download the file directly, but I currently don't see any easy way. Maybe someone else could figure it out.
QUESTION B: About video
I've used an embed link that shows all the attributes that can be used.
There is a pretty strange but working way to give the exact timestamp. Change starttime from the link below:
https://berkeley.granicus.com/MediaPlayer.php?publish_id=cbebb4e6-5b83-11eb-920e-0050569183fa&starttime=0&stoptime=undefined&autostart=1
So replacing 0 for 3600 will rewind the video forward by one hour (3600 seconds):
https://berkeley.granicus.com/MediaPlayer.php?publish_id=cbebb4e6-5b83-11eb-920e-0050569183fa&starttime=3600&stoptime=undefined&autostart=1
The problem here is that ... you cannot rewind back manually that particular hour (it just gets kind cropped out). But it works to show the exact episode.
That's a pretty strange site.
I have a bit of an issue with page formatting when I navigate away, and then hit browser back to a page.
Here is an example:
I have security questions on a form in a drop down list like so:
http://i.stack.imgur.com/ib32z.jpg
When the user selects [Type in your own question] from the drop down list, I have some jquery that animates a CSS change that pushes the form down, and makes visible a hidden field for 'custom security question'. When selected, the form looks like this:
http://i.stack.imgur.com/uVPKo.jpg
Now my dilemma is when I navigate away from this page, and then navigate back using the browsers back button, my formatting gets screwed up and looks like this:
http://i.stack.imgur.com/5Xhpi.jpg
The javascript that I have written does not trigger again on the back button so the browser doesn't know to move the form back down to accomodate the change in spacing. Is there anyway I can force the document.ready to reload or clear some kind of cache?
Thanks!
EDIT: Sorry guys, I need to reupload the images to a host and repost. Sorry for the delay.
There are basically four mechanisms for persisting state on the web:
Browser-based - the browser, if you're lucky, will save answers to form fields and re-display them when it sees an INPUT with the same name; also, some browsers will preserve some state between forward<=>back navigation
Cookie-based - pretty self-explanatory; you save a cookie with the state info, and check it later to recover the state
URL-based - navigate to a different hash of your URL, with the info you want in it (eg. "?roll_down=true")
HTML5/Local Storage - Look it up if you're interested :-)
We can basically throw 1 and 4 out, because they both rely too much on browser behavior/support, and we can't reliably rely on all browsers to handle them the way we want. That leaves #2 or #3.
Cookies allow you to save more info (as much as a cookie holds, ie. about 4k). URLs allow less info, but they have the added benefit of bookmark-ability; if the user saves the URL as a bookmark (or as a link they send a friend, or whatever), the state still gets preserved.
So, take your pick of the above, decide on how to persist your "my form is rolled down" state ... and then comes the part that (I think) you're really interested in: how do you check this state and fix things when the user clicks "back"?
That part I humbly defer to another SO post, which has already answered it:
Is there a way to catch the back button event in javascript?
I am trying to figure out the best way to acompish "unobtrusive" forms for a user (within a web app).
The purpose: keep user on the site by not asking to fill unnecessary form in. Ask for the details as only when such are needed.
The requrements are:
User should provide additional details only when it is required (email to receive notifications, login required for account page, save credit card details when checking out).
User should not leave the current page providing the additional details.
The implementation would be fairly easy if all requests would be AJAX ones. It would be easy to analyse the response (401 or so) and show the appropriate lightbox-form.
I do not see how it can be done "the right way" with plain anchors and form submits as in both cases the user actually leaves the page (by following the link or submitting a form) and there is no way to analyse the response on the client side.
Converting all links and forms to AJAX ones would be just silly.
The closest analog to what I want to achieve is the default Basic Authentication dialog in most of the browser. But obviously that just doesn't fit my requirements.
Any creative suggestions how to do that for non-AJAX requests?
Regards,
Dmytrii.
In a page sense, where "page" refers to what the user sees and not what the URL is, I only can think of following ways to update independent parts in a page with JavaScript (and thus Ajax) switched off:
Frames
Iframes
Using held-open connections there are two more ways to update a page, however these do not work reliably in all cases:
Animated GIF
CSS DIV tags with absolute positioning.
Note that this needs that your Server can keep open a session for each person looking at the page, which can be thousands. If this does not work the only possible workaround is with FRAMEs and automatic refresh, which is somewhat clumsy.
As I think that you do not want to use Frames and you do not want to render animated GIFs, I explain the CSS DIV way:
When you load the page you do not finish loading it. Instead the connection is kept open by the web server and the script handling the connection waits for additional information to arrive. When there is additional data, this is sent to the browser by encapsulating it into additional DIV tags which can overwrite other parts of the page.
Using "style" in the DIV tag and CSS position:absolute these can overwrite other information on the page like a new layer. However you need either position:absolute or must add this data to the end of the page.
How does this work with forms?
Forms usually have a known size so you can put them into IFRAMEs. These IFRAMEs get submitted to the webserver. The script there notifies the other script that new data must be output, so the waiting script renders the response and displays it in the page while the script which took the submit redisplays the form with fresh values only.
How does this work with 404 and anchors?
I don't really know because this must be tested, but here is a hint how I would try to implement this:
We have 2 issues here.
First the URL must not point to other pages but back to a server script again, so the href is under control. This script then notifies the waiting script to update the page accordingly, for example by retrieving the page and sending it to your browser. The script can check for 404 as well.
Second you must hinder the browser to switch the page when clicking on the anchor. This probably involves some clever tricks using CSS, target and server side status codes (like "gone" or redirect to the current page, whatever) to keep the browser from switching the page. I am not completely sure if that works, but if you remember download pages, these show URLs which do not switch the page but have an effect (downloading the file). That's where to start to try to hack browsers not leaving the current page without using JavaScript.
One idea not followed here is not keeping the connection of the page open but the CSS file and send new css information to the browser which then "fills in empty stubs" using the CSS way. But I doubt that this works very well, most browsers probably will parse the CSS only after loading finished, but perhaps I am wrong.
Also note that keeping a connection open never finishes the page loading, so you will see the busy-logo spinning all the time, which is unavoidable with this technique.
Having said this all I doubt you get around JavaScript.
What I wrote here is very difficult to do and therefor usually is not used because it scales badly. And it is a lot more difficult than using JavaScript alone (that's why I explained it).
With proper AJAX it is much more easy to reach your goal. Also note that you do not need to change your page source much, all you need is to add a script which augments the page content such, that for example forms suddenly use AJAX instead of a direct POST with re-rendering the page. Things which cannot be detected easily then need some hints in the tags such that the tag scanner knows how to handle the tag. The good thing then is, that with JavaScript switched off your page still works - however it then "leaves the page".
Normal HTML just was not designed to create application-like web pages like we want to see today. This all was added using JavaScript.
About popup forms
The Basic-Auth-Handler reloads the page after the user enters something into this dialog, only if cancel is hit the current page is displayed.
But there are two ways to present additional query-popups in a page using JavaScript:
The first one is the javascript "prompt", like in following example:
http://de.selfhtml.org/javascript/objekte/anzeige/window_prompt_vor.htm
(Click on the "Hier").
The second one is "JavaScript forms" which are like popups within an HTML-page.
However I consider popups to be far too intrusive and bad design.
Ajax and JavaScript is the easiest way
Unfortunately using JavaScript is never easy, but if you think JavaScript is improper or too difficult, there is no other technique which is easier, that's why JavaScript is used everywhere.
For example your page onload-Script can cycle through all Anchor-Tags and modify them such, that clicking on them invokes a function. This function then must do something clever.
Same is true for Forms. Fields which can be modified (like the user's eMail address) then have two views, on is visible, the other one hidden. The hidden one is a form. Clicking on the eMail address then switches the view (disables the first div and enables the second), such that suddenly instead of the eMail address a text form field is there containing the eMail address. If you click on the "OK" button the button changes the look into a spinner until the data is submitted, then the view switches back to the normal one.
That's the usual way to do it using JavaScript and Ajax. And this involves a lot of programming until it works well.
Sorry for not shortening this post and missing code snippets, I am currently lacking time ;)
Hidden iframe.
Set target attribute of the form to the name of the iframe. use the onload event of the iframe to determine what is the response.
Or, if you really dont like any javascript, don't hide the iframe and instead present it in a creative manner.
CSS to hide an element
#myiframe { position:absolute; left: -999em; display: none; visibility: hidden; }
But normally, display: none is enough. This is just an overkill.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I am currently building a registration page where if the user leaves, I want to pop up a CSS box asking him if he is sure or not. I can accomplish this feat using confirm boxes, but the client says that they are too ugly. I've tried using unload and beforeunload, but both cannot stop the page from being redirected. Using those to events, I return false, so maybe there's a way to cancel other than returning false?
Another solution that I've had was redirecting them to another page that has my popup, but the problem with that is that if they do want to leave the page, and it wasn't a mistake, they lose the page they were originally trying to go to. If I was a user, that would irritate me.
The last solution was real popup window. The only thing I don't like about that is that the main winow will have their destination page while the pop will have my page. In my opinion it looks disjoint. On top of that, I'd be worried about popup blockers.
Just to add to everyones comments. I understand that it is irritating to prevent users from exiting the page, and in my opinion it should not be done. Right now I am using a confirm box at this point. What happens is that it's not actually "preventing" the user from leaving, what the client actually wants to do is make a suggestion if the user is having doubts about registering. If the user is halfway through the registraiton process and leaves for some reason, the client wants to offer the user a free coupon to a seminar (this client is selling seminars) to hopefully persuade the user to register. The client is under the impression that since the user is already on the form, he is thinking of registering, and therefore maybe a seminar of what he is registering for would be the final push to get the user to register. Ideally I don't have to prevent the user from leaving, what would be just as good, and in my opinion better is if I can pause the unload process. Maybe a sleep command? I don't really have to keep the user on the page because either way they will be leaving to go to a different page.
Also, as people have stated, this is a terriable title, so if someone knows a better one, I'd really appreciate it if they could change the title to something no so spammer inviting.
As soon as I saw the words "prevent the user" I started to wail in agony. Never prevent the user, only help them.
If they see your registration page and run off, that's their choice. Pop up a javascript confirm box if they've already filled in some data (because they might be navigating away accidentally) but leave it at that. If they haven't touched the form, leave them alone - they don't want to fill in your form.
Look at other methods of engaging users. If your form is huge and scary, break it into simple manageable chunks or better yet, simplify things so much that the user only gives you data when you need it. For example, you might not need their address until you want to post something to them.
By breaking it into multiple parts you can hook them with a simple form and once they've invested that time, they'll be more likely to continue the process.
But don't harass users. If they don't want to register, pestering them with pop-ups and jaavscript dialogues will just chase them off the site completely.
With that in mind, assuming you're just trying to stop people half-filling-in forms, there are a couple of options to genuinely help people:
Detect if the form has changed and ask them a simple confirm() message.
This is all you can do. A CSS "pop-in" just won't work because you can't control* the window location in the unload event.
*You can put an event listener on all your page's links to fire off something to check the form, but this only helps if the user clicks on one of those links. It won't help if, for example, the user clicks back or closes the window. You could do both CSS and javascript but you end up with a bit of a mess.
Persist the state of the form behind the scenes.
An extension to #1. Instead of squabbling with the user, let them go where they want but save the content of the form either to session or cookie (if it'll fit) and put something on the page (like SO's orange prompt bars at the top of the page) that reminds them that they've started filling in a form and gives them a link back to the form.
When they click that link, you load the data out of the cookie (or session) back into the form and let them carry on. This has the clear benefit of letting them do what they like on your site and keeps the data safe.. ish.
If they don't come back and their cookie/session expire, that's their fault. You can only lead a horse to water. It's not your job to force it to drink.
Don't do it.
But if you want, try this. Record mouse positions and detect a quick upward thrust -- the user is reaching for the BIG X or the top left or top right. Now might be your chance for an unobtrusive box in the screen.
I've seen this implement on the web and it is evil.
If you want to trap links, you could rewrite the links in the page to go to a "you really want to leave?" javascript function, passing the destination URL as an argument.
If you wanna keep users from using their "Back" button, or keep them from putting another URL in the address bar, stop. Stop now. (1) Browsers were made to prevent exactly that kind of obnoxious behavior, and (2) Even if they allowed it, see the last two words of (1). It's freaking rude. Your site is not that special, no matter how cool you think it is.
window.onbeforeunload = function() { return "Message"; };
Use a JavaScript like this to display a leave confirmation message.
Here are just a couple of approaches I could think of but they are not without flaw:
Whatcha Gonna Do technique
Detect the mouse position going towards the edges of the browser as the user might be going to close the tab, window, go back, navigate elsewhere among other things. If so, immediately prompt them that that may be a mistake and they are going to lose out on something very valuable. However, the catch here is that you don't know for sure what their intentions were and you might piss them off with that popup. Also, they might use a bunch of shortcuts such as Ctrl+W etc to do the same.
You've Got Mail technique
If you've managed to get hold of the user's email address before they closed the page, you've hit a jackpot. As soon as the user types anything into the email box and then leaves it, immediately send it to the server using AJAX. Save the state of the page into localStorage or on the server using a cookie or something so it can be recreated later. Every couple of hours send them an email giving them a direct link to the previously saved form, and maybe with special offers this time.
History Repeats Itself technique
Then there's the infamous history manipulation where you keep stacking the current page into the document history so the back button renders effectively worthless.
Don't Put All Your Eggs In One Basket technique
Another technique off the top of my head is to create multiple windows in the background with the registration form and keep them all in sync when any the fields in any one changes. This is a classical technique and really puts the "don't put all your eggs in one basket" saying into real-life usage.
Another advantage of this awesome technique is even if the user closes one of the windows, and later comes across an identical cloned window with all the fields they filled up-to-date populated, they might get confused and think that they never closed the page. And guess what, this time they might just go ahead and fill out the registration form. But you have to be cautious with this as anything more than 2 or 3 clones will make it obvious as to what's going on.
You're Winner technique
Another technique is to tell every user they they are the Xth visitor on the site and use a good rounded number for X such as 1000, 10000, 50000, etc. Tell them that they can claim their prize once they register on the site. Imagine how special each user feels when they land on your site. The prize doesn't have to be anything tangible, it can simply be free coupons that you find on the intertubes.
Where Do You Want To Go Today? technique 1
This is basically a rip-off of your answer. Use document.location.href = 'some url' inside your onbeforeunload callback to navigate to a different page before it is unloaded.
1 Firefox only.
Note: there is no silver bullet solution here unless you write your own browser with your own security policies, but these are all optimizations that you can do to make it utterly impossible for users to leave.
Not all browsers support a modal popup, without which your page would go ahead and navigate anyway.
This is real awful requirement. The sort of requirement that is reasonable in a desktop application but entire unreasonable feature of a web site. Imagine being unable to leave a website.
The answer is either use the horrible confirm box and lump it. Or don't ask the user to enter too much data per page. Use a step by step wizard style data entry, the loss due to accidental navigation is minimised.
You can change the Value of the url using document.location.href = "www.website.com"
I can accomplish this feat using confirm boxes, but the client says that they are too ugly.
If the problem is the ugliness of the standard JS popup boxes, try something like this: http://www.sohtanaka.com/web-design/inline-modal-window-w-css-and-jquery/
Apart from that I second what most people are saying: do this with extreme caution if you don't want to lose users.
Background
I'm using POST form submissions instead of links with concatenated arguments inside a Web application, so I can exercise control over input.
Unfortunately this means users can't use the quick shortcuts they know to open links in new windows, like control-clicking or middle-clicking.
Problem
I've got what seems to be a workable way to use jQuery to capture mouse input and the various chord keys, based on the W3C DOM mouse events specification:
$("span#clickspan").click( function(event) {
buttonpress = event.button;
ctrlpress = event.ctrlKey;
$("#clickresult").empty();
$("#clickresult").append("<p>Click!</p>");
$("#clickresult").append("<p>Button # " + buttonpress + " pressed!</p>");
if (ctrlpress) {
$("#clickresult").append("<p>Control-click!</p>");
}
//form submission code would go here
event.preventDefault();
}
);
I can capture control-clicks this way (tested in Firefox 3 and IE7), and it correctly (?) reports left-clicks as coming from mouse button #0, but for some reason this code still isn't capturing middle-clicks on the span, and right-clicks still pop up the context menu. I'd like to capture middle-clicks, at least.
Can somebody help there?
What "control" means
The problem with a GET submission/link with concatenated arguments is that anybody can edit the address bar and type in anything. I've already worried about authentication and validation server-side. That's not why I want to work with POST.
I should only be showing users information that's meaningful. Internal database IDs aren't. I should only be letting users interact with the application in meaningful ways. Arbitrarily editing the address bar isn't one of those.
People make typos all the time. From the system's perspective, though, there's no difference between a typo in the address bar and a flaw in the application logic, and I'd rather not push the responsibility for deciding which one just happened off onto the users.
Short answer? Can't do it.
Long answer? Javascript mouse events. Still can't do it.
This begs the question: do you need to use POST or do you simply want to? The only reason why you'd need to is query string length. As to wanting to, you mention "controlling user input". What does that mean exactly?
You should never ever ever rely on the browser for input validation. You can do it there as a convenience but the server should always validate input.
Or is the reason aesthetic (ie shorter, "nicer" URLs)?
You're reinventing the wheel that is hyperlinks. I'm just trying to make sure you have a pretty darn good reason for doing so because it's counterproductive and you're never going to get the same browser support and compatibility as actual hyperlinks. Plus you're likely to simply annoy your users by having things they expect to work not work. Not good.