I am writing some automated tests in Selenium 2.0 using the Firefox driver. The site uses many bindings, for example an 'input' tag with a 'data-val-method-to-execute' attribute which triggers a javascript function.
Some 10% of my tests fail randomly because the driver clicks an element before the corresponding javascript function got loaded - so nothing will happen.
One solution could be Thread.Sleep - problems are: I would have to implement in all of my tests(quite a lot). They will slow my tests drastically, and time is an issue. A simple dropdown with a sleep of 1 second doesn't always work (so I would have to increase the timeout)
You need to figure out a way to make Selenium wait until the page is ready. There is no "one size fits all" solution for this. It really depends on what you do.
For some complex AJAX/JavaScript, I had to use phantomjs and add code which counts the number of open/active network connections. The test would wait until the number changes (so I know the AJAX request has been sent) and then until the number of active connections drops back to 0 (so I know the AJAX is completed).
Alternatively, try to add a hidden DIV to the page which tells the test "all scripts have finished". Wait for the DIV to appear. The problem here is to make sure that your AJAX handlers create the DIV.
In order to avoid polluting your code, use an empty function which creates the DIV and insert an additional <script> element in the head when running the tests which overwrites the function.
Related
I am a lowly operations employee without authorization to change the programs and permissions on my machine, and I would like to automate some highly repetitive data entry. I know there are a lot of programs that can do that, however, for the sake of this discussion we'll assume that I'm not allowed to have any of them and I can only script through the debug F12 menu in Chrome. I also probably don't understand half of these words as well as I should.
I have to run test cases on a third-party vendor's highly dynamic website, and I've already successfully written javascript which adds texts to elements in the DOM and presses the "next" button.
The problem is, upon .click()ing the "next" button, it takes time for the page to update, and the update creates new elements which weren't in the DOM when the script was initialized. I need to find a way to delay the execution of the script until the DOM contains all the elements I need to update.
As a really, really crude proof of concept I wrote the pre-filler for each page as a function, and I serially called each function at the end of the previous function, using setTimeout(nextfunct, 10000) to let the page update before executing the next line. (I was going to refine that by trying to create some kind of object listener instead of an arbitrary 10 second delay, but I wasn't even able to get that far.) This approach creates two errors.
1) The script seems to be checking whether the elements are on the DOM before the end of the setTimeout(), so it still gives me an error. If nextfunct is defined as
document.getElementById("doesntexistyet").value = "Fill Me";
console.log("nextfunct ran");
I will get the error message stating there is no element with the id "doesntexistyet" immediately, not after a delay of 10 seconds. The element on the next page will not update.
2) The DOM updating interrupts my script. In the above code, the console output will not ever appear in my console. If I comment out the missing element, so the function only prints a comment, it will still not appear in my console. However, if I comment out the code and I switch the setTimeout to 1ms, "nextfunct ran" will appear in my console, until the page updates, at which time the console will be deleted.
Are there ways around this which I can implement using only vanilla JS and a browser? I'm sure there's a keyword I can search for where someone has discussed this before, but it seems like the vast majority of JS autofilling discussions are oriented towards people designing code to be integrated into a website,
Thanks
I'm trying to do web scraping using puppeteer. The element I need to handle loads lately. When I click on the search button the result loads in AJAX and I need to pick the element I am trying to pick is in the search results but not in the initial load of the page. The page screenshot it is producing contains search results too and if it output the HTML source I can see the element there too. but not sure why I cannot pick it.
You can use await page.waitForSelector(cssSelector); to ask Puppeteer to wait for any element to be displayed in the UI before continuing on to further steps in your script. By default, the timeout for the wait is 30 seconds but you can set it to any timeout you wish.
So in your case I would:
Enter your search text into the search bar.
Click on the search button (this will execute your AJAX call to load the results).
Use await page.waitForSelector(cssSelector); to ask Puppeteer to wait until some element you are sure will be displayed in the UI after executing the search is visible.
Now that Puppeteer has registered the element as visible, you know that any actions you wish to perform on it will also execute correctly.
What you might find happens, if you don't use that waitForSelector() call is that the element is displayed but Puppeteer will timeout, for example, if you wish to execute a click command on an element. This is because the timeouts for click events (and other Puppeteer events which interact with elements) is very short and sometimes the script (especially in headless mode) can move to the next instruction too quickly to allow for the UI to update fast enough to keep up.
So by adding the additional waitForSelector calls, you're also making your scripts much more robust. Especially when data is being generated dynamically as they are in your case.
I hope I won't make a fool of myself but I'm trying to understand what is happening in those two lines of code:
document.body.innerHTML = 'something';
alert('something else');
What I am observing is that alert shows before HTML has been updated (or maybe it has but the page hasn't been refreshed/repainted/whatever)
Checkout this codepen to see what I mean.
Please note that even putting alert in setTimeout(..., 0) does not help. Looks like it takes more event loops for innerHTML to actually update page.
EDIT:
I forgot to mention I am using Chrome and did not check other browsers. Looks like it's only visible in Chrome. Nevertheless I am still interested why is that happening.
Setting innerHTML is synchronous, as are most changes you can make to the DOM. However, rendering the webpage is a different story.
(Remember, DOM stands for "Document Object Model". It's just a "model", a representation of data. What the user sees on their screen is a picture of how that model should look. So, changing the model doesn't instantaneously change the picture - it take some time to update.)
Running JavaScript and rendering the webpage actually happen separately. To put it simplistically, first all of the JavaScript on the page runs (from the event loop - check out this excellent video for more detail) and then after that the browser renders any changes to the webpage for the user to see. This is why "blocking" is such a big deal - running computationally intensive code prevents the browser from getting past the "run JS" step and into the "render the page" step, causing the page to freeze or stutter.
Chrome's pipeline looks like this:
As you can see, all of the JavaScript happens first. Then the page gets styled, laid out, painted, and composited - the "render". Not all of this pipeline will execute every frame. It depends on what page elements changed, if any, and how they need to be rerendered.
Note: alert() is also synchronous and executes during the JavaScript step, which is why the alert dialog appears before you see changes to the webpage.
You might now ask "Hold on, what exactly gets run in that 'JavaScript' step in the pipeline? Does all my code run 60 times per second?" The answer is "no", and it goes back to how the JS event loop works. JS code only runs if it's in the stack - from things like event listeners, timeouts, whatever. See previous video (really).
https://developers.google.com/web/fundamentals/performance/rendering/
Yes, it is synchronous, because this works (go ahead, type it in your console):
document.body.innerHTML = 'text';
alert(document.body.innerHTML);// you will see a 'text' alert
The reason you see the alert before you see the page changing is that the browser rendering takes more time and isn't as fast as your javascript executing line by line.
The innerHTML property actual does get updated synchronously, but the visual redraw that this change causes happens asynchronously.
The visual rendering the DOM is asynchronous in Chrome and will not happen until after the current JavaScript function stack has cleared and the browser is free to accept a new event. Other browsers might use separate threads to handle JavaScript code and browser rendering, or they might let some events get priority while an alert is halting the execution of another event.
You can see this in two ways:
If you add for(var i=0; i<1000000; i++) { } before your alert, you've given the browser plenty of time to do a redraw, but it hasn't, because the function stack has not cleared (add is still running).
If you delay your alert via an asynchronous setTimeout(function() { alert('random'); }, 1), the redraw process will get to go ahead of the function delayed by setTimeout.
This does not work if you use a timeout of 0, possibly because Chrome gives event-queue priority to 0 timeouts ahead of any other events (or at least ahead of redraw events).
I am doing GUI automation of my website via Selenium (RobotFramework). The problem I am facing is:
When my automation script clicks on some element (button,link etc) which is supposed to perform some action, it dose nothing. This happening randomly. When we test is manually, it works all the time. One observation is, with slower machine/environment, this tends to happen more.
I suspect this is happening either due to some corresponding JS is not loaded yet or if there is any such thing called "action binding" with each elemetnt, has not happened.
Some question
- Is there a way to find out if all the JS calls are over?
- In case action binding happens, has it already bound or not.
Please share if you have any other solution.
do you know what is last to load on the page? This should be very easy to find out via Developer Tools in your browser of choice.
You can then easily use a Wait Until Keyword (there are many variations) to wait until that last item appears as you expect, then continue with your test. Setting the timeout length and interval will help control the overhead of time/performance.
Wait Until Element Is Visible id=finalElement 10 finalElement did not appear on the screen before timeout period
http://robotframework.org/Selenium2Library/Selenium2Library.html - please see the documentation for further examples and options in terms of keywords
As many developers will be I'm producing web based application that are using AJAX to retrieve data and HTML.
I'm new to web development and javascript but have a couple of decades experience in programming in other languages.
I'm using mootools, which is a great framework, but have been battleing with the lack of destructors in javascript or even onDestroys/ unloads for the dom elements.
I've written a number of UI classes ( mostly to learn ) and alot of them use setInterval timers to periodically get data from the WebServer and update elements on the page (mostly images from cameras).
Most issue occur when another page is requested with the menu and the content div is reloaded with new HTML and Javascript ( using Request.HTML ). This simple replaces all the elements already in the div with the new one and runs the new scripts. Any timers in the old scripts or old objects created will continue to run. This was leaving me with lots of orphaned Clases, elements and timers.
I've been reading more on the mootools site and have realized a number of mistakes I've been making and have started to correct alot of the issues. The biggest of which was not using Element.store and Element.retrieve instead of linking my classes directly to the Elements.
I've already found that the contents of the div being reloaded need to be freed by calling destroy on all its child elements before calling the Request.HTML but that will not remove (clear) any timers that are running.
So I've done a JSFiddle here deinitialize classes to show what i've been trying, its appears to work fine but the following and what i want to know is,
Is it a good idea?
are there any other issues I might have missed?
can you see any problem with this type of implementation ?
or am I reinventing the wheel and missed
something?
Explanation
When the class is initialized it stores itself with the element.
It also appendes (makes if necessary) itself into an AssocClasses array also stored with the element.
I've created a ClearElement function that is called whenever the contents of an element are to be replace with and AJAX call or other method, which gets all elements within the div and if they have and AssocClasses array attached, calls the deinitialize on each of the Classes in the array, then it calls destroy on each of its direct children to free the elements/storage.
Any information, pointers etc would be most greatfully recieved.
Most issue occur when another page is requested with the menu and the content div is reloaded with new HTML and Javascript ( using Request.HTML ). This simple replaces all the elements already in the div with the new one and runs the new scripts. Any timers in the old scripts or old objects created will continue to run. This was leaving me with lots of orphaned Clases, elements and timers.
I would rethink your timer storage and use of evalScripts in your ajax calls.
Keep these outside of your AJAX requests. When doing peer code reviews rarely have I seen an instance where these were needed and could be done in a better way.
Maybe on the link that is clicked have it trigger a callback function on Complete or onSuccess
Without seeing your exact code it will be hard to advise further.