I have the following pop-up alert that I want to handle after a file upload. I have used the code below and it throws the error below.
wait.until(EC.alert_is_present())
driver.switch_to.alert().accept()
Traceback (most recent call last):
File "update.py", line 45, in
driver.switch_to.alert().accept()
TypeError: 'Alert' object is not callable
Why is this happening? I have handled a similar alert (that one had a cancel button?) in this manner.
There are two ways to accept alert available in Python + selenium (there is also JavaScript code for execute_script(), but it's not related to current issue):
driver.switch_to_alert().accept() # deprecated, but still works
driver.switch_to.alert.accept()
Note that in second line you don't need to call alert() as you did in your code
Problem with alert boxes (especially sweet-alerts is that they have a
delay and Selenium is pretty much too fast)
An Option that worked for me is:
while True:
try:
driver.find_element_by_xpath('//div[#class="sweet-alert showSweetAlert visible"]')
break
except:
wait = WebDriverWait(driver, 1000)
confirm_button = driver.find_element_by_xpath('//button[#class="confirm"]')
confirm_button.click()
Another option to handle alerts in Selenium Python would be to eliminate the notifications all together, if they are not needed.
You can pass in options to your webdriver browser that disables the notifications.
Example Python code using Chrome as the browser with options:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-notifications")
driver = webdriver.Chrome(ChromeDriverManager().install(),options=chrome_options)
driver.get('https://google.com')
print("opened Google")
driver.quit()
Related
I am trying to automate the download of research articles from scihub (https://sci-hub.scihubtw.tw/) based on their corresponding article titles. I am using a library called scholarly (https://pypi.org/project/scholarly/) to get the url, author information related to the given article title as shown in the code below.
I use the fetched url (as described above) to emulate the download process using scihub. But I am unable to download directly, since I can't press the open button on the search page (https://sci-hub.scihubtw.tw/). And pressing enter after populating the query forwards me to another page with an open button. I am unable to fetch and press the open button for some reason and it always returns me a null element using the selenium library.
However, I am able to execute the following in the browser console and successfully download the pape,
document.querySelector("#open-button").click()
But, trying to get similar response from selenium is failing.
Kindly help me resolve this issue.
## This part of code fetches url using scholarly library from google scholar
from scholarly import scholarly
search_query = scholarly.search_pubs('Hydrogen-hydrogen pair correlation function in liquid water')
search_query = [query for query in search_query][0]
## This part of code uses selenium to automate download process
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
import time
download_dir = '/Users/cacsag4/Downloads'
# setup the browser
options = webdriver.ChromeOptions()
options.add_experimental_option('prefs', {
"download.default_directory": download_dir, #Change default directory for downloads
"download.prompt_for_download": False, #To auto download the file
"download.directory_upgrade": True,
"plugins.always_open_pdf_externally": True #It will not show PDF directly in chrome
})
browser = webdriver.Chrome('./chromedriver', options=options)
browser.delete_all_cookies()
browser.get('https://sci-hub.scihubtw.tw/')
# Find the search element to send the url string to it
searchElem = browser.find_element(By.CSS_SELECTOR, 'input[type="textbox"]')
searchElem.send_keys(search_query.bib['url'])
# Emulate pressing enter two different ways, either by pressing return key or by executing JS
#searchElem.send_keys(Keys.ENTER) # This produces the same effect as the next line
browser.execute_script("javascript:document.forms[0].submit()")
# Wait for page to load
time.sleep(10)
# Try to press the open button using JS or by fetching the button by its ID
# This returns error since its unable to fetch open-button id
browser.execute_script('javascript:document.querySelector("#open-button").click()')
#openElem = browser.find_element(By.ID, "open-button") ## This also returns a null element
Ok, so I got the answer to this question. Sci-hub stores its pdf inside an iframe, so all you got to do is fetch the src attribute of the iframe after pressing enter on the first page. The following code does the job.
from scholarly import scholarly
search_query = scholarly.search_pubs('Hydrogen-hydrogen pair correlation function in liquid water')
search_query = [query for query in search_query][0]
print(search_query.bib['url'])
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
import time
download_dir = '/Users/cacsag4/Downloads'
# setup the browser
options = webdriver.ChromeOptions()
options.add_experimental_option('prefs', {
"download.default_directory": download_dir, #Change default directory for downloads
"download.prompt_for_download": False, #To auto download the file
"download.directory_upgrade": True,
"plugins.always_open_pdf_externally": True #It will not show PDF directly in chrome
})
browser = webdriver.Chrome('./chromedriver', options=options)
browser.delete_all_cookies()
browser.get('https://sci-hub.scihubtw.tw/')
# Find the search element to send the url string to it
searchElem = browser.find_element(By.CSS_SELECTOR, 'input[type="textbox"]')
searchElem.send_keys(search_query.bib['url'])
# Emulate pressing enter two different ways, either by pressing return key or by executing JS
#searchElem.send_keys(Keys.ENTER) # This produces the same effect as the next line
browser.execute_script("javascript:document.forms[0].submit()")
# Wait for page to load
time.sleep(2)
# Try to press the open button using JS or by fetching the button by its ID
# This returns error since its unable to fetch open-button id
#browser.execute_script('javascript:document.querySelector("#open-button").click()')
openElem = browser.find_element(By.CSS_SELECTOR, "iframe") ## This also returns a null element
browser.get(openElem.get_attribute('src'))
Background:
So I have very little coding knowledge, I have tried to learn coding several times through guides and youtube tutorials but never got very far in my learning. This time around I am trying a different approach, of simply having an idea and doing it.
I am trying to make a program interact with the website fitbit. Very simply it needs to login and change the email of my account.
I have successfully used Selenium python to login and access the settings page, but I cannot interact with any of the elements beyond this.
Once the program has made it to the settings page it needs to click the "Change Email Address"
Then fill in the form and click submit. That is all I need the program to do.
My code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chromedriver = webdriver.Chrome(executable_path=r'C:/Utility/BrowserDrivers/chromedriver.exe')
browser = chromedriver
browser.get('https://www.fitbit.com/settings/profile')
browser.implicitly_wait(20)
username = browser.find_element_by_xpath('//*[#id="loginForm"]/fieldset/dl/dd[1]/input')
username.click()
username.send_keys('Email')
password = browser.find_element_by_xpath('//*[#id="loginForm"]/fieldset/dl/dd[2]/input')
password.click()
password.send_keys('Password')
browser.find_element_by_xpath('//*[#id="loginForm"]/div[1]/button').click()
browser.implicitly_wait(30)
#browser.find_element_by_xpath('//*[#id="ember798"]/div[3]/div[2]/button').click()
new_email = browser.find_element_by_xpath('//*[#id="ember1077"]')
new_email.click()
new_email.send_keys('New Email')
confirm_new_email = browser.find_element_by_xpath('//*[#id="ember1107"]')
confirm_new_email.click()
confirm_new_email.send_keys('New Email')
password = browser.find_element_by_xpath('//*[#id="ember1119"]')
password.click()
password.send_keys('Password')
#browser.find_element_by_css_selector('#ember832 > div:nth-child(4) > div.column.medium-5.change-email-button-container > button').click()
#browser.find_element_by_xpath('//*[#id="ember748"]/div[3]/div[2]').click()
My Error:
DevTools listening on ws://127.0.0.1:51658/devtools/browser/12ae1773-a1f7-473f-8128-32f2090202fc
[12820:7500:1215/095935.812:ERROR:shader_disk_cache.cc(257)] Failed to create shader cache entry: -2
[12820:7500:1215/095959.060:ERROR:shader_disk_cache.cc(257)] Failed to create shader cache entry: -2
Traceback (most recent call last):
File "looptest.py", line 25, in <module>
new_email = browser.find_element_by_xpath('//*[#id="ember1077"]')
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_
by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_resp
onse
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"
method":"xpath","selector":"//*[#id="ember1077"]"}
(Session info: chrome=70.0.3538.110)
(Driver info: chromedriver=2.45.615291 (ec3682e3c9061c10f26ea9e5cdcf3c53f3f74387),platform=Windows NT 1
0.0.17134 x86_64)
I appreciate the code is messy, and I plan to tidy things up and add in if statements, trys and everything else once the program actually works.
Solutions:
I have tried to make the browser wait for the elements to load.
I have tried fiddling around with the iframe command, I do not believe the first button is on a different iframe, but the second part maybe.
I have tried using xpath, and tried using the CSS selector command as well.
The HTML:
IMAGE OF SETTINGS PAGE
Above is an image of the settings page, I want to click the button circled in red
Here is the xpath: //*[#id="ember815"]/div[3]/div[2]/button
sol the element 100% exists.
HTML OF BUTTON
There is a screenshot of the inspect of the element.
CHANGE EMAIL FIELDS
Once the button has been click this box will appear and I need to then fill in these fields.
THE HTML OF THE NEW EMAIL FIELD
Any help on this will be greatly appreciated. I know there is a quite a bit of Javascript on this page, and I reckon that is my problem. The Javascript is making it so I cannot interact with these elements but I am unsure why.
UPDATE 1.0
A user suggest I screenshot all the parts I am unsure about using the command
browser.save_screenshot('error2.png')
Interestingly the screenshot is provides is blank. SEE IMAGE
Update 2.0
Thank you for the responses so far, I have implemented the suggestion.
I implemented the this line of code
browser.find_element_by_xpath('//button[#class="button change-email-button"]').click()
and it has progressed the program so it takes me to this stage.
Here
Now I am facing the same problem with filling out the form.
Traceback (most recent call last):
File "looptest.py", line 28, in <module>
new_email = browser.find_element_by_xpath('(//input[#data-test-qa="new-email"])[2]')
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_
by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_resp
onse
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"
method":"xpath","selector":"(//input[#data-test-qa="new-email"])[2]"}
(Session info: chrome=70.0.3538.110)
(Driver info: chromedriver=2.45.615291 (ec3682e3c9061c10f26ea9e5cdcf3c53f3f74387),platform=Windows NT 1
0.0.17134 x86_64)
I have added waits and things, but still having no luck
browser.find_element_by_xpath('//button[#class="button change-email-button"]').click()
browser.implicitly_wait(30)
#new_email = browser.find_element_by_xpath('//*[#id="ember1077"]')
new_email = browser.find_element_by_xpath('(//input[#data-test-qa="new-email"])[2]')
it seem the ID is dynamic try use the folowing Xpath
# Change Email Address button
//button[#class="button change-email-button"]
# email
(//input[#data-test-qa="new-email"])[2]
# confirm email
(//input[#data-test-qa="confirm-new-email"])[2]
# password
(//input[#data-test-qa="password-for-email-change"])[2]
after clicking the Change Email button you need to wait until form popup appear
Thank you to the users that responded. We have a solution
Here the key fact as #ewwink correctly said, I failed to notice that the elements I was trying to access had dynamic ids.
I managed to fix this issue by typing out my own xpath and came up with the following:
//*[#data-test-qa="new-email"]
//*[#data-test-qa="confirm-new-email"]
//*[#data-test-qa="password-for-email-change"]
These work each time the program is run!
I need to capture the console logs (category: info) of a browser using Ruby & Capybara. Until now I have tried using driver.manage.logs.get(:browser) or (:client) but, using this, the result is not what I want. It gives out the interaction results between selenium and browser where I can see my javascript statements sent for execution, but the resulting output fails to get captured.
Whether or not logs are available when using selenium depends on what browser you are using with Selenium. If you were using Firefox you'd be out of luck since it doesn't support the log retrieval API, however since you're using Chrome they are accessible. The issue you're having is that, by default, only WARN or ERROR level logs are captured. You can change this in the driver registration through the loggingPrefs capability
Selenium 3
Capybara.register_driver :logging_selenium_chrome do |app|
caps = Selenium::WebDriver::Remote::Capabilities.chrome(loggingPrefs:{browser: 'ALL'})
browser_options = ::Selenium::WebDriver::Chrome::Options.new()
# browser_options.args << '--some_option' # add whatever browser args and other options you need (--headless, etc)
Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options, desired_capabilities: caps)
end
Selenium 4
Capybara.register_driver :logging_selenium_chrome do |app|
options = Selenium::WebDriver::Chrome::Options.new
options.add_option("goog:loggingPrefs", {browser: 'ALL'})
browser_options = ::Selenium::WebDriver::Chrome::Options.new()
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new(app,
capabilities: options,
browser: :chrome)
end
end
and then specify to use :logging_selenium_chrome as your driver
Capybara.javascript_driver = :logging_selenium_chrome # or however else you're specifying which driver to use
which should then allow you to get the logs in your tests with
page.driver.browser.manage.logs.get(:browser)
Thomas Walpole answer is correct but it seems that nowadays if you are using chrome as your driver you should use
Selenium::WebDriver::Remote::Capabilities.chrome( "goog:loggingPrefs": { browser: 'ALL' } )
Notice goog:loggingPrefs instead of loggingPrefs only with this solution i was able to get console.log printed in the log.
Took me a while and got it from here https://intellipaat.com/community/5478/getting-console-log-output-from-chrome-with-selenium-python-api-bindings after several frustrating attempts.
November 2022 update, use:
page.driver.browser.logs.get(:browser)
Not sure that this is what you want, but take a look at https://github.com/dbalatero/capybara-chromedriver-logger.
It helps me identify the problem with dynamic modules import(''). Works both locally and in Github Actions / Circle CI by displaying failed loads of assets (which i believe outputs as console.error).
I am using the Perl Selenium package, WWW::Selenium.
Trying to resize the browser window, I am getting a mysterious JavaScript error:
"Threw an exception: missing ; before statement".
Here is the code:
use strict;
use warnings;
use 5.014;
use autodie;
use warnings qw< FATAL utf8 >;
use Carp;
use Carp::Always;
use WWW::Selenium;
my $url = 'http://www.google.com'; #for example
my $sel = WWW::Selenium->new( host => 'localhost',
port => 4444,
browser => '*firefox F:\WIN 7 programs\Web & Internet\Firefox 8 bit\firefox.exe',
browser_url => $url,
);
$sel->open( $url );
$sel->wait_for_page_to_load(10000);
my $res = $sel->window_maximize(); # So far, this works fine
$res = $sel->get_eval( q{ WebDriver driver = ((WebDriverBackedSelenium) selenium).getWrappedDriver();
driver.manage().window().setSize(1040,720);} );
# (Following this: http://stackoverflow.com/questions/1522252/, Eli Colner's post)
The program then crashes here with:
"Threw an exception: missing ; before statement"
If I drop the first JavaScript line and just leave in the 2nd line, namely:
$res = $sel->get_eval( q{driver.manage().window().setSize(1040,720);} );
It bumps with: "driver not defined".
Help will be appreciated - Thanks in advance
Helen
Note: cross posted here: http://www.perlmonks.org/?node_id=1092355
I see invalid javascript in your code, you made a mistaken assumption. Regarding the referenced SO thread that you base your code on:
How to resize/maximize Firefox window during launching Selenium Remote Control?
what makes you think Eli Corner's answer/solution is "javascript"? That is Java, or C# otherwise, because only those language bindings for WebDriver (or Selenium 2) expose a WebDriverBackedSelenium feature. All other language bindings, including Perl have no such option. So even if the code syntax is correct, on execution it will fail because that's not javascript (or shall I say the referenced classes/objects are not javascript).
Your options for a solution the way I see it are:
use real javascript code and Dave Hunt's solution (in that same SO thread) ideally should work, adapted for Perl:
$sel->get_eval("window.resizeTo(1024, 768); window.moveTo(0,0);");
use Perl WebDriver binding to correctly use Eli Corner's solution (adapted for Perl), not Selenium (RC) binding that you are currently using. Perl WebDriver binding is Selenium::Remote::Driver, not WWW:Selenium. You should then be able to do something like this (there is no need for the WebDriverBackedSelenium part in Perl, but it does mean you have to switch off using Selenium RC moving to WebDriver, there's no backward compatibility support, you need Java or C# for that):
$driver->set_window_position(0, 0);
$driver->set_window_size(640, 480);
what should be right way to click on a javascript generated link on a regular time interval using python and selenium bindings? should it be using a thread?
as i would need to continue to process the input data, i need to refresh/reset a timer to continue to receive data, clicking on this given link to do this refresh (and this link is html directly generated by javascript).
best regards
You don't need thread to do this.
Use javascript function setInterval to continuously click the link.
For example:
import time
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://jsfiddle.net/falsetru/4UxgK/show/')
# Click the link every 3000 ms.
driver.execute_script('''
// argument passed from Python can be accessed by `arguments` array.
var link = arguments[0];
var timer = setInterval(function() {
link.click();
}, 3000);
''', driver.find_element_by_id('activity'))
while True:
data = driver.find_element_by_id('counter').text
print(data)
time.sleep(1)
NOTE
If you get error like follow, upgrade selenium to recent version. I experienced following error with Firefox 23.0 + selenium 2.32.0. Error was gone with selenium 2.35.0.
Traceback (most recent call last):
File "t2.py", line 12, in <module>
print driver.execute_script('''return 1 + 2;''')
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 397, in execute_script
{'script': script, 'args':converted_args})['value']
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 165, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 158, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: u'waiting for evaluate.js load failed' ; Stacktrace:
at r (file:///tmp/tmpm1sJhH/extensions/fxdriver#googlecode.com/components/driver_component.js:8360)
at fxdriver.Timer.prototype.runWhenTrue/g (file:///tmp/tmpm1sJhH/extensions/fxdriver#googlecode.com/components/driver_component.js:392)
at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmpm1sJhH/extensions/fxdriver#googlecode.com/components/driver_component.js:386)
Alternative: using thread
import threading
import time
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://jsfiddle.net/falsetru/4UxgK/show/')
def click_loop(link, interval):
while True:
link.click()
time.sleep(interval)
link = driver.find_element_by_id('activity')
threading.Thread(target=click_loop, args=(link, 3)).start()
while True:
data = driver.find_element_by_id('counter').text
print(data)
time.sleep(1)