We Keep Coding

Selenium WebDriver — Start Chrome with DevTools open to Console panel

I know how to start Chrome with the DevTools open so please don't tell me it is a duplicate of How to open Chrome Developer console in Selenium WebDriver using JAVA
I'm trying to have the DevTools open to a specific panel. By default it opens on the Elements panel but I want it to open on the Console panel instead:
I've seen this command line switch --devtools-flags but I haven't found any example usage of it. Essentially what I'm trying to achieve is something similar to that. Obviously that doesn't work but you get the gist:
const { Options } = require('selenium-webdriver/chrome');
// …
const options = new Options().addArguments([
'auto-open-devtools-for-tabs',
'devtools-flags="panel=console"' /* <- That doesn't work. What else would? */
]);
// …

I figured out how to do this for my Ruby on Rails app that uses RSpec and Capybara. Here's the code that I use to configure my Capybara driver to select the Console tab and dock the devtools to the bottom:
options = Selenium::WebDriver::Chrome::Options.new
options.add_preference(
'devtools',
'preferences' => {
'currentDockState' => '"bottom"', # Or '"undocked"', '"right"', etc.
'panel-selectedTab' => '"console"',
}
)
...
Capybara::Selenium::Driver.new(
app,
browser: :chrome,
options: options,
desired_capabilities: capabilities,
You should be able to call the setUserPreferences function to set user preferences: https://www.selenium.dev/selenium/docs/api/javascript/module/selenium-webdriver/chrome_exports_Options.html#setUserPreferences
const { Options } = require('selenium-webdriver/chrome');
// …
const options = new Options().addArguments([
"auto-open-devtools-for-tabs"
]).setUserPreferences({
"devtools": {
"preferences": {
"panel-selectedTab": "\"console\""
// "currentDockState": "\"bottom\"" // Or "\"undocked\"", "\"right\"", etc.
}
}
});
(I haven't tested this for JS, so please try it out and let me know if it works.)
I figured out how to set these preferences by looking at ~/Library/Application Support/Google/Chrome/Default/Preferences. This is where my main Google Chrome installation stores my user preferences, and it's JSON data.
You can view all of the possible settings under devtools => preferences. Note that all the values are strings that are parsed as JSON, so you need to "double-wrap" any strings in your code, e.g. "\"console\"".
You can open your main Google Chrome browser and change the settings in the UI, and then re-open your Preferences file to see what JSON you need to set.

Selenium chrome browser won't launch in background using Electron

i've been trying for hours launching a selenium web browser and now it's done, i can't put it in headless mode ( want it to perform tasks in background ). My electron app is a simple quick start with a button launching a Selenium webdriver. Here's my code :
document.getElementById("test").onclick = function () {
require('chromedriver');
var webdriver = require('selenium-webdriver');
var chromeCapabilities = webdriver.Capabilities.chrome();
//setting chrome options
var chromeOptions = {
'args': ['--headless']
};
chromeCapabilities.set('chromeOptions', chromeOptions);
var driver = new webdriver.Builder().withCapabilities(chromeCapabilities).build();
driver.get('http://www.google.com/');
};
What it makes : When you press the test button, selenium browser is launched and appears in my screen instead of being headless or in background :(
It seems like it doens't take in consideration the args of the chromeOption variable. I tried to put many different flags with many different syntaxes but noone of them worked. Does anyone have a solution please ?

Is there a way to get a webpage's Network activity (which you can see on Chrome Dev Tools) on load via Python?

I want to listen to the Network events (basically all of the activity that you can see when you go to the Network tab on Chrome's Developer Tools / Inspect) and record specific events when a page is loaded via Python.
Is this possible? Thanks!
Specifically:
go to webpage.com
open Chrome Dev Tools and go to the Network tab
add api.webpage.com as a filter
refresh page [scroll]
I want to be able to capture the names of these events because there are specific IDs that aren't available via the UI.

Update 2021
I had to make few changes to Zach answer to make it work. Comments with ### are my comments
def get_perf_log_on_load(url, headless=True, filter=None):
# init Chrome driver (Selenium)
options = Options()
options.add_experimental_option('w3c', False) ### added this line
options.headless = headless
cap = DesiredCapabilities.CHROME
cap["loggingPrefs"] = {"performance": "ALL"}
### installed chromedriver.exe and identify path
driver = webdriver.Chrome(r"C:\Users\asiddiqui\Downloads\chromedriver_win32\chromedriver.exe", desired_capabilities=cap, options=options) ### installed
# record and parse performance log
driver.get(url)
if filter:
log = [item for item in driver.get_log("performance") if filter in str(item)]
else:
log = driver.get_log("performance")
driver.close()
return log

Although it didn't completely answer the question, #mihai-andrei's answer got me the closest.
If anyone is looking for a Python solution than the following code should do the trick:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome.options import Options
def get_perf_log_on_load(self, url, headless = True, filter = None):
# init Chrome driver (Selenium)
options = Options()
options.headless = headless
cap = DesiredCapabilities.CHROME
cap['loggingPrefs'] = {'performance': 'ALL'}
driver = webdriver.Chrome(desired_capabilities = cap, options = options)
# record and parse performance log
driver.get(url)
if filter: log = [item for item in driver.get_log('performance')
if filter in str(item)]
else: log = driver.get_log('performance')
driver.close()
return log

You could side step chrome and use a scriptable proxy like mitmproxy.
https://mitmproxy.org/
Another ideea is to use selenium to drive the browser and get the events from perf logs
https://sites.google.com/a/chromium.org/chromedriver/logging/performance-log

tor browser selenium javascript

I'm using selenium with Tor but it's not working , i saw that there is a library for doing that but only with python . Can this be done with javascript ? I tried that but it doesn't work.
const {Builder, By, Key, until} = require('selenium-webdriver');
var driver = new Builder()
.forBrowser('tor')
.build();
driver.get('https://www.google.com')

As far as I remember, this can be done only from Java and Python.

I had trouble getting the latest geckodriver (0.21.0) and Selenium (3.13.0) to fetch a web page after launching the Tor Browser Bundle. I think it may be incompatibilities with the older Firefox version Tor uses and the geckodriver but am not sure.
If you're just trying to use selenium-webdriver to use the Tor network, try this:
const webdriver = require('selenium-webdriver');
const firefox = require('selenium-webdriver/firefox');
var options = new firefox.Options();
options.setPreference('network.proxy.type', 1) // manual proxy config
.setPreference('network.proxy.socks', '127.0.0.1')
.setPreference('network.proxy.socks_port', 9050)
.setPreference('network.proxy.socks_remote_dns', true) // resolve DNS over Tor
.setPreference('network.proxy.socks_version', 5)
let driver = new webdriver.Builder()
.forBrowser('firefox')
.setFirefoxOptions(options)
.build();
driver.get('https://example.com/')
You will need to run Tor using the expert bundle or install and run it natively.
Here's what I tried for actually getting Tor Browser to automate. It launches everything correctly but never navigates to the page.
const webdriver = require('selenium-webdriver');
const firefox = require('selenium-webdriver/firefox');
var options = new firefox.Options();
options.setBinary('/home/me/Desktop/tor-browser_en-US/Browser/start-tor-browser');
options.addArguments('--detach');
(async function run() {
let driver = await new webdriver.Builder()
.forBrowser('firefox')
.setFirefoxOptions(options)
.build();
await driver.get('https://example.com/')
})();
Just to re-iterate, this second example isn't working. On Mint 18 and Tor Browser 7.5.6 (FF ESR 52.9.0) it launches Tor and the browser just fine, but will not navigate to a page.

Disable Javascript in Selenium? Noscript doesn't seem to work

I am trying to disable javascript in selenium using noscript extension, as suggested here -->How to disable Javascript when using Selenium by JAVA?
But, it looks like it no longer works,
Here is what I have written :
FirefoxProfile profile = new FirefoxProfile();
File extPath = new File("noscript.xpi");
profile.addExtension(extPath);
//profile.setPreference("javascript.enabled", false);
WebDriver driver = new FirefoxDriver(profile);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.get("http://enable-javascript.com");
Even tried it by loading a profile that has javascript disabled, that doesn't seem to work either.
Code :
File profileDirectory = new File("Profiles/4hsi6txm.testing");
FirefoxProfile profile = new FirefoxProfile(profileDirectory);
DesiredCapabilities cap = DesiredCapabilities.firefox();
//
cap.setCapability(FirefoxDriver.PROFILE, profile);
WebDriver driver = new FirefoxDriver(cap);
driver.get("http://enable-javascript.com");
I am trying it on Firefox version-43.0.1 and selenium version-2.48.2
Any fix for this ?
Thanks :)