Docker/Selenium/Headless Chrome: Configure SUID sandbox correctly - javascript

I want to run selenium and headless chrome in my docker container for testing purpose.
I have tried to run selenium in headless chrome (outside my docker container) with the following in my .js file. This worked:
const client = webdriverio.remote({
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
args: ['--headless', '--disable-gpu']
},
binary: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
},
baseUrl: CONFIG.host,
logLevel: 'verbose',
waitForTimeout: 3000
})
But I can't get this to work in my docker container. In my docker container I use "FROM selenium/standalone-chrome". There does not seem to be any problem with my dockerfile. The problem occurs when I try to run my selenium tests. I changed the binary_path in my .js file to /opt/google/chrome/google-chrome. But the tests fails and client can not even be initiated.
So I tried to just run /opt/google/chrome/google-chrome in order to see if chrome works, but then I get this error:
[0711/005304.226472:ERROR:nacl_helper_linux.cc(311)] NaCl helper
process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
I am pretty new to this (and stack overflow) so there might be some basic things I have missed.

Try to include --no-sandbox
chromeOptions: {
args: ['--headless', '--disable-gpu', '--no-sandbox']
},
As I'm doing at docker-selenium

This error message...
[1003/144118.702053:ERROR:nacl_helper_linux.cc(310)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
...implies that you have no setuid sandbox in your system, hence the program was unable to initiate/spawn a new Browsing Context i.e. Chrome Browser session.
Solution
The easiest (not so clean) solution is, if you want to run Chrome and only use the namespace sandbox, you can set the flag:
--disable-setuid-sandbox
This flag will disable the setuid sandbox (Linux only). But if you do so on a host without appropriate kernel support for the namespace sandbox, Chrome will not spin up. As an alternative you can also use the flag:
--no-sandbox
This flag will disable the sandbox for all process types that are normally sandboxed.
Example:
chromeOptions: {
args: ['--disable-setuid-sandbox', '--no-sandbox']
},
You can find a detailed discussion in Security Considerations - ChromeDriver - Webdriver for Chrome
Deep dive
As per the documentation in Linux SUID Sandbox Development google-chrome needs a SUID helper binary to turn on the sandbox on Linux. In majority of the cases you can install the proper sandbox for you using the command:
build/update-linux-sandbox.sh
This program will install the proper sandbox for you in /usr/local/sbin and tell you to update your .bashrc if required.
However, there can be some exceptions as an example, if your setuid binary is out of date, you will get messages such as:
NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
Or
Running without the SUID sandbox!
In these cases, you need to:
Build chrome_sandbox whenever you build chrome (ninja -C xxx chrome chrome_sandbox instead of ninja -C xxx chrome)
After building, execute update-linux-sandbox.sh.
# needed if you build on NFS!
sudo cp out/Debug/chrome_sandbox /usr/local/sbin/chrome-devel-sandbox
sudo chown root:root /usr/local/sbin/chrome-devel-sandbox
sudo chmod 4755 /usr/local/sbin/chrome-devel-sandbox
Finally, you have to include the following line in your ~/.bashrc (or .zshenv):
export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox

Related

Send MQTT from Browser with simple nodejs [duplicate]

I have a virtual machine that is supposed to be the host, which can receive and send data. The first picture is the error that I'm getting on my main machine (from which I'm trying to send data from). The second picture is the mosquitto log on my virtual machine. Also I'm using the default config, which as far as I know can't cause these problems, at least from what I have seen from other examples. I have very little understanding on how all of this works, so any help is appreciated.
What I have tried on the host machine:
Disabling Windows defender
Adding firewall rules for "mosquitto.exe"
Installing mosquitto on a linux machine
Starting with the release of Mosquitto version 2.0.0 (you are running v2.0.2) the default config will only bind to localhost as a move to a more secure default posture.
If you want to be able to access the broker from other machines you will need to explicitly edit the config files to either add a new listener that binds to the external IP address (or 0.0.0.0) or add a bind entry for the default listener.
By default it will also only allow anonymous connections (without username/password) from localhost, to allow anonymous from remote add:
allow_anonymous true
More details can be found in the 2.0 release notes here
You have to run with
mosquitto -c mosquitto.conf
mosquitto.conf, which exists in the folder same with execution file exists (C:\Program Files\mosquitto etc.), have to include following line.
listener 1883 ip_address_of_the_machine(192.168.1.1 etc.)
By default, the Mosquitto broker will only accept connections from clients on the local machine (the server hosting the broker).
Therefore, a custom configuration needs to be used with your instance of Mosquitto in order to accept connections from remote clients.
On your Windows machine, run a text editor as administrator and paste the following text:
listener 1883
allow_anonymous true
This creates a listener on port 1883 and allows anonymous connections. By default the number of connections is infinite. Save the file to "C:\Program Files\Mosquitto" using a file name with the ".conf" extension such as "your_conf_file.conf".
Open a terminal window and navigate to the mosquitto directory. Run the following command:
mosquitto -v -c your_conf_file.conf
where
-c : specify the broker config file.
-v : verbose mode - enable all logging types. This overrides
any logging options given in the config file.
I found I had to add, not only bind_address ip_address but also had to set allow_anonymous true before devices could connect successfully to MQTT. Of course I understand that a better option would be to set user and password on each device. But that's a next step after everything actually works in the minimum configuration.
For those who use mosquitto with homebrew on Mac.
Adding these two lines to /opt/homebrew/Cellar/mosquitto/2.0.15/etc/mosquitto/mosquitto.conf fixed my issue.
allow_anonymous true
listener 1883
you can run it with the included 'no-auth' config file like so:
mosquitto -c /mosquitto-no-auth.conf
I had the same problem while running it inside docker container (generated with docker-compose).
In docker-compose.yml file this is done with:
command: mosquitto -c /mosquitto-no-auth.conf

Adding third-party cookie exception to Cypress configuration

Using Cypress, JavaScript end to end testing framework, is there a supported mechanism to add a third-party cookie exception, or some other Chromium preference (or Electron/ if not using e.g. --browser chromium)?
I have found a mechanism to do this but it involves wrapping the browser executable, effectively a man-in-the-middle between Cypress and the "browser" binary.
Presently, Cypress does not provide direct support for this. There is a standing change request to provide API support for this and other Chromium/browser preferences: https://github.com/cypress-io/cypress/issues/3671
A solution I've created is to run Cypress with a custom browser which pre-populates the Chromium user data directory ("profile") with these preferences: https://github.com/cypress-io/cypress/issues/3671#issuecomment-549660119
#!/usr/bin/env bash
USER_DATA_DIR=`cut -d "=" -f 2 <<< ${*: -2:1}`
mkdir $USER_DATA_DIR/Default > $USER_DATA_DIR/Default/Preferences
cat << END_Preferences > $USER_DATA_DIR/Default/Preferences
{
"profile":{"content_settings":{"exceptions":{"cookies":{"example.com,*":{"setting":1}}}}}
}
END_Preferences
exec chromium $#

How to access chromedriver logs for Protractor test

I have seen that chromedriver can output a logfile (https://sites.google.com/a/chromium.org/chromedriver/logging)
This page shows how to set this up when executing the exe directly:
chromedriver.exe --verbose --log-path=chromedriver.log
I cannot figure out how to set this up in Protractor however
My current protractor.conf.js
require('babel/register');
exports.config = {
framework: 'jasmine2',
seleniumServerJar: './node_modules/protractor/selenium/selenium-server-standalone-2.45.0.jar'
};
From #alecxe's answer below and protractor's browser setup docs I tried adding the following (with and without --s) but with no apparent effect:
capabilities: {
browserName: "chrome",
chromeOptions: {
args: [
"--verbose",
"--log-path=chromedriver.log"
]
}
}
I also tried specifying an absolute path (log-path=/chromedriver.log) which also didn't work.
You can always start up your own instance of chromedriver in a separate process and tell Protractor to connect to that. For example, if you start chromedriver with:
chromedriver --port=9515 --verbose --log-path=chromedriver.log
Then you could use a configuration file for Protractor like so:
exports.config = {
seleniumAddress: 'http://localhost:9515',
capabilities: {
'browserName': 'chrome'
},
specs: ['example_spec.js'],
};
We use a shell script to add chromedriver logging, among other checks. You can then point protractor at the shell script:
protractor config:
// When running chromedriver, use this script:
chromeDriver: path.resolve(topdir, 'bin/protractor-chromedriver.sh'),
bin/protractor-chromedriver.sh
TMPDIR="/tmp"
NODE_MODULES="$(dirname $0)/../node_modules"
CHROMEDRIVER="${NODE_MODULES}/protractor/selenium/chromedriver"
LOG="${TMPDIR}/chromedriver.$$.log"
fatal() {
# Dump to stderr because that seems reasonable
echo >&2 "$0: ERROR: $*"
# Dump to a logfile because webdriver redirects stderr to /dev/null (?!)
echo >"${LOG}" "$0: ERROR: $*"
exit 11
}
[ ! -x "$CHROMEDRIVER" ] && fatal "Cannot find chromedriver: $CHROMEDRIVER"
exec "${CHROMEDRIVER}" --verbose --log-path="${LOG}" "$#"
According to the protractor's source code, chromedriver service is started without any arguments and there is no direct way to configure the arguments. Even though the chromedriver's Service Builder that protractor uses actually has an ability to specify the verbosity and the log path:
var service = new chrome.ServiceBuilder()
.loggingTo('/my/log/file.txt')
.enableVerboseLogging()
.build();
Old (incorrect) answer:
You need to set the chrome arguments:
capabilities: {
browserName: "chrome",
chromeOptions: {
args: [
"verbose",
"log-path=chromedriver.log"
]
}
},
See also:
Viewing outstanding requests
Since, the previous answer by #P.T. didn't work for me on Windows 7, I started with his suggestions and got it working on Windows. Here is a working solution for Windows 7 users.
STEP 1: Install BASH and JQ and confirm they are working on your Windows box
Download bash (for Windows 10
https://itsfoss.com/install-bash-on-windows/ ; for Windows 7
download latest here:
https://sourceforge.net/projects/win-bash/files/shell-complete/latest/ ; for Windows Server 2012 or any Windows OS that already has Git installed on it, you already have a bash.exe and sh.exe installed at C:\Program Files\Git\usr\bin or C:\Program Files (x86)\Git\usr\bin already
)
Install bash - For Windows 7/ download it and extract the zip files to a directory.
Download jq (https://stedolan.github.io/jq/) and install it in the same directory location as bash
Make SURE that you add your above directory (for Windows 7- where you extracted the bash zip files to; for other applicable OSes that have git, the path it is installed at) to your PATH system environment variable.
Once the above is installed and added to your PATH, close ALL and reopen Webstorm and any CMD windows you wish to run your work in.
Test that bash is actually installed by simply typing it on a windows command prompt
C:\git\> bash .
Doing so should produce a bash cmd prompt like this
bash$
STEP 2: Add Custom Files for Redirecting Chromedriver to user Debug Logging
Add the following files to the top level of the project (wherever your protractor-conf.js is located). These files allow us to add custom debug switches to the chromedriver.exe execution.
Note that this is necessary because these switches are not exposed through protractor and cannot be done directly in the protractor.conf.js file via the chromeOptions/args flags as you would normally expect
chromedriver.cmd -- exact source shown below:
bash protractor-chromedriver.sh %*
protractor-chromedriver.sh -- exact source shown below:
TMPDIR="$(dirname $0)/tmp"
NODE_MODULES="$(dirname $0)/node_modules"
SELENIUM="${NODE_MODULES}/protractor/node_modules/webdriver-manager/selenium"
UPDATECONFIG="${SELENIUM}/update-config.json"
EXEFILENAME="$(cat ${UPDATECONFIG} | jq .chrome.last | tr -d '""')"
CHROMEDRIVER="${SELENIUM}/${EXEFILENAME##*'\\'}"
LOG="${TMPDIR}/chromedriver.$$.log"
fatal() {
# Dump to stderr because that seems reasonable
echo >&2 "$0: ERROR: $*"
# Dump to a logfile because webdriver redirects stderr to /dev/null (?!)
echo >"${LOG}" "$0: ERROR: $*"
exit 11
}
[ ! -x "$CHROMEDRIVER" ] && fatal "Cannot find chromedriver: $CHROMEDRIVER"
exec "${CHROMEDRIVER}" --verbose --log-path="${LOG}" "$#"
/tmp -- create this directory at the top level of your project (same as the location of the protractor.conf.js file.
STEP 3: Update protractor.conf.js file.
In the protractor.conf.js file, add the following line as a property in the exports.config object. As in:
exports.config = {
.. ..
chromeDriver: 'chromedriver.cmd',
.. ..
STEP 4: Launch your tests
your test should now run and if the chrome driver outputs any log information it will appear in a file called chromedriver.???.log in the tmp directory under your project.
Important caveats
This script set up assumes you install and run protractor (and the chrome driver under it) within the local node_modules directory inside your project. That is how I run my code, because I want it complete self-contained and re-generated in the build process/cycle. If you have protractor/chromedriver installed globally you should change the CHROMEDRIVER variable within the protractor-chromedriver.sh file to match your installation of protractor/chrome driver.
hope that helps.
If you're using the seleniumServerJar, in protractor.conf.js set the logfile path to wherever you want it to write the file:
seleniumArgs: [
'-Dwebdriver.chrome.logfile=/home/myUsername/tmp/chromedriver.log',
]
If you're using webdriver-manager start to run a local selenium server, you'll need to edit the webdriver-manager file:
// insert this line
args.push('-Dwebdriver.chrome.logfile=/home/myUsername/tmp/chromedriver.log');
// this line already exists in webdriver-manager, add the push to args before this line
var seleniumProcess = spawnCommand('java', args);
In case you use webdriver-manager: webdriver manager has the chrome_logs option (you can find it in its source code (in opts.ts or opts.js in the compiled code)), so you can use it something like:
webdriver-manager start --chrome_logs /path/to/logfile.txt
I'm using this as a global afterEach hook (mocha):
afterEach(() => {
browser.manage().logs().get('browser').then(function(browserLog) {
if(browserLog && browserLog.length) {
console.log('\nlog: ' + util.inspect(browserLog) + '\n');
}
});
});

What is a good headless browser to run with protractor?

New User here.
After hours of building my smoke and regression tests, I found out after reading many cases online that phantomjs is known to be a trouble to run with protractor. Jenkins has been running phantomjs for all the tasks it has been given so far.
They need these tests to run as part of ci which does not have a windowing system installed.
So I would appreciate it if there is a recommendation for completely headless browser or a headless chrome(that would be most beneficial) and a step by step to set it up. I already have a conf.js and a e2e.conf.js file. My code works perfectly fine with chrome.
I am on a iMac and selenium webdriver, I believe.
Edit: Problem = protractor doesn't work with phantomjs.
What I have done = use different web elements and googled if anyone has faced a similar situation. Also googled for headless browsers that worked for protractor, unable to find a suitable solution.
If anyone reached here - the answers are outdated.
Chromium (on next release) now supports headless mode. no need to work hard.
You can read more here:
https://developers.google.com/web/updates/2017/04/headless-chrome
Here is an example from command line
chrome \
--headless \ # Runs Chrome in headless mode.
--disable-gpu \ # Temporarily needed for now.
--remote-debugging-port=9222 \
https://www.chromestatus.com # URL to open. Defaults to about:blank.
And you can simply trigger protractor with capabilities for chrome:
Activating chrome language flags when activating from protractor (selenium)
Here is the configuraiton I am using
capabilities: {
'browserName': browserName,
chromeOptions: {
binary: '/Users/guymograbi/Downloads/chrome-mac/Chromium.app/Contents/MacOS/Chromium',
args: ['--headless','--disable-gpu']
}
},
Update - new versions of chrome doesn't require binary property
In my environments I found I can remove the binary property as new version of chrome is available on stable branches
My protractor configuration is
capabilities: {
'browserName': 'chrome',
chromeOptions: {
args: [ '--headless', '--disable-gpu', '--no-sandbox', '--window-size=1920x1200' ]
},
},
And it works smoothly for weeks now. highly recommended.
Update - how to do this in karma is super easy
Using headless chrome in karma is super easy:
browsers: 'ChromeHeadless'
it should work with the chrome loader and everything. more info
Your best bet is to continue with Chrome. With a bit of work you can get it to work via a CI and in a headless manner - we do this using Jenkins and Docker Ubuntu servers which are headless.
You will need to configure Chrome to run headless using XVFB. You can start off by following the gist here https://gist.github.com/addyosmani/5336747
You state you are on a Mac so you can either run the headless tests via Docker on your machine or you could set up a second config for the CI tests.
Another resource http://tobyho.com/2015/01/09/headless-browser-testing-xvfb/
I would continue testing in normal browsers with a head, but would use a remote selenium server as a service - Sauce Labs or BrowserStack, see:
Integration Testing with Protractor, WebdriverJS and Sauce Labs
Running Protractor tests on Browserstack Automate
automate-node-samples
You could run your Protractor tests against CodeShip or Drone.io, both of which offer Chrome and/or Firefox running headless for free. No really...
If you've got Chrome 59+ installed, start Chrome with the following flag:
--headless
please let me know if you need more help, will write the config for you :) enjoy

Cloud 9 and Grunt.js

I'm running grunt.js on Cloud9. Most everything is working correctly, except when I try to set this option:
'watch: {
options: {livereload:true},'
If I add that, the terminal states: 'Fatal error: listen EACCES'
I'm guessing I need to use the sudo command to run that and since c9 doesn't let you run sudo, the command fails.
But why is livereload a feature that requires permission? And is there any way around it?

Categories