Summery of the question
The question is about how I can incorporate javascript modules in python.
I need to operate the fakebrowser's driver in python the same way it does in javascript so it can keep all the manipulated values of the browser fingerprint & system.
Full question
I have created an automated script for a website using selenium & python. What my code does is that it manages multiple accounts but after some time the website can detect that the accounts are being used from the same devices even after using various methods to make it undetectable (Proxies, User-Agent, Driver Options, and a ton more).
So the best module I found that can get my job done is fakebrowser because it can bypass most of the known ways websites use to detect automation. Fakebrowser can build a chromium driver but the problem I am facing is that it is using javascript and I don't want to rewrite all my code in javascript while I have already written it in Python (because I know only the basics in Javascript).
I have tried a lot of other solutions, below are only two of them but nothing I have tried so far have worked.
Attempt #1
I have tried using the chromium build which fakebrowser creates.
driver = webdriver.Chrome(executable_path=r"C:\Users\Hidden\.cache\puppeteer\chrome\win64-1069273\chrome-win\chrome.exe")
driver.get('https://pixelscan.com/')
It still leaks most of the information about my system & browser fingerprint because most of the modifications to the browser are done when you open the chrome build using fakebrowser's module.
Attempt #2
I have tried to open the driver using javascript and then start controlling it using selenium python which I found in here:
https://stackoverflow.com/questions/8344776/can-selenium-interact-with-an-existing-browser-session
Fakebrowser Javascript Code
const {FakeBrowser} = require('fakebrowser');
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
!(async () => {
const createBrowserAndGoto = async (userDataDir, url) => {
const builder = new FakeBrowser.Builder()
.vanillaLaunchOptions({
headless: false,
args: ['--remote-debugging-port=9223']
})
.userDataDir(userDataDir);
const fakeBrowser = await builder.launch();
const page = await fakeBrowser.vanillaBrowser.newPage();
await page.goto(url);
await sleep(1000000000);
await fakeBrowser.shutdown();
};
createBrowserAndGoto('./fakeBrowserUserData1', 'http://pixelscan.net/').then(e => e);
})();
Selenium Python Code
from selenium import webdriver
from selenium.webdriver.chromium.options import ChromiumOptions
class Driver:
def __init__(self):
self.options = ChromiumOptions()
self.options.add_experimental_option("debuggerAddress", "127.0.0.1:9223")
def launch(self):
driver = webdriver.Chrome(r"C:\Users\User\.cache\puppeteer\chrome\win64-1069273\chrome-win\chrome.exe", chrome_options=self.options)
driver.get('http://pixelscan.net/')
Driver().launch()
Observation
Instead of connecting to the already open browser, it creates a new one but it's not using the same values as the fakebrowser driver does. So it's still leaking most of the information of my system & browser.
I need a way to control the same one that is already open or maybe I can use the driver variable created in JavaScript and pass it to the python code but I am not sure if that's even possible
The fakebrowser package has two versions:
The basic version is puppeteer based which uses JavaScript hooks to modify properties and provides a simple api to make your web bot undetectable.
In the advanced version fakechrome recompiled Chromium to complete the simulation more robostly.
This usecase
Now you being able able to open the driver using javascript, but using Selenium-Python client when you initiate a new browsing session you still won't be able to attach it to the previous session.
This issue was discussed at length within the thread Allow webdriver to attach to a running browser.
#simon.m.stewart, WebDriver creator took a final call on this requirement as NotFeasible and further added:
I'm going to make a call on this one: it's a browser specific feature,
and not something that we can implement in a general way. With IE,
it's possible to iterate over the open windows in the OS and find the
right IE process to attach to.
Firefox and Chrome, OTOH, need to be started in a specific mode and
configuration, which means that just attaching to a running instance
isn't technically possible.
Closing as "not feasible" here as this is a browser specific feature.
Conclusion
It's still not possible to attach webdriver to a running browser instance.
As #undetected-selenium allready in his answer mentioned, this seems not to be possible.
However, if, your goal is to use the bowser for different profiles, you cantry using Selenium-Profiles.
It bypasses bot-protections like cloudfare and currently allows the following properties for the profile:
{
"options": {
"browser": {
"sandbox": true,
"window_size": {"x":1024,"y":648},
"headless": false,
"load_images": true,
"incognito": true,
"app": false,
"gpu": false,
"proxy": null,
"proxy_method": null
},
"extensions": {
"extension_paths": [],
"auth_proxy": {"host":"host","port":9000,"username":"user", "password":"password"}
},
"option_args": ["--my-arg1", "..."],
"capabilities": [],
"adb": false,
"adb_package": "com.android.chrome",
"use_running_app": true
},
"cdp": {
"browser": {
"pointer_as_touch": false,
"darkmode": false,
"mobile": true
},
"touch": true,
"maxtouchpoints": 5,
"cdp_args": [],
"emulation": {"mobile":true,"width": 384, "height": 700, "deviceScaleFactor": 10,
"screenOrientation": {"type": "portrait-primary", "angle": 0}},
"useragent": {
"platform": "Linux aarch64",
"acceptLanguage":"en-US",
"userAgent": "Mozilla/5.0 (Linux; Android 11; HD1913) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Mobile Safari/537.36",
"userAgentMetadata": {
"brands": [{"brand": "Google Chrome", "version": "105"}, {"brand": "Not)A;Brand", "version": "8"},
{"brand": "Chromium", "version": "105"}],
"fullVersionList": [{"brand": "Google Chrome", "version": "105.0.5195.136"},
{"brand": "Not)A;Brand", "version": "8.0.0.0"},
{"brand": "Chromium", "version": "105.0.5195.136"}],
"fullVersion": "105.0.5195.136",
"platform": "Android",
"platformVersion": "11.0.0",
"architecture": "",
"model": "HD1913",
"mobile": true,
"bitness": "",
"wow64": false}
}}
}
A example profile for Windows would be:
"options": {
"browser": {
"gpu": false,
"window_size": {"x":1024,"y":648}}
},
"cdp": {
"touch": true,
"maxtouchpoints": 10,
"emulation": {"mobile":false,"width": 1024, "height": 648, "deviceScaleFactor": 1,
"screenOrientation": {"type": "portraitPrimary", "angle": 0}},
"useragent": {
"platform": "Win32",
"acceptLanguage":"en-US",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36",
"userAgentMetadata": {
"brands": [{"brand":"Google Chrome","version":"107"},
{"brand":"Chromium","version":"107"},
{"brand":"Not=A?Brand","version":"24"}],
"fullVersionList": [{"brand":"Google Chrome","version":"107.0.5304.88"},
{"brand":"Chromium","version":"107.0.5304.88"},
{"brand":"Not=A?Brand","version":"24.0.0.0"}],
"fullVersion": "107.0.5304.88",
"platform": "Windows",
"platformVersion": "10.0.0",
"architecture": "x86",
"model": "",
"mobile": false,
"bitness": "64",
"wow64": false}
}
}
}
and of course it also supports settiing, deleting and getting cookies :)
I've followed the Create a JAR with the scripts to deploy instruction to create a custom Javascript policy for checking user attributes and it seems to be deployed successfully, however, I can't see the policy name in the create policy dropdown. Is there any setting I need to enable for this feature?
After uploading the .jar file to /keycloak/standalone/deployments, it says WFLYSRV0010: Deployed "script.jar" (runtime-name : "script.jar") without any error. I can also see the {filename}.jar.deployed next to my script file.
Setup: I use docker-compose to launch a jboss/keycloak version 10.0.1 container. Here's my docker-compose file:
version: "3.1"
services:
keycloak:
image: jboss/keycloak
restart: always
ports:
- 8080:8080
environment:
KEYCLOAK_PASSWORD: admin
KEYCLOAK_USER: admin
volumes:
- ./deployments:/opt/jboss/keycloak/standalone/deployments
keycloak-scripts.json:
{
"policies": [
{
"name": "my-policy",
"fileName": "my-script-policy.js",
"description": "My Policy from a JS file"
}
]
}
my-script-policy.js:
var context = $evaluation.getContext();
var contextAttributes = context.getAttributes();
if (contextAttributes.containsValue('kc.client.network.ip_address', '127.0.0.1')) {
$evaluation.grant();
}
Client policy setting
Thank you in advance for any suggestions.
Ok so I'm trying to put the capabilities declaration for my Appium tests into another file and just reference it but I'm having no luck so far with using classes, functions, arrays and even global variables. I have also tried pulling it from a csv file but since JS is asynchronous that didn't work either.
I am not very familiar with JS and honestly stumped so if anyone could point me in the right direction that would be greatly appeciated!
Here's the code that I'm trying to seperate/make a reference for:
const opts = {
port: 4723,
desiredCapabilities: {
platformName: "Android",
platformVersion: "8.1.0",
deviceName: "Nexus 6P",
app: "C:/Users/reina.reinhart/KonyWorkspace/temp/dcpApp/build/luaandroid/dist/luavmandroid.apk",
automationName: "UiAutomator2",
noReset: true
}
}
electron-builder version: 20.9.2
Target: windows/portable
I'm building a portable app with electron-builder and using socket.io to keep a real-time connection with a backend service but I have an issue with the firewall. Because this is a portable app everytime the app is opened it looks that it is extracted in the temporary folder, which will generate a new folder (so the path to the app will be different) in every run which will make the firewall think that this is another app asking for the connection permissions. How can I change the extraction path when I run the app?
(This is the screen that I get every time I run the app)
This is my socket.io configuration
const io = require("socket.io")(6524);
io.on("connect", socket => {
socket.on("notification", data => {
EventBus.$emit("notifications", JSON.parse(data));
});
});
My build settings in package.json
"build": {
"productName": "xxx",
"appId": "xxx.xxx.xxx",
"directories": {
"output": "build"
},
"files": [
"dist/electron/**/*",
"!**/node_modules/*/{CHANGELOG.md,README.md,README,readme.md,readme,test,__tests__,tests,powered-test,example,examples,*.d.ts}",
"!**/node_modules/.bin",
"!**/*.{o,hprof,orig,pyc,pyo,rbc}",
"!**/._*",
"!**/{.DS_Store,.git,.hg,.svn,CVS,RCS,SCCS,__pycache__,thumbs.db,.gitignore,.gitattributes,.editorconfig,.flowconfig,.yarn-metadata.json,.idea,appveyor.yml,.travis.yml,circle.yml,npm-debug.log,.nyc_output,yarn.lock,.yarn-integrity}",
"!**/node_modules/search-index/si${/*}"
],
"win": {
"icon": "build/icons/myicon.ico",
"target": "portable"
}
},
Any idea about how at least I could specify an extraction path or make this extract it the execution folder?
BTW I already created an issue about this in the electron-builder repo
In version 20.40.1 they added a new configuration key unpackDirName
/**
* The unpack directory name in [TEMP](https://www.askvg.com/where-does-windows-store-temporary-files-and-how-to-change-temp-folder-location/) directory.
*
* Defaults to [uuid](https://github.com/segmentio/ksuid) of build (changed on each build of portable executable).
*/
readonly unpackDirName?: string
Example
config: {
portable: {
unpackDirName: "0ujssxh0cECutqzMgbtXSGnjorm",
}
}
More info #3799.
I'm attempting to setup Nightwatch.js for the first time. I'm following the following tutorial: https://github.com/dwyl/learn-nightwatch
Unfortunately I've hit a roadblock, and I'm in need of help resolving it.
Error retrieving a new session from the selenium server.
Connection refused! Is selenium server started?
nightwatch.conf.js
module.exports = {
"src_folders": [
"test"// Where you are storing your Nightwatch e2e/UAT tests
],
"output_folder": "./reports", // reports (test outcome) output by nightwatch
"selenium": {
"start_process": true, // tells nightwatch to start/stop the selenium process
"server_path": "./node_modules/nightwatch/bin/selenium.jar",
"host": "127.0.0.1",
"port": 4444, // standard selenium port
"cli_args": {
"webdriver.chrome.driver" : "./node_modules/nightwatch/bin/chromedriver"
}
},
"test_settings": {
"default": {
"screenshots": {
"enabled": true, // if you want to keep screenshots
"path": './screenshots' // save screenshots here
},
"globals": {
"waitForConditionTimeout": 5000 // sometimes internet is slow so wait.
},
"desiredCapabilities": { // use Chrome as the default browser for tests
"browserName": "chrome"
}
},
"chrome": {
"desiredCapabilities": {
"browserName": "chrome",
"javascriptEnabled": true // set to false to test progressive enhancement
}
}
}
}
guinea-pig.js
module.exports = { // addapted from: https://git.io/vodU0
'Guinea Pig Assert Title': function(browser) {
browser
.url('https://saucelabs.com/test/guinea-pig')
.waitForElementVisible('body')
.assert.title('I am a page title - Sauce Labs')
.saveScreenshot('ginea-pig-test.png')
.end();
}
};
Based on the configuration setup. I kept it as basic as possible. I cannot pinpoint the source where it would suggest another selenium server has started. Any ideas?
EDIT: TIMEOUT ERROR
In your nightwatch.json file, within "selenium"
Make sure your server path is correct.
Make sure your webdriver.chrome.driver path is correct.
Those are specific to your machine. If those do not refer to the correct file in the correct location, you'll get problems starting the selenium server.
After that, you want to make certain that the version of the selenium server you have works with the version of chrome driver you have and that that will work with the version of the Chrome browser you have.
But as Krishnan Mahadevanindicated, without the whole error message, we can't be of much more help.
The solution involved deleting my instance of Chrome (although it was the most recent version) and simply reinstalling the browser again.
I encourage all facing the same problems to first look at QualiT's response above as it's the more conventional troubleshooting strategy.
I had got the same issue when I used vue-cli init on my project. after I updated to Java 9, this problem was resolved.