I'm trying to use a puppeteer to access my bank in this scenario capital one. Although when I try I get "There was an issue accessing your account, please try again, if you continue to see this message, give us a call so we can help." I'm not too surprised that they don't appreciate me trying to log in with a headless browser.
this is the code I'm using
async function credit(creditcost) {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.capitalone.com/');
await page.mouse.click(167, 121, {button: 'left'})
await page.keyboard.type('username')
await page.mouse.click(431, 123, {button: 'left'})
await page.keyboard.type('password')
await page.mouse.click(672, 121, {button: 'left'})
await delay(5000);
await page.screenshot({ path: './test/example.png' });
await browser.close();
}
Is there any way to get around this issue, or a possibly better option? I'm trying to get balances and transfer funds.
I'm not sure if that is the smartest/safest way to do this, but remember to make sure that your credentials are safe. The first thing is that the website does probably detect the headless browser.
You may want to try puppeteer-extra, (the Stealth package) when using headless mode.
Next thing - since you're using headless mode, you may want to click the elements using their selectors or id's instead of using mouse coordinates. page.$() or page.waitForSelector() should suffice.
More about puppeteer-extra-plugin-stealth here.
Related
I use Playwright framework on JS to autofill unknow Google form (which means i dont know Xpath to specify the answer, i just know to question. In my situation, form ask about address, name, size, phone number).
const { webkit } = require('playwright');
const URL = 'https://forms.gle/B4r6qZKdyxZCApTWA';
(async () => {
const browser = await webkit.launch({ headless: false });
const page = await browser.newPage();
await page.goto(URL);
await page.fill('input:below(:has-text("Họ và tên"))','name');
await page.fill('input:below(:has-text("Số điện thoại"))','phone number');
await page.fill('input:below(:has-text("Địa chỉ"))','Address');
await page.fill('input:below(:has-text("CMND"))','id');
await page.fill('input:below(:has-text("Game"))','LOL');
await page.pause();
await browser.close();
})();
URL: https://forms.gle/B4r6qZKdyxZCApTWA
The name and number field is fine but in the address field, things get mess up. It skip and jump to the id field and fill 'address'->'LOL'->'id'
The answer field of Google Form has 2 kind: input and textarea. I just need to change it. But any better way to do more "general" to fit that kind of GForm?
I was trying to query a website: const url = "https://personal.vanguard.com/us/FixedIncomeHome" with the hope to automate some functionality within puppeteer.
I noticed if i create a screen shot: page.screenshot("preclick.png") it will show the page data with tabs. When i try to follow it up with a query, it seems to not return the second tab (denoted by the following selector: a[container="CD"]
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto(url, {waitUntil: 'networkidle2'})
page.screenshot("start.png")
page.evaluate( () => {
document.querySelectorAll("a[container='CD']")[0].click()
})
///...
and i dont really know why this is the case. Ideally, i am trying to click CD and then click an empty search. I noticed that since session ids are tracked, I wanted to do this as a sort of E2E test in order to get the resulting table data.
I see that the Content of tab etc is dynamically loaded, so somehow there is an issue with the page being able to query.
I was attempting something else to see what would occur, waiting for the tag to appear, BUT it would just timeout after 30 seconds:
await page.waitForSelector("a[container='CD']").then( async resolve => {
page.execute( () => document.querySelector("a[container='CD']").click() );
});
I dont know why the screenshot shows the HTML, but when attempting to query for it from within execute it fails. It doesnt make sense to me why this occurs. Ideally, I want to click CD tab, then i want to click Search, then i want to loop through the 20 results in the table.
EDIT I was noticing that evaluate was not querying the component correctly because of an iframe. If i want to develop e2e testing though, i assumed there was a way to somehow get a reference to the button and click it, or simulate a click.*
You can get the iframe from a selector. As the iframe has the ID TWRIFrame, you can wait for that selector, then get the contentFrame from that element.
Once you have the frame, the frame class has almost the same functions as the page class, e.g. click.
Notice that, as that iframe is from other domain, with the --disable-features=site-per-process flag.
const browser = await puppeteer.launch({headless: false, args: ['--disable-features=site-per-process']});
const page = await browser.newPage();
await page.goto('https://personal.vanguard.com/us/FixedIncomeHome', {waitUntil: 'networkidle2'});
await page.screenshot("start.png");
await page.waitForSelector('#TWRIFrame');
const frameElement = await page.$('#TWRIFrame');
const frame = await frameElement.contentFrame();
await frame.click("a[container='CD']");
I am making a puppeteer program were it checks Google Docs
I want to be able to select all the text in the Google Docs and be able to make it into a string or a variable
I tried to Copy the text in the Google docs But didn't know where to go from there
await page2.keyboard.down('Control');
await page2.keyboard.press('KeyA');
await page2.keyboard.press('KeyC');
await page2.keyboard.up('Control');
I don't know where to go from here
try
await page.keyboard.down('ControlLeft')
await page.keyboard.press('KeyA')
await page.keyboard.press('KeyC');
await page.keyboard.up('ControlLeft')
I'm trying to automate certain tasks for work. We have a portal that requires you to sign in through Google. I've created a Puppeteer instance that navigates to the Google auth page, types in my email and password, then stores the cookies so I can navigate through and manipulate the portal.
This works perfectly on my local environment, but I've deployed it to Heroku and Google adds a sign in challenge. After entering the password, I'm given the 'Verify it's you' page that says 'This device isn't recognized' and asks me to complete 2-FA auth.
I know I can't turn off 2-FA, so what would be the best way to bypass this?
Alternatively, is there an easier way to log in to a website guarded by Google auth and store the session cookies?
Here's my puppeteer code, any help would be much appreciated:
async function getCookies() {
const browser = await puppeteer.launch({
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-gpu'
]
})
const page = await browser.newPage()
await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36')
await page.goto(process.env.URL)
await page.waitForSelector('#identifierId')
await page.type('#identifierId', process.env.EMAIL, { delay: 5 })
await page.click('#identifierNext')
await page.waitForSelector('#password input[type="password"]', { visible: true });
await page.type('#password input[type="password"]', process.env.PASS, { delay: 5 })
await page.click('#passwordNext')
await page.waitFor(3000)
const cookies = await page.cookies()
await browser.close()
return cookies
}
Not possible I am afraid and not the answer you want.
I know I can't turn off 2-FA, so what would be the best way to bypass
this?`
If it was possible to bypass then it kinda opens the door for hackers as Two-factor authentication works as an extra step in the process, a second security layer, that will reconfirm your identity. Its purpose is to make attackers' life harder and reduce fraud risks!
I would have added an Android app in the mix too. You can set up the 2FA with SMS codes and an Android app with SMS read permission can read the SMS and connect with a backend.
The backend can send push message, probably using Firebase Cloud Messaging to the local Node.js instance where the headless Chrome is running to input it in the 2FA screen.
I don't think there's any other way to do it. Although I would recommend not doing it, since it may open some backdoor for security issues.
I is actually possible using Twilio API within Puppeteer to programatically receive the SMS code. You will have to setup a special Google account for this to work with the Twilio number as mobile phone OR change your current Google account primary mobile number for the Twilio number, and use your regular number as a secondary contact in your Google account info.
My working solution (needs some refactoring)
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({
headless: false, // for debugging only
ignoreHTTPSErrors: true // This happens when you use a self signed certificate locally
})
const page = await browser.newPage()
await page.setViewport({ width: 1280, height: 800 })
await page.goto('https://myawesomesystem/loginFrm01')
const navigationPromise = page.waitForNavigation()
// Clicks on the login button
const googleLoginButtonSelector = 'body > section > ... > div'
await page.waitForSelector( googleLoginButtonSelector )
await page.click( googleLoginButtonSelector )
// wait for the google oauth page to open
const googleOAuthTarget = await browser.waitForTarget( target => {
// console.log( target.url() ); // debugging
return target.url().indexOf('https://accounts.google.com/signin/oauth/identifier') !== -1
})
const googleOAuthPage = await googleOAuthTarget.page()
await googleOAuthPage.waitForSelector('#identifierId')
await googleOAuthPage.type('#identifierId', CRED.user, { delay: 5 } )
await googleOAuthPage.click('#identifierNext')
await googleOAuthPage.waitForSelector('input[type="password"]', { visible: true })
await googleOAuthPage.type('input[type="password"]', CRED.pass )
await googleOAuthPage.waitForSelector('#passwordNext', { visible: true })
await googleOAuthPage.click('#passwordNext')
await navigationPromise
// HERE:
// the user has been authenticated
// or login window was closed
// or whatever else, please check
await browser.close()
})()
Hello,
I´m currently working on an Javascript based on Nodejs where I use Puppeteer as an help to scrape the web. As you can read in the title I´m trying to click on a dropdown menu Item, where the dropdown changes if you type in something diffrent. First here is my code:
// Navigate to the Homepage
await page.goto('https://www.futbin.com/');
await page.click('#player_search');
await page.keyboard.type(playerName);
await page.keyboard.press('ArrowDown');
await page.keyboard.press('ArrowDown');
// Create a screenshot
await page.screenshot({
path: 'screenshot.png'
});
So Basically i do the screenshot just for proving that the headless-browser does the right thing.
The website is futbin, if you want to see how their website works and take a look at the inspect, i think that could help.
But my real problem is that normally when you press enter you directly go to the player page (where i want to get). but after my script there comes always the error "no target". So the keyboard.press('Enter') don´t works. Also other Suggestions from SO didn´t work for me as the dropdown isn´t native and don´t hast counting indexes.
I would really appreciate some Suggestions !
at the end i wanted leave the html code from the first row of the dropdown as well, but i never worked so i would appreciate if you´d take a look at the website please !
You were very close, you dont need to use the arrows as you can find the item to click another way. The secret is to add a waitForSelector as the dropdown makes a call to a api endpoint. Also notice the waitForSelector on the final page to render before we take the screenshot.
Therefore just do this:-
const puppeteer = require('puppeteer');
async function run() {
const browser = await puppeteer.launch( {
headless: false
});
const page = await browser.newPage();
await page.goto('https://www.futbin.com/');
await page.type('#player_search', "Dave");
await page.waitForSelector("ul li a[data-id]");
await page.click("ul li a[data-id]");
await page.waitForSelector('#cal');
await page.screenshot( { path: "./dave.png"});
await browser.close();
};
run();
EDIT ADDTIONAL
To select any index use:-
let index = 3;
let selector = "ul li:nth-child(" + index +") a[data-id]"
await page.click(selector);
This goes to the third item in the dropdown. HTH