I'm getting started with pupeteer and node and using vscode in win 10. I'm trying to log into a site and scrape a table. So far I have:
(async () => {
const browser = await puppeteer.launch({
headless: false,
});
var page = await browser.newPage();
await page.goto('thesite.com/login/');
await page.click(USERNAME_SELECTOR);
await page.keyboard.type(CREDS.username);
await page.click(PASSWORD_SELECTOR);
await page.keyboard.type(CREDS.password);
await page.click(BUTTON_SELECTOR);
await page.waitForNavigation();
const TABLE_ROW_SELECTOR = '.gv-container.gv-container-133 > table > tbody';
await page.waitForSelector(TABLE_ROW_SELECTOR);
await page.waitForSelector(TABLE_ROW_SELECTOR);
await page.screenshot({ path: 'example.png' });
const data = await page.evaluate(SELECTOR => document.querySelectorAll(SELECTOR), TABLE_ROW_SELECTOR);
await browser.close();
})();
This is mostly working. however in my console I see a list of objects but as far as I can tell no values. Heres the fiest object:
0:Object {}
__proto__:Object {constructor: , __defineGetter__: , __defineSetter__: , …}
__defineGetter__:function __defineGetter__() { … }
__defineSetter__:function __defineSetter__() { … }
__lookupGetter__:function __lookupGetter__() { … }
__lookupSetter__:function __lookupSetter__() { … }
constructor:function Object() { … }
hasOwnProperty:function hasOwnProperty() { … }
No debug adapter, can not send 'variables'
isPrototypeOf:function isPrototypeOf() { … }
No debug adapter, can not send 'variables'
What does " No debug adapter, can not send 'variables'" mean?
edit:
I updated to the latest vscode and checked that all extensions were updated. Now when I run LAUNCH PROGRAM
E:\nodejs\node.exe --inspect-brk=27108 index.js
Debugger listening on ws://127.0.0.1:27108/e5928c71-370c- 4111-9ec3-77bb2cd85075
For help, see: https://nodejs.org/en/docs/inspector
(node:12844) ExperimentalWarning: The fs.promises API is experimental
warning.js:18
Array(25) [ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, …]
index.js:64
length:25
__proto__:Array(0) [, …]
concat:function concat() { … }
[[Scopes]]:Scopes[0]
arguments:TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them
Any idea what this means?
I had this issue when trying to use the integratedConsole rather than integratedTerminal or externalTerminal as part of my Node configuration within launch.json:
Setting it back to:
"console": "integratedTerminal"
Fixed it. Only took an hour to figure out. See docs for more information.
You can also try:
"outputCapture": "std"
in your launch.json
Here is reference on Github
The reason this happens is that the debugger stops after the code execution ends. Then there is no more debug adapter available to send the variables. What I did is add an extra line on the bottom of the code execution, and set a breakpoint on that. It isn't pretty, but it works.
I had the same problem, but that was me who caused this error ...
I have a conditional breakpoint defined in some scope of code, I have tried to use it in another conditional breakpoint but in a different scope.
The variable used on this condition was not found in the new scope. that's why the debugger can not start and gives us this error.
When I figured out, I used the defined variables on the scope to get the condition work properly.
Not a very big deal, but I hope this can help someone
I had a similar problem when the JavaScript script was erroring-out because of a missing async keyword on an await-using function during an initial evaluation step before normal execution and the vscode debugger was not catching the error in a meaningful error reporting context before the execution context exited.
Related
As we all know, we can easy to see the line of an output statement in the browser, just like follow picture
enter image description here
but in the nodejs env, how do I know what line is 'output statement' in.
I have this need because I want to know better during development where the information is coming from when the program fails. Of course, I could have each output statement carry a unique character, like console.log('1', '...'), console.log('2', '...') but that feels silly and unhackable to me.
I'll show you a simple piece of code as an illustration
try {
throw new Error('something error')
} catch (error) {
console.log(error.stack)
}
Run the above code I can see the output:
Error: something error
at file:///c:/Users/Linhieng/Desktop/tmp/a.js:2:9
at ModuleJob.run (node:internal/modules/esm/module_job:198:25)
at async Promise.all (index 0)
at async ESMLoader.import (node:internal/modules/esm/loader:385:24)
at async loadESM (node:internal/process/esm_loader:88:5)
at async handleMainPromise (node:internal/modules/run_main:61:12)
the above output tell us what line is the error in, but I want to know the line of console.log.
You can monkeypatch console.log with code that parses the stack string, since this is Node.js and the format only varies across releases (unlike on browsers, where it can vary by browser as well):
const realLog = console.log;
console.log = (...msgs) => {
try {
throw new Error("something error");
} catch (error) {
const lines = error.stack.split(/(?:\r\n|\r|\n)+/);
msgs.push(`[${lines[2].trim()}]`);
}
realLog(...msgs);
};
Then for instance, this:
function example() {
console.log("Hi there");
}
example();
shows
Hi there [at example (file:///____/temp.js:13:13)]
At present, the "parsing" is really easy — just split on newlines and take the third element of the array. But you might want to extract more information, or the format could change in future versions to require more complicated "parsing."
I thing woth simple logging the only solution here is a manually passing the according line number. Something like that:
console.log(error, LINE_NUMBER);
Another possible solution probably could be when you use some kind of a external configurable NodeJS debugger software. There probably will be provided a functionality related you your needs.
I am using axios (npm module) to send a request to my server. This routine is part of a separate Node.js process I start, which should automatically close after its work is done.
However, I seem to be misinterpreting something about the promise. Consider the function I use to log these requests:
async log(request) {
return await axios.post('example.com', {
screenshot: await page.screenshot({type: 'png', encoding: 'base64'}),
});
}
I push all these calls to an array, so I know when to close the process:
this.awaitPromises.push(this.log(request));
later on in the code I call:
await Promise.all(this.awaitPromises);
As you can see there is a nested await page.screenshot(...) inside the parameters object of the axios call which creates a screenshot of the browser (puppeteer). This log(request) code however gives me an error indicating that the browser has closed unexpectedly: Protocol error (Page.captureScreenshot): Target closed.. This means that the await Promise.all() has resolved to soon because after that I close the browser.
The code does work when I add an extra .then() to the log function like so:
async log(request) {
return await axios.post('example.com', {
screenshot: await page.screenshot({type: 'png', encoding: 'base64'}),
}).then(() => {
// Do nothing.
});
}
The code now works as intended and the browser is only closed after the log function has (completely) been completed.
Anyone knows what the difference is here and why the last example works? It must have something to do with the inner await page.screenshot right?
I'm getting started with pupeteer and node and using vscode in win 10. I'm trying to log into a site and scrape a table. So far I have:
(async () => {
const browser = await puppeteer.launch({
headless: false,
});
var page = await browser.newPage();
await page.goto('thesite.com/login/');
await page.click(USERNAME_SELECTOR);
await page.keyboard.type(CREDS.username);
await page.click(PASSWORD_SELECTOR);
await page.keyboard.type(CREDS.password);
await page.click(BUTTON_SELECTOR);
await page.waitForNavigation();
const TABLE_ROW_SELECTOR = '.gv-container.gv-container-133 > table > tbody';
await page.waitForSelector(TABLE_ROW_SELECTOR);
await page.waitForSelector(TABLE_ROW_SELECTOR);
await page.screenshot({ path: 'example.png' });
const data = await page.evaluate(SELECTOR => document.querySelectorAll(SELECTOR), TABLE_ROW_SELECTOR);
await browser.close();
})();
This is mostly working. however in my console I see a list of objects but as far as I can tell no values. Heres the fiest object:
0:Object {}
__proto__:Object {constructor: , __defineGetter__: , __defineSetter__: , …}
__defineGetter__:function __defineGetter__() { … }
__defineSetter__:function __defineSetter__() { … }
__lookupGetter__:function __lookupGetter__() { … }
__lookupSetter__:function __lookupSetter__() { … }
constructor:function Object() { … }
hasOwnProperty:function hasOwnProperty() { … }
No debug adapter, can not send 'variables'
isPrototypeOf:function isPrototypeOf() { … }
No debug adapter, can not send 'variables'
What does " No debug adapter, can not send 'variables'" mean?
edit:
I updated to the latest vscode and checked that all extensions were updated. Now when I run LAUNCH PROGRAM
E:\nodejs\node.exe --inspect-brk=27108 index.js
Debugger listening on ws://127.0.0.1:27108/e5928c71-370c- 4111-9ec3-77bb2cd85075
For help, see: https://nodejs.org/en/docs/inspector
(node:12844) ExperimentalWarning: The fs.promises API is experimental
warning.js:18
Array(25) [ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, ElementHandle, …]
index.js:64
length:25
__proto__:Array(0) [, …]
concat:function concat() { … }
[[Scopes]]:Scopes[0]
arguments:TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them
Any idea what this means?
I had this issue when trying to use the integratedConsole rather than integratedTerminal or externalTerminal as part of my Node configuration within launch.json:
Setting it back to:
"console": "integratedTerminal"
Fixed it. Only took an hour to figure out. See docs for more information.
You can also try:
"outputCapture": "std"
in your launch.json
Here is reference on Github
The reason this happens is that the debugger stops after the code execution ends. Then there is no more debug adapter available to send the variables. What I did is add an extra line on the bottom of the code execution, and set a breakpoint on that. It isn't pretty, but it works.
I had the same problem, but that was me who caused this error ...
I have a conditional breakpoint defined in some scope of code, I have tried to use it in another conditional breakpoint but in a different scope.
The variable used on this condition was not found in the new scope. that's why the debugger can not start and gives us this error.
When I figured out, I used the defined variables on the scope to get the condition work properly.
Not a very big deal, but I hope this can help someone
I had a similar problem when the JavaScript script was erroring-out because of a missing async keyword on an await-using function during an initial evaluation step before normal execution and the vscode debugger was not catching the error in a meaningful error reporting context before the execution context exited.
I am doing processing using Puppeteer and I am closing the browser like using browser.close() for example something like this
const browser = await puppeteer.launch({
headless: true,
executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
});
try {
// more processing
browser.close();
// more processing
} catch(err) {
console.log(err);
browser.close();
}
I am using browser.close(); in catch block, but sometimes the browser.close() is already executed in try block when the exception occurs.
I want to know if there is a way to check if browser.open? and issue browser.close() only in that case.
Any help in this would be really great. Thanks.
Take a look about try...catch
You should add finally block and add there browser close :)
The finally-block contains statements to execute after the try-block and catch-block(s) execute, but before the statements following the try...catch...finally-block. Note that the finally-block executes regardless of whether an exception is thrown. Also, if an exception is thrown, the statements in the finally-block execute even if no catch-block handles the exception.
Edit 1
After understanding that issue is detecting if browser is closed I edit and input solution for that problem puppeteer : how check if browser is still open and working
This is an automation framework using page object model. I use the async/await approach over promises. I am using TS, compiling it to JS (protractor) and then executing the scripts.
Page Object:
async addProjectDetails(): Promise<void> {
expect(await this.currentStep.getText()).toBe("Records"); //There is no element like this, which I know.
await this.projectTitle.sendKeys("Project Feb 1");
await this.projectDescription.sendKeys("Project Description");
}
Spec:
it('should create a project successfully', async () => {
try {
await dashboard.createNewProject();
await dashboard.addProjectDetails();
}
The page loads and the element doesn't exist. I get the error:
NoSuchElementError: No element found using locator: By(xpath, //custom-expandable-title[#class='not-completed active']//span[#class='child-title'])
However instead of failing the spec passes. Shouldn't it fail. Isn't this a false positive?
Errors raised in try wouldn't affect final result, they will stop execution of try block and start execution of catch block. If you want test to behave like this you have to move call of addProjectDetails away of try block.