How to get request initiator with puppeteer? - javascript

I am trying to log all network requests in a page. My goal is to later filter for particular requests and check their initiator. The code below runs fine if I comment line 12 const request_initiator = interceptedRequest.initiator();. But the initiator is something really need. According to puppeteer's docs I am calling it correctly.
Here is my code:
const puppeteer = require('puppeteer');
(async() => {
const results = [];
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on('request', interceptedRequest => {
const request_url = interceptedRequest.url();
const request_method = interceptedRequest.method();
const request_payload = interceptedRequest.postData();
const request_initiator = interceptedRequest.initiator();
results.push({
request_url,
request_method,
request_payload,
request_initiator
})
interceptedRequest.continue();
});
await page.goto('https://testurl.com');
await browser.close();
return results
})().then(function(result) {
console.log(result)
});
Here is the error I am getting:
UnhandledPromiseRejectionWarning: TypeError: interceptedRequest.initiator is not a function
at /Users/test/Desktop/code/tagv2/tcv2/index.js:12:54
at /Users/test/Desktop/code/tagv2/tcv2/node_modules/peteer/lib/cjs/vendor/mitt/src/index.js:51:62
at Array.map (<anonymous>)
at Object.emit (/Users/test/Desktop/code/tagv2/tcv2/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
at Page.emit (/Users/test/Desktop/code/tagv2/tcv2/node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
at /Users/test/Desktop/code/tagv2/tcv2/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:143:100
at /Users/test/Desktop/code/tagv2/tcv2/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62
at Array.map (<anonymous>)
at Object.emit (/Users/test/Desktop/code/tagv2/tcv2/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
at NetworkManager.emit (/Users/test/Desktop/code/tagv2/tcv2/node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
(node:22974) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:22974) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Related

How do I copy to clipboard using clipboardy?

I am trying to use clipboard in order to copy 'longlat' to the clipboard by passing it through the function 'copyData()'. However, it keeps giving me errors.
This is the error (1. Clipboardy.writeSync not a function and 2.UnhandledPromiseRejectionWarning) :
(node:16596) UnhandledPromiseRejectionWarning: TypeError: clipboardy.writeSync is not a function
at copyData (C:\Users\Nicky\index.js:46:14)
at C:\Users\Nicky\index.js:32:13
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:16596) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:16596) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
This is my code
const clipboardy = (...args) => import('clipboardy').then(({default: clipboardy}) => clipboardy(...args));
const puppeteer = require("puppeteer");
const url = "https://www.google.com/";
async function StartScraping() {
await puppeteer
.launch({
headless: false,
})
.then(async (browser) => {
const page = await browser.newPage();
await page.setViewport({
width: 2000,
height: 800,
});
page.on("response", async (response) => {
if (response.url().includes("General")) {
const location = await response.text()
let intlong = location.indexOf("Example:")
const longlat = location.substring(intlong, intlong + 46)
console.log(longlat);
copyData(longlat)
}
});
await page.goto(url, {
waitUntil: "load",
timeout: 0,
});
});
}
function copyData(data1) {
clipboardy.writeSync(data1)
}
StartScraping();
cy.window().its('navigator.clipboard')
.invoke('readText').then((data) => {
console.log(data);
})
Should do the trick. If you are using cypress on the browser

fingerprintjs fingerprint.get is not a function

currently I'm struggling with fingerprint in my previous project i solved this in this way:
const test = async (req, res, next) => {
// other code
const FingerprintJS = require('#fingerprintjs/fingerprintjs');
const store = require('store')
FingerprintJS.get({
preprocessor: function(key, value) {
return value
}
},function(components){
let values = components.map(function (component) { return component.value });
let device_id = FingerprintJS.x64hash128(values.join(''), 31);
store.set('print', { device: device_id })
});
await new Promise(resolve => setTimeout(resolve, 1000));
// other code
};
But now I cant set it up to work.. I'm getting error in console.log :
(node:4940) UnhandledPromiseRejectionWarning: TypeError: FingerprintJS.get is not a function
at login (C:\Users\PC06\Desktop\project\backend\controllers\clients-controllers.js:53:17)
(node:4940) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:4940) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a
non-zero exit code.
Any help for this?

ipfs.cat(cid) in node js gives error "Error: Invalid version, must be a number equal to 1 or 0 at Function.validateCID"

I am using the ipfs-http-client module.
This is my script
async function main() {
const ipfs_client = require('ipfs-http-client');
const client = ipfs_client({recursive: false});
async function addFile(file_data) {
const cid = function (data) {
return client.add(data).then(cid =>{return cid;}).catch(err=>{
console.log(err)})
}
const real_cid = cid(file_data).then(result=>{return result}).catch(err=>{console.log(err)});
return real_cid;
}
const sendFileToNet = async (file) => {
return await addFile(file);
}
async function getFile(cid){
//error occurs here
for await (const chunk of client.cat(cid)) {
console.info(chunk)
}
return content;
}
const getFileFromNet = async()=>{
return await getFile();
}
getFileFromNet("QmU77sHBUuCT12324e9Re59bNHekpKg1PdCpiVAj3MFLiH").then(r => console.log(r));
module.exports = {sendFileToNet, getFileFromNet
};
}
main();
(node:22889) UnhandledPromiseRejectionWarning: Error: Invalid version, must be a number equal to 1 or 0
at Function.validateCID (/home/chipego/Documents/MedChain/node_modules/ipfs-http-client/node_modules/cids/src/index.js:346:13)
at new CID (/home/chipego/Documents/MedChain/node_modules/ipfs-http-client/node_modules/cids/src/index.js:174:9)
at Object.cat (/home/chipego/Documents/MedChain/node_modules/ipfs-http-client/src/cat.js:16:48)
at cat.next ()
at getFile (/home/chipego/Documents/MedChain/logic/ipfs_submit_controller.js:23:26)
at getFileFromNet (/home/chipego/Documents/MedChain/logic/ipfs_submit_controller.js:30:22)
at main (/home/chipego/Documents/MedChain/logic/ipfs_submit_controller.js:32:5)
at Object. (/home/chipego/Documents/MedChain/logic/ipfs_submit_controller.js:39:1)
at Module._compile (internal/modules/cjs/loader.js:1063:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
(Use node --trace-warnings ... to show where the warning was created)
(node:22889) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:22889) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with
Your getFile function takes an argument. You are invoking it in getFileFromNet but not passing the argument in.

How would I capture screenshots of an array of URLs using Puppeteer?

I set up a loop inside of an async function to iterate over a list of URLs. The data is coming from an .xls file converted to .json. My goal is to capture a screenshot of each URL in the array but I keep getting UnhandledPromiseRejectionWarning. Any ideas how how i might achieve this? Any help is appreciated!
The Code:
const puppeteer = require("puppeteer");
const excel = require("./excel");
const data = excel.data;
async function run(arr) {
for (let i = 0; i < data.length; i++) {
const url = data[i]["Seller"];
const sku = data[i]["Seller Name"];
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({
width: 1000,
height: 840,
deviceScaleFactor: 1
});
await page.goto(url, { waitUntil: "load" });
await page.screenshot({ path: `screenshots/${sku}.jpg` });
await browser.close();
}
}
run(data)
full error with try ... catch
(node:24804) UnhandledPromiseRejectionWarning: ReferenceError: err is not defined
at run (C:\Users\ray\OneDrive\Desktop\nodeScrappingProject\app\index.js:24:19)
(node:24804) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which
was not handled with .catch(). (rejection id: 1)
(node:24804) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Full error message without try .. catch
(node:34456) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open 'C:\Users\ray\OneDrive\Desktop\nodeScrappingProject\app\screenshots\fileName.jpg'
-- ASYNC --
at Page.<anonymous> (C:\Users\ray\OneDrive\Desktop\nodeScrappingProject\node_modules\puppeteer\lib\helper.js:111:15)
at run (C:\Users\ray\OneDrive\Desktop\nodeScrappingProject\app\index.js:20:16)
at processTicksAndRejections (internal/process/task_queues.js:85:5)
(node:34456) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which
was not handled with .catch(). (rejection id: 1)
(node:34456) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Your code seems correct to me. Did you try looking at what is coming out in data Seller and SellerName? I am guessing that you have issues in data. As you are not asserting anything you need not close the browser each time. Checkout the following snippet.
const puppeteer = require('puppeteer');
(async () => {
const data = [{"sellerUrl" : "https://www.amazon.com/",
"sellerName" : "amazon"
},
{"sellerUrl" : "https://www.ebay.com/",
"sellerName" : "ebay"
},
];
const browser = await puppeteer.launch();
const page = await browser.newPage();
for(const x of data){
await page.goto(x.sellerUrl, { waitUntill: 'networkidle2'});
await page.screenshot({path: `${x.sellerName}.jpg`});
}
await browser.close();
})();

How to stop Unhandled Promise error when running node.js file in BigQuery?

I'm running the following node.js code in BigQuery but I'm getting an unhandled promise error:
I'm not too familiar with node.js - do I have to add a .catch() in there at the end?
const BigQuery = require('#google-cloud/bigquery');
const bigquery = new BigQuery({
projectId: 'xxxxx',
keyFilename: 'key.json'
});
const query = `SELECT total_amount, pickup_datetime, trip_distance
FROM \`nyc-tlc.yellow.trips\`
ORDER BY total_amount DESC
LIMIT 1;`
bigquery.createQueryJob(query).then((data) => {
const job = data[0];
return job.getQueryResults({timeoutMs: 10000});
}).then((data) => {
const rows = data[0];
console.log(rows[0]);
});
Error:
(node:1035) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either
by throwing inside of an async function without a catch block, or by rejecting a promise which was not
handled with .catch(). (rejection id: 1)
(node:1035) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, p
romise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Thanks
I'm not familiar with #google-cloud/bigquery anyway you should do :
.then((data) => {
const rows = data[0];
console.log(rows[0]);
}).catch(e=>{
//handle exception
console.log(e)
})

Categories