Using Puppeteer&Python on Node.JS to solve captchas - javascript

I'm using puppeteer library to access some sites and leach data from their HTML files. for that manner, I've got a Python script that should help me solve each captcha if there is.
now, to run the Python script, I'm using python-shell from npm, and committing this:
let options = {
mode: 'text',
args: [captchas[0].sitekey],
scriptPath: path.resolve('bypass'),
pythonPath: '/usr/bin/python',
}
console.log(`options.scriptPath`, options.scriptPath)
PythonShell.run(
`bypass/bypass.py`,
options,
(err, [, captchaKey]) => {
if (err) throw err
let solutions = [
{
_vendor: captchas[0]._vendor,
id: captchas[0].id,
text: captchaKey,
hasSolution: true,
},
]
resolve({ solutions })
}
)
I've got this error -
Solving captcha...
options.scriptPath MY_PATH/backend/bypass
file:///MY_PATH/backend/bypass/captchaBypasser.js:18
(err, [, captchaKey]) => {
^
TypeError: object null is not iterable (cannot read property Symbol(Symbol.iterator))
at file:///MY_PATH/backend/bypass/captchaBypasser.js:18
at PythonShell._endCallback (/MY_PATH/backend/node_modules/python-shell/index.js:254:20)
at terminateIfNeeded (/MY_PATH/backend/node_modules/python-shell/index.js:209:39)
at ChildProcess.<anonymous> (/MY_PATH/backend/node_modules/python-shell/index.js:182:13)
I've tried also to use spawn of child_process, like this:
const location = path.resolve('bypass/bypass.py')
console.log(`python ${location} ${captchas[0].sitekey}`)
const run = spawn('python', [location, captchas[0].sitekey])
run.stdout.on('data', ([, captchaKey]) => {
let solutions = [
{
_vendor: captchas[0]._vendor,
id: captchas[0].id,
text: captchaKey,
hasSolution: true,
},
]
resolve({ solutions })
})
and it appears to just crush without even an outcome.
So I've tried to try the actual command line:
python /MY_PATH/backend/bypass/bypass.py ae7317...TOKEN...d0dab8b75
BY THE WAY - I've TRYED ALSO python3 WITH VERSION 3.10.0
And now I've got this error:
from hcapbypass import bypass
File "/MY_PATH/backend/bypass/hcapbypass.py", line 35
def N_Data(req) -> str:
^
So I tried to use the latest version of Python, 3.10.0, that came out like a few weeks ago, and that's the outcome:
File "/MY_PATH/backend/bypass/bypass.py", line 4, in <module>
captcha_solved = bypass(sys.argv[1], 'google.com', True)
TypeError: bypass() missing 1 required positional argument: 's'
For your full understanding, here is the bypass and the solver codes:
bypass.py
from hcapbypass import bypass
import sys
captcha_solved = bypass(sys.argv[1], 'google.com', True)
print(captcha_solved)
hcapbypass
Link to github
Does anyone know what to do?
Thanks a lot!!!

OK, ive got the solution:
inside the hcapbypass.py file, the httpx module isn't presented, so I've created an environment with bin & lib folders to install it locally.
In addition, few functions got an argument that doesn't exist.
And of course, destructuring an array that is null / undefined - throws an error!

Related

Replacing variable with #rollup/plugin-replace throws error

Problem
I am trying to inject / replace environment variables with #rollup/plugin-replace. Unfortunately, I get this error:
TypeError: can't convert undefined to object
Code
// rollup.config.js
import replace from "#rollup/plugin-replace";
import { config } from "dotenv";
config();
export default {
// ...
plugins: [
replace({
values: { YOUTUBE_API: JSON.stringify(process.env.YOUTUBE_API) },
preventAssignment: true,
}),
// ...
}
And I call it like this:
onMount(() => {
(async function getPopular() {
videos = await axios.get("https://www.googleapis.com/youtube/v3/videos", {
part: "id, snippet, suggestions",
chart: "mostPopular",
key: YOUTUBE_API,
});
})();
});
What I tried
I logged out the variable and so can confirm that it exists. Also, if I remove the stringify function, I get another error:
ReferenceError: blablabblub is not defined
I have done this successfully in other projects. What the heck is wrong here?
So after some digging around with the same issue, I found it was related to object assignment. For example:
export default {
// ...
plugins: [
replace({
values: {
env: {
API_URL: process.env.API_URL,
API_VERSION: process.env.API_VERSION,
}
},
preventAssignment: true,
}),
// ...
}
// in some JS or Svelte file
const config = {
host: env.API_URL,
version: env.API_VERSION
}
// The above will result in a reference error of 'env' not being defined.
// in the same JS or Svelte file..
const envVars = env;
const config = {
host: envVars.API_URL,
version: envVars.API_VERSION
}
// this works just fine!
I haven't had anymore time to investigate, but my gut feeling is that rollup won't replace variable names when they are nested inside an object assignment. It might be nice for an optional flag to allow this, but it might also get very messy hence why they didn't do it.
I hope this helps if it's still an issue for you.

React - Error when importing node package (Object prototype may only be an Object or null)

I am trying to implement this library with my React applicaiton however, I get an error when I import the package.
Error - TypeError: Object prototype may only be an Object or null: undefined
Package - https://www.npmjs.com/package/dukascopy-node
I tried importing it both ways, using require and using import from however it gives the same output
Function
import { getHistoricRates } from "dukascopy-node";
export const getHistoricalData = async (
instrument = "btcusd",
from = new Date("2018-01-01"),
to = new Date("2019-01-01"),
timeframe = "d1"
) => {
try {
const data = await getHistoricRates({
instrument: instrument,
dates: {
from: from,
to: to,
},
timeframe: timeframe,
format: "json",
});
console.log(data);
} catch (error) {
console.log("error", error);
}
};
As the import statement looks OK, there must be something wrong with your packages. I would try to delete the node_modules folder and reinstall all packages.
This is my go-to solution in such strange, unexplainable cases with imports.

Realm.objects() in react-native app return empty objects

I'm creating a db in realm, but when i want to consult the data with Realm.objects() return empty objects, like this: {"0":{},"1":{},"2":{}...etc}, my code is the next:
import Realm from "realm";
const nombreEsquema = 'Imagen12';
class Imagen {}
let esquema = Imagen.schema = {
name: nombreEsquema,
properties: {
nombre: {
type: 'string'
},
uri: {
type: 'string'
},
url: {
type: 'string'
},
}
};
let dbRealm = Realm.open({
schema: [esquema]
});
functionRealm() {
dbRealm.then(realm => {
realm.write(() => {
realm.create(nombreEsquema, {
nombre: 'David',
url: 'My URL',
uri: 'My URI'
});
});
let images = realm.objects(nombreEsquema);
console.log("------------------------------");
for (let i of images) {
console.log(i);
}
});
}
i read the realm's documentation but i don't see anything with that problem, my realm version is 5.0.2, and i don't know what i'm doing bad, i hope you can help me with my problem.
So I had the same issue as well. I was able to resolve it by following these steps:
Upgrade the realm package to 5.0.3 yarn upgrade realm
Run npx react-native link realm (Realm does not fully support auto-linking yet)
Verify that the linking happened properly by following the instructions at the top of: https://realm.io/docs/javascript/latest/
Make sure to add this line --> import io.realm.react.RealmReactPackage; to the top of your MainApplication.java in android/app/src/main/java/com/<your-app-name>/MainApplication.java
<------------------------------------- || **Optional ** || ------------------------------------->
**You shouldn't have to do this, but just incase...
Check your package list in android/app/src/main/java/com/<your-app-name>/MainApplication.java to ensure it is configured as followed:
#Override
protected List<ReactPackage> getPackages() {
#SuppressWarnings("UnnecessaryLocalVariable")
List<ReactPackage> packages = new PackageList(this).getPackages();
// Packages that cannot be autolinked yet can be added manually here, for example:
// packages.add(new MyReactNativePackage());
+ packages.add(new RealmReactPackage()); // <-- This line adds in the realm package.
packages.add(new RNSensitiveInfoPackage());
return packages;
}
cd into /android directory and clean by running ./gradlew clean
Re-compile and run again from your project's root directory npx react-native run-android
** If you get an error that looks something like this:
Native module realm tried to override realm for module name realm. If this was your intention, set canOverrideExistingModule=true
Open <your-app-name>/node_modules/realm/android/src/main/java/io/realm/react/RealmReactModule.java
Add the following lines inside the RealmReactModule class:
#Override
public boolean canOverrideExistingModule() {
return true;
}
Hope this helps! 🙂
Indeed you can just upgrade realm to 5.0.3 because there is a bug on 5.0.2
See on the release log: https://github.com/realm/realm-js/releases/tag/v5.0.3

Electron-dl Cannot read property 'then' of undefined

I am using electron-dl to download a file but I cant seem to make it work.
I did almost exactly same as defined in docs but i dont know what is wrong.
Error:
I get the following error on main process
TypeError: Cannot read property 'then' of undefined
at EventEmitter.ipcMain.on ( \electron-dl-test\main.js:24:7)
How to reproduce (setup):
use repo: https://github.com/mafar/electron-dl-test
npm install and then npm start
main.js:
ipcMain.on('download', (ev, args) => {
download(BrowserWindow.getFocusedWindow(), args.url, args.properties)
.then(dl => console.log(dl.getSavePath()))
.catch(console.error);
})
index.html as renderer:
document.getElementById("download-file").onclick = function () {
//
var ipcRenderer = require('electron').ipcRenderer;
ipcRenderer.send('download', {
url: 'https://textfiles.com/100/ad.txt',
properties: {
saveAs: true,
directory: 'C:\\'
}
});
//
};
Preview:
The download function is part of an object exported by electron-dl. You need to destructure (ES6) or reference it directly (ES5) when requiring the module:
const { download } = require("electron-dl") // ES6
or
var download = require("electron-dl").download // ES5

How to get Electron + rxdb to work?

I want to learn and develop a desktop app by using electron + rxdb.
My file structure:
main.js (the main process of electron)
/js-server/db.js (all about rxdb database, include creation)
/js-client/ui.js (renderer process of electron)
index.html (html home page)
main.js code:
const electron = require('electron')
const dbjs = require('./js-server/db.js')
const {ipcMain} = require('electron')
ipcMain.on('search-person', (event, userInput) => {
event.returnValue = dbjs.searchPerson(userInput);
})
db.js code:
var rxdb = require('rxdb');
var rxjs = require('rxjs');
rxdb.plugin(require('pouchdb-adapter-idb'));
const personSchema = {
title: 'person schema',
description: 'describes a single person',
version: 0,
type: 'object',
properties: {
Name: {type: 'string',primary: true},
Age: {type: 'string'},
},
required: ['Age']
};
var pdb;
rxdb.create({
name: 'persondb',
password: '123456789',
adapter: 'idb',
multiInstance: false
}).then(function(db) {
pdb = db;
return pdb.collection({name: 'persons', schema: personSchema})
});
function searchPerson(userInput) {
pdb.persons.findOne().where('Name').eq(userInput)
.exec().then(function(doc){return doc.Age});
}
module.exports = {
searchPerson: searchPerson
}
ui.js code:
const {ipcRenderer} = require('electron');
function getFormValue() {
let userInput = document.getElementById('searchbox').value;
displayResults(ipcRenderer.sendSync("search-person",userInput));
document.getElementById('searchbox').value = "";
}
Whenever I run this app, I got these errors:
(node:6084) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: RxError:
RxDatabase.create(): Adapter not added. (I am sure I've installed the pouched-adapter-idb module successfully)
Type error, cannot read property "persons" of undefined. (this error pops out when I search and hit enter to the form in index.html)
I am new to programming, especially js, I've been stuck on these errors for a week, just can't get it to work. Any help? Thanks.
The problem is that this line is in main.js:
const dbjs = require('./js-server/db.js')
Why? Because you're requiring RxDB inside the main process and using the IndexedDB adapter. IndexedDB is a browser API and thus can only be used in a rendering process. In Electron, the main process is a pure Node/Electron environment with no access to the Chromium API's.
Option #1
If you want to keep your database in a separate thread then consider spawning a new hidden browser window:
import {BrowserWindow} from 'electron'
const dbWindow = new BrowserWindow({..., show: false})
And then use IPC to communicate between the two windows similarly to how you have already done.
Option #2
Use a levelDB adapter that only requires NodeJS API's so you can keep your database in the main process.

Categories