Problem Statement
I wrote a JavaScript code to pull data from Uniswap V3 SDK (https://www.npmjs.com/package/#uniswap/v3-sdk) and I would like to open-source this code on GitHub because currently this isn't possible (for specific use cases). I am not an amazing system designer/programmer so please bare with me (I can only code in Python and barely understand JavaScript).
Essentially, I have a JavaScript code that pulls data and I wanted to wrap it in a Python code, so I can then take the data and do other things with the data (create alerts, flask app, etc.)
I am using Ubuntu, JS6, npm v8.19.2 and Node v16.18.1.
In my research, I found two packages that could help enable me wrapping this JS code in Python:
py2js - https://github.com/PiotrDabkowski/Js2Py
stpyv8 - https://github.com/cloudflare/stpyv8
Coding Attempt
py2js attempt
This package only works for JS5 and below, not JS6. Here is a quote from the GitHub that helps explain the workaround for this issue (which uses babel) https://github.com/PiotrDabkowski/Js2Py:
JavaScript 6 support was achieved by using Js2Py to translate javascript library called Babel. Babel translates JS 6 to JS 5 and afterwards Js2Py translates JS 5 to Python.
While trying this solution out, I am getting the following error:
/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/es6/__init__.py:10: UserWarning:
Importing babel.py for the first time - this can take some time.
Please note that currently Javascript 6 in Js2Py is unstable and slow. Use only for tiny scripts!
warnings.warn(
/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/base.py:2854: FutureWarning: Possible nested set at position 5
self.pat = re.compile(
Initialised babel!
Traceback (most recent call last):
File "/home/bobby/uni_balances/main.py", line 104, in <module>
result = js2py.eval_js6(js) # executing JavaScript and converting the result to python string
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/evaljs.py", line 120, in eval_js6
return eval_js(js6_to_js5(js))
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/evaljs.py", line 115, in eval_js
return e.eval(js)
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/evaljs.py", line 204, in eval
self.execute(code, use_compilation_plan=use_compilation_plan)
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/evaljs.py", line 199, in execute
exec (compiled, self._context)
File "<EvalJS snippet>", line 4, in <module>
File "<EvalJS snippet>", line 3, in PyJs_LONG_0_
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/base.py", line 949, in __call__
return self.call(self.GlobalObject, args)
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/base.py", line 1464, in call
return Js(self.code(*args))
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/host/jseval.py", line 17, in Eval
py_code = translate_js(code.to_string().value, '')
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/translators/translator.py", line 70, in translate_js
parsed = parse_fn(js)
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/js2py/translators/translator.py", line 62, in pyjsparser_parse_fn
return parser.parse(code)
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 3008, in parse
program = self.parseProgram()
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 2974, in parseProgram
body = self.parseScriptBody()
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 2963, in parseScriptBody
statement = self.parseStatementListItem()
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 2110, in parseStatementListItem
return self.parseStatement()
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 2718, in parseStatement
self.consumeSemicolon()
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 1120, in consumeSemicolon
self.throwUnexpectedToken(self.lookahead)
File "/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pyjsparser/parser.py", line 1046, in throwUnexpectedToken
raise self.unexpectedTokenError(token, message)
js2py.internals.simplex.JsException: SyntaxError: Line 34: Unexpected token function
What have I done to try to fix this?
I have updated node and even downgraded to version 5 as some suggested, but nothing worked.
JavaScript Code
Here is the JavaScript code (note that you would need an Alchemy API ID to run this script):
import { JSBI } from "#uniswap/sdk";
import { ethers } from 'ethers';
import * as fs from 'fs';
// ERC20 json abi file
let ERC20Abi = fs.readFileSync('Erc20.json');
const ERC20 = JSON.parse(ERC20Abi);
// V3 pool abi json file
let pool = fs.readFileSync('V3PairAbi.json');
const IUniswapV3PoolABI = JSON.parse(pool);
// V3 factory abi json
let facto = fs.readFileSync('V3factory.json');
const IUniswapV3FactoryABI = JSON.parse(facto);
let NFT = fs.readFileSync('UniV3NFT.json');
const IUniswapV3NFTmanagerABI = JSON.parse(NFT);
const provider = new ethers.providers.JsonRpcProvider(ALCHEMY_API_ID)
// V3 standard addresses (different for celo)
const factory = "0x1F98431c8aD98523631AE4a59f267346ea31F984";
const NFTmanager = "0xC36442b4a4522E871399CD717aBDD847Ab11FE88";
async function getData(tokenID){
let FactoryContract = new ethers.Contract(factory, IUniswapV3FactoryABI, provider);
let NFTContract = new ethers.Contract(NFTmanager, IUniswapV3NFTmanagerABI, provider);
let position = await NFTContract.positions(tokenID);
let token0contract = new ethers.Contract(position.token0, ERC20, provider);
let token1contract = new ethers.Contract(position.token1, ERC20, provider);
let token0Decimal = await token0contract.decimals();
let token1Decimal = await token1contract.decimals();
let token0sym = await token0contract.symbol();
let token1sym = await token1contract.symbol();
let V3pool = await FactoryContract.getPool(position.token0, position.token1, position.fee);
let poolContract = new ethers.Contract(V3pool, IUniswapV3PoolABI, provider);
let slot0 = await poolContract.slot0();
let pairName = token0sym +"/"+ token1sym;
let dict = {"SqrtX96" : slot0.sqrtPriceX96.toString(), "Pair": pairName, "T0d": token0Decimal, "T1d": token1Decimal, "tickLow": position.tickLower, "tickHigh": position.tickUpper, "liquidity": position.liquidity.toString()}
return dict
}
const Q96 = JSBI.exponentiate(JSBI.BigInt(2), JSBI.BigInt(96));
const MIN_TICK = -887272;
const MAX_TICK = 887272;
function getTickAtSqrtRatio(sqrtPriceX96){
let tick = Math.floor(Math.log((sqrtPriceX96/Q96)**2)/Math.log(1.0001));
return tick;
}
async function getTokenAmounts(liquidity,sqrtPriceX96,tickLow,tickHigh,token0Decimal,token1Decimal){
let sqrtRatioA = Math.sqrt(1.0001**tickLow).toFixed(18);
let sqrtRatioB = Math.sqrt(1.0001**tickHigh).toFixed(18);
let currentTick = getTickAtSqrtRatio(sqrtPriceX96);
let sqrtPrice = sqrtPriceX96 / Q96;
let amount0wei = 0;
let amount1wei = 0;
if(currentTick <= tickLow){
amount0wei = Math.floor(liquidity*((sqrtRatioB-sqrtRatioA)/(sqrtRatioA*sqrtRatioB)));
}
if(currentTick > tickHigh){
amount1wei = Math.floor(liquidity*(sqrtRatioB-sqrtRatioA));
}
if(currentTick >= tickLow && currentTick < tickHigh){
amount0wei = Math.floor(liquidity*((sqrtRatioB-sqrtPrice)/(sqrtPrice*sqrtRatioB)));
amount1wei = Math.floor(liquidity*(sqrtPrice-sqrtRatioA));
}
let amount0Human = (amount0wei/(10**token0Decimal)).toFixed(token0Decimal);
let amount1Human = (amount1wei/(10**token1Decimal)).toFixed(token1Decimal);
console.log("Amount Token0 wei: "+amount0wei);
console.log("Amount Token1 wei: "+amount1wei);
console.log("Amount Token0 : "+amount0Human);
console.log("Amount Token1 : "+amount1Human);
return [amount0wei, amount1wei]
}
async function start(positionID){
let data = await getData(positionID);
let tokens = await getTokenAmounts(data.liquidity, data.SqrtX96, data.tickLow, data.tickHigh, data.T0d, data.T1d);
}
start(273381)
// Also it can be used without the position data if you pull the data it will work for any range
getTokenAmounts(12558033400096537032, 20259533801624375790673555415)
Python Code
Here is the python code I wrote using the js2py package. To save space, I removed the JS Code in the js variable below, but you can copy it from above and place it in the triple quotes:
import js2py
js = """
<<< JS CODE COPIED HERE >>>
""".replace("document.write", "return ")
result = js2py.eval_js6(js) # executing JavaScript and converting the result to python string
One other note, you will need to install babel as well to make it work. Babel will help convert the code from JS6 to JS5.
stpyv8 attempt
This package is maintained by Cloudflare, which makes this a much more attractive approach, but I am having difficulty installing the pyv8 package on my machine.
I was able to successfully run:
sudo apt install python3 python3-dev build-essential libboost-dev libboost-system-dev libboost-python-dev libboost-iostreams-dev
The probem is that I now need to run setup.py, which I can't find. I also below, if you look at the pyv8 installation I tried below, it does it automatically, but it throws an error. Seems like I am getting stuck at trying to run setup.py
pyv8 attempt (alternative to stpyv8 I believe)
https://code.google.com/archive/p/pyv8/
When I run pip3 install -v pyv8, I get the following error:
Using pip 22.0.2 from /home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/pip (python 3.10)
Collecting pyv8
Using cached PyV8-0.5.zip (22 kB)
Running command python setup.py egg_info
running egg_info
creating /tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info
writing /tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info/SOURCES.txt'
writing manifest file '/tmp/pip-pip-egg-info-g__o7shf/PyV8.egg-info/SOURCES.txt'
Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for pyv8, since package 'wheel' is not installed.
Installing collected packages: pyv8
Running command Running setup.py install for pyv8
running install
/home/bobby/uni_balances/uni_venv/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.10
copying PyV8.py -> build/lib.linux-x86_64-3.10
running build_ext
building '_PyV8' extension
creating build/temp.linux-x86_64-3.10
creating build/temp.linux-x86_64-3.10/src
x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBOOST_PYTHON_STATIC_LIB -Ilib/python/inc -Ilib/boost/inc -Ilib/v8/inc -I/home/bobby/uni_balances/uni_venv/include -I/usr/include/python3.10 -c src/Context.cpp -o build/temp.linux-x86_64-3.10/src/Context.o
In file included from src/Wrapper.h:8,
from src/Context.h:7,
from src/Context.cpp:1:
src/Exception.h:6:10: fatal error: v8.h: No such file or directory
6 | #include <v8.h>
| ^~~~~~
compilation terminated.
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
error: subprocess-exited-with-error
× Running setup.py install for pyv8 did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/bobby/uni_balances/uni_venv/bin/python3 -u -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
# import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
# setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
# manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute `setup.py` since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
__file__ = %r
sys.argv[0] = __file__
if os.path.exists(__file__):
filename = __file__
with tokenize.open(__file__) as f:
setup_py_code = f.read()
else:
filename = "<auto-generated setuptools caller>"
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/tmp/pip-install-b9d0zoul/pyv8_c9dfd2e4ec0e4c66a2bff47f44972004/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' install --record /tmp/pip-record-pel12p8h/install-record.txt --single-version-externally-managed --compile --install-headers /home/bobby/uni_balances/uni_venv/include/site/python3.10/pyv8
cwd: /tmp/pip-install-b9d0zoul/pyv8_c9dfd2e4ec0e4c66a2bff47f44972004/
Running setup.py install for pyv8 ... error
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> pyv8
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
Conclusion
I am not married to using the two options I mentioned above, but I would like to pull the data into Python from the JS code.
Also this is the output from the JS code that I need to port over into Python:
Windows Git Bash specific problem...
Pretty simple script which takes some user input, and does not echo it to the output. Works fine when called like node secret.js but acts strange when called as ./secret.js, needing a ctrl+c to exit, and echoing the output as you type.
#!/usr/bin/env node
var prompt = require('prompt');
prompt.start();
prompt.colors = false;
prompt.message = '';
prompt.delimiter = '';
prompt.get([{
name: 'secret',
description: 'tell me your darkest secret: ',
hidden: true
}], function(err, result){
console.log('Hey guys! He said "' + result.secret.slice(0, 5) + '..." only kidding, I won\'t tell.');
});
What is a safe way to make script run on all platforms, including git bash?
update: added env result in case it is useful...
IEUser#ie8winxp MINGW32 ~/projects/issue (develop)
$ env
HOMEPATH=\Documents and Settings\IEUser
MANPATH=/mingw32/share/man:/usr/local/man:/usr/share/man:/usr/man:/share/man:
APPDATA=C:\Documents and Settings\IEUser\Application Data
HOSTNAME=ie8winxp
SHELL=/usr/bin/bash
TERM=xterm
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 10, GenuineIntel
WINDIR=C:\WINDOWS
TMPDIR=/tmp
OLDPWD=/c/Documents and Settings/IEUser/projects
USERDOMAIN=IE8WINXP
OS=Windows_NT
ALLUSERSPROFILE=C:\Documents and Settings\All Users
TEMP=/tmp
COMMONPROGRAMFILES=C:\Program Files\Common Files
USERNAME=IEUser
PROCESSOR_LEVEL=6
PATH=C:\Documents and Settings\IEUser\projects\issuemd\node_modules\.bin:C:\Documents and Settings\IEUser\projects\issue\node_modules\.bin:C:\Documents and Settings\IEUser\projects\node_modules\.bin:/c/Documents and Settings/IEUser/bin:/mingw32/bin:/usr/local/bin:/usr/bin:/bin:/mingw32/bin:/usr/bin:/c/Documents and Settings/IEUser/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/Program Files/nodejs:/c/Documents and Settings/IEUser/Application Data/npm:/usr/bin/vendor_perl:/usr/bin/core_perl
EXEPATH=C:\Program Files\Git
FP_NO_HOST_CHECK=NO
PWD=/c/Documents and Settings/IEUser/projects/issue
SYSTEMDRIVE=C:
LANG=en_US.UTF-8
USERPROFILE=C:\Documents and Settings\IEUser
CLIENTNAME=Console
PS1=\[\033]0;$TITLEPREFIX:${PWD//[^[:ascii:]]/?}\007\]\n\[\033[32m\]\u#\h \[\033[35m\]$MSYSTEM \[\033[33m\]\w\[\033[36m\]`__git_ps1`\[\033[0m\]\n$
LOGONSERVER=\\IE8WINXP
PROCESSOR_ARCHITECTURE=x86
SSH_ASKPASS=/mingw32/libexec/git-core/git-gui--askpass
SHLVL=1
HOME=/c/Documents and Settings/IEUser
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
PLINK_PROTOCOL=ssh
HOMEDRIVE=C:
MSYSTEM=MINGW32
COMSPEC=C:\WINDOWS\system32\cmd.exe
TMP=/tmp
SYSTEMROOT=C:\WINDOWS
PROCESSOR_REVISION=170a
PKG_CONFIG_PATH=/mingw32/lib/pkgconfig:/mingw32/share/pkgconfig
ACLOCAL_PATH=/mingw32/share/aclocal:/usr/share/aclocal
INFOPATH=/usr/local/info:/usr/share/info:/usr/info:/share/info:
PROGRAMFILES=C:\Program Files
DISPLAY=needs-to-be-defined
NUMBER_OF_PROCESSORS=1
SESSIONNAME=Console
COMPUTERNAME=IE8WINXP
_=/usr/bin/env
Turns out cygwin is not supported by node (and I assume git bash too).
Seems that git bash is not a real tty.
Looks like someone did something about it by bundling winpty with git bash.
Solution...
From within git bash, run winpty bash, then rest should work as expected.
I am trying to load in a PFX and passphrase from a file so that I can make a HTTPS request. Before I start, I already know that the PFX is good and that is not the issue.
I am doing the following:
config.options.pfx = fs.readFileSync('file.pfx');
config.options.passphrase = 'passphrase';
I am passing my options into an agent.
config.options.agent = new https.Agent(options);
I then try to build the rquest where I get the following error:
crypto.js:143
c.context.loadPKCS12(pfx, passphrase);
^
Error: header too long
at Object.exports.createCredentials (crypto.js:143:17)
at Object.exports.connect (tls.js:1334:27)
at Agent.createConnection (https.js:79:14)
at Agent.createSocket (http.js:1293:16)
at Agent.addRequest (http.js:1269:23)
at new ClientRequest (http.js:1416:16)
at Object.exports.request (https.js:123:10)
I checked this out from a work repository where I know that this works for the original author of it. For some reason my set-up is not running it, though.
I had a similar issue. It turned out I was using fs.readFileSync('file.pfx', 'utf8'), which is correct for PEM files, but since PKCS12 files are binary, you should just pass in fs.readFileSync('file.pfx').
I had the same problem but in my case i was using
pfx: fs.readFileSync('certs/keystore.p12')
In my case, the problem was that the pfx I was using was not correctly generated from the jks. Exporting with keytool was the right solution
keytool -v -importkeystore -srckeystore keystore.jks -destkeystore keystore.p12 -deststoretype PKCS12
I was kind of new to casperjs more specifically javascript on native enviroments, some casperjs script has the following code fragment that gives me error on executing:
system.stdout.write("Old \"" + password.name + "\" password: ");
var oldPassword = system.stdin.readLine().trim();
I tried to install commonjs npm library but does not solved my problem, I echoed the properties of available system library as follows:
for(var tmp in system){
console.log(tmp);
}
The out put is:
objectName
pid
args
env
os
isSSLSupported
destroyed(QObject*)
destroyed()
deleteLater()
_isCompletable()
_getCompletions(QString)
The first lines that execute the require commands as follows:
system = require('system');
casper = require('casper').create();
require = patchRequire(require, ['./adapters']);
config = require('./config').config;
And the complete source of the casperjs program I try to use is here passup.js
How can I solve this issue? I mean load system so that it has .stdout and .stdin properties. Any help would be appreciated.
Which version of PhantomJS do you use? Support for standard I/O was introduced in the 1.9 version.
Read more here: http://phantomjs.org/release-1.9.html
Are there equivalent to perl -c syntax check for JavaScript from command? Given that I have NodeJS installed?
JSLint is not considered as it is not a real parser. I think YUI compressor is possible but I don't want to install Java on production machines, so I am checking if Node.JS already provided this syntax check mechanism.
If you want to perform a syntax check like that way we do in perl ( another scripting language) you can simply use node -c <js file-name>
e.g. a JS file as test.js has:
let x = 30
if ( x == 30 ) {
console.log("hello");
else {
console.log( "world");
}
now type in node -c test.js
it will show you
test.js:5
else {
^^^^
SyntaxError: Unexpected token else
at startup (bootstrap_node.js:144:11)
at bootstrap_node.js:509:3
Now after fixing the syntax issue as
let x = 30
if ( x == 30 ) {
console.log("hello");
} else {
console.log( "world");
}
check syntax - node -c test.js will show no syntax error!!
Note - we can even use it to check syntax for all files in a dir. - node -c *.js
Try uglify. You can install it via npm.
Edit: The package name has changed. It is uglify-js.
nodejs --help
explains the -p switch: it evaluates the supplied code and prints the results. So using nodejs -p < /path/to/file.js would be a disastrous way to check the validity of node.js code on your server. One possible solution is the one indicated in this SO thread. The one thing not so good about it - the syntax error messages it reports are not terribly helpful. For instance, it tell you something is wrong but without telling you where it is wrong.