So, I'm teaching myself various AI techniques, and I figured the best way to do so would be to create my own Python bot which can play the game Pokemon using an online open-source Pokemon simulator.
The issue is that this simulator has all its source code written in JavaScript, whereas my bot uses Python. At first, I solved the problem of grabbing the full list of valid Pokemon and such by simply downloading the server's .js files, which (handily) contained all valid Pokemon/moves as JSON objects that I could easily import into Python. This was all well and good when I just needed Pokemon names and such, but now that I'm building out the actual engine I've discovered that it uses these same JSON files for its actual gameplay code.
For example, here's the Pokemon "Ability" Aftermath. Pay attention to onAfterDamage:
"aftermath": {
desc: "If this Pokemon is knocked out with a contact move, that move's user loses 1/4 of its maximum HP, rounded down. If any active Pokemon has the Damp Ability, this effect is prevented.",
shortDesc: "If this Pokemon is KOed with a contact move, that move's user loses 1/4 its max HP.",
id: "aftermath",
name: "Aftermath",
onAfterDamageOrder: 1,
onAfterDamage: function (damage, target, source, move) {
if (source && source !== target && move && move.flags['contact'] && !target.hp) {
this.damage(source.maxhp / 4, source, target);
}
},
rating: 2.5,
num: 106,
}
As you can see, onAfterDamage is a JavaScript function, one that I can't easily call from Python (to my knowledge). So now I'm at a crossroads:
Stop grabbing the .js files from the server and write my own code entirely in Python, essentially "translating" the JavaScript code myself.
Rewrite my entire bot in JavaScript instead of Python, although I'm far less experienced and comfortable in JavaScript.
Find a way to run the JavaScript code I'm downloading from the server from within my Python bot, calling arbitrary functions from within the code and returning the results.
Option 1 has the issue of rapidly becoming out of date -- when new Pokemon mechanics are introduced or new Pokemon are created, I wouldn't get it "for free" -- I'd have to figure out what changed and implement the changes myself.
I'd like to avoid Option 2 just because I'm not very comfortable writing JavaScript -- it's something I've dabbled with in the past, but it's far from my favorite language and not something I want to deal with when it's not the primary focus of my project.
I realize that Option 3 is a pretty massive security risk -- while I'm reasonably sure that a man-in-the-middle attack won't happen to a bot that likely nobody will ever use but myself, I'm still just downloading random JavaScript from a server somewhere and trying to execute it.
I've seen things like PyExecJS, but they're no longer maintained, and I haven't really seen anyone recommending a good replacement for them. PyExecJS's functionality is exactly what I'm looking for (namely the function execjs.eval(js_code)), but I'm not sure I want to use something that's reached EOL. I've seen the answers here, but they don't recommend any replacements for PyExecJS; they only show that user that they're using it wrong.
I've also seen people suggest Selenium or running a headless browser, but I'm not sure if I want to deal with all that extra overhead. The very first programming project I ever did outside of school was very similar to this, and I wound up coding it in Java using Selenium. I wasn't happy with that experience, and I'm not looking forward to having to do it again, especially when I only need to execute JavaScript I already have downloaded locally.
Am I completely off-base here? Is there another approach I should take, or should I just get to work translating all this JS code to Python by hand?
Ideally you could run your .js-file from cli just like any other executable. With node -e you can execute your JavaScript code on cli.
Relating to this post https://stackoverflow.com/a/450329/6478277 you could similarly run your script with your local node.
node -e "function func(){console.log(123)};func();"
From a file
print subprocess.check_output(['node -e "require(\"./file.js\").test()"'])
Where file.js could be something like
module.exports = {
test: () => console.log(123),
}
Related
I am using a WebRequest() function within the MetaTrader Terminal 4 codebase (MQL4) that allows one to download a HTML-response from a website.
Example site: http://www.forexfactory.com/docphoenix66#acct.57-tab.list
Here is an example how it is used in the MQL4 function call:
res = WebRequest( "GET",
"http://www.forexfactory.com/docphoenix66#acct.57-tab.list",
cookie,
NULL,
timeout,
post,
0,
result,
headers
);
and the documentation for the function WebRequest()
However, if I compare what is downloaded using a WebRequest() call with what you see when you right click and inspect element using Chrome or Safari, the bits I want available are missing!
In particular I want the trade information from below the following columns:
Instrument Price Open/Close Date Open/Close Lots Return
Profit Pips Chart Balance Swap Duration
And if you see below an example of what is missing from the htm file downloaded using the MQL4 function.
<td class="slidetable__cell slidetable__cell--fixed" style="width: 62px; min-width: 62px;"> <a id="snap_48205_trade_109309333" class="explorer__anchor explorer__anchor--trade"></a>
EUR/USD
</td>
If you download the HTML file, turn off your wifi and then open the file to see what was downloading, you see everything in the trade explorer still loading. Am I clear on what my problem is?
Short version: Yes, there is.
Long version: TL;DR;
Well, first, welcome to the Wild Worlds of MQL4
Given the intention is clear and given you were "promised" that there is "a possible way to read a HTML-page", I have to tell you it is not possible in all cases you will meet in real-world.
One may spend ages in MQL4-code domain to re-design html-sort-of Mark-Up syntax-(b)LOBs, suffering from all the restricted constraints the MQL4-code execution engine provided.
Nevertheless, the much faster, joyfull and a sure and future-proof ( read other posts on historically painfull creeping of the language syntax relief and crippled man*decades in API-integration code-base efforts ) approach exists .
Integrate MQL4-side via a professional fast & low-latency SIG/MSG-infrastructure with external, distributed processes, that can provide high-performance & robust services to MetaTrader Terminal ecosystem.
Using this approach we have prototyped and operate fast Mixed-Technical-and-Fundamental AI/ML-Inputs, including Web-page-feeds of Fundamental Data and News Announcements into the FX-trading realm 24/7/365 and it works blessingly well, independently of the limits the common MQL4 execution has.
If still in doubts, just try to read a page on rss.provider.com:6322/FED_actuals URL via a call to WebReqest() and you know, where is the dog burried.
I want to know if there is a way to open a exe file from client side with jQuery or JavaScript.
I tried with windows.open("..") but it didn't work.
First off, Dalorzo is correct - you cannot escape the sandbox of a browser without earning lots of money. Recently Pinky Pie won over $60k from Google for performing such a task on Google Chrome.
http://blog.chromium.org/2012/05/tale-of-two-pwnies-part-1.html
However, sometimes I ask questions to try to find out-of-the-box solutions, so I'm assuming that you have a legitimate use-case for this question. I can help you think of a "hack" solution if you have a legitimate use for this. I'll probably get downvoted to oblivion for this response, but theoretically you could do this...
Back in college, I took a computer forensics class in which I wrote a chat web application which would forensically clean a user's computer from having ever accessed the chat webpage. This was done through the user running an executable they could download from the site. However, I wanted to have the users to not have to run the executable every time so I wanted a sandbox escape like you're asking about.
It's messy, but if your users are a participating party they can modify their computer to allow executables to be run from Chrome webpages.
Please Note: This will make your computer less secure, I only ever did it on Virtual Machines or Lab Machines. Reconsider whether you actually wanna try this or not.
Step One: Create a Chrome custom browser:
This is the python that I eventually compiled to a packed *.exe that accomplished creating a Google Chrome custom browser.
import sqlite3 as lite
con = lite.connect('C:\Users\USERNAME\AppData\Local\Google\Chrome\User Data\Default\Web Data')
c = con.cursor()
#idmax = c.execute('SELECT MAX(id) FROM keywords')
#c.execute("INSERT INTO keywords VALUES('cleanupeverybodyeverywhere','cleanupeverybodyeverywhere','','forensichat://C:>Windows>System32>calc.exe',0,'',0,0,'',0,'',0,0,'',0,'','','','','','','','','')")
c.execute("SELECT * FROM keywords WHERE short_name='cleanupeverybodyeverywhere'")
foo = c.fetchall()
if foo == []:
c.execute("INSERT INTO keywords (short_name,keyword,url,favicon_url) VALUES ('cleanupeverybodyeverywhere','cleanupeverybodyeverywhere','forensichat://C:>Windows>System32>calc.exe','')")
con.commit()
con.close()
Note that now when I open a Chrome tab and use the custom browser cleanupeverybodyeverywhere to run the key forensichat and it will open calc.exe as proof of concept.
Step Two: Modify your Windows Registry:
import _winreg as wreg
import getpass
user = getpass.getuser()
key = wreg.CreateKey(wreg.HKEY_CLASSES_ROOT, '')
wreg.SetValue(key,'forensichat',wreg.REG_SZ,'URL:forensichat Protocol')
# Create new subkey
kkey = wreg.CreateKey(wreg.HKEY_CLASSES_ROOT, 'forensichat')
wreg.SetValueEx(kkey, 'URL Protocol', 0, wreg.REG_SZ,'')
# Create new value
wreg.CreateKey(wreg.HKEY_CLASSES_ROOT,"forensichat\shell")
nkey = wreg.CreateKey(wreg.HKEY_CLASSES_ROOT,"forensichat\shell\open")
wreg.SetValue(nkey,"command",wreg.REG_SZ,'C:\Users\\'+user+'\Downloads\\forensicleaner.exe')
key.Close()
Note that I'm creating a key called forensichat which will run the exe found in the user's Downloads folder.
Of course, I ended up cleaning this all up and packing it so Anti-Virus wouldn't flag it, next thing I know Google Chrome has marked the site lol. It's still up if you wanna check it out - check Secure Chat and Repeat Customer. Chrome will flag it, but there's obviously ways around that.
Well here's a problem.
I've got a website with large javascript backend. This backend talks to a server over a socket with a socket bridge using http://blog.deconcept.com/swfobject/
The socket "bridge" is a Flex/Flash .swf application/executable/plugin/thing for which the source is missing.
I've got to change it.
More facts:
file appExePluginThing.swf
appExePluginThing.swf Macromedia Flash data (compressed), version 9
I've used https://www.free-decompiler.com/flash/ to decompile the .swf file and I think I've sorted out what's the original code vs the libraries and things Flash/Flex built into it.
I've used FDT (the free version) to rebuild the decompiled code into MYappExePluginThing.swf so I can run it with the javascript code and see what happens.
I'm here because what happens isn't good. Basically, my javascript code (MYjavascript.js) gets to the point where it does
window.log("init()");
var so = new SWFObject("flash/MYappExePluginThing.swf"", socketObjectId, "0", "0", "9", "#FFFFFF");
window.log("init() created MYappExecPluginThing!!!");
so.addParam("allowScriptAccess", "always");
log("init() added Param!!");
so.write(elId);
log("init() wrote!");
IE9's console (yeah, you read that right) shows
init()
created MYappExecPluginThing!!!
init() added Param!!
init() wrote!
but none of the debugging i've got in MYappExePluginThing.as displays and nothing else happens.
I'm trying to figure out what I've screwed up/what's going on? Is MYappExePluginThing.as running? Is it waiting on something? Did it fail? Why aren't the log messages in MYappExePluginThing.as showing up?
The first most obvious thing is I'm using FDT which, I suspect, was not used to build the original. Is there some kind of magic "build javascript accessible swf thing" in FlashBuilder or some other IDE?
First noteworthy thing I find is:
file MYappExePluginThing.swf
MYappExePluginThing.swf Macromedia Flash data (compressed), version 14
I'm using Flex 4.6 which, for all I know, may have a completely different mechanism for allowing javascript communication than was used in appExePluginThing.swf
Does anyone know if that's true?
For example, when FDT runs this thing (I can compile but FDT does not create a .swf unless i run it) I get a warning in the following method:
private function init() : void
{
Log.log("console.log", "MYappExePluginThing init()");
//var initCallback:String = Application.application.parameters.initCallback?Application.application.parameters.initCallback:"MYjavascript.MYappExePluginThing_init";
var initCallback:String = FlexGlobals.topLevelApplication.parameters.initCallback?FlexGlobals.topLevelApplication.parameters.initCallback:"MYjavascript.MYappExePluginThing_init";
try
{
ExternalInterface.addCallback("method1Callback",method1);
ExternalInterface.addCallback("method2Callback",method2);
ExternalInterface.call(initCallback);
}
catch(err:Error)
{
Log.log("console.log", "MYappExePluginThing init() ERROR err="+err);
}
}
I got a warning that Application.application was deprecated and I should change:
var initCallback:String = Application.application.parameters.initCallback?Application.application.parameters.initCallback:"MYjavascript.MYappExePluginThing_init";
to:
var initCallback:String = FlexGlobals.topLevelApplication.parameters.initCallback?FlexGlobals.topLevelApplication.parameters.initCallback:"MYjavascript.MYappExePluginThing_init";
which I did but which had no effect on making the thing work.
(FYI Log.log() is something I added:
public class Log{
public static function log(dest:String, mssg:String):void{
if(ExternalInterface.available){
try{
ExternalInterface.call(dest, mssg);
}
catch(se:SecurityError){
}
catch(e:Error){
}
}
trace(mssg);
}
}
)
Additionally, in MYjavascript.js MYappExePluginThing_init looks like this:
this.MYappExePluginThing_init = function () {
log("MYjavascript.js - MYappExePluginThing_init:");
};
Its supposed to be executed when MYappExePluginThing finishes initializing itself.
Except its not. The message is NOT displaying on the console.
Unfortunately, I cannot find any references explaining how you allow javascript communication in Flex 4.6 so I can check if I've got this structured correctly.
Is it a built in kind of thing all Flex/Flash apps can do? Is my swf getting accessed? Is it having some kind of error? Is it unable to communicate back to my javascript?
Does anyone have any links to references?
If this was YOUR problem, what would you do next?
(Not a full solution but I ran out of room in the comment section.)
To answer your basic question, there's nothing special you should need to do to allow AS3-to-JS communication beyond what you've shown. However, you may have sandbox security issues on localhost; to avoid problems, set your SWFs as local-trusted (right-click Flash Player > Global Settings > Advanced > Trusted Location Settings). I'm guessing this not your problem, though, because you'd normally get a sandbox violation error.
More likely IMO is that something is broken due to decompilation and recompilation. SWFs aren't meant to do that, it's basically a hack made mostly possible due to SWF being an open format.
What I suggest is that you debug your running SWF. Using break-points and stepping through the code you should be able to narrow down where things are going wrong. You can also more easily see any errors your SWF is throwing.
Not really an answer, but an idea to get you started is to start logging everything on the Flash side to see where the breakage is.
Since you're using IE, I recommend getting the Debug flash player, installing it, then running Vizzy along side to show your traces.
Should give you a good idea of where the app is breaking down.
Vizzy
Debug Player
While looking at how to make JavaScript source code more secure I came upon a lot of 'solutions'. but most people said the same thing; "It's not possible to make your source code 100% secure", "try obfuscation", "run your code server side", etc, etc. After reading a lot posts here on stackoverflow, and other sites I came to the conclusion that a combination of minifying and obfuscating would do the job (for me).
But here is the problem: we are currently using soma.js with dependency injection, and the way we set it up it does not work well with obfuscation. It's basically this:
var session = function(id, sessionModel){
this._sessionmodel = sessionModel;
}
mapping:
injector.mapClass("sessionModel", project.SessionModel, true);
Obfuscation will then rename the sessionModel in the function to for example 'A', but the mapping that was done on SessionModel by the injector still remains 'sessionModel' and not 'A', this then basically breaks the code.
I've read this post which is about the same subject Dependency Injection and Code Obfuscation, but it does not provide a real answer to my problem so I decided to write my own question.
Any tips/hint/suggestions are welcome.
Thanks in advance.
EDIT
It seems you can tell Yuicompressor to exclude certain identifiers by putting in 'hints' into the files like this: "identifier:nomunge, identifier2:nomunge".
var session = function(id, sessionModel){
"sessionModel:nomunge";
this._sessionmodel = sessionModel;
}
I tested this and it works but that means you'll have to put it in yourself which is a lot of work if you have to do that for every script, especially if you have a very big project..
Gonna look into it further, and update this post if anything new pops up
EDIT 2
It's been a while, I only work 1 day a week on this =S.
As said before you can get it working by telling it which identifiers to exclude.
For that I looked into regular expression to get the "mapped classes" programmatically, since doing it by hand is just insane.
What I basically did was instead of putting every hint in by hand, I made a identifier, for example "#nomunge"; and used a simple replaceregexp task to find it and replace it with a string containing all the identifiers. This string is build by loading the script and going through it with a tokenfilter.
<target name="build hints">
<loadfile property="hints" srcFile="${temp.loc}/all.js">
<filterchain>
<tokenfilter delimoutput=":nomunge,">
<ignoreblank/>
<containsregex pattern="${regexp}"/>
</tokenfilter>
</filterchain>
</loadfile>
<echo message="${hints}"/>
</target>
<replaceregexp file="${temp.loc}/all.js"
match="#nomunge"
flags = "g"
replace = "target:nomunge, dispatcher:nomunge, injector:nomunge,${hints}"
/>
This seems to do the job, for now...
I'm behind the soma.js framework, feel free to ask me questions on the google group, happy to help.
This might help a bit more:
https://groups.google.com/forum/#!topic/somajs/noOX2R4K58g
Romu
There is a website called Gild.com that has different coding puzzles/challenges for users to do. They can be completed in wide array of languages including Javascript. I am interested in solving these puzzles in Javascript, but I am unsure of the following:
How am I supposed to access the input file which is supposed to be passed as an argument?
How am I supposed to output the result?
My understanding of Javascript is that it is run from within an HTML page and that output really is only in the form of placing values in the HTML, modifying the DOM, etc. For that reason it is not clear to me how Javascript can be used for solving these types of problems. Can someone who has used Gild before or has some insights into my question suggest how to proceed?
An example of a problem would be: the given input file contains a positive integer, find the sum of all prime numbers smaller than that integer and output it.
EDIT: Some of the solutions below involve using external resources, but on Gild, I am supposed to put my solution in their editor and then submit it that way, like the following picture shows:
In other words, I don't think my solution can have access to Node.js or other external resources.
Edit: Here are some interesting articles that I have found that I think are the answer to my question:
http://www.phpied.com/installing-rhino-on-mac/
http://www.phpied.com/javascript-shell-scripting/
I haven't spent much time on Gild, but I do a lot of similar types of problems on Project Euler. I think the best way to go is probably Node.js.
If you're not familiar, Node is basically server-side JavaScript that runs in Google's V8 engine. Installing it on your own Mac/Windows machine takes about 2 minutes. It's also really fast (considering it's JavaScript).
And you'd use it like this:
var fs = require('fs'); // the filesystem module
var contents = fs.readFileSync('theFile.txt', 'utf-8');
// Do stuff with the file contents...
Everything after those first two lines can be done with the same JS you'd write in the browser, right down to calling console.log() to spit out the answer.
So, if you wrote your script in a file on your desktop called getprimes.js, you'd open up your terminal and enter node ~/Desktop/getprimes.js (assuming you're on a Mac)
If you're:
on a Mac,
planning to do a lot of these puzzles, and
willing to pay $10, then
I highly recommend CodeRunner. It encapsulates runtimes for a variety of languages — from C to JavaScript — and lets you quickly build and run any sort of one-off code. Just hack together your code, ⌘R, and the results are printed right there in the same window.
I haven't used any file-based JavaScript in CodeRunner, but I imagine kennis's suggestions would apply. To output your results:
console.log(...)
Easy as pie!