Udacity gives students a web editor to enter Python programs. The editor recognizes Python keywords and built-in functions and allows to run a program. Do you know how this technology works? Are programs submitted to a backend and executed by the standard Python interpreter or is it a JavaScript based Python interpreter? Does the editor simply hold a static list of Python keywords and built-in functions or does it interact with the standard or JavaScript based Python to obtain these?
While javascript python interpreters do exist: http://syntensity.com/static/python.html , they don't appear to be using one. It would be far too easy to cheat if they didn't at least run the programs once for verification on their own interpreter.
After looking at the network activity on Udacity I can see that they make an ajax call with a bunch of data and then get the results of the program run back in JSON.
At a guess they have the standard python interpreter running in a sandbox that will execute the assignments, then the results are packed into JSON, returned to the client and updated to the screen. At the same time the results of your submission will be recorded as part of your class results.
Very late to the party here, but I work as an engineer at Udacity, so I figured I'd give it a shot.
There are two fundamental things going on:
The current syntax highlighting and editing is provided by a Codemirror implementation, although we have used several different editors in the past couple years.
When you hit submit (or run), your code is packaged up and shipped off to a sandboxed cluster we run for execution. If you're hitting submit, this is where we run our own tests against your code and "grade" it to see if it passes. The output from that (in various forms) is piped back to the front end, and you get your feedback.
Not quite as fast as running it locally, but it sure beats supporting a few hundred thousand people trying to install Python for the first time ;)
I haven't tried Udacity, but for the syntax highlighting parts, it can be easily done with a simple backend code, which is updated using some Ajax. One of the easiest ways can be used as lexical analyzing as in compilers or interpreters ..
Related
I am a few hundred lines of code into developing a Google Apps Script web app, with code that I have mostly tested as I went (thank you, GAS, for making the back-end language basically JavaScript, so that I could test in-browser). However, I am in the process of moving some of the functions and classes that I wrote on the client-side onto the server-side (the server end should have those functions, too, as its job is to serve the data!), and adding a few methods on the server side.
I would like to know how best to test this code I'm writing on top of what I'm implementing, without having to move it to client-side (or JavaScript console) every time that I write it. It would be nice to have results and tests that I can actually persist, without having to clutter my browser's console's command history.
I just created another google apps script project in another google doc. It's the easiest way for me.
There are numerous log files that I have to review daily for my job. Several good parsers already exist for these log files but I have yet to find exactly what I want. Well, who could make something more tailored to you than you, right?
The reason I am using JavaScript (other than the fact that I already know it) is because it's portable (no need to install anything) but at the same time cross-platform accessible. Before I invest too much time in this, is this a terrible method of accomplishing my goal?
The input will be entered into a text file, delimited by [x] and the values will be put into an array to make accessing these values faster than pulling the static content.
Any special formatting (numbers, dates, etc) will be dealt with before putting the value in the array to prevent a function from repeating this step every time it is used.
These logs may contain 100k+ lines which will be a lot for the browser to handle. However, each line doesn't contain a ton of information.
I have written some of it already, but with even 10,000 lines it's starting to run slow and I don't know if it's because I wasn't efficient enough or if this just cannot be effectively done. I'm thinking this is because all the data is in one giant table. I'd probably be better off paginating it, but that is less than desirable.
Question 1: Is there anything I failed to mention that I should consider?
Question 2: Would you recommend a better alternative?
Question 3: (A bit off topic, so feel free to ignore). Instead of copy/pasting the input, I would like to 'open' the log file but as far as I know JavaScript cannot do this (for security reasons). Can this be accomplished with a input="file" without actually having a server to upload to? I don't know how SSJS works, but it appears that I underestimated the limitations of JavaScript.
I understand this is a bit vague, but I'm trying to keep you all from having to read a book to answer my question. Let me know if I should include additional details. Thanks!
I think JavaScript is an "ok" choice for this. Using a scripting language to parse log files for personal use is a perfectly sane decision.
However, I would NOT use a browser for this. Web browsers place limitations on how long a bit of javascript can run, or on how many instructions it is allowed to run, or both. If you exceed these limits, you'll get something like this:
Since you'll be working with a large amount of data, I suspect you're going to hit this sooner or later. This can be avoided by clever use of setTimeout, or potentially with web workers, but that will add complexity to your project. This is probably not what you want.
Be aware that JavaScript can run outside of browsers as well. For instance, Windows comes with the Windows Script Host. This will let you run JavaScript from the command prompt, without needing a browser. You won't get the "Script too long" error. As an added bonus, you will have full access to the file system, and the ability to pass command-line arguments to your code.
Good luck and happy coding!
To answer your top question in bold: No, it is not a terrible idea.
If JS is the only language you know, you want to avoid setting up any dependencies, and you want to stay platform-independent... JavaScript seems like a good fit for your particular case.
As a more general rule, I would never use JS as a language to write a desktop app. Especially not for doing a task like log parsing. There are many other languages which are much better suited to this type of problem, like Python, Scala, VB, etc. I mention Python and Scala because of their script-like behaviour and minimal setup requirements. Python also has very similar syntax to JS so it might be easier to pick up then other languages. VB (or any .NET language) would work too if you have a Visual Studio license because of it's easy to use GUI builder if that suits your needs better.
My suggested approach: use an existing framework. There are hundreds, if not thousands of log parsers out there which handle all sorts of use-cases and different formats of logs that you should be able to find something close to what you need. It may just take a little more effort than Google'ing "Log Parsers" to find one that works. If you can't find one that suits your exact needs and you are willing to spend time making your own, you should use that time instead to contribute to one of the existing ones which are open source. Extending an existing code base should always be considered before trying to re-invent the wheel for the 10th gillion time.
Given your invariants "javascript, cross-platform, browser ui, as fast as possible" I would consider this approach:
Use command line scripts (windows: JScript; linux: ?) to parse log files and store 'clean'/relevant data in a SQLite Database (fall back: any decent scripting language can do this, the ready made/specialized tools may be used too)
Use the SQLite Manager addon to do your data mining with SQL
If (2) gets clumsy - use the SQLite Manager code base to 'make something more tailored'
Considering your comment:
For Windows-only work you can use the VS Express edition to write an app in C#, VB.NET, C++/CLI, F#, or even (kind of) Javascript (Silverlight). If you want to stick to 'classic' Javascript and a browser, write a .HTA application (full access to the local machine) and use ADO data(base) access and try to get the (old) DataGrid/Flexgrid controls (they may be installed already; search the registry).
the same problem haunting me a month ago is still haunting me now. i know ive asked several questions regarding this on this site and i am truly sorry for that. your suggestions have all been excellent but the answer is still elusive. i now realize that this is a direct result of me not being able to phrase my question properly and for that i am sorry.
to give you guys a generalized view of things, here i go: the situation is like this, i have 2 server side scripts that i want to run.
a python program/script that continuously spouts some numbers
based on the output from that python script, a javascript script will perform some action on a webpage (e.g., change background color, display alert message, change some text)
ive studied the replies to my previous posts and have found that what i want to accomplish is more or less accomplished by json. it is my understanding that json transforms 'program-specific' variables into a format that is more 'standard or general or global'.
two different programs therefore now have the means to 'talk' with each other because they are now speaking the same 'language'.
the problem is then this, how do i actually facilitate their communication? what is the 'cellphone' between these server side scripts? do they even need one?
thank you!
If I understand what you're asking, the "cellphone" is TCP/IP. The javascript is not server-side; it runs on the client side, and alters what the client's browser displays based on json data that it downloads from the server -- data that in this case is generated by Python.
This question provides a relevant example, though it's a bit technical: JSON datetime between Python and JavaScript
Here's a very basic tutorial that explains how to create a dynamic webpage using python and javascript. It doesn't appear to use json, but it should familiarize you with the fundamentals. Once you understand what's there, using json to transport more complicated data should be fairly straightforward.
http://kooneiform.wordpress.com/2010/02/28/python-and-ajax-for-beginners-with-webpy-and-jquery/
I assume you mean: Python is on the web server, and Javascript is running in the client's web browser.
Because browsers are all different (IE6 is terrible, Chrome is great), there are a huge number of ways people found to "hack" this "cellphone" into place. These techniques are called AJAX and COMET techniques. There is no one "cellphone", but a whole bunch of them! Hopefully, you can find a library to select the right technique for the browser, and you just have to worry about the messages.
Comet is harder to do, but lets the server "push" messages to the client.
Ajax can be easier - you just periodically "pull" messages from the server.
Start with Ajax, then look at comet if you really need it. Just start by have the client (javascript) make a "GET" request, to see if the number has changed.
I don't know Javascript or json, but...
if you've ever seen an Unix-like operating system, you know about pipes. Like program1 | program2 | program3 ... Why don't you just connect Python and Javascript programs with pipes? The first one writes to stdout, and the next one reads from stdin.
This probably isn't the answer that you are looking for, and without links to your previous posts, I don't have much to go on, but nonetheless...
javascript is client side. I can interpret your question 2 different ways...
Your python script is running on your computer, and you want a script to actually alter your current browser window.
Not too sure, but writing a browser plugin may be the answer here.
Your python script is running on the server, and as a result of the script running, you want the display of your site to be changed for viewing persons.
In this case, you will could use ajax polling (or similar) on your site. Have your site be polling the server with ajax, call a server method that checks the output of the script (maybe written to a file?), and see if it has changed.
When 2 process need to communicate, they need to decide of a common/shared way to express things and a protocol to exchange those things.
In your case, since one of the processes is a browser, the protocol of choice is http. So the browser needs to do an http request or regular http request to your python process.
This python process Will need in Some way or another to be exposed via http.
There are several ways to build a web server in python. You should read this article : http://fragments.turtlemeat.com/pythonwebserver.php as a jumpstart.
Once you have this, your browser Will be able to issue HTTP GET requests to your server and your server can reply with a string.
This string can be whatever you like. Nevertheless if your answer contains structured data it can be a good start to use the XML notation or the json notation.
Json (stands for Javascript object notation) is very easy to use in javascript and this is why many people advised you to choose this notation.
I hope this will help you
Jérome wagner
A lot of questions have been asked and answered about running server-side javascript on Google App Engine, but all of the answers deal with Java instances in order to make use of Java-based JS interpreters like Rhino, Rhino for Webapps, etc.
Is there any way to execute server-side javascript code on a Python GAE instance? I'm thinking something exactly along the lines of pyv8, but with support for App Engine (which I guess would mean a pure python implementation of the interpreter).
The only solution I can come up with at the moment is to use some sort of gross hack to run a Java and Python GAE instance side-by-side (via different versions) so they can both talk to the same datastore, let the Java instance host the JS code, and use an API to talk back'n'forth. Not very appealing.
No need to get into all the "this is unnecessary, you shouldn't be doing this" discussion -- I know this isn't ideal and I'm simply curious if it can be done.
As far as I can find: No
I've done a bit of searching, but it seems that nobody has tried to implement a pure Python Javascript engine, and I can't blame them: it would be a huge amount of work for very few use cases (unfortunately, yours is one of those). A couple of projects—Grailbrowser and Pybrowser—have Python code to render HTML, so might one day aim to run javascript, but it's not even started, and neither of them look in active development.
The most likely way it would ever happen is if Google were to offer the Parrot VM (which can run various dynamic languages) on Appengine. That's a cool idea, but I'm not holding my breath.
What might work is to run Jython (and Rhino) in a Java instance. Of course, then you'd have to get to any App services through the Java API, not the Python one, which would be ugly.
Actually, it can indeed be done, using either AppEngineJs or ESXX:
http://www.appenginejs.org/
http://esxx.blogspot.com/2009/06/esxx-on-google-app-engine.html
I am currently trying to solvevthe same problem with PyJON
http://code.google.com/p/pyjon/
Seems to be a pure Python JavaScrit parser an interpreter.
I have a legacy code to maintain and while trying to understand the logic behind the code, I have run into lots of annoying issues.
The application is written mainly in Java Script, with extensive usage of jQuery + different plugins, especially Accordion. It creates a wizard-like flow, where client code for the next step is downloaded in the background by injecting a result of a remote AJAX request. It also uses callbacks a lot and pretty complicated "by convention" programming style (lots of events handlers are created on the fly based on certain object names - e.g. current page name, current step name).
Adding to that, the code is very messy and there is no obvious inner structure - the functions are scattered in the code, file names do not reflect the business role of the code, lots of functions and code snippets are most likely not used at all etc.
PROBLEM: How to approach this code base, so that the inner flow of the code can be sort-of "reverse engineered" using a suite of smart debugging tools.
Ideally, I would like to be able to attach to the running application and step through the code, breaking on each new function call.
Also, it would be nice to be able to create a "diagram of calls" in the application (i.e. in order to run a particular page logic, this particular flow of function calls was executed in a particular order).
Not to mention to be able to run a coverage analysis, identifying potentially orphaned code fragments.
I would like to stress out once more, that it is impossible to understand the inner logic of the application just by looking at the code itself, unless you have LOTS of spare time and beer crates, which I unfortunately do not have :/ (shame...)
An IDE of some sort that would aid in extending that code would be also great, but I am currently looking into possibility to use Visual Studio 2010 to do the job, as the site itself is a mix of Classic ASP and ASP.NET (I'd say - 70% Java Script with jQuery, 30% ASP).
I have obviously tried FireBug, but I was unable to find a way to define a breakpoint or step into the code, which is "injected" into the client JS using AJAX calls (i.e. the application retrieves the code by invoking an URL and injects it to the client local code). Venkman debugger had similar issues.
Any hints would be welcome. Feel free to ask additional questions.
You can try to use "dynaTrace Ajax" to create a diagram of calls, but it is not debugger. If you have access to that application you can use keyword "debugger" to define a breakpoint explicitly in javascript files. Visual Studio theoretically shows all of evaluated and loaded javascript code in Solution Explorer when you've attached to IE.
I'd start with Chrome's developer tools and profiling single activities to find functions to set breakpoints on. Functions can be expanded to get their call stack.
That may or may not be helpful as it's entirely possible to get code that's too -- uh, unique, shall we say -- to make much sense of.
Good luck?