Wrapping Python console/plotting in HTML/CSS/JS - javascript

I am looking to build a graphical frontend for some python functionality.
Since I am very proficient in HTML/CSS/JS I decided I might check this out as a possibility.
The main idea is to make it easier to explore data/variables, I am thinking of a datagrid for example.
I was also thinking of creating a GUI for things such as plotting with matplotlib (ie: Choose variable for X axis, choose variable for Y axis, pick a color from a dropdown, ...) in order to cut out some of the tediousness in exploring data with several variables.
I would then probably use something like Node-webkit to make a desktop application out of it if possible (and if not, just keep it as a browser project, perhaps).
Essentially what it would boil down to is kind of creating an IDE for python, specialized for data exploration/plotting and perhaps data manipulation.
An example I found for a web-based python console is repl.it
However I have spent a fair amount of time looking for viable ways to go about this, since sending and receiving data from the python interpreter wouldn't be too hard, but it might get a bit harder for exploring variables or plotting data.
So the question is: Are there any projects or libraries out there to facilitate this kind of functionality, or which provide a good starting point for interacting with Python through JavaScript? Are there any pointers you guys can give me to make the experience for users as seamless as possible?
Thanks!

Related

does it make sense to use something like backbone for a single page application created using Raphael?

For the past six months or so I've been teaching myself Javascript by building a single page application that uses Raphael to create all of the elements the user sees and interacts with (see the landing page for the project for a small-scale example of what the tool will do). As I've been learning more about the language I've been coming across a lot of talk about using something like Backbone to structure the code such that it will be easier to read and maintain. While I'm not in love with the idea of taking what I've written and shoehorning it into an MVP paradigm (which would also require me to learn the paradigm), I am in favor of writing quality code that won't cause me headaches later. However, in the course of doing some google research, I've discovered that backbone doesn't seem to play nicely with SVG elements. True, there are some workarounds posted, but it seems to me that if I have to implement a kludgey work-around in order to get everything to work, the value of implementing Backbone in this case is lost as my code is still wonky, albeit in a different way. (ego compels me to point out that as it stands, the code is reasonably un-spagettified)
Given that I'm relatively new to Javascript, and to serious web programming in general, I'm posting this question here in the hopes of soliciting advice from persons more learned than I in matters such as these. If you were in my position, would you implement Backbone (or some equivalent) or would you proceed without it?
I would suggest you keep doing what you are doing. The website looks good and responsive. Do not change anything as long as you think the code is easily maintainable and readable. When you see heavy loading and spaghetti code, then you might consider about using some JS structure.

Decision about web application architecture

I am facing a decision about the web application architecture I am going to work on.
We are a small team and actually I will work on it alone (everybody work on something else).
This application will consist of front-end build on the ExtJS library
and it will use the model "load page, build GUI and never refresh".
On the web "desktop" there will be a lot of data windows, map views (using openlayers + GeoExt) and other stuff.
GUI should be flexible and allow every user to modify (and persist) the layout to fit his/her needs.
It should be possible to divide the application into modules / parts / ... and then let users in specific groups use only the specific modules. In other words, each group of users
can have different GUI available on the web "desktop".
Questions are:
First of all, is this approach good?
There will be a lot of AJAX calls from clients,
may be this could be a problem.
How to handle code complexity on client side?
So far I have decided to use dojo.require / dojo.provide feature and divide the client side code into modules
(for production they will be put together using dojo build system)
I am thinking about to use kind of IoC container on client side, but not sure which one yet.
It is very likely that I will write one for myself, it should not be difficult in dynamic language like JavaScript.
How to handle AJAX calls on server ?
Should I use WCF on server side ? Or just ordinary ashx handler ?
How to handle code complexity on server side ?
I want to use Spring.NET. May be this approach could help with modularity problem.
Data access - here I am pretty sure what to use:
For DAL classes I will use nHibernate. Then I compose them with business classes using Spring.NET.
I would really appreciate some advice about which way to go.
I know about a lot of technologies, but I have used only little part of it.
I don't have time to explore all of them and be fine with the decision.
We do this type of single page interface where I work on a pretty large scale for our clients. (Our site is not an internet site)
This seems to work pretty well for us. The more js you have the more difficult it gets to maintain, so have as many automated js tests as you can and try to break up your js logic in an mvc fashion. 4.0 is supposed to make this much easier.
Ext 4.0 has this built in if you are trying to limit the code you bring down. If you have the same users day after day, then I think it would be best to just bring all the source down (compressed and minified) and cache it.
We've found asmx to work really well. I have nothing against wcf, but last I looked it seemed like more trouble than it was worth. I know they have made many improvements recently. asmx just works though (with a few request header changes and managing the "d." on the client side).
Our Server side data access layer is pretty complex, but the interface for the ajax calls is pretty simple. You have not really given enough info to answer this part. I would start as simple as possible and refactor often.
We are also using nHibernate. Works fine for us. We have built a DDD model around it. It can take a lot of work to get that right though (not sure if we have it right after months of working at it).
If I were you I'd start with just extjs, your web service technology, and nHibernte.
I would recommend ASP.NET MVC 3 with Razor instead of a lot Javascript and calls to Service you can just do ajax calls to an Action in a Controller and that will let you have more maintainable code and use a IoC like Ninject. EF instead of NHibernate.
But it's your decision.
I would look into using a tool like Google Closure Compiler, especially if you're dealing with a very large project. I don't have too much experience with ExtJS, but large projects in JavaScript are hard and something like Closure Compiler tends to make it easier.

Need help determining how to approach building my project

I'd like to create something similar to a family tree online app (like geni.com). I'm unsure what languages I should use to build it with. My IT strong points aren't in programming and this project is going to require me to sit down and learn some languages. My problem right now is that I don't know what languages I should use.
So with the idea of a family tree online app in mind here are some of the specifications.
- I do not want to use flash.
- The app needs to be zoomable and scrollable (sort of like google maps)
- The app needs to be able to add content without reloading the page. Perhaps there's a little "+" sign and when I click it, I can add a tag/title/description/picture
- The app needs to be able to save your work for that user to retrieve later on.
- The layout that a user is able to create in is sort of widget based where the user can add a new bubble and then in that bubble they are able to add text or content.
I started programming this with HTML5 canvas and Javascript, but I'm stuck on creating a connection to the database that isn't directly from Javascript (because that seems very insecure to me). But I'm not just stumped on how to interact securely with the database (I don't even have a database picked out), but also I'm concerned that I won't be able to build out the app with just javascript and may need something else like ajax or something but I'm unfamiliar with what each language does nowadays.
If you are starting from scratch, then the best language to use is the one you are most comfortable with. Alternately, if you don't plan to be developing the whole thing yourself and you already have some other interested parties on board then the best language to use is the one that the majority of you are comfortable with. If it's just you and you do not yet have any favorites, then look around and play with a few - it's the only way to find out if you will actually like / be effective with them.
That being said, a few of the more likely candidates these days are:
JavaScript: Long gone are the days when this language was simply a way to put the D in DHTML. These days JavaScript is a viable client and server-side language. (Others here have already recommended Node.js -- I'd also recommend NPM (node package manager) to handle your dependencies). With a little bit of planning you can reuse most of your application code on both the client and the server side. On the downside, most of the server side technology is very new (only a few years old at most) and so you may find yourself implementing tools for use in your application rather than your application itself. Finding servers that support it will also be harder, again on account of it's age.
Perl: At the opposite end of the spectrum of age, we find Perl - the first commonly deployed language used to make web applications it still powers a great variety of useful websites out there (include new ones such as Pinboard.) The tools that are popular on CPAN have been vetted under fire. The good news is that it is not going anywhere anytime soon. The bad news is, you might have to search a little harder to find a module that supports [that newest, baddest thing that just came out yesterday].
PHP: The BASIC (or Perl, depending on who you ask) of the modern web, PHP was designed from the ground up to do one thing - make building dynamic web pages easier. Its popularity means that there is quite a lot of server support (PHP + Apache + MySQL is the Model T Ford of web servers -- everyone can afford one) and an enormous amount of pre-built code available for perusal. However, like BASIC, PHP's strength is also its greatest weakness. Almost anyone can write something that works in PHP ... how well it works depends on who wrote it. The caveat emptor that applies to all code snippets found on the web applies in spades to snippets of code written in PHP.
Python: The language that made programing fun again (at least for those who can see past the significant whitespace and lack of blocks / anonymous functions and overlook < 3.x's issues with non-ASCII out of the box.) It's a general-purpose, flexible and multi-paradigm language with quite a substantial standard library (but without .NET or Java's incredible bloat). In addition, quite a large amount of work has been done in it, so there is a good chance that what you need has been already developed by somebody else. Plus, it can make you fly.
Ruby (with or without Rails): The language that made the web fun, coupled, if you so desire, with the framework that made MVC cool. There is lots of documentation out there, and a great community, with many prebuilt tools (called gems) from which to pick and choose - free and cheap servers are not as common as their PHP counterparts, but they are likely to be of higher quality (when chosen at random).
All that being said, they are all great languages for web development. What matters is not what we think you should use ... but what you are most likely to be effective with. All of the languages listed above are mainstream (or will be in the next two years), easy-to-learn and easy-to-write languages. You cannot go wrong, no matter what you choose to start off with.
Alternately, if you want something a little more difficult, or less mainstream ... I am working with .NET applications at work, and with Lisp (SBCL)-based services in my spare time. I have heard great things about Lua and Java too ... there are at least two C++ web frameworks out there ... and I'm sure that there is somebody is having fun building a web service in COBOL with a FORTRAN backend. ;-)
As someone has already pointed out, you will need to work with a server side language as well. (Ruby, Python, PHP) You are exactly right there you should not be attempting a database connection via javascript in the browser.
You'll need to build out a server side application to handle the basic operations of your application.
I'd strongly recommend reading up on the MVC design pattern, and possibly looking into Ruby on Rails as your backend framework, it plays very nicely with ajax like features, and has a somewhat shorter learning curve, I believe, than some other frameworks / languages.
You will need server side scripts in a language like PHP or Ruby on Rails to interact with a database.
If you're already familiar with HTML5 and JavaScript, may I recommend using Node.JS? It's about the closest you'll get to what you already know with browser development. It can also hook in with database systems which are closer to the HTML5-suggested IndexedDB.
with that in mind...
If you're inexperienced with programming and programming languages, then the app you describe will involve a pretty big learning curve. While Flash and Flex have really nice interfaces to build apps with click-and-drag, the tools for HTML5 are much less mature.
That's not saying it's not possible with HTML5. Just that there's still some time to wait before people create tools to bring the app building process closer to what Adobe provides.

Issues with web application

The previous programmer left the website in pretty unusable state, and I am having difficulty modifying anything. I am new to web design so I don't know whether my skills are a mismatch to this kind of job or is it normal in the real industry to have websites like these
The Home page includes three frames
Each of these frames have their own javascript functions ( between <head>, and also call other common javascript functions (using <script src=..>
Excessive usage of document.all - in fact the elements are referred or accessed by document.all only.
Excessive usage of XSLT and Web Services - Though I know that using Web Services is generally considered a good design choice - is there any other way I can consume these services other than using xslt. For example, the menu is created using the data returned by a web method.
Every <div>, <td> and every other element has an id, and these id's are manipulated by the javascript functions, and then some appropriate web service and the xslt files are loaded based on these..
From the security perspective, he used T-SQL's for xml auto for most of the data that is returned by the web service - is it a good choice from the security standpoint to expose the table names and column names to the end user??
I am a lot confused about the state of the application itself. Should I learn about the intricacies that he has developed and continue working on it, or should I start rewriting everything? What I am perplexed a lot is the lack of alternatives - and whether this is the common way web projects are handled in the real world or was it an exception?
Any suggestions, any pointers are welcome. Thanks
No, it is not acceptable in this industry that people keep writing un-maintainable code.
My advice to you is to go up the chain and convince everyone that this needs to be rewritten. If they question you, find an external consultant with relevant web development skills to review the application (for 1 day).
Keeping this website as-is, because it 'works' is like keeping a working model Ford-T car on today's highways, very dangerous. Security and maintenance costs are likely the most persuading topics to convince anyone against keeping this site 'as-is'.
Next, get yourself trained, it will pay off if you can rewrite this application knowing the basics. Todays technology (asp.net MVC) allows you to implement core business value faster than trying to maintain this unconventionally written app.
Tough spot for an inexperienced developer (or any) to be left in. I think you have a few hard weeks a head of you where you really need to read up on the technologies involved to get a better understanding of them and what is best practice. You will also need to really dig down into the existing code to understand how it all hangs together.
When you done all that you really need to think about your options. Usually re-writing something from scratch (especially if it actually works) is a bad idea. This obviously depend on the size of the project, for a smaller projects with only a couple of thousand lines of code it might be OK. When looking at someone elses code it is also easy to overlook that all that weird shit going on could actually be fixes for valid requirements. Things often start out looking neat, but then the real words comes visiting.
You will need to present the business with time estimates for re-writing to see if that is an option at all, but I'm guessing you will need to accept the way things are and do your best with what you have. Maybe you could gradually improves things.
I would recommend moving the project to MVC3 and rewriting the XSLT portions to function using views and/or partial views with MVC. The Razor model binding syntax is very clean and should be able to quickly cleave out the dirty XSLT code and be left with just the model properties you would need.
I would then have those web services invoked from MVC serverside and for you to deserialize the object results into real objects (or even just use straight XQuery or Json traversing to directly pull stuff out for your model) and bind those to your views.
This could be a rather gargantuan leap for technology at your company though. Some places have aversion to change.
I'd guess this was written 6-7 years ago, and hacked on since then. Every project accumulates a certain amount of bubble gum and duct tape. Sounds like this one's got it bad. I suggest breaking this up into bite size chunks. I assume that the site is actually working right now? So you don't want to break anything, the "business" often thinks "it was working just fine when the last guy was here."
Get a feel for your biggest pain points for maintaining the project, and what you'll get the biggest wins from fixing. a rewrite is great, if you have the time and support. But if it's a complex site, there's a lot to be said for a mature application. Mature in the sense that it fulfills the business needs, not that it's good code.
Also, working on small parts will get you better acquainted with the project and the business needs, so when you start the rewrite you'll have a better perspective.

Processing language vs javascript?

I'm going to develop a comprehensive educational software which runs on the browser and has many visualization and simulation works (electrostatic and electromagnetic visualization, 2D and 3D).
Which language(Processing, javascript or something else) is the best toward my purpose?
The question is indeed broad but I will answer from the experience I've had.
Javascript is not really meant to do mathematical calculations, which is what might be necessary to calculate a lot of E&M phenomenon quickly (Especially if they are not represented as a closed form solution). It really goes into how much detail you want in your graphs as well (More steps = more calculations). You may find yourself needing to do more optimizations to make up for the performance difference.
I did some visualizations of antenna arrays (They had closed form solutions, only simple arrays) in Flash and it worked out ok. Javascript will definitely not be up to par with any 3D simulations you might want to do.
I wonder if Silverlight might be a better solution, because you may find more mathematics libraries for .NET than for Actionscript, that could save you a lot of work of writing the math out yourself (But you might end up doing this anyways because of the performance issues).
As others have suggested javascript is not that strong of a language when it comes to visualization.
Processing is a really good language for what you're trying to do, it's easy to learn and is Java based. Data visualization is built directly into the language, as well as temporal space (ie advance "1 tick" in time and have the visualization react to that.)
Also if you're interested in going that route I'd suggest picking up Visualizing Data which is pretty much a Processing primer.
Flash may be the more common application stack right now for what you are looking for, but Silverlight is looking primed to take the title from them based on the powerful features that it contains.
I would go Flex or Silverlight myself
Plenty of re-usable libraries
Native support for multimedia
Native support for graphics and animation
I'm a little late to the show, but what you want, has been implemented in JavaScript, and you'll find this incredibly useful. I recommend running it under Chrome as the JS processing engine is extremely fast. (You may even want to try Chrome 2 which is even faster)
http://ejohn.org/blog/processingjs/
http://ejohn.org/apps/processing.js/examples/basic/ (91 basic demos.)
http://ejohn.org/apps/processing.js/examples/topics/ (51 larger, topical, demos.)
http://ejohn.org/apps/processing.js/examples/custom/ (4 custom "in the wild" demos.)
See also: http://www.chromeexperiments.com/
I second LFSR Consulting's opinion: Processing is used a lot for educational purposes, it is free, and fast (Java is faster than Flash in general) and easy to learn, so you have faster results. It supports 3D, you can tackle Java libraries for simulation and computing, etc. And it has a great community! :-)
JavaScript is a bit light for such usage. JavaFX is hype, but it hasn't really 3D (although one used Java3D with it) and it is still a bit young.
Flash and Silverlight: no comment, not much experience in the field. OpenLazlo can be an alternative...
You really have two choices ActionScript in Flash or VB.NET/C#/other in Silverlight.
So first you need to decide which of these platforms you will target.
You may be able to split the problem into two parts, the user-interaction and display part, and the heavy calculations part.
If you can move the heavy calculations to a server then you can still show everything in javascript.
One difficulty with javascript is that it is interpreted and you will need to write more of the equations yourself, so there is a performance hit and development time, but it will work without any plugins, unless you don't want to do 3D in the canvas tag.
Flash and Silverlight may have better options, but then you are learning new languages and requiring plugins, depending on what version of Flash you want to use.
Check out processing.js, xcode, and iprocessing!
ProcessingJS is great for data visualization but lacks in interactivity.
You should probably try python. It is a really good language for educational and computational purposes it has a pretty decent community and the syntax is not so tough. Even though it was designed to for command line you can create front end gui's for it using some external package and it also provides packages like Scipy, Numpy and Matplotlib for advanced plotting and data visualization.

Categories