Add syntax to JavaScript with Clojure - javascript

I want to add new syntax to my JavaScript files much like Sweet.js, but using Clojure/ClojureScript to do the transformation.
Here is what I would like to do:
//original
specialFunction add(a, b) {
return a + b;
}
//transformed
function add(a, b) {
return a + b;
}
registerSpecialFunction({
name: "add",
args: ["a", "b"],
func: add
});
Correct me if I'm wrong, but I think it would work best by:
parse the JS file into an AST
do the transformation
print the resulting AST back out as JavaScript
Any idea on how to do parts 1 and 3? Do I even have the right idea here?

Just as a heads up, this may be a non-trivial task, but probably a great learning exercise.
You'll have to re-implement the concept or macros from Sweet.js or the idea of custom parser extensions from Babylon (parser for Babel) if you want the same control from Clojure.
Either way, you'll need to write a parser that understands a superset of JavaScript's syntax. You might want to look at parser generators such as instaparse (Clojure) and peg.js (JavaScript).
Then you need to decide whether you want to make a fixed number of additions to the language grammar (like Babel) or allow macros to define their own grammar extensions/replacement rules (like Sweet.js). At this point, you'll need to write some kind of engine for transforming the AST generated by your parser.
Macros can be implemented in a number of ways, everything from replacement rules like you'd find in C and C++ to full blown compile-time evaluated functions that work directly with the AST like you'd find in Clojure.
After parsing and transforming the AST with this new tool, you'll need to transform it into a valid JavaScript AST. It'll make things easier to maintain compatibility with the ESTree specification as this will allow you to use tools like escodegen to actually generate the JavaScript code from the AST itself.
Of course, piggybacking tools like peg.js and escodegen is only possible if you're writing your tool as ClojureScript and compiling and running it against NodeJS. The other option is to find compatible tools within the JVM ecosystem and use them with JVM compiled Clojure instead.
The JavaScript ecosystem has a range of good tools available for parsing, transforming and generating ES code (have a look through the Babel packages for example/inspiration) but you'll have to remember that if you are writing ClojureScript and running it under Node, you are in fact creating a JavaScript executable and it might have just been easier to start with JavaScript in the first place.

Related

How to convert LLVM IR br back to a while loop

I am wondering how to do sort of transpilation of source to source using LLVM at a high level. Given that LLVM converts while loops to using br and the like, I am wondering how to then take that specific IR chunk and convert it back to a while loop in a language such as JavaScript.
C while loop -> LLVM IR -> JavaScript while loop
This article suggests that Emscripten converts LLVM code to JavaScript, so it probably handles this somewhere.
I'm wondering just the general strategy for converting it, if there is one. It seems a bit tricky from a distance, figuring out the statements to piece together a while loop from IR.
During the translation from C to LLVM IR, instructions that are deemed necessary can be decorated with Metadata, this metadata can then be used to convert LLVM IR to JavaScript, e.g indicating if the circular branching between basic blocks is a while loop or not (This information is present in the C context). See Intrinsics & Metadata Attributes.
For more information regarding LLVM Metadata see LLVM-Metadata.
In Emscripten the algorithm of recreating high-level language structures is called Relooping and is described in this parer. I'm not sure, though, that it is up-to-date information, but it probably answers your question.

Javascript operator overloading on lists

JavaScript does not support operator overloading. Matrix libraries in JavaScript could not simplify notation. I would like to create operator overloading with a simple trick by adding using syntax like z = x++y. This is not a valid statement in JavaScript.
That is why I would like to create an include method which will parse existing JavaScript files and replace those statements with actual JavaScript code. This is somehow related to coffescript where the compiler is inside JavaScript. What would be the best way to approach that.
I have string manipulation solution:
"z=x++y;".replace(/(.*)=(.*)\+\+(.*)/i,"for(var _i_=0;_i_<$1.length;_i_++){ $1[i] = $2[i]+$3[i]}")
Example run:
for(var _i_=0;_i_<c.length;_i_++){ c[i] = data[0][i]+data[1][i];}
Obtaining Matlab, numpy like environment in JavaScript would be very convenient for easily deploying scientific models as web applications and avoiding computational burden in server side. Also, parallelization would be as easy as opening another browser tab from somewhere.

Java-based interpreter for JavaScript

As a project in school i have to make a JavaScript interpreter. (Everything incl. the entire backend has to be made by me).
Everything has to be written in Java - i use ANTLR for parsing and generating ASTs.
currently i can parse some .js code into an AST - and therefore need to translate this AST into som kind of intermediate-representation that can be executed on a bytecode machine.
I have some experience writing compilers for statically typed languages, but im very much in doubt how to proceed from here since JS is a dynamically typed language.
If you can give me some good advices on how to proceed i would be gratefull!
Personally i think i have to make the bytecode-machine first and then make the IR fit this machine afterwards. Unfortunatly i cant really find any good tutorials on how to write a bytecode-machine.
PS. im familiar with following books on the topic :
"modern compiler implementation in Java (Appel)",
"Programming language processors in Java (Watt & Brown)",
"Language implementation patterns (Parr)"
Regards Sune
If you only want to execute the javascript you do not need to transform the AST into IR and then into (some?) bytecode that would also force you to do a bytecode executer.
Why not just execute the javascript AST in a java "engine"? You can store all values as a Map<String, Object> and interpret them as you walk the AST. A new function get an environment/context (a new Map<...>).
If you cannot find a value in the current context you will have to fall back on the global context (=Map).
For the "dynamic" behaviour: If you need an double for an addition you only parse the Object.toString() value to an double using the standard way (more dynamic than that is hard to get :) ):
String value = contextMap.get(key);
Double dvalue = Double.parseDouble(value.toString());
....

Is metaprogramming possible in Javascript?

During my routine work, i happened to write the chained javascript function which is something like LINQ expression to query the JSON result.
var Result = from(obj1).as("x").where("x.id=5").groupby("x.status").having(count("x.status") > 5).select("x.status");
It works perfectly and give the expected result.
I was wondering this looks awesome if the code is written like this (in a more readable way)
var Result = from obj1 as x where x.status
groupby x.status having count(x.status) > 5
select x.status;
is there a way to achieve this??
Cheers
Ramesh Vel
No. JavaScript doesn't support this.
But this looks quite good too:
var Result = from(obj1)
.as("x")
.where("x.id=5")
.groupby("x.status")
.having(count("x.status") > 5)
.select("x.status");
Most people insist on trying to metaprogram from inside their favorite language. That doesn't work if the language doesn't support metaprogramming well; other answers have observed that JavaScript does not.
A way around this is to do metaprogramming from outside the language, using
program transformation tools. Such tools can parse source code, and carry out arbitrary transformations on it (that's what metaprogramming does anyway) and then spit the revised program.
If you have a general purpose program transformation system, that can parse arbitrary languages, you can then do metaprogramming on/with whatever language you like. See our DMS Software Reengineering Toolkit for such a tool, that has robust front ends for C, C++, Java, C#, COBOL, PHP, and ECMAScript and a number of other programming langauges, and has been used for metaprogramming on all of these.
In your case, you want to extend the JavaScript grammar with new syntax for SQL queries, and then transform them to plain JavaScript. (This is a lot like Intentional Programming)
DMS will easily let you build a JavaScript dialect with additional rules, and then you can use its program transformation capabilities to produce the equivalent standard Javascript.
Having said, that, I'm not a great fan of "custom syntax for every programmer on the planet" which is where Intentional Programming leads IMHO.
This is a good thing to do if there is a large community of users that would find this valuable. This idea may or may not be one of them; part of the problem is you don't get to find out without doing the experiment, and it might fail to gain enough social traction to matter.
although not quite what you wanted, it is possible to write parsers in javascript, and just parse the query (stored as strings) and then execute it. e.g.,using libraries like http://jscc.jmksf.com/ (no doubt there are others out there) it shouldnt be too hard to implement.
but what you have in the question looks great already, i m not sure why you'd want it to look the way you suggested.
Considering that this question is asked some years ago, I will try to add more to it based on the current technologies.
As of ECMAScript 6, metaprogramming is now supported in a sense via Symbol, Reflect and Proxy objects.
By searching on the web, I found a series of very interesting articles on the subject, written by Keith Kirkel:
Metaprogramming in ES6: Symbols and why they're awesome
In short, Symbols are new primitives that can be added inside an object (without practically being properties) and are very handy for passing metaprogramming properties to it among others. Symbols are all about changing the behavior of existing classes by modifying them (Reflection within implementation).
Metaprogramming in ES6: Part 2 - Reflect
In short, Reflect is effectively a collection of all of those “internal methods” that were available exclusively through the JavaScript engine internals, now exposed in one single, handy object. Its usage is analogous to the Reflection capabilities of Java and C#. They are used to discover very low level information about your code (Reflection through introspection).
Metaprogramming in ES6: Part 3 - Proxies
In short, Proxies are handler objects, responsible for wrapping objects and intercepting their behaviors through traps (Reflection through intercession).
Of course, these objects provide specific metaprogramming capabilities, much more restrictive compared to metaprogramming languages, but still can provide handy ways of basic metaprogramming, mainly through Reflection practices, in fact.
In the end, it is worth mentioning that there is some worth-noticing ongoing research work on staged metaprogramming in JavaScript.
Well, in your code sample:
var Result = from(obj1)
.as("x")
.where("x.id=5")
.groupby("x.status")
.having(count("x.status") > 5)
.select("x.status");
The only problem I see (other than select used as an identifier) is that you embed a predicate as a function argument. You'd have to make it a function instead:
.having(function(x){ return x.status > 5; })
JavaScript has closures and dynamic typing, so you can do some really nifty and elegant things in it. Just letting you know.
In pure JS no you can not. But with right preprocessor it is possible.
You can do something similar with sweet.js macros or (God forgive me) GPP.
Wat you want is to change the javascript parser into an SQL parser. It wasn't created to do that, the javascript syntax doesn't allow you to.
What you have is 90% like SQL (it maps straight onto it), and a 100% valid javascript, which is a great achievement. My answer to the question in the title is: YES, metaprogramming is possible, but NO it won't give you an SQL parser, since it's bound to use javascript grammar.
Maybe you want something like JSONPath if you've got JSON data. I found this at http://www.json.org/. Lots of other tools linked to from there if it's not exactly what you need.
(this is being worked on as well: http://docs.dojocampus.org/dojox/json/query)

Parse JavaScript to instrument code

I need to split a JavaScript file into single instructions. For example
a = 2;
foo()
function bar() {
b = 5;
print("spam");
}
has to be separated into three instructions. (assignment, function call and function definition).
Basically I need to instrument the code, injecting code between these instructions to perform checks. Splitting by ";" wouldn't obviously work because you can also end instructions with newlines and maybe I don't want to instrument code inside function and class definitions (I don't know yet). I took a course about grammars with flex/Bison but in this case the semantic action for this rule would be "print all the descendants in the parse tree and put my code at the end" which can't be done with basic Bison I think. How do I do this? I also need to split the code because I need to interface with Python with python-spidermonkey.
Or... is there a library out there already which saves me from reinventing the wheel? It doesn't have to be in Python.
Why not use a JavaScript parser? There are lots, including a Python API for ANTLR and a Python wrapper around SpiderMonkey.
JavaScript is tricky to parse; you need a full JavaScript parser.
The DMS Software Reengineering Toolkit can parse full JavaScript and build a corresponding AST.
AST operators can then be used to walk over the tree to "split it". Even easier, however, is to apply source-to-source transformations that look for one surface syntax (JavaScript) pattern, and replace it by another. You can use such transformations to insert the instrumentation into the code, rather than splitting the code to make holds in which to do the insertions. After the transformations are complete, DMS can regenerate valid JavaScript code (complete with the orignal comments if unaffected).
Why not use an existing JavaScript interpreter like Rhino (Java) or python-spidermonkey (not sure whether this one is still alive)? It will parse the JS and then you can examine the resulting parse tree. I'm not sure how easy it will be to recreate the original code but that mostly depends on how readable the instrumented code must be. If no one ever looks at it, just generate a really compact form.
pyjamas might also be of interest; this is a Python to JavaScript transpiler.
[EDIT] While this doesn't solve your problem at first glance, you might use it for a different approach: Instead of instrumenting JavaScript, write your code in Python instead (which can be easily instrumented; all the tools are already there) and then convert the result to JavaScript.
Lastly, if you want to solve your problem in Python but can't find a parser: Use a Java engine to add comments to the code which you can then search for in Python to instrument the code.
Why not try a javascript beautifier?
For example http://jsbeautifier.org/
Or see Command line JavaScript code beautifier that works on Windows and Linux
Forget my parser. https://bitbucket.org/mvantellingen/pyjsparser is great and complete parser. I've fixed a couple of it's bugs here: https://bitbucket.org/nullie/pyjsparser

Categories