Let's say you use several different programming languages and frameworks in your infrastructure to handle large amounts of traffic etc.
Example Stack:
event driven API servers (using Scala, node.js, Ruby EM)
a standard full stack webapp (e.g. Rails)
(maybe more technologies)
When using different languages and frameworks I usually end up duplicating most of the model validations because every "customer entry point" needs to validate its input. This is of course a pain to keep in sync.
How would you handle this without something like CORBA?
Your best bet would be a framework that allows you to specify model validation in a language agnostic format, like JSON. You might end up with a validation schema of sorts, say:
{
"name": [
{
"validate": "length",
"minLength": 6,
"maxLength": 10
},
...
],
...
}
You would then have language-specific validators that could parse this format. The validators only need to be written once, and then you maintain a single schema for each model.
However, this is probably sounding a lot like CORBA/SOAP/Thrift/ProtocolBuffers/etc. at this point. That's because they were written to solve these types of problems, and you'll end up reinventing a few wheels if you write it yourself.
I would go on a "dictionary" of Regular Expressions.
Regular Expressions are supported by all the languages you counted - and - translating their string representation from one language to another can be done by a passing the expressions themselves through regular expressions...
To my observation - it's a lot less work then composing a parse and execute mechanism for each language...
Like advised before - you can save this "dictionary" of Reg-Exps in an agnostic format, such as JSON.
This narrows down the duplicated work to -
one source file of validation expressions that you maintain
per programming language:
converter from the main file to the format of the target language
thin mechanism of
reading the JSON,
selecting from it the configured checks
executing them
edge cases (if any)
Have fun :)
To add to #Nathan Ostgard's post, XML, XSD and, when necessary, XSLT might work as well. The advantage to this would be a) XSD has the simple validation built into it b) most languages have good support for this c) you wouldn't have to write validation in each language; stuff that isn't handled in the schema could be written once in XSLT (with the caveat that XSLT implementations tend to vary :) )
If you want to go the way of full validation, I'd use SOAP. At least, as long as you have SOAP libraries, all you need to feed them is the WSDL for your interfaces.
Related
Disclaimer: My experience is mainly development with statically typed languages, and even if I understand how dynamic types works, I'm not very familiar with the common practices, neither I'm very aware of the usual tips and tricks yet.
I recently started to work in a project where we use serverless and deploy some lambdas in AWS with javascript. After deep diving the existing code I found out a common practice that shocked me a bit:
When a lambda is invoked and it receives a JSON payload, there is an initial pre-processing, converting all the values in the object to strings.
const mapValues = require('lodash/mapValues')
const escape = require('validator/lib/escape')
...
const body = mapValues(requestBody, value => escape('' + value))
...
Then after that pre-processing, the real work begins (validation, processing, invoking other services, etc...)
My first thoughts this are:
Benefit: Helps to reduce the cognitive load of the possible different types, enabling to assume that everything will be always a string.
Downside: Extra complexity dealing with non-string values, such as numbers.
Q1: Are there any other benefits and pitfalls to using this approach?
Q2: Could this be considered a bad practice? If so, why? (facts, not opinions please)
Thanks in advance! :)
Interesting question. I suggest you to ask the author if they 're available. Here are some of my thoughts:
I argue having string values does not reduce but increase cognitive load, as you have to watch out to convert them when doing basic arithmetics. I am sure all of us have seen at least once 1+1 being 11.
Also, handling every data as strings is a so bad practice it even has its own mock name: stringly typed. This is the cookbook example of that.
The only valid reason I can possibly imagine is to prevent various injection attacks against the service. In some scenarios, if the user is allowed to send in arbitrary json, it is possible to make the service execute code paths that it would not do normally. (something whacky like {"__proto__":[],"length":1,"0":"foo"} coerects to "foo" but is typeof object, this could bypass some flawed validation logic. mongodb is also prone to some attacks alike). But even in this case, proper validation would be much better than converting every value to string.
I have a grammar for a domain specific language, and I need to create a javascript code editor for that language. Are there any tools that would allow me to generate
a) a javascript incremental parser
b) a javascript auto-complete / auto-suggest engine?
Thanks!
An Example of implementing content assist (auto-complete)
using Chevrotain Javascript Parsing DSL:
https://github.com/SAP/chevrotain/tree/master/examples/parser/content_assist
Chevrotain was designed specifically to build parsers used (as part of) language services tools in Editors/IDEs.
Some of the relevant features are:
Automatic Error Recovery / Fault tolerance because editors and IDEs need to be able to handle 'mostly valid' inputs.
Every Grammar rule may be used as the starting rule as an Editor/IDE may only want to implement incremental parsing for performance reasons.
You may want jison, a js parser generator. In terms of auto-complete / auto-suggest...most of the stuff out there I know if more based on word completion rather than code completion. But once you have a parser running I don't think that part is too difficult..
This is difficult. I'm doing the same sort of thing myself.
One approach is:
You need is a parser which will give you an array of the currently possible ASTs for the text up until the token before the current cursor position.
From there you can see the next token can be of a number of types (usually just one), and do the completion, based on the partial text.
If I ever get my incremental parser working, I'll send a link.
Good luck, and let me know if you find a package which does this.
Chris.
I was wondering about this if people have strong opinions about the best way to generate HTML on the fly, especially with Ajax based applications.
Do you just create the HTML code using server side scripting and then send it out to the page or perhaps just return a JSON string and let Javascript do the job.
In my personal opinion, the first way ties the presentation layer way too much to the logic and makes it harder to change and a nightmare to maintain. The second way, although is my preferred method, also becomes a nightmare to maintain when the complexity of the project grows.
I was thinking of using a Javascript templating system as another layer, just to make the code more robust and less rigid. Anyone has good ideas of a light and really good JS templating system?
http://ejohn.org/blog/javascript-micro-templating/ is a devilishly brilliant hack for this. End result is very clean.
I too prefer a JSON response with client-side HTML creation logic.
Unfortunately, most real-world client-side HTML writing scripts are broken, containing many HTML-injection flaws that can easily become cross-site-scripting security holes. I think the belief is that because you're talking to your own server rather than directly to a hostile user you're somehow ‘safe’ and can get away without correctly strings when interpolating them into HTML. Which is of course nonsense.
I always see stuff like:
$('#mydiv').append('<em>Deleted '+response.title+'!</em>');
or:
mydiv.innerHTML= '<p>Renamed to '+response.name+'</p>;
or indeed Resig's microtemplating hack, where there is no HTML-escaping done by default. Come on, people! We've only just started cleaning up the legacy of broken PHP scripts serving up server-side XSS, now you want to introduce a whole new massive range of client-side XSS exploits?
Sigh. That's the Lure Of Strings. We think we understand them and can sling them together willy-nilly. But strings are treacherous, with hidden contexts and escaping requirements. If you must generate HTML on the client side you will need a function like this:
function h(s) {
return s.split('&').join('&').split('<').join('<').split('"').join('"');
}
mydiv.innerHTML= '<p>Renamed to '+h(response.name)+'</p>;
But personally I prefer DOM methods. Like with parameterisation for SQL, using the DOM methods takes the string-slinging out of the equation by talking raw strings directly to the components that will consume them. OK, the problem with the DOM is that it's rather verbose:
var p= document.createElement('p');
p.appendChild(document.createTextNode('Renamed to '+response.name))
mydiv.appendChild(p);
But you can always define helper functions to cut down on that, eg.:
mydiv.appendChild(makeElement('p', {}, 'Renamed to'+response.name));
(The new element creation stuff in jQuery 1.4 uses a similar style.)
+1 year ago, we started a new web app, and needed a way to render HTML from JSON data, in the browser.
We wanted it as fast as an XML/XSLT transformation.
Our answer to that was the JS template engine PURE.
Unlike most of the JS templating libs around, it keeps the HTML untouched(no strange tags at all) and except a few notations, it doesn't bring a new language to learn, only JS and HTML.
The way I use it:
Build the static HTML directly in the page
Then add the JS logic step by step, and the HTML becomes alive progressively
Once you get used to it, both HTML and JS can have a safe separate development life, and can be split between a designer and a JS developer job
We had a system where a lot of data was being passed as JSON from the server and then handled through a javascript templating engine on the client side. In .Net 4.0 (maybe even 3.5 sp1, i am not sure), this can be done using Client Templates.
I would prefer passing JSON over sending html. Its always good to keep data and view separate.
If you want to preserve the MVC framework, you should let your template framework do the templating.
The AJAX should only perform the server request, which performs all DB and business logic and then returns the URL to the template that should be loaded.
During my routine work, i happened to write the chained javascript function which is something like LINQ expression to query the JSON result.
var Result = from(obj1).as("x").where("x.id=5").groupby("x.status").having(count("x.status") > 5).select("x.status");
It works perfectly and give the expected result.
I was wondering this looks awesome if the code is written like this (in a more readable way)
var Result = from obj1 as x where x.status
groupby x.status having count(x.status) > 5
select x.status;
is there a way to achieve this??
Cheers
Ramesh Vel
No. JavaScript doesn't support this.
But this looks quite good too:
var Result = from(obj1)
.as("x")
.where("x.id=5")
.groupby("x.status")
.having(count("x.status") > 5)
.select("x.status");
Most people insist on trying to metaprogram from inside their favorite language. That doesn't work if the language doesn't support metaprogramming well; other answers have observed that JavaScript does not.
A way around this is to do metaprogramming from outside the language, using
program transformation tools. Such tools can parse source code, and carry out arbitrary transformations on it (that's what metaprogramming does anyway) and then spit the revised program.
If you have a general purpose program transformation system, that can parse arbitrary languages, you can then do metaprogramming on/with whatever language you like. See our DMS Software Reengineering Toolkit for such a tool, that has robust front ends for C, C++, Java, C#, COBOL, PHP, and ECMAScript and a number of other programming langauges, and has been used for metaprogramming on all of these.
In your case, you want to extend the JavaScript grammar with new syntax for SQL queries, and then transform them to plain JavaScript. (This is a lot like Intentional Programming)
DMS will easily let you build a JavaScript dialect with additional rules, and then you can use its program transformation capabilities to produce the equivalent standard Javascript.
Having said, that, I'm not a great fan of "custom syntax for every programmer on the planet" which is where Intentional Programming leads IMHO.
This is a good thing to do if there is a large community of users that would find this valuable. This idea may or may not be one of them; part of the problem is you don't get to find out without doing the experiment, and it might fail to gain enough social traction to matter.
although not quite what you wanted, it is possible to write parsers in javascript, and just parse the query (stored as strings) and then execute it. e.g.,using libraries like http://jscc.jmksf.com/ (no doubt there are others out there) it shouldnt be too hard to implement.
but what you have in the question looks great already, i m not sure why you'd want it to look the way you suggested.
Considering that this question is asked some years ago, I will try to add more to it based on the current technologies.
As of ECMAScript 6, metaprogramming is now supported in a sense via Symbol, Reflect and Proxy objects.
By searching on the web, I found a series of very interesting articles on the subject, written by Keith Kirkel:
Metaprogramming in ES6: Symbols and why they're awesome
In short, Symbols are new primitives that can be added inside an object (without practically being properties) and are very handy for passing metaprogramming properties to it among others. Symbols are all about changing the behavior of existing classes by modifying them (Reflection within implementation).
Metaprogramming in ES6: Part 2 - Reflect
In short, Reflect is effectively a collection of all of those “internal methods” that were available exclusively through the JavaScript engine internals, now exposed in one single, handy object. Its usage is analogous to the Reflection capabilities of Java and C#. They are used to discover very low level information about your code (Reflection through introspection).
Metaprogramming in ES6: Part 3 - Proxies
In short, Proxies are handler objects, responsible for wrapping objects and intercepting their behaviors through traps (Reflection through intercession).
Of course, these objects provide specific metaprogramming capabilities, much more restrictive compared to metaprogramming languages, but still can provide handy ways of basic metaprogramming, mainly through Reflection practices, in fact.
In the end, it is worth mentioning that there is some worth-noticing ongoing research work on staged metaprogramming in JavaScript.
Well, in your code sample:
var Result = from(obj1)
.as("x")
.where("x.id=5")
.groupby("x.status")
.having(count("x.status") > 5)
.select("x.status");
The only problem I see (other than select used as an identifier) is that you embed a predicate as a function argument. You'd have to make it a function instead:
.having(function(x){ return x.status > 5; })
JavaScript has closures and dynamic typing, so you can do some really nifty and elegant things in it. Just letting you know.
In pure JS no you can not. But with right preprocessor it is possible.
You can do something similar with sweet.js macros or (God forgive me) GPP.
Wat you want is to change the javascript parser into an SQL parser. It wasn't created to do that, the javascript syntax doesn't allow you to.
What you have is 90% like SQL (it maps straight onto it), and a 100% valid javascript, which is a great achievement. My answer to the question in the title is: YES, metaprogramming is possible, but NO it won't give you an SQL parser, since it's bound to use javascript grammar.
Maybe you want something like JSONPath if you've got JSON data. I found this at http://www.json.org/. Lots of other tools linked to from there if it's not exactly what you need.
(this is being worked on as well: http://docs.dojocampus.org/dojox/json/query)
I have a web application that uses TONS of javascript, and as usual, there are a lot of textual constants that will be displayed to the user embedded in the code itself.
What do you think is the best way to make this localizable?
I know I need to take those strings off of the code and replace them with constants, which will be defined into some external place.
For the server side, ASP.Net provides some very neat capabilities for dealing with this.
What's the best to do this in Javascript?
The best idea I have is to have a JS file with ALL the string constants of the JS of the site (i'd have different copies of this, for each language), and then on each page, I include this script first, before all the others.
This seems like the most centralized way, that also wastes the least bandwidth.
Are there any other better approaches?
Thanks!
here's how we did it (in ASP.net), pretty much along the lines of what you've mentioned:
1) Created two javascript files: one which defines all javascript functions/DOM manipulations as required by the site, and, second called Messages.js: this defines all the string literals that need to be localized, something like var ALERT_MSG = "Alert message in english".
2) Created different version of the Messages.js, one for each locale that we are supporting and localized the strings. The localized js files were named using messages.locale.js naming convention (for eg. messages.fr-FR.js).
3) Included the js files within the "ScriptManager" and provided the ResourceUICultures for the Messages.js file: this ensures that the correct localized file is embedded in the html output (if you are not using ASP.net you can build this basic functionality by doing some culture sniffing and including the appropriate js file).
4) Voila!
Your approach makes sense. Details:
I'd have the strings for each language in an object.
localized={"cat":"chat","dog":"chien"};
Then in code:
localized["cat"]
The quotations around of the keys and the array notation (rather than the more common object dot notation) are to avoid collisions with JavaScript reserved words.
There is a gettext library but I haven't used it.
Your approach sounds good enough.
If you have lots of strings and you are concerned about the bulkiness of the file you may want to consider a script that creates a single javascript file for each language by concatenating the code javascript and the locale javascript and then applying something like Minify.
You'll waste some CPU cycles on publishing but you'll save some round trips...
There's a library for localizing JavaScript applications: https://github.com/wikimedia/jquery.i18n
The strings are stored in JSON files, as pretty much everybody else suggests, but it has a few more features:
It can do parameter replacement, supports gender (clever he/she handling), number (clever plural handling, including languages that have more than one plural form), and custom grammar rules that some languages need.
The only requirement is jQuery.