JavaScript log parser project: Bad idea? - javascript

There are numerous log files that I have to review daily for my job. Several good parsers already exist for these log files but I have yet to find exactly what I want. Well, who could make something more tailored to you than you, right?
The reason I am using JavaScript (other than the fact that I already know it) is because it's portable (no need to install anything) but at the same time cross-platform accessible. Before I invest too much time in this, is this a terrible method of accomplishing my goal?
The input will be entered into a text file, delimited by [x] and the values will be put into an array to make accessing these values faster than pulling the static content.
Any special formatting (numbers, dates, etc) will be dealt with before putting the value in the array to prevent a function from repeating this step every time it is used.
These logs may contain 100k+ lines which will be a lot for the browser to handle. However, each line doesn't contain a ton of information.
I have written some of it already, but with even 10,000 lines it's starting to run slow and I don't know if it's because I wasn't efficient enough or if this just cannot be effectively done. I'm thinking this is because all the data is in one giant table. I'd probably be better off paginating it, but that is less than desirable.
Question 1: Is there anything I failed to mention that I should consider?
Question 2: Would you recommend a better alternative?
Question 3: (A bit off topic, so feel free to ignore). Instead of copy/pasting the input, I would like to 'open' the log file but as far as I know JavaScript cannot do this (for security reasons). Can this be accomplished with a input="file" without actually having a server to upload to? I don't know how SSJS works, but it appears that I underestimated the limitations of JavaScript.
I understand this is a bit vague, but I'm trying to keep you all from having to read a book to answer my question. Let me know if I should include additional details. Thanks!

I think JavaScript is an "ok" choice for this. Using a scripting language to parse log files for personal use is a perfectly sane decision.
However, I would NOT use a browser for this. Web browsers place limitations on how long a bit of javascript can run, or on how many instructions it is allowed to run, or both. If you exceed these limits, you'll get something like this:
Since you'll be working with a large amount of data, I suspect you're going to hit this sooner or later. This can be avoided by clever use of setTimeout, or potentially with web workers, but that will add complexity to your project. This is probably not what you want.
Be aware that JavaScript can run outside of browsers as well. For instance, Windows comes with the Windows Script Host. This will let you run JavaScript from the command prompt, without needing a browser. You won't get the "Script too long" error. As an added bonus, you will have full access to the file system, and the ability to pass command-line arguments to your code.
Good luck and happy coding!

To answer your top question in bold: No, it is not a terrible idea.
If JS is the only language you know, you want to avoid setting up any dependencies, and you want to stay platform-independent... JavaScript seems like a good fit for your particular case.
As a more general rule, I would never use JS as a language to write a desktop app. Especially not for doing a task like log parsing. There are many other languages which are much better suited to this type of problem, like Python, Scala, VB, etc. I mention Python and Scala because of their script-like behaviour and minimal setup requirements. Python also has very similar syntax to JS so it might be easier to pick up then other languages. VB (or any .NET language) would work too if you have a Visual Studio license because of it's easy to use GUI builder if that suits your needs better.
My suggested approach: use an existing framework. There are hundreds, if not thousands of log parsers out there which handle all sorts of use-cases and different formats of logs that you should be able to find something close to what you need. It may just take a little more effort than Google'ing "Log Parsers" to find one that works. If you can't find one that suits your exact needs and you are willing to spend time making your own, you should use that time instead to contribute to one of the existing ones which are open source. Extending an existing code base should always be considered before trying to re-invent the wheel for the 10th gillion time.

Given your invariants "javascript, cross-platform, browser ui, as fast as possible" I would consider this approach:
Use command line scripts (windows: JScript; linux: ?) to parse log files and store 'clean'/relevant data in a SQLite Database (fall back: any decent scripting language can do this, the ready made/specialized tools may be used too)
Use the SQLite Manager addon to do your data mining with SQL
If (2) gets clumsy - use the SQLite Manager code base to 'make something more tailored'
Considering your comment:
For Windows-only work you can use the VS Express edition to write an app in C#, VB.NET, C++/CLI, F#, or even (kind of) Javascript (Silverlight). If you want to stick to 'classic' Javascript and a browser, write a .HTA application (full access to the local machine) and use ADO data(base) access and try to get the (old) DataGrid/Flexgrid controls (they may be installed already; search the registry).

Related

How can I obfuscate the client side source code of my ES6 / React / Redux / Electron project? [duplicate]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I want to make a JavaScript application that's not open source, and thus I wish to learn how to can obfuscate my JS code? Is this possible?
Obfuscation:
Try YUI Compressor. It's a very popular tool, built, enhanced and maintained by the Yahoo UI team.
You may also use:
Google Closure Compiler
UglifyJS
UPDATE: This question was originally asked on 2008, and The mentioned technologies are deprecated. you can use:
terser - more information in web.dev.
Private String Data:
Keeping string values private is a different concern, and obfuscation won't really be of much benefit. Of course, by packaging up your source into a garbled, minified mess, you have a light version of security through obscurity. Most of the time, it's your user who is viewing the source, and the string values on the client are intended for their use, so that sort of private string value isn't often necessary.
If you really had a value that you never wanted a user to see, you would have a couple of options. First, you could do some kind of encryption, which is decrypted at page load. That would probably be one of the most secure options, but also a lot of work which may be unnecessary. You could probably base64 encode some string values, and that would be easier.. but someone who really wanted those string values could easily decode them. Encryption is the only way to truly prevent anyone from accessing your data, and most people find that to be more security than they need.
Sidenote:
Obfuscation in Javascript has been known to cause some bugs. The obfuscators are getting a little better about it, but many outfits decide that they see enough benefit from minifying and gzipping, and the added savings of obfuscation isn't always worth the trouble. If you're trying to protect your source, maybe you'll decide that it's worth your while, just to make your code harder to read. JSMin is a good alternative.
I'm surprised no one has mentioned Google's Closure Compiler. It doesn't just minify/compress, it analyzes to find and remove unused code, and rewrites for maximum minification. It can also do type checking and will warn about syntax errors.
JQuery recently switched from YUI Compresser to Closure Compiler, and saw a "solid improvement"
Obfuscation can never really work. For anyone who really wants to get at your code, it's just a speed bump. Worse, it keeps your users from fixing bugs (and shipping the fixes back to you), and makes it harder for you to diagnose problems in the field. Its a waste of your time and money.
Talk to a lawyer about intellectual property law and what your legal options are. "Open Source" does not mean "people can read the source". Instead, Open Source is a particular licensing model granting permission to freely use and modify your code. If you don't grant such a license then people copying your code are in violation and (in most of the world) you have legal options to stop them.
The only way you can really protect your code is to not ship it. Move the important code server-side and have your public Javascript code do Ajax calls to it.
See my full answer about obfuscators here.
You can obfuscate the javascript source all you want, but it will always be reverse-engineerable just by virtue of requiring all the source code to actually run on the client machine... the best option I can think of is having all your processing done with server-side code, and all the client code javascript does is send requests for processing to the server itself. Otherwise, anyone will always be able to keep track of all operations that the code is doing.
Someone mentioned base64 to keep strings safe. This is a terrible idea. Base64 is immediately recognizable by the types of people who would want to reverse engineer your code. The first thing they'll do is unencode it and see what it is.
There are a number of JavaScript obfuscation tools that are freely available; however, I think it's important to note that it is difficult to obfuscate JavaScript to the point where it cannot be reverse-engineered.
To that end, there are several options that I've used to some degree overtime:
YUI Compressor. Yahoo!'s JavaScript compressor does a good job of condensing the code that will improve its load time. There is a small level of obfuscation that works relatively well. Essentially, Compressor will change function names, remove white space, and modify local variables. This is what I use most often. This is an open-source Java-based tool.
JSMin is a tool written by Douglas Crockford that seeks to minify your JavaScript source. In Crockford's own words, "JSMin does not obfuscate, but it does uglify." It's primary goal is to minify the size of your source for faster loading in browsers.
Free JavaScript Obfuscator. This is a web-based tool that attempts to obfuscate your code by actually encoding it. I think that the trade-offs of its form of encoding (or obfuscation) could come at the cost of filesize; however, that's a matter of personal preference.
What i would do:
A. Troll the hacker!
This is will be in the second part my fake/obfuscated secret javascript code LAUNCHER.
The one you see in the source code.
What does this code?
loads the real code
sets a custom header
posts a custom variable
var ajax=function(a,b,d,c,e,f){
e=new FormData();
for(f in d){e.append(f,d[f]);};
c=new XMLHttpRequest();
c.open('POST',a);
c.setRequestHeader("Troll1","lol");
c.onload=b;
c.send(e);
};
window.onload=function(){
ajax('Troll.php',function(){
(new Function(atob(this.response)))()
},{'Troll2':'lol'});
}
B. Obfuscate the code a little
What is that?
thats the same code as above in base64
this is not the SECRET javascript code
(new Function(atob('dmFyIGFqYXg9ZnVuY3Rpb24oYSxiLGQsYyxlLGYpe2U9bmV3IEZvcm1EYXRhKCk7Zm9yKGYgaW4gZCl7ZS5hcHBlbmQoZixkW2ZdKTt9O2M9bmV3IFhNTEh0dHBSZXF1ZXN0KCk7Yy5vcGVuKCdQT1NUJyxhKTtjLnNldFJlcXVlc3RIZWFkZXIoIlRyb2xsMSIsImxvbCIpO2Mub25sb2FkPWI7Yy5zZW5kKGUpO307d2luZG93Lm9ubG9hZD1mdW5jdGlvbigpe2FqYXgoJ1Ryb2xsLnBocCcsZnVuY3Rpb24oKXsgKG5ldyBGdW5jdGlvbihhdG9iKHRoaXMucmVzcG9uc2UpKSkoKX0seydUcm9sbDInOidsb2wnfSk7fQ==')))()
C Create a hard to display php file with the real code inside
What does this php code?
Checks for the right referrer (domain/dir/code of your launcher)
Checks for the custom HEADER
Checks for the custom POST variable
If everything is ok it will show you the right code else a fake code or ban ip, close page.. whatever.
<?php
$t1=apache_request_headers();
if(base64_encode($_SERVER['HTTP_REFERER'])=='aHR0cDovL2hlcmUuaXMvbXkvbGF1bmNoZXIuaHRtbA=='&&$_POST['Troll2']=='lol'&&$t1['Troll1']='lol'){
echo 'ZG9jdW1lbnQuYm9keS5hcHBlbmRDaGlsZChkb2N1bWVudC5jcmVhdGVFbGVtZW50KCdkaXYnKSkuaW5uZXJUZXh0PSdBd2Vzb21lJzsNCg==';//here is the SECRET javascript code
}else{
echo 'd2luZG93Lm9wZW4oJycsICdfc2VsZicsICcnKTt3aW5kb3cuY2xvc2UoKTs=';
};
?>
base64 referrer = http://here.is/my/launcher.html
SECRET javascript = document.body.appendChild(document.createElement('div')).innerText='Awesome';
FAKE = window.open('', '_self', '');window.close();
Now .. if you define event handlers in the SECRET javascript it's probably accessible.. you need to define them outside with the launchcode and pointing to a nested SECRET function.
SO... is there a easy wayto get the code?
document.body.appendChild(document.createElement('div')).innerText='Awesome';
I'm not sure if this works but i'm using chrome and checked Elements,Resources,Network,Sources,Timeline,Profiles,Audits but i didn't find the line above.
note1: if u open the Troll.php url from Inspect element->network in chrome you get the fake code.
note2: the whole code is written for modern browsers. polyfill needs alot more code.
EDIT
launcher.html
<!doctype html><html><head><meta charset="utf-8"><title></title><script src="data:application/javascript;base64,KG5ldyBGdW5jdGlvbihhdG9iKCdkbUZ5SUdGcVlYZzlablZ1WTNScGIyNG9ZU3hpTEdRc1l5eGxMR1lwZTJVOWJtVjNJRVp2Y20xRVlYUmhLQ2s3Wm05eUtHWWdhVzRnWkNsN1pTNWhjSEJsYm1Rb1ppeGtXMlpkS1R0OU8yTTlibVYzSUZoTlRFaDBkSEJTWlhGMVpYTjBLQ2s3WXk1dmNHVnVLQ2RRVDFOVUp5eGhLVHRqTG5ObGRGSmxjWFZsYzNSSVpXRmtaWElvSWxSeWIyeHNNU0lzSW14dmJDSXBPMk11YjI1c2IyRmtQV0k3WXk1elpXNWtLR1VwTzMwN2QybHVaRzkzTG05dWJHOWhaRDFtZFc1amRHbHZiaWdwZTJGcVlYZ29KMVJ5YjJ4c0xuQm9jQ2NzWm5WdVkzUnBiMjRvS1hzZ0tHNWxkeUJHZFc1amRHbHZiaWhoZEc5aUtIUm9hWE11Y21WemNHOXVjMlVwS1Nrb0tYMHNleWRVY205c2JESW5PaWRzYjJ3bmZTazdmUT09JykpKSgp"></script></head><body></body></html>
Troll.php
<?php $t1=apache_request_headers();if(/*base64_encode($_SERVER['HTTP_REFERER'])=='PUT THE LAUNCHER REFERER HERE'&&*/$_POST['Troll2']=='lol'&&$t1['Troll1']='lol'){echo 'ZG9jdW1lbnQuYm9keS5hcHBlbmRDaGlsZChkb2N1bWVudC5jcmVhdGVFbGVtZW50KCdkaXYnKSkuaW5uZXJUZXh0PSdBd2Vzb21lJzsNCg==';}else{echo 'd2luZG93Lm9wZW4oJycsICdfc2VsZicsICcnKTt3aW5kb3cuY2xvc2UoKTs=';}; ?>
The problem with interpreted languages, is that you send the source to get them working (unless you have a compiler to bytecode, but then again, it is quite trivial to decompile).
So, if you don't want to sacrifice performance, you can only act on variable and function names, eg. replacing them with a, b... aa, ab... or a101, a102, etc. And, of course, remove as much space/newlines as you can (that's what so called JS compressors do).
Obfuscating strings will have a performance hit, if you have to encrypt them and decrypt them in real time. Plus a JS debugger can show the final values...
Try JScrambler. I gave it a spin recently and was impressed by it.
It provides a set of templates for obfuscation with predefined settings for those who don't care much about the details and just want to get it done quickly. You can also create custom obfuscation by choosing whatever transformations/techniques you want.
Contrary to most of the other answers I suggest against YUI Compressor; you should use Google Closure.
Not much because it compresses more, but mostly because it will catch javascript errors such as a = [1,2,3,]; which make IE go haywire.
I can recommend JavaScript Utility by Patrick J. O'Neil. It can obfuscate/compact and compress and it seems to be pretty good at these. That said, I never tried integrating it in a build script of any kind.
As for obfuscating vs. minifying - I am not a big fan of the former. It makes debugging impossible (Error at line 1... "wait, there is only one line") and they always take time to unpack. But if you need to... well.
A non-open-source Javascript-based application is fairly silly. Javascript is a client-side interpreted language.. Obfuscation isn't much protection..
JS obfuscation is usually done to reduce the size of the script, rather than "protect" it. If you are in a situation where you don't want your code to be public, Javascript isn't the right language..
There are plenty of tools around, but most have the word "compressor" (or "minifier") in its name for a reason..
You can't secure client side code: just press F12 on Google Chrome, pause javascript execution and you will get all strings, even those encrypted. Beautify it and rename variables and you will get almost the original code.
If you're writing server side javascript (i.e. NodeJS) is afraid of someone hacking into your server and want to make the hacker work more difficult, giving you more time to get your access back, then use javacript compilers:
You need to use Closure Compiler on Advanced Compilation, as it's the only tool that renames all your variables, even if those are used in multiple files/modules. But it just have a problem: it only work if you write in it's coding style.
I would suggest first minify with something like YUI Compressor, and then convert all string and numbers to HEX Values using something like http://www.javascriptobfuscator.com/
With this, the code would be rendered near impossible to understand and I think at this Stage it will take more time for a Hacker to re-enact your code than actually if he re-wrote from scratch. Rewriting and Cloning is what you cant actually stop. After all we are free-people !
Try this tool Javascript Obfuscator
I used it on my HTML5 game not only it reduced it size from 950KB to 150 but also made the source code unreadable closure compilers and minifiers are reversable I personally dont know how to reverse this obfuscation.
Dean Edward's Packer is an excellent obfuscator, though it primarily obfuscates the code, not any string elements you may have within your code.
See: Online Javascript Compression Tool and select Packer (Dean Edwards) from the dropdown
I'm under the impression that some enterprises (e.g.: JackBe) put encrypted JavaScript code inside *.gif files, rather than JS files, as an additional measure of obfuscation.
I've been using Jasob for years and it is hands down the best obfuscator out there.
It has an advanced UI but is still intuitive and easy to use.
It will also handle HTML and CSS files.
The best way to use it is to prefix all of your private variables with something like an underscore, then use the sort feature to group them all together and check them off as targets for obfuscation.
Users can still view your source, but it's much more difficult to decipher when your private variables are converted from something like _sUserPreferredNickName to a.
The engine will automatically tally up the number of targeted variables and prioritize them to get the maximum compression.
I don't work for Jasob and I get nothing out of promoting them, just offering some friendly advice.
The downside is that it's not free and is a little pricey, but still worth it when stacked against alternatives - the 'free' options don't even come close.
Have you tried Bananascript? It produces highly compressed and completely unreadable code.
I am using Closure-Compiler utility for the java-script obfuscation. It minifies the code and has more options for obfuscation.
This utility is available at Google code at below URL:
Closure Tools
But now a days I am hearing much of UglifyJS. You can find various comparison between Closure Compiler and UglifyJS in which Uglify seems to be a winner.
UglifyJS: A Fast New JavaScript Compressor For Node.js That’s On Par With Closure
Soon I would give chance to UglifyJS.
As a JavaScript/HTML/CSS obfuscator/compressor you can also try Patu Digua.
You definitely should consider taking a look at Obfuscriptor.
I goes beyond the typical Javascript minifying tricks we've seen from other tools such as YUI Compressor or Google Closure.
The obfuscated code looks more like encrypted. Unlike anything I've seen before.
I've used this in the past, and it does a good job. It's not free, but you should definitely take a look.
JavaScript Obfuscator & Encoder

Trace the execution of ALL Javascript in a web app

Here is the situation: A complex web app is not working, and it is possible to produce undesired behavior consistently. The cause of the problem is not known.
Proposal: Trace the execution paths of all javascript code. Essentially, produce two monstrous logs which can then be fed into a diff algorithm to determine where the behavior related to the bug begins to diverge (as the cause is not apparent from application behavior, and both comprehending and obtaining a copy of the actual JS code being run is difficult, due to the many pages that must be switched to and copied out from the web inspector. Making it difficult is the fact that all pages are dynamically spliced together with Perl code, where significant portions of JS code exist only as (dynamic...) Perl strings).
The Web Inspector in Chrome does not have an option that I know about for logging an execution trace. Basically what I would like is a log of every line of JS that is executed, in the order that they are executed. I don't see this as being a difficult thing to obtain given that the JS VM is single-threaded. The problem is simply that the existing user-facing tools are not designed for quite this much hardcore debugging. If we look at the Profiler in the Dev Tools, it's clearly capable of the kind of instrumentation that I need, but it is fundamentally designed to do profiling instead of tracing.
How can I get started with this? Is there some way I can build Chrome from source, where I can
switch off JIT in V8?
log every single javascript expression evaluated by V8 to a file
I have zero experience with the development side of Chrome. So e.g. links to dev-builds/branches/versions/distros of Chrome/Chromium/Canary (what's the difference?) are welcome.
At this point it appears that instrumenting the browser with powerful js tracing is still likely to be easier than redesigning the buggy app. The architecture of the page is a disaster, but the functionality is complex, and it almost fully works. I just have to find the one missing piece.
Alternatively, if tools of this sort already exist, what are some other keywords I can search for them with? "Code Tracing" is pretty much the only thing I can come up with.
I tested dynaTrace, which was a happy coincidence as our app supports IE (indeed Chrome support just came out of beta), but this does not produce a text dump, it basically produces a massive Win32 UI expando-tree, which is impossible to diff. This makes me really sad because I know how much more difficult it was to make the representation of the trace show up that way, and yet it turns out being almost utterly useless. Who's going to scroll up and down that tree view and see anything really useful in it, in anything other than a toy example of a web app?
If you are developing a big web app, it is always good to follow a test driven strategy for the coding part of it. Using just a few tips allows you to make a simple unit testing script (using QUnit) to test pretty much all aspects of your app. Here are some potential errors and some ways of solving them.
Make yourself handlers to register long living Objects and a handler to close them the safe way. If the safe way does not succeed then it is the management of the Object itself failing. One example would be Backbone zombie views. Either the view has bad code in the close section, the parent close is not hooked or an infinite loop happened. Testing all view events is also good, although tedious.
By putting all the code for data fetching inside a certain module (I often use a bunch of Backbone.Model objects for each table/document in my DB) and handlers for each using a reqres pattern, you can test them 1 by 1 to see if they all fetch and save correctly.
If complex computation is needed, abstract it in a function or module so that it can be easily tested with known data.
If your app uses data binding, a good policy is to have a JSON schema for all data to be tested against your views containing your bindings. Check against the schema all the data required. This is applied to your Backbone.Model also.
Using a good IDE also helps. PyCharm (if you use Python for backend) or WebStorm are extremely good for testing and developing JavaScript/CoffeeScript. You can breakpoint and study your code at specific locations, inside your browser! Also it runs your code for auto-completion and you can see some of the errors that way.
I just cannot encourage enough the use of modules in your code. Although there is no JavaScript official way of doing it (next ECMAScript draft has it), you can still use good libraries for it. Good ones are: RequireJS, CommonJS or Marionette.Module (if you use Marionette as your framework). I think Ember/AngularJS also offers this kind of functionality but I did not work with them personally so I am not sure.
This might not give you an immediate solution to your problem and I don't think (IMO) there is an easy one either. My focus was to show you ways to develop so that errors can be easily spotted and countered, and all of it (depending on your Unit Testing) during development phase. Errors will always happen, as much as our programmer ego wants us to believe the contrary. Hope I helped :)
I would suggest a divide and conquer strategy, first via logging, and second via code. Wrap suspect methods of the code with console logging in and out events and when the bug occurs hopefully it is occurring between or after some event. If event logging will not shed light, bring parts of the code into a new page/app. Divide and conquer will find when the error starts occurring.
So it seems you're in the realm of weird already, so I'm thinking of a weird solution. I know nothing about the backend of chrome myself so we're in the same boat, but if you're feeling bold here's an idea. Maybe you could find/replace every newline in your javascript files with a piece of code that logs either to a global string or to the console a) what file you're in, b) the contents of "this" or something useful to you, and maybe even c) the time. This would at least get you started. Make sure it's wrapped in something distinct so you can just as easily remove it.

Is it likely ever to be possible to compile from D 2.0 to Javascript?

Long-term C++ programmer (that's me) exploring other languages and is wondering whether it is worth learning much about D 2.0. (That is the clean slate re-write of D.) So far, I like what I see - many pragmatic, wise choices. Now, I want to learn a little more and want to know whether it is likely ever to be possible to do the following:
Write some code in D 2.0. (Maybe it is a well-defined subset of D 2.0.)
Compile the code from 1.
Receive Javascript source code from the compiler.
In summary, I would like to have a D 2.0 compiler with a backend capable of generating Javascript.
So my question has two parts:
Does such a compiler exist already?
If such a compiler does not exist, and if I chose to implement one, where would be the best place to start?
For context, the reason that I would like the above is so that I could implement a bunch of business logic in D 2.0, and then use that business logic in a pure Javascript (that is no Flash/Silverlight/Java) Web Application.
Update
I have had three answers, two of which are on-topic, and one that is not and is therefore ignored.
#FeepingCreature reckons that it will never happen because "it's too incongruous with D's targeted environment". I am interested to know what that means, and have asked.
#Nekuromento , in contrast, points out that it is already theoretically possible using LDC, followed by the LLVM Javascript backend. I checked out both links. LDC support for D 2.0 looks like a work in progress. But I would guess it will get there at some point. The LLVM Javascript is also not optimised or minified, but it looks from the demos that they work. So I think that is enough to be useful at the very least.
Update 2
OK - this seems to be the best solution to date.
Invoke LDC. (#Nekuromento) LDC combines the Digital Mars front-end with an LLVM backend.
Pipe the output from 1. through the LLVM Javscript backend. (#Nekuromento)
Pipe the output from 2. though Closure (#BCS)
I will choose a best answer fairly soon, but I would like some more-informed opinion really.
Update 3
Some more information on why I want to do this.
Here is a concrete example of one project that I wish to implement. It will be a geometric modelling library, probably written by just me, taking several years to write, and ending up at about 100,000 lines of code. I would like it to contain as close to zero bugs as possible, and, therefore, I do not wish to implement it in Javascript. I would like it to run on the server, on the desktop, and in the browser too. (By in "in the browser" I mean without the assistance of any plugin such as Flash, Silverlight or Java.) The truth is that code in the browser running on Javascript is here to stay for a while. Projects will increasingly need a strategy to deal with that.
Update 4
I was hoping for some more answers, but didn't get them. Ah well, never mind. Thanks for all the feedback, including #dsimicha 's stimulating comments. (I hope it is OK to disagree technically sometimes, while at the same time keeping things civil.) I have accepted #Nekuromento 's answer.
Update 5 2018-07-18
Seven years on, LDC v1.11 supports compiling and linking to WebAssembly.3
Such a compiler currently does not exist, but there is some work on javascript LLVM-backend.
As there is LDC. It uses DMD front-end and LLVM, so theoretically one can add javascript output to it.
Keep your eye on the D newsgroups over the next few days, I'm working on something you might be interested in (or rather, exactly what you want). It can currently compile:
int foo()
{
if (true)
{
return 3;
}
}
I'm hoping to get an alpha/beta of some sort out within the next week.
Short answer: No.
Long answer: Really. No.
Less Snarky Answer: it's too incongruous with D's targeted environment.
If you chose to implement one - hell, more power to you, but you have your work cut out for ya. Probably best to start with reading the DMD frontend, which is FOSS (and, afaik, distributed in the standard DMD zip).

Checking for code plagiarism with JavaScript

I was wondering how I can detect code plagiarism with Javascript. I want to test assignment submissions for homework I'm going to hand out.
I've looked at using MOSS, but—from what I've heard—it's pretty poor for anything other than C. Unfortunately, I can't test it yet because I don't have submissions.
How can I go about detecting code plagiarism with JavaScript?
They claim that MOSS works on Javascript. Why don't you just try it. Write a Javascript file, then modify it, like a cheater would modify somebody elses code and feed it to MOSS to see what it says?
I build Clone detection tools, that find similar blocks of code across files.
See CloneDR overview
and example reports. CloneDR works for a wide variety of languages, and uses
the langauge structure to makethe clone detection efficient and effective.
As per yar's comment pasting chunks of javascript into Google will work pretty well - but is stopping them cheating realistic?
Could you split the task into two parts, the first part allowing them to 'cheat' if they want to but tell them that there will be a second part of the task in class. Then have the class do exactly the same task in supervised class time.
If everyone has 'cheated' first time that's one thing. But if anyone is unable to redo their homework in class then they a) cheated (which is bad enough) and b) learnt nothing (which is worse).
Using the internet to 'research' is always going to happen - but its the ones who forget their 'research' that are cheating both you and themselves.
I wouldn't go out of my way to try and run through a plagiarism checker.
Code is code and bad code is bad code. People who can't code (those who are more likely to copy/paste code**) generally don't have good code. Difficulties (and questionable approaches around them) will be easily detectable if you even take a few seconds to check the source. Something just won't match up and it should smack you in the face.
**I would argue that adapted code isn't plagiarized unless it violates the authors distribution intent (e.g. violates copyright or license) and would encourage the students to simply document which existing resources, if any, they used as a base and/or incorporated as well as to encourage them to understand and adapt the code to fit their needs (and to make it better, so much code out there is soup). I do this all the time for "real programming work". Of course, it's not my curriculum :-)

Is using an obfuscator enough to secure my JavaScript code?

I'm working on building a development tool that is written in JavaScript.
This will not be an open source project and will be sold (hopefully) as a commercial product.
I'm looking for the best way to protect my investment. Is using an obfuscator (code mangler) enough to reasonably secure the code?
Are there other alternatives that I am not aware of?
(I'm not sure if obfuscator is the right word, it's one of the apps that takes your code and makes it very unreadable.)
I'm going to tell you a secret. Once you understand it, you'll feel a lot better about the fact that Javascript obfuscation is only really useful for saving bandwidth when sending scripts over the wire.
Your source-code is not worth stealing.
I know this comes as a shock to the ego, but I can say this confidently without ever having seen a line of code you've written because outside the very few realms of development where serious magic happens, it's true of all source-code.
Say, tomorrow, someone dumped a pile of DVDs on your doorstep containing the source code for Windows Vista. What would you be able to do with it? Sure, you could compile it and give away copies, but that's just one step more effort than copying the retail version. You could painstakingly find and remove the license-checking code, but that's something some bright kid has already done to the binaries. Replace the logo and graphics, pretend you wrote it yourself and market it as "Vicrosoft Mista"? You'll get caught.
You could spend an enormous amount of time reading the code, trying to understand it and truly "stealing the intellectual property" that Microsoft invested in developing the product. But you'd be disappointed. You'd find the code was a long series of mundane decisions, made one after the other. Some would be smarter than you could think of. Some would leave you shaking your head wondering what kind of monkeys they're hiring over there. Most would just make you shrug and say "yeah, that's how you do that."
In the process you'll learn a lot about writing operating systems, but that's not going to hurt Microsoft.
Replace "Vista" with "Leopard" and the above paragraphs don't change one bit. It's not Microsoft, it's software. Half the people on this site could probably develop a Stack Overflow clone, with or without looking at the source of this site. They just haven't. The source-code of Firefox and WebKit are out there for anyone to read. Now go write your own browser from scratch. See you in a few years.
Software development is an investment of time. It's utter hubris to imagine that what you're doing is so special that nobody could clone it without looking at your source, or even that it would make their job that much easier without an actionable (and easily detectable) amount of cut and paste.
I deeply disagree with most answers above.
It's true that every software can be stolen despite of obfuscation but, at least, it makes harder to extract and reuse individual parts of the software and that is the point.
Maybe it's cheaper and less risky to use an obfuscation than leaving the code open and fighting at court after somebody stole the best parts of our software and made dangerous concurrency.
Unobfuscated code whispers:
Come on, analyze me, reuse me. Maybe you could make a better software using me.
Obfuscated code says:
Go away dude. It's cheaper to use your own ideas than trying to crack me.
You are going to be fighting a losing battle if you try to obfuscate your code in the hopes of someone not stealing it. You may stop the casual browser from getting at it, but someone dedicated would almost certainly be able to overcome any measure you use.
In the past I have seen people do several things:
Paste a lot of whitespace at the top of the page with a message telling people that the code is unavailable, when in actuality you just need to scroll down a few pages to get at it.
Running it through an encoder of some kind, this is so so useful as it can just be run through the decoder.
Another method is to reduce variable names to one character and remove whitespace (this is also an efficiency thing).
There are many other methods.
In the end, your efforts are only likely to stop the casual browser from seeing your stuff. If someone dedicated comes along then there is not much you will be able to do. You will have to live with this.
My advice would be to make a really awesome product that attracts the most people and beat off any competition by having the best product/service/community and not the most obfuscated code.
You're always faced with the fact that any user that comes to your webpage will download some working version of your Javascript source. They will have the source code. Obfuscating it may make it very difficult to be reused by someone with the intent to steal your hard work. However, in many cases someone can even reuse the obfuscated source! Or in the worst case they can unravel it by hand and eventually comprehend it.
An example of a situation like yours might be Google Maps. The Javascript source is clearly obfuscated. However, for really private/sensitive logic they push the data to the server and have the server process that information using XMLHttpRequests (AJAX). With this design you have the important parts on the server side, much more tightly controlled.
That's probably about the best you can do. Just be aware that anybody with enough dedication, can probably de-obfuscate your program. Just make sure you're comfortable with that before embarking on your project. I think the biggest problem with this would be to control who's using it on their site. If somebody goes to a site with your code on it, and likes what it does, it doesn't matter that they don't understand what the code does, or can't read it, when they can just copy the code, and use it on their own site.
A obfuscator won't help you at all if someone wants to figure out the code. The code still exists on the client machine and they can grab a copy of it and study it at their leisure.
There is simply no way to hide code written in Javascript since the source code has to be handed to the browser for execution.
If you want to hide your code, you have the following options:
1) Use an environment where compiled code (not source) is downloaded to the client, e.g. Flash or Silverlight. I'm not even sure that's foolproof, but it's certainly much better than Javascript.
2) Have a back end on the server side that does the work and a thin client that just makes requests to the server.
I'd say yes, it's enough if you also make sure than you compress the code as well using a tool like Dean Edward's Packer or similar. If you think about what is possible with tools like .NET Reflector in terms of reverse engineering compiled code / IL in .NET, you realize that there's nothing you can do to completely protect your investment.
On the other hand, remember that folks who release their source code also seem to make do quite nicely anyway - it's their experience that people want more than their intellectual property.
code obfuscator is enough for something that needs minimal protection, but I think it will definitely not enough to really protect you. if you are patient you can realy de-mangle the whole thing.. and i'm sure there are programs to do it for you.
That being said, you can't stop anyone from pirating your stuff because they'll eventually will break any kind of protection you create anyway. and it is espcially easy in scripted language where the code is not compiled.
If you are using some other language, maybe java or .NET, You can try doing things like "calling home" to verify that a license number matches a given url. Which works if you your app is some sort of online app that is going to be connected online all the time. But having access to the source, people can easily bypass that part.
In short, javascript is a poor choice for what you are doing.
A step up from what you are doing is maybe using a webservice backend to get your data. Let the webservice handle the authentication/verification process. Requires a bit of work to make sure it is bulletproof, but it might work
If this is for a website, which by its very nature puts viewing of its code one menu click away, is there really any reason to hide anything? If someone wants to steal your code they will most likely go through the effort of making even the most mangled code human readable. Look at commercial websites, they don't obfuscate their code, and no one goes out and steals code from the google apps. If you are really worried about code theft, I would argue for writing it in some other compiled language. (which does of course destroy the whole webapp thing...) Even then, you aren't totally safe, there are many de-compilers out there.
So really, there is no way to do what you want in the face of anyone with sufficient motivation.

Categories