We Keep Coding

Framework to structure existing JS code

I have some procedural javascript code that I have written for an open-source application and I'd Like to refactor it into OOP and since I have very little experience with javascript frameworks I have trouble finding a one suitable for my needs, though I haven't tried anything yet, I just read about AngularJS, Backbone.js and Knockout.
I want to structure the code, because, at the moment, there's a mess, global variables and functions scattered around.
I have to mention that all the business logic is handled at the server level, so the client-side code handles just the UI using the data it receives or requests from the server.
The code can be found here:
https://github.com/paullik/webchat/blob/asp.net/webchat/Static/Js/chat.js
Do you have any suggestions?

Object-Oriented JavaScript is not necessarily the answer to all your
problems.
My advice is to be careful the choice you pick on this topic.
In practice, OO-JS can add more complexity to your code for the sake of trying to be more like traditional object-oriented languages. As you probably know, JS is unique.
It is important to know that there are Design Patterns that will structure your code and keep implementation light and flexible.
It is Design Patterns that I see structuring advanced JS
implementations, not OO. To paraphrase Axel Rauchmeyer - "Object
Oriented methodology does not fit into basic JavaScript syntax, it is
a twisted and contorted implementation, and JS is far more expressive
with out it."
The root of this analysis boils down to the fact that JS has no class. In essence, since everything is an object, you already have object-oriented variables and functions. Thus the problem is slightly different than the one found in compiled languages (C/Java).
What Design Patterns are there for JavaScript?
An excellent resource to check is Addy O' Somani and Essential Design Patterns.
He wrote this book on Design Patterns in JavaScript.
But there is more... much more.
A. require.js - There is a way to load modules of JS code in a very impressive way.
These are generally called a module loaders, and are widely considered the future of loading js files since they optimize performance at runtime. yepnope and others exist. Keep this in mind if you are loading more than a few js files. (moved to top by request).
B. MVC - There are dozens of Model View Controller frameworks to help you structure code.
It is a Pattern, but may be unreasonable for your purposes. You mentioned backbone, knockout and angular... Yes. These can do the trick for you, but I would be concerned that they might be 1) high learning-curve, and 2) overkill for your environment.
C. Namespace or Module Pattern. Are probably the most important for your needs.
To solve global variables just wrap them in a namespace and reference that.
These are very good patterns that give rise to module loaders.
D. Closure - You mentioned OO JS. On piece of this that is useful is the notion of closures to provide yourself with... private members. At first this is a cryptic idea, but after you recognize the pattern, it is trivial practice.
E. Custom Events - It becomes very important to not use hard references between objects. Example: AnotherObject.member; This is because it will tightly couple the two objects together and make both of them inflexible to change. To solve this, trigger and listen to events. In traditional Design Patterns this is the Observer. In JS it is called PubSub.
F. The Callback - The callback pattern is what enabled AJAX and it is revolutionizing development in terms of Window 8, Firefox OS, and Node.js - because of something called non-blocking-io. Very important.
Do not be afraid. This is the direction to go for long-term and advanced JavaScript implementations.
Once you recognize the patterns, it is down hill from there.
Hope this helps.

Shortcut programming languages am I missing something?

I'm seeing a rise in the number of projects online that require additional "short" syntax to generate underlying code (commonly HTML and JS), and every time I see another pop up I wonder if I'm missing something.
Surely the additional time taken to learn and remember these shortcodes is lost when passing projects between different team members (designers et al) who then have to relearn basic languages such as HTML and JS?
Point in question: YAML being used for generating templates, and projects (such as) CoffeeScript used to generate Javascript. I'm just wondering if other Stackoverflow folk think that there's major benefits to this granular level of abstraction, or if there's a feeling that these types of projects have any lifespan, considering the improvements in the base technologies (I'm thinking HTML5 / CSS / JQuery) here. Have markup languages not evolved enough to really warrant these?
Realise this isn't really a question as such, but would be interested to hear the community viewpoint. I'm sure there are genuinely good reasons for these types of projects when creating self generating code, but outside of that, I'm puzzled - with many folks using them day to day. Is this a bandwagon worth jumping on?

Personally I don't think that it's worth it as most things you can do in javascript are already simplified by libraries such as jQuery. As far as their lifespan goes it's hard to tell. However, as long as you know javascript, and you understand the code output, moving to a new project that isn't using CoffeeScript for example is a simple matter of taking the output code with you.

Any sufficiently complicated program contains an implementation of Lisp.
This is the ideology that Lisp and friends promote over many years. Basically, you're discouraged to code any application logic in a "raw" language. Raw languages are universal and low level, so your logic quickly gets verbose and contaminated with lots of code needed to support the language itself.
Instead, create a DSL that suits your application best and code your logic, and nothing but the logic, in this DSL. Hide all nasty language details behind the DSL. This makes the logic much easier to improve and support.
On the other side, things like Coffescript, Groovy etc are IMO the wrong way to go. They are basically trying to create new universal languages but with a "better" syntax, however, this doesn't solve the fundamental problem - these languages still describe abstract calculations rather than your problem domain. A more productive way is to have a very basic underlying language and a well-developed macro facility that would simplify creating DLSs. This is how lisp itself works, another example of this approach is Haxe/Neko.

In my opinion I prefer jQuery for writing short JS instructions and prevent everyone from bloating the final product. I don't like css frameworks because we are building huge portals and you don't always need all of the stuff hidden inside these. I prefer writing modular css based on some common rules like using clearfix for clearing an using a css reset. :)

Should writing Javascript be avoided in favour of GWT/WebSharper or some other abstraction?

I'm curious what's the view on "things that compile into javascript" e.g. GWT, Script# and WebSharper and their like. These seem to be fairly niche components aimed at allowing folks to write javascript without writing javascript.
Personally I'm comfortable writing javascript (using JQuery/Prototype/ExtJS or some other such library) and view things like GWT these as needless abstractions that may end up limiting what a developer needs to accomplish or best-case providing a very long-winded workaround. In some cases you still end up writing javascript e.g. JSNI.
Worse still if you don't know what's going on under the covers you run the risk of unintended consequences. E.g. how do you know GWT is creating closures and managing namespaces correctly?
I'm curious to hear others' opinions. Is this where web programming is headed?

Should JavaScript be avoided in favor of X? By all means!
I will start with a disclaimer: my answer is very biased as I am on the WebSharper developer team. The reason I am on this team in the first place is that I found myself a complete failure in writing pure JavaScript, and then suggested to my company that we try and write a compiler from our favorite language, F#, to JavaScript.
For me, JavaScript is the portable assembly of the web, fulfilling the same role as C does in the rest of the world. It is portable, widely used, and it will stay. But I do not want to write JavaScript, no more than I want to write assembly. The reasons that I do not want to use JavaScript as a language include:
There is no static analysis, it does not even check if functions are called with the right number of arguments. Typos bite!
There is no or a very limited concept of libraries, namespaces, modules, classes, therefore every framework invents their own (a similar situation to that of R5RS Scheme).
The tooling (code editors, debuggers, profilers) is rather poor, and most of it because of (1) and (2): JavaScript is not amenable to static analysis.
There is no or a very limited standard library.
There are many rough edges and a bias to using mutation. JavaScript is a poorly designed language even in the untyped family (I prefer Scheme).
We are trying to address all of these issues in WebSharper. For example, WebSharper in Visual Studio has code completion, even when it exposes third-party JavaScript APIs, like Ext Js. But whether we have or will succeed or fail is not really the point. The point is that it is possible, and, I would hope, very desirable to address these issues.
Worse still if you don't know what's
going on under the covers you run the
risk of unintended consequences. E.g.
how do you know GWT is creating
closures and managing namespaces
correctly?
This is just about writing the compiler the right way. WebSharper, for instance, maps F# lambdas to JavaScript lambdas in a 1-1 manner (in fact, it never introduces a lambda). I would perhaps accept your argument if you mentioned that, say, WebSharper is not yet mature and tested enough and therefore you are hesitant to trust it. But then GWT has been around for a while and should produce correct code.
The bottom line is that strongly typed languages are strictly better than untyped languages - you can easily write untyped code in them if you need to, but you have the option of using the type-checker, which is the programmer's spell-checker. Why wouldn't you? A refusal to do so sounds a bit luddite to me.

Although, I don't personally favor one style over another, I don't think that abstraction from Javascript is the only benefit that these frameworks bring to the table. Surely, in abstracting the entire language, there will be things that become impossible that were previously possible, and vice-versa. The decision to go with a framework such as GWT over writing vanilla JavaScript depends on many factors.
Making this a discussion of JavaScript vs language X is fruitless as each language has its strengths and weaknesses. Instead, do an objective cost-benefit analysis on what is to be gained or lost by going with such a framework, and that can only be done by you and not the SO community unfortunately.
The issue of not knowing what goes on under the hood applies to JavaScript just as much as it does to any translated source. How many people do you think would know exactly what is going on in jQuery when they try to do a comparison such as $("p") == $("p") and get back false as a result. This is not a hypothetical situation and there are several questions on SO regarding the same. It takes time to learn a language or framework, and given sufficient time, developers could just as well understand the compiled source of these frameworks.
Another related aspect to the above question is of trust. We continuously build higher level abstractions upon lower level abstractions, and rely on the fact that the lower level stuff is supposed to work as expected. What was the last time you dug down into the compiled binary of a C++ or Java program just to ensure that it worked correctly? We don't because we trust the compiler.
Moreover, when using such a framework, there is no shame in falling back to JavaScript using JSNI, for example. It's all about solving the problem in the best possible manner with the tools at hand. There is nothing sacred about JavaScript, or Java, or C#, or Ruby, etc. They are all tools for solving problems, and while it may be a barrier for you, it might be a real time-saver and advantageous to someone else.
As for where I think web programming is headed, there are many interesting trends that I think or rather hope will succeed such as JavaScript on the server side. It solves very real problems for me at least in that we can avoid code duplication easily in a web application. Same validations, logic, etc. can be shared on the client and server sides. It also allows for writing a simple (de)serialization mechanism so RPC or RMI communication becomes possible very easily. I mean it would be really nice to be able to write:
account.debit(200);
on the client side, instead of:
$.ajax({
url: "/account",
data: { amount: 200 },
success: function(data) {
..
}
error: function() {
..
}
});
Finally, it's great that we have all this diversity in frameworks and solutions for building web applications as the next generation of solutions can learn from the failures of each and focus on their successes to build even better, faster, and more awesome tools.

I have three big practical issues I have with websharper and other compilers that claim to avoid the pain of Javascript.
If you won’t know Javascript well you can’t understand most of the examples on the web of using the DOM/ExtJs etc., so you have to learn Javascript whatever. (For the some reason all F# programmers must be able to at least read C# or VB.NET otherwise they cannot access most information about the .net framework)
On any web project you need a few web experts that know the DOM and CSS inside out; would such a person be willing to work with F# rather than Javascript?
Being tied into the provider of the compiler, will they be about in 5 years’ time; I want full open source or the tools to be supported by Microsoft.
The 4 big positives I see with these frameworks are:
Shareing code between the server/client
Having fewer languages a programmer needs to know (javascript is a real pain as it looks like Jave/C# but is not anything like them)
The average quality of a F# programmer is a lot better than a jscript programmer.

My opinion for what it's worth is that every framework has its pros/cons and a project team should evaluate their use cases before including one. To me any framework is just a tool to be used to solve a problem, and you should pick the best one for the job.
I prefer to stick to pure JavaScript solutions myself, but that being said I can think of a few cases where GWT would be helpful.
GWT would allow a team to share code between the server/client, reducing the need to write the same code twice (JS and Java). Or if someone was porting a Java client to a web UI, they may find it easier to stick to GWT ( of course then again it may make it harder :-) ).

I know this is a gross over-simplification, because there are many other things that frameworks like GWT offer, but here is how I view it: if you like JavaScript, write JavaScript; if you don't, use GWT or Cappuccino or whatever.
The reason people use frameworks like GWT is not necessarily the abstraction that they give--you can have that with JavaScript frameworks like ExtJS--but rather the fact that they allow you to write web applications in something other than JavaScript. If I were a Java programmer who wanted to write a web application, I would use GWT because I would not have to learn a new language.
It's all preference, really. I prefer to write JavaScript, but many people don't.

Ideas for a comprehensive Javascript assignment

I am likely to be part of the teaching team for the web programming course at my University next semester and I was wondering what kind of Javascript assignment to hand out to the students. The course is not an introductory one from a programming perspective.
It is assumed that the students are familiar with OOP, data structures and algorithms, functional programming concepts and working knowledge of networking protocols (HTTP included). This is the first course in which they come in contact with JavaScript
I was thinking to give out something framework-specific (using jQuery perhaps) that involves DOM traversal, some animations and AJAX. The three questions I have in mind are:
should they use a framework or should I have them write vanilla JavaScript?
should I focus more on the functional programming part and on the prototypal inheritance part (more on the language than on working with the DOM)?
how do I automate testing for this? It's better if they have a clear idea on how they will be evaluated. Also, automated testing ensures objectivity and saves me time :).
Outcome
I made them do Tic Tac Toe as a jQuery plugin and the results were mostly satisfactory (70% of the students submitted, generally the submissions were ok).
To prevent copying code from the net, I thought out an API which they had to implement. At least, they'd have to understand the code they found on the net before copy&pasting it into the methods :).
I used QUnit for automated testing, but I also tested each assignment manually because this was the first JavaScript assignment they'd had and I wanted to give relevant feedback.
Thank you all for your ideas, they all helped a lot.
Cheers,
Alex

I think its useful for the students to know fundamentals about the language before working with frameworks. They need to know about JS Scopes, closures, prototypes, the memory model, and everything that makes JS unique.
After that, introduce them to frameworks and the DOM. They'll appreciate them much more since they'll be able to understand the implementation.
As for testing, automated testing might be easy if you have them generate a DOM that you can walk and validate. Mozilla might be able to help you out, esp with JSUnit. You can find info here

I always like the idea of making games to learn new programming concepts. You get a well-defined problem domain that's as simple or complex as you need it, and it's usually more interesting and fun to implement than other problems.
When I wanted to learn Ajax programming I used jQuery and Java server-side to implement the game of chess. It was a fun project, but pretty complicated (at least for me, but I'm primarily a server side programmer). I think something like Tic-Tac-Toe would be substantially simpler, and might be a good idea for a project assignment.
As for the 3 questions:
If this is the only JavaScript assignment, then I'd probably use vanilla instead of jQuery. But if they have a chance to do some assignments before this, I'd consider jQuery, because it just makes JavaScript so much less annoying, and it's also good to know jQuery for future employment possibilities.
I'd place an equal emphasis on both the language and the DOM, because the primary purpose of the language IS to work on the DOM, and the DOM does take some getting used to.
I think Selenium might work for the testing you're trying to do. JsUnit could also be used for unit testing the individual methods.

Start off with vanilla JavaScript to learn the basics. You don't want to create a group that relies on any particular framework that wouldn't know how to do things without it.

I would most definitely have them write vanilla JavaScript. It will encourage all students to better understand the abstractions that frameworks/libraries provide in particular environments i.e. for the most part, in the browser working with the DOM.
I highly recommend having a good text for the course. Object Oriented JavaScript by Stoyan Stefanov is in my mind a great text for learning the language, including some of the topics that many people have difficulty with (prototypes, objects, closures, inheritance, etc). I've read numerous JavaScript books and feel that this particular text best balances the core of the language and it's application in the modern client-side development realm.
You may then want to look at dissecting certain pieces of the source of a particular JavaScript library to gain insight into patterns and practices used in a real-world scenario.

I would have them write vanilla javascript AND also learn how to use jQuery. jQuery is javascript after all, and they need a working knowledge of the language anyway. They'll also need to become SWAT (skills with advanced tools), and I believe anyone not using one of the JS frameworks (or at least their own!) in today's environment is at a serious disadvantage.
See answer 1. I'd teach them about prototypal inheritance in vanilla JS, and about DOM manipulation in jQuery.
Automated testing could be achieved in several ways. 1: produce the correct output given some sample code to start with for the parts that deal with learning JS. 2: for the parts that deal with jQuery, you could provide a reference image for how you expect the result to look, provide an original document, and have them recreate the reference image using jQuery manipulation... sort of like the ACID tests http://acid3.acidtests.org/

Should they use a framework or should I have them write vanilla JavaScript?
To me, it is overwhelmingly import that people new to the language start with the language proper, not modified versions or advanced/fancy libraries that do a lot of the work for you. Besides, if you're starting off not working with the DOM, then you're not getting much benefit from using almost any library, as the bulk of most JavaScript libraries has to do with handling the DOM. Also, it's easier to spot "bad" or ill-performant code when teaching and learning in a "vanilla" environment since you don't have libraries abstracting away the nitty-gritty.
Should I focus more on the functional programming part and on the prototypal inheritance part (more on the language than on working with the DOM)?
Yes! On one hand, the DOM is not not that big of a deal; yet it is also the core of what JavaScript is used to interact with. For starters, I suggest that if you're going to be using a browser environment, you should initially avoid the DOM by using Firebug's console.* methods for output so that you can focus on the "functional programming part and on the prototypal inheritance" and other core concepts. After these core concepts have been covered, then start introducing the DOM. It's best to introduce the DOM later as time will need to be dedicated to cross-browser compatibility, which will only confuse the subject if you are trying to teach the core concepts in tandem.
How do I automate testing for this? It's better if they have a clear idea on how they will be evaluated. Also, automated testing ensures objectivity and saves me time :).
Before (and after) the DOM is introduced, you could use something like JSUnit. Also, see this question: Automated Unit Testing with JavaScript. Once you introduce the DOM, you may want to have the students generate a document that you can walk and validate as SB suggested.

Learning JavaScript... Should I skip straight to the good stuff (the frameworks)?

I learnt HTML/CSS a good few years back, then PHP a little later. I've recently become interesting in web development again, just started playing with frameworks like Django and RoR. I'm curious as to how much time/effort I should spend learning straight JS before looking at frameworks. I've been reading through a let of articles called Mastering AJAX by Brett McLaughlin which seems quite good, but I'm seeing a lot of stuff (such as cross browser compatibility - even for things like XMLHttpRequest) coming up which look like they would be non-issues if using a framework.
So, should I keep reading through these articles and try to build stuff using basic JS, or should I just start looking into jQuery and the like?
Also, I've been watching a few videos regarding GWT from Google I/O. I've been learning Java over the last year, built a few medium sized apps in it. I'm wondering if GWT is something that's worth going straight to, along with gQuery?

Starting with the basics of JavaScript is a good idea, IMHO.
Read JavaScript: The Good Parts, by Douglas Crockford. Very, very good book.
You should also check out Douglas Crockford's web site.
I also had to come back here and mention this in an update:
Douglas Crockford presented an illuminating talk about JavaScript - past, present, future - at the Microsoft MIX10 conference earlier this year. You'll find the full video for Crockford's talk at Microsoft MIX10 - The Tale of JavaScript. I Mean ECMAScript.

No.
Just as when you are learning to program you are taught first C/Pascal then Java/C++ and finally Python/Ruby/Smalltalk/Lisp, and when learning any language you start with simple language constructs, you should first learn ECMAScript, then learn DOM and finally frameworks.
Why? Because you'll have a deeper understanding of the language, and will be able to debug things that might seem odd unless you've got that learning experience.
If you are a seasoned developer, you can speed up each phase, but don't skip them, or you will have problems due to not fully understanding the small oddities.
Javascript is an interesting and fun language, but can act rather odd at times (Date has bitten me a couple of times in the ass).
Use frameworks to avoid repetitive tasks and to simplify your code, but not as a starting point. Simplicity is a final goal, not the starting point, and frameworks are for that, simplicity, not for learning a language. Frameworks are intended for simplifying things for experienced developers.
Learning the differences between browsers (DOM implementations) will allow you to debug your framework. That is priceless.
I've been learning Java over the last
year...
Javascript is not Java. Never was never will.
Even if you can compile to Javascript from Java, it's still a framework, don't jump into it unless you already know what you are doing.

I think "both." Mix it up. Play around with a framework. You'll get stuck when you try to do something real, so you'll pick up some JavaScript to figure it out.
A lot of the good jQuery books teach you JavaScript along the way.

This is one of the best videos for beginner javascript developers that understand how to program in other languages:
It's a talk the John Resig did last year at Northeastern, most of it is devoted to talking about core javascript, then the last quarter of the talk jumps into jQuery:
http://video.google.com/videoplay?docid=-7485992465859932389&ei=jhZUSu73OpfSrQLgyYV3&q=john+resig&hl=en
It's actually a really interesting tech talk and presentation since he does live examples and Resig is pretty good at presenting.
Like others in this question, I also highly recommend reading Javascript: The Good Parts for a better understanding.

Yes, jumping straight to framework-based programming instead of DOM is a good idea.
I started doing JS before any major frameworks like jQuery came along, and was reluctant to switch at first, but when I first started using jQuery, it felt so good to be able to write selectors and stuff and not have to worry about cross-browser compatibility.
However, there are some areas of JS where frameworks wouldn't be available. One of them is in userscripting, where you have to make your script work on a site you don't control. Another such one is the use of JavaScript in applications such as XUL.
Overall, I suggest you start with some trivial JS applications, then switch to jQuery instead of going on to the complex topic of DOM.

What makes you think that the frameworks are the good stuff and the JavaScript is not?
If you ask me, I will say that JavaScript is a real fun language and you should learn it first. JavaScript has received bad rap because it was mainly used for browser scripting and those browsers were buggy making people think that JavaScript sucks.
Crockford says that JavaScript is the world's most misunderstood language.
If your prior experience is with PHP (or any non functional language for that matter), the concept of 'first class functions' will really give you an 'aha' moment. 'Closure' will be another tool which will simplify your code and will make you wonder why all languages don't have it. Prototype inheritance will show you that there are alternatives to OOP. I would definitely suggest that you learn JavaScript first before you jump to any frameworks. I must add that you will also have to learn the concepts (first class functions/closure/prototype inheritance), to use any JS framework efficiently as all frameworks exploit the features of JavaScript.
To learn JavaScript, get Crockford's 'JavaScript, The Good Parts' book and try to learn the language using a standards compliant browser (say Firefox [with FireBug], Safari, Chrome) without focusing on DOM manipulation. This presentation by Simon Wilson is also good.
Once you have a good feel for the language, move on to the next step i.e. manipulating the DOM. I would personally suggest that you try to do some DOM manipulation using bare bone JavaScript to get better understanding of the DOM and the pain points involved. [E.g., when I attach a method to onClick of event, 'this' doesn't refer to what I think it refers to?]
After you have suffered a little bit of pain by doing DOM manipulation by hand, move on to a JavaScript framework which removes all the pain and makes JavaScript fun again. Personally, I would highly recommend jQuery over other frameworks.
And if you have any questions while on your JavaScript journey, you can always ask them on SO! :) Good luck.

I wouldn't touch any framework in any language until I have good basic understanding of underlying technology. Worth type of coder is one that uses tool without a knowledge.
JavaScript has somewhat sad history but in its latest incarnation it's surprisingly powerful and even fascinating language. I say - learn it well, then use whatever framework suits your current needs

It depends on where you want to invest your time. Ideally, we're all expert in assembly language, but that's not realistic or practical. We have to pick our battles. Then generally we attack each other for picking the wrong battle, which we call "cargo cult coding."
Personally, out of all the things I could spend a lot of time banging my head against, the intricacies of cross-platform JavaScript seemed less interesting and rewarding than other choices, so I decided to jump straight to jQuery. I'm happy with how it worked out.

I had the same background as you. After 6 months of MooTools I found out that mootools was indirectly teaching me "vanilla" javascript.
I've heard people say that mootools feels more like plain ol' javascript than jquery (after all, jquery's tagline is "jQuery is designed to change the way that you write JavaScript.").
I'd recommend starting with a framework. Plenty of the best javascript developers use frameworks. Once you're comfortable there you'll likely be able to pick up the others frameworks and plain ol' javascript quite readily.