We Keep Coding

Framework to structure existing JS code

I have some procedural javascript code that I have written for an open-source application and I'd Like to refactor it into OOP and since I have very little experience with javascript frameworks I have trouble finding a one suitable for my needs, though I haven't tried anything yet, I just read about AngularJS, Backbone.js and Knockout.
I want to structure the code, because, at the moment, there's a mess, global variables and functions scattered around.
I have to mention that all the business logic is handled at the server level, so the client-side code handles just the UI using the data it receives or requests from the server.
The code can be found here:
https://github.com/paullik/webchat/blob/asp.net/webchat/Static/Js/chat.js
Do you have any suggestions?

Object-Oriented JavaScript is not necessarily the answer to all your
problems.
My advice is to be careful the choice you pick on this topic.
In practice, OO-JS can add more complexity to your code for the sake of trying to be more like traditional object-oriented languages. As you probably know, JS is unique.
It is important to know that there are Design Patterns that will structure your code and keep implementation light and flexible.
It is Design Patterns that I see structuring advanced JS
implementations, not OO. To paraphrase Axel Rauchmeyer - "Object
Oriented methodology does not fit into basic JavaScript syntax, it is
a twisted and contorted implementation, and JS is far more expressive
with out it."
The root of this analysis boils down to the fact that JS has no class. In essence, since everything is an object, you already have object-oriented variables and functions. Thus the problem is slightly different than the one found in compiled languages (C/Java).
What Design Patterns are there for JavaScript?
An excellent resource to check is Addy O' Somani and Essential Design Patterns.
He wrote this book on Design Patterns in JavaScript.
But there is more... much more.
A. require.js - There is a way to load modules of JS code in a very impressive way.
These are generally called a module loaders, and are widely considered the future of loading js files since they optimize performance at runtime. yepnope and others exist. Keep this in mind if you are loading more than a few js files. (moved to top by request).
B. MVC - There are dozens of Model View Controller frameworks to help you structure code.
It is a Pattern, but may be unreasonable for your purposes. You mentioned backbone, knockout and angular... Yes. These can do the trick for you, but I would be concerned that they might be 1) high learning-curve, and 2) overkill for your environment.
C. Namespace or Module Pattern. Are probably the most important for your needs.
To solve global variables just wrap them in a namespace and reference that.
These are very good patterns that give rise to module loaders.
D. Closure - You mentioned OO JS. On piece of this that is useful is the notion of closures to provide yourself with... private members. At first this is a cryptic idea, but after you recognize the pattern, it is trivial practice.
E. Custom Events - It becomes very important to not use hard references between objects. Example: AnotherObject.member; This is because it will tightly couple the two objects together and make both of them inflexible to change. To solve this, trigger and listen to events. In traditional Design Patterns this is the Observer. In JS it is called PubSub.
F. The Callback - The callback pattern is what enabled AJAX and it is revolutionizing development in terms of Window 8, Firefox OS, and Node.js - because of something called non-blocking-io. Very important.
Do not be afraid. This is the direction to go for long-term and advanced JavaScript implementations.
Once you recognize the patterns, it is down hill from there.
Hope this helps.

Is mootools alternative of jquery + backbone / spine / sprouteCore

I was full time java developers, now I also work on JavaScript. couple of years back when I started learning JavaScript, first library I tried was jquery like most of the people. But it made my life harder, and after sometime I started writing fairly large JavaScript app. It wasn't coming together for me using jquery. I had huge codebase without much of a structure. It was method blocks updating HTML blocks using selectors.
Then I tried mootools and obliviously as a java developer it appealed to me a lot. And I was able to write managible web apps having huge code base.
As per my understanding Mootools is not considered a preferred way to write JavaScript because it mimic conventional OO over default prototype-based OO language. So now to really understand javascript and desire of walking with the world, i decided to try other approaches, so again I turned back to jquery, and realise that only jquery is not enough. So started looking at current trending frameworks like backbone, spine, ember.js, sprouteCore. Strangly I found that these frameworks at their core tries to mimic conventional OO like mootools only by having constructors and creating a object of class and reusing this class object to create instance objects. So
Am I missing something?
Is mootools way really wrong?
Mootools project is very alive and releases new versions/features, but I don't
see many people talking about it on Internet, also there are no comparisions vs backbone/spine etc.

As per my understanding Mootools is not considered a preferred way to write JavaScript because it mimic conventional OO over default prototype-based OO language.
Where do you get that from? The greatest thing about javascript is that is so loosely typed (see what I did there) - you can write the same thing in a platitude of ways. There are also so many ways to abstract it and repackage it - and this applies from a simple new Array() vs [] to how you structure your app.
If you love JavaScript (or just know it and secretly hate it), you will be fine with MooTools. The API is mostly either native js or ES5 spec or - rarely - an extra utility that also feels 'natural'. The notable exception that stands out is Class. And the fact that you can abstract having to deal with prototypical inheritance by passing an object to a special constructor Type function that returns your instance is ... oh wait. It looks different but what it does pretty much sounds like normal javascript. Only easier -- why wouldn't you prefer that?
A lot is being made these days of the clientside MVC boom and this 'new way of developing apps'. Suddenly, jQuery folks were given the luxury of tap water! I have spoken to a lot of MooTools developers over this and (un)surprisingly discovered that most think MooTools rarely needs anything like that. I tend to agree with them. The only gaping hole is a view controller with templating but there are a fair few solutions.
The thing is, you cannot directly compare a MVC framework with MooTools, it is not the same. At all. You can compare the so called Model constructors vs Classes.
I have now spent a while researching various MVC framework solutions and patterns to see if our new app can be moulded into a 'best practice' shape.
Basically, I tried backbone.js (with and w/o a mootools adapter) and found it awkward to use after MooTools - it felt like a step back. When I say use I don't mean I can't use it but it feels awkward to extend and build on. I am sure it's just down to experience, though, have yet to read all the backbone patterns examples out there.
Typical issue I run into - wanted to have a special Model property that tells it to use localStorage to fetch/save. Not obvious how - examples tend to show you can either route Backbone.sync to one or the other but not both at the same time. I had to actually decorate the function and extend it, keeping the original referenced in case the model did not require localStorage. NOT the best / most obvious pattern to maintain and leaves me dependent on their changes not breaking my code.
In MooTools, I would have just extended my Model class and could have defined a custom Class Mutator property (like Binds or Implements). Done. Write what you know, they say, and not for nothing...
Another issue - it's tightly coupled with data and you cannot reuse models like classes - eg, a User model loads a user and renders through a User Edit view. You then want to create a new customer and suddenly, you cannot reuse the old object that easily and just render the same view but with empty values. I think it will also be down to inexperience on my part or bad architecture.
Ember.js I found slightly more moo-ish as an interface though It did not quite click either. Frankly, backbone was less trouble to setup.
There are other attempts. Composer is one - once again for mootools but it tries too hard to be backbone and is written by people that are relatively new to the framework so I would not call it mature. Knockout etc etc. There's a new one every day, literally.
Garrick Cheung released a framework called Neuro which has huge potential.
I wrote Epitome - a full MVP implementation based upon classes and events and wrapped in AMD modules, feel free to check it out. It also comes with a builder, documentation builder and many little goodies to get you started.
SeanMonstar released Shipyard, which is used by Mozilla Flight Deck - http://seanmonstar.github.com/Shipyard/. Whereas it's not native mootools, it's mootools-ish with mootools class etc - only w/o extending natives, so a great alternative.
BTW, try irc.freenode.net #mootools or the mail list and you will always get a good answer.
Anyway, enough on MVC. The points about MooTools have been made countless of times. Haters will be haters. Those that love it don't look back. If you are a programmer from an OOP background or are looking for something that renders itself well to patterns, do yourself a favour and stick with it. Exciting times are ahead. Roadmap for 1.5: AMD, for 2.0 (aka, Prime) Host object prototyping optional. These have been the two biggest talking-down points in the eyes of critics. No more 'dirty' prototypes so people can get on with using for ... in loops incorrectly on non-objects and without hasOwnProperty checks. Anyway...
Other things to worry about may be of importance. Like, the size of the 'community'. I think having a healthy community is a great thing but even if you look at jquery, the amount of actual contributors vs users is low. The ratio of quality CODE vs good looking effects is bad. The plugins you can use - a lot are not well written or dead and unsupported. When you draw the line, it's a lot less glamorous than you'd think!
I am not saying that mootools or other frameworks don't have these problems. It is fair to say MooTools people and especially the core devs are fairly private and less vocal about what they do. It may send the wrong impressions, I don't know. It certainly is no jQuery.
Ultimately - if you have the resources and the know-how, use what works best and what will scale. There are even these that use coffeescript and swear by it. Who am I to judge...
In the interests of full disclosure - you will find it MUCH harder to find a decent mootools dev when you recruit. Cannot be ignored...

Javascript: How to make sense of all the frameworks and design philosophies?

I have been a user of jQuery (and some of its minor plugins) for a while. The Javascript code I've developed over the years could be described best as... messy. It used a ton of global variables and functions here and there, didn't use standard ways of organizing the code, nor any design patterns whatsoever.
I am currently building the new version of a website, and I have completed doing the backend with PEAR::MDB2 and Smarty templates. The rest is just homebrew PHP with some classes.
Now I am at the point where I'll add the Javascript layer on top of the website to improve the user-friendliness of some features. (while making sure everything degrades gracefully) I want to write better, cleaner, more organized Javascript than I used to, so I did a little research. I read Stefanov's Object-Oriented Javascript to have a better grasp on some concepts I knew only loosely about (prototypes, constructors, etc.) as well. Now I'm stuck at a point where I wonder which Javascript frameworks I should use, and how to organize it all.
After conducting my research, I understood Cappuccino & Objective-J, and Sproutcore were not what I was looking for. To quote Cappucino's about page:
Cappuccino is not designed for building web sites, or making existing sites more "dynamic". We think these goals are too far removed from those of application development to be served well by a single framework. Projects like Prototype and jQuery are excellent at those tasks
So there's that. Then I found out about Coffee Script, which is more of a one-to-one "compiler" and wouldn't help me with the actual organization of my code.
I also stumbled on some articles that give guidelines:
Using Inheritance Patterns to Organize Large jQuery Applications
A JavaScript Module Pattern
I also found out about Backbone.js, Shoestring, JavaScriptMVC, Google Loader, jQuery Tools, jQuery UI. I don't really know what to do of all this... The things I know:
I don't want to invest too much time in learning something too complex, I want to keep things simple and flexible as much as possible (that is why I don't use Symfony on the backend, for example), yet clean and organized.
I want to use jQuery, the question is, what should I use with it? (that is compatible too)
Right now, I'd use jQuery and jQuery Tools and "organize" all that in a simple namespace/object literal with simple properties and methods and also, since the site is localized, I just plan on using the simple vsprintf (as I do on the backend) with key:value pairs loaded from an object literal provided by the backend. JavaScriptMVC seems interesting, but I fear it would bring way too much complexity for a project that is fairly small sized. That is where I need your advice! Thank you very much in advance.

Ok, my attempt at an answer:
There is no 'best' to way to do it. You now know what's there and I think you might have a preference for yourself for what you want. In that case, pick a framework and learn it inside-out. (sorry to burst your bubble, but each framework has a learning curve, some steep, some very easy, but in the end to use it well you have to invest in it. Just do it, you won't be sorry).
You of course have an preference for clean code, so you might take some considerations into account. You also say you have a preference for jQuery, which is fine, but there are some limitations (as also pointed out in the link provided by eskimoblood).
There are some nice lectures / and tutorials with advice on how to structure your code in jQuery:
How to manage large jquery apps
On Large jQuery apps
Essential Javascript and jQuery patterns (free ebook)
Some style guides:
Jquery core UI Styleguide
Google Closure Javascript Style Guide
Tools for checking your code
JSLint
JSHint (a more forgiving/practical fork)
Closure Linter (haven't tried it yet, but intend to)
Standard works (javascript)
Everything by Douglas Crockford
Quirksmode
There might be more.. perhaps more people can contribute, but I also think that you've almost reached the end of what you can learn before getting your hands dirty. Many of these guides are written in a very generic way, but the interesting thing is that javascript is called upon in many specific situations. It might be useful to just post some of the code that you regard as "messy" and we can help you figure out how to do it better. Good luck!

You should watch the video and read the links in this article and then you should ask yourself again if jquery is the right tool. Maybe you will use dojo, that is much better for larger projects or you take a look at backbone and where you can stay with jquery. After all both of them are more "javascriptish" then something like sproutcore, cappuciono or even GWT. And also much easier to understand when you come from jquery.

One framework that is to consider is definitely ReactJS from Facebook. This framework is pretty slick in many ways.
First thing you have to know is that it is a view framework. It can be used server-side to do the pre-rendering of pages, but it really shines on client side. Since it's a view framework, it can be used with backbone or any other "back-front"-end framework.
One of the main point of React is its rapidity. It keeps a virtual DOM in memory and virtualize all the webpages events. So the virtuals event are used to keep events browser agnostics.
The virtual DOM kind of make programming a dynamic site as if you were programming an old static website. You can just shoot the whole HTML to render to the view engine (as if you were "re-rendering" the whole page) and it will manage the DOM operations. It does a diff between the new virtual DOM and the current virtual DOM and only inserts nodes that needs to be inserted. This way you reduce the number of DOM ops and thus increase your render speed by a lot.
A good place to start is this tutorial which shows how to use "Flux" (the web flow designed by Facebook for its site) in order to realize a Todo application!

Wrap jQuery, dojo with custom library?

Coming from Java, I'm wondering if a Java best practice applies to JavaScript.
In Java, there's a separation of interface and implementation, and mixing them up is considered a bad practice. By the same token, it is recommended to hide implementation details of your library from end developers.
For example, log4J is one of the most popular logging libraries out there but it is recommended to write code to the slf4j library or the Commons Logging library that "wraps" log4j. This way, if you choose to switch to another logging framework such as logback, you can do so without changing your code. Another reason is that you, as a user of a logging library, how logging is done is none of your concern, as long as you know what logging does.
So back to JavaScript, most non-trivial web applications have their own custom JavaScript libraries, many of which use open source libraries such as jQuery and dojo. If a custom library depends on, say jQuery, not as an extension, but as implementation, do you see the need to add another layer that wraps jQuery and makes it transparent to the rest of JavaScript code?
For example, if you have the foo library that contains all your custom, front-end logic, you'd introduce the bar library that just wraps jQuery. This way, your foo library would use the bar library for jQuery functions, but it is totally oblivious to jQuery. In theory, you could switch to other libraries such as dojo and google web toolkit without having a big impact on the foo library.
Do you see any practical value in this? Overkill?

Although it makes sense from a theoretical standpoint, in practice I'd say it's overkill. If nothing else for these two reasons:
Anything that adds to the size of
the request (or adds more requests)
is bad - in web world, less is more.
If you're using say jQuery, the
chances of you switching to
something like Mootools is (imho) slim to none. From what I've seen, the top libraries each aim to solve different problems (at least in the case of Mootools and jQuery - see this great doc for more info on that). I'd assume that you'd incur a tremendous amount of headache if you were to try to implement a middleware library that could easily switch between the two.

In my experience and being a Java developer myself, sometimes we tend to take the whole "abstraction" layer pattern too far, I've seen implementations where someone decided to completely abstract a certain framework just for the sake of "flexibility" but it ends up making things more complicated and creating more code to maintain.
Bottom line is you should look at it on a case by case basis, for example you wouldn't try to create an abstraction layer on top of struts, or on top of JPA, just in case you then go to a different framework (which I've rarely seen done).
My suggestion is, regardless of the framework you are using, create objects and components that use the framework internally, they should model your problem and be able to interact between them without the need of any specific framework.
Hope this helps.

There are a lot of good answers here, but one thing I don't see mentioned is feature sets. If you try to write a library to wrap the functionality provided by, say, jQuery, but you want to be able to easily swap out for something like prototype, you have a problem. The jQuery library doesn't provide all the features prototype provides, and prototype doesn't provide all the features jQuery provides. On top of that, they both provide their features in radically different ways (prototype extends base objects -- that's damn near impossible to wrap).
In the end, if you tried to wrap these libraries in some code that adds 'abstraction' to try to make them more flexible, you're going to lose 80% of what the frameworks provided. You'll lose the fancy interfaces they provide (jQuery provides an awesome $('selector') function, prototype extends base objects), and you'll also have to decide if you want to leave out features. If a given feature is not provided by both frameworks, you have to either ditch it or reimplement it for the other framework. This is a big can of worms.
The whole problem stems from the fact that Java is a very inflexible language. A library provides functionality, and that's it. In JavaScript, the language itself is insanely flexible, and lets you do lots of crazy things (like writing a library, and assigning it to the $ variable). The ability to do crazy things lets developers of javascript libraries provide some really creative functionality, but it means you can't just find commonalities in libraries and write an abstraction. I think writing javascript well requires a significant change in perspective for a Java developer.

Someone wise once said "premature optimization is the root of all evil." I believe that applies in this case.
As others have expressed, you don't want to abstract for the sake of flexibility until you have an actual need for the abstraction. Otherwise you end up doing more work than necessary, and introducing unnecessary complexity before it is required. This costs money and actually makes your code more brittle.
Also, if your code is well organized and well tested, you should not be afraid of major changes. Code is always changing, and trying to anticipate and optimize for a change that may or may not come will almost always get you in more trouble than it saves you.
Acknowledgement: I should give credit to Agile programming and my practice and readings on the topic. What I've said comes directly from my understanding of Agile, and I've found it to be an extremely good razor to cut out the extra fat of my work and get lots done. Also none of what I've said is actually JavaScript specific... I'd apply those principles in any language.

There are good arguments calling this development practice - wrapping in order to switch later - into question in any language.
A good quote by Oren Eini, from his writeup on wrapping ORMs:
Trying to encapsulate to make things
easier to work with, great. Trying to
encapsulate so that you can switch
OR/Ms? Won’t work, will be costly and
painful.

This is definitely something that is done in enterprise environments.
Take for example a company that has their own custom javascript framework that is used on all of their projects. Each of the projects decide to use their own framework (jQuery, Dojo, Prototype) to add functionality to the underlying modules of the company framework. Employees that move between projects can now easily do so because their API with working the project's codebase is still the same, even though the underlying implementation could be different for each project. Abstraction is helpful in these situations.

It is overkill. Javascript is not Java and is not in any way related to Java. It is a completely different language that got J-a-v-a in the name for marketing reasons.
If you are concerned with availability of add-on libraries, then choose a framework with a large ecosystem. In an enterprise environment you will be further ahead by standardising on a vanilla off-the-shelf uncustomised web framework that you can upgrade every year or so tracking the rest of the world. And then supplement that with a SMALL in-house add-on library which you will, of course, have to maintain yourself, not to mention training any new programmers that you hire.
Since you are talking about Javascript in the client (web browser) it is more important that you limit the complexity of the things that people do with it. Don't build huge amounts of client side code, and don't make stuff that is so brittle that another programmer can't maintain it. A web framework helps you both keep the linecount down, and keep your own code reasonably simple.
It is not a question of Javascript best practice, because that would be different for server-side JS such as Rhino or node.js.

Adapter pattern is not a common solution in this case. The only example I know to use this pattern is extjs. Javascript projects are usually too small and they aren't worth the effort you would make by creating such an abstraction layer.
The common solution for this problem is that you try to use multiple frameworks together for example with jquery.noConflict.

I've done this before, and can talk a bit about the experience of writing a library/toolkit wrapper.
The plan was to move from Prototype to some other library. Dojo was the first choice, but at the time I wasn't sure whether that's the library to move everything to (and by everything I mean ~5MB of Prototype-happy JS). So coming from a world of clean interfaces, I was set to write one around Prototype and Dojo; an awesome interface that would make switching out from dojo a breeze, if that was in fact necessary.
That was a mistake that cost a lot of time and effort for a few reasons. The first one is that although two libraries can provide the same functionality, (a) their API will almost always be different, and most importantly (b) the way you program with one library will be different.
To demonstrate, let's take something as common as adding a class-name:
// Prototype
$("target").addClassName('highlighted');
// Dojo
dojo.addClass("target", "highlighted");
// jQuery
$("target").addClass("highlighted");
// MooTools
$('target').set('class', 'highlighted');
Pretty straight-forward so far. Let's complicate it a bit:
// Prototype
Element.addClassName('target', 'highlighted selected');
// Dojo
dojo.addClass("target", ["highlighted", "selected"]);
// jQuery
$("target").addClass(function() {
return 'highlighted selected';
});
// MooTools
$("target").set({
"class": "highlighted selected"
});
Now after choosing an interface for your version of the addClass you have two options: (1) code to the lowest common denominator, or (2) implement all of the non-intersecting features of the libraries.
If you go with the 1st -- you'll loose the "personality" / best qualities of each of the library. If you go with #2 -- your addClass' code will be at 4 times larger than the ones provided by any of the libraries, since for example when Dojo is included, you'll have to write the code for the function as the first param (jQuery) and the Object as the first param (MooTools).
Therefore, although it is theoretically possible, it isn't practical, but is a very nice way to understand the intricacies of the libraries out there.

Is there a general purpose JavaScript library out there?

We started a new project and realized that we needed a general purpose javascript library that contains a nice set of string functions, MD5, base64, allows extensions, etc. Also, copying and pasting functions from other libraries doesn't sound very attractive.
So, I guess the question is which javascript library contains the most general purpose functionality out there? or maybe there is a good collection of global functions out there we could use/extend. We know DOM manipulation is covered by many AJAX libraries including JQuery.
*Mind you, we could alternatively extend ExtJS, JQuery, etc. Is that what you guys are doing?

Google Closure Library
It contains (quoted from link):
a large set of reusable UI widgets and controls, and from lower-level utilities for DOM manipulation, server communication, animation, data structures, unit testing, rich-text editing, and more.
It also contains a nice set of string manipulating methods, in goog.string namespace.
Underscore
Underscore is a utility-belt library for JavaScript that provides a lot of the functional programming support that you would expect in Prototype.js
Underscore is intended to go along with other library, like jQuery or prototype.
It's not extensible like jQuery or Google Closure, though.

*Mind you, we could alternatively extend ExtJS, JQuery, etc. Is that what you guys are doing?
Yes, I do and I think most are. A lot of what you describe as a "general purpose library" is covered by Frameworks like JQuery, Prototype or Moo. And short of clipping the webmaster's nails, there's a JQuery plugin for everything that's not already in the core.
Still, I'm interested to see whether any other "general purpose" libraries come up here. There are fields - like string manipulation, as stated in one of the comments to another answer, and advanced date operations - where none of these frameworks is the holy grail AFAIK.

I use the jQuery library and a bunch of plugins. jQuery's plugin directory contains a lot of useful tools. There's also jQuery UI, a set of interactive components and effects, which you can use if you don't want to use a more complex library like ExtJS.
Of course every project is different, and you will probably end up writing some helper functions on your own.

I realize that others are going to say the same, but jQuery truly amazes me every time I learn something new about it.
DOM, CSS, and event manipulation along with easy AJAX, extensibility, and the plethora of existing extensions make jQuery a wonderful tool for web development.

jQuery is incredibly useful for UI manipulation. However, being open source, it contains some less than optimal code. If you start running into performance issues, don't be afraid to delve into the source and see what's going on.

I have been using jQuery for some time now and that seems to handle most of the basic operations I need. It has a healthy library of plug-ins and you can always write your own. It is a very good lightweight js library and even if it doesn't do all that you need it to you it is a good starting point.