I've started work recently at a new company and they have an existing application with 1000s of lines of Javascript code. The baseline contains dozens of JS files with easily over 10,000 custom lines of code, they also use multiple 3rd party libraries such as Jquery, Livequery, JQTransform and others. One of the major complaints they have been receiving from users is the slowness of the client side operation of the site. I've been tasked with optimizing and improving the performance of the JS. My first step will be obviously to move forward to the newest Jquery library, and incorporate JSMin into the build process. Other than that I'm wondering if anyone has some tips on where to begin with optimization on such a huge code base?
You could try installing DynaTrace Ajax Edition (free download here) and see what that tells you. It supports only IE8 I think, but that's probably as good a place to start as any. It's got a much more thorough and understandable profiler interface than do either Firebug or Chrome, in my opinion.
One thing that jumps out at me is "Livequery", which if not used very carefully can cause gigantic performance problems.
Remember this: in a code base that big, developed over time and possibly not with the most "modern" Javascript techniques available, your real problems are going to be bad algorithms in your own code. Newer libraries and minification/optimization methods are good ideas, but the first thing you need to do is find the pages that seem sluggish and then start profiling. In my experience, in a big old codebase like that, you'll find something terrible really quickly. Install a desktop gadget that tracks CPU utilization. That's a great way to see when page code is causing the browser to slow down directly, and not just network lag. Any big spike in browser CPU usage for any significant amount of time should be a big red flag.
Profile that code. Don't optimize something if you just "feel" it could be optimized. Remember the 80% 20% rule. 80% of time is spent in 20% of code.
Use Google's Closure tools. They can optimize and reduce your JS code, which will at least cause it to load faster on your client's computers.
The way to go is to find bottlenecks. If you find the actual situtation where the app is slow, you can use Firebug to profile your code and tell how much time spent on every function and how many times they have been called. From this information it's pretty easy to determine what areas need some improvement.
Generally the bottlenecks of a webapplication are:
Working with the DOM extensively (repaints, reflows)
Heavy network communication (AJAX)
You have a long road ahead of you mate, and I dont envy you.
Here are some Performance Optimization Techniques for Javascript that I wrote down after working in a similar role as yours recently.
They are broken down into 5 broad categories in order of the performance difference they make.
However given what you said about the codebase, I think the second section on Managing and Actively reducing your Dependencies is the most relevant, particularly:
Modifying code to reduce library dependencies, and
Using a post-load dependency manager for your libraries and modules
However all 25 techniques listed there are useful for improving performance.
I hope that you find them useful.
Related
For a quite long time I was used to use the Flash Builder (previously called "Flex") enviroment to build complex applications, and in my current project (cant say what project is, sory) Im part of a RIA made in Flex, using mxml, AS3, some JS, etc.
The performance is below average.
The site feels like u need a powerful PC to run it something close to smooth.
And Im working with very talented ppl here, or at least thats what I can perceive.
So my question is, there is a posibility that the next version of the website will be made in pure HTML + CSS + JS, to obtain a more responsive feeling when u use the site, is something at least close to what FB can do for you in terms of a project-oriented enviroment?
On another hand, this general laggy and low performance thing is a programmer related issue, or it cant be avoided since "its just Flash" ?
Thanks!
(Im a big fan of flash tbh, but I try to keep my mind open in this)
Everything depends on what you're trying to do and if your programmers know how do to their jobs. If they don't take care, then it's very easy to create an application in Flash, and especially Flex (as you really need to know the internals of Flex to understand what it will do and when), that saps memory and power so that you need a beast of a machine to run it. Optimisation is a large topic, but fortunately, the web is covered in links on what not to do, or tips on how to use Flash to it's full potential.
As a quick test, open up the task manager, then run your app (close down all other programs using flash, including gmail, web radio etc), then watch the memory usage. If it's continually climbing, then there's a problem in your code.
Alternatively, if you have a bit more time, build some debug tools into your app so you can see what's going on in terms of framerate and memory usage - try the FlashPreloadProfiler for a quick fix: http://jpauclair.net/flashpreloadprofiler/
If your app shouldn't be complex but it's lagging (by complex, I don't mean the actual purpose of the app, but rather what it's showing on the screen, effects etc), then the problem is that there's misuse somewhere.
Go the HTML5 route if you feel it's the best solution for your site/app, but you'll probably encounter the same problems. In some situations, it's slower, in others Flash wins by a country mile. Again, it comes down to the devs
We built Dedoose in Flash Buidler 4, check it out... the performance is freaking great, and it's is a GIANT web app that handles an enormous amount of data. Flash Player in general does have performance problems, but with great engineering and knowledge of how to squeeze good performance out of it, I personally do not see a better alternative, excerpt for maybe silverlight if your requirements fit. For guidance on performance improvements a good place to start is all of Grant Skinner's articles over at http://gskinner.com/blog
Additionally, I can't really see what the actual question your asking is, what are the alternatives? HTML5+CSS is good, but good luck getting any good productivity out of your team developing a large scale web app in it. Silverlight is pretty dang good if you can force plugins down the users throat, Java is suffers the same dilemna. Personally I would read up more on getting good performance out of flash, then learn exactly how the flex framework works in terms of what it builts upon flash with, then... if u want real performance, ditch most of the flex framework and use things like MinimalComps and custom built components. Have you used the Flex Builder Profiler to analyze where your slow downs are occuring?
~ JT
If it's just an IDE you want, I'll give a nod to Flash Develop: http://www.flashdevelop.org/wikidocs/index.php?title=Main_Page
Not all the power of Flash (Flex) Builder, and without the design view. But it's free, open source, and community driven. I personally prefer the style of context highlighting and code completion hints in Flash Develop.
So I have been playing around with a home project that includes a lot of js. I having been using Script# to write my own library etc. Personally I wouldn't write a lot of js if I didn't have a tool like Script# or GWT to help maintain it.
So far it includes these external libraries:
– ASP.NET AJAX
– ExtJS
– Google Maps
– Google Visulisations
– My own library to wrap the above libraries and add extra functionality...
So that works out to be a heap of js. It runs fine on my pc. I however have little faith in js/browsers and I am concerned that loading too much js will cause the browser to die or perform poorly.
Is this a valid concern?
Does anyone have any experience with loading a lot of js into the browser that has resulted in performance issues? I however know there are a lot of variables here, for example browser type (I assume IE is worse than others) the client PCs RAM etc, but it would be good to get other peoples experiences. I would hate to invest a lot of time into js only to find that I am painting myself into a corner.
The more I use Script# the more client classes I have as I move more processing onto the client. At what point would this start becoming an issue? I'm sure the browser could easily handle 100 MS Ajax classes but at what would be too far for a browser?
NOTE: I am not concerned about the actual js file sizes but more the runtime environment that gets loaded.
There is nothing wrong with having large number of js files or big js files, the project currently am working on got more than 60 core framework libraries and 30 of each module got average of 5 to 6 js files.
So the only concern is how you design your website that make use of the JS best practices & optimization techniques. like
Minimize the JS using YUI or any other compression libraries to address the download size issues.
Enable proper caching in your webserver to reduce the file downloads.
Put your javascript in the bottom of the page, or make it a separate file.
Make your AJAX response cachable.
And finally, design your page that handles the on-demamnd script loading.
- Microsoft DOLOTO is a good example for this one. download it here
And Check out the High Performance Web Sites && latest Even Faster Web Sites by Steve Souders. Its a must read for the web developers. This book addresses all the common problems web developers facing today.
with modern browsers routinely occupying 250 MB of RAM or more, script caching, and optimized javascript engines, keeping the script library resident would probably be negligible added load in most reasonable scenarios.
the biggest bottleneck would probably be intitial load time of the scripts - downloading and parsing them. but once that's done, the scripts are cached and the per-page initialization isn't very noticeable.
I highly doubt a browser would ever crash running your JS scripts, but it will become really slow and may not perform what you want. Most people are more concerned about how fast it runs, not if it will run!
I agree with jspcal, you should be able to load quite a lot of javascript with no problems. The javascript engines in all the modern browsers are a lot faster than they were a few years ago. The initial load will be the biggest issue. If possible I'd suggest lazy loading scripts that aren't needed for the page to render.
Also, Steve Souders has a lot of great material about improving page load times, such as this article, which gives several techniques for loading scripts without blocking.
http://www.stevesouders.com/blog/2009/04/27/loading-scripts-without-blocking/
If you're really concerned about performance then I would take a look at your target audience. If you think you'll have a relatively high number of IE6 users then test it out in IE6-- on an older machine if possible. IE Tester is great for this.
We currently have a quite complex business application that contains a huge lot of JavaScript code for making the user interface & interaction feel as close to working with a traditional desktop application as possible (since that's what our users want). Over the years, this Javascript code has grown and grown, making it hard to manage & maintain and making it ever more likely that adding new functionallity will break some existing one. Needless to say, lots of this code also isn't state of the art anymore.
Thus, we have some ongoing discussion whether the client-side part of the application should be written anew in either Flex or Silverlight, or written anew with some state of the art JavaScript framework like jQuery, or whether we should simply carry on with what we have and gradually try to replace the worst bits of the existing code. What makes this even harder to decide is that writing the UI anew will probable cost us 6-12 person months.
I'd like to hear your thoughts on that issue (maybe some of you have already had to make a similar decission).
EDIT: To answer some of the questions that came up with the answers: The back-end code is written in C#, the target audience are (usually) non-technical users from the companies we sell the software to (not the general public, but not strictly internal users either), the software 'only' has to run in desktop browsers but not necessarily on mobile devices, and the client app is a full-blown UI.
In all honesty, I would refactor the old JavaScript code and not rewrite the application. Since you are asking about which platform to put it in, I would guess that your team isn't an expert in any of them (not slamming the team, it's just a simple fact that you have to consider when making a decision). This will work against you as you'll have double duty rewriting and learning how to do things on the new platform.
By keeping it in JavaScript, you can slowly introduce a framework if you choose and do it iteratively (Replace on section of code, test it, release it, and fix any bugs). This will allow you to do it at a slower pace and get feedback along the way. That way too, if the project is canceled part way through, you aren't out all the work, because the updated code is being used by the end users. Remember the waterfall model, which is essentially what a full swap out of will be almost never works.
As much as I hate to admit this, as it is always the most fun for developers, shifting platforms, and replacing an entire system at once rarely works. There are countless examples of this, Netscape for one. Here is the post from Spolsky on it. (I would also recommend the book Dreaming in Code. It is an excellent example of a software project that failed and how and why). Remember to rewrite a system from scratch you are essentially going to have to go through every line of code and figure what it does and why. At first you think you can skip it, but eventually it comes down to this. Like you said, your code is old, and that means there are most likely hacks in it to get something done. Some of these you can ignore, and others will be, "I didn't know the system needed it to do that."
These things spring to mind:
As you have a .Net backend and you have some ability to force your customers onto a specific platform, Silverlight is an option;
Since your client is a full-blown UI you want widgets and possibly other features like Drag and Drop;
I haven't seen any requirements that to me would justify starting over (which often doesn't work out) in Flex/Silverlight (eg streaming video, SVG support. Added to your team's familiarity with Javascript, I think you can't make a compelling case for doing it in anything other than Javascript.
But of course Javascript is lots of things and there are [lots of Javascript frameworks1. The most important divider is whether your intent is to "decorate" a set of Web pages or you need a full set of Widgets to create a desktop-like application on the Web. Your question indicate it is the latter.
As such--and I may get downvoted for saying this--I don't think jQuery is the answer and I say this as someone who loves jQuery. jQuery (imho) is great to enhance Webpages and abstract cross-browser low-level functionality but the most important factor for complex UI developer is this:
It's all about the widgets.
And yes I'm aware of jQuery UI but it's a lot sparser than the others when it comes to widgets. I suggest you take a look at the samples and widget galleries of some frameworks:
YUI Examples Gallery;
ExtJS demo; and
SmartClient feature explorer.
The others (jQuery, Dojo, Mootools, Prototype) are more "compact" frameworks arguably less suited to your purpose.
Also consider the license of each framework when making your decision.
My thoughts on the above three are:
ExtJS has somewhat angered the community in that it started out as LGPL but had a controversial license change (that thread is at 76 pages!) to GPL/commercial at version 2.1. The problem with that the community no longer has an active participation in the framework's development. Not the mainline version anyway. This means it's being developed and supported by a small team (possibly one person) rather than the community. IMHO it's not worth paying a commercial license for that and GPL is probably prohibitive in your situation;
YUI is supported by Yahoo and available under a far more permissive and far less invasive BSD license. It's mature, well-used and well worth serious consideration; and
SmartClient impresses me a lot. It has perhaps the most permissive license of all (LGPL), is roughly seven years old, has an incredibly impressive array of widgets available. Check out their feature explorer.
Your decision should be based on how you get as much of your application "for free" as possible. You don't want to spending valuable developer time doing things like:
Coding UI widgets like trees and accordions;
Testing and fixing cross-browser Javascript and CSS issues;
Creating homegrown frameworks that greatly duplicate what existing frameworks do and do well.
I would seriously look at one of the above three as your path forward.
This decision is usually less about the technology, and more about your skill sets and comfort zones.
If you have guys that eat and breathe Javascript, but know nothing about .net or Flash/Flex then there's nothing wrong with sticking with Javascript and leaning on a library like jQuery or Prototype.
If you have skills in either of the others then you might get a quicker result using Silverlight or Flex, as you get quite a lot of functionality "for free" with both of them.
My opinion on this one's pretty simple: unless the app needs to be accessible publicly, unless it needs to be search-engine optimized and findable, and/or there's an otherwise compelling case for its having to remain strictly text-based, then the chips are stacked in favor of rich-client runtimes like Flash or Silverlight right out of the gate.
A big reason, if not the biggest, is that they eliminate the complexities of developing for multiple browsers and platforms. Again: they eliminate the runtime-environment variable. No more debugging old versions of Netscape and IE, no more object detection and consequent branching, no more wacky CSS hacks -- one codebase, and you're done. Offloading the runtime environment to Adobe or Microsoft will save you time, money and headaches, all else equal. (Sure, there's YUI, JQuery, etc., but they don't eliminate that variable -- they just abstract it. And they don't abstract all of it, either -- only some of it; ultimately, it's still up to you to test, debug, retest, debug, repeat.)
Of course, your situation's a bit more complicated by the existing-codebase problem, and it's difficult to say definitively which way you should go, because only you've got the code, and we're just geeks with opinions. But assuming, just by your asking the question, that a refactoring of your existing codebase would involve a significant-enough undertaking as to warrant even considering an alternative (and probably comparatively foreign) technology in the first place, which it sounds like it would, then my response is that your curiosity is well-placed, and that you should give them both a serious look before making a decision.
For my part, I'm a longtime server-side guy, ASP.NET/C# for the past several years, and I've written many a text-based line-of-business app in my time, the last few with strong emphasis on delivering rich soverign UIs with JavaScript. I've also spent the last couple of years almost exclusively with Flex. I've got experience in both worlds. And I can tell you without hesitation that right now, it's everyone else's job to beat Flex: it's just an amazingly versatile and productive product, and for line-of-business apps, it remains leaps and bounds ahead of Silverlight. I just can't recommend it highly enough; the data-binding and event-handling features alone are incredible time-savers, to say nothing of the complete freedom you'll have over layout, animation, etc. The list goes on.
So, my advice: take a long, careful look at Flex. In the end, you might find a ground-up rewrite is just too massive an undertaking to justify, and that's fine -- only you can make that determination. (And to be fair, you've just as much ability to make a mess of a Flex project as you do with a JavaScript project -- I know. I've done it.) But all told, Flex is probably the least-limiting, most flexible, most feature-rich and productive option out there right now, so it's certainly worth considering. Good luck!
Any javascript you have that has been developed 'Over the years' probably doesn't look anything like what's possible today. You undoubtedly have a lot of useful code there. nonetheless. So my recommendation would be re-write in javascript using jQuery and make use of one of the available GUI add-ons, perhaps look at Yahoos stuff. You will also be targeting the widest audience this way.
The GUI technology should be first and foremost determined by your target audience. For instance, if you target englobes iPhone users, I would not recommend Flex, because iPhone doesn't have a flash player at the moment.
Bear in mind that if you switch to a full fledged GUI toolkit like Silverlight, your users may find the L&F unnatural, since the usual request-reply cycle is not so evident with client-side frameworks.
After that, it is your developers that should have a word to say. Every toolkit needs maintenance, and if you are switching to a whole new toolkit the developers will have to familiarize with the new toolkit, which can be costly.
My suggestion is that you stick to javascript, since your devs are familiar with it, and gradually replace the old javascript with a new toolkit like prototype, jQuery or any other. You will probably redo some of the old stuff faster using a state-of-the-art toolkit. Remember that you can build beautiful apps with any tookit.
We have developed an extremely rich application using EXTJS with C# and a some C++ on the server. Not only do we have clients who are happy with the interface in their desktop browsers but with very little tweaking to the Javascript we were able to provide web browser support. Also, we have clients in third-world countries who cannot use Flash or Silverlight apps due to their field personnel using kiosks in internet cafes (many of which don't have Flash installed - forget Silverlight!). I think these issues and others make up for the difficulty of coding a complex app in javascript...
Check this comparison table for Flex vs Javascript:
I'm going to develop a comprehensive educational software which runs on the browser and has many visualization and simulation works (electrostatic and electromagnetic visualization, 2D and 3D).
Which language(Processing, javascript or something else) is the best toward my purpose?
The question is indeed broad but I will answer from the experience I've had.
Javascript is not really meant to do mathematical calculations, which is what might be necessary to calculate a lot of E&M phenomenon quickly (Especially if they are not represented as a closed form solution). It really goes into how much detail you want in your graphs as well (More steps = more calculations). You may find yourself needing to do more optimizations to make up for the performance difference.
I did some visualizations of antenna arrays (They had closed form solutions, only simple arrays) in Flash and it worked out ok. Javascript will definitely not be up to par with any 3D simulations you might want to do.
I wonder if Silverlight might be a better solution, because you may find more mathematics libraries for .NET than for Actionscript, that could save you a lot of work of writing the math out yourself (But you might end up doing this anyways because of the performance issues).
As others have suggested javascript is not that strong of a language when it comes to visualization.
Processing is a really good language for what you're trying to do, it's easy to learn and is Java based. Data visualization is built directly into the language, as well as temporal space (ie advance "1 tick" in time and have the visualization react to that.)
Also if you're interested in going that route I'd suggest picking up Visualizing Data which is pretty much a Processing primer.
Flash may be the more common application stack right now for what you are looking for, but Silverlight is looking primed to take the title from them based on the powerful features that it contains.
I would go Flex or Silverlight myself
Plenty of re-usable libraries
Native support for multimedia
Native support for graphics and animation
I'm a little late to the show, but what you want, has been implemented in JavaScript, and you'll find this incredibly useful. I recommend running it under Chrome as the JS processing engine is extremely fast. (You may even want to try Chrome 2 which is even faster)
http://ejohn.org/blog/processingjs/
http://ejohn.org/apps/processing.js/examples/basic/ (91 basic demos.)
http://ejohn.org/apps/processing.js/examples/topics/ (51 larger, topical, demos.)
http://ejohn.org/apps/processing.js/examples/custom/ (4 custom "in the wild" demos.)
See also: http://www.chromeexperiments.com/
I second LFSR Consulting's opinion: Processing is used a lot for educational purposes, it is free, and fast (Java is faster than Flash in general) and easy to learn, so you have faster results. It supports 3D, you can tackle Java libraries for simulation and computing, etc. And it has a great community! :-)
JavaScript is a bit light for such usage. JavaFX is hype, but it hasn't really 3D (although one used Java3D with it) and it is still a bit young.
Flash and Silverlight: no comment, not much experience in the field. OpenLazlo can be an alternative...
You really have two choices ActionScript in Flash or VB.NET/C#/other in Silverlight.
So first you need to decide which of these platforms you will target.
You may be able to split the problem into two parts, the user-interaction and display part, and the heavy calculations part.
If you can move the heavy calculations to a server then you can still show everything in javascript.
One difficulty with javascript is that it is interpreted and you will need to write more of the equations yourself, so there is a performance hit and development time, but it will work without any plugins, unless you don't want to do 3D in the canvas tag.
Flash and Silverlight may have better options, but then you are learning new languages and requiring plugins, depending on what version of Flash you want to use.
Check out processing.js, xcode, and iprocessing!
ProcessingJS is great for data visualization but lacks in interactivity.
You should probably try python. It is a really good language for educational and computational purposes it has a pretty decent community and the syntax is not so tough. Even though it was designed to for command line you can create front end gui's for it using some external package and it also provides packages like Scipy, Numpy and Matplotlib for advanced plotting and data visualization.
I'm talking about things like page/stylesheet caching, minifying javascript, etc.
Half of me thinks it's better to do these things as early as possible while still in development so I can be consciously aware of more realistic speed and response issues as well as interacting with something that more closely resembles what will be deployed into production, but the other half of my brain thinks it makes more sense to not do anything until just before launch so that I'm constantly working with the raw data that has not been optimized while in development.
Is there common or conventional wisdom on this subject?
I do all optimizations at the end. That way I know when something doesn't work it is because the code is wrong. I've tried to optimize things too early at times, and realized that I wasted an hour because I was caching something etc.
Realize that a user spents most of his time waiting on frontend objects to (down)load. Your application may generate html in 0.1 second but the user spends at least 2 seconds waiting on all the images etc to load. Getting these download times to a small number will positively increase the user experience.
A lot can be done already by enabling GZIP and using minified javascript libraries. You should download and install YSlow and configure your webserver with appropriate caching headers. This alone can save hundreds of miliseconds loading time.
The last step is to optimize the amount of images using CSS sprites. Other steps can include minimizing css and javascript, but this will gain the least of all methods I mentioned already.
To summarize, most of this can be done by properly configuring your webserver, the sprites however should be done during development.
I'm a fan of building the site first, then using a user experience profiler like YSlow to do the optimizations at the very end.
http://developer.yahoo.com/yslow/
I should add that a profiler of some sort is essential. Otherwise, you're optimizing without before/after data, which is not exactly scientific (not to mention you won't be able to quantify how much improvement you made).
Premature optimization is the root of all evil :)
Especially early in development and even more so when the optimizations will interfere with your ability to debug the code or understand the flow of the program.
That said, it is important to at least plan for certain optimizations during the design phase so you don't code yourself into a corner where those optimizations are no longer easy to implement (certain kinds of internal caching being a good example).
I agree with your premise that you should do as much optimization in the early stages as you can. It will not only improve development time (think: saving 1/2 seconds per refresh adds up when you're spamming control+r!) but it will keep your focus in the end -- during the time when you're refacting the actual code you've implemented -- on the important stuff. Minify everything that you won't be modifying right off of the bat.
I agree with the other half of your brain that says 'do what you have to do, and then do it faster'. But for anything you do you must know and keep in mind how it can be done faster.
The main problem I see with this approach is that at the end its easier to ignore the fact that you have to optimise, especially if everything seems to be 'good enough'. The other problem is that if you are not experienced with optimisation issues you may indeed 'code your self in a corner' and that's where things tend to get really ugly.
I think this might be one where it's difficult to get a clear answer, as different projects will have different requirements, depending on how much work they are doing on the client side.
Mine rule of thumb would be probably later rather than sooner. Only because a lot of the typical front-end optimisation techniques (at least the stuff I'm aware of) tend to be fairly easy to implement. I'm thinking of whitespace stripping and changing http headers and so forth here. So I would favour focusing on work that directly answers the problem your project is tackling; once that is being addressed in an effective way move onto stuff like optimising front-end response times.
After coding for a number of years, you get to have a pretty clear idea what the performance bottle necks will be.
DB Queries
JS/Jquery functions
cfc's
images
javascripts
By identifying the bottlenecks ahead of time, while you create/modify each piece of the application that deals with those bottlenecks, you can spend time tweaking, knowing that by spending time while coding, gives you better performance in the end.
Also tends to make us less lazy, by always creating optimal code, and learning what that is in real life.