i am currently investigating possibilities for profiling/benchmarking javascript code in terms of runtime behavior. I know that there are some decent tools if you run the code in your browser (firebug, chrome dev-tools, etc.), but I am looking for a method that can be automated for testing.
If possible, I would like to integrate performance tests (e.g. execution time of certain functions/code blocks) into a CI cycle. Additionally, i would like to collect runtime information about memory allocations.
Currently i track the execution time of functions/code blocks by adding tracking-code by hand. Has anybody implemented a better solution to do this? How do you profile/benchmark your JS code?
Please share your experience with me!
Thanks
John Resig wrote in 2009 about a free tool that profiles IE (and now Firefox, too) in detail. According to the vendor the Premium Version also integrates with Selenium and supports Web Driver scripts.
Related
Some code in my page is making my browser slow after 20-30 min. I need to know which one. What tools can i use to debug this out.
Following js files are being loaded
Jquery
Jquery ui
History
Mustache
Yes i had settimeout and thought that to be the culprit but alas after removing it too, it's still a bit slow.
You should use Google's Speed Tracer. It's a Chrome extension.
Speed Tracer is a tool to help you identify and fix performance
problems in your web applications. It visualizes metrics that are
taken from low level instrumentation points inside of the browser and
analyzes them as your application runs. Speed Tracer is available as a
Chrome extension and works on all platforms where extensions are
currently supported (Windows and Linux).
Alternatively, you have Yahoo!'s YUI 2: Profiler.
The YUI Profiler is a simple, non-visual code profiler for JavaScript.
Unlike most code profilers, this one allows you to specify exactly
what parts of your application to profile. You can also
programmatically retrieve profiling information as the application is
running, allowing you to create performance tests YUI Test or other
unit testing frameworks.
An addendum to #Julio Santos ' answer
You can use Dynatrace Ajax which has a good free version of their product
Is there a way to check the resource usage with a javascript code?
Can i check RAM usage and cpu usage of the script?
Since there are various ways to do something, i might write the code using different methods, and save it in as 2 different files, and check which method is more optimized. Esp when i'm calling a function recursively.
This way, i'll get to learn which methods are better and what to use.
Anything like maybe an addon, or maybe a script to be added that does this. It would be much better if it shows function wise. I'm not sure if something like this exists.
On a note, with chrome inspector, i tried CPU profiling, but it seems to show me values according to time, and does not show the RAM/CPU usage.
There's no uniform cross-browser way to do this. You can get an overall sense of your javascript performance by using Chrome's Timeline tool. Switch between the Timelines and Memory tabs on the left and hit the record button at the bottom. I believe Safari has a similar tool.
Internet Explorer has the excellent DynaTrace tool, although I don't know it well enough to provide detailed instruction on how to use it.
Note that javascript interpreters vary wildly in implementation, so they may have very different performance characteristics. Rather than get caught up in the details of implementation (premature optimization is evil and all that), write your code with good coding practices in mind. That means that if you do have performance bottlenecks (which the profiling tools available will help you locate), you can refactor more readily.
I don't know any specific way to test the Ram and CPU stats of javascript. But I often user http://jsperf.com/ to see how two different functions perform.
Have you tried with Crome Tools > Task Manager in Chome Tools menu options? It shows the CPU and Memory Usage for each process running in the borwser (comprise chrome-extensions).
I'm developing a fairly large page with several external javascript files. In IE6, the page hangs for a few seconds just as it's finished loading. I'm not (directly) asking for help debugging this problem, I'm only asking for help finding a tool to help debug this problem.
What I'm looking for is something similar to Chrome Developer Tools's "Timeline" tab. Something that will tell me the run time of my various external scripts (or functions, event handlers, whatever level of granularity is available).
I'd much prefer something that is free, but if that isn't available, I may be convinced to shell out some cash for a little extra quality.
Please keep in mind that I'm looking for tools that will work with Internet Explorer 6.
I think Dynatrace could be what you're looking for.
It's quite complex but integrates well with IE and gives you more data than you could ever use: Number of times functions get called, CPU usage per function, a step-by-step look at the DOM tree being built... It's pretty intense.
Godspeed.
Fiddler2 will work well for tracking all aspects of the load time... unfortunately no pretty timeline chart though.
I've realized I need a full-fledged browser automation tool for testing user interactions with our JavaScript widget library. I was using qunit, starting with unit testing and then I unwisely started incorporating more and more functional tests. That was a bad idea: trying to simulate a lot of user actions with JavaScript. The timing issues have gotten out of control and have made the suite too brittle. Now I spend more time fixing the tests, then I do developing.
Is it possible to find a browser automation tool that works in:
Windows XP: IE6,7,8, FF3
OSX: Safari, FF3
?
I've looked into SeleniumIDE and RC, but there seems to be some IE8 problems.
I've also seen some things about Google's WebDriver, which confusingly seems to work with Selenium.
Our organziation has licenses for IBM's Rational Functional Tester, but I don' think that will work on the MAC.
The idea is to try to run tests on all the browsers our organization supports. Doable? Are my requirements unrealistic? Any recommendations as far as software to try?
Thanks!
I would recommend using Selenium but I say that as a Selenium Committer.
Selenium works on any browser that supports JavaScript since the framework has been written in JavaScript. This means if your browser on any OS supports JavaScript it will run in Selenium. That documentation it out of date, you can see that since it is talking about IE8b1 and IE9 preview is out now.
Selenium and WebDriver (which isn't a Google thing since it started at ThoughtWorks) are currently being merged as they both have their strengths and weaknesses. The current merged work will be called Selenium 2 and you can start using the alpha release now at http://code.google.com/p/selenium/. It will still work on any OS as that is still the main driving force behind the work being done.
Selenium IDE only works on Firefox because it is a Firefox add on.
I personally would avoid Rational Functional Tester because it has a lot of weaknesses that its not even worth contemplating.
If you start with Selenium there are some tutorials on my site at http://www.theautomatedtester.co.uk
Try Sahi (http://sahi.co.in/) It works across browsers and operating systems. It has a powerful recorder, and great APIs for object identification. It supports HTTPS, proxy tunneling etc. and has drivers in sahi script, java and ruby. It also has parallel playback inbuilt. It is 5 yr old mature project hosted on SourceForge, with releases almost every month.
It automatically waits for AJAX and page loads, and does not use XPaths for object identification. It also handles sites with dynamic ids.
Selenium is probably your best bet out of the tools you mentioned. What are the issues it has with IE8? You might want to check out HttpUnit to see if that meets your needs, also.
Selenium RC is a great tool if you invest the time to use it. With significant modifications to the existing library I've gotten it to fulfill all of my front end testing needs.
The confusion you are having about Webdriver is understandable. Selenium 2 is in development and will be a merge of Webdriver and Selenium. Check out: http://www.youtube.com/watch?v=RQD4EzWI4qk to get more detail.
The only browser that I have found to be unusable with Selenium is IE6. IE7 and IE8 work fine as does Firefox (which I have modified to include firebug for debugging purposes).
I'm in the same boat. It is a difficult problem to solve. Windmill and Selenium are the 2 best I've found. Though they both have issues. Selenium can only record scripts in Firefox and I haven't managed to get the proxy chaining to work as advertised. Windmill you can record in any browser and you can supposedly tweak the proxy to put extra logic in there, but the js mechanism for recording across page loads has been in my experience very brittle at least on the app I have to test.
I don't think anyone can get it quite right as long as there is more than one browser that needs to be supported.
Maybe have a look at SIKULI. It's a different paradigm but, depending on what you want to test exactly, it may do the job and will work with any browser, on any platform.
Have a look at their official blog for some examples of interactions with web applications.
So I wrote some of my more problematic tests in Selenium RC, using the Python driver. It was a better experience than writing the same tests in pure JavaScript, but I still had some of the same issues.
Testing something like an ajax autocomplete widget, meant forking some of the code depending on IE, or Firefox, and I still can't get typeKeys or a combination of type with typeKeys to work in Safari.
So, I am not sure if having cross browser clean, extensive ui tests is a bit unrealistic.
Should I try webdriver/Selenium 2? Would that make things better, or is that product not ready for prime time yet? How's the Python binding for that? I don't know Java, but I would learn some if need be.
I'm developing large application using ExtJS framework. Because it grows too fast, I realized that this might be the time to start doing tests.
I want to go for BDD technique, I found several BDD frameworks for JavaScript around (Screw.Unit, JSpec, JSSpec), but I'm still not sure which one to choose. There are some articles about this topic, but I'm more interested in your own experiences/suggestions.
So my questions are:
Which one do you use and why?
Any other hints/tips are welcome.
Do you use Selenium alongside a BDD testing?
Do you use any other technique?
We use Selenium, yes.
I wrote a Unit Test framework (well, most of one, functional, not entirely pluggable) that I have used a few times. These sorts of topics keep coming up so maybe I should finish it... the url is http://code.google.com/p/jasproject/
I use buster.js because I develop in node.js as well as client-side javascript. It copes with both scenarios using a single API. The documentation is still lacking, but I really like the support for asynchronous testing.
Buster can also be used in a similar way to Selenium's WebDriver - you can slave various browsers to a buster "server" and run your tests in all those browsers at the same time.
Functional web testing has several challenges. Tests tend to be...
Slow to run (http requests are slow, dom traversal can be slow too)
Slow to write (write a test, start the app, run the test, realise you made a mistake, start again)
Hard to read (xpaths, css selectors etc)
Brittle (when tightly coupled to your HTML)
Expensive to maintain (if you don't use an abstraction layer such as the page object pattern)
Unrealistic (when run in a fake browser)
For these reason my preferred stack is
JavaScript - development time is fast since there's no compilation time
CasperJS or Zombie JS - very fast, webkit based (Safari & Chrome [for now])
Yadda for true BDD - Makes the test easy to read and provides an abstraction layer, mitigating the brittleness and maintenance
Downsides of this stack is that you're only testing in webkit, not firefox, IE or Chrome (when Google move to Blink)