I am working on a tiny webapp project using ruby/sinatra. It is displaying a dashboard which is loading its data periodically using AJAX call to the REST API part of the app. (Basically the json results are transformed to some HTML on the page.)
I was driving the development of the REST API and the HTML using RSPec tests, but when I started with AJAX calls I couldn't use TDD.
So, how can I test the result of my javascript/AJAX calls? I would like to examine the resulting HTML.
I would like to avoid to use Selenium with browser to keep the project as lightweight as possible.
Cheers
Alex
The MockJax library (for example) will allow you to use TDD for your javascript and jQuery code. It will enable you to mock your AJAX calls, hence allowing you to test your client-side components in isolation.
As for testing the result of the AJAX calls, you could use the Capybara test framework; as well as supporting Selenium (which you want to avoid), it also supports a headless webdriver Capybara-webkit which should be "lighter" than Selenium as it does not load the entire browser.
Related
I'm currently writing a library in typescript that retrieves videos from certain pages. I'm using jest to write the test but I'm facing the problem of testing the scraper since some content is dynamic and the library makes some post requests to store some cookies mocks are not feasible, what would be the correct way to test such a library?
I need to crawl a website, however, its content is dynamic. Are there any packages in Python that could call js functions? For example, suppose I have a link and JS functions 1, 2, and 3 in JS that I should call on that webpage, and I need the final webpage after all JS function calls.
Executing client-side javascript can get very complicated, so the most reliable way to run all javascript on a page just like a user would would be to use a real browser in headless mode.
Specifically for Python, there is a Python+Selenium combo for working with headless Chrome.
If you are willing to trade Python for Nodejs, a more powerful toolset is Puppeteer+headless Chrome (it lets you do a lot more than Selenium). There is also an early unofficial port of Puppeteer to Python but I haven't tried it and can't comment on how stable it is https://pypi.org/project/pyppeteer/
I want to convert some web pages with javascript to plain html, and I found there several ways(pls tell me if I'm wrong):
Use Jython, an example: http://blog.databigbang.com/web-scraping-ajax-and-javascript-sites/
Use Java together with htmlunit
Use a proxy, an example: http://grep.codeconsult.ch/2007/02/24/crowbar-scrape-javascript-generated-pages-via-gecko-and-rest/
Use python together with qt or PyV8
Because I want to make a tiny tool to meet my request, and I thought it somewhat complicated to install V8 and qt, although python is my first choice.
So I tried to make a proxy with gecko, but it seems need a DISPLAY which I can not afford in a remote Linux server.
Now I am trying to use jython, but it seems there is no simple way to just convert a whole page to plain html.
Actually, I want to ask is there a way to convert a web page contains javascript to plain html, just like the brower does. Can node.js do this job?
I've recently built a server on top of PhantomJS that does this. I highly recommend this route.
http://phantomjs.org/
Basically, you write a quick script that has PhantomJS run the page, and configure a trigger method that lets you know the page is finished and sends the data off. My version used the built-in HTTP server, so PhantomJS easily served up the results on its own. This takes about 15 lines of code to do. (Sorry, can't paste it here... wrote it on work time. But, check out the example on their home page. It's almost complete!)
I am building a web app using BackboneJS and RequireJS and need to implement some form of unit testing for UI interaction and data retrieval via AJAX. I have come across QUnit and Jasmine but don't really know how I can integrate this into my app.
If I am testing things such as:
Is the user logged in alright?
Has the data been received from the server ok?
Does clicking a button trigger the expected response?
Do click events work on dynamically loaded html content?
Does the app respond correctly to changes in hash/push-state urls?
I would imagine the testing has to be directly integrated into my app so as to have access to specific JS objects, work with session specific data and respond to changes in push state URLs.
How can I integrate QUnit or Jasmine (or other suggestions) into my modular app to unit test such features?
Unit testing is really simple.
You make a test HTML page. You include QUnit/NodeUnit/Jasmine/TestLibraryOfChoice
You then use requireJS and load one of your javascript modules,
and you simply test the exported object or function. That means testing the valid inputs of your module and asserting the outputs are correct.
You may have to mock out ajax and write HTML mocks
Dojo Objective Harness (DOH) is a very good unit test framework, which is browser agnostic and supports testing asynchronous functions, see here for a walkthrough guide.
However, from your test cases it looks like you want something more like an integration test?
If so Selenium is a good browser automation tool.
Crucially, neither of these tools will require you to modify your code (unless you find bugs :))
If you want to see an example where requireJS based modules are unit tested with QUnit, download the javascript reference architecture at http://boilerplatejs.org.
Disclaimer: I'm the main author of it.
This is kind of tricky. There is this webpage which, I am guessing, uses some kind of AJAX to pull out content based on the search query. When I fetch the page using get in Perl, it fetches the script code behind the php/html, but not the results which are displayed when the query is searched manually. I need to be able to fetch the content of the results page. Is there anyway to do this in Perl?
Take a look at Selenium RC and the WWW::Selenium module in Perl. With them you can control a real web browser.
Another option is WWW::HtmlUnit which uses the HtmlUnit Java library to execute the JavaScript without a web browser. WWW::HtmlUnit uses Inline::Java to give Perl access to the library. I have found that when installing, it is best to say No to the question "Do you wish to build the JNI extension?".
If you are writing tests that need to check the rendered page, you can have a look at Schwern's javascript-tap-harness, which works with Selenium and handles all the scaffolding.
I also found Using WWW::Selenium To Test Or Automate An Ajax Website pretty useful.