Tuesday, November 20, 2012

A glance at Autopilot

So, as has been already mentioned, automated testing is going to come into focus this cycle. To that end, I'd like to talk about some of the tools and methods for automated testing that exist and are being utilized inside ubuntu.

I'm sure everyone has used unity at some point, and you will be happy to know that there is an automated testsuite for unity. Perhaps you've even heard the name autopilot. The unity team has built autopilot as a testing tool for unity. However, autopilot has broader applications beyond unity to help us do automated testing on a grander scale. So, to introduce you to the tool, let's check out a quick demo of autopilot in action shall we? Run the following command to install the packages needed (you'll need quantal or raring in order for this to work):

sudo apt-get install python-autopilot unity-autopilot

Excellent, let's check this out. A word of caution here, running autopilot tests on your default desktop will cause your computer to send mouse and keyboard commands all by itself ;-) So, before we go any further, let's hop over into a 'Guest Session'. You should be able to use the system indicator in the top right to select 'Guest Session'. Once you are there, you'll be in a new desktop session, so head back over to this page. Without further ado, open a terminal and type:

autopilot run unity.tests.test_showdesktop.ShowDesktopTests.test_showdesktop_hides_apps

This is a simple test to check and see if the "Show Desktop" button works. The test will spawn a couple of applications, click the show desktop button and verify clicking on it will hide your applications. It'll clean up after itself as well, so no worries. Neat eh?

You'll notice there's quite a few unity testcases, and you've installed them all on your machine now.

autopilot list unity

As of this writing, I get 461 tests returned. Feel free to try and run them. Pick one from the list and see what happens. For example,

autopilot run unity.tests.test_dash.DashRevealTests.test_alt_f4_close_dash

Just make sure you run them in a guest session -- I don't want anyone's default desktop to get hammered by the tests!

If you are feeling adventurous, you can actually run all the unity testcases like this (this will take a LONG TIME!).

autopilot run unity

As a sidenote, you are likely to find some of the testcases fail on your machine. The testsuite is run constantly by the unity developers, and the live results of commit by commit success or failure is actually available on jenkins. Check it out.

So in closing, this cycle we as a community have some goals surrounding easing the burden for ourselves in testing, freeing our resources and minds towards the deeper and more thorough testing that automation cannot handle. To help encourage this move of our basic testcases towards automation, the next series of blog posts will be a walkthrough on how to write Autopilot testcases. I hope to learn, explore and discover along with all of you. Autopilot tests themselves are written in python, but don't let that scare you off! If you are able to understand how to test, writing a testcase that autopilot can run is simply a matter of learning syntax -- non-programmers are welcome here!


  1. So I ran a few tests and now I want to report them (some failed). How do I send my test reports and to who? Thx.

    1. Sid, thanks for wanting to report your results! The qatracker is the place where we record results as a quality team. However, these testcases were intended as a demonstration of autopilot, and as such, the testsuite is not yet finished. Therefore no reporting of the results was sought (yet!). If your interested in running and reporting autopilot tests that are ready, join the quality team https://wiki.ubuntu.com/QATeam/. We are currently working on converting some of our manual tests to autopilot tests and we will run and report results for them as part of our cadence testing this cycle.

  2. Hey Nicholas, I have a small grammatical suggestion. Can you change the phrase "Without your control" to "out of control" in the third paragraph?

  3. I am not able to install autopilot. It is giving following error. Please let me know how to fix it.

    sudo apt-get install python-autopilot unity-autopilot
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    E: Unable to locate package python-autopilot
    E: Unable to locate package unity-autopilot

    1. Dipan, can you tell me what version of ubuntu you are running? You need quantal or newer in order for those commands to work. Precise doesn't have those packages in it's default repository.