Thursday, October 1, 2015

Introducing Pilot

It's finally here! We've been working on a way to allow those who have a ubuntu phone to participate more directly in testing the software that runs on their device. This includes things like helping test OTA updates before they are shipped and to verify and look for bugs in applications like the core apps and system services.

Introducing Pilot, a new application you can find today in the ubuntu store. The application utilizes checkbox as a way of distributing tests to you on the phone. This first round of testing includes tests from 4 of your favorite core applications including dekko, clock, music, and weather.

To help test, search for Pilot in the store and install it.

Start the app, and click the Start Testing button once it's loaded.

Select a test plan to run. Right now you can choose to test specific features of the different core apps.

Select the tests to run. You can choose to run all of tests for that feature, or just one if you wish.

Run through the test, following each step. If everything works as listed in the test, press the Pass button. Otherwise press Fail.

You can also add comments about the test or skip the test using the buttons at the top of this page.

Finally, submit your results back to the QA team by pressing the Submit Results to Community Practitest button. You'll need to supply your ubuntu SSO information to do so. You may also view your submitted results on this screen by pressing the corresponding button.

It's that easy. Over time, we'll push new tests via application updates, so you can help test new things as they are developed. As the number of devices grows, we want to ensure every device has the same level of quality. With your help, we can make sure ubuntu gets better with each update. Thanks for your help!


  1. This is great! Will non-Canonical app developers be able to get feedback from our users on our apps? Thanks.

    1. Steve, that's an interesting idea. At the moment it's not really setup to handle something like that, but I'm thinking it would be possible to do so. The app would need properly formatted testcases, and we'd have to expand on reporting. If you are curious, file a bug asking for the feature here:

      We can run through ideas and get a clear idea of what it might take to implement.

  2. I found a bug in the Pilot app. When i had chosen the tests for the music player app, there were urls to screenshots in the test description, but not the images shown.

    1. David, good catch. Thanks for pointing it out. There's a bug speaking about this here: We'll try and figure out a solution for the next round.