Thursday, October 1, 2015

Introducing Pilot

It's finally here! We've been working on a way to allow those who have a ubuntu phone to participate more directly in testing the software that runs on their device. This includes things like helping test OTA updates before they are shipped and to verify and look for bugs in applications like the core apps and system services.

Introducing Pilot, a new application you can find today in the ubuntu store. The application utilizes checkbox as a way of distributing tests to you on the phone. This first round of testing includes tests from 4 of your favorite core applications including dekko, clock, music, and weather.

To help test, search for Pilot in the store and install it.

Start the app, and click the Start Testing button once it's loaded.

Select a test plan to run. Right now you can choose to test specific features of the different core apps.

Select the tests to run. You can choose to run all of tests for that feature, or just one if you wish.

Run through the test, following each step. If everything works as listed in the test, press the Pass button. Otherwise press Fail.

You can also add comments about the test or skip the test using the buttons at the top of this page.

Finally, submit your results back to the QA team by pressing the Submit Results to Community Practitest button. You'll need to supply your ubuntu SSO information to do so. You may also view your submitted results on this screen by pressing the corresponding button.

It's that easy. Over time, we'll push new tests via application updates, so you can help test new things as they are developed. As the number of devices grows, we want to ensure every device has the same level of quality. With your help, we can make sure ubuntu gets better with each update. Thanks for your help!

Monday, September 28, 2015

Ubuntu Font Testing: Arabic

Some of you may remember the birth of the ubuntu font family during the 10.10 cycle. The time has come to finish that work as well as fix a few issues with the current font set. To start with, the design team has been working on Arabic, and is ready for some feedback on how the font looks and interacts.

To help gather your feedback, we've made a simple survey. It contains the information you need to get the font, as well as the opportunity to leave feedback.

We would love to hear from you! If you encounter any issues trying to test or use the survey, feel free to get in touch, but otherwise leave your feedback on the font in the survey. Thanks again for your help!

For those of you who don't happen to speak Arabic or a related language, an opportunity to test the full ubuntu font family is coming up soon. Get ready!

Wednesday, September 23, 2015

Final Beta Testing for Wily

Another cycle draws to a close, and it's time to test our images and make sure Wily is in good shape. We're entering crunch time.

How can I help? 
To help test, visit the iso tracker milestone page for final beta.  The goal is to verify the images in preparation for the release. Find those bugs! The information at the top of the page will help you if you need help reporting a bug or understanding how to test. 

There's a first time for everything! Check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help.

How long is this going on?
The testing runs through tomorrow, Thursday September 24th, when the the images for final beta will be released. If you miss the deadline we still love getting result Test against the daily image milestone instead.

Thanks and happy testing everyone!

Wednesday, July 29, 2015

Opportunity: Automated Installer Testing

I wanted to share a unique opportunity to get invovled with ubuntu and testing. Last cycle, as part of a datacenter shuffle, the automated installer testing that was occurring for ubuntu flavors stopped running. The images were being test automatically via a series of autopilot tests, written originally by the community (Thanks Dan et la!). These tests are vital in helping reduce the burden of manual testing required for images by running through the base manual test cases for each image automatically each day.

When it was noticed the tests didn't run this cycle, wxl from Lubuntu accordingly filed an RT to discover what happened. Unfortunately, it seems the CI team within Canonical can no longer run these tests. The good news however is that we as a community can run them ourselves instead.

To start exploring the idea of self-hosting and running the tests, I initially asked Daniel Chapman to take a look. Given the impending landing of dekko in the default ubuntu image, Daniel certainly has his hands full. As such Daniel Kessel has offered to help out and begun some initial investigations into the tests and server needs. A big thanks to Daniel and Daniel!

But they need your help! The autopilot tests for ubiquity have a few bugs that need solving. And a server and jenkins need to be setup, installed, and maintained. Finally, we need to think about reporting these results to places like the isotracker. For more information, you can read more about how to run the tests locally to give you a better idea of how they work.

The needed skillsets are diverse. Are you interested in helping make flavors better? Do you have some technical skills in writing tests, the web, python, or running a jenkins server? Or perhaps you are willing to learn? If so, please get in touch!

Friday, July 10, 2015

Ubuntu SDK Autopilot Plugin

Those of you who have developed an application using the Ubuntu SDK understand how nice it is to have a tool to support your workflow for writing an application. You can code, build, run and iterate on your code easily right from inside the SDK. However, to test your application, it was necessary to open a terminal and execute some commands. Leaving the Ubuntu SDK is an interruption to your workflow! It's even enough to throw you off your coding zen! It certainly may have dissuaded you from running tests. Seeing as testing should be a positive experience, this certainly won't do!

Thankfully Akiva thought the same thing. Thus he created a new plugin for the SDK. I'd like to celebrate and thank him for making all of our lives easier. Thanks Akiva! A big thank you to Benjamin from the SDK team as well for reviewing and helping get the plugin in shape.

The plugin scans your project for autopilot tests, and then creates a run configuration for them. From there, it's as easy as hitting the run button to run the application. See for yourself!

To learn more about how to install the plugin, or how it works, checkout the documentation on running autopilot tests found on 

Go forth and test all the things! Try out using the plugin in your existing workflow. I'd love to hear feedback. If you are interested in making the plugin better, or expanding it to include other things, get in touch. As always, code is welcome!

Wednesday, July 1, 2015

Have you tried writing a testsuite?

It's never been easier to write tests for your application! I wanted to share some details on the new documentation and other tidbits that will help you ensure your application has a nice testsuite. If you've used the SDK in the past, you understand how nice it can make your development workflow. Writing code and running it on your desktop, device, or emulator is a snap.

Fortunately, having a nice testsuite for your application can also be just as easy. First, you will notice that now all of the wizards inside the SDK now come with nice testsuites already in place. They are ready for you to simply add-on more tests. The setup and heavy lifting is done. See for yourself!

Secondly, has a great section on every level of testing; no matter which language you use with the SDK. You'll find API references for the tools and technology used, along with helpful guides to get you in the proper mindset.

For autopilot itself, there's also API documentation for the various 'helpers' that will make writing tests much easier for you. In addition, there's a guide to running autopilot tests. This has been made even easier by the addition of Akiva's Autopilot plugin inside the SDK. I'll be sharing details on this as soon as it's packaged, but you can see a sneak peek in this video.

Finally, you will find a guide on how to structure your functional tests. These are the most demanding to write, and it's important to ensure you write your tests in a maintainable way. Don't forget about the guide on writing good functional tests either.

No matter what language or level you write tests for, the guides are there to help you. Why not trying adding a test or two to your project? If you are new, check out one of the wizards and try adding a simple testcase. Then apply the same knowledge (and templated code!) to your own project. Happy test writing!

Snappy Open House!

Introducing Snappy Open Houses! Snappy represents some new and exciting possibilities for ubuntu. A snappy open house is your chance to get familiar with the technology while helping test and break things ! We plan to do an open house before each release as a chance for everyone to interact and provide feedback and help with testing. As such, this is a great way to get started in the snappy world!

So what exactly do I mean by an open house? We want to encourage the community to test with us, explore new features, check for possible regressions and exercise the documentation.  An open house is a chance to come and meet the snappy team developers and help QA test the new image.

During the open house, we'll host a live broadcast on As part of the broadcast, we'll speak with some of the developers behind snappy and show off new features in the upcoming release. We'll also demonstrate how to flash and test the new release so you can follow along and help test. Finally we'll answer any questions you have and stick around on IRC for a bit to discuss any issues found during testing.

In other words, it's time intended for you to come and try out snappy! I know what you are thinking, "I don't have a cool IoT device to run snappy with". You are in luck! You can run snappy on your desktop and laptop. You don't need a device as you can install snappy on your local machine via kvm. If you do have a device, bring it and prepare to have some fun!

The first of these snappy open houses will be July 7th at 1400 UTC. Please stop by and help test with us, try out snappy, and meet the snappy team!

You can find out more information on the wiki. Mark your calendars and see you next Tuesday!