FWN/Beats/QualityAssurance

QualityAssurance

In this section, we cover the activities of the QA team.

Contributing Writer: Adam Williamson

Test Days

There have been no Test Days for the last two weeks, due to the pressures of the Fedora 12 release.

No Test Day is currently planned for this week. If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac.

Weekly meetings

As the QA beat was unfortunately not present for Fedora Weekly News #201, we will cover two weeks' worth of events below.

QA group weekly meetings were held on 2009-11-02 and 2009-11-09. The full logs are available, .

During the meeting of 2009-11-02, Adam Williamson reported that Milos Jakubicek had not yet followed up on his idea regarding an event to work on FTBFS problems. Jóhann Guðmundsson was also not present to report on his work on creating a test case for keyboard layout issues. Will Woods was making good progress on the action items for AutoQA from the previous meeting.

The group reviewed the status of the Fedora 12 code base with regards to the then-impeding release candidate phase. It was generally agreed that the status was promising and it should be possible to make the release candidate phase on time, based on a review of the blocker bug list.

Will Woods and Kamil Paral reported on the progress of the AutoQA project. Will had been working on revising the AutoQA code to provide a Python library interface. He had been moving all shared or potentially shareable code from all current AutoQA tests into the library. He hoped to have it merged into the master branch by the end of the week. He also noted that a newer version of autotest was currently being packaged and implemented into the AutoQA system, which may cause strange results if any bugs emerged.

Adam Williamson reviewed upcoming events. The release candidate date was Wednesday 2009-11-04 and the go/no-go date Monday 2009-11-09. Jesse Keating clarified that for an RC build to be done, the blocker bug list must be clear, but new issues that emerged during RC compose and testing could be resolved up until the date of the go/no-go meeting. James Laska promised to co-ordinate with the anaconda team to ensure there were no remaining blocker issues in installation.

The meeting of 2009-11-09 was held during the final run-up to the Fedora 12 go/no-go meeting, so there was some last-minute blocker bug discussion. Jóhann Guðmundsson had not yet been able to work on creating a test case for keyboard layout issues. James Laska had followed up with the anaconda team and verified no blocker bugs remained in the installation process. Adam Williamson noted that one of the anaconda bugs that definitely wasn't left had been fixed the previous day.

Will Woods and Kamil Paral reported on the progress of the AutoQA project. Kamil had added a check to rpmguard for the case where an old version of rpmdiff is installed. Will had the new python library ready but wanted more testing before merging it into master. The new Koji watcher (for running AutoQA tests on new builds as they hit Koji) was now functional.

James Laska pointed out that Matthias Clasen had asked the group to test Fedora 12 0-day updates by enabling the Fedora 12 updates and possibly updates-testing repositories and updating their systems. James thought it would be a good idea to create a test case for testing the update repositories for a release before they were generally enabled.

A Bugzappers group weekly meeting was held on 2009-11-03. No meeting was held on 2009-11-10. The full log is available. Richard June reported that he was continuing to work on kernel triage. He had not been in touch with Jeff Hann regarding his volunteering to help out yet, but would attempt to co-ordinate via the mailing list.

Adam Williamson asked if anyone had concerns about unaddressed issues for the Fedora 12 release, and no-one did. Adam asked Matej Cepl how he was coping with X.org triage while François Cami was mostly unable to help, and he said it was difficult to stay on top of the large number of bugs. Adam promised to continue to try and manage the nouveau driver bugs, and Thomas Janssen volunteered to help out with others. Matej said he would work with Thomas to bring him up to speed on X triaging.

Edward Kirk reported that he had worked on outstanding Fedora 10 bugs, and managed to update some and close others. He also reported that the maintainer warning email for the upcoming Fedora 12 housekeeping Bugzilla changes had been sent out. John Poelstra was ready to do the Rawhide bug rebase (moving open Rawhide bugs to Fedora 12) and Fedora 10 bug end-of-life warning operations.

The next QA weekly meeting will be held on 2009-11-16 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-11-17 at 1500 UTC in #fedora-meeting.

Fedora 12 release

Much of the group activity in the last two weeks was centred around Fedora 12 release testing and validation. James Laska posted a recap of the final blocker bug review meeting for Fedora 12. James, Adam Williamson and Will Woods worked with the release engineering team throughout the final week before release to continuously monitor blocker bug status, test fixes, and monitor for newly identified blocker bugs and regressions.

Four release candidate builds were produced by the release engineering group with the help of testing and feedback from QA. Liam Li co-ordinated the planned installation testing through the test compose and release candidate process , and maintained the test results matrix. He also sent a post mortem on the testing process. Many members of the group contributed valuable test reports to the matrix. Several group members, including Gianluca Cecchi, Gene Czarcinski and others posted test installation reports which helped identify important issues that were fixed or documented during the release process.