FWN/Beats/QualityAssurance

QualityAssurance

In this section, we cover the activities of the QA team.

Contributing Writer: Adam Williamson

Test Days

Last week's Test Day was on the the use of software RAID arrays with Anaconda. Our hardened group of install testers worked hard to test as many RAID scenarios as possible. We'd like to send a big thanks to Alexey Torkhov (greenlion) for contributing so much valuable testing.

Next week's Test Day will be on internationalization (also known as i18n) - an event which usually has a strong focus on input methods, but can also cover issues like fonts. The Test Day will run all day on Thursday 2009-10-15 in the #fedora-test-day IRC channel. Please come along and help ensure Fedora works just as well no matter what language you use!

No Fit and Finish track Test Day is planned for next week.

If you would like to propose a main track Test Day for the Fedora 12 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac.

Weekly meetings

The QA group weekly meeting was held on 2009-10-05. The full log is available. Adam Williamson asked Jesse Keating if there had been any follow-up on the development process issues discussed at the last meeting but one. Jesse explained that he had repeated to Red Hat management that it was too late to land feature changes in Fedora 12, whatever their desirability for Red Hat products.

Will Woods reported on the progress of the AutoQA project. He had implemented a UI for the israwhidebroken.com project, allowing QA group members to submit results for the manual tests which will form a part of it. Implementing a nicer AJAX UI and automating yum and network tests can be done later, so now only backlinks from israwhidebroken.com to autotest need to be implemented before it can go public. Kamil Paral mentioned that he is working on a test for comparing the old and new versions of a package each time a new build is uploaded, looking for important changes such as dependency changes, configuration file changes, and file mode changes. He hopes to integrate this test into AutoQA in future.

The group then looked at the status of Fedora 12 Beta, with regard to the go/no-go decision meeting which would come later that day. There was a broad consensus that the QA team's position would be against shipping the Beta on schedule, due to several blocker bugs still being open and apparently unfixed. The group spent some time discussing individual bugs in detail and agreeing action plans to address several of them.

James Laska summarized upcoming events, including the imminent Beta go/no-go meeting, and the RAID Test Day.

The Bugzappers group weekly meeting was held on 2009-10-06. The full log is available. The group noted that Edward Kirk was absent, so his plans to improve Triage Day events could not be discussed.

Richard June gave an update on the kernel triage project. He had found some bugs where useful triage could be conducted, and would contact John Linville to ensure the information requests he was sending out were sensible. Adam Williamson noted that the amount of time it had taken to find a few bugs where triage could be useful might suggest the kernel triage project overall was not worth pursuing, but Richard said it was still too early to tell and he would continue to investigate different types of bugs.

Edward Kirk also provided a draft of the meeting organization SOP that he had begun work on. The group felt it was a good start. Edward wondered where the page should end up, and Adam Williamson suggested it should be a page of its own, linked from the main Meetings page, as other SOP pages are. Edward promised to work on finalizing the page for the next meeting. Jeff Hann volunteered to help Richard with the project, and they agreed to co-ordinate after the meeting.

Adam Williamson started a discussion on Jóhann Guðmundsson's proposal to standardize the Wiki pages containing instructions on debugging problems in various Fedora components. The group generally agreed that it was a good idea, and Jóhann agreed to work on a proposed standard template for these pages, to be discussed and refined on the mailing list.

The next QA weekly meeting will be held on 2009-10-12 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-10-13 at 1500 UTC in #fedora-meeting.

Wiki debugging page standardization

Following discussion in the meeting, Jóhann Guðmundsson submitted his proposal for standardizing Wiki pages which explain how to debug and file proper bug reports on Fedora components. The proposal was enthusiastically received, with additional suggestions coming from Giovanni Tirloni, Christopher Beland, Adam Williamson and others. A template page is now in place and being refined, and some debugging pages have already been revised to use the template. The group will convert other pages one by one.

Fedora 12 Beta release process

Adam Williamson represented the QA group at the 2009-10-05 release engineering meeting. During the discussion of whether or not to release the Fedora 12 Beta according to the original schedule, Adam presented the QA group's position that the Beta release, and all subsequent events, should be delayed by a week due to the fact that all release blocker bugs had not been resolved. Other groups present agreed, and the meeting resulted in an agreement to delay the release due to this wide consensus.

As a result, the group continued to work on aiding the process of resolving release blocker bugs during the week, and a final blocker bug review meeting was held on Friday 2009-10-09. James Laska provided a recap of this meeting.