FWN/Beats/QualityAssurance

QualityAssurance

In this section, we cover the activities of the QA team.

Contributing Writer: Adam Williamson

Test Days

There was no Test Day last week, as we finally released Fedora 11.

Currently, no Test Day is scheduled for next week - it is still very early in the Fedora 12 cycle. If you would like to propose a test day which could result in changes for post-release updates for Fedora 11, or an early test day for Fedora 12, please contact the QA team via email or IRC, or file a ticket in QA Trac.

Weekly meetings

The QA group weekly meeting was held on 2009-06-17. The full log is available. Adam Williamson reported that he had added a suggested draft for the previously agreed change to the minimum hardware requirements to an existing bug report.

James Laska reported on the Fedora 11 retrospective meeting which had taken place the previous day, and which he, Adam Williamson, Jesse Keating and Edward Kirk had attended to represent the QA and BugZappers groups. All agreed that the meeting had been well-run and productive. Jesse pointed out that the real test of its success would be if any of the items discussed had led to actual changes within the next month or two. James promised to update the QA team's Goals page to incorporate the lessons learned from the Fedora 11 release cycle.

The group discussed whether some kind of voice format for the meeting would be better than IRC, but in the end there was wide agreement that it would not be.

James Laska mentioned that the period for open feedback on proposals stemming from the earlier Fedora Development Cycle Activity Day was nearing an end: feedback on these proposals will be accepted up to June 30th. The proposals can be found in an email from Jesse Keating. Anyone interested is encouraged to read the proposals and provide feedback.

Will Woods reported on progress of the AutoQA project. He noted that one of the FAD proposals, israwhidebroken.com, is essentially an AutoQA project, and so he has established it as the first AutoQA milestone, with a set of tickets. He noted that Jesse Keating is working on packaging autotest, which will be the harness used to create the automated tests for this project. Jesse pointed out that autotest required Google Web Toolkit, which is not yet packaged either, so packaging autotest is a big project, but he was confident that he will be successful.

The Bugzappers group weekly meeting was held on 2009-06-16. The full log is available. Adam Williamson reported that Brennan Ashton is working on having a components page as part of the triage metrics system.

Edward Kirk reported on his attempts to find out how the critical component list was generated so it can be accurately updated. He later spoke with Jon Stanley and Jesse Keating and was advised to use the critical path packages proposal to help re-generate the list. He will report further next week.

Adam Williamson reported on the progress of the kernel triage project. He had sent an email to all interested parties, asking the kernel team to provide information on the current workflow used for kernel bugs, but had not yet received a reply, so this project is currently waiting on that important information from the kernel team.

Adam Williamson also reported on a request received from the EPEL team for some help with a Bug Day they have planned for July 11th. Kevin Fenzi, who is part of the EPEL project, provided some explanations: a copy of RHEL is not required to help, Bugzappers could help with only a CentOS box, or even just Fedora in some cases. Help asked of the Bugzappers team is mostly in typical Bugzappers tasks of triaging and pinging dormant bugs for further information. The EPEL project follows the Fedora bug workflow. The group agreed that they would be happy to help out with the Bug Day, and asked the EPEL project to provide more information closer to the date.

The next QA weekly meeting will be held on 2009-06-24 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-23 at 1500 UTC in #fedora-meeting.

Improved and more detailed QA / Release Engineering scheduling for Fedora 12

John Poelstra announced that he had worked on a draft for an improved and more detailed schedule for release engineering tasks for Fedora 12, which also affects QA. He suggested that "we should move to more of a standard software model (just as we did with the naming of the schedule, etc.) where there is more separation between Releng and QA. IOW, Releng provides the service of packing the bits and composing an installable distro and QA provides the service of testing them and giving a thumbs up/down on them", and asked "What tweaks should I make to better reflect QA's needs?" The proposed schedule has blocker bug reviews happening the Friday before key freezes, exact dates for release engineering composes, and exact dates for compose testing. James Laska replied with broad support for each of the proposals.

Installer test plan

Liam Li announced a test plan for installation for Fedora 12, and asked for feedback. James Laska replied, noting that he had made some minor changes to the Wiki page, and providing some comments on the plan. He pointed out that it may be a good idea to consider the yum install cleanup feature in the plan, and suggested only listing the test priority order once.

Rawhide acceptance test plan

Will Woods announced the creation of a Rawhide acceptance test plan, which is part of the israwhidebroken.com proposal discussed during the weekly meeting (see above). This outlines the overall set of features which should be tested to determine if Rawhide is currently in a usable state or not. Jóhann Guðmundsson suggested a test for whether a basic wired network connection could be established. Adam Williamson suggested a test for whether basic input (keyboard and mouse) are working. Tom London suggested a test for encrypted root filesystems, but Will explained that this was beyond the scope of basic functionality testing.