FWN/Beats/QualityAssurance

QualityAssurance

In this section, we cover the activities of the QA team.

Contributing Writer: Adam Williamson

Test Days

Last week's main track Test Day was on NetworkManager. There was a solid turnout of testers and developers, and several bugs were filed and fixed. A report on this Test Day is available.

Last weeks' Fit and Finish Test Day was on peripherals. Several people turned out to help test, and a variety of different bugs with different types of peripheral were reported.

Next week's main track Test Day will be on ABRT changes for Fedora 12. ABRT is the Automatic Bug Reporting Tool which helps users file bug reports automatically when applications fail, and it has been extensively improved for Fedora 12. It's an easy component to test and it will help improve the quality of future Fedora releases, so please come along and help out! The Test Day will be held on Thursday 2009-08-20 in IRC #fedora-qa.

The Fit and Finish Test Day track will be holding its own Test Day, on printing. This is a vital area for many users and has lots of potential quirks with different types of printer connected in different ways, so please come out and help make sure the printing user experience is as smooth as possible! Live images will be available before the Test Day. The Test Day will be held on Tuesday 2009-08-18 in IRC #fedora-fit-and-finish (note this is not the same channel where main track Test Days take place).

If you would like to propose a main track Test Day for the Fedora 12 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac.

Weekly meetings

The QA group weekly meeting was held on 2009-08-10. The full log is available. James Laska asked for feedback on the quality of downloads of the Alpha test compose from the alt.fedoraproject.org server. Adam Williamson reported that his download had been fast and trouble-free. Kamil Paral's had been slower, but that was tracked down to bandwidth limitations on his end.

James Laska asked why Rawhide still contained anaconda 12.7, when later versions had been released and built. Jesse Keating stated that later versions of anaconda had been entirely broken and thus had not passed his critical path package checks. Adam Williamson asked why major regressions in anaconda seemed to be being introduced during an Alpha freeze. Jesse Keating explained that anaconda development was treated as an independent 'upstream project', like rpm, and so did not respect Fedora freezes. Adam suggested that, in that case, Fedora packaging of anaconda should not accept new upstream versions as a matter of course, especially during freezes, but cherry-pick appropriate fixes, due to the sensitivity of anaconda to changes and its position of fundamental importance in any Fedora release.

James Laska called for those who had filed or were monitoring critical bugs for the Alpha release to continue to work on verifying fixes for them and closing them where appropriate.

James Laska asked for a general overview of Rawhide's readiness for the Alpha release. The consensus was that anaconda was still not yet ready, but most other components were in decent shape. Adam Williamson noted that packages fixing the known major breakage in xorg-x11-server-1.6.99-25 had been tagged into Rawhide over the weekend. James also worried that many features on the Fedora 12 feature list did not seem to be complete in terms of development or have complete test plans yet, but no action was thought to be possible on this.

Will Woods reported on the progress of the AutoQA project. He had completed the automated installation tests, and refactored the pre-existing autoqa tests into the new autotest system. He also had some tests starting to send their results to a mailing list, and hoped to have this process available to the public soon.

The Bugzappers group weekly meeting was held on 2009-08-11. The full log is available. Adam Williamson admitted he had not found time to ask the desktop development team for their position on the new triage process, or check which Bugzilla changes generate an email by default.

The group wanted to take a final decision on the question of changing the process by which bugs are marked as triaged. After a long discussion, it was agreed to go ahead with a plan to switch to using the Triaged keyword rather than the ASSIGNED state, starting with bugs for Fedora 13. Adam Williamson agreed to send a wrap-up email to the mailing list.

Edward Kirk brought up the recent fedora-devel-list mail which had mentioned the need for triaging of XMMS bugs. However, several group members had looked over the list of bugs on XMMS that were still open or had been closed due to age, and found nothing that could be pursued.

Edward Kirk also worried that meetings were being planned only by himself and Richard June and were not being planned according to a defined policy. He intended to write a SOP for planning meetings, and encourage the use of the agenda item submission process to make sure no important issues were not making it to the meeting agenda.

The next QA weekly meeting will be held on 2009-08-17 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-08-18 at 1500 UTC in #fedora-meeting.

Fedora 12 Alpha delay

At the final go/no-go meeting on Monday 2009-08-10, it was decided with the unanimous agreement of release engineering and QA groups to slip Fedora 12 Alpha's release by one week due to several blocker bugs still outstanding, including several bugs which could cause installations to fail completely in very common circumstances. Jesse Keating announced the slip to the development mailing list.

Fedora 12 Alpha blocker bug reviews

James Laska reminded the group that several bugs blocking the Alpha release (as of Sunday 2009-08-09) were in MODIFIED state and required further testing. Later, he sent a follow-up with updated status on several of the bugs listed.