QualityAssurance
In this section, we cover the activities of the QA team[1].
Contributing Writer: Adam Williamson
Test Days
There was no main track Test Day last week. The Fit and Finish project's Test Day track continued with its second Test Day, on power management and suspend/resume[1]. The event was a success, with several testers turning out, many bugs filed, and some fixed during the day or soon afterwards, especially relating to laptops with multiple batteries.
No Test Day is scheduled for next week. If you would like to propose a main track Test Day for the Fedora 12 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[2].
Weekly meetings
The QA group weekly meeting[1] was held on 2009-07-22. The full log is available[2]. James Laska reported that he had published a blog post asking people to help with the process of writing Debugging pages[3]. Adam Williamson mentioned that he had looked into creating some of the desired pages, but did not know what kind of information was actually required for any of the components concerned. Jesse Keating suggested doing an informal interview-style session with maintainers to discover what information is needed, and then having QA take responsibility for turning that information into a finished Wiki page.
James Laska had created a meeting time matrix[4] for the purpose of re-scheduling the QA meeting to make it possible for as many group members as possible to attend. The group agreed that the new meeting day and time should be Mondays at 16:00 UTC, moved from Wednesdays at 16:00 UTC.
James Laska noted that a Fedora 12 Alpha blocker bug review meeting was scheduled for Friday 2009-07-24. It was agreed that Adam Williamson would send out an announcement of the meeting, and James would send out a recap after it had finished. Jesse Keating mentioned it would be good to do some Rawhide install testing prior to the meeting, but a combination of two significant bugs was preventing almost any Rawhide install from working.
James Laska explained that a test compose for Fedora 12 Alpha was scheduled for 2009-07-29, and Liam Li had made an announcement requesting help on install testing[5]. Jesse Keating pointed out that it would not be easy for the general public to take part, as the test compose would not be generally distributed. This led to another long discussion about the practicality of distributing time-critical test composes to the public. No definite conclusion was reached, but a tentative agreement was made to look into a system which would allow access to such composes to members of the QA group in FAS.
Jóhann Guðmundsson noted that there were some problems with Dracut, the nash/mkinitrd replacement being introduced as a feature in Fedora 12. It has no implementation plan by which the progress of the feature can be externally measured, and no detailed contingency plan beyond 'revert to mkinitrd'. Jóhann agreed to contact the feature mantainer, Harald Hoyer, to help develop a full test plan and contingency plan.
Will Woods reported on the progress of the AutoQA project. He has now automated the first four test cases in the Rawhide Acceptance Test Plan[6], and is now working on automating the installation tests. He noted that separate i386, x86-64 and PowerPC test hosts would be necessary for some tests, and that PPC might be difficult in the absence of the Fedora standard libvirt virtualization framework on that platform. Jesse Keating worried that the installation tests may be adding too much complexity to the system, and asked how much faster the process would be if only repository level tests were considered. Adam Williamson pointed out that the full set of repository level tests were the ones that had already been automated. Will promised that they would be updated to send the results somewhere publicly accessible soon.
Sebastian Dziallas brought up the topic of a Test Day for the Sugar on a Stick project[7] - essentially for the integration of Sugar with a stock Fedora distribution. It was agreed that the SoaS project would host the Test Day themselves using the SOP created for this purpose[8]. A tentative date of 2009-09-03 was agreed for the test day.
The Bugzappers group weekly meeting[9] was held on 2009-07-21. The full log is available[10]. No-one had heard from Brennan Ashton regarding the status of the triage metrics project. Adam Williamson agreed to contact him by email to find out the current status, and ask if he would be interested in having a co-maintainer on the project, in the interest of smoother development.
The group discussed the current draft of the critical path-based triage component list[11]. There was a general feeling that the list was very long and might contain components that, practically speaking, would not benefit hugely from triage. It also seemed to contain at least some binary (rather than source) package names, while Bugzilla is based on source package names. Niels Haase and Matej Cepl volunteered to adjust the list to use source package names, and break it up into groups for ease of digestion, for further review at next week's meeting.
Adam Williamson gave an update on the status of the kernel bug triage project. He admitted it had not progressed very far as he had been focussing on anaconda triage. He outlined a plan under which a volunteer would, as a test, triage bugs on one particular component of the kernel, to see if the process could be made to work. Edward Kirk thought the proposal a sound one, and Adam agreed to try and put in into practice in the next week.
Finally, the group discussed the 'Bugzilla Semantics' proposal Adam Williamson had made to the mailing list, involving various ways in which the triage process could be tweaked and the use of the NEW and ASSIGNED states changed. Initially discussion was in favour of retaining the status quo, but Jesse Keating and Josh Boyer made it clear that the development groups they were involved in used ASSIGNED in a different way to its use by the Bugzappers group, and they would prefer if Bugzappers marked bugs as having been triaged in some other way, so their groups could take advantage of the triage process. It became clear that there would be both benefits and costs involved in changing the triage process. Adam Williamson agreed to send a follow-up email to the mailing list to summarize the current state of the debate, and to see if a consensus could be found on a future path.
The next QA weekly meeting will be held on 2009-07-27 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-07-28 at 1500 UTC in #fedora-meeting.
- ↑ http://fedoraproject.org/wiki/QA/Meetings
- ↑ http://fedoraproject.org/wiki/QA/Meetings/20090722
- ↑ http://jlaska.livejournal.com/5693.html
- ↑ http://fedoraproject.org/wiki/QA_Meeting_Matrix
- ↑ http://www.redhat.com/archives/fedora-test-list/2009-July/msg00429.html
- ↑ http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
- ↑ http://wiki.sugarlabs.org/go/Sugar_on_a_Stick
- ↑ http://fedoraproject.org/wiki/User:Adamwill/Draft_test_day_SOP
- ↑ http://fedoraproject.org/wiki/BugZappers/Meetings
- ↑ http://meetbot.fedoraproject.org/fedora-meeting/2009-07-21/fedora-meeting.2009-07-21-15.02.log.html
- ↑ http://fedoraproject.org/wiki/User:Arxs/CPCL
F12 Alpha blocker bug review meeting
Adam Williamson announced[1] the second blocker bug review meeting for Fedora 12, to be held on 2009-07-24, mainly to review blocker bug status for the upcoming Alpha release. Later, James Laska posted a recap of the meeting[2].
Xfce spin testing
Adam Miller announced[1] the second test live image with the Xfce desktop, and would appreciate testing and reporting of problems. He noted that the known bugs in Anaconda at the time of the compose may make the image very difficult to install, but it should be usable on most hardware as a live boot.
KDE QA tester request
Kevin Kofler posted a request[1] for volunteers to help with KDE testing. He noted that the requirements for testers were quite low, and asked interested people to reply to the fedora-kde mailing list or #fedora-kde on IRC. Two people, Aioanei Rares and Marco Crosio, were quick to volunteer, and were accepted as the new KDE testers.
Bugzilla semantics debate
The Bugzilla semantics debate[1] continued throughout the week, especially following the input from developers at the QA meeting (see above) and the subsequent summary[2] posted by Adam Williamson. He proposed three options: leaving the current triage process unchanged and encouraging development teams who currently use ASSIGNED to mean a bug has been accepted by a certain developer to use ON_QA instead; changing Bugzappers practice to use a keyword to mark triaged bugs going forward, but leave all existing bugs as they are; or changing Bugzappers practice going forwards and also attempting to 'fix' existing bug reports to use the keyword where appropriate. Jesse Keating seemed to favor the second option[3], and John Poelstra agreed[4].