FWN/Beats/QualityAssurance

QualityAssurance

In this section, we cover the activities of the QA team.

Contributing Writer: Adam Williamson

Test Days

There was no Test Day last week.

No Test Day is scheduled on the main track next week. However, the new Fit and Finish Test Day track will be holding its first event, on display configuration (particularly multiple displays, and projectors). The Test Day page already includes several test scenarios, and a live CD for testing will soon be available. The Fit and Finish project is a great effort to improve the details of the Fedora project, so please show up to support their first event! The Test Day will be held on 2009-07-07 (Tuesday) in IRC #fedora-fitandfinish (note this is not the same channel where main track Test Days take place).

If you would like to propose a main track Test Day for the Fedora 12 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac.

Weekly meetings

The QA group weekly meeting was held on 2009-07-01. The full log is available. James Laska reported that he had not yet been able to update the QA Goals page, but had started work on it and hoped to have it complete soon. He noted that he had completed his survey of feedback on the Test Day review questionnaire.

Will Woods reported that he had added two more test cases to the Rawhide acceptance test plan, based on suggestions received to the initial plan: post-install networking and post-install input.

Matthias Clasen reported on the new Fit and Finish project, discussed on mailing lists under the topic 'raising the bar'. He explained it would be based on a series of Test Days concentrating on user experience rather than developer-driven 'features', and hoped members of the QA team would come along to help drive testing. Jóhann Guðmundsson suggested that the Test Day pages should provide more instructions on what information to include with bug reports for the Fit and Finish topics.

Will Woods gave a quick further update on the status of AutoQA (which includes the rawhide acceptance testing discussed earlier). He noted that the test plan was complete and seven of twelve test cases had been written. The next step is to start automating cases, which is waiting on the packaging of autotest. This is being tracked in a ticket. James Laska and Jesse Keating are working on this together, and making good progress. Will hopes that by next week, all test cases will be written and one will be automated. He also noted he was working on how to allow manual test result reports, as there will inevitably be cases that cannot be automated.

James Laska reported on behalf of Liam Li, on his draft for a installation test event SOP. This is intended to document the procedure for running an event where multiple people work together in real time to test installation. James reported that Liam had produced an initial draft, and it had been revised with help from Christopher Beland and Niels Haase.

Finally, Jóhann Guðmundsson reported on his project to improve the quality and quantity of information contained in bug reports. He had filed an RFE to improve the interface of abrt (the automated bug reporting tool). Matthias Clasen suggested asking the Design team for help in further refining the interface, and James Laska pointed out they had come up with good suggestions for the SELinux troubleshooting GUI design.

The Bugzappers group weekly meeting was held on 2009-06-30. The full log is available.

Brennan Ashton reported on the status of the triage metric project. He had got FAS integration and component grouping implemented, but adding some features requested by James Laska had caused significant breakage which he was working on.

Niels Haase provided an update on his efforts to update the triage critical component list. He had written a draft for a list based on the critical path component list being developed for that proposal, and proposed to add components from this list to the existing list. The group approved this in principle, but pointed out several components that should be removed as they do not make sense from the triage perspective, and also noted that any packages which the components in the list depend on should be considered for addition as well. Niels promised to provide a list of these dependencies for the next meeting.

Adam Williamson reported on the project to improve (and create) pages explaining how to debug problems in particular components, and what information to include when reporting bugs in these components. This had grown out of Jóhann Guðmundsson's 'improve reporting' proposal. He encouraged all group members to contribute to the page for their triage components, and create a page if there is not yet one. The page for X.org was cited as a good example or template. Niels Haase mentioned a page for NetworkManager currently in draft status. Adam also mentioned he had received a proposal from Ubuntu's James Westby for the distributions to work together on combining their pages of this type, and contributing them to the original upstream projects. The group agreed this was a good idea in principle and they would wait for further information from James, relayed via Adam.

Matej Cepl reported that the latest rewrite of the triage team GreaseMonkey script was nearly complete, and appealed for volunteers to help test it.

Adam Williamson reported that the project to implement use of the severity and priority fields in Bugzilla was now more or less complete, and asked all triagers if they had not already to begin assessing the severity of bugs when triaging them. He referred to the instructions on this in the Triage Guide on how to assign severity appropriately.

The next QA weekly meeting will be held on 2009-07-08 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-07-07 at 1500 UTC in #fedora-meeting.

Test Day survey results

James Laska posted a round-up of the results of his survey on the Fedora 11 cycle Test Days. It provides an overview and summary of the feedback received both publicly and privately in response to the survey.