From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 227 qa beat)
(whoops, remove a chunk of 225 that was left in my draft)
Line 11: Line 11:


The QA group spent the last two weeks completing Fedora 13 testing and documentation. Following the one week delay of the Fedora 13 release discussed in FWN 225<ref>http://fedoraproject.org/wiki/FWN/Issue225#QualityAssurance</ref>, the team got down to testing the third release candidate<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/090918.html</ref>. We were able to produce a full installation test matrix<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC3_Install</ref> and desktop test results for GNOME and KDE<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC3_Desktop</ref>, thanks to the contributions of many team members. With the help of these results, the group was able to confidently support the nomination of RC3 as the final Fedora 13 release compose at the 2010-05-18 go/no-go meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-05-18/f-13-final-eng-readiness.2010-05-18-23.58.html</ref>. Finally, the group tested an updated preupgrade package for Fedora 12<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/091132.html</ref> which was being prepared in order to be ready for the final Fedora 13 public release date, to ensure that preupgrade-based upgrades from Fedora 12 would run smoothly.
The QA group spent the last two weeks completing Fedora 13 testing and documentation. Following the one week delay of the Fedora 13 release discussed in FWN 225<ref>http://fedoraproject.org/wiki/FWN/Issue225#QualityAssurance</ref>, the team got down to testing the third release candidate<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/090918.html</ref>. We were able to produce a full installation test matrix<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC3_Install</ref> and desktop test results for GNOME and KDE<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC3_Desktop</ref>, thanks to the contributions of many team members. With the help of these results, the group was able to confidently support the nomination of RC3 as the final Fedora 13 release compose at the 2010-05-18 go/no-go meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-05-18/f-13-final-eng-readiness.2010-05-18-23.58.html</ref>. Finally, the group tested an updated preupgrade package for Fedora 12<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/091132.html</ref> which was being prepared in order to be ready for the final Fedora 13 public release date, to ensure that preupgrade-based upgrades from Fedora 12 would run smoothly.
Testing of the test compose (TC1) began on 2010-04-29<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090520.html</ref>, with as usual installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_TC1_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_TC1_Desktop</ref> validation tests.
Thanks to this testing and also more general long-term Fedora 13 use, a large pool of release blocking bugs built up, with over 60 release blocking bugs needing to be addressed before final release could be considered. At the regular blocker bug review meetings, the group - along with the development and release engineering groups - gradually tackled this set of bugs until the list was eventually cleared for the RC1 release<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/090707.html</ref> on 2010-05-07, only a day behind the planned schedule.
RC1 testing - installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC1_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC1_Desktop</ref> - was generally positive, but two issues led to the release of a minimally-changed RC2<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/090721.html</ref>. RC2 installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC2_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC2_Desktop</ref> testing was again mostly positive, but identified a bug<ref>http://bugzilla.redhat.com/show_bug.cgi?id=590640</ref> with NFS installation, which at the 2010-05-12 go/no-go meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-05-12/f-13-readiness.2010-05-12-00.00.html</ref> was agreed to be potentially significant enough to delay the final release. The slip was announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-May/090881.html</ref> on 2010-05-12; a RC3 build will now be provided by the release engineering team for QA testing during the next few days and, we hope, validation for release on 2010-05-25.


<references/>
<references/>

Revision as of 23:13, 26 May 2010

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

Fedora 13 testing

The QA group spent the last two weeks completing Fedora 13 testing and documentation. Following the one week delay of the Fedora 13 release discussed in FWN 225[1], the team got down to testing the third release candidate[2]. We were able to produce a full installation test matrix[3] and desktop test results for GNOME and KDE[4], thanks to the contributions of many team members. With the help of these results, the group was able to confidently support the nomination of RC3 as the final Fedora 13 release compose at the 2010-05-18 go/no-go meeting[5]. Finally, the group tested an updated preupgrade package for Fedora 12[6] which was being prepared in order to be ready for the final Fedora 13 public release date, to ensure that preupgrade-based upgrades from Fedora 12 would run smoothly.

  1. ↑ http://fedoraproject.org/wiki/FWN/Issue225#QualityAssurance
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-May/090918.html
  3. ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC3_Install
  4. ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_13_Final_RC3_Desktop
  5. ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-05-18/f-13-final-eng-readiness.2010-05-18-23.58.html
  6. ↑ http://lists.fedoraproject.org/pipermail/test/2010-May/091132.html

Fedora 13 QA Retrospective

At the 2010-05-17 weekly meeting[1], James Laska reminded the group that there was a page[2] to gather thoughts about the QA process throughout the Fedora 13 cycle, including things that had gone well, things which had not gone so well, and ideas for future enhancements to the QA process.

  1. ↑ http://fedoraproject.org/wiki/QA/Meetings/20100517
  2. ↑ http://fedoraproject.org/wiki/Fedora_13_QA_Retrospective

Making QA sexy

At the same meeting, Adam Miller initiated a heroic attempt to achieve the improbable: making QA work sexy. Adam suggested providing customized swag for top QA contributors, such as a special QA group t-shirt. Others were thinking of ways to identify top contributors. James Laska suggested looking at Bodhi feedback. Will Woods thought about a way for developers to nominate testers and bug reporters who had made significant contributions.


Triage scripting

At the 2010-05-18 Bugzappers weekly meeting[1], Matej Cepl mentioned that he was rewriting his browser scripts to assist the process of triaging, and would be presenting on the topic at GUADEC on 2010-07-28[2]. He explained that "the idea is a) to propagate existence of the scripts around among developers, b) to make them compatible with multiple instances and making upstream ... I have somebody working on something similar for (Mozilla)".

  1. ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-05-18/fedora-meeting.2010-05-18-15.03.log.html
  2. ↑ http://live.gnome.org/GUADEC/2010/Schedule/Main

AutoQA

James Laska proposed an email update instead of a weekly meeting for 2010-05-24. In his email[1], he noted that a new version of AutoQA was due, which would include backlinks from test results to the test logs[2], and the ability to subscribe to test results for a specific package[3]. Replying[4], Kamil Paral noted that his package sanity tests were now working "for the most basic cases" and he was now looking into sandboxing the testing via libvirt. Will Woods reported[5] that he was continuing to work on a watcher script for noticing Bodhi updates (as part of the dependency check tests), and hoped to have the post-bodhi-update hook "up and running by the end of this week".

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-May/091164.html
  2. ↑ http://fedorahosted.org/autoqa/ticket/130
  3. ↑ http://fedorahosted.org/autoqa/ticket/151
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-May/091180.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2010-May/091188.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=176594"