From Fedora Project Wiki

< FWN‎ | Beats

(typo)
(create fwn 181 draft)
Line 18: Line 18:
=== Weekly meetings ===
=== Weekly meetings ===


The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was to be held on 2009-06-10, but was cancelled for the week due to many key group members being busy with the Fedora Development Cycle Activity Day<ref>http://fedoraproject.org/wiki/Fedora_Activity_Day_Fedora_Development_Cycle_2009</ref>. Next week's meeting will cover the ground.
The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-06-17. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090617</ref>. [[User:Adamwill|Adam Williamson]] reported that he had added a suggested draft for the previously agreed change to the minimum hardware requirements to an existing bug report<ref>http://bugzilla.redhat.com/show_bug.cgi?id=499585</ref>.


The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-06-09. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Jun-09</ref>. The group discussed revising the components and active triagers page<ref>http://fedoraproject.org/wiki/BugZappers/Components_and_Triagers</ref>, as is traditional at the start of a new release cycle. [[User:Adamwill|Adam Williamson]] suggested that, once the planned change to have the FAS 'triagers' group automatically grant membership of the 'fedorabugs' group, have new members apply to 'triagers' rather than 'fedorabugs', and ensure all current triagers are members of 'triagers' went through, the 'triagers' group membership list should become the canonical source of active triagers. The group agreed, but also decided to keep the Wiki page up to date. There was some discussion about whether changes directly from FAS, or from FAS via the triage metrics system, could be automatically fed into the Wiki page, but no decision was reached. In the end, [[User:arxs | Niels Haase]] volunteered to update the page by hand.
[[User:Jlaska|James Laska]] reported on the Fedora 11 retrospective meeting which had taken place the previous day, and which he, [[User:Adamwill|Adam Williamson]], [[User:jkeating|Jesse Keating]] and [[User:Tk009|Edward Kirk]] had attended to represent the QA and BugZappers groups. All agreed that the meeting had been well-run and productive. Jesse pointed out that the real test of its success would be if any of the items discussed had led to actual changes within the next month or two. James promised to update the QA team's Goals page<ref>http://fedoraproject.org/wiki/QA/Goals</ref> to incorporate the lessons learned from the Fedora 11 release cycle.


[[User:Tk009|Edward Kirk]] proposed removing yum and anaconda from the list of components requiring triage, as their maintainers did not want help from the Bugzappers group. This prompted [[User:Adamwill|Adam Williamson]] to report that he had been working on engaging the kernel and anaconda teams in the Bugzappers process, at the request of [[User:Jlaska|James Laska]]. [[User:andyl|Andy Lindeberg]], who currently works on triaging anaconda, is working on a Wiki page that will document the process used in Bugzilla by the anaconda team, and then Adam will try to work with her and the Bugzappers group to reconcile the process with the normal Bugzappers process.
The group discussed whether some kind of voice format for the meeting would be better than IRC, but in the end there was wide agreement that it would not be.


Matej Cepl pointed out that the group had made a conscious decision at the start of the Fedora 11 cycle not to triage kernel bugs, as in the past it had taken a lot of time for little result. However, two group members - Brennan Ashton and [[User:Rjune|Richard June]] - said they were interested in attempting some kernel triage, if a good process could be found. [[User:Adamwill|Adam Williamson]] promised to continue the discussion with the kernel maintainers and bring in Edward and Richard with a view to agreeing a workable process for kernel bug triage.
[[User:Jlaska|James Laska]] mentioned that the period for open feedback on proposals stemming from the earlier Fedora Development Cycle Activity Day<ref>http://fedoraproject.org/wiki/Fedora_Activity_Day_Fedora_Development_Cycle_2009</ref> was nearing an end: feedback on these proposals will be accepted up to June 30th. The proposals can be found in an email from [[User:jkeating|Jesse Keating]]<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00385.html</ref>. Anyone interested is encouraged to read the proposals and provide feedback.


The next QA weekly meeting will be held on 2009-06-17 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-16 at 1500 UTC in #fedora-meeting.
[[User:Wwoods|Will Woods]] reported on progress of the AutoQA project. He noted that one of the FAD proposals, israwhidebroken.com<ref>http://fedoraproject.org/wiki/Israwhidebroken.com_Proposal</ref>, is essentially an AutoQA project, and so he has established it as the first AutoQA milestone, with a set of tickets<ref>http://fedorahosted.org/autoqa/milestone/israwhidebroken.com</ref>. He noted that [[User:jkeating|Jesse Keating]] is working on packaging autotest<ref>http://autotest.kernel.org/</ref>, which will be the harness used to create the automated tests for this project. Jesse pointed out that autotest required Google Web Toolkit, which is not yet packaged either, so packaging autotest is a big project, but he was confident that he will be successful.
 
The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-06-16. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Jun-16</ref>. [[User:Adamwill|Adam Williamson]] reported that Brennan Ashton is working on having a components page as part of the triage metrics system<ref>http://publictest14.fedoraproject.org/triageweb/</ref>.
 
[[User:Tk009|Edward Kirk]] reported on his attempts to find out how the critical component list was generated so it can be accurately updated. He later spoke with [[User:Jstanley| Jon Stanley]] and [[User:jkeating|Jesse Keating]] and was advised to use the critical path packages proposal<ref>http://fedoraproject.org/wiki/Critical_Path_Packages_Proposal</ref> to help re-generate the list. He will report further next week.
 
[[User:Adamwill|Adam Williamson]] reported on the progress of the kernel triage project. He had sent an email to all interested parties, asking the kernel team to provide information on the current workflow used for kernel bugs, but had not yet received a reply, so this project is currently waiting on that important information from the kernel team.
 
[[User:Adamwill|Adam Williamson]] also reported on a request received from the EPEL team<ref>http://fedoraproject.org/wiki/EPEL</ref> for some help with a Bug Day they have planned for July 11th<ref>http://fedoraproject.org/wiki/EPEL_Bug_Day_July_2009</ref>. [[User:Kevin|Kevin Fenzi]], who is part of the EPEL project, provided some explanations: a copy of RHEL is not required to help, Bugzappers could help with only a CentOS box, or even just Fedora in some cases. Help asked of the Bugzappers team is mostly in typical Bugzappers tasks of triaging and pinging dormant bugs for further information. The EPEL project follows the Fedora bug workflow. The group agreed that they would be happy to help out with the Bug Day, and asked the EPEL project to provide more information closer to the date.
 
The next QA weekly meeting will be held on 2009-06-24 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-23 at 1500 UTC in #fedora-meeting.


<references/>
<references/>


=== Fedora 11 DeltaISO availability ===
=== Improved and more detailed QA / Release Engineering scheduling for Fedora 12 ===


Andre Robatino announced<ref>https://www.redhat.com/archives/fedora-test-list/2009-June/msg00336.html</ref> that he had built and made available delta ISOs - files containing the difference between two ISO images, allowing the reconstruction of the latest final image - for Fedora 11 final release, from the Fedora 11 preview image. He also noted that he had built but could not publish ISOs for Fedora 10 to Fedora 11, and suggested that these could be provided as torrents on the official Fedora torrent tracker, but this has not yet been adopted.
[[User:poelstra|John Poelstra]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00507.html</ref> that he had worked on a draft for an improved and more detailed schedule for release engineering tasks for Fedora 12, which also affects QA. He suggested that "we should move to more of a standard software model (just as we did with the naming of the schedule, etc.) where there is more separation between Releng and QA. IOW, Releng provides the service of packing the bits and composing an installable distro and QA provides the service of testing them and giving a thumbs up/down on them", and asked "What tweaks should I make to better reflect QA's needs?" The proposed schedule has blocker bug reviews happening the Friday before key freezes, exact dates for release engineering composes, and exact dates for compose testing. [[User:Jlaska|James Laska]] replied<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00513.html</ref> with broad support for each of the proposals.


<references/>
<references/>


=== Components and Triagers page revision ===
=== Installer test plan ===


[[User:arxs | Niels Haase]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00401.html</ref> that he had revised the Components and Triagers page as agreed at the weekly Bugzappers meeting, to list only triagers known to be active. He recommended everyone check the diff for his changes<ref>http://fedoraproject.org/w/index.php?title=BugZappers%2FComponents_and_Triagers&diff=107583&oldid=107350</ref>, and make appropriate corrections if they had been incorrectly added to, removed from or kept on the list.
[[User:Liam|Liam Li]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00529.html</ref> a test plan for installation for Fedora 12<ref>http://fedoraproject.org/wiki/QA/TestPlans/Fedora12Install</ref>, and asked for feedback. [[User:Jlaska|James Laska]] replied<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00538.html</ref>, noting that he had made some minor changes to the Wiki page, and providing some comments on the plan. He pointed out that it may be a good idea to consider the yum install cleanup feature<ref>http://fedoraproject.org/wiki/Anaconda/Features/YuminstallCleanup</ref> in the plan, and suggested only listing the test priority order once.


<references/>
<references/>


=== QA, Bugzappers and others involvement in release documentation ===
=== Rawhide acceptance test plan ===


A post<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00345.html</ref> by Scott Robbins, suggesting a particular issue in Fedora 11 be noted on the download page, led to an extensive discussion of how those involved in the QA and BugZappers group, as well as those involved in front-line user support, could best document important issues at release time. [[User:Adamwill|Adam Williamson]] opposed documenting common problems on the download page as it would be hard to draw a line to prevent too extensive a list of problems complicating the page and discouraging people from downloading Fedora at all<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00347.html</ref>. In that post and others in the thread, Adam advocated trying to have all teams contribute known issues to a well-defined set of canonical pages, so that these pages would gain widespread use and acceptance among the community, particularly the Release Notes and Common Bugs pages. Adam also suggested<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00370.html</ref> that members of the QA, BugZappers and other teams with an interest in documenting significant issues with releases should join the Documentation project<ref>http://fedoraproject.org/wiki/Docs_Project</ref> in order to improve the communication between these teams and the docs team, and hopefully ensure that future Release Notes cover all the material they would like to see covered.
[[User:Wwoods|Will Woods]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00547.html</ref> the creation of a Rawhide acceptance test plan<ref>http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan</ref>, which is part of the israwhidebroken.com proposal discussed during the weekly meeting (see above). This outlines the overall set of features which should be tested to determine if Rawhide is currently in a usable state or not. [[User:Johannbg|Jóhann Guðmundsson]] suggested<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00552.html</ref> a test for whether a basic wired network connection could be established. [[User:Adamwill|Adam Williamson]] suggested<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00549.html</ref> a test for whether basic input (keyboard and mouse) are working. Tom London suggested<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00565.html</ref> a test for encrypted root filesystems, but Will explained<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00566.html</ref> that this was beyond the scope of basic functionality testing.


<references/>
<references/>

Revision as of 22:04, 19 June 2009

QualityAssurance

In this section, we cover the activities of the QA team[1].

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA

Test Days

There was no Test Day last week, as we finally released Fedora 11.

Currently, no Test Day is scheduled for next week - it is still very early in the Fedora 12 cycle. If you would like to propose a test day which could result in changes for post-release updates for Fedora 11, or an early test day for Fedora 12, please contact the QA team via email or IRC, or file a ticket in QA Trac[1].

  1. ↑ https://fedorahosted.org/fedora-qa/

Weekly meetings

The QA group weekly meeting[1] was held on 2009-06-17. The full log is available[2]. Adam Williamson reported that he had added a suggested draft for the previously agreed change to the minimum hardware requirements to an existing bug report[3].

James Laska reported on the Fedora 11 retrospective meeting which had taken place the previous day, and which he, Adam Williamson, Jesse Keating and Edward Kirk had attended to represent the QA and BugZappers groups. All agreed that the meeting had been well-run and productive. Jesse pointed out that the real test of its success would be if any of the items discussed had led to actual changes within the next month or two. James promised to update the QA team's Goals page[4] to incorporate the lessons learned from the Fedora 11 release cycle.

The group discussed whether some kind of voice format for the meeting would be better than IRC, but in the end there was wide agreement that it would not be.

James Laska mentioned that the period for open feedback on proposals stemming from the earlier Fedora Development Cycle Activity Day[5] was nearing an end: feedback on these proposals will be accepted up to June 30th. The proposals can be found in an email from Jesse Keating[6]. Anyone interested is encouraged to read the proposals and provide feedback.

Will Woods reported on progress of the AutoQA project. He noted that one of the FAD proposals, israwhidebroken.com[7], is essentially an AutoQA project, and so he has established it as the first AutoQA milestone, with a set of tickets[8]. He noted that Jesse Keating is working on packaging autotest[9], which will be the harness used to create the automated tests for this project. Jesse pointed out that autotest required Google Web Toolkit, which is not yet packaged either, so packaging autotest is a big project, but he was confident that he will be successful.

The Bugzappers group weekly meeting[10] was held on 2009-06-16. The full log is available[11]. Adam Williamson reported that Brennan Ashton is working on having a components page as part of the triage metrics system[12].

Edward Kirk reported on his attempts to find out how the critical component list was generated so it can be accurately updated. He later spoke with Jon Stanley and Jesse Keating and was advised to use the critical path packages proposal[13] to help re-generate the list. He will report further next week.

Adam Williamson reported on the progress of the kernel triage project. He had sent an email to all interested parties, asking the kernel team to provide information on the current workflow used for kernel bugs, but had not yet received a reply, so this project is currently waiting on that important information from the kernel team.

Adam Williamson also reported on a request received from the EPEL team[14] for some help with a Bug Day they have planned for July 11th[15]. Kevin Fenzi, who is part of the EPEL project, provided some explanations: a copy of RHEL is not required to help, Bugzappers could help with only a CentOS box, or even just Fedora in some cases. Help asked of the Bugzappers team is mostly in typical Bugzappers tasks of triaging and pinging dormant bugs for further information. The EPEL project follows the Fedora bug workflow. The group agreed that they would be happy to help out with the Bug Day, and asked the EPEL project to provide more information closer to the date.

The next QA weekly meeting will be held on 2009-06-24 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-23 at 1500 UTC in #fedora-meeting.

  1. ↑ http://fedoraproject.org/wiki/QA/Meetings
  2. ↑ http://fedoraproject.org/wiki/QA/Meetings/20090617
  3. ↑ http://bugzilla.redhat.com/show_bug.cgi?id=499585
  4. ↑ http://fedoraproject.org/wiki/QA/Goals
  5. ↑ http://fedoraproject.org/wiki/Fedora_Activity_Day_Fedora_Development_Cycle_2009
  6. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00385.html
  7. ↑ http://fedoraproject.org/wiki/Israwhidebroken.com_Proposal
  8. ↑ http://fedorahosted.org/autoqa/milestone/israwhidebroken.com
  9. ↑ http://autotest.kernel.org/
  10. ↑ http://fedoraproject.org/wiki/BugZappers/Meetings
  11. ↑ http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Jun-16
  12. ↑ http://publictest14.fedoraproject.org/triageweb/
  13. ↑ http://fedoraproject.org/wiki/Critical_Path_Packages_Proposal
  14. ↑ http://fedoraproject.org/wiki/EPEL
  15. ↑ http://fedoraproject.org/wiki/EPEL_Bug_Day_July_2009

Improved and more detailed QA / Release Engineering scheduling for Fedora 12

John Poelstra announced[1] that he had worked on a draft for an improved and more detailed schedule for release engineering tasks for Fedora 12, which also affects QA. He suggested that "we should move to more of a standard software model (just as we did with the naming of the schedule, etc.) where there is more separation between Releng and QA. IOW, Releng provides the service of packing the bits and composing an installable distro and QA provides the service of testing them and giving a thumbs up/down on them", and asked "What tweaks should I make to better reflect QA's needs?" The proposed schedule has blocker bug reviews happening the Friday before key freezes, exact dates for release engineering composes, and exact dates for compose testing. James Laska replied[2] with broad support for each of the proposals.

  1. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00507.html
  2. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00513.html

Installer test plan

Liam Li announced[1] a test plan for installation for Fedora 12[2], and asked for feedback. James Laska replied[3], noting that he had made some minor changes to the Wiki page, and providing some comments on the plan. He pointed out that it may be a good idea to consider the yum install cleanup feature[4] in the plan, and suggested only listing the test priority order once.

  1. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00529.html
  2. ↑ http://fedoraproject.org/wiki/QA/TestPlans/Fedora12Install
  3. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00538.html
  4. ↑ http://fedoraproject.org/wiki/Anaconda/Features/YuminstallCleanup

Rawhide acceptance test plan

Will Woods announced[1] the creation of a Rawhide acceptance test plan[2], which is part of the israwhidebroken.com proposal discussed during the weekly meeting (see above). This outlines the overall set of features which should be tested to determine if Rawhide is currently in a usable state or not. Jóhann Guðmundsson suggested[3] a test for whether a basic wired network connection could be established. Adam Williamson suggested[4] a test for whether basic input (keyboard and mouse) are working. Tom London suggested[5] a test for encrypted root filesystems, but Will explained[6] that this was beyond the scope of basic functionality testing.

  1. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00547.html
  2. ↑ http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
  3. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00552.html
  4. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00549.html
  5. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00565.html
  6. ↑ http://www.redhat.com/archives/fedora-test-list/2009-June/msg00566.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=109367"