From Fedora Project Wiki

< FWN‎ | Beats

(create 179 qa beat)
(create fwn 180 qa beat)
Line 10: Line 10:
=== Test Days ===
=== Test Days ===


There was no Test Day last week, as we are deep in the Fedora 11 final release run-up.
There was no Test Day last week, as we finally released Fedora 11.


Currently, no Test Day is scheduled for next week - it is too close to the scheduled release of Fedora 11 for any testing to produce results directly in Fedora 11 final release, but if you would like to propose a test day which could result in changes for post-release updates, or an early test day for Fedora 12, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>https://fedorahosted.org/fedora-qa/</ref>.
Currently, no Test Day is scheduled for next week - it is still very early in the Fedora 12 cycle. If you would like to propose a test day which could result in changes for post-release updates for Fedora 11, or an early test day for Fedora 12, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>https://fedorahosted.org/fedora-qa/</ref>.


<references/>
<references/>
Line 18: Line 18:
=== Weekly meetings ===
=== Weekly meetings ===


The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-06-03. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090603</ref>. [[User:Adamwill|Adam Williamson]] reported that he had finally remembered to ask the Bugzilla team to add a link to the Fedora bug workflow page<ref>http://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow</ref> from the Bugzilla page<ref>https://bugzilla.redhat.com/page.cgi?id=fields.html</ref>. This has been done, and the link will show up with the next refresh of Bugzilla.
The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was to be held on 2009-06-10, but was cancelled for the week due to many key group members being busy with the Fedora Development Cycle Activity Day<ref>http://fedoraproject.org/wiki/Fedora_Activity_Day_Fedora_Development_Cycle_2009</ref>. Next week's meeting will cover the ground.


[[User:Jlaska|James Laska]] reported that he has now sent out the survey about Fedora 11 Test Days, asking participants for feedback on how the events went and any possible improvements that could be made<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00160.html</ref>. Some feedback had already been received, and much more was expected.
The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-06-09. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Jun-09</ref>. The group discussed revising the components and active triagers page<ref>http://fedoraproject.org/wiki/BugZappers/Components_and_Triagers</ref>, as is traditional at the start of a new release cycle. [[User:Adamwill|Adam Williamson]] suggested that, once the planned change to have the FAS 'triagers' group automatically grant membership of the 'fedorabugs' group, have new members apply to 'triagers' rather than 'fedorabugs', and ensure all current triagers are members of 'triagers' went through, the 'triagers' group membership list should become the canonical source of active triagers. The group agreed, but also decided to keep the Wiki page up to date. There was some discussion about whether changes directly from FAS, or from FAS via the triage metrics system, could be automatically fed into the Wiki page, but no decision was reached. In the end, [[User:arxs | Niels Haase]] volunteered to update the page by hand.


[[User:Wwoods|Will Woods]] reported that he had added two test cases for preupgrade <ref>http://fedoraproject.org/wiki/QA:Testcase_Preupgrade</ref>, <ref>http://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release</ref>, and updated the release candidate test matrix for RC3<ref>http://fedoraproject.org/wiki/QA:Fedora_11_RC3_Install_Test_Results</ref>.
[[User:Tk009|Edward Kirk]] proposed removing yum and anaconda from the list of components requiring triage, as their maintainers did not want help from the Bugzappers group. This prompted [[User:Adamwill|Adam Williamson]] to report that he had been working on engaging the kernel and anaconda teams in the Bugzappers process, at the request of [[User:Jlaska|James Laska]]. [[User:andyl|Andy Lindeberg]], who currently works on triaging anaconda, is working on a Wiki page that will document the process used in Bugzilla by the anaconda team, and then Adam will try to work with her and the Bugzappers group to reconcile the process with the normal Bugzappers process.


The group discussed how to handle the installation test result matrix wiki page<ref>https://fedoraproject.org/wiki/QA:Fedora_11_RC2_Install_Test_Results</ref> between release candidate revisions. [[User:Jlaska|James Laska]] committed to work out his best solution and send it to the mailing list.
Matej Cepl pointed out that the group had made a conscious decision at the start of the Fedora 11 cycle not to triage kernel bugs, as in the past it had taken a lot of time for little result. However, two group members - Brennan Ashton and [[User:Rjune|Richard June]] - said they were interested in attempting some kernel triage, if a good process could be found. [[User:Adamwill|Adam Williamson]] promised to continue the discussion with the kernel maintainers and bring in Edward and Richard with a view to agreeing a workable process for kernel bug triage.


[[User:Adamwill|Adam Williamson]] reported that he had added an entry to the Fedora 11 Common Bugs page<ref>http://fedoraproject.org/wiki/Common_F11_bugs</ref> for bug #502077<ref>http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=502077</ref>, but that the bug would now be fixed for final release and so the note should be removed. He clarified that issues which will be fixed for final release should just be removed from the page, not moved to the planned 'Resolved Issues' section.
The next QA weekly meeting will be held on 2009-06-17 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-16 at 1500 UTC in #fedora-meeting.
 
The group discussed the state of Fedora 11 final release preparation. In general building of release candidates and testing was progressing smoothly. [[User:Jlaska|James Laska]] asked that the group make an effort to confirm the fixes for the nine release-critical issues marked as MODIFIED in Bugzilla.
 
The group then discussed the appropriate way to document bug #503824<ref>http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=503824</ref>, where installation fails in certain circumstances on an x86-64 system with only 512MB of memory. In the end it was decided the most appropriate way to address this would be in the minimum hardware requirements. [[User:Adamwill|Adam Williamson]] volunteered to add a request for some appropriate text to be added to an existing bug report on revision of the minimum requirements.
 
[[User:Jlaska|James Laska]] then started a brainstorming session for a general review of QA's role during the Fedora 11 cycle. Many ideas were contributed by the entire group. A summary of these is available on the meeting page<ref>http://fedoraproject.org/wiki/QA/Meetings/20090603#F-11_QA_Post-mortem_discussion</ref>.
 
The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-06-02. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Jun-02</ref>. [[User:poelstra|John Poelstra]] reported on progress of the housekeeping changes for Fedora 11's release, and the group agreed that he was doing a fine job and should keep it up.
 
[[User:Adamwill|Adam Williamson]] reported on the progress of the triage metric system. The system<ref>http://publictest14.fedoraproject.org/triageweb/</ref> is now running on the real Bugzilla data, updated nightly. The system is now in its beta stage, and the developer Brennan Ashton asks that people experiment with it and report bugs or feature requests to trac<ref>http://fedorahosted.org/triage</ref> (component triageweb).
 
[[User:Adamwill|Adam Williamson]] also reported on the progress of the proposal to include setting the priority / severity fields as part of triage. It is now waiting on a change by the Bugzilla maintainers to restrict access to the priority and severity fields. This is being tracked in a bug report<ref>http://bugzilla.redhat.com/show_bug.cgi?id=495985</ref>. [[User:arxs | Niels Haase]] noted that he had already begun setting severity on reports he is triaging, according to the policy, and had not yet met with any resistance on the part of reporters. The group agreed that triagers could go ahead and begin setting the severity field ahead of the change to Bugzilla, if they would like to.
 
[[User:arxs | Niels Haase]] flagged up a bug<ref>http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=459323</ref> for possible inclusion in the Fedora 11 Common Bugs page. It involves resume from suspend failing when using the nouveau graphics driver. After some discussion, the group agreed it should be added to the list.
 
[[User:poelstra|John Poelstra]] announced that he would be stepping back from some of his leadership role within the BugZappers group, though remaining involved in many ways. The group thanked him for all his efforts so far. [[User:Adamwill|Adam Williamson]], [[User:Tk009|Edward Kirk]] and [[User:arxs | Niels Haase]] will cover meeting arrangements for the foreseeable future.
 
[[User:StevenParrish | Steven Parrish]] mentioned that he intended to go through all still-open Fedora 9 bugs for the components he triages, and try to determine whether they were still valid for a current release (and if so change them to that release), in advance of the automated closing of Fedora 9 bugs for EOL. [[User:Adamwill|Adam Williamson]] suggested he also mention this idea on the mailing list.
 
The next QA weekly meeting will be held on 2009-06-10 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-09 at 1500 UTC in #fedora-meeting.
 
<references/>
 
=== Release candidate build availability ===
 
Following on from last week's discussion of the availability of release candidate builds, Andre Robatino announced<ref>https://www.redhat.com/archives/fedora-test-list/2009-May/msg01372.html</ref> that he had built and made available delta ISOs - files containing the difference between two ISO images, allowing the reconstruction of the latest final image - for RC2, from Fedora 11 Preview. He later made delta ISOs available for RC3 and RC4. The group continued to discuss the feasibility of getting quickly-revised pre-release builds available from the public mirror system using various methods, but no conclusion has yet been reached.


<references/>
<references/>


=== Bugzilla statistics ===
=== Fedora 11 DeltaISO availability ===


Brennan Ashton released<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00093.html</ref> the first weekly Bugzilla statistics roundup, derived from the new triage metrics system. The response was enthusiastic, with requests and suggestions for more information from [[User:Johannbg|Jóhann Guðmundsson]]<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00097.html</ref> and [[User:Beland|Christopher Beland]]<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00121.html</ref>. There were also several positive responses on the development mailing list, where the information was also posted.
Andre Robatino announced<ref>https://www.redhat.com/archives/fedora-test-list/2009-June/msg00336.html</ref> that he had built and made available delta ISOs - files containing the difference between two ISO images, allowing the reconstruction of the latest final image - for Fedora 11 final release, from the Fedora 11 preview image. He also noted that he had built but could not publish ISOs for Fedora 10 to Fedora 11, and suggested that these could be provided as torrents on the official Fedora torrent tracker, but this has not yet been adopted.


<references/>
<references/>


=== 'How to report bugs' page revised ===
=== Components and Triagers page revision ===


[[User:Adamwill|Adam Williamson]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00137.html</ref> that he had made some changes to the main Wiki page on how to report bugs<ref>https://fedoraproject.org/wiki/Bugs_and_feature_requests</ref>. In particular, he had revised the section providing advice on what information to include in particular types of bug report to be more consistent. He encouraged everyone to contribute this type of information: if you know of specific information which is usually required when filing a particular type of bug (or a bug on a particular component), add this information following the layout used in the appropriate section of the page<ref>http://fedoraproject.org/wiki/Bugs_and_feature_requests#Tips_by_Type_of_Bug</ref>.
[[User:arxs | Niels Haase]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00401.html</ref> that he had revised the Components and Triagers page as agreed at the weekly Bugzappers meeting, to list only triagers known to be active. He recommended everyone check the diff for his changes<ref>http://fedoraproject.org/w/index.php?title=BugZappers%2FComponents_and_Triagers&diff=107583&oldid=107350</ref>, and make appropriate corrections if they had been incorrectly added to, removed from or kept on the list.


<references/>
<references/>


=== Fedora 11 Test Day survey ===
=== QA, Bugzappers and others involvement in release documentatation ===


[[User:Jlaska|James Laska]] posted a survey<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00160.html</ref> on the Fedora 11 Test Day process, asking for feedback on various facets of the process and suggestions for future improvements. The response was wide and enthusiastic, across both the QA and the development mailing lists, with many useful and constructive suggestions from testers and developers alike. James and [[User:Adamwill|Adam Williamson]] responded to several of the suggestions, affirming that many would be considered for implementation during the Fedora 12 Test Day cycle.
A post<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00345.html</ref> by Scott Robbins, suggesting a particular issue in Fedora 11 be noted on the download page, led to an extensive discussion of how those involved in the QA and BugZappers group, as well as those involved in front-line user support, could best document important issues at release time. [[User:Adamwill|Adam Williamson]] opposed documenting common problems on the download page as it would be hard to draw a line to prevent too extensive a list of problems complicating the page and discouraging people from downloading Fedora at all<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00347.html</ref>. In that post and others in the thread, Adam advocated trying to have all teams contribute known issues to a well-defined set of canonical pages, so that these pages would gain widespread use and acceptance among the community, particularly the Release Notes and Common Bugs pages. Adam also suggested<ref>http://www.redhat.com/archives/fedora-test-list/2009-June/msg00370.html</ref> that members of the QA, BugZappers and other teams with an interest in documenting significant issues with releases should join the Documentation project<ref>http://fedoraproject.org/wiki/Docs_Project</ref> in order to improve the communication between these teams and the docs team, and hopefully ensure that future Release Notes cover all the material they would like to see covered.


<references/>
<references/>

Revision as of 01:03, 13 June 2009

QualityAssurance

In this section, we cover the activities of the QA team[1].

Contributing Writer: Adam Williamson

Test Days

There was no Test Day last week, as we finally released Fedora 11.

Currently, no Test Day is scheduled for next week - it is still very early in the Fedora 12 cycle. If you would like to propose a test day which could result in changes for post-release updates for Fedora 11, or an early test day for Fedora 12, please contact the QA team via email or IRC, or file a ticket in QA Trac[1].

Weekly meetings

The QA group weekly meeting[1] was to be held on 2009-06-10, but was cancelled for the week due to many key group members being busy with the Fedora Development Cycle Activity Day[2]. Next week's meeting will cover the ground.

The Bugzappers group weekly meeting[3] was held on 2009-06-09. The full log is available[4]. The group discussed revising the components and active triagers page[5], as is traditional at the start of a new release cycle. Adam Williamson suggested that, once the planned change to have the FAS 'triagers' group automatically grant membership of the 'fedorabugs' group, have new members apply to 'triagers' rather than 'fedorabugs', and ensure all current triagers are members of 'triagers' went through, the 'triagers' group membership list should become the canonical source of active triagers. The group agreed, but also decided to keep the Wiki page up to date. There was some discussion about whether changes directly from FAS, or from FAS via the triage metrics system, could be automatically fed into the Wiki page, but no decision was reached. In the end, Niels Haase volunteered to update the page by hand.

Edward Kirk proposed removing yum and anaconda from the list of components requiring triage, as their maintainers did not want help from the Bugzappers group. This prompted Adam Williamson to report that he had been working on engaging the kernel and anaconda teams in the Bugzappers process, at the request of James Laska. Andy Lindeberg, who currently works on triaging anaconda, is working on a Wiki page that will document the process used in Bugzilla by the anaconda team, and then Adam will try to work with her and the Bugzappers group to reconcile the process with the normal Bugzappers process.

Matej Cepl pointed out that the group had made a conscious decision at the start of the Fedora 11 cycle not to triage kernel bugs, as in the past it had taken a lot of time for little result. However, two group members - Brennan Ashton and Richard June - said they were interested in attempting some kernel triage, if a good process could be found. Adam Williamson promised to continue the discussion with the kernel maintainers and bring in Edward and Richard with a view to agreeing a workable process for kernel bug triage.

The next QA weekly meeting will be held on 2009-06-17 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-16 at 1500 UTC in #fedora-meeting.

Fedora 11 DeltaISO availability

Andre Robatino announced[1] that he had built and made available delta ISOs - files containing the difference between two ISO images, allowing the reconstruction of the latest final image - for Fedora 11 final release, from the Fedora 11 preview image. He also noted that he had built but could not publish ISOs for Fedora 10 to Fedora 11, and suggested that these could be provided as torrents on the official Fedora torrent tracker, but this has not yet been adopted.

Components and Triagers page revision

Niels Haase announced[1] that he had revised the Components and Triagers page as agreed at the weekly Bugzappers meeting, to list only triagers known to be active. He recommended everyone check the diff for his changes[2], and make appropriate corrections if they had been incorrectly added to, removed from or kept on the list.

QA, Bugzappers and others involvement in release documentatation

A post[1] by Scott Robbins, suggesting a particular issue in Fedora 11 be noted on the download page, led to an extensive discussion of how those involved in the QA and BugZappers group, as well as those involved in front-line user support, could best document important issues at release time. Adam Williamson opposed documenting common problems on the download page as it would be hard to draw a line to prevent too extensive a list of problems complicating the page and discouraging people from downloading Fedora at all[2]. In that post and others in the thread, Adam advocated trying to have all teams contribute known issues to a well-defined set of canonical pages, so that these pages would gain widespread use and acceptance among the community, particularly the Release Notes and Common Bugs pages. Adam also suggested[3] that members of the QA, BugZappers and other teams with an interest in documenting significant issues with releases should join the Documentation project[4] in order to improve the communication between these teams and the docs team, and hopefully ensure that future Release Notes cover all the material they would like to see covered.