FWN/Beats/QualityAssurance: Difference between revisions

Revision as of 00:30, 28 September 2009

QualityAssurance

In this section, we cover the activities of the QA team^[1].

↑ http://fedoraproject.org/wiki/QA

Test Days

Last week's Test Day^[1] was on Xfce^[2], the popular alternative desktop. including PulseAudio^[3]. The always-loyal Xfce enthusiast base turned out to check on the polish of the Xfce desktop for Fedora 12, and filed many useful bug reports. Thanks to everyone who came out.

Next week's Test Day^[4] on 2009-10-01 will be on the installer's (Anaconda) storage system. As always, the Test Day will run all day in the #fedora-test-day IRC channel. The complete rewrite of Anaconda's storage code which first appeared in Fedora 11 continues to be refined, so please come out to the Test Day and help us ensure Anaconda is tested on a wide range of storage hardware and configurations.

No Fit and Finish track Test Day is planned for next week.

If you would like to propose a main track Test Day for the Fedora 12 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac^[5].

Weekly meetings

The QA group weekly meeting^[1] was held on 2009-09-21. The full log is available^[2]. Adam Williamson noted that the zsync issues raised at the previous meeting had been discussed in a development mailing list thread^[3], but no real movement had yet occurred.

Adam Williamson mentioned that the beta release process was getting underway, and the test compose would be happening on 2009-09-23. He also reminded the group that a blocker bug review meeting would be taking place on 2009-09-25. He asked the group to test the test compose, Rawhide itself, and the nightly Rawhide live CDs as much as possible, and particularly to try and test beta blockers bugs that required re-testing.

David Pravec said he had been talking to the Anaconda team and they were unhappy about significant changes being made to components on which Anaconda depends so close to the beta release time. The specific example raised was the major changes made to the nss package in the days before the meeting. The group discussed the problem, and agreed that there needed to be better communication between the Anaconda team and developers responsible for the components on which it relies. Jesse Keating pointed out that such changes sometimes result from the Red Hat Enterprise Linux development process, with Red Hat staff being requested to make changes in Fedora for the ultimate benefit of that product. The group unanimously felt that there should be a process for such changes which ensures that they do not negatively affect the Fedora development process, and that Red Hat Enterprise Linux engineers should be required to be considerate of Fedora deadlines and processes when making changes to Fedora. Jesse said he should be able to raise this issue with the appropriate Red Hat management staff.

Seth Vidal asked the group whether it would be acceptable to submit a new version of yum which includes history support to Rawhide prior to the release of Fedora 12 Beta. After some discussion, the group agreed that it would be an acceptable risk to make this change, as Seth had already performed some good testing on it and had a good plan for backing out the change if it became necessary.

The Bugzappers group weekly meeting^[4] was held on 2009-09-22. The full log is available^[5]. Edward Kirk noted there had been no real movement on the triage day improvement topic, but he was hoping to pull something together soon.

Edward Kirk was also looking for a way to improve the tracking of action items from previous meetings, some of which he felt had fallen through the cracks in the past. After some discussion, John5342 suggested using the team's trac instance^[6] for this purpose, and the group agreed this was a good idea. Edward Kirk volunteered to transfer any outstanding action items from previous meetings to Trac.

No-one had heard from Brennan Ashton regarding the triage metrics project in the week before the meeting.

Richard June gave an update on the kernel triage project. He was still finding that the wireless tickets he was checking were all well-filed and would not much benefit from triage. He was therefore moving to look at a wider range of kernel bugs to see if this impression held true. He was also planning to get in touch with kernel developers to get some instruction on what kinds of information they would require on particular bugs.

The next QA weekly meeting will be held on 2009-09-28 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-09-29 at 1500 UTC in #fedora-meeting.

Fedora 12 Beta Test compose

Liam Li announced^[1] the Fedora 12 Beta test compose, a test build to check for major showstopper bugs that would appear only in the traditional installer physical media builds. Earlier, he had sent a mail detailing the types of testing^[2] that would be useful, and the results matrix page^[3] for reporting results. Several people reported failures with the test compose images, and Liam later announced an updated test compose^[4] which fixed the most significant bug affecting the earlier compose.

Bug resolution discussion

Sergey Rudchenko asked^[1] whether triagers could ever close a bug as NEXTRELEASE without reference to the maintainer responsible for the bug. Adam Williamson explained^[2] that this was rarely the case, as it would require certain knowledge of the maintainers intentions regarding which releases they intended to fix the bug in.Karel Volny asked^[3] why some of what Adam had said did not seem to match the policy documented on the Bugzilla page^[4]. Adam explained^[5] that the canonical reference for Fedora's bug process is in fact on the Wiki^[6], and explained some wrinkles around distribution versioning, which he promised to explain on the Wiki page.

[1] ttp://fedoraproject.org/wiki/QA

[2] ttp://fedoraproject.org/wiki/Test_Day:2009-09-24_XFCE

[3] ttp://fedoraproject.org/wiki/Xfce

[4] ttp://fedoraproject.org/wiki/Test_Day:2009-09-16_Audio

[5] ttp://fedoraproject.org/wiki/Test_Day:2009-10-01_Anaconda/Features/StorageFiltering

[6] ttp://fedorahosted.org/fedora-qa/

[7] ttp://fedoraproject.org/wiki/QA/Meetings

[8] ttp://fedoraproject.org/wiki/QA/Meetings/20090921

[9] ttp://www.redhat.com/archives/fedora-devel-list/2009-September/msg00525.html

[10] ttp://fedoraproject.org/wiki/BugZappers/Meetings

[11] ttp://meetbot.fedoraproject.org/fedora-meeting/2009-09-22/fedora-meeting.2009-09-22-15.13.log.html

[12] ttp://fedorahosted.org/triage/

[13] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00560.html

[14] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00446.html

[15] ttp://fedoraproject.org/wiki/Test_Results:Fedora_12_Beta_TC_Install

[16] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00656.html

[17] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00549.html

[18] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00552.html

[19] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00563.html

[20] ttp://bugzilla.redhat.com/page.cgi?id=fields.html#status

[21] ttp://www.redhat.com/archives/fedora-test-list/2009-September/msg00565.html

[22] ttp://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow

[1]

[1]

[2]

[3]

[4]

[5]

[1]

[2]

[3]

[4]

[5]

[6]

[1]

[2]

[3]

[4]

[1]

[2]

[3]

[4]

[5]

[6]

@@ Line 10: / Line 10: @@
 === Test Days ===
-Last week saw two main track Test Days. The first was on audio, including PulseAudio<ref>http://fedoraproject.org/wiki/Test_Day:2009-09-16_Audio</ref>. The second was on virtualization<ref>http://fedoraproject.org/wiki/Test_Day:2009-09-17_Virtualization</ref>. The audio Test Day saw good turnout and encouraging results, with basic audio functionality failing for only two out of 21 testers. The virtualization event saw a group of experienced testers carry out much detailed testing on a wide range of features. Thanks to all who turned out to both events.
+Last week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2009-09-24_XFCE</ref> was on Xfce<ref>http://fedoraproject.org/wiki/Xfce</ref>, the popular alternative desktop. including PulseAudio<ref>http://fedoraproject.org/wiki/Test_Day:2009-09-16_Audio</ref>. The always-loyal Xfce enthusiast base turned out to check on the polish of the Xfce desktop for Fedora 12, and filed many useful bug reports. Thanks to everyone who came out.
-Next week's Test Day will be on Xfce<ref>https://fedoraproject.org/wiki/Test_Day:2009-09-24_XFCE</ref>. This popular alternative desktop environment has an enthusiastic Fedora maintenance team and has produced polished live CD releases for the last few Fedora releases, so if you're an Xfce fan, please come along to the Test Day and help make sure it's in good working shape for the Fedora 12 release!
+Next week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2009-10-01_Anaconda/Features/StorageFiltering</ref> on 2009-10-01 will be on the installer's (Anaconda) storage system. As always, the Test Day will run all day in the #fedora-test-day IRC channel. The complete rewrite of Anaconda's storage code which first appeared in Fedora 11 continues to be refined, so please come out to the Test Day and help us ensure Anaconda is tested on a wide range of storage hardware and configurations.
 No Fit and Finish track Test Day is planned for next week.
@@ Line 22: / Line 22: @@
 === Weekly meetings ===
-The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-09-14. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090914</ref>. [[User:Adamwill|Adam Williamson]] mentioned that the new test-announce mailing list<ref>http://admin.fedoraproject.org/mailman/listinfo/test-announce</ref> was up and running and already being used for announcements, and thanked [[User:dpravec|David Pravec]] for the idea and the implementation.
+The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-09-21. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090921</ref>. [[User:Adamwill|Adam Williamson]] noted that the zsync issues raised at the previous meeting had been discussed in a development mailing list thread<ref>http://www.redhat.com/archives/fedora-devel-list/2009-September/msg00525.html</ref>, but no real movement had yet occurred.
-[[User:Kparal|Kamil Paral]] gave an update on his investigation into the possible use of zsync to reduce the size of live image transfers. He had written a blog post<ref>http://kparal.wordpress.com/2009/09/01/zsync-transfer-large-files-efficiently/</ref> on the potential benefits. They seemed significant and worthwhile, but [[User:Kevin|Kevin Fenzi]] had pointed out a problem: zsync is not packaged in Fedora, so cannot be installed on the server, and acceptance of zsync as a Fedora package is currently blocked by its use of an internal copy of the zlib library<ref>https://bugzilla.redhat.com/show_bug.cgi?id=490140</ref>. Various attempts have been made to have the zlib changes involved, which are also included in rsync, split out into a separate package or merged into upstream zlib, but this has not yet been successfully accomplished. After some discussion, [[User:Adamwill|Adam Williamson]] promised to send a follow-up email to all interested parties to try to restart this process.
+[[User:Adamwill|Adam Williamson]] mentioned that the beta release process was getting underway, and the test compose would be happening on 2009-09-23. He also reminded the group that a blocker bug review meeting would be taking place on 2009-09-25. He asked the group to test the test compose, Rawhide itself, and the nightly Rawhide live CDs as much as possible, and particularly to try and test beta blockers bugs that required re-testing.
-[[User:Wwoods|Will Woods]] reported on the progress of the AutoQA project. Failure reports sent to the mailing lists now include traceback information, where available. The installation tests now monitor both the serial console and the installer log files. Will is working on a TurboGears application to cover exporting the necessary information from the autoqa system to support the proposed israwhidebroken.com Rawhide status page. His current code is available in a git repository<ref>git://git.fedorapeople.org/~wwoods/israwhidebroken.git</ref>. He expects to be able to implement the page within a few weeks, and also to make a public instance of the autotest/autoqa systems available.
+[[User:dpravec|David Pravec]] said he had been talking to the Anaconda team and they were unhappy about significant changes being made to components on which Anaconda depends so close to the beta release time. The specific example raised was the major changes made to the nss package in the days before the meeting. The group discussed the problem, and agreed that there needed to be better communication between the Anaconda team and developers responsible for the components on which it relies. [[User:jkeating|Jesse Keating]] pointed out that such changes sometimes result from the Red Hat Enterprise Linux development process, with Red Hat staff being requested to make changes in Fedora for the ultimate benefit of that product. The group unanimously felt that there should be a process for such changes which ensures that they do not negatively affect the Fedora development process, and that Red Hat Enterprise Linux engineers should be required to be considerate of Fedora deadlines and processes when making changes to Fedora. Jesse said he should be able to raise this issue with the appropriate Red Hat management staff.
-[[User:Adamwill|Adam Williamson]] led a Test Day roundup. [[User:Sdz|Sebastian Dziallas]] reported on the Sugar on a Stick Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2009-09-03_SoaS</ref>. He was very happy with the participation and feedback received, and felt that the experimental use of the Semantic plugin for result reporting had been successful. He promised to follow up with Adam and [[User:Jlaska|James Laska]] on the use of Semantic. Adam reported on the Graphics Test Week which had happened the previous week, saying that turnout had been good and he was working on a round up. He also trailed the then-upcoming audio and virtualization Test Days. [[User:Markmc|Mark McLoughlin]] reported that preparations for the virtualization Test Day were advanced, but asked people to check over the Wiki page and suggest any improvements.
+[[User:Skvidal|Seth Vidal]] asked the group whether it would be acceptable to submit a new version of yum which includes history support to Rawhide prior to the release of Fedora 12 Beta. After some discussion, the group agreed that it would be an acceptable risk to make this change, as Seth had already performed some good testing on it and had a good plan for backing out the change if it became necessary.
-[[User:poelstra|John Poelstra]] reminded the group of the then-upcoming blocker bug review meeting for Fedora 12 Beta on Friday 2009-09-18.
+The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-09-22. The full log is available<ref>http://meetbot.fedoraproject.org/fedora-meeting/2009-09-22/fedora-meeting.2009-09-22-15.13.log.html</ref>. [[User:Tk009|Edward Kirk]] noted there had been no real movement on the triage day improvement topic, but he was hoping to pull something together soon.
-The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-09-15. The full log is available<ref>http://meetbot.fedoraproject.org/fedora-meeting/2009-09-15/fedora-meeting.2009-09-15-15.08.log.html</ref>. Brennan Ashton reported he was close to having the new version of the triage metrics system available, and hoped to have it available in a few days' time. He had looked for a co-maintainer for the project, but not found one yet.
+[[User:Tk009|Edward Kirk]] was also looking for a way to improve the tracking of action items from previous meetings, some of which he felt had fallen through the cracks in the past. After some discussion, [[User:John5342|John5342]] suggested using the team's trac instance<ref>http://fedorahosted.org/triage/</ref> for this purpose, and the group agreed this was a good idea. [[User:Tk009|Edward Kirk]] volunteered to transfer any outstanding action items from previous meetings to Trac.
-[[User:Tk009|Edward Kirk]] made some proposals to improve the Triage Day project. He suggested moving it to the weekend and making it bi-monthly. Niels Haase said he was interested in driving the changes and leading the events, as Edward would not be able to do this regularly. Niels suggested requiring new members attend a Triage Day before their membership was approved, but Edward felt this would not be a good idea. Ben Williams suggested creating a Fedora classroom for triaging, and the other group members felt this was a promising idea. [[User:Adamwill|Adam Williamson]] suggested Edward and Niels develop their thoughts and bring them forward on the mailing list or at the next week's meeting.
+No-one had heard from Brennan Ashton regarding the triage metrics project in the week before the meeting.
-Niels Haase asked a question about the Test Day live CD images, wondering if the default configuration could make it easier to access the system via ssh, to make it easier to access logs in the case of failures which make the system unusable directly. [[User:Adamwill|Adam Williamson]] explained that there were considerable security problems in doing this. Niels suggested making it optional via the use of a kernel parameter. Adam said that would involve development work, and suggested posting the suggestion to the development mailing list.
+[[User:Rjune|Richard June]] gave an update on the kernel triage project. He was still finding that the wireless tickets he was checking were all well-filed and would not much benefit from triage. He was therefore moving to look at a wider range of kernel bugs to see if this impression held true. He was also planning to get in touch with kernel developers to get some instruction on what kinds of information they would require on particular bugs.
-The next QA weekly meeting will be held on 2009-09-21 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-09-22 at 1500 UTC in #fedora-meeting.
+The next QA weekly meeting will be held on 2009-09-28 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-09-29 at 1500 UTC in #fedora-meeting.
 <references/>
-=== Pre-beta install testing ===
+=== Fedora 12 Beta Test compose ===
-[[User:Liam|Liam Li]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00298.html</ref> a pre-beta installation testing session, to assess the state of the installation process shortly before the process of building the Fedora 12 Beta begins. He explained that a test matrix was available on the Wiki<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_12_PreBeta_Install</ref>, and asked for group members' help in filling out the tests.
+[[User:Liam|Liam Li]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00560.html</ref> the Fedora 12 Beta test compose, a test build to check for major showstopper bugs that would appear only in the traditional installer physical media builds. Earlier, he had sent a mail detailing the types of testing<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00446.html</ref> that would be useful, and the results matrix page<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_12_Beta_TC_Install</ref> for reporting results. Several people reported failures with the test compose images, and Liam later announced an updated test compose<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00656.html</ref> which fixed the most significant bug affecting the earlier compose.
 <references/>
-=== Intel graphics information request ===
+=== Bug resolution discussion ===
-[[User:ajax|Adam Jackson]] asked<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00378.html</ref> users of Intel graphics adapters to send him some information to help with reliable display detection. He asked for a dump of the system video BIOS, together with information on the connectors available on the hardware in question. All Intel graphics adapter owners are encouraged to read Adam's request and send him the information from their systems - it is very easy and takes only a minute.
+[[User:Rudchenkos|Sergey Rudchenko]] asked<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00549.html</ref> whether triagers could ever close a bug as NEXTRELEASE without reference to the maintainer responsible for the bug. [[User:Adamwill|Adam Williamson]] explained<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00552.html</ref> that this was rarely the case, as it would require certain knowledge of the maintainers intentions regarding which releases they intended to fix the bug in.[[User:kvolny|Karel Volny]] asked<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00563.html</ref> why some of what Adam had said did not seem to match the policy documented on the Bugzilla page<ref>http://bugzilla.redhat.com/page.cgi?id=fields.html#status</ref>. Adam explained<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00565.html</ref> that the canonical reference for Fedora's bug process is in fact on the Wiki<ref>http://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow</ref>, and explained some wrinkles around distribution versioning, which he promised to explain on the Wiki page.
-<references/>
-=== Graphics test week recap ===
-[[User:Adamwill|Adam Williamson]] reported<ref>http://www.redhat.com/archives/fedora-test-list/2009-September/msg00388.html</ref> on the Graphics Test Week, providing a summary of all the bugs reported across the three Test Days, and thanking the developers and testers who showed up to help with testing.
 <references/>

Search