FWN/Beats/QualityAssurance: Difference between revisions

Revision as of 18:04, 28 April 2010

QualityAssurance

In this section, we cover the activities of the QA team^[1]. For more information on the work of the QA team and how you can get involved, see the Joining page^[2].

Contributing Writer: Adam Williamson

Test Days

Last week's Test Day^[1] was on Anaconda (the Fedora installer)'s storage support^[2]. The turnout was unfortunately low, perhaps due to the hardware requirements for testing, but nevertheless the few testers present managed to expose four bugs.

This week will see the final two Test Days of the Fedora 13 cycle. First up is Preupgrade Test Day^[3] on Thursday 2010-04-29, followed by Xfce Test Day^[4] on Friday 2010-04-30.

These are two juicy topics: preupgrade^[5] is the recommended method for upgrading from one Fedora release to the next, and Xfce is one of the most popular 'alternate' Linux desktops, and has a very dedicated Fedora SIG^[6] which works hard to provide a smooth experience and live spin, and has organized the Test Day to make sure the Fedora 13 Xfce experience is second to none.

As always, the Test Days will run all day in the #fedora-test-day channel on Freenode IRC. If you're not sure about IRC, read the guide or use WebIRC^[7]. If you can't do the testing on the Test Day, you can still run through the tests and provide your results earlier or later. You can do the Xfce testing with a live image which will be provided on the Test Day page. Obviously this isn't possible for the preupgrade testing, but you can test it in a virtual machine if you don't want to (or can't) mess with your real Fedora installation.

If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac^[8].

Update acceptance testing

At the weekly QA meeting^[1], Adam Miller asked if a vote would be needed to move ahead with the proventesters policy he was drafting. Adam Williamson said votes were not normally needed and suggested just sending a final draft to the mailing list with a note that it would go into effect if there were no major objections. Adam Miller subsequently sent the final draft to the list^[2], where Adam Williamson^[3], Shmuel Siegel^[4] and others suggested small amendments, which Adam Miller accepted^[5].

Kamil Paral reported that the package update acceptance test plan^[6] could be completed before the end of the Fedora 13 cycle, for automation and implementation in the Fedora 14 cycle. Kamil later posted a final call for comments^[7] on the plan, to which James Laska^[8] and Jesse Keating^[9] provided detailed responses.

Remote accessibility release criterion proposal

Adam Williamson proposed^[1] a new release criterion stating that it must be possible to install a system in such a way that it is immediately remotely accessible. This was in response to reports on the list that it was not possible to do this in Fedora 13, in contrast to previous releases, which was causing sysadmins dealing with remote machines some problems.

↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090255.html

Fedora 11 end-of-life notification

John Poelstra announced^[1] that he would soon send out the end-of-life notification for Fedora 11 on bugs reported against that release. Jason Tibbitts noted^[2] that he had adjusted open package review requests filed against that release, so these would not erroneously receive notifications.

Fedora 13 testing

This week saw the second blocker bug review meeting^[1] for the final release. The group reviewed all open blocker bugs with assistance from the development team; in general, progress is being made on all blockers and we don't foresee any major problems resolving them in time for release.

Al Dunsmuir reported^[2] a crash in Firefox with the common Noscript extension when browsing the release notes, which he filed as a bug^[3].

Joachim Backes reported^[4] a problem booting to runlevel 3 with the echoing of login information: he had seen cases where the username was not echoed back to the screen when typed, but the password was (obviously, the behavior should be opposite). Steven I Usdansky confirmed^[5] that he had seen a similar issue.

[1] ttp://fedoraproject.org/wiki/QA

[2] ttp://fedoraproject.org/wiki/QA/Join

[3] ttp://fedoraproject.org/wiki/Test_Day:2010-04-22_StorageFiltering

[4] ttp://fedoraproject.org/wiki/Anaconda/Features/StorageFiltering

[5] ttp://fedoraproject.org/wiki/Test_Day:2010-04-29_Preupgrade

[6] ttp://fedoraproject.org/wiki/Test_Day:XFCE_F13_20100430

[7] ttp://fedoraproject.org/wiki/How_to_use_PreUpgrade

[8] ttp://fedoraproject.org/wiki/Xfce

[9] ttp://webchat.freenode.net/?channels=fedora-test-day

[10] ttp://fedorahosted.org/fedora-qa/

[11] ttp://fedoraproject.org/wiki/QA/Meetings/20100419

[12] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090252.html

[13] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090257.html

[14] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090260.html

[15] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090288.html

[16] ttp://fedoraproject.org/wiki/QA:Package_Update_Acceptance_Test_Plan

[17] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090287.html

[18] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090313.html

[19] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090317.html

[20] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090255.html

[21] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090326.html

[22] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090328.html

[23] ttp://meetbot.fedoraproject.org/fedora-bugzappers/2010-04-23/fedora-bugzappers.2010-04-23-16.06.log.html

[24] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090238.html

[25] ttp://bugzilla.redhat.com/show_bug.cgi?id=583826

[26] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090330.html

[27] ttp://lists.fedoraproject.org/pipermail/test/2010-April/090335.html

[1]

[2]

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[1]

[1]

[2]

[1]

[2]

[3]

[4]

[5]

@@ Line 10: / Line 10: @@
 === Test Days ===
-Last week saw Graphics Test Week, with NVIDIA Test Day on Tuesday 2010-04-13<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-13_Nouveau</ref>, ATI/AMD Test Day on Wednesday 2010-04-14<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-14_Radeon</ref>, and Intel graphics Test Day on Thursday 2010-04-15<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-15_Intel</ref>. We had a great turnout again, with 164 total adapters tested by slightly fewer testers (extra thanks to those diligent souls who tested multiple systems!) and great support from the Fedora X.org developers and triagers. [[User:Adamwill|Adam Williamson]] provided a recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090271.html</ref> of the week, with some statistics on the numbers of bugs filed, and on the numbers of bugs from previous graphics Test Days that were fixed.
+Last week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-22_StorageFiltering</ref> was on Anaconda (the Fedora installer)'s storage support<ref>http://fedoraproject.org/wiki/Anaconda/Features/StorageFiltering</ref>. The turnout was unfortunately low, perhaps due to the hardware requirements for testing, but nevertheless the few testers present managed to expose four bugs.
-This week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-22_StorageFiltering</ref> will be on Anaconda (the Fedora installer)'s storage support<ref>http://fedoraproject.org/wiki/Anaconda/Features/StorageFiltering</ref>: we will aim to test all the various exotic storage device options Anaconda makes available, including various types of RAID array, iSCSI (with iBFT), FCoE (if we can find someone with the hardware - please do come along if you have it!) and multipath devices. The broader the base of devices we can test the better, so please do come along and help if you can, particularly if you have, say, a motherboard that supports BIOS RAID and a couple of hard disks you can use temporarily. Unlike normal Test Days, it's impractical to do this testing with a live image, but there is some testing that can be done in a virtual machine. The Test Day will take place all day on Thursday 2010-04-22 in the #fedora-test-day channel on Freenode IRC (if you're not sure how to use IRC, there's an instruction page<ref>http://fedoraproject.org/wiki/How_to_use_IRC</ref>, or you can use WebIRC<ref>http://webchat.freenode.net/?channels=fedora-test-day</ref>). If you can't make it on the day, you can still provide your results on the Wiki page before or after the event.
+This week will see the final two Test Days of the Fedora 13 cycle. First up is Preupgrade Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-29_Preupgrade</ref> on Thursday 2010-04-29, followed by Xfce Test Day<ref>http://fedoraproject.org/wiki/Test_Day:XFCE_F13_20100430</ref> on Friday 2010-04-30.
+These are two juicy topics: preupgrade<ref>http://fedoraproject.org/wiki/How_to_use_PreUpgrade</ref> is the recommended method for upgrading from one Fedora release to the next, and Xfce is one of the most popular 'alternate' Linux desktops, and has a very dedicated Fedora SIG<ref>http://fedoraproject.org/wiki/Xfce</ref> which works hard to provide a smooth experience and live spin, and has organized the Test Day to make sure the Fedora 13 Xfce experience is second to none.
+As always, the Test Days will run all day in the #fedora-test-day channel on Freenode IRC. If you're not sure about IRC, read the guide or use WebIRC<ref>http://webchat.freenode.net/?channels=fedora-test-day</ref>. If you can't do the testing on the Test Day, you can still run through the tests and provide your results earlier or later. You can do the Xfce testing with a live image which will be provided on the Test Day page. Obviously this isn't possible for the preupgrade testing, but you can test it in a virtual machine if you don't want to (or can't) mess with your real Fedora installation.
 If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
@@ Line 20: / Line 24: @@
 === Update acceptance testing ===
-[[User:Adamwill|Adam Williamson]] and [[User:Jlaska|James Laska]] got together with Bodhi developers [[User:Lmacken|Luke Macken]] and [[User:Bochecha|Mathieu Bridon]] to find out about their plans for implementing different feedback types in Bodhi, following the proposals by Doug Ledford<ref>http://lists.fedoraproject.org/pipermail/devel/2010-March/131799.html</ref> and Adam<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089690.html</ref>. Luke and Mathieu indicated that this work was occurring in the tg2 (TurboGears 2) branch of Bodhi, which they plan to put into production in the Fedora 14 timeframe. [[User:maxamillion|Adam Miller]] continued to revise the draft Proven Testers policy<ref>http://fedoraproject.org/wiki/QA/JoinProvenTesters:Draft</ref> based on the group's feedback.
+At the weekly QA meeting<ref>http://fedoraproject.org/wiki/QA/Meetings/20100419</ref>, [[User:maxamillion|Adam Miller]] asked if a vote would be needed to move ahead with the proventesters policy he was drafting. [[User:Adamwill|Adam Williamson]] said votes were not normally needed and suggested just sending a final draft to the mailing list with a note that it would go into effect if there were no major objections. Adam Miller subsequently sent the final draft to the list<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090252.html</ref>, where Adam Williamson<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090257.html</ref>, [[ShmuelSiegel|Shmuel Siegel]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090260.html</ref> and others suggested small amendments, which Adam Miller accepted<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090288.html</ref>.
+[[User:Kparal|Kamil Paral]] reported that the package update acceptance test plan<ref>http://fedoraproject.org/wiki/QA:Package_Update_Acceptance_Test_Plan</ref> could be completed before the end of the Fedora 13 cycle, for automation and implementation in the Fedora 14 cycle. Kamil later posted a final call for comments<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090287.html</ref> on the plan, to which [[User:Jlaska|James Laska]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090313.html</ref> and [[User:jkeating|Jesse Keating]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090317.html</ref> provided detailed responses.
 <references/>
-=== Kernel triage ===
+=== Remote accessibility release criterion proposal ===
-At the weekly Bugzappers meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-04-13/fedora-meeting.2010-04-13-15.01.log.html</ref>, [[User:Kevin|Kevin Fenzi]] reported that he had begun to investigate kernel triage, an area [[User:Rjune|Richard June]] had previously been looking into but had been lacking free time. Kevin had begun to contact kernel team members and consider an overall strategy for approaching kernel triage, and asked other interested group members to join him.
+[[User:Adamwill|Adam Williamson]] proposed<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090255.html</ref> a new release criterion stating that it must be possible to install a system in such a way that it is immediately remotely accessible. This was in response to reports on the list that it was not possible to do this in Fedora 13, in contrast to previous releases, which was causing sysadmins dealing with remote machines some problems.
 <references/>
-=== Fedora 13 Beta Delta ISOs ===
+=== Fedora 11 end-of-life notification ===
-Andre Robatino announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090082.html</ref> the availability of Delta ISOs for Fedora 13 Beta. As a quick reminder, Delta ISOs include just the difference between two ISO images, allowing you to reconstruct one image from the other and the Delta ISO, making it much faster to download a new ISO if you have a similar previous ISO. Andre provided deltas from Fedora 12 to 13 Beta (around 40% of the size of the full F13 Beta images), and from 13 Alpha to 13 Beta (around 10% of the size of the full images).
+[[User:Poelstra|John Poelstra]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090326.html</ref> that he would soon send out the end-of-life notification for Fedora 11 on bugs reported against that release. [[User:tibbs|Jason Tibbitts]] noted<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090328.html</ref> that he had adjusted open package review requests filed against that release, so these would not erroneously receive notifications.
 <references/>
@@ Line 38: / Line 44: @@
 === Fedora 13 testing ===
-Planned Fedora 13 testing was much quieter this week with the successful release of the Beta, but we did see the first final blocker review meeting<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2010-04-16/f-13-blocker-bug-review.2010-04-16-15.12.log.html</ref>, which was expertly summarized<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090224.html</ref> by [[User:Jlaska|James Laska]]. All outstanding blocker bugs for Fedora 13 were reviewed and assigned for action by testers or the development team.
+This week saw the second blocker bug review meeting<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2010-04-23/fedora-bugzappers.2010-04-23-16.06.log.html</ref> for the final release. The group reviewed all open blocker bugs with assistance from the development team; in general, progress is being made on all blockers and we don't foresee any major problems resolving them in time for release.
-Several group members were engaged in testing the final Beta release. [[TomHorsley|Tom Horsley]] reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090186.html</ref> a README file was present on the DVD image which discussed the boot.iso image, which is no longer included in the DVD. [[User:Sundaram|Rahul Sundaram]] suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090202.html</ref> he file a bug report. Tom also noticed<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090198.html</ref> a large amount of debugging messages from GDM in his system logs; Al Dunsmuir also observed this<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090239.html</ref>. [[User:Adamwill|Adam Williamson]] thought<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090254.html</ref> this was due to debugging statements that were temporarily enabled in plymouth to track down a bug, but [[User:rstrode|Ray Strode]] later mentioned in IRC conversation that it was simply because the current gdm package is a development release. Tom wasn't done yet; he also reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090199.html</ref> results from testing three ATI video cards, complete with bug reports.
-Wolfgang Rupprecht reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090097.html</ref> that upgrading to Fedora 13 Beta using preupgrade had failed for him; Birger also had problems<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090215.html</ref> despite definitely having a large enough /boot partition. [[User:Kparal|Kamil Paral]] suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090222.html</ref> waiting for preupgrade 1.1.5 and trying again, as it has fixes for several significant bugs.
+Al Dunsmuir reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090238.html</ref> a crash in Firefox with the common Noscript extension when browsing the release notes, which he filed as a bug<ref>http://bugzilla.redhat.com/show_bug.cgi?id=583826</ref>.
-Tommy He tried out the new backup tool Deja Dup and found it lacking<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090156.html</ref>: it could not restore a backup it had created. [[User:Sundaram|Rahul Sundaram]] swung into action and made sure the upstream authors were aware of the bug, reporting back<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090168.html</ref> that they would look into it over the coming weekend.
+Joachim Backes reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090330.html</ref> a problem booting to runlevel 3 with the echoing of login information: he had seen cases where the username was not echoed back to the screen when typed, but the password was (obviously, the behavior should be opposite). Steven I Usdansky confirmed<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090335.html</ref> that he had seen a similar issue.
 <references/>

Search