From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 221 qa beat)
(create fwn 222 qa beat)
Line 10: Line 10:
=== Test Days ===
=== Test Days ===


Last week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-08_Virtualization</ref> was on virtualization<ref>http://fedoraproject.org/wiki/Virtualization</ref>. This was mainly focused on the Fedora virtualization stack, based around KVM, libvirt, and virt-manager. A small band of hardened virtualization testers were able to expose 14 bugs, which the developers are now investigating. Thanks to everyone who came out to help with the testing.
Last week saw Graphics Test Week, with NVIDIA Test Day on Tuesday 2010-04-13<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-13_Nouveau</ref>, ATI/AMD Test Day on Wednesday 2010-04-14<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-14_Radeon</ref>, and Intel graphics Test Day on Thursday 2010-04-15<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-15_Intel</ref>. We had a great turnout again, with 164 total adapters tested by slightly fewer testers (extra thanks to those diligent souls who tested multiple systems!) and great support from the Fedora X.org developers and triagers. [[User:Adamwill|Adam Williamson]] provided a recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090271.html</ref> of the week, with some statistics on the numbers of bugs filed, and on the numbers of bugs from previous graphics Test Days that were fixed.


This week is a big moment in the Test Day schedule: Graphics Test Week. There will be three Test Days focusing on the three major graphics drivers: NVIDIA Test Day on Tuesday 2010-04-13<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-13_Nouveau</ref>, ATI/AMD Test Day on Wednesday 2010-04-14<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-14_Radeon</ref>, and Intel graphics Test Day on Thursday 2010-04-15<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-15_Intel</ref>. As always, widespread graphics testing is critical to the development of these drivers. Around 75% of all bugs reported in the last Graphics Test Week have been closed (either as fixed, or as duplicates), so the information gathered isn't ignored! Testing can be done with a live image, so there's no need to have an unstable Fedora installation to do the testing, and the tests are easy to do and come with full instructions. Almost everyone has an NVIDIA, AMD/ATI or Intel graphics adapter, so please come out to help us test! The events will take place all day in the #fedora-test-day channel on Freenode IRC (if you're not sure how to use IRC, there's an instruction page<ref>http://fedoraproject.org/wiki/How_to_use_IRC</ref>, or you can use WebIRC<ref>http://webchat.freenode.net/?channels=fedora-test-day</ref>. If you can't make it on the day, you can still provide your results on the Wiki page before or after the event.
This week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-04-22_StorageFiltering</ref> will be on Anaconda (the Fedora installer)'s storage support<ref>http://fedoraproject.org/wiki/Anaconda/Features/StorageFiltering</ref>: we will aim to test all the various exotic storage device options Anaconda makes available, including various types of RAID array, iSCSI (with iBFT), FCoE (if we can find someone with the hardware - please do come along if you have it!) and multipath devices. The broader the base of devices we can test the better, so please do come along and help if you can, particularly if you have, say, a motherboard that supports BIOS RAID and a couple of hard disks you can use temporarily. Unlike normal Test Days, it's impractical to do this testing with a live image, but there is some testing that can be done in a virtual machine. The Test Day will take place all day on Thursday 2010-04-22 in the #fedora-test-day channel on Freenode IRC (if you're not sure how to use IRC, there's an instruction page<ref>http://fedoraproject.org/wiki/How_to_use_IRC</ref>, or you can use WebIRC<ref>http://webchat.freenode.net/?channels=fedora-test-day</ref>). If you can't make it on the day, you can still provide your results on the Wiki page before or after the event.


If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
Line 18: Line 18:
<references/>
<references/>


=== Fedora 13 testing ===
=== Update acceptance testing ===


This week saw the group wrap up Fedora 13 Beta validation testing. After the previous week's delay, the fourth<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/089919.html</ref> and fifth<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/089962.html</ref> release candidate builds for the Beta arrived during the week. Installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Beta_RC4_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Beta_RC4_Desktop</ref> validation testing for the RC4 build were both broadly successful, but [[User:Adamwill|Adam Williamson]] realized that the build included a critical bug which would cause systems containing a certain common network adapter to be unable to boot<ref>http://bugzilla.redhat.com/show_bug.cgi?id=577463</ref>, so the RC5 build provided an updated kernel to fix that issue. Adam posted a call for testing of the updated kernel<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/089944.html</ref> which drew an overwhelming response, with dozens of group members confirming the kernel worked on their systems. The group re-ran the validation tests<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Beta_RC5_Install</ref>, and subsequently agreed with the development and release engineering groups at the go/no-go meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-04-08/f-13-beta-eng-readiness.2010-04-08-00.01.html</ref> that the RC5 build met all the release criteria<ref>http://fedoraproject.org/wiki/Fedora_13_Beta_Release_Criteria</ref> and so was suitable for release as Fedora 13 Beta. [[User:Rhe|Rui He]] summarized the validation test results<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090015.html</ref> and encouraged more group members to be involved in the validation testing for future releases.
[[User:Adamwill|Adam Williamson]] and [[User:Jlaska|James Laska]] got together with Bodhi developers [[User:Lmacken|Luke Macken]] and [[User:Bochecha|Mathieu Bridon]] to find out about their plans for implementing different feedback types in Bodhi, following the proposals by Doug Ledford<ref>http://lists.fedoraproject.org/pipermail/devel/2010-March/131799.html</ref> and Adam<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089690.html</ref>. Luke and Mathieu indicated that this work was occurring in the tg2 (TurboGears 2) branch of Bodhi, which they plan to put into production in the Fedora 14 timeframe. [[User:maxamillion|Adam Miller]] continued to revise the draft Proven Testers policy<ref>http://fedoraproject.org/wiki/QA/JoinProvenTesters:Draft</ref> based on the group's feedback.


<references/>
<references/>


=== Testing non-English keyboard layouts ===
=== Kernel triage ===


Petri Laine reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/089995.html</ref> that he had experienced problems using a non-default keyboard mapping in Fedora 13 Beta RC5. [[User:Adamwill|Adam Williamson]] replied<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/089998.html</ref> that similar bugs had occurred during previous release periods, and then announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090001.html</ref> that he had extended an installation validation test case<ref>http://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28encrypted%29_install</ref> and created a desktop validation test case<ref>http://fedoraproject.org/wiki/QA:Testcase_desktop_login</ref> to try to ensure that similar issues are caught in future testing rounds. Petri appended his report to an existing bug report<ref>http://bugzilla.redhat.com/show_bug.cgi?id=571900</ref> and followed up on the problem there.
At the weekly Bugzappers meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-04-13/fedora-meeting.2010-04-13-15.01.log.html</ref>, [[User:Kevin|Kevin Fenzi]] reported that he had begun to investigate kernel triage, an area [[User:Rjune|Richard June]] had previously been looking into but had been lacking free time. Kevin had begun to contact kernel team members and consider an overall strategy for approaching kernel triage, and asked other interested group members to join him.


<references/>
<references/>


=== Ensuring packages are signed ===
=== Fedora 13 Beta Delta ISOs ===


[[User:Jlaska|James Laska]] proposed<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090025.html</ref> a new release criterion and validation test to ensure that all packages are signed with a valid Feora GPG signature. [[User:Notting|Bill Nottingham]] pointed out<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090026.html</ref> that this would not slot easily into the existing package release workflow. He also noted that Bodhi is supposed to reject un-signed packages. [[User:jkeating|Jesse Keating]] explained<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090028.html</ref> that this was a mash configuration option which had been disabled intentionally for initial Branched composes as some packages were known not to be signed at that time. Bill ultimately suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090032.html</ref> that the signature check should be re-enabled in the relevant mash configurations.
Andre Robatino announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090082.html</ref> the availability of Delta ISOs for Fedora 13 Beta. As a quick reminder, Delta ISOs include just the difference between two ISO images, allowing you to reconstruct one image from the other and the Delta ISO, making it much faster to download a new ISO if you have a similar previous ISO. Andre provided deltas from Fedora 12 to 13 Beta (around 40% of the size of the full F13 Beta images), and from 13 Alpha to 13 Beta (around 10% of the size of the full images).


<references/>
<references/>


=== Bugzappers screencasts ===
=== Fedora 13 testing ===


At the weekly Bugzappers meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-04-06/fedora-meeting.2010-04-06-15.00.log.html</ref>, [[User:Adamwill|Adam Williamson]] noted that he had not yet forwarded [[User:Shakthimaan|Shakthi Kannan's]] suggestion of making Bugzapping screencasts to the mailing list. The next day, he did so<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/089972.html</ref>. [[User:etank|Eric Lake]], [[User:campbecg|Chris Campbell]] and James Gledhill all posted in support of the idea, but no-one yet had the combination of free time and expertise to make the screencasts.
Planned Fedora 13 testing was much quieter this week with the successful release of the Beta, but we did see the first final blocker review meeting<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2010-04-16/f-13-blocker-bug-review.2010-04-16-15.12.log.html</ref>, which was expertly summarized<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090224.html</ref> by [[User:Jlaska|James Laska]]. All outstanding blocker bugs for Fedora 13 were reviewed and assigned for action by testers or the development team.  


<references/>
Several group members were engaged in testing the final Beta release. [[TomHorsley|Tom Horsley]] reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090186.html</ref> a README file was present on the DVD image which discussed the boot.iso image, which is no longer included in the DVD. [[User:Sundaram|Rahul Sundaram]] suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090202.html</ref> he file a bug report. Tom also noticed<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090198.html</ref> a large amount of debugging messages from GDM in his system logs; Al Dunsmuir also observed this<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090239.html</ref>. [[User:Adamwill|Adam Williamson]] thought<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090254.html</ref> this was due to debugging statements that were temporarily enabled in plymouth to track down a bug, but [[User:rstrode|Ray Strode]] later mentioned in IRC conversation that it was simply because the current gdm package is a development release. Tom wasn't done yet; he also reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090199.html</ref> results from testing three ATI video cards, complete with bug reports.


=== AutoQA ===
Wolfgang Rupprecht reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090097.html</ref> that upgrading to Fedora 13 Beta using preupgrade had failed for him; Birger also had problems<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090215.html</ref> despite definitely having a large enough /boot partition. [[User:Kparal|Kamil Paral]] suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090222.html</ref> waiting for preupgrade 1.1.5 and trying again, as it has fixes for several significant bugs.


With Fedora 13 validation testing winding down, work on AutoQA was picking up steam again, with the team working on dependency checking<ref>http://fedorahosted.org/autoqa/milestone/autoqa depcheck</ref>, tests<ref>http://fedorahosted.org/autoqa/milestone/Package%20Sanity%20Tests</ref> to implement the Package Sanity Test Plan<ref>http://fedoraproject.org/wiki/QA:Package_Sanity_Test_Plan</ref>, the results database idea<ref>http://fedorahosted.org/autoqa/milestone/Resultdb</ref> and the automated installation test plan<ref>http://fedorahosted.org/autoqa/milestone/Automate%20installation%20test%20plan</ref>.
Tommy He tried out the new backup tool Deja Dup and found it lacking<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090156.html</ref>: it could not restore a backup it had created. [[User:Sundaram|Rahul Sundaram]] swung into action and made sure the upstream authors were aware of the bug, reporting back<ref>http://lists.fedoraproject.org/pipermail/test/2010-April/090168.html</ref> that they would look into it over the coming weekend.


<references/>
<references/>

Revision as of 07:08, 21 April 2010

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

Test Days

Last week saw Graphics Test Week, with NVIDIA Test Day on Tuesday 2010-04-13[1], ATI/AMD Test Day on Wednesday 2010-04-14[2], and Intel graphics Test Day on Thursday 2010-04-15[3]. We had a great turnout again, with 164 total adapters tested by slightly fewer testers (extra thanks to those diligent souls who tested multiple systems!) and great support from the Fedora X.org developers and triagers. Adam Williamson provided a recap[4] of the week, with some statistics on the numbers of bugs filed, and on the numbers of bugs from previous graphics Test Days that were fixed.

This week's Test Day[5] will be on Anaconda (the Fedora installer)'s storage support[6]: we will aim to test all the various exotic storage device options Anaconda makes available, including various types of RAID array, iSCSI (with iBFT), FCoE (if we can find someone with the hardware - please do come along if you have it!) and multipath devices. The broader the base of devices we can test the better, so please do come along and help if you can, particularly if you have, say, a motherboard that supports BIOS RAID and a couple of hard disks you can use temporarily. Unlike normal Test Days, it's impractical to do this testing with a live image, but there is some testing that can be done in a virtual machine. The Test Day will take place all day on Thursday 2010-04-22 in the #fedora-test-day channel on Freenode IRC (if you're not sure how to use IRC, there's an instruction page[7], or you can use WebIRC[8]). If you can't make it on the day, you can still provide your results on the Wiki page before or after the event.

If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[9].

  1. ↑ http://fedoraproject.org/wiki/Test_Day:2010-04-13_Nouveau
  2. ↑ http://fedoraproject.org/wiki/Test_Day:2010-04-14_Radeon
  3. ↑ http://fedoraproject.org/wiki/Test_Day:2010-04-15_Intel
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090271.html
  5. ↑ http://fedoraproject.org/wiki/Test_Day:2010-04-22_StorageFiltering
  6. ↑ http://fedoraproject.org/wiki/Anaconda/Features/StorageFiltering
  7. ↑ http://fedoraproject.org/wiki/How_to_use_IRC
  8. ↑ http://webchat.freenode.net/?channels=fedora-test-day
  9. ↑ http://fedorahosted.org/fedora-qa/

Update acceptance testing

Adam Williamson and James Laska got together with Bodhi developers Luke Macken and Mathieu Bridon to find out about their plans for implementing different feedback types in Bodhi, following the proposals by Doug Ledford[1] and Adam[2]. Luke and Mathieu indicated that this work was occurring in the tg2 (TurboGears 2) branch of Bodhi, which they plan to put into production in the Fedora 14 timeframe. Adam Miller continued to revise the draft Proven Testers policy[3] based on the group's feedback.

  1. ↑ http://lists.fedoraproject.org/pipermail/devel/2010-March/131799.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089690.html
  3. ↑ http://fedoraproject.org/wiki/QA/JoinProvenTesters:Draft

Kernel triage

At the weekly Bugzappers meeting[1], Kevin Fenzi reported that he had begun to investigate kernel triage, an area Richard June had previously been looking into but had been lacking free time. Kevin had begun to contact kernel team members and consider an overall strategy for approaching kernel triage, and asked other interested group members to join him.

  1. ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-04-13/fedora-meeting.2010-04-13-15.01.log.html

Fedora 13 Beta Delta ISOs

Andre Robatino announced[1] the availability of Delta ISOs for Fedora 13 Beta. As a quick reminder, Delta ISOs include just the difference between two ISO images, allowing you to reconstruct one image from the other and the Delta ISO, making it much faster to download a new ISO if you have a similar previous ISO. Andre provided deltas from Fedora 12 to 13 Beta (around 40% of the size of the full F13 Beta images), and from 13 Alpha to 13 Beta (around 10% of the size of the full images).

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090082.html

Fedora 13 testing

Planned Fedora 13 testing was much quieter this week with the successful release of the Beta, but we did see the first final blocker review meeting[1], which was expertly summarized[2] by James Laska. All outstanding blocker bugs for Fedora 13 were reviewed and assigned for action by testers or the development team.

Several group members were engaged in testing the final Beta release. Tom Horsley reported[3] a README file was present on the DVD image which discussed the boot.iso image, which is no longer included in the DVD. Rahul Sundaram suggested[4] he file a bug report. Tom also noticed[5] a large amount of debugging messages from GDM in his system logs; Al Dunsmuir also observed this[6]. Adam Williamson thought[7] this was due to debugging statements that were temporarily enabled in plymouth to track down a bug, but Ray Strode later mentioned in IRC conversation that it was simply because the current gdm package is a development release. Tom wasn't done yet; he also reported[8] results from testing three ATI video cards, complete with bug reports.

Wolfgang Rupprecht reported[9] that upgrading to Fedora 13 Beta using preupgrade had failed for him; Birger also had problems[10] despite definitely having a large enough /boot partition. Kamil Paral suggested[11] waiting for preupgrade 1.1.5 and trying again, as it has fixes for several significant bugs.

Tommy He tried out the new backup tool Deja Dup and found it lacking[12]: it could not restore a backup it had created. Rahul Sundaram swung into action and made sure the upstream authors were aware of the bug, reporting back[13] that they would look into it over the coming weekend.

  1. ↑ http://meetbot.fedoraproject.org/fedora-bugzappers/2010-04-16/f-13-blocker-bug-review.2010-04-16-15.12.log.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090224.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090186.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090202.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090198.html
  6. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090239.html
  7. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090254.html
  8. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090199.html
  9. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090097.html
  10. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090215.html
  11. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090222.html
  12. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090156.html
  13. ↑ http://lists.fedoraproject.org/pipermail/test/2010-April/090168.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=168055"