From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 235 qa beat)
(create fwn 236 qa beat)
Line 8: Line 8:
<references/>
<references/>


=== Instructions and process for moving bug reports upstream ===
=== QA metrics ===


[[User:Ankursinha|Ankur Sinha]] began a discussion<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092018.html</ref>, <ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092036.html</ref> on providing instructions for users on the Wiki for reporting bugs to upstream bug trackers. [[User:Johannbg|Jóhann Guðmundsson]] felt that this was the wrong approach<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092040.html</ref> and that it should be the responsibility of package maintainers to submit reports upstream when appropriate. In general, though, the group felt that providing instructions for users would be helpful in at least some cases, and Ankur said he would work on a draft of such a page<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092091.html</ref>.
[[User:Kparal|Kamil Paral]] asked for feedback<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092196.html</ref> on the topic of gathering statistics on QA activities, as part of a Fedora-wide 'Fedora Hall Of Fame' project<ref>http://fedoraproject.org/wiki/User:Kparal/Idea:Fedora_Hall_of_Fame</ref>. He asked what would be the most important tools to monitor and the most important characteristics to gather. [[User:Johannbg|Jóhann Guðmundsson]] pointed out some of the pitfalls that can come with trying to measure and reward voluntary contributions<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092198.html</ref>. [[User:Jlaska|James Laska]] pointed out that gathering statistics would be valuable in ways beyond trying to reward contributors<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092200.html</ref>. Jóhann clarified that he was only concerned with the 'Hall of Fame' concept and agreed that statistics could be useful in monitoring and improving overall team performance<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092205.html</ref>. Kamil thought that in practice the negative effects Jóhann feared would not be so pronounced<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092214.html</ref>. Jóhann followed up with some concrete suggestions of areas and numbers that would be interesting<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092215.html</ref>.  


<references/>
<references/>


=== Rawhide acceptance testing ===
=== Localization testing ===


[[User:Jlaska|James Laska]] reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092095.html</ref> on the first automated Rawhide acceptance test plan<ref>http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan</ref> run for Fedora 14, and linked to the full results<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Pre-Alpha_Rawhide_Acceptance_Test_1</ref>. The run exposed four bugs, which were significant enough that the test images could not be declared 'last known good'.
Following on from last week's coverage of [[User:Igor|Igor Pires Soares]]'s localization testing discussion<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092029.html</ref>, [[User:Noriko|Noriko Mizumoto]] suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092193.html</ref> using the newly open-sourced Nitrate tool<ref>http://fedorahosted.org/nitrate/</ref> for tracking the tests. Igor asked<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092201.html</ref> if the tool was yet packaged for Fedora and deployed in Fedora Infrastructure. [[User:Jlaska|James Laska]] reported<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092203.html</ref> that it was not, but said he thought it would be a good candidate to be included. Later, Igor announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092219.html</ref> that he had made an initial set of updates to the template<ref>http://fedoraproject.org/wiki/QA:Fedora_14_l10n_Results_Template</ref> for Fedora 14.


<references/>
<references/>


=== Proven testers ===
=== Boot menu release criterion proposal ===


[[User:Adamwill|Adam Williamson]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092075.html</ref> that he had updated the Wiki proven testers material, making the main page<ref>http://fedoraproject.org/wiki/Proven_tester</ref> cover all aspects of the project (based on his previous draft mentioned in last week's issue) and turning the JoinProvenTesters page into a redirect to it. Later in the week he updated the instructions again<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092101.html</ref>.
After a contentious first Fedora 14 blocker bug review meeting<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092147.html</ref>, [[User:Adamwill|Adam Williamson]] proposed two alternative ways to cover boot menu functionality in the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092148.html</ref>. He suggested either having a basic test of essential functionality (the installer eventually boots without manual interaction) at the Alpha stage and a more advanced test (the graphical boot menu appears as intended) at Beta or Final stage, or simply having the more advanced test at Alpha stage. [[User:Jlaska|James Laska]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092152.html</ref> and [[User:Jkeating|Jesse Keating]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092150.html</ref> both came down in favour of simply requiring the menu to work correctly at Alpha stage.
 
<references/>
 
=== AutoQA package acceptance testing ===
 
During the QA weekly meeting of 2010-07-12<ref>http://fedoraproject.org/wiki/QA/Meetings/20100712</ref>, [[User:Wwoods|Will Woods]] explained that the AutoQA team had decided to consider automating the package update acceptance test plan<ref>http://fedoraproject.org/wiki/QA:Package_Update_Acceptance_Test_Plan</ref> as its highest priority. They felt this would provide a very visible and useful example of the potential of AutoQA as soon as possible. Automated package acceptance testing would prevent packages with certain types of serious problems, such as broken dependencies, from being accepted as updates. Will had identified clear areas of focus which are necessary to implement automated package acceptance testing, and these can be tracked on the AutoQA roadmap<ref>http://fedorahosted.org/autoqa/roadmap</ref>.


<references/>
<references/>


=== Localization testing ===
=== Bootloader menu release criterion ===


[[User:Igor|Igor Pires Soares]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092029.html</ref> that he would be working to update the Fedora 13 localization results template<ref>http://fedoraproject.org/wiki/QA:Fedora_13_l10n_Results_Template</ref> for Fedora 14 use at FUDCon Santiago, and asked for feedback and ideas on improving it. [[User:Jlaska|James Laska]] suggested<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092094.html</ref> a page to explain the general testing procedure, and asked where test results are recorded. Igor accepted the general procedure idea, and explained that the template was intended to allow local teams to record results in whatever way they felt most appropriate<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092096.html</ref>.
[[User:Adamwill|Adam Williamson]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092231.html</ref> that, following the list discussion, he had added the simpler version of the proposed release criterion mentioned in last week's issue to the Fedora 14 Alpha release criteria<ref>http://fedoraproject.org/wiki/Fedora_14_Alpha_Release_Criteria#Alpha_Release_Requirements</ref>. [[User:Rhe|Rui He]] added a validation test to check this<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092248.html</ref>.


<references/>
<references/>


=== Boot menu release criterion proposal ===
=== Lessons learned about update process ===


After a contentious first Fedora 14 blocker bug review meeting<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092147.html</ref>, [[User:Adamwill|Adam Williamson]] proposed two alternative ways to cover boot menu functionality in the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092148.html</ref>. He suggested either having a basic test of essential functionality (the installer eventually boots without manual interaction) at the Alpha stage and a more advanced test (the graphical boot menu appears as intended) at Beta or Final stage, or simply having the more advanced test at Alpha stage. [[User:Jlaska|James Laska]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092152.html</ref> and [[User:Jkeating|Jesse Keating]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092150.html</ref> both came down in favour of simply requiring the menu to work correctly at Alpha stage.
[[User:Kevin|Kevin Fenzi]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092236.html</ref> that he had created a page to track lessons learned from problems in the update process<ref>http://lists.fedoraproject.org/pipermail/test/2010-July/092236.html</ref>, and asked the group to document any serious issues they had experienced involving updates to stable releases so that FESCo could benefit from the information in planning changes to the update processes and tooling.


<references/>
<references/>

Revision as of 02:45, 29 July 2010

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

QA metrics

Kamil Paral asked for feedback[1] on the topic of gathering statistics on QA activities, as part of a Fedora-wide 'Fedora Hall Of Fame' project[2]. He asked what would be the most important tools to monitor and the most important characteristics to gather. Jóhann Guðmundsson pointed out some of the pitfalls that can come with trying to measure and reward voluntary contributions[3]. James Laska pointed out that gathering statistics would be valuable in ways beyond trying to reward contributors[4]. Jóhann clarified that he was only concerned with the 'Hall of Fame' concept and agreed that statistics could be useful in monitoring and improving overall team performance[5]. Kamil thought that in practice the negative effects Jóhann feared would not be so pronounced[6]. Jóhann followed up with some concrete suggestions of areas and numbers that would be interesting[7].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092196.html
  2. ↑ http://fedoraproject.org/wiki/User:Kparal/Idea:Fedora_Hall_of_Fame
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092198.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092200.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092205.html
  6. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092214.html
  7. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092215.html

Localization testing

Following on from last week's coverage of Igor Pires Soares's localization testing discussion[1], Noriko Mizumoto suggested[2] using the newly open-sourced Nitrate tool[3] for tracking the tests. Igor asked[4] if the tool was yet packaged for Fedora and deployed in Fedora Infrastructure. James Laska reported[5] that it was not, but said he thought it would be a good candidate to be included. Later, Igor announced[6] that he had made an initial set of updates to the template[7] for Fedora 14.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092029.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092193.html
  3. ↑ http://fedorahosted.org/nitrate/
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092201.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092203.html
  6. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092219.html
  7. ↑ http://fedoraproject.org/wiki/QA:Fedora_14_l10n_Results_Template

Boot menu release criterion proposal

After a contentious first Fedora 14 blocker bug review meeting[1], Adam Williamson proposed two alternative ways to cover boot menu functionality in the release criteria[2]. He suggested either having a basic test of essential functionality (the installer eventually boots without manual interaction) at the Alpha stage and a more advanced test (the graphical boot menu appears as intended) at Beta or Final stage, or simply having the more advanced test at Alpha stage. James Laska[3] and Jesse Keating[4] both came down in favour of simply requiring the menu to work correctly at Alpha stage.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092147.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092148.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092152.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092150.html

Bootloader menu release criterion

Adam Williamson announced[1] that, following the list discussion, he had added the simpler version of the proposed release criterion mentioned in last week's issue to the Fedora 14 Alpha release criteria[2]. Rui He added a validation test to check this[3].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092231.html
  2. ↑ http://fedoraproject.org/wiki/Fedora_14_Alpha_Release_Criteria#Alpha_Release_Requirements
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092248.html

Lessons learned about update process

Kevin Fenzi announced[1] that he had created a page to track lessons learned from problems in the update process[2], and asked the group to document any serious issues they had experienced involving updates to stable releases so that FESCo could benefit from the information in planning changes to the update processes and tooling.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092236.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-July/092236.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=189138"