From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 276 qa beat)
(create fwn 279 qa beat)
Line 10: Line 10:
=== Test Days ===
=== Test Days ===


The Fedora 15 Test Day track is now finished, and the Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
A special Test Day<ref>http://fedoraproject.org/wiki/QA/Test_Day:2011-06-08_IPv6</ref> was held on Wednesday 2011-06-08 for World IPv6 Day<ref>http://www.worldipv6day.org/</ref>, thanks to Linda Wang. Despite the complexity involved in implementing an IPv6 setup for testing, a good group of testers were able to run through the various tests and identify some bugs which have already received developer attention.
 
The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.


<references/>
<references/>


=== Fedora 15 validation and preparation ===
=== Security spin testing ===


This was a quiet week after the declaration that Fedora 15 was gold on Tuesday, so the group worked on updating the Fedora 15 common bugs page<ref>http://fedoraproject.org/wiki/Common_F15_bugs</ref> and tried to help with getting the Sugar desktop into a releasable state<ref>http://bugzilla.redhat.com/show_bug.cgi?id=697649</ref>, and made sure 0-day updates for the release were being properly tested. [[User:Jlaska|James Laska]] worked on<ref>http://fedorahosted.org/fedora-qa/ticket/164#comment:7</ref> and announced<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100175.html</ref>providing a validation framework for the newly-introduced multi-desktop DVD live image<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_15_Final_Multi_Boot_DVD</ref>, and along with Andre Robatino and [[User:Cwickert|Christoph Wickert]], performed the required testing.
[[User:Athmane|Athmane Madjoudj]] announced some testing he had done on the Fedora 15 security spin<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100484.html</ref>. [[User:Adamwill|Adam Williamson]] thanked him and asked if he had contacted the security spin authors about it<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100485.html</ref>, and Athmane replied that he would. Later, Athmane announced that he had added some more tests<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100496.html</ref>, and Adam suggested writing them up as test cases<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100497.html</ref>. Athmane did this<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100511.html</ref>, and then continued to add more test cases regularly. In the course of this work, Athmane noticed and helped to address some problems with the rendering of tags and templates in the wiki<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100668.html</ref> <ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100763.html</ref>.


<references/>
<references/>
Line 22: Line 24:
=== Release criteria revisions ===
=== Release criteria revisions ===


[[User:Adamwill|Adam Williamson]] proposed some more release criteria changes. First up was logging<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100126.html</ref>. [[User:Jlaska|James Laska]] suggested a refinement<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100128.html</ref>, and Adam posted a revised proposal<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100131.html</ref>, which was met with general approval. Later, Adam announced that he had created the criteria pages for Fedora 16<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100183.html</ref>, and including the new logging criterion, along with some other criteria which had previously been agreed upon but not added to the Fedora 15 criteria. He also re-started the discussion of how to refer to desktops that are considered able to block the release as compared to those that are not, and suggested the term 'release-blocking desktops'. [[User:Johannbg|Jóhann Guðmundsson]] re-raised the question of which desktops should be considered to block the release<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100188.html</ref>, and Adam maintained that this was a question that was beyond the authority of the QA group to decide<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100189.html</ref>. Finally, Adam also proposed a criterion regarding security issues<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100237.html</ref> for discussion by the QA group along with the security and development groups.
[[User:Adamwill|Adam Williamson]] continued with revisions of the release criteria. He announced that the wording 'release-blocking desktops' to describe the desktops that are capable of blocking release had been generally well received, so he had updated the criteria to use this wording<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100646.html</ref>.
 
<references/>
 
=== Installer validation test revisions ===
 
[[User:Rhe|Rui He]] reported that she had refined the various installation source tests that form part of the installation validation matrix by creating some new tests and renaming and adjusting others<ref>http://fedorahosted.org/fedora-qa/ticket/173#comment:7</ref>. [[User:Jlaska|James Laska]] followed up with some questions and comments<ref>http://fedorahosted.org/fedora-qa/ticket/173#comment:8</ref>, and Rui He continued with further improvements.


<references/>
<references/>


=== Housekeeping tasks ===
=== Overly similar application names ===


[[User:Adamwill|Adam Williamson]] noted that there are several tasks nominally under the Bugzappers group's remit that had not been happening recently<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100150.html</ref>, and suggested running a meeting to ensure they would be looked after. [[User:Rbergero|Robyn Bergeron]] replied that several of the tasks were really her responsibility as program manager<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100165.html</ref>, but agreed that it would be a good idea to improve the scheduling and planning of these tasks to make it less likely they would not be completed in future.
Vitezslav Humpa started a discussion and made some proposals around the issue of applications with very similar names and icons in the system menus<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100684.html</ref>. This resulted in an active and productive debate across several teams about the best way to move forward in addressing the problem. In the end it was agreed that it would be best and fastest to work on a case-by-case basis through the most commonly-encountered name collisions.


<references/>
<references/>


=== QA approval of release candidates ===
=== Draft btrfs test case ===


[[User:Adamwill|Adam Williamson]] reported<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100196.html</ref> that he had updated the Go/No-Go meeting wiki page<ref>http://fedoraproject.org/wiki/Go_No_Go_Meeting</ref> to define the parameters for QA's approval or otherwise of release candidate builds, to make it clear that QA's decision in this regard is entirely determined by concrete criteria (whether all necessary validation tests have been completed and no unaddressed accepted release blocker bugs remain), so that there is no subjectivity to the decision and it can be reported to the Go/No-Go meeting by any member of the QA group (or simply inferred by anyone present at the meeting, whether a member of the QA team or not). [[User:Johannbg|Jóhann Guðmundsson]] questioned whether the Go/No-Go meeting was even necessary, given the improved procedures<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100199.html</ref>. Adam agreed that this was a reasonable question<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100202.html</ref>, but suggested it might be a good idea to preserve the meeting as a 'human in the loop' safeguard against particularly strange and unforeseeable circumstances.
[[User:Jdulaney|John Dulaney]] announced that he had been working on a btrfs test case in advance of Fedora 16, where it is likely to be the default filesystem, and that he had a draft available for review<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100749.html</ref>. [[User:Rhe|Rui He]], [[User:Sundaram|Rahul Sundaram]] and JB replied with comments and suggestions.


<references/>
<references/>


=== Triage scripts updated (again!) ===
=== Fedora 15 QA retrospective ===


Following quickly on the heels of last week's 1.0 RC1, [[User:Mcepl|Matej Cepl]] announced the release of version 1.0 of his Firefox extension to aid in bug triage, bugzilla-triage-scripts<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100277.html</ref>. He asked all Bugzappers with Firefox 4 to update to it and report back on how it worked.
[[User:Jlaska|James Laska]] announced that he had completed the Fedora 15 QA retrospective wiki page, and drafted a set of recommendations for review by the group. He asked for feedback on the retrospective and the recommendations<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100904.html</ref>.


<references/>
<references/>
Line 46: Line 54:
=== AutoQA ===
=== AutoQA ===


The AutoQA team updated their progress as usual at the weekly QA meeting of 2011-05-23<ref>http://fedoraproject.org/wiki/QA/Meetings/20110523</ref>. [[User:Kparal|Kamil Paral]] reported that the team had been working on the proposed 'pretty' plaintext logs, with two proposals: one<ref>http://fpaste.org/sExW/</ref> and two<ref>http://fpaste.org/I7O0/</ref>. [[User:Tflink|Tim Flink]] had been working on the proposed 'spam reduction' code, making AutoQA output less overwhelming for developers, and would be pushing it soon. [[User:Jskladan|Josef Skladanka]] had been working on a wiki page giving an overview of the ResultsDB project<ref>http://fedoraproject.org/wiki/ResultsDB_Overview</ref>.
There were two big AutoQA developments since the last newsletter. One was the so-called 'pretty patch', which improved the layout and legibility of AutoQA results, particularly the dependency check test. It was submitted to the mailing list on 2011-06-06<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002344.html</ref> by [[User:Kparal|Kamil Paral]] and pushed on 2011-06-10<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002385.html</ref>. The other was a patch from [[User:Tflink|Tim Flink]] to reduce the volume of messages sent out by AutoQA, notably by not sending emails to maintainers when the tests are entirely successful. This patch was submitted on 2011-06-06<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002346.html</ref> and merged on 2011-06-09<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002372.html</ref>.


<references/>
<references/>

Revision as of 02:04, 23 June 2011

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

Test Days

A special Test Day[1] was held on Wednesday 2011-06-08 for World IPv6 Day[2], thanks to Linda Wang. Despite the complexity involved in implementing an IPv6 setup for testing, a good group of testers were able to run through the various tests and identify some bugs which have already received developer attention.

The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[3].

Security spin testing

Athmane Madjoudj announced some testing he had done on the Fedora 15 security spin[1]. Adam Williamson thanked him and asked if he had contacted the security spin authors about it[2], and Athmane replied that he would. Later, Athmane announced that he had added some more tests[3], and Adam suggested writing them up as test cases[4]. Athmane did this[5], and then continued to add more test cases regularly. In the course of this work, Athmane noticed and helped to address some problems with the rendering of tags and templates in the wiki[6] [7].

Release criteria revisions

Adam Williamson continued with revisions of the release criteria. He announced that the wording 'release-blocking desktops' to describe the desktops that are capable of blocking release had been generally well received, so he had updated the criteria to use this wording[1].

Installer validation test revisions

Rui He reported that she had refined the various installation source tests that form part of the installation validation matrix by creating some new tests and renaming and adjusting others[1]. James Laska followed up with some questions and comments[2], and Rui He continued with further improvements.

Overly similar application names

Vitezslav Humpa started a discussion and made some proposals around the issue of applications with very similar names and icons in the system menus[1]. This resulted in an active and productive debate across several teams about the best way to move forward in addressing the problem. In the end it was agreed that it would be best and fastest to work on a case-by-case basis through the most commonly-encountered name collisions.

Draft btrfs test case

John Dulaney announced that he had been working on a btrfs test case in advance of Fedora 16, where it is likely to be the default filesystem, and that he had a draft available for review[1]. Rui He, Rahul Sundaram and JB replied with comments and suggestions.

Fedora 15 QA retrospective

James Laska announced that he had completed the Fedora 15 QA retrospective wiki page, and drafted a set of recommendations for review by the group. He asked for feedback on the retrospective and the recommendations[1].

AutoQA

There were two big AutoQA developments since the last newsletter. One was the so-called 'pretty patch', which improved the layout and legibility of AutoQA results, particularly the dependency check test. It was submitted to the mailing list on 2011-06-06[1] by Kamil Paral and pushed on 2011-06-10[2]. The other was a patch from Tim Flink to reduce the volume of messages sent out by AutoQA, notably by not sending emails to maintainers when the tests are entirely successful. This patch was submitted on 2011-06-06[3] and merged on 2011-06-09[4].