(create fwn 273 qa beat) |
(create fwn 275 qa beat) |
||
| Line 10: | Line 10: | ||
=== Test Days === | === Test Days === | ||
Thursday 2011-04- | Thursday 2011-04-28 was cloud Test Day, with events for BoxGrinder<ref>http://fedoraproject.org/wiki/Test_Day:2011-04-28_Cloud_SIG_BoxGrinder_Build</ref> and EC2<ref>http://fedoraproject.org/wiki/Test_Day:2011-04-28_Cloud_SIG_Fedora_EC2</ref>. Both Test Days went off successfully, with a decent turnout of testers and some productive results. Marc Savy posted a recap of the BoxGrinder event<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000243.html</ref>: "In the course of testing only two major unanticipated bugs were encountered, along with several known issues for Fedora 15 appliances, but no blockers preventing inclusion in the distribution. These problems have since been remediated in the upcoming 0.9.2 release...Aside from the bugs exposed, the opportunity to meet new users, garner ideas and solicit opinions was extremely worthwhile." | ||
These were the final Test Days for the Fedora 15 cycle. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. | |||
The | <references/> | ||
=== Fedora 15 validation and preparation === | |||
Over the last three weeks the group has been busy with preparation and testing for the final Fedora 15 release, leading to the absence of this beat from recent FWN issues, for which we apologize. The final blocker review meeting took place on Friday 2011-05-06<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2011-05-06/f15-blocker-review.2011-05-06-17.02.html</ref>. The first test compose landed on Tuesday 2011-05-03<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000236.html</ref>, followed by three release candidates, the first on Wednesday 2011-05-11<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000239.html</ref>, the second on Friday 2011-05-13<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000240.html</ref>, and the third and final also on that date<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000241.html</ref>. The group completed desktop and installation validation testing for all four composes, viewable on the Wiki<ref>http://fedoraproject.org/wiki/Category:Fedora_15_Final_RC_Test_Results</ref> <ref>http://fedoraproject.org/wiki/Category:Fedora_15_Final_TC_Test_Results</ref>. As can be inferred from the existence of the second and third release candidates, the group was able to identify release blocking issues and ensure these were successfully resolved. The RC3 testing confirmed that this image set met all release criteria, and it was duly approved as the gold image set for Fedora 15 release at the Go/No-Go meeting of 2011-05-17<ref>http://meetbot.fedoraproject.org/fedora-meeting/2011-05-17/fedora_15_go_or_no_go_meeting.2011-05-17-21.00.html</ref>, a decision which was later announced by [[User:Rbergero|Robyn Bergeron]]<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000245.html</ref>. | |||
<references/> | |||
=== Sample data and configurations for testing === | |||
Samuel Greenfeld asked whether the group had a repository of sample items of data and configuration files for testing updates<ref>http://lists.fedoraproject.org/pipermail/test/2011-April/099337.html</ref>. [[User:Jlaska|James Laska]] replied that there was not such a repository, and suggested that such files could be added to the Wiki and incorporated in appropriate test cases<ref>http://lists.fedoraproject.org/pipermail/test/2011-April/099353.html</ref>. Samuel explained that he thought such files could be useful in multiple cases, so a separate structure from the test case setup was required<ref>http://lists.fedoraproject.org/pipermail/test/2011-April/099374.html</ref>. | |||
<references/> | |||
=== Problematic glibc update === | |||
The group (along with the development team) was quick to catch a problematic glibc update<ref>http://admin.fedoraproject.org/updates/glibc-2.13.90-13</ref> which constituted a significant ABI break shortly before Fedora 15 release. This was spotted both on the mailing list<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/099636.html</ref> and via the Bodhi update feedback system<ref>http://admin.fedoraproject.org/updates/glibc-2.13.90-13</ref>. The updates policy<ref>http://fedoraproject.org/wiki/Updates_Policy</ref> ensured that the update was not accepted until the ABI change had been reverted. | |||
<references/> | |||
=== Triage scripts updated === | |||
[[User:Mcepl|Matej Cepl]] announced the release of a major update to his Firefox extension to aid in bug triage, bugzilla-triage-scripts 1.0 RC1<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/099784.html</ref>. He asked all Bugzappers with Firefox 4 to update to it and report back on how it worked. | |||
<references/> | <references/> | ||
=== | === Desktop release criteria revisions === | ||
[[MatthiasClasen|Matthias Clasen]] proposed several revisions to the desktop-related release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/099787.html</ref>, prompted by discussions about a then-release-blocking bug report on the presence of an Other category in the Shell's applications view<ref>http://bugzilla.redhat.com/show_bug.cgi?id=697834</ref>. He proposed removing the criterion requiring there to be no Other category in the application menus, among other changes. [[User:Adamwill|Adam Williamson]] commented on the proposed changes<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/099789.html</ref>, as did [[User:Jlaska|James Laska]]<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/099792.html</ref>. | |||
<references/> | <references/> | ||
=== | === AutoQA === | ||
During the QA meeting of 2011-05-02<ref>http://fedoraproject.org/wiki/QA/Meetings/20110502</ref>, [[User:Kparal|Kamil Paral]] explained that the AutoQA team had decided the upcoming 0.5.0 release would have a tight focus on making AutoQA feedback more friendly to developers, following several comments from developers on the verbosity and usefulness of current AutoQA results. [[User:Tflink|Tim Flink]] provided minutes from the meeting where this plan was discussed<ref>http://fedorahosted.org/autoqa/wiki/Planning050series</ref>. Plans include reducing the number of AutoQA comments in Bodhi and improving the structure and contents of the dependency check test log. The group had also fixed a problem with disk space exhaustion resulting in the deletion of recent test logs, which turned out to be caused by tests stuck in a loop writing huge logs. | |||
During the meeting of 2011-05-16<ref>http://fedoraproject.org/wiki/QA/Meetings/20110516</ref>, Kamil reported that he had implemented package caching in AutoQA, which can drastically speed up dependency check test runs when enabled. [[User:Jskladan|Josef Skladanka]] said that he had introduced an algorithm to filter out key information from the dependency check test log explaining why a given package has dependency issues. The team had started to document its test cases, beginning with the upgradepath test<ref>http://fedoraproject.org/wiki/AutoQA_tests/Upgradepath</ref>. Tim was working further on the looping tests issue, and Vitezslav Humpa created a template for future test reports<ref>http://vhumpa.fedorapeople.org/prettylog_upgradepath_example2.txt</ref>. | |||
<references/> | <references/> | ||
Revision as of 22:33, 18 May 2011
QualityAssurance
In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].
Contributing Writer: Adam Williamson
Test Days
Thursday 2011-04-28 was cloud Test Day, with events for BoxGrinder[1] and EC2[2]. Both Test Days went off successfully, with a decent turnout of testers and some productive results. Marc Savy posted a recap of the BoxGrinder event[3]: "In the course of testing only two major unanticipated bugs were encountered, along with several known issues for Fedora 15 appliances, but no blockers preventing inclusion in the distribution. These problems have since been remediated in the upcoming 0.9.2 release...Aside from the bugs exposed, the opportunity to meet new users, garner ideas and solicit opinions was extremely worthwhile."
These were the final Test Days for the Fedora 15 cycle. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[4].
Fedora 15 validation and preparation
Over the last three weeks the group has been busy with preparation and testing for the final Fedora 15 release, leading to the absence of this beat from recent FWN issues, for which we apologize. The final blocker review meeting took place on Friday 2011-05-06[1]. The first test compose landed on Tuesday 2011-05-03[2], followed by three release candidates, the first on Wednesday 2011-05-11[3], the second on Friday 2011-05-13[4], and the third and final also on that date[5]. The group completed desktop and installation validation testing for all four composes, viewable on the Wiki[6] [7]. As can be inferred from the existence of the second and third release candidates, the group was able to identify release blocking issues and ensure these were successfully resolved. The RC3 testing confirmed that this image set met all release criteria, and it was duly approved as the gold image set for Fedora 15 release at the Go/No-Go meeting of 2011-05-17[8], a decision which was later announced by Robyn Bergeron[9].
- ↑ http://meetbot.fedoraproject.org/fedora-bugzappers/2011-05-06/f15-blocker-review.2011-05-06-17.02.html
- ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000236.html
- ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000239.html
- ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000240.html
- ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000241.html
- ↑ http://fedoraproject.org/wiki/Category:Fedora_15_Final_RC_Test_Results
- ↑ http://fedoraproject.org/wiki/Category:Fedora_15_Final_TC_Test_Results
- ↑ http://meetbot.fedoraproject.org/fedora-meeting/2011-05-17/fedora_15_go_or_no_go_meeting.2011-05-17-21.00.html
- ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-May/000245.html
Sample data and configurations for testing
Samuel Greenfeld asked whether the group had a repository of sample items of data and configuration files for testing updates[1]. James Laska replied that there was not such a repository, and suggested that such files could be added to the Wiki and incorporated in appropriate test cases[2]. Samuel explained that he thought such files could be useful in multiple cases, so a separate structure from the test case setup was required[3].
Problematic glibc update
The group (along with the development team) was quick to catch a problematic glibc update[1] which constituted a significant ABI break shortly before Fedora 15 release. This was spotted both on the mailing list[2] and via the Bodhi update feedback system[3]. The updates policy[4] ensured that the update was not accepted until the ABI change had been reverted.
Triage scripts updated
Matej Cepl announced the release of a major update to his Firefox extension to aid in bug triage, bugzilla-triage-scripts 1.0 RC1[1]. He asked all Bugzappers with Firefox 4 to update to it and report back on how it worked.
Desktop release criteria revisions
Matthias Clasen proposed several revisions to the desktop-related release criteria[1], prompted by discussions about a then-release-blocking bug report on the presence of an Other category in the Shell's applications view[2]. He proposed removing the criterion requiring there to be no Other category in the application menus, among other changes. Adam Williamson commented on the proposed changes[3], as did James Laska[4].
AutoQA
During the QA meeting of 2011-05-02[1], Kamil Paral explained that the AutoQA team had decided the upcoming 0.5.0 release would have a tight focus on making AutoQA feedback more friendly to developers, following several comments from developers on the verbosity and usefulness of current AutoQA results. Tim Flink provided minutes from the meeting where this plan was discussed[2]. Plans include reducing the number of AutoQA comments in Bodhi and improving the structure and contents of the dependency check test log. The group had also fixed a problem with disk space exhaustion resulting in the deletion of recent test logs, which turned out to be caused by tests stuck in a loop writing huge logs.
During the meeting of 2011-05-16[3], Kamil reported that he had implemented package caching in AutoQA, which can drastically speed up dependency check test runs when enabled. Josef Skladanka said that he had introduced an algorithm to filter out key information from the dependency check test log explaining why a given package has dependency issues. The team had started to document its test cases, beginning with the upgradepath test[4]. Tim was working further on the looping tests issue, and Vitezslav Humpa created a template for future test reports[5].