FWN/Beats/QualityAssurance: Difference between revisions

Revision as of 04:55, 30 March 2011

QualityAssurance

In this section, we cover the activities of the QA team^[1]. For more information on the work of the QA team and how you can get involved, see the Joining page^[2].

Contributing Writer: Adam Williamson

Test Days

Thursday 2011-03-24 was power management Test Day^[1]. The event produced a great turnout of testers who provided results on a wide range of hardware, and the developers are already looking at the bugs filed. Thanks to everyone who came out to help.

This Tuesday, 2011-03-29, was printing Test Day^[2]. The turnout was a little smaller than we hoped, but the testers who did come managed to find lots of bugs, and Fedora's printing maintainer, Tim Waugh, was very happy with all the results.

This Thursday, 2011-03-31, will be ABRT Test Day^[3]. As well as checking that ABRT (Fedora's automated crash report tool) is working as expected for Fedora 15, we'll be testing out a big new feature, the retrace server^[4]. This allows you to submit crash reports to a remote server which will generate the backtrace - avoiding the need for you to download and install often large debuginfo packages in order to submit reports. Please come along and help us test this exciting new feature!

There is currently no Test Day planned for next Thursday, 2011-04-07. Take the day off!

Anaconda testing

James Laska requested testing of an installer image which included updated versions of anaconda and lorax that needed testing before being approved as stable updates^[1]. Several testers responded, and the updates were soon approved.

↑ http://lists.fedoraproject.org/pipermail/test/2011-March/098018.html

GNOME Tweak Tool testing

Michel Salim announced that GNOME Tweak Tool, an application for tweaking various advanced settings in GNOME 3, was now available for testing^[1]. Several group members thanked Michel for the announcement and provided feedback on the tool.

↑ http://lists.fedoraproject.org/pipermail/test/2011-March/098107.html

Fedora 15 Beta preparation

The third Beta blocker/nice-to-have review meeting took place on 2011-03-25^[1], and the team worked through the full list of proposed Beta blocker and nice-to-have bugs. The first test compose was scheduled for Tuesday 2011-03-22, but could not be completed on that day due to various outstanding bugs and the lack of the planned NetworkManager 0.9 packages. At a special release schedule meeting on Wednesday 2011-03-23^[2], the QA and release engineering teams, and the Fedora Project Leader, agreed that the late arrival of NetworkManager 0.9 justified a one week slip in the release schedule. The first test compose was then set for Tuesday 2011-03-29.

Improving pre-release download page

Tim Flink passed on some feedback^[1] he had received from a new tester who had downloaded a pre-release and asked some questions in the #fedora IRC channel, which does not handle pre-release issues. He suggested that it would be a good idea to add some text to the pre-release download page^[2] explaining the right places to go to post feedback and get help with pre-release issues. Jóhann Guðmundsson and Adam Williamson both suggested filing a request with the websites team, and Tim reported^[3] that he had filed a ticket^[4].

[1] ttp://fedoraproject.org/wiki/QA

[2] ttp://fedoraproject.org/wiki/QA/Join

[3] ttp://fedoraproject.org/wiki/Test_Day:2011-03-24

[4] ttp://fedoraproject.org/wiki/Test_Day:2011-03-29

[5] ttp://fedoraproject.org/wiki/Test_Day:2011-03-31_ABRT_Retrace_Server

[6] ttp://fedoraproject.org/wiki/Features/RetraceServer

[7] ttp://lists.fedoraproject.org/pipermail/test/2011-March/098018.html

[8] ttp://lists.fedoraproject.org/pipermail/test/2011-March/098107.html

[9] ttp://meetbot.fedoraproject.org/fedora-bugzappers/2011-03-25/f-15-beta-blocker.2011-03-25-16.59.html

[10] ttp://meetbot.fedoraproject.org/fedora-meeting/2011-03-23/fedora-meeting.2011-03-23-19.30.html

[11] ttp://lists.fedoraproject.org/pipermail/test/2011-March/098160.html

[12] ttp://fedoraproject.org/get-prerelease

[13] ttp://lists.fedoraproject.org/pipermail/test/2011-March/098173.html

[14] ttps://fedorahosted.org/fedora-websites/ticket/45

[1]

[2]

[1]

[2]

[3]

[4]

[1]

[1]

[1]

[2]

[1]

[2]

[3]

[4]

@@ Line 10: / Line 10: @@
 === Test Days ===
-Thursday 2011-03-17 was preupgrade Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-17_Preupgrade</ref>. A good group of testers showed up, but testing was somewhat complicated by underlying bugs in anaconda. However, the testers were able to check that most of preupgrade itself was working as intended. [[User:Rhe|Rui He]] posted a recap to the list<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097977.html</ref>.
+Thursday 2011-03-24 was power management Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-24</ref>. The event produced a great turnout of testers who provided results on a wide range of hardware, and the developers are already looking at the bugs filed. Thanks to everyone who came out to help.
-This Thursday, 2011-03-24, is power management Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-24</ref>, where we'll be testing how well various common power management operations work on a range of systems. The main focus of this event is to check common power management features on a wide range of hardware, so we really need as many people as possible to come along. Please come out if you have a bit of spare time.
+This Tuesday, 2011-03-29, was printing Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-29</ref>. The turnout was a little smaller than we hoped, but the testers who did come managed to find lots of bugs, and Fedora's printing maintainer, Tim Waugh, was very happy with all the results.
-Next Tuesday (note the special day!), 2011-03-29, will be printing Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-29</ref>. We'll be aiming to test out Fedora 15's printing stack with as many different printers as possible, so if you have a printer and a pulse, please come and join us!
+This Thursday, 2011-03-31, will be ABRT Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-31_ABRT_Retrace_Server</ref>. As well as checking that ABRT (Fedora's automated crash report tool) is working as expected for Fedora 15, we'll be testing out a big new feature, the retrace server<ref>http://fedoraproject.org/wiki/Features/RetraceServer</ref>. This allows you to submit crash reports to a remote server which will generate the backtrace - avoiding the need for you to download and install often large debuginfo packages in order to submit reports. Please come along and help us test this exciting new feature!
-Next Thursday, 2011-03-31, will be ABRT Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-31_ABRT_Retrace_Server</ref>. As well as checking that ABRT (Fedora's automated crash report tool) is working as expected for Fedora 15, we'll be testing out a big new feature, the retrace server<ref>http://fedoraproject.org/wiki/Features/RetraceServer</ref>. This allows you to submit crash reports to a remote server which will generate the backtrace - avoiding the need for you to download and install often large debuginfo packages in order to submit reports. Please come along and help us test this exciting new feature!
+There is currently no Test Day planned for next Thursday, 2011-04-07. Take the day off!
 <references/>
-=== Release criteria revisions ===
+=== Anaconda testing ===
-[[User:Adamwill|Adam Williamson]] proposed several alterations to the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097859.html</ref>, covering several issues that had become clear at recent blocker meetings. The alterations attempted to account for bugs that were specific to particular keymaps or languages, cover firstboot, cover unintended installations, and clarify a few other points. Andre Robatino proposed an additional criterion requiring a working media check process in the installer<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097861.html</ref>, an idea Adam supported<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097863.html</ref>. [[User:Jlaska|James Laska]] recommended caution to avoid tying the criterion too closely to a specific implementation<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097870.html</ref>.
+[[User:Jlaska|James Laska]] requested testing of an installer image which included updated versions of anaconda and lorax that needed testing before being approved as stable updates<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/098018.html</ref>. Several testers responded, and the updates were soon approved.
 <references/>
-=== Nightly builds via Koji ===
+=== GNOME Tweak Tool testing ===
-In response to a query from Gregory Woodbury about nightly builds being stalled<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097856.html</ref>, [[User:Kevin|Kevin Fenzi]] explained<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097860.html</ref> that this was due to a problem on the compose machine. He went on to announce that the nightly builds would soon be done via Koji, which will result in the builds happening much faster, guarantee that they are built in exactly the same way as official composes, and allow for an archive of recent nightly builds to be kept, making regression testing easier.
+[[User:Salimma|Michel Salim]] announced that GNOME Tweak Tool, an application for tweaking various advanced settings in GNOME 3, was now available for testing<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/098107.html</ref>. Several group members thanked Michel for the announcement and provided feedback on the tool.
 <references/>
-=== QA group governance ===
+=== Fedora 15 Beta preparation ===
-[[User:Johannbg|Jóhann Guðmundsson]] raised some questions regarding the governance of the QA group<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097891.html</ref>, including the idea of an elected board to handle some decision making when representing the group as a whole. [[StephenSmoogen|Stephen Smoogen]] noted that SIGs do not generally have formal voting structures<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097899.html</ref>, while leaving open the question of whether QA is (or should be) a SIG or a Steering Committee. Clyde E. Kunkel noted that the functions of the QA team are mostly advisory rather than decision-making, and expressed a desire for an organization chart (or 'wiring diagram') for the various groups that make up the Fedora project<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097902.html</ref>. [[User:Adamwill|Adam Williamson]] pointed out that the Board had worked on such a diagram at FUDCon Tempe, and a copy of the image may be available<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097910.html</ref>. He replied to Jóhann's question about the status of Red Hat employees in the group by saying they tried not to see their position as giving them automatic decision-making power on behalf of the group, but admitted that in practice, other groups often seem to come directly to the Red Hat employees in the group and consider their responses as representing the QA group as a whole.
+The third Beta blocker/nice-to-have review meeting took place on 2011-03-25<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2011-03-25/f-15-beta-blocker.2011-03-25-16.59.html</ref>, and the team worked through the full list of proposed Beta blocker and nice-to-have bugs. The first test compose was scheduled for Tuesday 2011-03-22, but could not be completed on that day due to various outstanding bugs and the lack of the planned NetworkManager 0.9 packages. At a special release schedule meeting on Wednesday 2011-03-23<ref>http://meetbot.fedoraproject.org/fedora-meeting/2011-03-23/fedora-meeting.2011-03-23-19.30.html</ref>, the QA and release engineering teams, and the Fedora Project Leader, agreed that the late arrival of NetworkManager 0.9 justified a one week slip in the release schedule. The first test compose was then set for Tuesday 2011-03-29.
 <references/>
-=== Fedora 15 Beta preparation ===
+=== Improving pre-release download page ===
-The second Beta blocker/nice-to-have review meeting took place on 2011-03-18<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2011-03-18/f-15-beta-blocker.2011-03-18-17.00.html</ref>, and the team worked through the full list of proposed Beta blocker and nice-to-have bugs. The first automated Beta acceptance test run was scheduled for 2011-03-17, but this was not possible due to underlying issues in anaconda at the time, as discussed at the weekly meeting of 2011-03-21<ref>http://fedoraproject.org/wiki/QA/Meetings/20110321</ref>. At the same meeting, [[User:Jlaska|James Laska]] noted that the Beta test compose was at risk due to the same problems, and committed to providing updated images to test candidate fixes for the problems.
+[[User:Tflink|Tim Flink]] passed on some feedback<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/098160.html</ref> he had received from a new tester who had downloaded a pre-release and asked some questions in the #fedora IRC channel, which does not handle pre-release issues. He suggested that it would be a good idea to add some text to the pre-release download page<ref>http://fedoraproject.org/get-prerelease</ref> explaining the right places to go to post feedback and get help with pre-release issues. [[User:Johannbg|Jóhann Guðmundsson]] and [[User:Adamwill|Adam Williamson]] both suggested filing a request with the websites team, and Tim reported<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/098173.html</ref> that he had filed a ticket<ref>https://fedorahosted.org/fedora-websites/ticket/45</ref>.
 <references/>

Search