From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 263 qa beat)
(create fwn 265 qa beat)
Line 10: Line 10:
=== Test Days ===
=== Test Days ===


No Test Day took place last week due to the delay of the FreeIPA Test Day. This week two Test Days are taking place.
Thursday 2011-02-17 was Xfce 4.8<ref>http://fedoraproject.org/wiki/Features/Xfce48</ref> Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-02-17_Xfce</ref>. The event was well organized by the Xfce team, and a dedicated group of testers was able to expose some important bugs to be fixed.


Tuesday 2011-02-15 was FreeIPA v2<ref>http://fedoraproject.org/wiki/Features/FreeIPAv2</ref> Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-02-15_FreeIPAv2</ref>. IPA is an identity server, and 2.0 is a major revision due to land in Fedora 15. The Test Day focused on checking that all the major changes work as intended. A small group of testers attended the event and exposed several issues, which the developers are now working on.
The week of 2011-02-21 saw the traditional Graphics Test Week. [[User:Adamwill|Adam Williamson]] posted a full recap of the event to the mailing list<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-March/000200.html</ref>. He noted that participation was up again compared to the Fedora 14 events and that some important testing had been carried out, but also noted a worrying trend in status of bugs from previous events, with many bugs reported during the Fedora 13 and 14 events remaining unfixed. He promised to investigate the causes of this.


Thursday 2011-02-17 is Xfce 4.8<ref>http://fedoraproject.org/wiki/Features/Xfce48</ref> Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-02-17_Xfce</ref>, where we are testing out all the new features of this major Xfce release and ensuring there are no regressions. A full report on the event will be printed next week.
This week (and next Tuesday!) is internationalization and localization Test Week, and [[User:Adamwill|Adam Williamson]] put up a blog post<ref>http://www.happyassassin.net/2011/02/28/internationalization-and-localization-test-week-this-week/</ref> explaining the three Test Days this includes: the Anaconda i18n and l10n Test Day on 2011-03-01<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-01_L10n_i18n_Installation</ref>, the desktop i18n Test Day on 2011-03-03<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-03_I18n_Desktop</ref> and the desktop l10n Test Day on 2011-03-08<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-08_L10n_Desktop</ref>. All these events are very important to ensure that non-US-English-speaking users of Fedora get a great experience with Fedora 15.
 
Thursday 2011-03-10 will be the second of three planned GNOME 3 Test Days<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-10_GNOME3_Beta</ref>, where we'll continue to work with the GNOME team to test GNOME 3.0 and its integration with Fedora 15 as rigorously as we can before the final release of both. We'll be repeating the tests from the previous event to see how things have progressed, and also running some new tests which have been added for this event. As before, this is a very important event for both Fedora and GNOME and affects most Fedora users, so if you have a few minutes to spare, please come along and help testing. Once again, live images will be available to make it easier to test, and we'll have a new process in place for reporting crasher bugs without the trouble of trying to install debuginfo packages on a live image!


<references/>
<references/>


=== AutoQA ===
=== Fedora 15 Alpha preparation ===
 
The QA team has been busy over the last two weeks validating the Fedora 15 Alpha release, with TC2<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-February/000186.html</ref>, RC1<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-February/000190.html</ref>, and RC2<ref>http://lists.fedoraproject.org/pipermail/test-announce/2011-February/000194.html</ref> candidate builds being posted and tested. As always, the whole team chipped in with the all-important testing. In the end the RC1 image was not accepted and the release delayed for a week<ref>http://lists.fedoraproject.org/pipermail/devel-announce/2011-February/000759.html</ref> due to a significant bug with many non-English keyboard layouts which was exposed during the testing. With this bug fixed, the RC2 image was accepted as gold at the go/no-go meeting of 2011-03-02<ref>http://meetbot.fedoraproject.org/fedora-meeting/2011-03-02/fedora_15_alpha_gono-go_meeting.2011-03-02-22.00.html</ref>.


[[User:Kparal|Kamil Paral]] reported that he had conducted an AutoQA workshop at the Developer Conference event<ref>http://fedoraproject.org/wiki/DeveloperConference2011</ref> in Brno<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-February/001623.html</ref>. He also published the sample workshop material<ref>http://kparal.fedorapeople.org/autoqa/doctest.tar.xz</ref>.
=== Bodhi improvements ===
 
At the weekly meeting of 2011-02-28<ref>http://fedoraproject.org/wiki/QA/Meetings/20110228</ref>, [[User:Lmacken|Luke Macken]] announced that package-specific test case integration into Bodhi is now live, meaning that packages which have test cases associated with them according to the package test plan SOP<ref>http://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation</ref> will now have the test cases displayed in Bodhi when an update for the package is under review. Also going live in the new Bodhi release are the improvements to the automated messages Bodhi sends to Bugzilla when an update's status changes, improvements discussed in previous issues of this newsletter.


<references/>
<references/>


=== Test Day SOP improvements ===
=== FreeIPA Test Day problems and proposals ===


[[User:Adamwill|Adam Williamson]] proposed an update<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/096978.html</ref> to the Test Day management SOP<ref>http://fedoraproject.org/wiki/QA/SOP_Test_Day_management</ref>, adding a paragraph recommending the creation of a 'how to debug' page for the component being tested. This was the result of a suggestion at the previous weekly meeting by [[User:Johannbg|Jóhann Guðmundsson]].
Dmitri Pal posted a comment to the FreeIPA Test Day trac ticket<ref>http://fedorahosted.org/fedora-qa/ticket/163#comment:10</ref> noting that he was unhappy with the way the event had turned out. [[User:Adamwill|Adam Williamson]], [[User:Jlaska|James Laska]] and [[User:Johannbg|Jóhann Guðmundsson]] joined in with suggestions to try and learn from this experience. Dmitri is considering re-running the event with some tweaks.


<references/>
<references/>


=== Fedora 15 Alpha preparation ===
=== Complications for DeltaISO users ===
 
Andre Robatino provided some detailed information<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097009.html</ref> on what a change to the xz compression scheme would mean for users of DeltaISOs. In a nutshell, users trying to use Fedora 14 -> Fedora 15 DeltaISOs will need to use a workaround, detailed in the post, if applying the ISO on a Fedora 14 system. Andre followed up later with a refinement of the workaround<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097284.html</ref>.
 
<references/>
 
=== IPv6 testing ===
 
A.J. Werkman provided a recap of some very solid testing he had performed on Fedora's out-of-the-box IPv6 capabilities<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097153.html</ref>. He identified some significant problems and reported them as bugs, including anaconda refusing to work without an IPv4 DHCP lease, and Fedora's Bugzilla not being available in the IPv6 domain.
 
<references/>
 
=== Using abrt during Test Days ===


Preparation for the release of Fedora 15 Alpha began in earnest this week. The first test compose was released<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/096917.html</ref> and the first blocker review meeting took place<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/096955.html</ref>. TC1 testing identified some major issues in both installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_15_Alpha_TC1_Install</ref> and desktop<ref>https://fedoraproject.org/wiki/Test_Results:Fedora_15_Alpha_TC1_Desktop</ref> areas.
A thread<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097271.html</ref> started by [[User:Mcloaked|Mike Cloaked]] highlighted the issue of abrt being difficult or impossible to use successfully from live images (such as during Test Days), in large part due to the size of debuginfo packages. [[User:Adamwill|Adam Williamson]] pointed out<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097289.html</ref> that the abrt team's retrace server project<ref>http://fedoraproject.org/wiki/Features/RetraceServer</ref> would be a perfect solution for this, but noted that he had not yet tested using it. When he did, it seemed not yet to be ready for Fedora 15<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097299.html</ref>. [[User:Jmoskovc|Jiri Moskovc]] said that the Fedora 15 support would soon be available<ref>http://lists.fedoraproject.org/pipermail/test/2011-February/097314.html</ref>.


<references/>
<references/>

Revision as of 04:05, 3 March 2011

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

Test Days

Thursday 2011-02-17 was Xfce 4.8[1] Test Day[2]. The event was well organized by the Xfce team, and a dedicated group of testers was able to expose some important bugs to be fixed.

The week of 2011-02-21 saw the traditional Graphics Test Week. Adam Williamson posted a full recap of the event to the mailing list[3]. He noted that participation was up again compared to the Fedora 14 events and that some important testing had been carried out, but also noted a worrying trend in status of bugs from previous events, with many bugs reported during the Fedora 13 and 14 events remaining unfixed. He promised to investigate the causes of this.

This week (and next Tuesday!) is internationalization and localization Test Week, and Adam Williamson put up a blog post[4] explaining the three Test Days this includes: the Anaconda i18n and l10n Test Day on 2011-03-01[5], the desktop i18n Test Day on 2011-03-03[6] and the desktop l10n Test Day on 2011-03-08[7]. All these events are very important to ensure that non-US-English-speaking users of Fedora get a great experience with Fedora 15.

Thursday 2011-03-10 will be the second of three planned GNOME 3 Test Days[8], where we'll continue to work with the GNOME team to test GNOME 3.0 and its integration with Fedora 15 as rigorously as we can before the final release of both. We'll be repeating the tests from the previous event to see how things have progressed, and also running some new tests which have been added for this event. As before, this is a very important event for both Fedora and GNOME and affects most Fedora users, so if you have a few minutes to spare, please come along and help testing. Once again, live images will be available to make it easier to test, and we'll have a new process in place for reporting crasher bugs without the trouble of trying to install debuginfo packages on a live image!

  1. ↑ http://fedoraproject.org/wiki/Features/Xfce48
  2. ↑ http://fedoraproject.org/wiki/Test_Day:2011-02-17_Xfce
  3. ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-March/000200.html
  4. ↑ http://www.happyassassin.net/2011/02/28/internationalization-and-localization-test-week-this-week/
  5. ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-01_L10n_i18n_Installation
  6. ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-03_I18n_Desktop
  7. ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-08_L10n_Desktop
  8. ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-10_GNOME3_Beta

Fedora 15 Alpha preparation

The QA team has been busy over the last two weeks validating the Fedora 15 Alpha release, with TC2[1], RC1[2], and RC2[3] candidate builds being posted and tested. As always, the whole team chipped in with the all-important testing. In the end the RC1 image was not accepted and the release delayed for a week[4] due to a significant bug with many non-English keyboard layouts which was exposed during the testing. With this bug fixed, the RC2 image was accepted as gold at the go/no-go meeting of 2011-03-02[5].

Bodhi improvements

At the weekly meeting of 2011-02-28[6], Luke Macken announced that package-specific test case integration into Bodhi is now live, meaning that packages which have test cases associated with them according to the package test plan SOP[7] will now have the test cases displayed in Bodhi when an update for the package is under review. Also going live in the new Bodhi release are the improvements to the automated messages Bodhi sends to Bugzilla when an update's status changes, improvements discussed in previous issues of this newsletter.

  1. ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-February/000186.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-February/000190.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test-announce/2011-February/000194.html
  4. ↑ http://lists.fedoraproject.org/pipermail/devel-announce/2011-February/000759.html
  5. ↑ http://meetbot.fedoraproject.org/fedora-meeting/2011-03-02/fedora_15_alpha_gono-go_meeting.2011-03-02-22.00.html
  6. ↑ http://fedoraproject.org/wiki/QA/Meetings/20110228
  7. ↑ http://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation

FreeIPA Test Day problems and proposals

Dmitri Pal posted a comment to the FreeIPA Test Day trac ticket[1] noting that he was unhappy with the way the event had turned out. Adam Williamson, James Laska and Jóhann Guðmundsson joined in with suggestions to try and learn from this experience. Dmitri is considering re-running the event with some tweaks.

  1. ↑ http://fedorahosted.org/fedora-qa/ticket/163#comment:10

Complications for DeltaISO users

Andre Robatino provided some detailed information[1] on what a change to the xz compression scheme would mean for users of DeltaISOs. In a nutshell, users trying to use Fedora 14 -> Fedora 15 DeltaISOs will need to use a workaround, detailed in the post, if applying the ISO on a Fedora 14 system. Andre followed up later with a refinement of the workaround[2].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097009.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097284.html

IPv6 testing

A.J. Werkman provided a recap of some very solid testing he had performed on Fedora's out-of-the-box IPv6 capabilities[1]. He identified some significant problems and reported them as bugs, including anaconda refusing to work without an IPv4 DHCP lease, and Fedora's Bugzilla not being available in the IPv6 domain.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097153.html

Using abrt during Test Days

A thread[1] started by Mike Cloaked highlighted the issue of abrt being difficult or impossible to use successfully from live images (such as during Test Days), in large part due to the size of debuginfo packages. Adam Williamson pointed out[2] that the abrt team's retrace server project[3] would be a perfect solution for this, but noted that he had not yet tested using it. When he did, it seemed not yet to be ready for Fedora 15[4]. Jiri Moskovc said that the Fedora 15 support would soon be available[5].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097271.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097289.html
  3. ↑ http://fedoraproject.org/wiki/Features/RetraceServer
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097299.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2011-February/097314.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=224782"