From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 242 qa beat)
(create fwn 243 qa beat)
Line 3: Line 3:


In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. For more information on the work of the QA team and how you can get involved, see the Joining page<ref>http://fedoraproject.org/wiki/QA/Join</ref>.
In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. For more information on the work of the QA team and how you can get involved, see the Joining page<ref>http://fedoraproject.org/wiki/QA/Join</ref>.
The QA section returns this week after a month's absence, for which I apologize! At first I was busy with Fedora 14 Beta work, and for the last two weeks I simply missed the deadline. If anyone would like to help write the QA beat so this doesn't happen in future, please do get in touch, I'd welcome the help.


Contributing Writer: [[User:Adamwill|Adam Williamson]]
Contributing Writer: [[User:Adamwill|Adam Williamson]]
Line 10: Line 12:
=== Test Days ===
=== Test Days ===


Last week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-02_Preupgrade</ref> on 2010-09-02 was on preupgrade<ref>http://fedoraproject.org/wiki/Preupgrade</ref>, the Fedora in-place upgrade system. There was a good turnout for the Test Day, and it exposed several serious bugs to be resolved for the Beta and final releases of Fedora 14. [[User:Rhe|Rui He]] provided a recap of the event<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093329.html</ref> with a list of the bugs discovered.
Since the last issue, there have been Test Days for systemd<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-07_Systemd</ref> (recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093463.html</ref>), anaconda non-English translations and keyboard layouts<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-16_AnacondaTranslationKeyboard</ref> (recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093871.html</ref>), virtualization<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-23_Virtualization</ref>, and the Graphics Test Week, including nouveau<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-28_Nouveau</ref>, radeon<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-29_Radeon</ref>, and intel<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-30_Intel</ref> (recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094439.html</ref>). See the recaps for details on each event, but they were all broadly successful in exposing bugs to be fixed for the Fedora 14 release.


This week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-07_Systemd</ref> on systemd<ref>http://fedoraproject.org/wiki/Features/systemd</ref>, the new initialization system being introduced in Fedora 14, has happened already, since it was moved up from the usual Thursday slot to Tuesday 2010-09-07. A great turnout of testers exercised systemd extensively and did a good job of discovering bugs to be fixed. Even though the test day itself is past, further testing of systemd would be valuable, so if you have some free time, do check out the page and contribute your results!
Next week's Test Day<ref>https://fedoraproject.org/wiki/Test_Day:2010-10-14_OpenLDAP/NSS</ref> on 2010-10-14 will be on the use of OpenLDAP with the NSS security library - the use of NSS with OpenLDAP is new in Fedora 14, replacing the previous use of OpenSSL. The Test Day will focus on ensuring that OpenLDAP with NSS behaves exactly as OpenLDAP with OpenSSL did. If you're an OpenLDAP user, please come along to the Test Day and help ensure there are no nasty surprises with OpenLDAP in Fedora 14! As always, the Test Day will run all day in the #fedora-test-day IRC channel. You can help out with testing using a virtual machine, so there's no need to alter your main system.


Next week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-09-16_AnacondaTranslationKeyboard</ref> on 2010-09-16 will be on translations and keyboard layout support in the Fedora installer. These features are key for Fedora users outside the United States, which is most of you! As always, the Test Day will run all day in the #fedora-test-day IRC channel. You can help out with testing using a live image or a virtual machine, so there's no need to alter your main system, although some of the tests do require you to complete an installation and check the installed system. Please drop by and help make sure as many languages and keyboard layouts as possible will be correctly supported in Fedora 14!
This will be the last Test Day of the Fedora 14 cycle. If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.


If you would like to propose a main track Test Day for the Fedora 14 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
<references/>
 
=== Fedora 14 Beta testing ===
 
The group has been busy for the last month or so with Fedora 14 Beta validation and testing. We performed validation testing on TC1<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093468.html</ref>, RC1<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093830.html</ref>, RC2<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093843.html</ref> and RC3<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093947.html</ref>, with many community members contributing extensive testing. Eventually we declared that RC3 had passed installation validation testing and desktop validation testing (including all four primary desktops), and along with the development and release engineering groups, approved the release of RC3 as Fedora 14 Beta at the go/no-go meeting of 2010-09-22<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-09-22/fedora-meeting.2010-09-22-21.01.log.html</ref>.


<references/>
<references/>
Line 22: Line 28:
=== Proven testers ===
=== Proven testers ===


During the QA weekly meeting of 2010-08-30<ref>http://fedoraproject.org/wiki/QA/Meetings/20100830</ref>, [[User:Jlaska|James Laska]] asked what to do about an email from a proven tester asking to become a group sponsor so he could sponsor new proven testers members. [[User:Adamwill|Adam Williamson]] pointed out the proven tester wiki page already has a procedure for this<ref>http://fedoraproject.org/wiki/Proven_tester#Becoming_a_mentor</ref>, albeit a fairly casual one. He recommended simply accepting the request for now, and suggested that if anyone wanted to make the process more systematic, they should send a proposal to the mailing list. James also asked about the progress of proven tester metrics, and Adam pointed out that a ticket requesting support from the Bodhi team already existed<ref>http://fedorahosted.org/bodhi/ticket/456</ref>, and promised to follow up on the ticket for the next meeting.
[[User:Adamwill|Adam Williamson]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093372.html</ref> a proven testers procedure clarification<ref>http://fedoraproject.org/w/index.php?title=Proven_tester&diff=195994&oldid=193694</ref>: if an update does not fix a bug it claims to fix, but otherwise works correctly and has other changes which do work, proven testers should not file negative karma against that update.


<references/>
<references/>


=== Outdated Firefox package in Fedora 14 ===
=== Release criteria ===
 
[[User:Jlaska|James Laska]] proposed some new release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093476.html</ref> defining when artwork should be ready for each Fedora release. [[User:Jdulaney|John Dulaney]] and [[User:Adamwill|Adam Williamson]] responded positively, so James pushed the change to the Wiki<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093568.html</ref>.


David Boles pointed out<ref>http://lists.fedoraproject.org/pipermail/test/2010-August/093178.html</ref> that the Firefox package in Fedora 14 was somewhat old. [[User:Adamwill|Adam Williamson]] contacted the maintainers and passed on their explanation<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093205.html</ref> that they had forgotten that Fedora 14 updates now had to be submitted through Bodhi, and that they would soon submit a newer Firefox package.
[[User:Adamwill|Adam Williamson]] proposed a new release criterion<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094302.html</ref> requiring there to be no unintentional conflicts or unresolved dependencies in the entire package repository frozen for release. This was proposed on behalf of release engineering following discussion at a blocker meeting. Some felt this was too high a standard to aim for. After some discussion, [[User:Notting|Bill Nottingham]] stated that "I'd be willing to extend/change this such that trees are tested for broken dependencies, all broken dependencies are filed, and these tickets are attached to the nice-to-have tracker".


<references/>
<references/>


=== Bugzilla voting disabled ===
=== Nice-to-have bug process ===


[[User:Kparal|Kamil Paral]] wondered why he had recieved a mail about votes being removed<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093243.html</ref>. [[User:Kevin|Kevin Fenzi]] explained that this was due to the Bugzilla voting feature being disabled<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/093263.html</ref>. FESCo had decided the system was so flawed it provided no useful data, and so should be disabled. They had not known Bugzilla would send out notification emails about this. He explained that FESCo was now looking at using CC lists or comment quantities to provide an indication of 'popular' bugs, but that this too was proving problematic, and pointed out a ticket where this was being tracked<ref>http://fedorahosted.org/fedora-engineering-services/ticket/24</ref>.
[[User:Adamwill|Adam Williamson]] submitted a proposal<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/094067.html</ref> for a formal process for handling nice-to-have bugs; those bugs that do not block a given release but for which fixes will be taken during the freeze period for that release. [[User:Jlaska|James Laska]] responded with some suggested modifications<ref>http://lists.fedoraproject.org/pipermail/test/2010-September/094122.html</ref>.


<references/>
<references/>

Revision as of 19:50, 6 October 2010

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

The QA section returns this week after a month's absence, for which I apologize! At first I was busy with Fedora 14 Beta work, and for the last two weeks I simply missed the deadline. If anyone would like to help write the QA beat so this doesn't happen in future, please do get in touch, I'd welcome the help.

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

Test Days

Since the last issue, there have been Test Days for systemd[1] (recap[2]), anaconda non-English translations and keyboard layouts[3] (recap[4]), virtualization[5], and the Graphics Test Week, including nouveau[6], radeon[7], and intel[8] (recap[9]). See the recaps for details on each event, but they were all broadly successful in exposing bugs to be fixed for the Fedora 14 release.

Next week's Test Day[10] on 2010-10-14 will be on the use of OpenLDAP with the NSS security library - the use of NSS with OpenLDAP is new in Fedora 14, replacing the previous use of OpenSSL. The Test Day will focus on ensuring that OpenLDAP with NSS behaves exactly as OpenLDAP with OpenSSL did. If you're an OpenLDAP user, please come along to the Test Day and help ensure there are no nasty surprises with OpenLDAP in Fedora 14! As always, the Test Day will run all day in the #fedora-test-day IRC channel. You can help out with testing using a virtual machine, so there's no need to alter your main system.

This will be the last Test Day of the Fedora 14 cycle. If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[11].

  1. ↑ http://fedoraproject.org/wiki/Test_Day:2010-09-07_Systemd
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093463.html
  3. ↑ http://fedoraproject.org/wiki/Test_Day:2010-09-16_AnacondaTranslationKeyboard
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093871.html
  5. ↑ http://fedoraproject.org/wiki/Test_Day:2010-09-23_Virtualization
  6. ↑ http://fedoraproject.org/wiki/Test_Day:2010-09-28_Nouveau
  7. ↑ http://fedoraproject.org/wiki/Test_Day:2010-09-29_Radeon
  8. ↑ http://fedoraproject.org/wiki/Test_Day:2010-09-30_Intel
  9. ↑ http://lists.fedoraproject.org/pipermail/test/2010-October/094439.html
  10. ↑ https://fedoraproject.org/wiki/Test_Day:2010-10-14_OpenLDAP/NSS
  11. ↑ http://fedorahosted.org/fedora-qa/

Fedora 14 Beta testing

The group has been busy for the last month or so with Fedora 14 Beta validation and testing. We performed validation testing on TC1[1], RC1[2], RC2[3] and RC3[4], with many community members contributing extensive testing. Eventually we declared that RC3 had passed installation validation testing and desktop validation testing (including all four primary desktops), and along with the development and release engineering groups, approved the release of RC3 as Fedora 14 Beta at the go/no-go meeting of 2010-09-22[5].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093468.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093830.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093843.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093947.html
  5. ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-09-22/fedora-meeting.2010-09-22-21.01.log.html

Proven testers

Adam Williamson announced[1] a proven testers procedure clarification[2]: if an update does not fix a bug it claims to fix, but otherwise works correctly and has other changes which do work, proven testers should not file negative karma against that update.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093372.html
  2. ↑ http://fedoraproject.org/w/index.php?title=Proven_tester&diff=195994&oldid=193694

Release criteria

James Laska proposed some new release criteria[1] defining when artwork should be ready for each Fedora release. John Dulaney and Adam Williamson responded positively, so James pushed the change to the Wiki[2].

Adam Williamson proposed a new release criterion[3] requiring there to be no unintentional conflicts or unresolved dependencies in the entire package repository frozen for release. This was proposed on behalf of release engineering following discussion at a blocker meeting. Some felt this was too high a standard to aim for. After some discussion, Bill Nottingham stated that "I'd be willing to extend/change this such that trees are tested for broken dependencies, all broken dependencies are filed, and these tickets are attached to the nice-to-have tracker".

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093476.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/093568.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-October/094302.html

Nice-to-have bug process

Adam Williamson submitted a proposal[1] for a formal process for handling nice-to-have bugs; those bugs that do not block a given release but for which fixes will be taken during the freeze period for that release. James Laska responded with some suggested modifications[2].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/094067.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2010-September/094122.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=200986"