From Fedora Project Wiki

< FWN‎ | Beats

(create 186 qa beat)
(create fwn 288 draft)
 
(102 intermediate revisions by 3 users not shown)
Line 2: Line 2:
== QualityAssurance ==
== QualityAssurance ==


In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>.
In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. For more information on the work of the QA team and how you can get involved, see the Joining page<ref>http://fedoraproject.org/wiki/QA/Join</ref>.
 
We apologize for the lack of a QA section for the last few issues of FWN: the QA team was very busy with Fedora 16 validation testing. This issue catches up with the QA team news from the last several weeks.


Contributing Writer: [[User:Adamwill|Adam Williamson]]
Contributing Writer: [[User:Adamwill|Adam Williamson]]
Line 10: Line 12:
=== Test Days ===
=== Test Days ===


There was no main track Test Day last week. The Fit and Finish project's Test Day track continued with its second Test Day, on power management and suspend/resume<ref>https://fedoraproject.org/wiki/Test_Day:2009-07-21_Fit_and_Finish:Batteries_and_Suspend</ref>. The event was a success, with several testers turning out, many bugs filed, and some fixed during the day or soon afterwards, especially relating to laptops with multiple batteries.
In the past few weeks, we finished up the Fedora 16 Test Day schedule, with Graphics Test Week taking place at the start of September<ref>http://fedoraproject.org/wiki/Test_Day:2011-09-06_Nouveau</ref> ref>http://fedoraproject.org/wiki/Test_Day:2011-09-07_Radeon</ref> ref>http://fedoraproject.org/wiki/Test_Day:2011-09-07_Radeon</ref>, virtualization test day taking place on 2011-09-15<ref>http://fedoraproject.org/wiki/Test_Day:2011-09-15_Virtualization</ref>, another i18n desktop test day on 2011-09-22<ref>http://fedoraproject.org/wiki/Test_Day:2011-09-22_I18n_Desktop</ref>, an ABRT test day on 2011-09-26<ref>http://fedoraproject.org/wiki/Test_Day:2011-09-26_ABRT</ref>, a power management test day on 2011-09-29<ref>http://fedoraproject.org/wiki/Test_Day:2011-09-29_PowerManagement</ref>, printing test day on 2011-10-06<ref>http://fedoraproject.org/wiki/Test_Day:2011-10-06_Printing</ref>, Fedora packager plugin for Eclipse test day on 2011-10-13<ref>http://fedoraproject.org/wiki/Test_Day:2011-10-13_Fedora_Packager_for_Eclipse</ref>, and Cloud SIG test day on 2011-10-20<ref>http://fedoraproject.org/wiki/Test_Day:2011-10-20_Cloud_SIG_Test_Day</ref>. Most of these test days passed off successfully with the work of the developers behind them, despite the QA team being very busy, so many thanks to those who organized and carried out these events, and those who turned up to do the testing.


No Test Day is scheduled for next week. If you would like to propose a main track Test Day for the Fedora 12 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>https://fedorahosted.org/fedora-qa/</ref>.
The Fedora 17 Test Day cycle has not yet started. We welcome proposals for test days for the Fedora 17 cycle, and we usually accept all the proposals that are made. You can propose a test day for almost anything, and organize it yourself following the handy guide we provide<ref>http://fedoraproject.org/wiki/QA/SOP_Test_Day_management</ref>, or alternatively we can help out with the organization of the event. Information on how to propose a test day is available on the Wiki<ref>http://fedoraproject.org/wiki/QA/Test_Days/Create</ref>.


<references/>
<references/>


=== Weekly meetings ===
=== Fedora 16 preparation ===
 
As mentioned above, Fedora 16 release validation took up almost all of the QA team's time during the last few months, with very challenging Beta and Final releases. There were a total of 12 candidate builds for Beta and Final combined, and the whole team put in tireless work running the set of validation tests against each build and then investigating and verifying the large number of blocker bugs identified. The team was able to contribute to the release eventually going ahead with only a one week slip to the Beta schedule and no slip of the Final schedule, a considerable achievement in the light of the many complex changes in the Fedora 16 feature list.


The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-07-22. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090722</ref>. [[User:Jlaska|James Laska]] reported that he had published a blog post asking people to help with the process of writing Debugging pages<ref>http://jlaska.livejournal.com/5693.html</ref>. [[User:Adamwill|Adam Williamson]] mentioned that he had looked into creating some of the desired pages, but did not know what kind of information was actually required for any of the components concerned. [[User:jkeating|Jesse Keating]] suggested doing an informal interview-style session with maintainers to discover what information is needed, and then having QA take responsibility for turning that information into a finished Wiki page.
<references/>


[[User:Jlaska|James Laska]] had created a meeting time matrix<ref>http://fedoraproject.org/wiki/QA_Meeting_Matrix</ref> for the purpose of re-scheduling the QA meeting to make it possible for as many group members as possible to attend. The group agreed that the new meeting day and time should be Mondays at 16:00 UTC, moved from Wednesdays at 16:00 UTC.
=== Release criteria updates ===


[[User:Jlaska|James Laska]] noted that a Fedora 12 Alpha blocker bug review meeting was scheduled for Friday 2009-07-24. It was agreed that [[User:Adamwill|Adam Williamson]] would send out an announcement of the meeting, and James would send out a recap after it had finished. [[User:jkeating|Jesse Keating]] mentioned it would be good to do some Rawhide install testing prior to the meeting, but a combination of two significant bugs was preventing almost any Rawhide install from working.
Largely as a result of the Fedora 16 validation process, there were several adjustments and additions to the release criteria in recent weeks. After discussion of the proposed kickstart / unattended installation release criterion concluded, [[User:Adamwill|Adam Williamson]] reported that he had committed his proposed modifications<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102599.html</ref>. He also committed a change to reflect the increased priority of EFI installations from Fedora 17 onwards<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102600.html</ref>.


[[User:Jlaska|James Laska]] explained that a test compose for Fedora 12 Alpha was scheduled for 2009-07-29, and [[User:Liam|Liam Li]] had made an announcement requesting help on install testing<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00429.html</ref>. [[User:jkeating|Jesse Keating]] pointed out that it would not be easy for the general public to take part, as the test compose would not be generally distributed. This led to another long discussion about the practicality of distributing time-critical test composes to the public. No definite conclusion was reached, but a tentative agreement was made to look into a system which would allow access to such composes to members of the QA group in FAS.
Adam also passed on a suggestion from [[User:Pjones|Peter Jones]] to improve the clarity of the virtualization criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102601.html</ref>. After an extensive discussion, an elegant wording suggestion from Albert Graham<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102636.html</ref> was eventually accepted and committed<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103680.html</ref>.


[[User:Johannbg|Jóhann Guðmundsson]] noted that there were some problems with Dracut, the nash/mkinitrd replacement being introduced as a feature in Fedora 12. It has no implementation plan by which the progress of the feature can be externally measured, and no detailed contingency plan beyond 'revert to mkinitrd'. Jóhann agreed to contact the feature mantainer, [[User:Harald|Harald Hoyer]], to help develop a full test plan and contingency plan.
[[User:Tflink|Tim Flink]] raised the question to what extent support for Xen virtualization should be included in the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/103127.html</ref>. After a similarly enthusiastic discussion, it was eventually agreed that Xen DomU support - effectively, the ability to install successfully as a Xen guest - should be a Final release criterion<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103678.html</ref>.


[[User:Wwoods|Will Woods]] reported on the progress of the AutoQA project. He has now automated the first four test cases in the Rawhide Acceptance Test Plan<ref>http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan</ref>, and is now working on automating the installation tests. He noted that separate i386, x86-64 and PowerPC test hosts would be necessary for some tests, and that PPC might be difficult in the absence of the Fedora standard libvirt virtualization framework on that platform. [[User:jkeating|Jesse Keating]] worried that the installation tests may be adding too much complexity to the system, and asked how much faster the process would be if only repository level tests were considered. [[User:Adamwill|Adam Williamson]] pointed out that the full set of repository level tests were the ones that had already been automated. Will promised that they would be updated to send the results somewhere publicly accessible soon.
Adam also proposed downgrading some rarely-used kickstart deployment methods from Beta to Final in the criteria, requiring only the most commonly-used to be working at Beta stage<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103557.html</ref>.


[[User:sdz|Sebastian Dziallas]] brought up the topic of a Test Day for the Sugar on a Stick project<ref>http://wiki.sugarlabs.org/go/Sugar_on_a_Stick</ref> - essentially for the integration of Sugar with a stock Fedora distribution. It was agreed that the SoaS project would host the Test Day themselves using the SOP created for this purpose<ref>http://fedoraproject.org/wiki/User:Adamwill/Draft_test_day_SOP</ref>. A tentative date of 2009-09-03 was agreed for the test day.
Finally, Adam proposed a criterion for i18n (translation) issues<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103588.html</ref>. After discussion, the proposal was agreed upon at a blocker review meeting later in the week<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103679.html</ref>.


The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-07-21. The full log is available<ref>http://meetbot.fedoraproject.org/fedora-meeting/2009-07-21/fedora-meeting.2009-07-21-15.02.log.html</ref>. No-one had heard from Brennan Ashton regarding the status of the triage metrics project. [[User:Adamwill|Adam Williamson]] agreed to contact him by email to find out the current status, and ask if he would be interested in having a co-maintainer on the project, in the interest of smoother development.
<references/>


The group discussed the current draft of the critical path-based triage component list<ref>http://fedoraproject.org/wiki/User:Arxs/CPCL</ref>. There was a general feeling that the list was very long and might contain components that, practically speaking, would not benefit hugely from triage. It also seemed to contain at least some binary (rather than source) package names, while Bugzilla is based on source package names. Niels Haase and [[User:Mcepl|Matej Cepl]] volunteered to adjust the list to use source package names, and break it up into groups for ease of digestion, for further review at next week's meeting.
=== Update policy changes ===


[[User:Adamwill|Adam Williamson]] gave an update on the status of the kernel bug triage project. He admitted it had not progressed very far as he had been focussing on anaconda triage. He outlined a plan under which a volunteer would, as a test, triage bugs on one particular component of the kernel, to see if the process could be made to work. [[User:Tk009|Edward Kirk]] thought the proposal a sound one, and Adam agreed to try and put in into practice in the next week.
In September, [[User:Karsten|Karsten Hopp]] raised the issue of a security update for Fedora 14 which had been languishing in the updates-testing repository for some time<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102493.html</ref>. [[User:Adamwill|Adam Williamson]] explained that the amount of testers working on older releases was limited, and that the actual karma requirements for updates to be accepted were controlled by FESCo (the Fedora engineering steering committee), not the QA group<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102497.html</ref>. [[User:Cra|Chuck Anderson]] noted that he had the update in question installed, but was struggling for lack of information on how to test it properly<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102503.html</ref>. [[User:Sundaram|Rahul Sundaram]] suggested that Karsten file a ticket with FESCo to raise the issue<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102502.html</ref>, and Karsten did<ref>http://fedorahosted.org/fesco/ticket/664</ref>.


Finally, the group discussed the 'Bugzilla Semantics' proposal [[User:Adamwill|Adam Williamson]] had made to the mailing list, involving various ways in which the triage process could be tweaked and the use of the NEW and ASSIGNED states changed. Initially discussion was in favour of retaining the status quo, but [[User:jkeating|Jesse Keating]] and [[User:jwboyer|Josh Boyer]] made it clear that the development groups they were involved in used ASSIGNED in a different way to its use by the Bugzappers group, and they would prefer if Bugzappers marked bugs as having been triaged in some other way, so their groups could take advantage of the triage process. It became clear that there would be both benefits and costs involved in changing the triage process. [[User:Adamwill|Adam Williamson]] agreed to send a follow-up email to the mailing list to summarize the current state of the debate, and to see if a consensus could be found on a future path.
That ticket was merged with another similar one reported by Doug Ledford<ref>http://fedorahosted.org/fesco/ticket/667</ref>, which became a topic of concern to FESCo. After several rounds of discussions, FESCo first decided to relax the requirements for critical path updates somewhat by allowing them to be sent through to the stable repository without the 'required' karma after a period of two weeks had elapsed<ref>http://fedorahosted.org/bodhi/ticket/642</ref>, and later proposed removing the requirement for critical path updates to receive positive karma from a proven tester<ref>http://fedorahosted.org/fesco/ticket/667#comment:26</ref>, effectively a proposal to end the proven tester system, as this is the only function it serves.


The next QA weekly meeting will be held on 2009-07-27 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-07-28 at 1500 UTC in #fedora-meeting.
The QA group discussed this proposal at the weekly meeting of 2011-11-07<ref>http://fedoraproject.org/wiki/QA/Meetings/20111107</ref>, agreeing that, while they had some reservations about the proposal, they were not definitely opposed to it, and recognized that critical path updates not receiving the currently-required karma is a significant problem.


<references/>
<references/>


=== F12 Alpha blocker bug review meeting ===
=== Update candidate notification ===


[[User:Adamwill|Adam Williamson]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00472.html</ref> the second blocker bug review meeting for Fedora 12, to be held on 2009-07-24, mainly to review blocker bug status for the upcoming Alpha release. Later, [[User:Jlaska|James Laska]] posted a recap of the meeting<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00498.html</ref>.
Samuel Greenfeld asked if there was any system to notify testers of new candidate updates for specific packages, and to determine what packages are being actively used on a system<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102981.html</ref>. There were no takers for the second question, but for the first, [[User:Adamwill|Adam Williamson]] suggested using yum parameters that would allow one to specify only certain packages be pulled from the updates-testing repository<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102982.html</ref>, and [[User:till|Till Maas]] pointed out that Bodhi can actually provide per-package RSS notifications<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102992.html</ref>.


<references/>
<references/>


=== Xfce spin testing ===
=== Proven tester meetings ===
 
[[User:maxamillion|Adam Miller]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00391.html</ref> the second test live image with the Xfce desktop, and would appreciate testing and reporting of problems. He noted that the known bugs in Anaconda at the time of the compose may make the image very difficult to install, but it should be usable on most hardware as a live boot.
 
<references/>


=== KDE QA tester request ===
As a response to the concerns about candidate updates not receiving enough karma, [[User:Kevin|Kevin Fenzi]] ran a series of weekly proven tester meetups<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/102869.html</ref> from 2011-09-21 to 2011-10-26. Recaps of these meetings are available in the mailing list archives<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/103000.html</ref> <ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103341.html</ref> <ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103585.html</ref> <ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103840.html</ref> <ref>http://lists.fedoraproject.org/pipermail/test/2011-October/104043.html</ref>.


[[User:kkofler|Kevin Kofler]] posted a request<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00423.html</ref> for volunteers to help with KDE testing. He noted that the requirements for testers were quite low, and asked interested people to reply to the fedora-kde mailing list or #fedora-kde on IRC. Two people, Aioanei Rares and Marco Crosio, were quick to volunteer, and were accepted as the new KDE testers.
Kevin also proposed an updates-testing-info mailing list, containing only the mails about new packages in updates-testing<ref>http://lists.fedoraproject.org/pipermail/test/2011-September/103163.html</ref>. However, the consensus was against the idea, as it was felt that it was easy enough to simply filter the desired mails from the test mailing list for those who did not want to read the other traffic.


<references/>
<references/>


=== Bugzilla semantics debate ===
=== QA group representation at FUDCon Pune ===


The Bugzilla semantics debate<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00309.html</ref> continued throughout the week, especially following the input from developers at the QA meeting (see above) and the subsequent summary<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00411.html</ref> posted by [[User:Adamwill|Adam Williamson]]. He proposed three options: leaving the current triage process unchanged and encouraging development teams who currently use ASSIGNED to mean a bug has been accepted by a certain developer to use ON_QA instead; changing Bugzappers practice to use a keyword to mark triaged bugs going forward, but leave all existing bugs as they are; or changing Bugzappers practice going forwards and also attempting to 'fix' existing bug reports to use the keyword where appropriate. [[User:jkeating|Jesse Keating]] seemed to favor the second option<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00412.html</ref>, and [[User:poelstra|John Poelstra]] agreed<ref>http://www.redhat.com/archives/fedora-test-list/2009-July/msg00415.html</ref>.
[[User:Ankursinha|Ankur Sinha]] asked whether anyone from the QA team would be present at the upcoming FUDCon in Pune, India and able to do a presentation on the group's activities<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103712.html</ref>. [[User:Adamwill|Adam Williamson]] replied that unfortunately none of the Red Hat team would be at the conference, but encouraged Ankur to take a shot at giving a presentation himself<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103728.html</ref>. A S Alam then stepped up to volunteer to lead a QA session<ref>http://lists.fedoraproject.org/pipermail/test/2011-October/103739.html</ref>. His session was scheduled for 2011-11-04<ref>http://fudcon.in/sessions/fedora-testing</ref>, but we have no report on the event - if you were present, please write to the mailing list and let us know how it went!


<references/>
<references/>

Latest revision as of 05:10, 17 November 2011

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

We apologize for the lack of a QA section for the last few issues of FWN: the QA team was very busy with Fedora 16 validation testing. This issue catches up with the QA team news from the last several weeks.

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

Test Days

In the past few weeks, we finished up the Fedora 16 Test Day schedule, with Graphics Test Week taking place at the start of September[1] ref>http://fedoraproject.org/wiki/Test_Day:2011-09-07_Radeon</ref> ref>http://fedoraproject.org/wiki/Test_Day:2011-09-07_Radeon</ref>, virtualization test day taking place on 2011-09-15[2], another i18n desktop test day on 2011-09-22[3], an ABRT test day on 2011-09-26[4], a power management test day on 2011-09-29[5], printing test day on 2011-10-06[6], Fedora packager plugin for Eclipse test day on 2011-10-13[7], and Cloud SIG test day on 2011-10-20[8]. Most of these test days passed off successfully with the work of the developers behind them, despite the QA team being very busy, so many thanks to those who organized and carried out these events, and those who turned up to do the testing.

The Fedora 17 Test Day cycle has not yet started. We welcome proposals for test days for the Fedora 17 cycle, and we usually accept all the proposals that are made. You can propose a test day for almost anything, and organize it yourself following the handy guide we provide[9], or alternatively we can help out with the organization of the event. Information on how to propose a test day is available on the Wiki[10].

  1. ↑ http://fedoraproject.org/wiki/Test_Day:2011-09-06_Nouveau
  2. ↑ http://fedoraproject.org/wiki/Test_Day:2011-09-15_Virtualization
  3. ↑ http://fedoraproject.org/wiki/Test_Day:2011-09-22_I18n_Desktop
  4. ↑ http://fedoraproject.org/wiki/Test_Day:2011-09-26_ABRT
  5. ↑ http://fedoraproject.org/wiki/Test_Day:2011-09-29_PowerManagement
  6. ↑ http://fedoraproject.org/wiki/Test_Day:2011-10-06_Printing
  7. ↑ http://fedoraproject.org/wiki/Test_Day:2011-10-13_Fedora_Packager_for_Eclipse
  8. ↑ http://fedoraproject.org/wiki/Test_Day:2011-10-20_Cloud_SIG_Test_Day
  9. ↑ http://fedoraproject.org/wiki/QA/SOP_Test_Day_management
  10. ↑ http://fedoraproject.org/wiki/QA/Test_Days/Create

Fedora 16 preparation

As mentioned above, Fedora 16 release validation took up almost all of the QA team's time during the last few months, with very challenging Beta and Final releases. There were a total of 12 candidate builds for Beta and Final combined, and the whole team put in tireless work running the set of validation tests against each build and then investigating and verifying the large number of blocker bugs identified. The team was able to contribute to the release eventually going ahead with only a one week slip to the Beta schedule and no slip of the Final schedule, a considerable achievement in the light of the many complex changes in the Fedora 16 feature list.


Release criteria updates

Largely as a result of the Fedora 16 validation process, there were several adjustments and additions to the release criteria in recent weeks. After discussion of the proposed kickstart / unattended installation release criterion concluded, Adam Williamson reported that he had committed his proposed modifications[1]. He also committed a change to reflect the increased priority of EFI installations from Fedora 17 onwards[2].

Adam also passed on a suggestion from Peter Jones to improve the clarity of the virtualization criteria[3]. After an extensive discussion, an elegant wording suggestion from Albert Graham[4] was eventually accepted and committed[5].

Tim Flink raised the question to what extent support for Xen virtualization should be included in the release criteria[6]. After a similarly enthusiastic discussion, it was eventually agreed that Xen DomU support - effectively, the ability to install successfully as a Xen guest - should be a Final release criterion[7].

Adam also proposed downgrading some rarely-used kickstart deployment methods from Beta to Final in the criteria, requiring only the most commonly-used to be working at Beta stage[8].

Finally, Adam proposed a criterion for i18n (translation) issues[9]. After discussion, the proposal was agreed upon at a blocker review meeting later in the week[10].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102599.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102600.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102601.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102636.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103680.html
  6. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/103127.html
  7. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103678.html
  8. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103557.html
  9. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103588.html
  10. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103679.html

Update policy changes

In September, Karsten Hopp raised the issue of a security update for Fedora 14 which had been languishing in the updates-testing repository for some time[1]. Adam Williamson explained that the amount of testers working on older releases was limited, and that the actual karma requirements for updates to be accepted were controlled by FESCo (the Fedora engineering steering committee), not the QA group[2]. Chuck Anderson noted that he had the update in question installed, but was struggling for lack of information on how to test it properly[3]. Rahul Sundaram suggested that Karsten file a ticket with FESCo to raise the issue[4], and Karsten did[5].

That ticket was merged with another similar one reported by Doug Ledford[6], which became a topic of concern to FESCo. After several rounds of discussions, FESCo first decided to relax the requirements for critical path updates somewhat by allowing them to be sent through to the stable repository without the 'required' karma after a period of two weeks had elapsed[7], and later proposed removing the requirement for critical path updates to receive positive karma from a proven tester[8], effectively a proposal to end the proven tester system, as this is the only function it serves.

The QA group discussed this proposal at the weekly meeting of 2011-11-07[9], agreeing that, while they had some reservations about the proposal, they were not definitely opposed to it, and recognized that critical path updates not receiving the currently-required karma is a significant problem.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102493.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102497.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102503.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102502.html
  5. ↑ http://fedorahosted.org/fesco/ticket/664
  6. ↑ http://fedorahosted.org/fesco/ticket/667
  7. ↑ http://fedorahosted.org/bodhi/ticket/642
  8. ↑ http://fedorahosted.org/fesco/ticket/667#comment:26
  9. ↑ http://fedoraproject.org/wiki/QA/Meetings/20111107

Update candidate notification

Samuel Greenfeld asked if there was any system to notify testers of new candidate updates for specific packages, and to determine what packages are being actively used on a system[1]. There were no takers for the second question, but for the first, Adam Williamson suggested using yum parameters that would allow one to specify only certain packages be pulled from the updates-testing repository[2], and Till Maas pointed out that Bodhi can actually provide per-package RSS notifications[3].

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102981.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102982.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102992.html

Proven tester meetings

As a response to the concerns about candidate updates not receiving enough karma, Kevin Fenzi ran a series of weekly proven tester meetups[1] from 2011-09-21 to 2011-10-26. Recaps of these meetings are available in the mailing list archives[2] [3] [4] [5] [6].

Kevin also proposed an updates-testing-info mailing list, containing only the mails about new packages in updates-testing[7]. However, the consensus was against the idea, as it was felt that it was easy enough to simply filter the desired mails from the test mailing list for those who did not want to read the other traffic.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/102869.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/103000.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103341.html
  4. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103585.html
  5. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103840.html
  6. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/104043.html
  7. ↑ http://lists.fedoraproject.org/pipermail/test/2011-September/103163.html

QA group representation at FUDCon Pune

Ankur Sinha asked whether anyone from the QA team would be present at the upcoming FUDCon in Pune, India and able to do a presentation on the group's activities[1]. Adam Williamson replied that unfortunately none of the Red Hat team would be at the conference, but encouraged Ankur to take a shot at giving a presentation himself[2]. A S Alam then stepped up to volunteer to lead a QA session[3]. His session was scheduled for 2011-11-04[4], but we have no report on the event - if you were present, please write to the mailing list and let us know how it went!

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103712.html
  2. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103728.html
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2011-October/103739.html
  4. ↑ http://fudcon.in/sessions/fedora-testing
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=261435"