From Fedora Project Wiki

< FWN‎ | Beats

(s/https/http/)
(create fwn 231 qa beat)
Line 8: Line 8:
<references/>
<references/>


=== AutoQA initscript testing ===
=== Proven testers ===


[[User:Jskladan|Josef Skladanka]] updated<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091466.html</ref> the status of the automated initscripts test effort<ref>http://fedorahosted.org/autoqa/wiki/initscripts</ref>. He explained that 30% of initscripts had now been reviewed, and again asked for people to help in completing the process.
During the QA weekly meeting of 2010-06-14<ref>http://fedoraproject.org/wiki/QA/Meetings/20100614</ref>, [[User:Adamwill|Adam Williamson]] reported that he had drafted a set of instructions<ref>http://fedoraproject.org/wiki/User:Adamwill/Draft_proventesters_instructions</ref> for proven testers (under the new proven tester policy<ref>http://fedoraproject.org/wiki/QA/JoinProvenTesters</ref>), and also had updated various wiki pages<ref>http://fedoraproject.org/wiki/QA</ref> <ref>http://fedoraproject.org/wiki/QA/Join</ref> <ref>http://fedoraproject.org/wiki/QA/Updates_Testing</ref> to reference the proven testers process. [[User:Jlaska|James Laska]] noted that he was monitoring the ticket<ref>http://fedorahosted.org/bodhi/ticket/424</ref> requesting the infrastructure team to configure Bodhi to require proven tester feedback on critical path updates. Adam subsequently announced his draft on the mailing list<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091513.html</ref>, followed by a second draft<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091552.html</ref>. Aaron Faanes stepped in<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091559.html</ref> with a much-improved revision of Adam's draft<ref>https://fedoraproject.org/wiki/User:Dafrito/Draft_proventesters_instructions</ref>. Adam replied<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091560.html</ref> to thank Aaron for his improvements.
 
=== AutoQA  ===
 
During the QA meeting, [[User:Kparal|Kamil Paral]] reported that the AutoQA team had decided to re-prioritize their goals with the aim of delivering concrete results as soon as possible, even where this meant not immediately meeting the whole range of aims for the project<ref>http://fedorahosted.org/pipermail/autoqa-devel/2010-June/000668.html</ref>. They had decided the current priorities were to test and finish the Bodhi hook, implementing the ability to run potentially dangerous tests in a virtual machine, creating a test instance of the autotest environment for testing new features without breaking the stable instance, getting a publicly accessible machine for the results database (resultdb), and working on resultdb.
 
<references/>
 
=== Critical path wiki update ===
 
[[User:Jlaska|James Laska]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091517.html</ref> that he had revised the critical path documentation on the Wiki. He had created a new page<ref>http://fedoraproject.org/wiki/Critical_Path_Packages</ref> to complement the existing critical path proposal page<ref>http://fedoraproject.org/wiki/Critical_Path_Packages_Proposal</ref>, which had initially been only the proposal of the critical path process but had come to be used as a general reference for the implemented process as well.
 
<references/>
 
=== Kernel triage ===
 
During the Bugzappers weekly meeting of 2010-06-15<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-06-15/bugzappers.2010-06-15-15.01.log.html</ref>, [[User:Tcpip4000|JP]] reported that he had updated the stock responses on the older kernel triage page<ref>http://fedoraproject.org/wiki/KernelBugTriage and BugZappers</ref> to match the style of the newer Bugzappers stock responses<ref>http://fedoraproject.org/wiki/BugZappers/StockBugzillaResponses</ref>, and asked for feedback on the changes.


<references/>
<references/>


=== Fedora 14 QA schedule ===
=== Triage metrics ===


[[User:Poelstra|John Poelstra]] posted<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091492.html</ref> the QA group schedule for Fedora 14<ref>http://poelstra.fedorapeople.org/schedules/f-14/f-14-quality-tasks.html</ref>, including all the significant dates for the team in the run-up to the next Fedora release.
During the Bugzappers meeting, [[User:jraber|Jeff Raber]] reported his progress on the new triage metrics project. He had created a Bugzilla query<ref>http://bugzilla.redhat.com/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=BugsTriaged-Last30Day&sharer_id=292687</ref> which lists bugs triaged in the previous 30 days, and was working on some modifications to python-bugzilla to provide output suited to triage statistics. [[User:Adamwill|Adam Williamson]] promised to put Jeff in touch with [[User:Wwoods|Will Woods]] to discuss merging the python-bugzilla changes.


<references/>
<references/>


=== Virtualized testing ===
=== Setting needinfo on impending end-of-life bugs ===


Bob Lightfoot asked<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091494.html</ref> if there was a consensus on the use of virtual machines as opposed to real systems in testing, and whether it is acceptable to run tests of the install media in virtual machines. Richard Ryniker's well-considered response<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091497.html</ref> pointed out that "just as an error observed on "real" hardware might be attributed to a quirk or fault in that platform, so too an error in a VM might be the result of some bug in the implementation of the VM," and that "errors observed in a VM environment...should be subjected to the same triage process that might elevate them to "critical" status because they seriously impact operation on many (real or virtual) platforms, or reduce them to "future consideration" status because they have little impact, they occur only on platforms rare enough to suggest a quirk or platform fault is their cause". [[User:Adamwill|Adam Williamson]] said<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091501.html</ref> that virtual testing is valuable, but testing on real hardware is also necessary, in both cases.
During the Bugzappers meeting, [[User:Mcepl|Matej Cepl]] suggested that when adding a comment to bugs on releases that will soon go end-of-life, we should also set the needinfo state to mark that additional input is needed to keep the bug open. After some discussion, everyone agreed that this was a good idea.


<references/>
<references/>


=== NSS dependency issue ===
=== Fedora 14 recommendations ===


During the QA weekly meeting of 2010-06-07<ref>http://fedoraproject.org/wiki/QA/Meetings/20100607</ref>, [[User:Adamwill|Adam Williamson]] brought up the problem with dependencies in the nss-softokn package which had caused dependency issues during updates for many users of the 64-bit edition of Fedora 13. The group concluded that there had been no failure in the QA processes, but also agreed that it would be a good idea to make sure the AutoQA dependency checks will be able to catch this particular type of problem when they go live. Adam promised to send [[User:Wwoods|Will Woods]] a summary of the issue for this purpose.
[[User:Jlaska|James Laska]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091538.html</ref> the list of recommendations for the Fedora 14 cycle based on the Fedora 13 QA retrospective<ref>http://fedoraproject.org/wiki/Fedora_13_QA_Retrospective#Recommendations</ref>. He noted that the next task would be to organize the recommendations into a set of trac tickets to track their implementation.


<references/>
<references/>


=== Triage metrics ===
=== Reopening bugs ===


During the Bugzappers weekly meeting of 2010-06-08<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-06-08/bugzappers.2010-06-08-14.59.log.html</ref>, [[User:Adamwill|Adam Williamson]] recapped the previous efforts to produce a system for monitoring the triage process and providing metrics on triage work, and proposed an alternative approach of producing some simple Bugzilla queries that would provide some basic information in the short term and without a lot of complex work. [[User:jraber|Jeff Raber]] stepped in and volunteered to attempt this.
Matt McCutchen brought up the topic<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091541.html</ref> of reopening bugs, specifically the fact that most Bugzilla users can only reopen bugs that they filed (or which are assigned to them). He mentioned that he had filed an RFE asking that all users be given permission to reopen bugs<ref>http://bugzilla.redhat.com/show_bug.cgi?id=573535</ref>. [[User:Adamwill|Adam Williamson]] said<ref>http://lists.fedoraproject.org/pipermail/test/2010-June/091546.html</ref> he would forward the proposal to the Bugzilla maintainers for consideration.


<references/>
<references/>

Revision as of 23:05, 22 June 2010

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

  1. ↑ http://fedoraproject.org/wiki/QA
  2. ↑ http://fedoraproject.org/wiki/QA/Join

Proven testers

During the QA weekly meeting of 2010-06-14[1], Adam Williamson reported that he had drafted a set of instructions[2] for proven testers (under the new proven tester policy[3]), and also had updated various wiki pages[4] [5] [6] to reference the proven testers process. James Laska noted that he was monitoring the ticket[7] requesting the infrastructure team to configure Bodhi to require proven tester feedback on critical path updates. Adam subsequently announced his draft on the mailing list[8], followed by a second draft[9]. Aaron Faanes stepped in[10] with a much-improved revision of Adam's draft[11]. Adam replied[12] to thank Aaron for his improvements.

AutoQA

During the QA meeting, Kamil Paral reported that the AutoQA team had decided to re-prioritize their goals with the aim of delivering concrete results as soon as possible, even where this meant not immediately meeting the whole range of aims for the project[13]. They had decided the current priorities were to test and finish the Bodhi hook, implementing the ability to run potentially dangerous tests in a virtual machine, creating a test instance of the autotest environment for testing new features without breaking the stable instance, getting a publicly accessible machine for the results database (resultdb), and working on resultdb.

  1. ↑ http://fedoraproject.org/wiki/QA/Meetings/20100614
  2. ↑ http://fedoraproject.org/wiki/User:Adamwill/Draft_proventesters_instructions
  3. ↑ http://fedoraproject.org/wiki/QA/JoinProvenTesters
  4. ↑ http://fedoraproject.org/wiki/QA
  5. ↑ http://fedoraproject.org/wiki/QA/Join
  6. ↑ http://fedoraproject.org/wiki/QA/Updates_Testing
  7. ↑ http://fedorahosted.org/bodhi/ticket/424
  8. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091513.html
  9. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091552.html
  10. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091559.html
  11. ↑ https://fedoraproject.org/wiki/User:Dafrito/Draft_proventesters_instructions
  12. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091560.html
  13. ↑ http://fedorahosted.org/pipermail/autoqa-devel/2010-June/000668.html

Critical path wiki update

James Laska announced[1] that he had revised the critical path documentation on the Wiki. He had created a new page[2] to complement the existing critical path proposal page[3], which had initially been only the proposal of the critical path process but had come to be used as a general reference for the implemented process as well.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091517.html
  2. ↑ http://fedoraproject.org/wiki/Critical_Path_Packages
  3. ↑ http://fedoraproject.org/wiki/Critical_Path_Packages_Proposal

Kernel triage

During the Bugzappers weekly meeting of 2010-06-15[1], JP reported that he had updated the stock responses on the older kernel triage page[2] to match the style of the newer Bugzappers stock responses[3], and asked for feedback on the changes.

  1. ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-06-15/bugzappers.2010-06-15-15.01.log.html
  2. ↑ http://fedoraproject.org/wiki/KernelBugTriage and BugZappers
  3. ↑ http://fedoraproject.org/wiki/BugZappers/StockBugzillaResponses

Triage metrics

During the Bugzappers meeting, Jeff Raber reported his progress on the new triage metrics project. He had created a Bugzilla query[1] which lists bugs triaged in the previous 30 days, and was working on some modifications to python-bugzilla to provide output suited to triage statistics. Adam Williamson promised to put Jeff in touch with Will Woods to discuss merging the python-bugzilla changes.

  1. ↑ http://bugzilla.redhat.com/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=BugsTriaged-Last30Day&sharer_id=292687

Setting needinfo on impending end-of-life bugs

During the Bugzappers meeting, Matej Cepl suggested that when adding a comment to bugs on releases that will soon go end-of-life, we should also set the needinfo state to mark that additional input is needed to keep the bug open. After some discussion, everyone agreed that this was a good idea.


Fedora 14 recommendations

James Laska announced[1] the list of recommendations for the Fedora 14 cycle based on the Fedora 13 QA retrospective[2]. He noted that the next task would be to organize the recommendations into a set of trac tickets to track their implementation.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091538.html
  2. ↑ http://fedoraproject.org/wiki/Fedora_13_QA_Retrospective#Recommendations

Reopening bugs

Matt McCutchen brought up the topic[1] of reopening bugs, specifically the fact that most Bugzilla users can only reopen bugs that they filed (or which are assigned to them). He mentioned that he had filed an RFE asking that all users be given permission to reopen bugs[2]. Adam Williamson said[3] he would forward the proposal to the Bugzilla maintainers for consideration.

  1. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091541.html
  2. ↑ http://bugzilla.redhat.com/show_bug.cgi?id=573535
  3. ↑ http://lists.fedoraproject.org/pipermail/test/2010-June/091546.html
Retrieved from "https://fedoraproject.org/w/index.php?title=FWN/Beats/QualityAssurance&oldid=181317"