From Fedora Project Wiki

< FWN‎ | Beats

(qa beat for 212)
(qa beat for 213)
Line 10: Line 10:
=== Test Days ===
=== Test Days ===


Last week's Test Day was on the introduction of NFSv4 by default in Fedora 13<ref>http://fedoraproject.org/wiki/Features/NFSv4Default</ref>. The NFS maintainer, [[User:stevedSteve Dickson]], was kind enough to provide several automated test suites, and a good turnout of testers ran them on a variety of NFS configurations, providing valuable results.
There was no Test Day last week. Next week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-02-18_Color_management</ref> will be on color management<ref>http://fedoraproject.org/wiki/Features/ColorManagement</ref>. This significant new feature for Fedora 13 makes it easy to set up color profiles for monitors, scanners and printers, meaning you can rely on seeing the true colors of images from production all the way through to final printing. This is particularly significant for photographers and designers, but a correct color profile can improve anyone's desktop experience, so please come along to make sure the new color management tools get tested on a wide range of hardware. We would particularly appreciate testing from anyone with access to a colorimeter. The Test Day will run all day on Thursday 2010-02-18 in the #fedora-test-day IRC channel.


No Test Day is currently planned for this week. If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.


<references/>
<references/>
Line 18: Line 18:
=== Weekly meetings ===
=== Weekly meetings ===


The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2010-02-01. The full logs are available<ref>http://fedoraproject.org/wiki/QA/Meetings/20100201</ref>. [[User:Jlaska|James Laska]] reported that [[User:Beland|Christopher Beland]] had added links to the Bugzilla common_bugs queries to the Fedora 13 common bugs page<ref>http://fedoraproject.org/wiki/Common_F13_bugs</ref>.
The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2010-02-08. The full logs are available<ref>http://fedoraproject.org/wiki/QA/Meetings/20100208</ref>. [[User:Jlaska|James Laska]] and [[User:Kevin|Kevin Fenzi]] arranged to update the list of RSS feeds tracked by the QA IRC bot, zodbot.


[[User:Kevin|Kevin Fenzi]] promised to look at updating the QA IRC bot, zodbot, to monitor the Fedora 13 blocker bugs instead of the Fedora 12 ones.
[[User:Jlaska|James Laska]] reported that [[User:Wwoods|Will Woods]] and [[User:Kparal|Kamil Paral]] had arranged a meeting the next day to discuss plans for the AutoQA results database. James also mentioned he had not had time to discuss updating the QA calendar with [[User:poelstra|John Poelstra]].


[[User:Wwoods|Will Woods]] and [[User:Kparal|Kamil Paral]] reported that they had not yet discussed design ideas for the proposed AutoQA results database. Kamil felt that more people should be involved as the project would become important to several groups, if implemented. [[User:Jlaska|James Laska]] suggested a micro-FAD<ref>http://fedoraproject.org/wiki/FAD</ref>. Kamil mentioned that [[User:Lmacken|Luke Macken]] had suggested looking at his Kobo project<ref>http://fedorahosted.org/kobo/</ref> for inspiration. [[User:jkeating|Jesse Keating]] felt it might be over-engineered for the purposes of AutoQA. Will and Kamil agreed to organize a meeting during the week to begin designing the system.
[[User:Jlaska|James Laska]], [[User:jkeating|Jesse Keating]] and [[User:Adamwill|Adam Williamson]] discussed the planned Wiki page for 'last known good' Rawhide builds. Jesse clarified the process from the release engineering side, and James planned to document everything in a trac ticket to track the creation of the Wiki page.


[[User:Adamwill|Adam Williamson]] reported that he had moved the privilege escalation policy discussion to the development mailing list<ref>http://lists.fedoraproject.org/pipermail/devel/2010-January/129978.html</ref>, and would go through the same feedback/revision cycle there as he had on the test list before finally escalating the draft policy to FESco.
[[User:Jlaska|James Laska]] noted that the third automated Rawhide acceptance test point had passed the previous week, and had been marked as a failure due to a bug<ref>http://bugzilla.redhat.com/show_bug.cgi?id=559290</ref> resulting in high memory requirements for installation.


[[User:Adamwill|Adam Williamson]] reported on his and [[User:Rhe|Rui He's]] progres with documenting installation testing as a QA group activity. They had finalized the draft installation validation testing page<ref>http://fedoraproject.org/wiki/QA:Installation_validation_testing</ref> and created a draft desktop validation testing page<ref>http://fedoraproject.org/wiki/QA:Desktop_validation_testing</ref>. They had added a paragraph to the Join page<ref>http://fedoraproject.org/wiki/QA/Join#Release_validation</ref> which briefly explains the testing and link to the two more detailed pages. [[User:Jlaska|James Laska]] asked if the QA schedule should be altered to refer to 'release validation' rather than 'installation validation', to leave room for non-installation testing, and Adam said he thought this would be a good idea.
[[User:Wwoods|Will Woods]] was not around to provide an AutoQA project update, but had provided a blog post with some information<ref>http://qa-rockstar.livejournal.com/9234.html</ref>. [[User:Kparal|Kamil Paral]] reported that he had not made any progress on rpmguard this week. [[User:Jlaska|James Laska]] passed on a report on installation automation from [[User:Liam|Liam Li]], who had been discussing the best approach for booting with custom kernel parameters with the virtualization team<ref>http://lists.fedoraproject.org/pipermail/virt/2010-February/001809.html</ref>. On gwt packaging, James reported that he had had feedback from upstream developers about the bundled JARs<ref>http://groups.google.com/group/google-web-toolkit-contributors/browse_thread/thread/fc5713c023919618/f423351722bc7afe</ref>. He was aiming to have one package in progress during the week.  


[[User:Jlaska|James Laska]] reported that another rawhide acceptance testing event had come around during the previous week. This involves the creation and automated testing of a tree complete with installer. This time the acceptance test suite passed, but the installed system was unbootable due to a bug preventing the creation of the initramfs<ref>http://bugzilla.redhat.com/show_bug.cgi?id=559597</ref>. [[User:jkeating|Jesse Keating]] pointed out that this meant the installer was ready for 'last known good' status, but not the package set, and further asked whether the 'last known good' idea is intended to refer only to the installer or also to an associated package tree. There was general agreement that the 'last known good' page should list the tested installer and package tree, and note that the good installer may work with different package trees but could not be guaranteed to. James and [[User:Adamwill|Adam Williamson]] agreed to work on the 'last known good' Wiki page with information provided by Jesse.
[[User:Adamwill|Adam Williamson]] briefly outlined upcoming events, noting the first Alpha test compose was due in the coming week, with installation and desktop validation testing planned. The second Alpha blocker bug meeting was also on the calendar.


[[User:Wwoods|Will Woods]] and [[User:Kparal|Kamil Paral]] provided an AutoQA update. Will had created a working prototype of the dependency checking test, based on yum, which was 147 lines of code and took around 20 seconds to run. [[User:Jlaska|James Laska]] mentioned that he was hoping to see the rpmfluff tool for generating fake test packages become an official Fedora package soon. Kamil went over some updates to rpmguard; he had made it notice when two packages it is asked to compare are identical, and compare a package only to the previous package from stable or updates (not updates-testing).  
The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2010-02-09. The full log is available<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-02-09/fedora-meeting.2010-02-09-15.06.log.html</ref>. [[User:Adamwill|Adam Williamson]] reported that he had sent a draft stock response text for bugs filed against orphaned packages to the mailing list for review<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088369.html</ref>. The group agreed that the draft looked good.


[[User:Jlaska|James Laska]] reported that [[User:Liam|Liam Li]] had added a dvd_install.py script to the autoqa repo<ref>http://fedorahosted.org/pipermail/autoqa-devel/2010-January/000178.html</ref>. He encouraged others to follow up with their thoughts on the script.
[[User:campbecg|Chris Campbell]] reported that he was looking into implementing multi-row button capability for the triage Jetpack script, as having a button for each stock response would result in more buttons than it is currently capable of handling. He said he would give another update next week.


[[User:Jlaska|James Laska]] had continued working on the packaging plan for gwt<ref>http://fedoraproject.org/wiki/User:Jlaska/gwt#JPackage_Dependencies</ref> and hoped to be able to start packaging soon.
[[User:Adamwill|Adam Williamson]] highlighted the upcoming second blocker bug review meeting for Fedora 13 Alpha, and asked group members to elevate any bugs they thought might constitute Alpha blockers. [[User:campbecg|Chris Campbell]] said he wasn't aware of any really serious nouveau bugs at present. Adam said he would check on intel and radeon bugs.


[[User:Kparal|Kamil Paral]] mentioned that he was working on a package update acceptance test plan, and asked the group to provide any information they had on how other projects have approached this issue. [[User:Adamwill|Adam Williamson]] mentioned the Mandriva policies<ref>http://wiki.mandriva.com/en/Policies/SoftwareMedia</ref> <ref>http://wiki.mandriva.com/en/Policies/Support</ref> in the area, and suggested Kamil might talk to [[User:Vdanen|Vincent Danen]], who had managed the update process for Mandriva before joining Red Hat.
[[User:Tk009|Edward Kirk]] reported that he had not yet been able to send out the email to discuss rearranging the weekly meeting time.


The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2010-02-02. The full log is available<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-02-02/fedora-meeting.2010-02-02-15.20.log.html</ref>. [[User:Adamwill|Adam Williamson]] admitted he had done nothing on the subject of bugs filed against orphaned packages.
[[User:Tk009|Edward Kirk]] reported for Juan P. Daza on his update of the active triager list<ref>http://fedoraproject.org/wiki/BugZappers/Components_and_Triagers</ref>. He had emailed all known triagers and found 23 did not respond (while 56 did). None of those 23 were listed on the active triagers page, so no adjustment would be necessary.


[[User:Rudchenkos|Sergey Rudchenko]] told the group about a script he had written to clean up abrt backtraces, providing bug 558883<ref>http://bugzilla.redhat.com/show_bug.cgi?id=558883</ref> as an example. He was interested in extending the script to compute the similarity of any two given backtraces, and also store multiple backtraces offline for comparison. [[User:Adamwill|Adam Williamson]] and [[User:Mcepl|Matej Cepl]] suggested he talk to the abrt team about integrating his ideas into abrt itself.
[[User:Tk009|Edward Kirk]] noted that he was still working on updating the housekeeping procedures for closing blocker tracking bugs for Fedora releases that have already gone out. He expected it would only need some Wiki page updates, and will report next week.


The next QA weekly meeting will be held on 2010-02-08 at 1600 UTC in #fedora-meeting. The next Bugzappers weekly meeting will be held on 2010-02-09 at 1500 UTC in #fedora-meeting.
During the open floor period, it became clear that almost all components on the components list<ref>http://fedoraproject.org/wiki/BugZappers/Components_and_Triagers</ref> were listed as not requiring help. The group thought this was probably not desired, so [[User:Adamwill|Adam Williamson]] and [[User:Tk009|Edward Kirk]] promised to look into it.
 
The next QA weekly meeting will be held on 2010-02-15 at 1600 UTC in #fedora-meeting. The next Bugzappers weekly meeting will be held on 2010-02-16 at 1500 UTC in #fedora-meeting.


<references/>
<references/>


=== Nouveau 3D acceleration testing ===
=== Fedora 13 Alpha test compose ===


[[User:Bruno|Bruno Wolff]] noted<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088305.html</ref> that experimental 3D acceleration for nouveau was now available in the mesa-dri-drivers-experimental package, and asked whether it was yet at a point where the developers would be interested in bug reports. [[User:Adamwill|Adam Williamson]] promised to pass the question along to the nouveau maintainer.
[[User:Rhe|Rui He]] outlined<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088396.html</ref> the upcoming Fedora 13 Alpha test compose validation testing schedule, with the release due 2010-02-11 and desktop and installation validation testing due to run through 2010-02-17. She linked to the installation<ref>http://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Install_Test_Results</ref> and desktop<ref>http://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Desktop_Test_Results</ref> results matrices, and asked group members to contribute results if they could.


<references/>
<references/>


=== Fedora 13 Alpha blocker bug review meeting ===
=== Automated live image testing ===


[[User:Adamwill|Adam Williamson]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088321.html</ref> and later recapped<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088334.html</ref> the first blocker bug review meeting for Fedora 13. The summary of the meeting is available from meetbot<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2010-02-05/alphablocker1.2010-02-05-16.05.html</ref>. All current alpha blocker bugs were reviewed at the meeting.
Jon Jaroker kindly explained<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088436.html</ref> some work he has been doing on implemented ongoing automated testing of the Fedora live desktop image. His goal is to have an automated testing system which tests both the boot process and some basic desktop functionality using automated scripts, and flags up regressions. [[User:Adamwill|Adam Williamson]] thanked Jon for his work<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088437.html</ref> and suggested he co-ordinate with [[User:maxamillion|Adam Miller]] on live image automation, and consider integrating his work into the AutoQA framework<ref>http://fedoraproject.org/wiki/AutoQA</ref>.


<references/>
<references/>

Revision as of 23:57, 14 February 2010

QualityAssurance

In this section, we cover the activities of the QA team[1].

Contributing Writer: Adam Williamson

Test Days

There was no Test Day last week. Next week's Test Day[1] will be on color management[2]. This significant new feature for Fedora 13 makes it easy to set up color profiles for monitors, scanners and printers, meaning you can rely on seeing the true colors of images from production all the way through to final printing. This is particularly significant for photographers and designers, but a correct color profile can improve anyone's desktop experience, so please come along to make sure the new color management tools get tested on a wide range of hardware. We would particularly appreciate testing from anyone with access to a colorimeter. The Test Day will run all day on Thursday 2010-02-18 in the #fedora-test-day IRC channel.

If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[3].

Weekly meetings

The QA group weekly meeting[1] was held on 2010-02-08. The full logs are available[2]. James Laska and Kevin Fenzi arranged to update the list of RSS feeds tracked by the QA IRC bot, zodbot.

James Laska reported that Will Woods and Kamil Paral had arranged a meeting the next day to discuss plans for the AutoQA results database. James also mentioned he had not had time to discuss updating the QA calendar with John Poelstra.

James Laska, Jesse Keating and Adam Williamson discussed the planned Wiki page for 'last known good' Rawhide builds. Jesse clarified the process from the release engineering side, and James planned to document everything in a trac ticket to track the creation of the Wiki page.

James Laska noted that the third automated Rawhide acceptance test point had passed the previous week, and had been marked as a failure due to a bug[3] resulting in high memory requirements for installation.

Will Woods was not around to provide an AutoQA project update, but had provided a blog post with some information[4]. Kamil Paral reported that he had not made any progress on rpmguard this week. James Laska passed on a report on installation automation from Liam Li, who had been discussing the best approach for booting with custom kernel parameters with the virtualization team[5]. On gwt packaging, James reported that he had had feedback from upstream developers about the bundled JARs[6]. He was aiming to have one package in progress during the week.

Adam Williamson briefly outlined upcoming events, noting the first Alpha test compose was due in the coming week, with installation and desktop validation testing planned. The second Alpha blocker bug meeting was also on the calendar.

The Bugzappers group weekly meeting[7] was held on 2010-02-09. The full log is available[8]. Adam Williamson reported that he had sent a draft stock response text for bugs filed against orphaned packages to the mailing list for review[9]. The group agreed that the draft looked good.

Chris Campbell reported that he was looking into implementing multi-row button capability for the triage Jetpack script, as having a button for each stock response would result in more buttons than it is currently capable of handling. He said he would give another update next week.

Adam Williamson highlighted the upcoming second blocker bug review meeting for Fedora 13 Alpha, and asked group members to elevate any bugs they thought might constitute Alpha blockers. Chris Campbell said he wasn't aware of any really serious nouveau bugs at present. Adam said he would check on intel and radeon bugs.

Edward Kirk reported that he had not yet been able to send out the email to discuss rearranging the weekly meeting time.

Edward Kirk reported for Juan P. Daza on his update of the active triager list[10]. He had emailed all known triagers and found 23 did not respond (while 56 did). None of those 23 were listed on the active triagers page, so no adjustment would be necessary.

Edward Kirk noted that he was still working on updating the housekeeping procedures for closing blocker tracking bugs for Fedora releases that have already gone out. He expected it would only need some Wiki page updates, and will report next week.

During the open floor period, it became clear that almost all components on the components list[11] were listed as not requiring help. The group thought this was probably not desired, so Adam Williamson and Edward Kirk promised to look into it.

The next QA weekly meeting will be held on 2010-02-15 at 1600 UTC in #fedora-meeting. The next Bugzappers weekly meeting will be held on 2010-02-16 at 1500 UTC in #fedora-meeting.

Fedora 13 Alpha test compose

Rui He outlined[1] the upcoming Fedora 13 Alpha test compose validation testing schedule, with the release due 2010-02-11 and desktop and installation validation testing due to run through 2010-02-17. She linked to the installation[2] and desktop[3] results matrices, and asked group members to contribute results if they could.

Automated live image testing

Jon Jaroker kindly explained[1] some work he has been doing on implemented ongoing automated testing of the Fedora live desktop image. His goal is to have an automated testing system which tests both the boot process and some basic desktop functionality using automated scripts, and flags up regressions. Adam Williamson thanked Jon for his work[2] and suggested he co-ordinate with Adam Miller on live image automation, and consider integrating his work into the AutoQA framework[3].