This page is intended to gather feedback from the Fedora QA community on things that worked well and things that could have been better with the testing of Fedora 12. The feedback will be used as a basis for identifying areas for improvement for Fedora 13 testing. Any thoughts, big or small, are valuable. If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.
For any questions or concerns, send mail to firstname.lastname@example.org.
Adding feedback is fairly straight forward. If you already have a Fedora account ...
- Login to the wiki
- Select [Edit] for the appropriate section below.
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Otherwise, if you do not have a Fedora account, follow the instructions below ...
- Select the appropriate page for your feedback...
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Things that went well
- jlaska - Having QA/SOP_Test_Day_management was invaluable and resulted in a consistent look'n'feel for Fedora 12 test days
- jlaska - The creation of the email@example.com mailing list made it much easier to announce test events. With a single mailing list for announcements, it's harder to miss any events.
- jlaska - Consistent announcement of ISO test events from Liam was great to ensure a consistent testing experience (see User:Liam/Draft_Install_Test_SOP).
- jlaska - We had more ISO install test runs, and they were earlier in each milestone
- jlaska - F-12 ISO Install testing solicited more testing from outside the core install test group. 27 different people contributed test feedback on the wiki test matrix during F-12. . 19 different people contributed test feedback on the wiki test matrix during F-11.
- jlaska - Automated test cases provided for the Power Management test day == awesome!
- Viking-Ice - Increase in SIG and maintainers for hosting and holding their own test days and specific application testing.
- Viking-Ice - Nightly-composes of rawhide test images.
- Viking-Ice - Improvement in advertisement testing/test day's
- Viking-Ice - Increase in request from sig/maintainers in Fedora QA Trac system.
- Viking-Ice - Increase in how to debug <component> for reporters to follow and provide better bug report.
- Viking-Ice - Consistency in how to debug <component> pages for better look and feel.
- Viking-Ice - Automatic bug reporting tool develop to help provide better bug reporting and allow technical challenge people to participate report and provide feedback.
- Viking-Ice - Better collaboration and participation from developers/maintainers with QA.
- He Rui - The SOP management of both test days and install tests
- He Rui - Traceback can be saved or reported automatically
- He Rui - Common f12 bugs page
- He Rui - Tickets could be created and traced in Trac system
- Adam Williamson - test days went well again. it was nice to see how many independent test days there were.
Could have been better
uname -a hera.localdomain 220.127.116.11-174.2.3.fc12.x86_64 #1 SMP Mon Jan 18 19:52:07 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux language brazilian portuguese
this works for me ls -l *.rpm
look at "./" preceding the char "*" this works for me ls -l ./*.pps
this does not works for me
ls -l *.pps ls: invalid option -- 'P' Experimente "ls --help" para mais informações.
I think this is a bug
- jlaska - it may be, but I believe you might want to confirm what the asterix is expanding to in the directory you are running the command
ls -l *.pps. It's possible it is expanding to a filename that has -P as the first 2 characters, which confuses the ls command. What does
echo *.ppsyield? You may wish to enclose the command in quotes,
ls -l "*.pps". If problems remain, please raise the issue on the users or test list.
- jlaska - The blocker bug meetings during the Final milestone were too long.
- Kparal - The preupgrade could receive more testing/test cases
- Kparal - test-announce wasn't used for announcing RC candidates IIRC, we should make sure all important events are mentioned there
- He Rui - Post-event review python could be used for install test like in test day
- He Rui - Transfer from manual-installation to Auto-installation test
- He Rui - Test days/Install test events could be put up obviously on wiki so that people who wanna contribute can find it easily from wiki except from mail lists.
- He Rui - 'Edit' on wiki could be more automatically by filling in and submitting a form without wiki syntax.
- Adam Williamson - in the end, we focused heavily on just three component areas for testing: anaconda, kernel, and X.org. This is primarily a function of the fact that these are the most vital bits; it feels like we're still at the point of doing let's make sure it's not totally broken testing than let's make sure it's really good testing. We didn't do stuff like making sure the desktop was polished.
- Matthias Clasen - I agree with this assessment. I would go even further and say it is to a large extent 'lets make sure the installer is not totally broken for exotic cases' testing.
- Adam Williamson - there was clearly a lot of uncertainty about RAID issues; it's something we obviously don't as a team test well enough (and some of us personally don't understand enough :>). In the end there didn't turn out to be any horrible issues, but the confusion was evident, and we did miss the Intel BIOS RAID stuff-ups. For F13 we should have better RAID testing both in Test Days and in pre-release test cycles.
- Adam Williamson - We weren't completely on top of X.org bugs for this release. The ones that wound up getting promoted to release blocker level were kind of an arbitrary selection. I think we got nouveau mostly right as I had a reasonable grip on nouveau triage, but we just had too few people to triage server / intel / ati bugs during the cycle, so when we hit beta / RC stage, we didn't have the whole bug set well enough triaged to be able to be sure we picked the most important bugs as blockers. For F13 we should stay on top of triage better so we can do blocker identification accurately. happily, matej is more active on X triage again now and we have some more assistance from Chris Campbell (thank you Chris!) further volunteers would be great. I will try to stay on top of nouveau, again.
- Adam Williamson - we have the big security thing (aka lack of a defined security policy and test plan) to deal with. I did start a thread on fedora-devel-list about that.
- Matthias Clasen - From my perspective, the two main avenues to a new Fedora release are the live installer and preupgrade, and those two should get all the attention they can get.
- Rahul Sundaram - IMO, RCs needs to advertised loudly. We need all the testing we can get and more alpha or beta snapshots would help as well, I think.
- Rahul Sundaram - PackageKit signed install policy was a major issue and although QA was not really responsible for it, the team does need to do it all can do to avoid anything like this in the future . Signing Rawhide packages automatically would have caught this. Not many users in forum or fedora-list complained about it however.
- Rahul Sundaram - Lot more users are trying Preupgrade and QA needs strong focus on the upgrade story including test days directed towards it. The small /boot is a major issue and another common bug (not listed in the wiki but I think needs to be added) is https://bugzilla.redhat.com/show_bug.cgi?id=538118 which makes the problem worse. We really need to make sure Anaconda creates a bigger /boot for Fedora 13, preupgrade is explicitly tested well and has solid workarounds for the small /boot case.
- Rahul Sundaram - KMS is still flaky in some cases. In particular, a few Intel users seem to be reporting lower resolution by default without nomodeset and ATI performance seems to have regressed. Although I am no fan of proprietary drivers, I must note that installing the proprietary Nvidia driver has become a bit more of a hassle.
- Scott Robbins - Although it's not much of a factor in the US, apparently bandwidth limits are quite common in Australia--there were several who seemed rather disgruntled about the somewhat confusing SHA1/SHA256 sum issue. Although it's now clear in the release notes, those who verify, as a rule, have done it before with other distributions, and are used to downloading an ISO, looking at the sums file, and seeing that it's md5 (if anyone's still using it), or SHAwhatever. Upon getting what seems to be a bad checksum, they download again, which in fairness, if you get one bad checksum, is probably the logical thing to do.
- Jim Haynes - I've learned it is necessary to install foomatic-db to have my particular printer show up in system->administration->printing. According to Adam Williamson, This has been documented in common bugs, but it does seem like something we ought to have caught - thanks for bringing it up. Perhaps we need more testing of common basic peripherals like printers.
- CAI Qian - Power Management Test Day is boring, because it provides a automated scripts to run, and not sure how to inspect that data from the testers. As the results, it lose the fun to hunt bugs. It is important to have some instructions to inspect the data we received in the future, so we can see if there any regression or issues.
- 18.104.22.168 - I would like a pony
- jlaska - I also want a pony
- Kparal - I would like to see more people in beta/RC testing. New templates are going to solve the practical part (display multiple results easily), now the issue is to attract more people. I would like to visit get-fedora page and see there "Fedora 13 Release Candidate available! Download <<here>>, help with testing <<here>>". And links to our wiki pages describing how to contribute on testing. I think Ubuntu is doing similar thing (large banners "Beta available" on their front page). That could bring a lot of new people.
- Kparal - Searching through our wiki I could not find a page named "Fedora 12", "Fedora 13", etc. Simply a page summarizing (linking) everything important about a release - schedules, features, current progress, (release notes). It's scattered in pages like these and these, but no page is standing out as an "outpost" of that particular release. I imagine if we have some page like that, we could add there links about the current testing efforts, so people would be easily navigated exactly where we need them to help ("current QA efforts: Fedora 13 RC2 testing"...). This "Fedora 13" overview page would of course be referenced from main wiki page, so people can easily find it.
- He Rui - More Asian testers could participate in the test events. (could inform them by forwarding announcements to their mail list/irc in native language)
- Jim Haynes - Over all this is the easiest Fedora version upgrade I have ever done; and when I did have problems I got quick and correct answers from this list (firstname.lastname@example.org) that I was not getting anywhere else. (Maybe that suggests we should have a current-version-install ombudsman list, where anybody can present questions but only authoritative people can give answers.)
- Kparal - People attending Test Days presentation asked which sources are most attractive for Test Day announcements and allure many people. Where certainly not to forget to advertise when arranging a test day.
- Kparal - People attending Test Days presentation asked if test day results are presented and published externally, like in magazines, etc.
- Kparal - People attending Test Days presentation asked if Fedora Test Days are on Facebook and Twitter, which could attract larger audience.
After enough feedback has been collected, the QA team will discuss and make recommendations on changes for Fedora 13. This section will be filled in at that time.