From Fedora Project Wiki

(creating initial F15 retrospective page)
 
(→‎Could have been better: Release note memory requirements)
 
(22 intermediate revisions by 5 users not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==


This page is intended to gather feedback from the Fedora community on things that worked well and things that could have been better during the Fedora 15 release cycle, including (X, Y, Z).  The feedback will be used as a basis for identifying areas for improvement for Fedora 16.  Any thoughts, big or small, are valuable.  If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.
This page is intended to gather feedback from the Fedora community on things that worked well and things that could have been better during the Fedora 15 release cycle, including (shifting the feature freeze until after FUDCon, NetworkManager 0.9, and so on).  The feedback will be used as a basis for identifying areas for improvement for Fedora 16.  Any thoughts, big or small, are valuable.  If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.
 
The [[QA]] team has their own retrospective page.  If you have specific feedback related to testing and/or QA, please visit the [[Fedora_15_QA_Retrospective | Fedora 15 QA Retrospective]] page.


== Providing feedback ==
== Providing feedback ==
* [[User:Gwjasu|Gwjasu]] - I like ____ about the new ____ process
* [[User:Gwjasu|Gwjasu]] - I like ____ about the new ____ process
Adding feedback is fairly straight forward.  If you already have a [https://admin.fedoraproject.org/accounts Fedora account] ...
Adding feedback is fairly straight forward.  If you already have a [https://admin.fedoraproject.org/accounts Fedora account] ...
# Login to the wiki
# Login to the wiki
# Select ''[Edit]'' for the appropriate section below.
# Select ''[Edit]'' for the appropriate section below.
# Add your feedback using the format: <pre>* ~~~ - I like ____ about the new ____ process</pre>
# When done, ''Submit'' your changes
Otherwise, if you do not have a [https://admin.fedoraproject.org/accounts Fedora account], follow the instructions below ...
# Select the appropriate page for your feedback...
#* [http://fedoraproject.org/w/index.php?title=Test_Results:Fedora_14_QA_Retrospective/good&action=edit Something that worked well]
#* [http://fedoraproject.org/w/index.php?title=Test_Results:Fedora_14_QA_Retrospective/bad&action=edit Something that didn't work well]
#* [http://fedoraproject.org/w/index.php?title=Test_Results:Fedora_14_QA_Retrospective/wishlist&action=edit Anything on your QA wishlist]
# Add your feedback using the format: <pre>* ~~~ - I like ____ about the new ____ process</pre>
# Add your feedback using the format: <pre>* ~~~ - I like ____ about the new ____ process</pre>
# When done, ''Submit'' your changes
# When done, ''Submit'' your changes
Line 22: Line 17:


=== Things that went well ===
=== Things that went well ===
{{Test_Results:Fedora_15_QA_Retrospective/good}}
# [[User:Jsmith|Jared Smith]] - ''BoxGrinder'' - I was very happy to see the BoxGrinder feature work well in Fedora 15, and even happier to see the efforts put into starting to get the packaging for parts of JBoss to a point where they can be included in Fedora.
# [[User:Jlaska|jlaska]] - ''Mailing list'' - Seeing an increased level of mailing list traffic discussing test issues with systemd and gnome3.  This is pretty exciting and exactly the type of communication I'd love to see on the list.  Testers helping testers identify and triage issues.
# [[User:Bruno|Bruno Wolff]] - ''slips'' - In sort of a mixed bag category, I thought the extra time provided by the two one week slips ended up with us releasing a better product. We had several ambitious features and the extra time seemed to be helpful for polishing them.
# [[User:Jlaska|jlaska]] - ''Beta'' - Because of the FESCO approval for the late arriving NM-0.9 changes (refer to [https://fedorahosted.org/fesco/ticket/572 FESCO#572), QA held firm that the schedule '''must''' slip one week.  At the time of the decision, 2 test milestones were missed (pre-beta acceptance, TC1).  Attempting to absorb the late changes, and the bugs introduced by those changes, would have negatively impacted the release and '''definitely''' resulted in a slip regardless.  It was the right decision to slip the release after accepting the late NM-0.9 change.
# [[User:Bruno|Bruno Wolff]] - ''packaging'' - This was the first release that used git for packaging all the way through. I have found it makes backporting updates easier than when we were using CVS.
# [[User:Bruno|Bruno Wolff]] - ''spins'' - The timing of XZ support for squashfs getting into the kernel was fortuitous as we would have had to either make hard decisions about cuts to keep several live spins CD sized or decide to use images that wouldn't fit on CDs.


=== Could have been better ===
=== Could have been better ===
{{Test_Results:Fedora_15_QA_Retrospective/bad}}


# [[User:Jlaska|jlaska]] - ''Alpha-TC wasn't branched'' - F15 Alpha test compose was created using pre-branched content. This was unexpected, but it appears that there is no clear guidance/documentation on what is expected (see https://fedorahosted.org/rel-eng/ticket/4399)
# [[User:Jsmith|Jared Smith]] -- ''NetworkManager bump'' - The move to NetworkManager 0.9 came after the feature freeze, and the Sugar desktop wasn't accounted for in the scramble to get alternate desktop environments working with it.
# [[User:jsmith|jsmith]] - ''Mass rebuild not included in schedule'' - The mass rebuild timing wasn't ideal, it landed at the same time of the branch.  While release engineering did an outstanding job resolving rebuild issues in a timely manner, the schedule didn't account for the mass rebuild, and test composes were delayed.
# [[User:Jsmith|Jared Smith]] -- ''artwork'' - There seemed to be a lack of consensus from the Design team on whether or not the artwork was considered ''ready'' or not
# [[User:jlaska|jlaska]] - ''insufficient live testing'' - Because of {{bz|672265}} and {{bz|676904}} we didn't get a lot of testing on the live images prior to, and during the Alpha test composes.  As a result, once those issues were fixed, we found 3 additional Alpha blockers
# [[User:Jsmith|Jared Smith]] -- ''systemd'' - The Fedora Packaging Committee had a hard time getting specifics from the systemd developers in a timely manner, which caused difficulties in creating packaging guidelines with respect to systemd
#* {{bz|679107}} - TypeError: argument 2 to map() must support iteration
# [[User:Jsmith|Jared Smith]] -- ''marketing'' - The Fedora Project Leader left the Red Hat press release and web copy until the last minute, and it should have been done much earlier.
#* {{bz|663294}} - RuntimeError: XOpenDisplay failed }} - TypeError: argument 2 to map() must support iteration
# [[User:Pfrields|Paul Frields]] -- ''marketing'' - There didn't seem to be the usual number of interviews and feature profiles coming out around the release, which was even more surprising since the look and feel of the release with GNOME 3 was so visibly different.  This was a lost opportunity to partner well with a major upstream where we have a lot of participants producing new code.
#* {{bz|672030}} - AttributeError: 'NoneType' object has no attribute 'format' }} - TypeError: argument 2 to map() must support iteration
# [[User:Jsmith|Jared Smith]] -- ''translation'' - We switched from a self-hosted version of Transifex 0.8 to a hosted instance at transifex.net mid-way through the F15 release cycle.  This caused many difficulties for the translation team, and I'm not sure how well the developers and packagers in Fedora understood the impact of the changes.
# [[User:jlaska|jlaska]] - ''missed first 2 blocker review meetings'' - Not sure what else to say ... we forgot about these meetings.  With FUDCon and PTO, the first two completely fell off my radar.
# [[User:Bruno|Bruno Wolff]] -- ''spins'' - The Spins SIG leader got busy (with work I think) and we ended up making rushed (though reasonable) decisions (after a number of meetings didn't happen) on which spins should be produced.
# [[User:jlaska|jlaska]] - ''i18n'' - i18n/l10n continues to highlight problems early on in the release.  F-15-Alpha '''slipped''' one week due to {{bz|676827}} (keyboard with german layout doesn't work in gdm).  We need release criteria for LANG/keymap issues.
# [[User:Bruno|Bruno Wolff]] -- ''hardware requirements'' - The amount of memory needed to install F15 increased significantly. There is a plan to reduce this for F16, but there wasn't time to do it for F15.
# [[User:jlaska|jlaska]] - ''Missing criteria'' - When proposing and publishing release criteria, use caution when making changes. Particularly when release criteria pages are already available for the next release.  You '''must''' be sure that any changes made to {{FedoraVersion||next}} are copied forward to {{FedoraVersion||next2}}.  This was problem for Fedora 15 Alpha with the desktop artwork criteria.  The artwork criteria was added to the Fedora 14 Alpha criteria page, but the Fedora 15 Alpha criteria page was already created.  The new criteria were not carried forward, and Fedora 15 Alpha released with the incorrect artwork.  As a result, a bug was filed, but not escalated.
# [[User:duffy|Máirín Duffy]] -- ''artwork'' - not having separate wallpapers per desktop environment would have been better, caused a lot of confusion in what artwork to use in producing collateral materials for the release and a burden on team
#* {{bz|677080}} - 'F14' artwork is shown during F-15 installation
# [[User:duffy|Máirín Duffy]] -- ''artwork'' - some kind of metrics for artwork quality as part of QA would be good
#* Also, the [http://rbergero.fedorapeople.org/schedules/f-15/f-15-design-tasks.html f15 design schedule] does not have a task to update fedora-logos with Alpha installer artwork.
# [[User:Jsmith|Jared Smith]] -- Amazon EC2 images were not prepared for release time. (Still trying to learn exact details of what held them up.)
# [[User:Rhe|Rhe]] 06:50, 14 March 2011 (UTC) - VNC test - Add password test for VNC to current test run since the bug was found:
# [[User:Gholms|Garrett Holmstrom]] -- ''release notes'' - At release time the release notes had not yet been updated to reflect the greater amount of memory needed to install with install media.
#* {{bz|678150}} - VNC install w/ password, fails to establish password for VNC session
# [[User:Vhumpa|vhumpa]] - GNOME 3 Test Day attendance - for some reason, for this #2 testing event, significantly less testers showed up: 19 compared to over 50 for #1. Still - a nice number of bugs have been encountered, 1.58 bugs per tester at #2 compared to 1.28 at #1, which is due to a larger number of tests this time (25 vs. 16). Although a test day was finally a success, it might be useful to reason about the lower turnout. Here are some ideas, but please note that they are just speculations:
#* Generally, there might have been less simply curious people as they'd already get the curiosity satisfied by #1
#* At the last moment, we wanted to add some of the latest updates that were not yet available in repositories, which caused:
#** We weren't sure if we wouldn't actually want to move the event, so perhaps notifications were sent a little late
#*** I have no data on this - but perhaps also less announce channels were used this time
#** We couldn't use the nightly composes as previously planned and had to make custom images - which (as Adamw's sleepless nights confirm) brings trouble and takes time. Particularly the i386 image wasn't available until the late Test Day, which could have driven away some of the testers
#* A significant number of test cases could also have discouraged some testers with less time. We had to divide the tests in two sections/tables and majority of the testers did go through just the first part - which can be seen as a little proof of this.
# [[User:jlaska|jlaska]] - ''Beta'' - Missed the 2011-03-17 - F-15 pre-beta acceptance milestone (https://fedorahosted.org/rel-eng/ticket/4530)
#* Some nasty glibc change caused an anaconda failure - (see https://bugzilla.redhat.com/show_bug.cgi?id=684742)
#* Delays in landing NetworkManager-0.9 resulted in anaconda build failures (https://fedorahosted.org/fesco/ticket/572)
# [[User:Jlaska|jlaska]] - ''Beta'' - Slipped release 1 week due to decision by FESCO (refer to [https://fedorahosted.org/fesco/ticket/572 FESCO#572]) to accept late NetworkManager-0.9 changes which impacted *all* desktop environments.  QA felt strongly that the right decision was to slip the release one week and not attempt to absorb the changes given that we already missed 2 test milestones (pre-beta acceptance and beta TC1). Could the late arriving changes have been seen/predicted earlier?  Ticket filed on 03/10/11


=== Wishlist ===
=== Wishlist ===

Latest revision as of 17:49, 7 June 2011

Introduction

This page is intended to gather feedback from the Fedora community on things that worked well and things that could have been better during the Fedora 15 release cycle, including (shifting the feature freeze until after FUDCon, NetworkManager 0.9, and so on). The feedback will be used as a basis for identifying areas for improvement for Fedora 16. Any thoughts, big or small, are valuable. If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.

The QA team has their own retrospective page. If you have specific feedback related to testing and/or QA, please visit the Fedora 15 QA Retrospective page.

Providing feedback

  • Gwjasu - I like ____ about the new ____ process

Adding feedback is fairly straight forward. If you already have a Fedora account ...

  1. Login to the wiki
  2. Select [Edit] for the appropriate section below.
  3. Add your feedback using the format: 
    * ~~~ - I like ____ about the new ____ process
  4. When done, Submit your changes

Feedback

Things that went well

  1. Jared Smith - BoxGrinder - I was very happy to see the BoxGrinder feature work well in Fedora 15, and even happier to see the efforts put into starting to get the packaging for parts of JBoss to a point where they can be included in Fedora.
  2. Bruno Wolff - slips - In sort of a mixed bag category, I thought the extra time provided by the two one week slips ended up with us releasing a better product. We had several ambitious features and the extra time seemed to be helpful for polishing them.
  3. Bruno Wolff - packaging - This was the first release that used git for packaging all the way through. I have found it makes backporting updates easier than when we were using CVS.
  4. Bruno Wolff - spins - The timing of XZ support for squashfs getting into the kernel was fortuitous as we would have had to either make hard decisions about cuts to keep several live spins CD sized or decide to use images that wouldn't fit on CDs.

Could have been better

  1. Jared Smith -- NetworkManager bump - The move to NetworkManager 0.9 came after the feature freeze, and the Sugar desktop wasn't accounted for in the scramble to get alternate desktop environments working with it.
  2. Jared Smith -- artwork - There seemed to be a lack of consensus from the Design team on whether or not the artwork was considered ready or not
  3. Jared Smith -- systemd - The Fedora Packaging Committee had a hard time getting specifics from the systemd developers in a timely manner, which caused difficulties in creating packaging guidelines with respect to systemd
  4. Jared Smith -- marketing - The Fedora Project Leader left the Red Hat press release and web copy until the last minute, and it should have been done much earlier.
  5. Paul Frields -- marketing - There didn't seem to be the usual number of interviews and feature profiles coming out around the release, which was even more surprising since the look and feel of the release with GNOME 3 was so visibly different. This was a lost opportunity to partner well with a major upstream where we have a lot of participants producing new code.
  6. Jared Smith -- translation - We switched from a self-hosted version of Transifex 0.8 to a hosted instance at transifex.net mid-way through the F15 release cycle. This caused many difficulties for the translation team, and I'm not sure how well the developers and packagers in Fedora understood the impact of the changes.
  7. Bruno Wolff -- spins - The Spins SIG leader got busy (with work I think) and we ended up making rushed (though reasonable) decisions (after a number of meetings didn't happen) on which spins should be produced.
  8. Bruno Wolff -- hardware requirements - The amount of memory needed to install F15 increased significantly. There is a plan to reduce this for F16, but there wasn't time to do it for F15.
  9. Máirín Duffy -- artwork - not having separate wallpapers per desktop environment would have been better, caused a lot of confusion in what artwork to use in producing collateral materials for the release and a burden on team
  10. Máirín Duffy -- artwork - some kind of metrics for artwork quality as part of QA would be good
  11. Jared Smith -- Amazon EC2 images were not prepared for release time. (Still trying to learn exact details of what held them up.)
  12. Garrett Holmstrom -- release notes - At release time the release notes had not yet been updated to reflect the greater amount of memory needed to install with install media.

Wishlist

References

Retrieved from "https://fedoraproject.org/w/index.php?title=Fedora_15_Retrospective&oldid=240640"