From Fedora Project Wiki

Revision as of 08:18, 17 June 2009 by Liam (talk | contribs)

Note.png
DRAFT
This page is currently a draft. This test plan has not been finalized yet. Stay tuned ...

Introduction

This document describes the tests that will be created and used to verify the functions/components of Fedora 12.

The goals of this plan are to:

  • Organize the test effort
  • Communicate the planned tests to all relevant stake-holders for their input and approval
  • Serve as a base for the test planning for future Fedora 12 releases

Test Strategy

Instead of outlining all possible installation inputs and outputs, this test plan will focus on defining inputs and outputs at different stages in anaconda. This will also allow different tests to be performed independently during a single installation. For example, one may execute a kickstart delivery via HTTP, raid0 partitioning using 3 physical disks, and a minimal package installation on a para-virtualized xen guest all in single installation. Scenarios where the stages are dependent will be indicated as such in the test case.

Where possible, SNAKE will be used to automate and aid in reproducibility.

New features of Fedora 12

Important.png
FIXME
Need confirmation on the following list of features for Fedora 12

As Fedora 11,Fedora 12 will bring us some new features.The following list of features was obtained from Anaconda/Features. Test plans for these features will be designed/developed on each feature page.

Schedule/Milestones

  • The Fedora 12 release schedule is available at Releases/12/Schedule
  • Each major milestone(Alpha,Beta,Preview,etc..) will demand a full regression run

Test Priority

This test plan will use a 3 tier classification for test execution priority.3 tier Priority to test cases at 3 tier priority

Tier1 is intended to verify that installation is possible on common hardware using common use cases. Verification includes:

  • Common boot media
  • Common Installation source
  • Installation using defaults installation options
  • Default Partitioning

Tier2 takes a step further to include more use cases. Tier2 verification consists of:

  • All boot media
  • All installation sources
  • All kickstart delivery methods
  • Some architecture specific verification

Tier3 captures the remaining identified use cases:

  • More exhaustive partitioning schemes
  • More complex networking scenarios
  • More architecture specific verification
  • Network device
  • Storage device
  • Upgrade testing

Test Pass/Fail Criteria

The milestone release of Fedora 12 should conform these criteria:

Entrance criteria

  • Trees must be generated using release engineering tools (not hand crafted)
  • There must be no unresolved dependencies for packages included in the installation tree
  • There must be no dependency conflicts for packages included in the installation tree
  • Any changes in composition of the installation tree are explainable by way of bugzilla

Beta criteria

  • Entrance criteria have been met
  • All tier1 tests have been executed

Preview criteria

  • Beta criteria have been met
  • All tier1 tests pass
  • All tier2 tests have been executed

GA criteria

  • Preview criteria have been met
  • All test tiers must pass
  • Any open defects have been documented as release notes

Scope and Approach

Testing will include:

  • Manually execute test cases using a live image of rawhide or updated rawhide system
  • Manually/kickstart boot the installation program to execute the installation
  • Automatically execute installation testing via cobbler/koan etc..,cobbler guide and scripts,please refer to cobbler

Items outside the scope of this test plan include:

  • Functional verification of software installed on the system
  • Installation from media not generated by fedora release engineering

Test Deliverables

  • This test plan
  • A test summary document for each major milestone
  • A list of defects filed
  • Any test scripts used for automation or verification

Testing Tasks

Testing will execute test cases to verify installation of Fedora 12 on different hardware platforms and gather installation test feedback.

Test Environment/Configs

For Fedora 12,test cases will be executed on these hardware platforms:

  • i386
  • ppc
  • x86_64

Responsibilities

  • who's doing what

Risks and Contingencies

  • If a particular build is uninstallable, a updates.img may be used to work around any test blockers. For information on building and using an updates.img please refer to Anaconda/Updates.
Note.png
An updates.img can't always save the day
Please note, any issues affecting stage#1 loader (networking segfaults) or kernel boot issues may not be resolvable by using an updates.img. For these issues, your defect will need to be prioritized as a test blocker so that it can be addressed in a subsequent build.
  • If a build is uninstallable due to a bad kernel or driver, you may be able to work around the problem. Anaconda provides several scripts for taking a newer kernel, and using that during installation. For details, please refer to [scripts/upd-kernel]


Reporting Bugs and Debugging Problems

If catch some bugs,please go ahead and file the bugs following the guide below:

References