From Fedora Project Wiki

Revision as of 16:47, 21 December 2010 by Adamwill (talk | contribs) (wikify bodhi link)

This SOP describes the process for creating a test plan for a specific package. Following the procedure outlined on this page is important as it ensures we use a similar process for testing all packages in Fedora, and makes it simpler to integrate these tests into the update release process.

Background

A test plan, for Fedora, is a co-ordinated set of test cases that, together, test the functionality of some particular piece of software, or process.

Preparation

In most cases, the best way to go about creating a test plan for a particular package is to consider the tasks or functions the package is capable of performing. For instance, if the package you wished to create a test plan for was yum, you might identify the following functions:

  • install a package
  • remove a package
  • provide information on a package
  • search for a package
  • update the system
  • set up a repository
  • disable a repository

and so on. Identify if any, some or all of the functions constitute part of the Fedora critical path. For our yum example, update the system would be a critical patch function. You may also want to split the functions into 'core' and 'extended' groups: those that are functions so basic the package would be considered severely broken if they were to fail, and those that are less important. Of course, if you are creating a test plan for a very complex package, it may be impractical to create test cases for every possible function, or even to identify them all. In this case simply concentrate on the most vital functions first of all, particularly Fedora critical path functions.

Test case creation

Once you have identified the functions you wish to create test cases for, create a test case for each function, according to the SOP for creating test cases.

Categorization

To link the cases together and ensure they can easily be found both manually and automatically (for the use of tools such as Bodhi and Fedora Easy Karma), use categories.

Simple (required) categorization

Add each test case for your package to the category named Category:Package_(packagename)_test_cases, where (packagename) is the name of the package - use the .src.rpm name, not binary package names. For our yum example, the category would be named Category:Package_yum_test_cases. If you identified any critical path functions, add the test case(s) for those function(s) to the category named Category:Critical_path_test_cases.

For our yum example, consider two test cases:

  • QA:Testcase_yum_upgrade - to test upgrading the system
  • QA:Testcase_yum_history - to test yum's history function

Both must be in the category Category:Package_yum_test_cases. QA:Testcase_yum_upgrade must also be in the category Category:Critical_path_test_cases.

Advanced (optional) categorization

If you want to split the test cases into groups, you can add sub-categories of the main package category, such as Category:Package_(packagename)_core_test_cases and Category:Package_(packagename)_extended_test_cases. This split can be helpful for testers, if there are many test cases for a package; it can also be expressed by tools such as Fedora Easy Karma.

For our yum example, consider a third test case added to our previous two test cases:

  • QA:Testcase_yum_install - to test installing a package
  • QA:Testcase_yum_upgrade - to test upgrading the system
  • QA:Testcase_yum_history - to test yum's history function

As before, all three must be in the category Category:Package_yum_test_cases and QA:Testcase_yum_upgrade must be in the category Category:Critical_path_test_cases. Additionally, you could place QA:Testcase_yum_install and QA:Testcase_yum_upgrade in a category Category:Package_yum_core_test_cases and QA:Testcase_yum_history in a category Category:Package_yum_extended_test_cases, and make both those categories sub-categories of Category:Package_yum_test_cases, if you wished to separate core and extended test cases for yum.