From Fedora Project Wiki

(Fedora Gooey Karma)
(add project repo link for BBA)
 
(39 intermediate revisions by 5 users not shown)
Line 2: Line 2:
 
{{autolang|base=yes}}
 
{{autolang|base=yes}}
  
__NOTOC__
+
The [[QA]] team develops tools to make testing easier and more repeatable.  We have several projects ranging from some that are pretty simple and new developer friendly to complex and challenging stuff at the cutting edge of the field. Contact [[User:tflink|Tim Flink]] or the {{fplist|qa-devel}} mailing list if you're interesting in helping to build new tools for Fedora QA.
  
The [[QA]] Project develops tools to make testing easier and more repeatable.  Developing open source quality assurance tools is complex and challenging stuff at the cutting edge of the field. Contact [[User:tflink|Tim Flink]] if you're up to the challenge of building new tools for Fedora QA.
+
= QA tools =
  
= General =
+
== Test Automation ==
; [http://git.fedorahosted.org/git/?p=fedora-qa.git Fedora QA scripts] : This is a Fedora QA Git repository containing various scripts that are related to our tasks.
 
  
= Test Automation =
+
; [[ResultsDB]] : A generic storage of (automated) test results. Designed to be lightweight, simple, scalable and fast. [https://docs.fedoraproject.org/en-US/ci/ Fedora CI] and [[OpenQA]] results are reported to ResultsDB.
  
; [[Taskotron]] : Taskotron is our current in-development framework for automatically executing a number of Fedora related tests. Both Fedora infrastructure generic and Fedora packages specific tests should be supported. Package maintainers should have an easy way of providing tests for their packages. This framework is intended to obsolete [[AutoQA]].
+
; [https://pagure.io/taskotron/resultsdb_conventions resultsdb_conventions] : A Python library that represents certain conventions for storing results in [[ResultsDB]] as code, and makes it easy to report results that comply with the conventions.
; [[AutoQA]] : AutoQA is an framework that provides tests, methods for automated execution, and displays for test results.  AutoQA relies on the [[Autotest]] test harness for job scheduling and execution.  Visit the [https://fedorahosted.org/autoqa AutoQA] trac instance for development milestones and source code.  AutoQA is an outgrowth of FudCon F11 [[Automated QA Testing Project|collaboration]]. ''This project is going to be replaced by Taskotron.''
 
; [https://fedorahosted.org/beaker Beaker] : [https://fedorahosted.org/beaker Beaker] is an up and coming fully automated test framework that provides bare metal hardware provisioning including console and power management, and job scheduling and execution.  The mailing list is [https://fedorahosted.org/mailman/listinfo/beaker-devel beaker-devel].
 
  
= Test Case Management =
+
; [[openQA]] : We use [http://open.qa this tool] (developed by our friends at [https://www.suse.com/ SUSE], now with contributions from us too) to automate certain release test cases, especially installation with [[Anaconda]]. We develop [https://pagure.io/fedora-qa/os-autoinst-distri-fedora Fedora tests] for openQA, along with a [https://pagure.io/fedora-qa/fedora_openqa scheduler / result forwarder] and a [https://pagure.io/fedora-qa/createhdds hard disk image creator]. The production Fedora instance is [https://openqa.fedoraproject.org here].
  
; [https://fedorahosted.org/nitrate Nitrate] : The nitrate project intends to provide a web-based test management framework for test planning and test execution tracking. Nitrate is being packaged for Fedora (refer to {{bz|690728}}) for updates on packaging.
+
; [https://github.com/Rorosha/testcloud testcloud] : testcloud is a tool to simplify and automate the local deployment of cloud based virtual machines for testing and development. It is used as part of Fedora CI workflow and also for manual Cloud images testing during release validation.
  
= Installation =
+
== Release validation testing ==
; [https://fedorahosted.org/snake SNAKE] : Smart Network Automated Kickstart Environment.  A small tool to assist with installing and pykickstart-based library for modelling sustainable kickstarts.
 
; [https://fedorahosted.org/cobbler Cobbler/koan] : Provisioning at it's best ... check out the [https://fedorahosted.org/cobbler trac] instance for documentation.
 
  
= Bugzilla =
+
; [https://pagure.io/fedora-qa/python-wikitcms/ python-wikitcms], [https://pagure.io/fedora-qa/relval/ relval] and [https://pagure.io/fedora-qa/testdays/ testdays]: A Python library and CLI based upon it for performing various [[QA:Release_validation_test_plan|release validation testing]] tasks in the [[Wikitcms]] 'test management system' (the set of wiki pages in which release validation test results are stored). testdays can perform a few basic operations on [[Test_Days|Test Day]] pages. [https://pagure.io/fedora-qa/relvalconsumer/ relvalconsumer] is a [https://www.fedmsg.com Fedmsg] consumer which creates validation test events.
  
; [https://fedorahosted.org/python-bugzilla/ python-bugzilla] : Command-line interaction with bugzilla.  Supports searching, creating and modifying bugs.
+
; [https://pagure.io/fedora-qa/check-compose/ check-compose]: A script for running informational checks on a Fedora compose and printing or mailing out the results.
; Improvements to [https://bugzilla.redhat.com/ Bugzilla] :
 
  
= Infrastructure =
+
; [https://pagure.io/fedora_nightlies fedora_nightlies]: A project comprising a static page generator and some fedmsg consumers that together produce [https://www.happyassassin.net/nightlies.html a page] linking to the most successful recent compose of each Fedora image for [[Rawhide]] and [[Branched]]. Also lists most recent successfully-tested image, for images tested by openQA or Autocloud.
  
; [https://fedorahosted.org/bodhi/ Bodhi] : Bodhi manages delivery of package updates for Fedora.
+
== Test case reporting ==
  
= Test update feedback =
+
; [[QA:TestdayApp|Test Days App]] : A web application for providing results from [[QA/Test Days|Test Days]] events. It uses [[ResultsDB]] as a backend.
  
; [[Fedora Gooey Karma]] : A graphical tool that makes it easy to report feedback for any updates-testing packages currently installed. Read [[Fedora Gooey Karma]] to learn more.
+
== Bug reporting ==
  
; [[Fedora Easy Karma]] : A command line tool that makes it easy to report feedback for any updates-testing packages currently installed. Install {{pkg|fedora-easy-karma}} package.
+
; [https://qa.fedoraproject.org/blockerbugs/ Blocker Bugs App] : A web application to display current [[QA:SOP blocker bug process|blocker bugs]] during our pre-release cycle. [https://pagure.io/fedora-qa/blockerbugs Project repo].
 +
 
 +
== Test update feedback ==
 +
 
 +
; [[Fedora Easy Karma]] : A command line tool that makes it easy to report feedback for any updates-testing packages currently installed.
 +
 
 +
== Mixed bag ==
 +
 
 +
; [https://pagure.io/fedora-qa/qa-misc Fedora QA tools] : A git repository containing various small tools.
 +
; [https://packager.fedorainfracloud.org Packager Dashboard] : A succinct status summary for all packages owned by a given packager. [https://pagure.io/fedora-qa/packager_dashboard Project repo].
 +
 
 +
== Statistics ==
 +
 
 +
; [https://pagure.io/fedora-qa/qa-stats Fedora QA statistics tools] : Tools for generating statistics on user contributions. [https://pagure.io/fedora-qa/relval/ relval] also has some functions for doing this. See [[QA:Recognition|Recognition]] for more details.
 +
 
 +
= Infrastructure tools =
 +
 
 +
These tools we use as our support infrastructure during QA tools development.
 +
 
 +
; [https://pagure.io/group/fedora-qa Pagure team projects] : Pagure repositories related to our tools and needs.
 +
 
 +
 
 +
[[Category:QA]]

Latest revision as of 14:36, 15 July 2020

QA.png

The QA team develops tools to make testing easier and more repeatable. We have several projects ranging from some that are pretty simple and new developer friendly to complex and challenging stuff at the cutting edge of the field. Contact Tim Flink or the qa-devel mailing list if you're interesting in helping to build new tools for Fedora QA.

QA tools

Test Automation

ResultsDB 
A generic storage of (automated) test results. Designed to be lightweight, simple, scalable and fast. Fedora CI and OpenQA results are reported to ResultsDB.
resultsdb_conventions 
A Python library that represents certain conventions for storing results in ResultsDB as code, and makes it easy to report results that comply with the conventions.
openQA 
We use this tool (developed by our friends at SUSE, now with contributions from us too) to automate certain release test cases, especially installation with Anaconda. We develop Fedora tests for openQA, along with a scheduler / result forwarder and a hard disk image creator. The production Fedora instance is here.
testcloud 
testcloud is a tool to simplify and automate the local deployment of cloud based virtual machines for testing and development. It is used as part of Fedora CI workflow and also for manual Cloud images testing during release validation.

Release validation testing

python-wikitcms, relval and testdays
A Python library and CLI based upon it for performing various release validation testing tasks in the Wikitcms 'test management system' (the set of wiki pages in which release validation test results are stored). testdays can perform a few basic operations on Test Day pages. relvalconsumer is a Fedmsg consumer which creates validation test events.
check-compose
A script for running informational checks on a Fedora compose and printing or mailing out the results.
fedora_nightlies
A project comprising a static page generator and some fedmsg consumers that together produce a page linking to the most successful recent compose of each Fedora image for Rawhide and Branched. Also lists most recent successfully-tested image, for images tested by openQA or Autocloud.

Test case reporting

Test Days App 
A web application for providing results from Test Days events. It uses ResultsDB as a backend.

Bug reporting

Blocker Bugs App 
A web application to display current blocker bugs during our pre-release cycle. Project repo.

Test update feedback

Fedora Easy Karma 
A command line tool that makes it easy to report feedback for any updates-testing packages currently installed.

Mixed bag

Fedora QA tools 
A git repository containing various small tools.
Packager Dashboard 
A succinct status summary for all packages owned by a given packager. Project repo.

Statistics

Fedora QA statistics tools 
Tools for generating statistics on user contributions. relval also has some functions for doing this. See Recognition for more details.

Infrastructure tools

These tools we use as our support infrastructure during QA tools development.

Pagure team projects 
Pagure repositories related to our tools and needs.