From Fedora Project Wiki

< QA

fp-wiki>ImportUser
(Imported from MoinMoin)
 
(→‎Where we are: Fixed link to Fedoras Beaker instance)
 
(4 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Beaker =
= Beaker =


Beaker is an adaptation of Red Hat's RHTS automated test system for Fedora (and the Free Software community in general). See ["QA/RHTS"]  for more info about RHTS.
Beaker is an automated test system created by Red Hat that can handle full stack hardware and software integration testing for Fedora, Red Hat Enterprise Linux, CentOS and derivatives.
 
Project home page: http://beaker-project.org/


=== Where we are ===
=== Where we are ===


As mentioned elsewhere, Red Hat QE has released the following pieces of their test system:
While Red Hat released Beaker as an open source project several years ago, none of the regression tests were released at that point in time.
 
* Documentation and APIs for tests, test running, test result reporting, etc.
* Automated tests which use those APIs
* A personal test execution engine (so individual testers can run tests on their systems)
 
=== Where we're going ===
 
Now that we have a suggested format / API for tests, and ways to run those tests, the next obvious step is collecting test results and displaying them in an attractive, easy-to-use way. So here's what's next:
 
* Streamlined test-writing
* Writing RHTS tests is not hard, but it could be easier. We're considering creating a simplified test format for Beaker.
* A few design goals:
1. Tests should not be responsible for anything other than doing testing and emitting output on stdout, stderr, and maybe writing some files (typically to an agreed-upon place)
1. Tests should run without needing a special test environment.
1. Tests should be able to be written in any language the author chooses.
1.  Tests should be verbose about what they are doing and how they are doing it.
<!-- should link to QA/BeakerTestFormat or something
-->
* A common test result reporting format.
* RHTS has an XML-RPC API for reporting test results.
* There are other test result schema out there - for example, Spikesource has their OSL-licensed Test Results Publication Interface, as defined here: http://developer.spikesource.com/wiki/index.php/Test_Results_Publication_Interface
* A test-runner
* Once we've got tests and a common reporting format, we need a tool that will take the test output and turn it into output in the common format.
* A results aggregator - a place to send, display, and query test results.
* This will take files using the common test result format and display them to users, testers, and other interested parties.


Here is a tentative roadmap/schedule for some important Beaker achievements:
This is now changing, with the essential installer regression tests being made available for use in Fedora as well: http://atodorov.org/blog/2013/11/19/open-source-quality-assurance-infrastructure-for-fedora-qa/


{| border="1"
The Fedora Beaker instance is at: http://beaker.qa.fedoraproject.org/
|- ||||style="color: white; background-color: #3070c0; font-weight: bold"
|Beaker Roadmap
|- style="color: white; background-color: #153560; font-weight: bold"
|Date||Task||Status/Notes
|- style="color: white; background-color: green"
|June 30||Red Hat releases sample tests, simple test execution engine||OK, test release ongoing
|- style="color: white; background-color: green"
|August 30||Test API documentation available||https://testing.108.redhat.com/test-writing-api/
|- style="color: white; background-color: #040"
|January 24||RHTS test-env packages in Extras||Under review
|-
| ||Test runner and common results format||
|-
| ||First prototype results server||
|-
| ||Completed results server||

Latest revision as of 00:53, 10 June 2016

Beaker

Beaker is an automated test system created by Red Hat that can handle full stack hardware and software integration testing for Fedora, Red Hat Enterprise Linux, CentOS and derivatives.

Project home page: http://beaker-project.org/

Where we are

While Red Hat released Beaker as an open source project several years ago, none of the regression tests were released at that point in time.

This is now changing, with the essential installer regression tests being made available for use in Fedora as well: http://atodorov.org/blog/2013/11/19/open-source-quality-assurance-infrastructure-for-fedora-qa/

The Fedora Beaker instance is at: http://beaker.qa.fedoraproject.org/