From Fedora Project Wiki

Revision as of 23:17, 4 February 2009 by Jkeating (talk | contribs) (Create page, brain dump from fudcon.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

It is time that Fedora starts doing automated QA testing.

Many minds met during FUDCon F11 Boston 2009 in order to brain storm on automated testing. (link to video here)

We identified a number of "triggers" to react to in order to test things, a number of tests to perform, and a set of places to notify of the results. Here is a quick dump of these three things.

Triggers

  • cvs checkin
  • koji build
  • bodhi request
  • rawhide compose
  • manual (periodic)
  • iso compose
  • new test creation
  • post test
  • comps change


Tests

  • repository Sanity (sublist here)
  • package installation
  • profile testing
  • tree sanity
  • rpmdiff
  • build log sanity
  • source sanity
  • build root sanity
  • fails to build from source
  • make check
  • comps grammar


Notifiers

  • <package>-owner@fedoraproject.org
  • project webpage
  • fedora-test-list@redhat.com or fedora-devel-list@redhat.com

Also discussed were timeline targets. Next 3 weeks, next 3 months, next 3 quarters. We feel we can accomplish a number of the triggers and tests within the next 3 weeks, focusing more package and repo sanity. In the next 3 months we can likely add functional testing of installer via Lab in a Box (link here). Over the next 3 quarters we hope to be able to expand functional testing to individual packages.