From Fedora Project Wiki

No edit summary
No edit summary
Line 10: Line 10:


* ''Goal'': eventually all packages involved in Fedora Atomic Host will have opted into the pipeline.
* ''Goal'': eventually all packages involved in Fedora Atomic Host will have opted into the pipeline.




Line 26: Line 25:
* Once the pipeline is triggered as part of the build process if unit tests exist they will be executed.
* Once the pipeline is triggered as part of the build process if unit tests exist they will be executed.
* The end result is package will be produced to then be used for further testing.  Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
* The end result is package will be produced to then be used for further testing.  Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
** Send build result on the fedmsg bus can be integration to any Fedora tools.
** Build result on the fedmsg bus will be plumbed through to bodhi via resultsDB


== Stage 3: Functional Tests on Package ==
== Stage 3: Functional Tests on Package ==


* Functional tests will be executed on the produced package from the previous stage of the pipeline if they exist.  This will help identify issues isolated to the package themselves. Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
* Functional tests will be executed on the produced package from the previous stage of the pipeline if they exist.  This will help identify issues isolated to the package themselves.  
** Send build result on the fedmsg bus can be integration to any Fedora tools.
* Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
** Functional tests results on the fedmsg bus will be plumbed through to bodhi via resultsDB
 
 
== Stage 4. The package built is pulled into a compose ==
 
* If functional tests are successful in the previous stage of the pipeline then an OStree compose is generated. 
* The compose occurs in Jenkins
** Compose result on the fedmsg bus will be plumbed through to bodhi via resultsDB
 
 
== Stage 5: Execute Integration Tests on OStree ==
 
* Integration tests are run on the OStree compose. 
* Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer. 
** Integration tests result on the fedmsg bus will be plumbed through to bodhi via resultsDB
 
 
== Stage 6: e2e Conformance Tests on Openshift Clusters ==
 
* If integration tests of the images are successful an Openshift cluster will be configured using the Atomic Openshift Installer with the new Fedora Atomic Host image as the base system. 
* Once the cluster is configured Kubernetes conformance tests will run.
* Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
** e2e Conformance Integration tests result on the fedmsg bus will be plumbed through to bodhi via resultsDB
 
 
== Stage 7: Image Generated From Successful Integration Tests On OStree ==
 
* An image will be initially generated at a certain interval when there has been successful integration test execution on an OStree compose.
* Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
** Image build result on the fedmsg bus will be plumbed through to bodhi via resultsDB




== Stage 4. The package built is pulled into a compose
== Stage 8: Image Smoke Test Validation ==


* The build occurs in Jenkins
* The validation left is to make sure the image can boot and more smoke tests may follow if required.
* Integration tests are run against this compose
* Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer. 
** Send test results on the fedmsg bus
** Image smoke tests result on the fedmsg bus will be plumbed through to bodhi via resultsDB




[[Category:FedoraAtomicCI]]
[[Category:FedoraAtomicCI]]


4. The packager builds the package in koji as usual
1. The packager builds the package in koji as usual


5. The packager creates the bodhi update as usual
2. The packager creates the bodhi update as usual


6. Gating happens in bodhi
3. Gating happens in bodhi


Build result and test results are gathered in Bodhi.
Build result and test results are gathered in Bodhi.

Revision as of 02:08, 26 April 2017

CI-Pipeline Architecture and Design

In this section we will describe the different steps that the pipeline aims to introduce and support.


Stage 0. The packager opts into the pipeline for one or more packages and agrees the tests of these packages will be gating their push to the Fedora repositories.

  • The tests are contained in dist-git repos and can be invoke via Ansible following the Changes/InvokingTestsAnsible proposal.
  • The packager should be able to opt into which tests and test suites gate their package.
  • Goal: eventually all packages involved in Fedora Atomic Host will have opted into the pipeline.


Stage 1. Trigger: The pipeline starts after a packaging change is pushed to dist-git repo

  • Once packages are pushed to Fedora dist-git this will trigger a message. (fedmsg and Jenkins integration is via the Jenkins JMS Plugin)
  • The change is what we is tested in the pipeline against the known-good (already merged) set of changes. The pre-merge is covered in "Stage 0", which is a CI inner loop that is a Pull Request workflow.
  • Only Fedora Atomic Host packages will be monitored for changes.
  • Requirement: This dist-git commit should not yet affect other packagers.

Stage 2. The package is built with the new change

  • The build occurs in Jenkins using Mock. We may leverage COPR in the future for builds
  • Once the pipeline is triggered as part of the build process if unit tests exist they will be executed.
  • The end result is package will be produced to then be used for further testing. Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
    • Build result on the fedmsg bus will be plumbed through to bodhi via resultsDB

Stage 3: Functional Tests on Package

  • Functional tests will be executed on the produced package from the previous stage of the pipeline if they exist. This will help identify issues isolated to the package themselves.
  • Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
    • Functional tests results on the fedmsg bus will be plumbed through to bodhi via resultsDB


Stage 4. The package built is pulled into a compose

  • If functional tests are successful in the previous stage of the pipeline then an OStree compose is generated.
  • The compose occurs in Jenkins
    • Compose result on the fedmsg bus will be plumbed through to bodhi via resultsDB


Stage 5: Execute Integration Tests on OStree

  • Integration tests are run on the OStree compose.
  • Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
    • Integration tests result on the fedmsg bus will be plumbed through to bodhi via resultsDB


Stage 6: e2e Conformance Tests on Openshift Clusters

  • If integration tests of the images are successful an Openshift cluster will be configured using the Atomic Openshift Installer with the new Fedora Atomic Host image as the base system.
  • Once the cluster is configured Kubernetes conformance tests will run.
  • Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
    • e2e Conformance Integration tests result on the fedmsg bus will be plumbed through to bodhi via resultsDB


Stage 7: Image Generated From Successful Integration Tests On OStree

  • An image will be initially generated at a certain interval when there has been successful integration test execution on an OStree compose.
  • Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
    • Image build result on the fedmsg bus will be plumbed through to bodhi via resultsDB


Stage 8: Image Smoke Test Validation

  • The validation left is to make sure the image can boot and more smoke tests may follow if required.
  • Success or failure will result with a fedmsg back to the Fedora reviewer/maintainer.
    • Image smoke tests result on the fedmsg bus will be plumbed through to bodhi via resultsDB

1. The packager builds the package in koji as usual

2. The packager creates the bodhi update as usual

3. Gating happens in bodhi

Build result and test results are gathered in Bodhi.

If a test passes CI then that change is allowed through Bodhi. At this point the change starts to affect further development. This results in future changes going through CI being layered on top of this change.

If any tests fail, then the change will not be allowed through Bodhi (packager will be notified of failures), and no later runs of CI will include the change. A different change would then be run through the pipeline.