Features/FirstClassCloudImages/KojiPlan

Plan for Koji

This is the basic plan for how automatic image generation will work in Koji.

Overview

Inputs

1. Kickstart files (from https://git.fedorahosted.org/cgit/cloud-kickstarts.git)
2. Repo to use (for manual builds of release candidates, etc.)
3. name (ditto; weekly should have deterministic name with "weekly" and date)

Outputs

1. Images in all EC2 regions
2. Images uploaded to https://dl.fedoraproject.org/pub/alt/cloud
3. Fedmsg message announcing new uploads
4. simple web service providing stateful report on current images (JSON)

Steps

1. Cron job starts image build in koji (manual initiation also possible)
2. As they complete, Koji sends messages on the Fedora Message Bus (fedmsg)
3. uploader service consumes consumes these messages and uploads AMIs images to Amazon and qcow2 and tar xz'd raw images to http://alt.fedoraproject.org/cloud/ (see note on formats)
4. uploader service sends Fedora Message Bus message when each image is successfully uploaded
5. web listener service consumes these messages and stores to DB, serves out as JSON (also other possible formats, including something pretty?)

Not Pictured

1. Automatic QE system will pick up fedora messages and run QE automatically
2. Pretty web site showing weekly and test candidate image builds, possibly with a way for releng to mark certain ones as the official release

Specifics

1. Cron Tasks

Right now, livecd nightly builds are launched by hand. A cron script should automate this instead. The current process requires Koji admin credentials; the new system should avoid that.

2. Koji Fedmsg Integration

Right now, Koji does not send notifications for scratch builds. We need to either change that or make these "real" builds.

3. The Uploader Service

This needs to listen to fedmsg