Changes/ELN Buildroot and Compose

ELN Buildroot and Compose

Summary

The goal of the ELN project is to continuously build Fedora Rawhide packages and composes in the way which resembles the CentOS and RHEL build process and to provide a feedback loop for Fedora maintainers on the status of those builds.

Owner

• Name: Aleksandra Fedorova
• Email: alpha@bookwar.info
• Name: Stephen Gallagher
• Email: sgallagh@redhat.com

Current status

• Targeted release: Fedora 33
• Last updated: 2020-03-23
• FESCo issue: <will be assigned by the Wrangler>
• Tracker bug: <will be assigned by the Wrangler>
• Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

This Change supersedes the previously-approved Change to enable an additional buildroot. During development, its scope has expanded to include the entire process of how the Fedora sources are built and composed into shippable artifacts.

This includes:

• buildroot configuration, rpm macro and compile flags,
• comps files and the compose content,
• compose itself and the pipeline which builds it.

Under this umbrella we are going to have:

• disttag which allows tweaking rpm spec files,
• Koji tag which allows control of the buildroot,
• fork of the pungi configuration which describes the compose configuration,
• pipeline which builds periodic compose based on that configuration,
• storage for such composes and additional pipelines which will verify the quality of the compose and test additional scenarios on top of it.

Benefit to Fedora

Who benefits from the implementation of this feature:

• Fedora Infrastructure
  • As we are going to try and test new, more easily maintainable infrastructure for composes.
• Fedora Minimization
  • As we are going to modify the compose content according to minimization goals and provide feedback to Fedora maintainers on the results
• CentOS Stream, EPEL and RHEL
  • We are going to build Fedora packages into a compose similar to the multi-repo structure of CentOS and RHEL.
• Fedora Community
  • The feedback pipelines built for the project will allow downstream developers to open up their work and bring it closer to Fedora. In particular, it will enable projects like OpenShift to do their work in Fedora.
  • Making Fedora friendlier to its downstream will justify bringing more resources and more contributions to Fedora Rawhide. So that downstream not only tries to catch up with Fedora going forward, but actively participates in Fedora development and testing.
  • The tooling developed for this project will allow further development of changes in Fedora Infrastructure without blocking the regular Fedora packaging and release process.

Scope

• Proposal owners:
  • Setup build environment for a new disttag (eln) .
  • Setup automation so that when build submitted to Fedora Rawhide there is a build submitted to the ELN buildroot. Result of the build task will be posted to Fedora Messaging and consumed by ResultsDB, so that it appears in Bodhi.
  • Setup automation to run periodic partial composes (via ODCS) without installation media to generate repositories with these packages.
  • Update packaging documentation to mention new disttag and how it can be used.

• Other developers:

  Anyone who wants to produce different content for the ELN compose will need to implement the new macros in their specfile. The overwhelming majority of packages will require no modification.

• Release engineering: #9154

• Policies and guidelines: #955

• Trademark approval: not needed. There is no new shippable artifact.

Upgrade/compatibility impact

N/A

Though it is a System Wide change it doesn’t impact Fedora upgrades, as it is a development environment tracking Fedora Rawhide.

How To Test

There is going to be a development snapshot of the state of the ELN repositories.

It is under discussion whether this snapshot will have its own installation media. For now the preferred way to test ELN composes would be to use standard Fedora Rawhide images and then include ELN as an additional repository.

User Experience

There is no user-facing part in this change. No ELN artifacts are going to be shipped to the end-user.

Dependencies

There is no dependency on other changes, though some of the infrastructure services like ODCS composes and Fedora CI will be involved.

Contingency Plan

• Contingency mechanism: If this effort doesn’t land in Fedora 33 we review it and decide whether it needs to be moved further to Fedora 34 or cancelled.
• Contingency deadline: N/A as it is a continuous process in Fedora Rawhide and it is not pinned to a specific release.
• Blocks release? No
• Blocks product? No

Additional risk: Infrastructure resources taken by ELN builds can create load on Fedora build system

Mitigation: In case of such load we cancel or reschedule ELN tasks (package and compose builds and tests) on request from the Fedora Infrastructure team. At least initially, we will also set ELN tasks to low-priority, similar to how COPR does it.

Documentation

https://fedoraproject.org/wiki/ELN

See also previous discussions:

• "Alternative buildroot as a development tool" @ devel

Release Notes

N/A

Though it is a System-Wide Change it has no user-facing component. We may announce it through other channels.