From Fedora Project Wiki
(→‎Scope: Adding a link to the releng ticket about this change)
(→‎Current status: mention F35 in targeted release)
 
(8 intermediate revisions by 2 users not shown)
Line 16: Line 16:


== Current status ==
== Current status ==
[[Category:ChangeAnnounced]]
[[Category:ChangeAcceptedF35]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 24: Line 24:
[[Category:SystemWideChange]]
[[Category:SystemWideChange]]


* Targeted release: N/A
* Targeted release: Q3 2021 ([[Releases/35 | Fedora 35 ]])
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Line 32: Line 32:
ON_QA -> change is fully code complete
ON_QA -> change is fully code complete
-->
-->
* FESCo issue: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/2582 #2582]
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1945406 #1945406]
* Release notes tracker: <will be assigned by the Wrangler>
* Release notes tracker: <will NOT be assigned by the Wrangler>


== Detailed Description ==
== Detailed Description ==


rpmautospec offers packagers who want to use it the possibility of replacing the content of the `Release` of their spec file by `Release:    %autorel` and/or replace the content of the `%changelog` section of their spec file by:
rpmautospec offers packagers who want to use it the possibility of replacing the content of the `Release` of their spec file by `Release:    %autorelease` and/or replace the content of the `%changelog` section of their spec file by:
   %changelog
   %changelog
   %autochangelog
   %autochangelog
   
   
Both `%autorel` and `%autochangelog` are RPM macros that will be expanded or replaced when the SRPM is built on the build system by their corresponding values according to rpmautospec.
Both `%autorelease` and `%autochangelog` are RPM macros that will be expanded or replaced when the SRPM is built on the build system by their corresponding values according to rpmautospec.


An overview of how rpmautospec works can be found at: https://docs.pagure.org/fedora-infra.rpmautospec/principle.html.
An overview of how rpmautospec works can be found at: https://docs.pagure.org/fedora-infra.rpmautospec/principle.html.
We will describe below how each macro works.
We will describe below how each macro works.


=== The %autorel macro ===
=== The %autorelease macro ===


To determine the next release information, rpmautospec relies on the build history of the package.
To determine the next release information, rpmautospec relies on the git history of the package.
This information is extracted from the buildsystem when running as a koji plugin and from git tags when running outside of the buildsystem.
This information is extracted from the package repository when run as a koji plugin or locally (e.g. using fedpkg).


Using the build history of the package (ie a list of NEVRs) as well as the information provided by the packager in the spec file, rpmautospec then computes the next best release number for the package.
Using the git history of the package as well as flags set by the packager in the spec file, rpmautospec then computes the next release value for the package.


Once defined, it prepends a suitably defined %autorel macro to the top of the spec file, freezing the computed value of the release number and thus allowing reproducible builds of the SRPM.
Once defined, it prepends a suitably defined %autorelease macro to the top of the spec file, freezing the computed value of the release number and thus allowing reproducible builds of the SRPM.


The [https://docs.pagure.org/fedora-infra.rpmautospec/autorel.html documentation of the autorel macro] describes how packagers can use it to provide extra information.
The [https://docs.pagure.org/fedora-infra.rpmautospec/autorelease.html documentation of the autorelease macro] describes how packagers can use it to provide extra information.


=== The %autochangelog macro ===
=== The %autochangelog macro ===
Line 86: Line 86:
== Scope ==
== Scope ==
* Proposal owners:
* Proposal owners:
** Finish the work on the %autorel macro to support some of the use cases not implemented currently
** Finish the work on the %autorelease macro to support some of the use cases not implemented currently and change the algorithm to only consider the number of commits since the last version change (as requested by FESCo)
** Adjust the packaging so rpmautospec does not live in a specific, versionized python environment (and thus could be use to bootstrap python)
** Adjust rpmdev-bumpspec (used by releng for mass-rebuilds) so it ignores spec files using %autorelease/%autochangelog  
** Adjust rpmdev-bumpspec (used by releng for mass-rebuilds) so it ignores spec files using %autorel/%autochangelog  
** Adjust the mass-rebuild script so it only adds a suitable git commit log message for spec files using %autorelease/%autochangelog (NB: uses rpmdev-bumpspec, so no change necessary here)
** Adjust the mass-rebuild script so it only adds a suitable git commit log message for spec files using %autorel/%autochangelog
** Adjust fedpkg to skip the NEVR check when using rpmautospec  
** Adjust fedpkg to skip the NEVR check when using rpmautospec  
** Add dependency on rpmautospec on redhat-rpm-config  
** Add dependency on rpmautospec on redhat-rpm-config  
Line 121: Line 120:
* Install `rpmautospec-rpm-macros` to build packages locally.
* Install `rpmautospec-rpm-macros` to build packages locally.
* Edit the spec file to use either macro (or both)
* Edit the spec file to use either macro (or both)
* (Optional) Run `rpmautospec` on the git repository of your choice to see what it does (you may have to run `rpmautospec tag-package` the first time you run it).
* (Optional) Run `rpmautospec calculate-release/generate-changelog/process-distgit` on the git repository of your choice to see what it does.
* Build the package
* Build the package (`fedpkg-stage build --skip-nvr-check` until changes have landed in rpkg/fedpkg)
* Check its release or changelog according to the macro set
* Check its release or changelog according to the macro set



Latest revision as of 15:54, 18 June 2021


rpmautospec - removing release and changelog fields from spec files

Summary

The goal of this change is to deploy in production the rpmautospec project.

With it, the content of the Release and %changelog fields in spec files can be auto-generated, either locally or in the builder using the information present in the git repo (in the form of git tags).


Note: This proposal is about changing the way the Release and %changelog sections of the spec files are filled, not about removing them from the SRPM or binary RPM.

Owner

Current status

  • Targeted release: Q3 2021 ( Fedora 35 )
  • Last updated: 2021-06-18
  • FESCo issue: #2582
  • Tracker bug: #1945406
  • Release notes tracker: <will NOT be assigned by the Wrangler>

Detailed Description

rpmautospec offers packagers who want to use it the possibility of replacing the content of the Release of their spec file by Release: %autorelease and/or replace the content of the %changelog section of their spec file by:

 %changelog
 %autochangelog

Both %autorelease and %autochangelog are RPM macros that will be expanded or replaced when the SRPM is built on the build system by their corresponding values according to rpmautospec.

An overview of how rpmautospec works can be found at: https://docs.pagure.org/fedora-infra.rpmautospec/principle.html. We will describe below how each macro works.

The %autorelease macro

To determine the next release information, rpmautospec relies on the git history of the package. This information is extracted from the package repository when run as a koji plugin or locally (e.g. using fedpkg).

Using the git history of the package as well as flags set by the packager in the spec file, rpmautospec then computes the next release value for the package.

Once defined, it prepends a suitably defined %autorelease macro to the top of the spec file, freezing the computed value of the release number and thus allowing reproducible builds of the SRPM.

The documentation of the autorelease macro describes how packagers can use it to provide extra information.

The %autochangelog macro

The %autochangelog macro is in fact more a placeholder that rpmautospec fills in during the creation of the SRPM (or when ran manually on a project).

rpmautospec uses two sources of information to automatically generate the changelog:

  • an optional changelog file that packagers can add to the repository
  • the git history of the spec file

rpmautospec will include the changelog as is in the %changelog section and will use all the commits made to the repository after the last change of the changelog file to generate the changelog.

In other words, if the packager has a repository created on January 10th and works on it for 6 months, adds a changelog file on June 1st. All the commits made before that commit of June 1st will be ignored in favor of the content of the changelog file. Similarly, if the packager then edits the file on July 1st, all the commits prior to that commit of July 1st will be ignored. All the commits involving files ending with either ".spec" or ".patch" (this list can be expended if desired) and made after July 1st will be used to generate the changelog.

Feedback

Benefit to Fedora

In short: easier packaging in Fedora for the packagers who opt-in.

The Release and %changelog fields are the two most conflicting fields in RPM spec files. They impact most pull requests if they involve updating the package or if the package is updated/rebuilt while pull-request are being reviewed.

We currently have three different change logs per package and while they target different audiences (package maintainer: git, sysadmin: %changelog and end-user: bodhi notes) they overlap a lot and in most cases are redundant. With this proposal, package maintainers will only have to cope with two changelogs: git and bodhi notes.

Scope

  • Proposal owners:
    • Finish the work on the %autorelease macro to support some of the use cases not implemented currently and change the algorithm to only consider the number of commits since the last version change (as requested by FESCo)
    • Adjust rpmdev-bumpspec (used by releng for mass-rebuilds) so it ignores spec files using %autorelease/%autochangelog
    • Adjust the mass-rebuild script so it only adds a suitable git commit log message for spec files using %autorelease/%autochangelog (NB: uses rpmdev-bumpspec, so no change necessary here)
    • Adjust fedpkg to skip the NEVR check when using rpmautospec
    • Add dependency on rpmautospec on redhat-rpm-config
  • Other developers:
    • Opt-in for those who want to use it
    • Test changes in staging
    • Provide feedback
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

Not applicable


How To Test

In staging:

  • Use stg-koji instead of koji
  • Use fedpkg-stage instead of fedpkg (dnf install fedpkg-stage)
  • Run kinit <accountname>@STG.FEDORAPROJECT.ORG
  • Test with rawhide as rpmautospec is only available there in staging at this point
  • Install rpmautospec-rpm-macros to build packages locally.
  • Edit the spec file to use either macro (or both)
  • (Optional) Run rpmautospec calculate-release/generate-changelog/process-distgit on the git repository of your choice to see what it does.
  • Build the package (fedpkg-stage build --skip-nvr-check until changes have landed in rpkg/fedpkg)
  • Check its release or changelog according to the macro set

In production:

  • Edit the spec file to use either macro (or both)
  • (Optional) Run rpmautospec on the git repository of your choice to see what it does (you may have to run rpmautospec tag-package the first time you run it).
  • Build the package
  • Check its release or changelog according to the macro set

Issues can be reported at: https://pagure.io/fedora-infra/rpmautospec/issues

Note that using this approach has some peculiarities, we've documented the ones we've encountered at: https://docs.pagure.org/fedora-infra.rpmautospec/peculiarities.html

User Experience

Packagers not opting-in should not be affected in any way by this change.

Packagers opting-in should be able to stop worrying (or worry much less) about the content of the Release/%changelog fields in their spec file.

Dependencies

None

Contingency Plan

  • Contingency mechanism: rpmautospec is not deployed in koji
  • Contingency deadline: N/A (this isn't tied to a Fedora release)
  • Blocks release? No


Documentation

All the upstream documentations can be https://docs.pagure.org/fedora-infra.rpmautospec/

Release Notes

N/A, this change will not affect end-users (except maybe for some changes in the content of the rpm changelog).