From Fedora Project Wiki
Line 96: Line 96:
 
|-
 
|-
 
|What about having one macro called %buildsys_packager instead of two?
 
|What about having one macro called %buildsys_packager instead of two?
| This is certainly doable and will simplify the code. Therefore, I will do it. I was formatted by the way name and email are separated in .gitconfig
+
| This is certainly doable, a good suggestion, and has been done in the last version of the code.
  
 
|-
 
|-

Revision as of 09:31, 5 July 2020

RPM-level auto release and changelog bumping

Summary

redhat-rpm-config will be updated so users of the auto framework get automated release and changelog bumping.

Owner

  • Email: <nicolas.mailhot at laposte.net>

Current status

  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

This is a system-wide change because all packages build with redhat-rpm-config, but it only concerns packages that opted to use this part of redhat-rpm-config (auto framework).

The change will make packages that use the %auto_<call> redhat-rpm-config macros auto-bump and auto-changelog at the rpm level, in an infrastructure-independent way. The %auto_<call> framework is proposed in https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs

In that context, auto-bumping means that a SRPM, produced in any compatible build system (that is, any build system that does not inhibit low-level rpmbuild behaviour), will rebuild by default to a release higher, than the last build release, in the next build system it is imported into, without any manual change to the SRPM source files.

Auto-changelog means that the build event will also be traced in the rpm changelog (again, without any manual change).

Feedback

How is it better than rpmdev-bumpsec? Unlike rpmdev-bumpsec, the feature is automatic.


It does not require explicit human action. Releases get bumped even if the human forgot a particular release has already been built.

It does not rely on an external tool, nor requires this external tool to be able to parse a spec file (which can be difficult for heavily automated spec files like the ones that take advantage of %auto<call> macros).

A rebuild does not touch the spec file at all. That means, the spec files changes tracked by your favorite scm, will show only spec logic changes, without drowning those in no-logic-change build events.

Where will it take changelog from? The previous changelog state will be taken from a separate rpm-changelog.txt included by default in SRPM sources.

The other information needed to bump reliably will be taken from another key = value buildsys.conf file also included by default in SRPM sources.

How does it affect reproducibility? The change makes SRPMs non-reproducible by default (reproducibility is the antithesis of auto-bumping).

To reproduce a build, you will need to take the SRPM produced by last build, and rebuild it with

 %reproducible_build = true

(or anything else except false, in your ~/.rpmmacros, or config_opts['macros'], or as rpmbuild --define flags)

It is not possible to reproduce a build from the SRPM as it existed at the start of last build, since that SRPM does not know the date at which the next build occurred.

Why is a separate "rpm-changelog.txt" file with manually maintained changelog better than current manually maintained changelog inside .spec? This change does not separates the changelog. The separation is already done in

https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs that this change builds upon

Without separation, we would lose the benefit of auto-bumping at the SCM level, since no-logic-change rebuilds would still result in a spec file change.

Separation makes automation a lot easier since adding to the changelog is just pre-pending some lines, and does not require any knowledge of rpm syntax. Auto-bumping will add a "* date name evr" line on the next rebuild, so changelog additions can limit themselves to plain text descriptions of new changes at the top of the existing file.

Separation is a requirement for auto-changelog bumping at the rpm level. Once rpmbuilt is lauched, it can not modify the processed spec file. Therefore making the changelog modifiable by the build process requires splitting it out of the spec file first.

How about using git commit log for changelog instead? This is a low level change that does not depend on any specific infrastructure, git included. It works directly at the rpm level.
How will the changelog be maintained? An infrastructure that uses git, can feed git commit events to the detached changelog file, using dumb or elaborate git commit hooks, and any other method it wants to implement. The auto-bump logic does not care, it will use the detached changelog file in the state it exists at the start of the build process.

Because the logic catches all rebuilds, regular manual trimming of the lines that add no value is recommended.

What about having one macro called %buildsys_packager instead of two? This is certainly doable, a good suggestion, and has been done in the last version of the code.
Can it be used without the %auto_call framework? Unfortunately, no, the implementation was simple and reliable and easy because the %auto_call framework already splits the release value into easy to process parts, puts them in variables that rpm can manipulate, and provides the entry points the feature uses at various stages of the spec file.

A separate implementation would need to reproduce all this work first.

Anything that lets the packager fill in Release: directly in the spec file can not bump at the RPM level without RPM changes, since the act of reading the release tag (and making it available as %[release} for further processing, sets the Release value in stone.

Can I see some spec file examples? You can take any spec file in

https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-auto-call-bump-changelog-fonts/builds/

except for the two macro packages those use. (Those are fonts packages, so the fonts macros are required, but autobumping is done at the generic redhat-rpm-config level, not at the fonts macro level)

However you won’t find any bumping specific changes in there. The spec files are strictly identical to those that were posted in

https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs

The feature does not require additional spec changes.

Can I see the code diff? The current code state is visible in

It’s one small commit on top of the huge change queued in:

That PR can still evolve based on feedback and testing. Therefore, I can’t promise that the auto-bumping logic will always apply directly, just that it will look more or less this way after rebasing. I do not rebase it on every change to the other PR.

This is very young code, there are probably lots of easy things to tidy up in there. However it works.

Are there other things to tidy up? A production implementation would probably split %{dist} in %{distcore}

and %{distprefix} (the .gitdatehash things we stuff in Releases and in rpm changelogs as opposed to the fcX part we want to appear in Release but not in changelogs). I know where the offending code is in fedora- release and the split up is trivial to implement, but there’s no point in worrying about this level of detail before the core of the feature is approved (or not).

Right now the implementation inhibits %dist in bulk (decorations included) in changelogs.



Benefit to Fedora

Autobumping removes a huge packager chore and makes time-stamping in changelogs more reliable.

Scope

  • Proposal owners: The feature is coded and works at the rpm level. Unfortunately, mock filters away the srpms containing the bump state, so it does not work in upper layers.
  • Other developers: The feature requires buy-in by mock developers (and probably koji developers) to lift the restrictions that block it above the rpm level. Also, it requires a mechanism to pass the user name and email that will be used in bumped changelogs (defining two variables in ~/.rpmmacros is sufficient at rpm level)
  • Policies and guidelines: maybe eventually if things work out on the technical level
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

This is a pure build tooling update, it changes how things are built not what is built.

How To Test

A redhat-rpm-config packages with the changes and some example packages are available in

 https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-auto-call-bump-changelog-fonts/builds/

Since the mock/copr layer is currently blocking the feature, you need to install the redhat-rpm-config and forge macro packages available in this repo locally. Afterwards you can take any of the example packages in the repo and rebuild them with rpmbuild -ba to your heart content, and see the releases bump and the changelogs being updated accordingly.

To get beautiful changelogs, you also need to add

%buildsys_name  Your name
%buildsys_email Your email

in ~/.rpmmacros

User Experience

N/A Packager experience change only

Dependencies

The change is a spin-off of

https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macros_-_Detached_rpm_changelogs

Therefore, it depends on the success of that other change and will probably need rebasing if the code in this other change evolves during the redhat-rpm-config merge.

It also depends on mock / copr/ koji buy-in and changes, that may add their own requirements.

Contingency Plan

There is no contingency plan because the change will happen or not at all.

Documentation

There is as much documentation as the average redhat-rpm-config change (ie comments in the macro files themselves)

Release Notes

N/A Packager productivity change only