Changes/ReproducibleBuildsClampMtimes

Reproducible builds: Clamp build mtimes to $SOURCE_DATE_EPOCH

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

The %clamp_mtime_to_source_date_epoch RPM macro will be set to 1. When an RPM package is built, mtimes of packaged files will be clamped to $SOURCE_DATE_EPOCH which is already set to the date of the latest %changelog entry. As a result, more RPM packages will be reproducible: The actual modification time of files that are e.g. modified in the %prep section will not be reflected in the RPM package.

Owner

• Name: Miro Hrončok, Zbigniew Jędrzejewski-Szmek
• Email: mhroncok at redhat.com, zbyszek at in.waw.pl

Current status

• Targeted release: Fedora Linux 38
• Last updated: 2022-11-08
• FESCo issue: <will be assigned by the Wrangler>
• Tracker bug: <will be assigned by the Wrangler>
• Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

This change exists to make RPM package builds more reproducible. A common problem that prevents build reproducibility is the mtime (modification times) of the packaged files.

Suppose we package an RPM package of software called skynet in version 1.0. Upstream released this version at datetime A. A Fedora packager creates the RPM package at datetime B. Unfortunately, the packager needs to patch the sources in the RPM %prep section. When the build runs at datetime C, the modification datetime of the patched file is set to C. When the build runs again in an otherwise identical environment at datetime D, the modification datetime of the patched file is set to D. As a result, the build is not bit-by-bit reproducible, because the datetime of the build is saved in the resulting package. Patching is not necessary to make this happen. When a source file is compiled into a binary file, the modification datetime is also set to the datetime of the build. In practice, the modification datetime of many files packaged in RPM packages is dependent on when the package was actually built.

To eliminate this problem, we propose to clamp build mtimes to $SOURCE_DATE_EPOCH. RPM build in Fedora already sets the $SOURCE_DATE_EPOCH environment variable based on the latest %changelog entry because the %source_date_epoch_from_changelog macro is set to 1. We will also set the %clamp_mtime_to_source_date_epoch macro to 1. As a result, when files are packaged to the RPM package, their modification datetimes are clamped to $SOURCE_DATE_EPOCH (to the latest changelog entry datetime). Clamping means that all files which would have a modification datetime higher than $SOURCE_DATE_EPOCH will have the modification datetime changed to $SOURCE_DATE_EPOCH; files with mtime lower (or equal) to $SOURCE_DATE_EPOCH will retain the original mtimes.

This functionality is already implemented in RPM. We will enable it by setting %clamp_mtime_to_source_date_epoch to 1.

Non-goal

We do not aim to make all Fedora packages reproducible (at least not as part of this change proposal). We just eliminate one problem that we consider the biggest blocker for reproducible builds.

Python bytecode

When Python bytecode cache (a .pyc file) is built, the mtime of the corresponding Python source file (.py) is included in it for invalidation purposes. Since the .pyc file is created before RPM clamps the mtime of the .py file, the mtime stored in the .pyc file might be higher than the corresponding mtime of the .py file.

To solve this, we will modify Python to clamp the stored mtime to $SOURCE_DATE_EPOCH as well (when building RPM packages). Upstream Python chooses to invalidate bytecode cache based on hashes instead of mtimes when $SOURCE_DATE_EPOCH is set, but that could cause performance issues for big files, so Fedora's Python already deviates from upstream behavior when building RPM packages. To avoid accidentally breaking the behavior when %clamp_mtime_to_source_date_epoch is not set to 1, RPM macros and buildroot policy scripts for creating the Python bytecode cache will be modified to unset $SOURCE_DATE_EPOCH when %clamp_mtime_to_source_date_epoch is not set to 1.

This behavior might be proposed upstream if it turns out to be superior to the current upstream choice, in case we won't redesign the bytecode-source relationship entirely instead.

Opting out

Packages broken by this new behavior can unset %clamp_mtime_to_source_date_epoch but packagers are encouraged to fix the problem instead.

Feedback

Enabling this RPM feature was proposed as a pull request to redhat-rpm-config in April 2021. It received good feedback with the exception of the following:

• it was said the change needs to be coordinated with the Python maintainers
• it was said the change should be done via a change process for better coordination and exposure

We believe that by proposing this via the change process and planning for the changes needed in Python, both issues are addressed.

Benefit to Fedora

We believe that many RPM packages will become reproducible and others will be more reproducible than before. The benefits of reproducible builds are better explained at https://reproducible-builds.org/

Scope

• Proposal owners:
  • Propose a PR for redhat-rpm-config (set %clamp_mtime_to_source_date_epoch to 1)
  • Propose a PR for python-rpm-macros (unset $SOURCE_DATE_EPOCH while creating .pyc files iff %clamp_mtime_to_source_date_epoch is not 1)
  • Propose a PR for the Python's bytecode invalidation mode patch for all Python versions that have it
  • Backport (the new portion of) the patch to older Pythons (python2.7, python3.6 and PyPys)
  • Test everything together in Copr and deploy it if it works.
  • Optional: Run some reproducibility tests before and after this change and produce some statistics.

• Other developers:
  • Test their packages with the new behavior, report problems, and opt-out if really needed.

• Release engineering: N/A (not needed for this Change)

• Policies and guidelines: N/A (not needed for this Change)

• Trademark approval: N/A (not needed for this Change)

• Alignment with Objectives: N/A (not needed for this Change)

Contingency Plan

• Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
• Contingency deadline: N/A (not a System Wide Change)
• Blocks release? N/A (not a System Wide Change), Yes/No

Documentation

N/A (not a System Wide Change)