From Fedora Project Wiki

< Changes

Revision as of 11:03, 23 May 2018 by Churchyard (talk | contribs) (Scope)



Move /usr/bin/python into a separate package

Summary

Reflecting the recent changes of PEP 394 -- The "python" Command on Unix-Like Systems, we are moving /usr/bin/python from the Package-x-generic-16.pngpython2 package into a separate package called Package-x-generic-16.pngpython-unversioned-command. Package-x-generic-16.pngpython2 will recommend this package.

This means Fedora users will get it automatically with Package-x-generic-16.pngpython2, but they might opt-out and remove it. In Fedora's build system, only packages explicitly buildrequiring /usr/bin/python will get it.

This change obsoletes "Avoid /usr/bin/python in RPM build" change.


Owner

  • Release notes owner:

Current status

  • Targeted release: Fedora 29
  • Last updated: 2018-05-23
  • Tracker bug: <will be assigned by the Wrangler>

Detailed Description

Motivation

The meaning of the python command is ambiguous: it might mean either Python 2 or Python 3, depending on context. The upstream recommendation (PEP 394), which we try to follow in Fedora, is that users -- not distros, and not sysadmins -- should be in control of the python command.

Specifically, this means distro-packaged software should use python2 or python3 explicitly. Fedora's Python packaging guidelines have suggested this since February 2015, and demand it since August 2017. However, enforcing this guideline (which we will need to actually allow users to safely change their python command) is problematic. "Avoid /usr/bin/python in RPM_Build" change didn't work: most of the packagers didn't change anything and too many packages still use the python command.

Instead of developing custom hacks, we are now utilizing a more systematic solution: If your package still needs /usr/bin/python to build, explicitly BuildRequire it. If your package needs /usr/bin/python to run, explicitly Require it. (In both cases, try to use /usr/bin/python2 or /usr/bin/python3 instead if possible.)

We are also expecting some buildsystems (autools, cmake, etc.) to automatically use /usr/bin/python2 if /usr/bin/python is unavailable, so the problem might go away more naturally.

PEP 394 and how it is mapped to this change

Upstream PEP 394 -- The "python" Command on Unix-Like Systems describes how the python command should behave. In Fedora, that's /usr/bin/python. The PEP was recently updated to reflect upstream's evolving views on the situation. Notable new information from the PEP is:

In controlled environments aimed at expert users, where being explicit is valued over user experience (for example, in test environments and package build systems), distributions may choose to not provide the python command even if python2 is available. (All software in such a controlled environment must use python3 or python2 rather than python, which means scripts that deliberately use python need to be modified for such environments.)

We consider Fedora's build machinery a controlled environments aimed at expert users.

What's changing

"Avoid /usr/bin/python in RPM_Build" change is reverted. Packages that follow its Quick Opt-Out section (i.e. set PYTHON_DISALLOW_AMBIGUOUS_VERSION) will be fixed by the owners of this change.

/usr/bin/python remains a symbolic link to /usr/bin/python2. However, it is moved to a new Package-x-generic-16.pngpython-unversioned-command package (technically a subpackage of Package-x-generic-16.pngpython2). Package-x-generic-16.pngpython2 package will only recommend /usr/bin/python.

Packages that need /usr/bin/python to build will need to BuildRequire it. Packages that need /usr/bin/python to be used by users will need to Require it. In both cases, the packager should avoid the need and only fallback to (Build)Requiring /usr/bin/python as a temporary workaround.

The new package will virtually provide python, ensuring that dnf install python will make the python command available.

Effect on automatic bytecompilation

When "No more automagic Python bytecompilation" change is done, packages that byte-compile files outside of Python directories should switch to the new behavior described in that change, and should not be impacted by this change. However, if that change is delayed or reverted, packagers that rely on the old behavior when byte compiling files will need to set %__python to python2 or python3 explicitly.

Effect on %__python and other ambiguous RPM macros

Using ambiguous Python macros (%{__python}, %{python_sitelib}...) is forbidden and your package will fail to build if you still use those without redefining %__python. Either switch to explicitly versioned macros (%{__python2}, %{python2_sitelib}, %{__python3}...) or set __python to an explicit Python version.

Benefit to Fedora

This change brings us one step closer to a seamless transition to Python 3, while following upstream recommendations.

This change allows experienced users to remove /usr/bin/python in a supported way.

Scope

  • Proposal owners:
    • Split the packages as described in Detailed Description.
    • Fix packages that use PYTHON_DISALLOW_AMBIGUOUS_VERSION to BuildRequire /usr/bin/python instead.
  • Other developers:
    • Maintainers of packages that use /usr/bin/python need to switch to using /usr/bin/python3 or /usr/bin/python2 explicitly (with help from Proposal owners if needed).
      • While doing that, consider switching your package to Python 3 only, if the Python 2 bits are unused in Fedora. (This is not necessarily required for this change, however it will make your packaging job easier.)
      • If that can't be done in a timely manner, fallback to (Build)Requiring /usr/bin/python as a temporary workaround.
  • Policies and guidelines:
    • Already existing: "packages in Fedora ... MUST call the proper executable for the needed python major version directly, either /usr/bin/python2 or /usr/bin/python3 as appropriate" from Packaging:Python#Multiple_Python_Runtimes.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

N/A (not a System Wide Change)

User Experience

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? N/A (not a System Wide Change), Yes/No
  • Blocks product? product

Documentation

N/A (not a System Wide Change)

Release Notes