No more automagic Python bytecompilation (phase 3)
See Changes/No_more_automagic_Python_bytecompilation and Changes/No_more_automagic_Python_bytecompilation_phase_2. Now,
%global _python_bytecompile_extra 1 won't be allowed anymore and raises an error with a link to this change.
- Name: Lumír Balhar
- Email: email@example.com
- Targeted release: Fedora 33
- Last updated: 2020-06-15
- FESCo issue: #2401
- Tracker bug: #1847153
- Release notes tracker: #513
See the Detailed Description of the previous Change Proposal and the Detailed Description of its phase 2.
Currently, Fedora rawhide contains ~130 packages with
%global _python_bytecompile_extra 1 in their specfiles but surprisingly only 42 of them actually ship any .pyc files outside the standard location
/usr/lib(64)?/python[0-9]\.[0-9]+. That might be caused by either of the following:
- there is nothing to byte-compile — the statement is a leftover to be removed
- The automagic bytecompilation uses
/usr/bin/pythonby default (for historical reasons) but
/usr/bin/pythonis not present in the buildroot by default.
Our plan is to finish the announced removal of this automagic Python bytecompilation, remove it from packages where it's not necessary and fix it with
%py_byte_compile for packages where it's needed.
The documentation already contains the following paragraph:
If you have *.py files outside of the /usr/lib(64)?/pythonX.Y/ directories and you require those files to be byte compiled (e.g. it’s an importable Python module) you MUST compile them explicitly using the %py_byte_compile macro. Note that not all Python files are importable Python modules; when in doubt, grep the sources for the appropriate import statement.
so no changes have to be made there.
bugzilla calamares ceph chromium cinnamon-screensaver edk2 eog-plugins fish fleet-commander-admin fleet-commander-client freecad gaupol gazebo gdb gedit gedit-latex gedit-plugins git-cola glusterfs gnome-code-assistance gnome-devel-docs grass gtk-doc gtranslator ibus ibus-anthy ibus-hangul ibus-libpinyin ibus-libzhuyin ibus-pinyin ibus-table ibus-typing-booster ibus-uniemoji insight kajongg kdevelop-python klatexformula libpeas libreoffice libsmbios libunity lirc lyx mate-menu mingw-glib2 mirrormanager2 odcs onboard otf2 paraview pcs php-opencloud-openstack pygobject2 pygtk2 python-cherrypy python-flask-silk python-genshi python-pycurl python-reportlab qpid-dispatch rhythmbox sems sigul soundconverter sugar sugar-abacus sugar-browse sugar-calculator sugar-castle sugar-chat sugar-clock sugar-colordeducto sugar-countries sugar-deducto sugar-distance sugar-finance sugar-flip sugar-flipsticks sugar-fototoon sugar-fractionbounce sugar-getiabooks sugar-imageviewer sugar-implode sugar-infoslicer sugar-jukebox sugar-kuku sugar-labyrinth sugar-locosugar sugar-log sugar-maze sugar-measure sugar-memorize sugar-moon sugar-nutrition sugar-paint sugar-physics sugar-pippy sugar-playgo sugar-portfolio sugar-pukllanapac sugar-read sugar-recall sugar-record sugar-ruler sugar-speak sugar-srilanka sugar-starchart sugar-stopwatch sugar-story sugar-terminal sugar-turtleart sugar-typing-turtle sugar-view-slides sugar-visualmatch sugar-words sugar-write sugar-xoeditor sugar-xoirc sugar-yupana synfigstudio system-config-repo system-switch-java system-switch-mail texlive totem transmageddon ufw-kde variety virt-manager vtk xed
Explicit compilation with
%py_byte_compile is EPEL 6/7/8 compatible.
In EPEL 6 and EPEL 7, the
brp-python-bytecompile script compiles all "extra" directories with
%__python (if it is available in the buildroot, this happens always, there is no opt out other than disabling the entire script). The
%_python_bytecompile_extra value has no meaning and is not used. EPEL 6/7 compatible spec files adapted to work in Fedora 33+ (via the recommended way outline in this proposal) will continue to be EPEL 6/7 compatible (regarding Python byte compilation of "extra" directories). In the worst case scenario, some files will be byte compiled twice (explicitly, then automagically).
In EPEL 8, the "extra" directories are not compiled by default, but they are compiled with
%_python_bytecompile_extra is set to
1 (same as in e.g. Fedora 32 or Fedora 33 prior to this chnage). Once this starts to be an error in Fedora 33+, spec files brought from EPEL 8 with
%global _python_bytecompile_extra 1 to Fedora will need to be adapted. EPEL 8 compatible spec files adapted to work in Fedora 33+ (via the recommended way outline in this proposal) will continue to be EPEL 8 compatible (regarding Python byte compilation of "extra" directories).
Benefit to Fedora
More explicit specfiles when it comes to Python byte compilation. This change also supports the dropping of unversioned
- Proposal owners: Change the brp-python-bytecompile script and macros in redhat-rpm-config package, help with transforming the packages. See https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/87
- Other developers: Maintainers of the mentioned python packages will have to change their packages to use explicit
- Release engineering: No Release engineering work is needed for this change
- Policies and guidelines: nothing to be changed
- Trademark approval: not needed
How To Test
Build a package with
%global _python_bytecompile_extra 1. The build should fail with an actionable error message.
The users of this change are packagers. The new behavior should make byte-compilation more obvious, explicit, and discoverable. Users of Fedora should not feel this (except if this change uncovers a packaging bug).
- Contingency mechanism: revert the changes in redhat-rpm-config
- Contingency deadline: none (not a System Wide Change)
- Blocks release? no (not a System Wide Change)
- Blocks product? no
The guidelines already contain up-to-date documentation.
This change does not deserve Release Notes, it is not user-facing.