From Fedora Project Wiki
m (releng issue)
m (fix typo)
 
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{DISPLAYTITLE:Make ambiguous python shebangs error}}
{{DISPLAYTITLE:Changes/Make ambiguous python shebangs error}}


<!-- Self Contained or System Wide Change Proposal?
<!-- Self Contained or System Wide Change Proposal?
Line 29: Line 29:
Note that motivation for the change should be in the Motivation section below, and this part should answer the question "What?" rather than "Why?". -->
Note that motivation for the change should be in the Motivation section below, and this part should answer the question "What?" rather than "Why?". -->


The <code>/usr/lib/rpm/redhat/brp-mangle-shebangs</code> buildroot policy script will be changed to make the build fail when it sees an ambiguous python shebang (such as <code>#!/usr/bin/python</code> or <code>#!/usr/bin/env python</code>). It currently says ''This will become an ERROR'' for 2 Fedora releases.
The <code>/usr/lib/rpm/redhat/brp-mangle-shebangs</code> buildroot policy script will be changed to make the build fail when it sees an ambiguous python shebang, such as <code>#!/usr/bin/python</code> or <code>#!/usr/bin/env python</code>. (The script has been warning in these cases for 2 Fedora releases already, saying ''This will become an ERROR''.)


== Owner ==
== Owner ==
Line 59: Line 59:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1626684 #1626684]


== Detailed Description ==
== Detailed Description ==
Line 75: Line 75:
The script will exit with nonzero exit code, rendering the build failed.
The script will exit with nonzero exit code, rendering the build failed.


The warning and a promis of the error was there for 2 releases (28 and 29).
The warning and a promise of the error was there for 2 releases (28 and 29). Taskotron check was also present.


There are standard mechanics to avoid this buildroot policy script or to block certain files form it. Those remain intact by this change. For details see [[Packaging:Guidelines#Shebang_lines|Shebang lines]] and [[Packaging:Guidelines#BRP_.28BuildRoot_Policy.29_Scripts|BuildRoot Policy Scripts]] sections of the packaging guidelines.
There are standard mechanics to avoid this buildroot policy script or to block certain files form it. Those remain intact by this change. For details see [[Packaging:Guidelines#Shebang_lines|Shebang lines]] and [[Packaging:Guidelines#BRP_.28BuildRoot_Policy.29_Scripts|BuildRoot Policy Scripts]] sections of the packaging guidelines.
=== Using pathfix.py to fix shebangs ===
Packagers are encouraged to use <code>pathfix.py</code> to fix shebangs. Here is an example. First, buildrequire the tool:
BuildRequires: /usr/bin/pathfix.py
Note: The examples bellow use some python macros (<code>%{__pythonX}</code> and <code>%{pyX_shbang_opts}</code>). BuildRequire {{package|python3-devel}} (and/or  {{package|python2-devel}}) to get those.
Fix the shebangs ideally in prep:
%prep
...setup...
# Fix all Python shebangs recursively in .
# -p preserves timestamps
# -n prevents creating ~backup files
# -i specifies the interpreter for the shebang
# Need to list files that do not match ^[a-zA-Z0-9_]+\.py$ explicitly!
pathfix.py -pni "%{__python3} %{py3_shbang_opts}" . whatnot/file-with-dashes.py else/no_py_extension
Sometimes shebanged files are created "magically" during install, so the fix needs to be applied after that:
%install
...install stuff...
# Some files got ambiguous python shebangs, we fix them after everything else is done
pathfix.py -pni "%{__python2} %{py2_shbang_opts}" %{buildroot}%{python2_sitearch}
pathfix.py -pni "%{__python3} %{py3_shbang_opts}" %{buildroot}%{python3_sitearch} %{buildroot}%{_bindir}/*


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 109: Line 138:
-->
-->


Packagers will be notified by build error if they accidentally have python2 shebangs (and python2 dependency) they didn't anticipated. It's up to them to decide what to do with such files, no automation can know.
Packagers will be notified by build error if they accidentally have python2 shebangs (and python2 dependency) they didn't anticipate. It's up to them to decide what to do with such files, no automation can know.


== Scope ==
== Scope ==
Line 181: Line 210:
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks release? N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->


Line 199: Line 228:
-->
-->


[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF30]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Latest revision as of 19:21, 20 March 2019



Make ambiguous python shebangs error

Summary

The /usr/lib/rpm/redhat/brp-mangle-shebangs buildroot policy script will be changed to make the build fail when it sees an ambiguous python shebang, such as #!/usr/bin/python or #!/usr/bin/env python. (The script has been warning in these cases for 2 Fedora releases already, saying This will become an ERROR.)

Owner

  • Name: Miro Hrončok
  • Email: <mhroncok@redhat.com>
  • Release notes owner:

Current status

Detailed Description

The buildroot policy script in /usr/lib/rpm/redhat/brp-mangle-shebangs currently changes all python shebangs to python2 with a message like:

*** WARNING: mangling shebang in /usr/bin/taskotron_result from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!

We will change it to:

*** ERROR: ambiguous python shebang in /usr/bin/taskotron_result: #!/usr/bin/python. Change it to python3 (or python2) explicitly.

The script will exit with nonzero exit code, rendering the build failed.

The warning and a promise of the error was there for 2 releases (28 and 29). Taskotron check was also present.

There are standard mechanics to avoid this buildroot policy script or to block certain files form it. Those remain intact by this change. For details see Shebang lines and BuildRoot Policy Scripts sections of the packaging guidelines.

Using pathfix.py to fix shebangs

Packagers are encouraged to use pathfix.py to fix shebangs. Here is an example. First, buildrequire the tool:

BuildRequires: /usr/bin/pathfix.py

Note: The examples bellow use some python macros (%{__pythonX} and %{pyX_shbang_opts}). BuildRequire Package-x-generic-16.pngpython3-devel (and/or Package-x-generic-16.pngpython2-devel) to get those.

Fix the shebangs ideally in prep:

%prep
...setup...

# Fix all Python shebangs recursively in .
# -p preserves timestamps
# -n prevents creating ~backup files
# -i specifies the interpreter for the shebang
# Need to list files that do not match ^[a-zA-Z0-9_]+\.py$ explicitly!
pathfix.py -pni "%{__python3} %{py3_shbang_opts}" . whatnot/file-with-dashes.py else/no_py_extension

Sometimes shebanged files are created "magically" during install, so the fix needs to be applied after that:

%install
...install stuff...

# Some files got ambiguous python shebangs, we fix them after everything else is done
pathfix.py -pni "%{__python2} %{py2_shbang_opts}" %{buildroot}%{python2_sitearch}
pathfix.py -pni "%{__python3} %{py3_shbang_opts}" %{buildroot}%{python3_sitearch} %{buildroot}%{_bindir}/*

Benefit to Fedora

Packagers will be notified by build error if they accidentally have python2 shebangs (and python2 dependency) they didn't anticipate. It's up to them to decide what to do with such files, no automation can know.

Scope

  • Other developers: fix their packages if they fail because of this
  • Release engineering: #7679 (a check of an impact with Release Engineering is needed)
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

Have an RPM package that tries to ship files with ambiguous python shebang. Observe the warning on Fedora 29 and the error on Fedora 30.

User Experience

Users should not notice this, expect there might be less unneeded python2 dependencies created by accident.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? N/A (not a System Wide Change)
  • Blocks product? product

Documentation

The Python Packaging Guidelines will be the documentation.

Release Notes

TBD