From Fedora Project Wiki
(a note about python 2 packages already built in rhel)
m (be more explicit in files)
 
(29 intermediate revisions by 3 users not shown)
Line 3: Line 3:
=== Why Python 3 in EPEL 7, Why not SCLs ===
=== Why Python 3 in EPEL 7, Why not SCLs ===


Due to Fedora plans for migration to Python 3 [1], it is becoming necessary to have Python 3 accessible in some form in EPEL 7. This will allow upstream projects to move to Python 3 (dropping Python 2 support) and still be usable through EPEL repositories. Some key stakeholders in this regard are Fedora Infra and DNF.
Due to [[Changes/Python 3 as Default|Fedora plans for migration to Python 3]], it is becoming necessary to have Python 3 accessible in some form in EPEL 7. This will allow upstream projects to move to Python 3 (dropping Python 2 support) and still be usable through EPEL repositories. Some key stakeholders in this regard are Fedora Infra and DNF.


Another important reason is further promotion of Python 3 and raising awareness for enterprise-level Python users.
Another important reason is further promotion of Python 3 and raising awareness for enterprise-level Python users.
Line 14: Line 14:
* Python minor versions are not ABI compatible, rebuilds are needed (both because of libpython soname change and because of bytecode magic numbers).
* Python minor versions are not ABI compatible, rebuilds are needed (both because of libpython soname change and because of bytecode magic numbers).
* Python has a good support for parallel installable stacks, even for minor versions of one major version (e.g. 3.3 and 3.4 can easily be installable alongside).
* Python has a good support for parallel installable stacks, even for minor versions of one major version (e.g. 3.3 and 3.4 can easily be installable alongside).
* EPEL policies try to discourage updates that break ABI and require rebuilds of other packages - due to the above fact, I consider Python minor version update a "major version update" as specified in EPEL update guidelines [2]. And as the guidelines say, this should be avoided if at all possible.
* EPEL policies try to discourage updates that break ABI and require rebuilds of other packages - due to the above fact, I consider Python minor version update a "major version update" as specified in [[EPEL/GuidelinesAndPolicies#A_major_version_update|EPEL update guidelines]]. And as the guidelines say, this should be avoided if at all possible.


=== Proposal ===
=== Proposal ===


* We will package Python 3 as python3X parallel-installable package (X is minor version, e.g. "4" ATM).
* Python 3 is packaged as python3X parallel-installable package (X is minor version, e.g. "4" ATM) that can coexist with other parallel installable python3Y stacks.
* All extension packages will be named python3X-foo. They will follow Fedora's python3 packaging guidelines with the exception that name contains both major and minor Python version (in Fedora it's only major).
* One of these stacks is always considered the "default" in the sense that it owns <code>/usr/bin/python3</code>.
* The whole python3X stack will be parallel installable with any other python3Y stack.
* All binary RPMs of extension packages must be named python3X-foo. They must follow Fedora's python3 packaging guidelines with the exception that name contains both major and minor Python version (in Fedora it's only major). See below for packaging instructions.
* In a situation when python3X is in EPEL and 3.X+1 is released upstream, the following should happen:
* Most of the time, there is only one python3X stack in EPEL. During transitional periods, after 3X+1 is released, there are two.
** python3X+1 package will be created for EPEL ASAP and all extension packages should be built also for this new python3X+1 stack. (While the python3X stack would stay untouched the whole time.)
* In a situation when python3X is in EPEL and 3.X+1 is released upstream, the following happens:
** When all packages are rebuilt for python3X+1, the old python3X stack will be retired after some amount of time (TODO: how long?, TODO: would this align with the EPIC proposal somehow?).
** python3X+1 package is created for EPEL ASAP and all extension packages are built also for this new python3X+1 stack.
** "/usr/bin/python3" will belong to the "stable" python3X stack, hence it should be only changed to point to the new stack once the old is obsoleted. Usage of "/usr/bin/python3" should be discouraged in favour of using /usr/bin/python3.X explicitly (TODO: at which point should "/usr/bin/python3" be switched from python3X to python3X+1?)
** When all packages are rebuilt for python3X+1, the old python3X stack is retired after certain period. This period gives everyone enough time to rebuild their packages while being as short as possible. There is intentionally no hard limit here, we will approach this case by case.
** <code>/usr/bin/python3</code> belongs to the "stable" python3X stack. Switching <code>/usr/bin/python3</code> from python3X to python3X+1 happens shortly before the end of transitional period (== before obsoleting python3X) and it is announced on epel-devel.
** Usage of <code>/usr/bin/python3</code> is discouraged in favour of using <code>/usr/bin/python3.X</code> explicitly.


==== Packaging Parallel python3X stacks ====
==== Packaging Parallel python3X stacks ====


* Automatic dependency generators and bytecompilation hooks should do fine.
* Automatic dependency generators and bytecompilation hooks work fine.
* SRPM/RPM naming:
* RPM naming:
** As mentioned above, binary RPMs will be named python3X for Python itself and python3X-foo for extension packages.
** As mentioned above, binary RPMs are named python3X for Python itself and python3X-<name> for extension packages.
** Source RPMs should also be named python3X-foo.srpm, although we could create python3-foo.srpm and build python3X-foo.rpm and python3X+1-foo.rpm out of it. Doing the latter is pretty similar to what we're doing in Fedora with parallel python2/python3 stacks and may be easier to maintain. (We could even create just python-foo.srpm, but that might be confusing, since the same SRPMs could exist in RHEL/CentOS 7, which are used for the Python 2 RHEL 7 stack.)
* SRPM naming, dist-git:
* Dist-git possibilities:
** We're using Fedora-like dist-git repos:
** Having python3X-foo dist-git repo for every package. Easy and obvious solution, but packages would have to be re-reviewed and new repos created with every Python 3 minor release. This would align with the first SRPM variant (python3X-foo SRPMS).
*** If a python3-<name> Fedora repo exists, it must be used. If a python-<name> SRPM exists in base RHEL, python3-<name> must be created and used (see [[#Explanation]]).
** Using current dist-git repos, e.g. branch epel-7 in python-foo (or python3-foo, if such Fedora dist-git exists). With this approach, we'd create python3-foo.srpm and build python3X and possibly python3X+1 RPMs out of it. The disadvantage is that some packages would be in python-foo and others in python3-foo dist git repos, which could get confusing. Also, their packaging could in time diverge a lot, so it might not make sense to do this.
*** Otherwise if python-<name> Fedora repo exists, it must be used.
** Or we could do the above, but create special epel7-python3-foo dist-git repos to keep things more organized and separated from standard Python repos (I'd prefer this).
*** Otherwise the package doesn't exist in Fedora and packager should decide before review which of the above he'll choose to have created, according to Fedora Python Packaging Guidelines.
** SRPM names are same as dist-git repo names. This means that we can have python-<name> dist-git repo and python-<name> SRPM which produces just python34-<name> RPM, while not producing python-<name> RPM.


==== What Will a Specfile Look Like? ====
==== Specfiles, Macros, Packaging Process ====


I experimented with several possibilities and this one seems to be the best:
{{admon/note|Libraries vs. Applications|For purposes of the following text, ''libraries'' are "python-<name>" packages - Python extension modules that are built for every interpreter and are imported by other ''libraries'' or ''applications''. ''Libraries'' specfiles should allow building subpackages for both "python3" and "python3_other". ''Applications'' are packages that provide "end user functionality" and it only makes sense to build them for one Python runtime}}
 
An example specfile of a ''library'' follows follows:


<pre>
<pre>
%global with_python3 1
# note: macros %%python3_pkgversion, %%python3_other_pkgversion and %%with_python3_other are defined
%if 0%{?rhel} > 6
# in the minimal buildroot; %%python3_pkgversion is also available in Fedora, so it's possible to have a common
%global python3_pkgversion_nodots 34
# specfile for EPEL and Fedora
%global with_python3_other 1
 
%global python3_other_pkgversion_nodots 35
%global srcname example
%else
 
%global python3_pkgversion_nodots 3
%global common_description %{expand:
An python module which provides a convenient example.  It is simple and has a
nice multiline description.}
 
%if %{defined el6} || %{defined el7}
%bcond_without python2
%endif
%endif


Name:          python-X
%bcond_without python3
Version:        1
 
%if %{defined el7}
%bcond_without python3_other
%endif
 
 
Name:          python-%{srcname}
Version:        1.2.3
Release:        1%{?dist}
Release:        1%{?dist}
Summary:        X
Summary:        An example python module
Source:         X
License:       MIT
URL:            https://pypi.org/project/%{srcname}/
Source0:        %pypi_source
BuildArch:      noarch


License:        MIT
 
URL:            http://X
%description %{common_description}
%if 0%{?with_python3}
 
# BR: python%{python3_pkgversion_nodots}-devel
 
# and other BR
%if %{with python2}
%package -n python2-%{srcname}
Summary:       %{summary}
BuildRequires:  python2-devel
BuildRequires:  python2-setuptools
BuildRequires:  python2-othermodule
Requires:      python2-othermodule
%{?python_provide:%python_provide python2-%{srcname}}
 
 
%description -n python2-%{srcname} %{common_description}
%endif
%endif


%if 0%{?with_python3_other}
# BR: python%{python3_other_pkgversion_nodots}-devel
# and other BR
%endif


%description
%if %{with python3}
X.
%package -n python%{python3_pkgversion}-%{srcname}
Summary:        %{summary}
BuildRequires:  python%{python3_pkgversion}-devel
BuildRequires:  python%{python3_pkgversion}-setuptools
BuildRequires:  python%{python3_pkgversion}-othermodule
Requires:      python%{python3_pkgversion}-othermodule
%{?python_provide:%python_provide python%{python3_pkgversion}-%{srcname}}


%if 0%{?with_python3}
%package -n python%{python3_pkgversion_nodots}-X
Summary: python%{python3_pkgversion_nodots} build of X


%description -n python%{python3_pkgversion_nodots}-X
%description -n python%{python3_pkgversion}-%{srcname} %{common_description}
python%{python3_pkgversion_nodots} build of X.
%endif
%endif


%if 0%{?with_python3_other}
%package -n python%{python3_other_pkgversion_nodots}-X
Summary: python%{python3_other_pkgversion_nodots} build of X


%description -n python%{python3_other_pkgversion_nodots}-X
%if %{with python3_other}
python%{python3_other_pkgversion_nodots} build of X.
%package -n python%{python3_other_pkgversion}-%{srcname}
Summary:        %{summary}
BuildRequires:  python%{python3_other_pkgversion}-devel
BuildRequires:  python%{python3_other_pkgversion}-setuptools
BuildRequires:  python%{python3_other_pkgversion}-othermodule
Requires:      python%{python3_other_pkgversion}-othermodule
%{?python_provide:%python_provide python%{python3_other_pkgversion}-%{srcname}}
 
 
%description -n python%{python3_other_pkgversion}-%{srcname} %{common_description}
%endif
%endif


%prep
%prep
%setup -q -n X-%{version}
%autosetup -n %{srcname}-%{version}


%if 0%{?with_python3}
# do what you do for Fedora's Python 3 stack
%endif


%if 0%{?with_python3_other}
%build
# do the same as above, just with pytohn3_other macro set
%{?with_python2:%py2_build}
%endif
%{?with_python3:%py3_build}
%{?with_python3_other:%py3_other_build}


%build
# do it like in Fedora, add with_python3_other


%install
%install
# do it like in Fedora, add with_python3_other
# Must do the python3_other install first, then python3 and then python2.
# The scripts in /usr/bin are overwritten with every setup.py install.
%{?with_python3_other:%py3_other_install}
%{?with_python3:%py3_install}
%{?with_python2:%py2_install}
 
 
%check
%{?with_python2:%{__python2} setup.py test}
%{?with_python3:%{__python3} setup.py test}
%{?with_python3_other:%{__python3_other} setup.py test}


%if 0%{?with_python3}
 
%files -n python%{python3_pkgversion_nodots}-X
# Note that there is no %%files section for the unversioned python module if we are building for several python runtimes
%if %{with python2}
%files -n python2-%{srcname}
%license COPYING
%doc README.rst
%{python2_sitelib}/%{srcname}
%{python2_sitelib}/%{srcname}-%{version}-py%{python2_version}.egg-info
%{_bindir}/sample-exec-%{python2_version}
%endif
%endif


%if 0%{?with_python3_other}
 
%files -n python%{python3_other_pkgversion_nodots}-X
%if %{with python3}
%files -n python%{python3_pkgversion}-%{srcname}
%license COPYING
%doc README.rst
%{python3_sitelib}/%{srcname}
%{python3_sitelib}/%{srcname}-%{version}-py%{python3_version}.egg-info
%{_bindir}/sample-exec
%{_bindir}/sample-exec-%{python3_version}
%endif
 
 
%if %{with python3_other}
%files -n python%{python3_other_pkgversion}-%{srcname}
%license COPYING
%doc README.rst
%{python3_other_sitelib}/%{srcname}
%{python3_other_sitelib}/%{srcname}-%{version}-py%{python3_other_version}.egg-info
%{_bindir}/sample-exec-%{python3_other_version}
%endif
%endif


%changelog
%changelog
</pre>
</pre>


===== Explanation =====
Note, that ''application'' specfile would be adapted from Fedora only by changing dependency tags (Requires/BuildRequires/...) to use<code>%{python3_pkgversion}</code>.
 
===== Guidelines =====


For start, packagers will be able to merge specfile from Fedora and use it - all macros like <code>%__python3</code> or <code>%python3_sitelib</code> will be provided by the current python3X-devel. It'll even be possible to use the same specfile for Fedora and EPEL with minimal effort. When merging specfiles from Fedora, packagers will need to:
For start, packagers are able to merge specfile from Fedora and use it - all macros like <code>%__python3</code> or <code>%python3_sitelib</code> are provided by the current python3X-devel. It's even be possible to use the same specfile for Fedora and EPEL with minimal effort.


* do initial Fedora import
The process of importing a specfile from Fedora to EPEL follows:
* add conditionalized defintions of <code>python3_pkgversion_nodots</code>, <code>with_other_python3</code> and <code>python3_other_pkgversion_nodots</code> (see the top of the specfile)
* change Fedora's <code>python3-X</code> subpacke name to <code>python%{python3_pkgversion_nodots}-X</code>
* add defintion and %prep, %build and %install steps of <code>python%{python3_other_pkgversion_nodots}-X</code> subpackage. which looks exactly the same as <code>python%{python3_pkgversion_nodots}-X</code> subpackage, it just uses different macros


Changing stacks when a new python3X+1 build is introduced or python3X is retired:
* Do initial Fedora import.
* during periods when only a single Python 3 runtime is in EPEL 7, packages will be built with <code>%with_python3</code> enabled and with <code>%with_python3_other</code> disabled
* when a new python3X+1 is built in EPEL - let's say that there is python34 and python35 has just been introduced:
** <code>%with_python3_other</code> is enabled in all packages, <code>%python3_pkgversion_nodots</code> is still set to 34, <code>%python3_other_pkgversion_nodots</code> is still set to 35
** mass rebuild is run to build as much of the new stack possible automatically
** at a certain point at time, annoucement is made that python34 is to be retired and python35 is to be *the* one - at this point, python34 and python35 will be rebuilt; python34-devel will provide the "other" macros from now on, while python35 will provide the "default" macros (note that all the specfiles still build just fine as they are)
** some time after python34 and python35 are rebuilt, <code>%python3_pkgversion_nodots</code> is set to "35", <code>%python3_other_pkgversion_nodots</code> is set to "36" and <code>%with_python3_other</code> is disabled
** no rebuild is needed at this point, but all packages build just python35-X subpackages
** now the whole python34 stack is disabled


All the painful parts in the above process can be done automatically (enabling/disabling "with" macros, changing values of "pkgversion" macros, mass rebuilding).
{{admon/warning|Packages already in RHEL|Some packages imported from Fedora, for example python-setuptools, would have same SRPM name (possibly different version) as packages in base RHEL. Since SRPM names must not conflict, it is necessary to create python3-<name> package for these (assuming it doesn't exist yet in Fedora), do a new package review and build it just for EPEL (the package will likely have python3- subpackage already in Fedora built from python-<name>).}}


{{admon/warning|subpackages for Python 2 already in RHEL|Some packages imported from Fedora, for example setuptools, already have their Python 2 versions built in RHEL and these must not be replaced/updated by EPEL packages. This means that for these packages, the packager will either need to remove the python 2 build parts from the specfile or he'll need to conditionalize them to be only built in Fedora.}}
* For ''applications''
** Change dependency tags (Requires/BuildRequires/...) from <code>python3-<depname></code> to <code>python%{python3_pkgversion}-<depname></code>
* For ''libraries''
** Change Fedora's <code>python3-X</code> subpackage name to <code>python%{python3_pkgversion}-X</code>, change dependency tags (Requires/BuildRequires/...) in the same way for this subpackage.
** Add definition and %prep, %build and %install (and %check) steps of <code>python%{python3_other_pkgversion}-X</code> subpackage. which looks exactly the same as <code>python%{python3_pkgversion}-X</code> subpackage, it just uses different macros; build of this subpackage has to be conditionalized by the <code>%with_python3_other</code> macro, which is defined in the minimal buildroot.
** Scripts/entry-points (if any) in <code>%{_bindir}</code> should be split like this among subpackages:
*** python-X subpackage should own <code>%{_bindir}/foo</code> and possibly <code>%{_bindir}/foo-2</code> and <code>%{_bindir}/foo-2.7</code> files
*** python%{python3_pkgversion}-X should own <code>%{_bindir}/foo-3</code> and <code>%{_bindir}/foo-%{python3_version}</code> files
*** python%{python3_other_pkgversion}-X should own just <code>%{_bindir}/foo-%{python3_other_version}</code>
*** ''Note: often, upstreams implement their own entrypoint versioning and there are several different schemes they use: "foo-3.X", "foo-3X", "foo3.X", "foo3X". If upstream has any of these, it is advisable to adopt it. Packager should only manually create versioned files when they don't exist.''
* For both ''applications'' and ''libraries''
** Make sure that all hashbangs in resulting package are in form "#!/usr/bin/python3.X". Macros <code>%{__python3}</code> and <code>%{__python3_other}</code> in EPEL have these values, so if you invoke <code>%{__python3} setup.py <action></code>, the entry points will automatically be set correctly. You must replace any upstream hardcoded hashbangs during build by using sed or other means.
* TODO: provide the macro names and definitions here


[1] http://fedoraproject.org/wiki/Changes/Python_3_as_Default
Lifecycle of python3X stacks, rebuilding:
[2] https://fedoraproject.org/wiki/EPEL/GuidelinesAndPolicies#A_major_version_update
* during periods when only a single Python 3 runtime is in EPEL, packages must be built with <code>%with_python3</code> enabled (<code>%with_python3_other</code> is disabled in minimal buildroot)
* when a new python3X+1 is built in EPEL - let's say that there is python34 and python35 has just been introduced:
** <code>%with_python3_other</code> is enabled in minimal buildroot, <code>%python3_pkgversion</code> is still set to 34, <code>%python3_other_pkgversion</code> is still set to 35
** (scripted) mass rebuild is run to build as much of the new stack possible automatically
** all ''libraries'' should be rebuilt to provide python35-<name> subpackages
** at a certain point at time, annoucement is made that python34 is to be retired and python35 is to be *the* one - at this point, python34 and python35 are rebuilt
*** python34-devel now provides the "other" macros, while python35 provides the "default" macros (note that all the specfiles still build just fine as they are)
** all ''applications'' should be rebuilt to depend on python35 instead of python34
** <code>%with_python3_other</code> is disabled in minimal buildroot
** <code>%python3_pkgversion</code> is set to "35", <code>%python3_other_pkgversion</code> is set to "36"
** no rebuild is needed at this point, but all packages build just python35-<name> subpackages
** now the whole python34 stack can be retired (TODO: how?)

Latest revision as of 04:47, 10 December 2018

EPEL 7 in Python 3 Plan Draft

Why Python 3 in EPEL 7, Why not SCLs

Due to Fedora plans for migration to Python 3, it is becoming necessary to have Python 3 accessible in some form in EPEL 7. This will allow upstream projects to move to Python 3 (dropping Python 2 support) and still be usable through EPEL repositories. Some key stakeholders in this regard are Fedora Infra and DNF.

Another important reason is further promotion of Python 3 and raising awareness for enterprise-level Python users.

Because of key stakeholders like Fedora Infra and DNF, SCLs as a solution to this problem were ruled out. SCLs are currently not allowed in EPEL (and it seems they won't be allowed in near future) and since these projects need/strongly prefer to be in EPEL itself, SCLs are not a viable alternative for them.

Some Facts

  • Python 3 is actively developed by upstream and new minor versions are released approximately every year. Latest Python 3 minor is 3.4.
  • Python minor versions are not ABI compatible, rebuilds are needed (both because of libpython soname change and because of bytecode magic numbers).
  • Python has a good support for parallel installable stacks, even for minor versions of one major version (e.g. 3.3 and 3.4 can easily be installable alongside).
  • EPEL policies try to discourage updates that break ABI and require rebuilds of other packages - due to the above fact, I consider Python minor version update a "major version update" as specified in EPEL update guidelines. And as the guidelines say, this should be avoided if at all possible.

Proposal

  • Python 3 is packaged as python3X parallel-installable package (X is minor version, e.g. "4" ATM) that can coexist with other parallel installable python3Y stacks.
  • One of these stacks is always considered the "default" in the sense that it owns /usr/bin/python3.
  • All binary RPMs of extension packages must be named python3X-foo. They must follow Fedora's python3 packaging guidelines with the exception that name contains both major and minor Python version (in Fedora it's only major). See below for packaging instructions.
  • Most of the time, there is only one python3X stack in EPEL. During transitional periods, after 3X+1 is released, there are two.
  • In a situation when python3X is in EPEL and 3.X+1 is released upstream, the following happens:
    • python3X+1 package is created for EPEL ASAP and all extension packages are built also for this new python3X+1 stack.
    • When all packages are rebuilt for python3X+1, the old python3X stack is retired after certain period. This period gives everyone enough time to rebuild their packages while being as short as possible. There is intentionally no hard limit here, we will approach this case by case.
    • /usr/bin/python3 belongs to the "stable" python3X stack. Switching /usr/bin/python3 from python3X to python3X+1 happens shortly before the end of transitional period (== before obsoleting python3X) and it is announced on epel-devel.
    • Usage of /usr/bin/python3 is discouraged in favour of using /usr/bin/python3.X explicitly.

Packaging Parallel python3X stacks

  • Automatic dependency generators and bytecompilation hooks work fine.
  • RPM naming:
    • As mentioned above, binary RPMs are named python3X for Python itself and python3X-<name> for extension packages.
  • SRPM naming, dist-git:
    • We're using Fedora-like dist-git repos:
      • If a python3-<name> Fedora repo exists, it must be used. If a python-<name> SRPM exists in base RHEL, python3-<name> must be created and used (see #Explanation).
      • Otherwise if python-<name> Fedora repo exists, it must be used.
      • Otherwise the package doesn't exist in Fedora and packager should decide before review which of the above he'll choose to have created, according to Fedora Python Packaging Guidelines.
    • SRPM names are same as dist-git repo names. This means that we can have python-<name> dist-git repo and python-<name> SRPM which produces just python34-<name> RPM, while not producing python-<name> RPM.

Specfiles, Macros, Packaging Process

Note.png
Libraries vs. Applications
For purposes of the following text, libraries are "python-<name>" packages - Python extension modules that are built for every interpreter and are imported by other libraries or applications. Libraries specfiles should allow building subpackages for both "python3" and "python3_other". Applications are packages that provide "end user functionality" and it only makes sense to build them for one Python runtime

An example specfile of a library follows follows:

# note: macros %%python3_pkgversion, %%python3_other_pkgversion and %%with_python3_other are defined 
# in the minimal buildroot; %%python3_pkgversion is also available in Fedora, so it's possible to have a common
# specfile for EPEL and Fedora

%global srcname example

%global common_description %{expand:
An python module which provides a convenient example.  It is simple and has a
nice multiline description.}

%if %{defined el6} || %{defined el7}
%bcond_without python2
%endif

%bcond_without python3

%if %{defined el7}
%bcond_without python3_other
%endif


Name:           python-%{srcname}
Version:        1.2.3
Release:        1%{?dist}
Summary:        An example python module
License:        MIT
URL:            https://pypi.org/project/%{srcname}/
Source0:        %pypi_source
BuildArch:      noarch


%description %{common_description}


%if %{with python2}
%package -n python2-%{srcname}
Summary:        %{summary}
BuildRequires:  python2-devel
BuildRequires:  python2-setuptools
BuildRequires:  python2-othermodule
Requires:       python2-othermodule
%{?python_provide:%python_provide python2-%{srcname}}


%description -n python2-%{srcname} %{common_description}
%endif


%if %{with python3}
%package -n python%{python3_pkgversion}-%{srcname}
Summary:        %{summary}
BuildRequires:  python%{python3_pkgversion}-devel
BuildRequires:  python%{python3_pkgversion}-setuptools
BuildRequires:  python%{python3_pkgversion}-othermodule
Requires:       python%{python3_pkgversion}-othermodule
%{?python_provide:%python_provide python%{python3_pkgversion}-%{srcname}}


%description -n python%{python3_pkgversion}-%{srcname} %{common_description}
%endif


%if %{with python3_other}
%package -n python%{python3_other_pkgversion}-%{srcname}
Summary:        %{summary}
BuildRequires:  python%{python3_other_pkgversion}-devel
BuildRequires:  python%{python3_other_pkgversion}-setuptools
BuildRequires:  python%{python3_other_pkgversion}-othermodule
Requires:       python%{python3_other_pkgversion}-othermodule
%{?python_provide:%python_provide python%{python3_other_pkgversion}-%{srcname}}


%description -n python%{python3_other_pkgversion}-%{srcname} %{common_description}
%endif


%prep
%autosetup -n %{srcname}-%{version}


%build
%{?with_python2:%py2_build}
%{?with_python3:%py3_build}
%{?with_python3_other:%py3_other_build}


%install
# Must do the python3_other install first, then python3 and then python2.
# The scripts in /usr/bin are overwritten with every setup.py install.
%{?with_python3_other:%py3_other_install}
%{?with_python3:%py3_install}
%{?with_python2:%py2_install}


%check
%{?with_python2:%{__python2} setup.py test}
%{?with_python3:%{__python3} setup.py test}
%{?with_python3_other:%{__python3_other} setup.py test}


# Note that there is no %%files section for the unversioned python module if we are building for several python runtimes
%if %{with python2}
%files -n python2-%{srcname}
%license COPYING
%doc README.rst
%{python2_sitelib}/%{srcname}
%{python2_sitelib}/%{srcname}-%{version}-py%{python2_version}.egg-info
%{_bindir}/sample-exec-%{python2_version}
%endif


%if %{with python3}
%files -n python%{python3_pkgversion}-%{srcname}
%license COPYING
%doc README.rst
%{python3_sitelib}/%{srcname}
%{python3_sitelib}/%{srcname}-%{version}-py%{python3_version}.egg-info
%{_bindir}/sample-exec
%{_bindir}/sample-exec-%{python3_version}
%endif


%if %{with python3_other}
%files -n python%{python3_other_pkgversion}-%{srcname}
%license COPYING
%doc README.rst
%{python3_other_sitelib}/%{srcname}
%{python3_other_sitelib}/%{srcname}-%{version}-py%{python3_other_version}.egg-info
%{_bindir}/sample-exec-%{python3_other_version}
%endif


%changelog

Note, that application specfile would be adapted from Fedora only by changing dependency tags (Requires/BuildRequires/...) to use%{python3_pkgversion}.

Guidelines

For start, packagers are able to merge specfile from Fedora and use it - all macros like %__python3 or %python3_sitelib are provided by the current python3X-devel. It's even be possible to use the same specfile for Fedora and EPEL with minimal effort.

The process of importing a specfile from Fedora to EPEL follows:

  • Do initial Fedora import.
Warning.png
Packages already in RHEL
Some packages imported from Fedora, for example python-setuptools, would have same SRPM name (possibly different version) as packages in base RHEL. Since SRPM names must not conflict, it is necessary to create python3-<name> package for these (assuming it doesn't exist yet in Fedora), do a new package review and build it just for EPEL (the package will likely have python3- subpackage already in Fedora built from python-<name>).
  • For applications
    • Change dependency tags (Requires/BuildRequires/...) from python3-<depname> to python%{python3_pkgversion}-<depname>
  • For libraries
    • Change Fedora's python3-X subpackage name to python%{python3_pkgversion}-X, change dependency tags (Requires/BuildRequires/...) in the same way for this subpackage.
    • Add definition and %prep, %build and %install (and %check) steps of python%{python3_other_pkgversion}-X subpackage. which looks exactly the same as python%{python3_pkgversion}-X subpackage, it just uses different macros; build of this subpackage has to be conditionalized by the %with_python3_other macro, which is defined in the minimal buildroot.
    • Scripts/entry-points (if any) in %{_bindir} should be split like this among subpackages:
      • python-X subpackage should own %{_bindir}/foo and possibly %{_bindir}/foo-2 and %{_bindir}/foo-2.7 files
      • python%{python3_pkgversion}-X should own %{_bindir}/foo-3 and %{_bindir}/foo-%{python3_version} files
      • python%{python3_other_pkgversion}-X should own just %{_bindir}/foo-%{python3_other_version}
      • Note: often, upstreams implement their own entrypoint versioning and there are several different schemes they use: "foo-3.X", "foo-3X", "foo3.X", "foo3X". If upstream has any of these, it is advisable to adopt it. Packager should only manually create versioned files when they don't exist.
  • For both applications and libraries
    • Make sure that all hashbangs in resulting package are in form "#!/usr/bin/python3.X". Macros %{__python3} and %{__python3_other} in EPEL have these values, so if you invoke %{__python3} setup.py <action>, the entry points will automatically be set correctly. You must replace any upstream hardcoded hashbangs during build by using sed or other means.
  • TODO: provide the macro names and definitions here

Lifecycle of python3X stacks, rebuilding:

  • during periods when only a single Python 3 runtime is in EPEL, packages must be built with %with_python3 enabled (%with_python3_other is disabled in minimal buildroot)
  • when a new python3X+1 is built in EPEL - let's say that there is python34 and python35 has just been introduced:
    • %with_python3_other is enabled in minimal buildroot, %python3_pkgversion is still set to 34, %python3_other_pkgversion is still set to 35
    • (scripted) mass rebuild is run to build as much of the new stack possible automatically
    • all libraries should be rebuilt to provide python35-<name> subpackages
    • at a certain point at time, annoucement is made that python34 is to be retired and python35 is to be *the* one - at this point, python34 and python35 are rebuilt
      • python34-devel now provides the "other" macros, while python35 provides the "default" macros (note that all the specfiles still build just fine as they are)
    • all applications should be rebuilt to depend on python35 instead of python34
    • %with_python3_other is disabled in minimal buildroot
    • %python3_pkgversion is set to "35", %python3_other_pkgversion is set to "36"
    • no rebuild is needed at this point, but all packages build just python35-<name> subpackages
    • now the whole python34 stack can be retired (TODO: how?)