From Fedora Project Wiki
m (The virtual provide should be versioned...)
 
(12 intermediate revisions by the same user not shown)
Line 6: Line 6:


== Current State ==
== Current State ==
python-* packages (built with Python 2) are installed by default (from LiveCD, as dependencies of applications that require some Python packages) and by <code>yum install python-foo</code>. Python 3 packages are named python3-* and are installed as <code>yum install python3-foo</code>.
<code>python-*</code> packages (built with Python 2) are installed by default (from LiveCD, as dependencies of applications that require some Python packages) and by <code>yum install python-foo</code>. Python 3 packages are named <code>python3-*</code> and are installed by <code>yum install python3-foo</code>.


From the packaging point of view, this is achieved by either
From the packaging point of view, this is achieved by either
* Having one SRPM python-foo that produces python-foo and python3-foo binary RPMs.
* Having one SRPM <code>python-foo</code> that produces <code>python-foo</code> and <code>python3-foo</code> binary RPMs.
or
or
* Having two SRPMs, python-foo (produces python-foo binary RPM) and python3-foo SRPM (produces python3-foo binary RPM).
* Having two SRPMs, <code>python-foo</code> (produces <code>python-foo</code> binary RPM) and <code>python3-foo</code> SRPM (produces <code>python3-foo</code> binary RPM).


== Future State Implied by the ''Change'' ==
== Future State Implied by the ''Change'' ==
After the switch, python3-* packages are going to be installed by default (from LiveCD, as dependencies of applications that require some Python packages).
After the switch, <code>python3-*</code> packages are going to be installed by default (from LiveCD, as dependencies of applications that require some Python packages).


== Proposal for Further Changes ==
== Proposal for Further Changes ==
* A brief discussion already happened at [2].
* A brief discussion already happened at [2].
* The SRPMs that produce binary RPMs for more Python runtimes should keep current names python-foo, the SRPMs that produce binary RPM for just one runtime should be named pythonX-foo.
* SRPMs that produce binary RPMs for more Python runtimes should keep current names (<code>python-foo</code>).
* All the binary RPMs must be named pythonX-foo, where X is the major version of the Python runtime that they use.
* SRPMs that produce binary RPM for just one runtime should be named <code>pythonX-foo</code>.
* For the time being (and maybe forever), every python2-foo package must have <code>Provides: python-foo</code>.
* Every binary RPM must be named <code>pythonX-foo</code>, where <code>X</code> is the major version of the Python runtime that it uses.
* This means that from user's perspective only one thing will change, and that is python-* packages will be renamed to python2-*, but will still keep the provide of python-*.
* For the time being (and maybe forever), every <code>python2-foo</code> package must have a virtual provide for its previous name. For most of the packages, this will be <code>Provides: python-foo</code>, but some of the packages haven't had the <code>python-</code> prefix historically, so these will just have <code>Provides: foo</code> (for example <code>Provides: PyYAML</code>).
* From packaging perspective, the above also means that all the <code>python2-foo</code> packages will need to have a proper <code>Obsoletes: ... </code> tag.
* From user's perspective this means:
** <code>python-*</code> packages will be renamed to <code>python2-*</code>, but will still keep the provide of <code>python-*</code> (so <code>yum install python-foo</code> will still work).
** Packages that didn't have the <code>python-</code> prefix, e.g. pyfoo, will newly be named <code>python2-pyfoo</code>, but they will still provide the old name (so again, <code>yum install pyfoo</code> will still work).


== Why? ==
== Why? ==
* This concept will easily scale to more Python runtimes (PyPy, Jython), as already proposed previously by Tom Spur. Please note, that scaling to other runtimes is '''not''' part of this proposal and should be discussed further in another proposal/ML thread.
* This concept will easily scale to more Python runtimes (PyPy, Jython), as already proposed previously by Tom Spur. Please note, that scaling to other runtimes is '''not''' part of this proposal and should be discussed further in another proposal/ML thread.
* Currently, upstream recommendation [3] is to point /usr/bin/python to Python 2, but as the PEP notes, it will be reviewed and it is anticipated that in time it will be updated to recommend pointing /usr/bin/python to /usr/bin/python3. When this time comes, we should also move the provides suggested above (<code>Provides: python-foo</code>) from python2-* packages to python3-* packages to keep things like <code>yum install /usr/bin/python python-foo</code> consistent.
* Currently, upstream recommendation [3] is to point <code>/usr/bin/python</code> to Python 2, but as the PEP notes, it will be reviewed and it is anticipated that in time it will be updated to recommend pointing <code>/usr/bin/python</code> to <code>/usr/bin/python3</code>. When this time comes, we should also move the provides suggested above (<code>Provides: python-foo</code>) from <code>python2-*</code> packages to <code>python3-*</code> packages to keep things like <code>yum install /usr/bin/python python-foo</code> consistent.
* Having python2-* vs. python3-* packages is a good way of explicitly distinguishing packages from the two stacks. This is connected with the reason above - we should recommend explicit usage of <code>/usr/bin/python{2,3}</code> and <code>yum install python{2,3}-foo</code>, and we should discourage use of implicit /usr/bin/python and python-* so that nothing breaks for users when these are changed in any way. (We already started to advertise usage of versioned /usr/bin/python{2,3} binary by deprecating %__python in current guidelines and recommending %__python{2,3}.)
* Having <code>python2-*</code> vs. <code>python3-*</code> packages is a good way of explicitly distinguishing packages from the two stacks. This is connected with the reason above - we should recommend explicit usage of <code>/usr/bin/python{2,3}</code> and <code>yum install python{2,3}-foo</code>, and we should discourage use of implicit <code>/usr/bin/python</code> and <code>python-*</code> so that nothing breaks for users when these are changed in any way. (We already started to advertise usage of versioned <code>/usr/bin/python{2,3}</code> binary by deprecating <code>%__python</code> in current guidelines and recommending <code>%__python{2,3}</code> instead.)


== Alternatives ==
== Alternatives ==
* Just keep what we have - doesn't seem to scale to other Python runtimes;  
* Just keep what we have - doesn't seem to scale to other Python runtimes; doesn't allow explicit naming (<code>python2-foo</code> vs. <code>python3-foo</code>) while keeping the possibility to move <code>Provides: python-foo</code> to <code>python3-foo</code> in the future.
* Only allow split SRPMs - too much maintenance work.
* Only allow split SRPMs - seems to be too much maintenance work.


= Example Specfile =
= Example Specfile =


Example of what a specfile would look like follows. Everything would work in the same way as it does now with building python3-* subpackages, but the same approach would be applied to python2-* subpackages, too.
Example of what a specfile would look like follows. Everything works in the same way as it does now with building <code>python3-*</code> subpackages; the same approach is applied to <code>python2-*</code> subpackages.


The specfile will produce two binary RPMs, python2-six and python3-six. Any of them can be disabled by setting with_python{2,3} to 0:
This specfile will produce two binary RPMs, <code>python2-six</code> and <code>python3-six</code>. Any of them can be disabled by setting with_python{2,3} to 0:
<pre>
<pre>
%global with_python2 1
%global with_python2 1
Line 57: Line 61:
BuildArch:      noarch
BuildArch:      noarch
%if 0%{?with_python2}
%if 0%{?with_python2}
BuildRequires:  python-devel
BuildRequires:  python2-devel
# For use by selftests:
# For use by selftests:
BuildRequires:  pytest
BuildRequires:  python2-pytest
BuildRequires:  tkinter
BuildRequires:  python2-tkinter
%endif
%endif


Line 81: Line 85:
Summary:        Python 2 and 3 compatibility utilities
Summary:        Python 2 and 3 compatibility utilities
Group:          Development/Languages
Group:          Development/Languages
Provides:      python-six = %{version}-%{release}
Obsoletes:      python-six < 1.4.1-1


%description -n python2-six
%description -n python2-six
Line 101: Line 107:




%prep, %build, %install and %check sections look pretty much the same, each has to explictly switch the directory to the respective py{2,3}dir.
<code>%prep, %build, %install and %check</code> sections look pretty much the same, each has to explictly switch the directory to the respective py{2,3}dir. I believe we could come up with some macros that would make this easier and more readable (TODO).
<pre>
<pre>
%prep
%prep
Line 140: Line 146:


%if 0%{?with_python2}
%if 0%{?with_python2}
pushd %{py3dir}
pushd %{py2dir}
%{__python2} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
%{__python2} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
popd
popd
Line 146: Line 152:
</pre>
</pre>


There is no default %files section, only %files sections for python{2,3}-six packages.
There is no default <code>%files</code> section, only <code>%files</code> sections for <code>python{2,3}-six</code> packages.
<pre>
<pre>
%if 0%{?with_python2}
%if 0%{?with_python2}

Latest revision as of 10:03, 5 December 2013

Prelude

This page summarizes a proposal of changes to Python packaging guidelines, that should occur with switch to Python 3 as a default, as proposed in [1] (will be referred to as Change).

Changes in Package Naming and SRPM <-> RPM Relations

Current State

python-* packages (built with Python 2) are installed by default (from LiveCD, as dependencies of applications that require some Python packages) and by yum install python-foo. Python 3 packages are named python3-* and are installed by yum install python3-foo.

From the packaging point of view, this is achieved by either

  • Having one SRPM python-foo that produces python-foo and python3-foo binary RPMs.

or

  • Having two SRPMs, python-foo (produces python-foo binary RPM) and python3-foo SRPM (produces python3-foo binary RPM).

Future State Implied by the Change

After the switch, python3-* packages are going to be installed by default (from LiveCD, as dependencies of applications that require some Python packages).

Proposal for Further Changes

  • A brief discussion already happened at [2].
  • SRPMs that produce binary RPMs for more Python runtimes should keep current names (python-foo).
  • SRPMs that produce binary RPM for just one runtime should be named pythonX-foo.
  • Every binary RPM must be named pythonX-foo, where X is the major version of the Python runtime that it uses.
  • For the time being (and maybe forever), every python2-foo package must have a virtual provide for its previous name. For most of the packages, this will be Provides: python-foo, but some of the packages haven't had the python- prefix historically, so these will just have Provides: foo (for example Provides: PyYAML).
  • From packaging perspective, the above also means that all the python2-foo packages will need to have a proper Obsoletes: ... tag.
  • From user's perspective this means:
    • python-* packages will be renamed to python2-*, but will still keep the provide of python-* (so yum install python-foo will still work).
    • Packages that didn't have the python- prefix, e.g. pyfoo, will newly be named python2-pyfoo, but they will still provide the old name (so again, yum install pyfoo will still work).

Why?

  • This concept will easily scale to more Python runtimes (PyPy, Jython), as already proposed previously by Tom Spur. Please note, that scaling to other runtimes is not part of this proposal and should be discussed further in another proposal/ML thread.
  • Currently, upstream recommendation [3] is to point /usr/bin/python to Python 2, but as the PEP notes, it will be reviewed and it is anticipated that in time it will be updated to recommend pointing /usr/bin/python to /usr/bin/python3. When this time comes, we should also move the provides suggested above (Provides: python-foo) from python2-* packages to python3-* packages to keep things like yum install /usr/bin/python python-foo consistent.
  • Having python2-* vs. python3-* packages is a good way of explicitly distinguishing packages from the two stacks. This is connected with the reason above - we should recommend explicit usage of /usr/bin/python{2,3} and yum install python{2,3}-foo, and we should discourage use of implicit /usr/bin/python and python-* so that nothing breaks for users when these are changed in any way. (We already started to advertise usage of versioned /usr/bin/python{2,3} binary by deprecating %__python in current guidelines and recommending %__python{2,3} instead.)

Alternatives

  • Just keep what we have - doesn't seem to scale to other Python runtimes; doesn't allow explicit naming (python2-foo vs. python3-foo) while keeping the possibility to move Provides: python-foo to python3-foo in the future.
  • Only allow split SRPMs - seems to be too much maintenance work.

Example Specfile

Example of what a specfile would look like follows. Everything works in the same way as it does now with building python3-* subpackages; the same approach is applied to python2-* subpackages.

This specfile will produce two binary RPMs, python2-six and python3-six. Any of them can be disabled by setting with_python{2,3} to 0:

%global with_python2 1
%global with_python3 1

# this macro is defined here only for testing purposes, it would
# be defined in macros.python2 provided by python2-devel
%global py2dir %{_builddir}/python2-%{name}-%{version}-%{release}

Name:           python-six
Version:        1.4.1
Release:        1%{?dist}
Summary:        Python 2 and 3 compatibility utilities

Group:          Development/Languages
License:        MIT
URL:            http://pypi.python.org/pypi/six/
Source0:        http://pypi.python.org/packages/source/s/six/six-%{version}.tar.gz

BuildArch:      noarch
%if 0%{?with_python2}
BuildRequires:  python2-devel
# For use by selftests:
BuildRequires:  python2-pytest
BuildRequires:  python2-tkinter
%endif

%if 0%{?with_python3}
BuildRequires:  python3-devel
# For use by selftests:
BuildRequires:  python3-pytest
BuildRequires:  python3-tkinter
%endif

Provide descriptions for both packages (and for the main package, since rpmbuild enforces that).

%description
python-six provides simple utilities for wrapping over differences between
Python 2 and Python 3.

%if 0%{?with_python2}
%package -n python2-six
Summary:        Python 2 and 3 compatibility utilities
Group:          Development/Languages
Provides:       python-six = %{version}-%{release}
Obsoletes:      python-six < 1.4.1-1

%description -n python2-six
python-six provides simple utilities for wrapping over differences between
Python 2 and Python 3.
%endif

%if 0%{?with_python3}
%package -n python3-six
Summary:        Python 2 and 3 compatibility utilities
Group:          Development/Languages

%description -n python3-six
python-six provides simple utilities for wrapping over differences between
Python 2 and Python 3.

This is the Python 3 build of the module.
%endif


%prep, %build, %install and %check sections look pretty much the same, each has to explictly switch the directory to the respective py{2,3}dir. I believe we could come up with some macros that would make this easier and more readable (TODO).

%prep
%setup -q -n six-%{version}
# possibly apply patches here

%if 0%{?with_python2}
rm -rf %{py2dir}
cp -a . %{py2dir}
%endif

%if 0%{?with_python3}
rm -rf %{py3dir}
cp -a . %{py3dir}
%endif


%build
%if 0%{?with_python2}
pushd %{py2dir}
%{__python2} setup.py build
popd
%endif

%if 0%{?with_python3}
pushd %{py3dir}
%{__python3} setup.py build
popd
%endif


%install
%if 0%{?with_python3}
pushd %{py3dir}
%{__python3} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
popd
%endif

%if 0%{?with_python2}
pushd %{py2dir}
%{__python2} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
popd
%endif

There is no default %files section, only %files sections for python{2,3}-six packages.

%if 0%{?with_python2}
%files -n python2-six
%doc LICENSE README documentation/index.rst
%{python2_sitelib}/*
%endif

%if 0%{?with_python3}
%files -n python3-six
%doc LICENSE README documentation/index.rst
%{python3_sitelib}/*
%endif

[1] https://fedoraproject.org/wiki/Changes/Python_3_as_Default

[2] https://lists.fedoraproject.org/pipermail/devel/2013-July/186822.html

[3] http://www.python.org/dev/peps/pep-0394/

[4] http://www.python.org/dev/peps/pep-0394/#future-changes-to-this-recommendation