From Fedora Project Wiki
No edit summary
Line 49: Line 49:


== Detailed Description ==
== Detailed Description ==
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
Historically, for over a decade, [https://pypi.org/project/setuptools/ setuptools] was omnipresent in Python packaging.
Historically, for over a decade, [https://pypi.org/project/setuptools/ setuptools] was omnipresent in Python packaging.
For that reason, the python3-devel package required python3-setuptools.
For that reason, the python3-devel package required python3-setuptools.
This is however changing, with [https://www.python.org/dev/peps/pep-0517/ PEP517] and [https://www.python.org/dev/peps/pep-0518/ PEP518] based upstream packaging, setuptools changed to a very popular yet optional way of building packages, many upstreams switched to poetry or flit instead.
This is however changing, with [https://www.python.org/dev/peps/pep-0517/ PEP517]- and [https://www.python.org/dev/peps/pep-0518/ PEP518]-based upstream packaging, setuptools changed to a very popular yet only optional way of building packages, many upstreams switched to poetry or flit instead.
Since we received multiple user reports about an unnecessary dependency, we wanted to [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ARFZB56ULRQNT3FEAC4YXXZDIUHG4M5P/ get rid of the dependency], but we have encountered a problem, which is best summarized in the Python spec file:
Since we received multiple user reports about an unnecessary dependency, we wanted to [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ARFZB56ULRQNT3FEAC4YXXZDIUHG4M5P/ get rid of the dependency], but it was not possible until now.


We'll do the two following changes (neither one depends on the other):
### === python3-devel will stop Requiring python3-setuptools ===
When we wanted to remove the dependency before,
we have encountered a problem, which is best summarized in the Python spec file:
````
  # This is not "API" (packages that need setuptools should still BuildRequire it)
  # This is not "API" (packages that need setuptools should still BuildRequire it)
  # However some packages apparently can build both with and without setuptools
  # However some packages apparently can build both with and without setuptools
Line 63: Line 70:
  # See https://fedoraproject.org/wiki/Packaging:Directory_Replacement
  # See https://fedoraproject.org/wiki/Packaging:Directory_Replacement
  Requires: (python3-setuptools if rpm-build)
  Requires: (python3-setuptools if rpm-build)
 
````
To avoid this problem, python3-devel currently requires `(python3-setuptools if rpmbuild)` for backwards compatibility.
To avoid this problem, python3-devel currently requires `(python3-setuptools if rpmbuild)` for backwards compatibility.
Other than that, python3-setuptools was always brought into the buildroot transitively with python3-rpm-generators.
Other than that, python3-setuptools was always brought into the buildroot transitively with python3-rpm-generators.
Line 70: Line 77:
Packages that don't BuildRequire python3-setuptools but use setuptools during the build will fall into one of the following three categories:
Packages that don't BuildRequire python3-setuptools but use setuptools during the build will fall into one of the following three categories:


# They will fail to build from source with errors like: `ModuleNotFoundError: No module named 'setuptools'`.
* They will fail to build from source with errors like: `ModuleNotFoundError: No module named 'setuptools'`.
# They will successfully build from source because they will fallback to the standard library distutils module - their `.egg-info` directory will become a text file instead, causing errors when RPM tries to update the package (and they will possibly miss some runtime dependencies because the metadata generated by distutils is not complete).
* They will successfully build from source because they will fallback to the standard library distutils module - their `.egg-info` directory will become a text file instead, causing errors when RPM tries to update the package (and they will possibly miss some runtime dependencies because the metadata generated by distutils is not complete).
# They will continue to build unchanged because one or more packages they BuildRequire pulls in python3-setuptools transitively - at any point in the future, this can change and either (1) or (2) will happen.
* They will continue to build unchanged because one or more packages they BuildRequire pulls in python3-setuptools transitively - at any point in the future, this can change and either (1) or (2) will happen.


We tested this change in the past when we [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GCPGM34ZGEOVUHSBGZTRYR5XKHTIJ3T7/ asked] other maintainers to explicitly BuildRequire python-3setuptools. Some of them responded many did not. Our analysis was based on grepping sources for any mention of setuptools, such as `import setuptools` or `from setuptools import `. Nowadays, the same grep-based query showed us roughly the same number (about 335 + 72 that failed to be analyzed) of packages relying on python3-setuptools without BuildRequiring it. To see the direct impact, we have also briefly removed the Requires from python3-devel package used in testing [https://copr.fedorainfracloud.org/coprs/g/python/python3.10/packages/ python3.10] in COPR. It revealed only 140 packages that fail to build from the source because of the change, however many packages bring in python3-setuptools transitively. Some of the packages might build successfully even without python3-setuptools, because they fallback to distutils if dependency on setuptools is not satisfied.
We tested this change in the past when we [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/GCPGM34ZGEOVUHSBGZTRYR5XKHTIJ3T7/ asked] other maintainers to explicitly BuildRequire python3-setuptools. Some of them responded, but many did not. Our analysis was based on grepping sources for any mention of setuptools, such as `import setuptools` or `from setuptools import `. Nowadays, the same grep-based query showed us roughly the same number (~350) of packages relying on python3-setuptools without BuildRequiring it. To see the direct impact, we have also briefly removed the Requires from python3-devel package used in testing [https://copr.fedorainfracloud.org/coprs/g/python/python3.10/packages/ python3.10] in COPR. It revealed only 140 packages that fail to build from the source because of the change, however many packages bring in python3-setuptools transitively. Some of the packages might build successfully even without python3-setuptools, because they fallback to distutils (from the Python standard library) if dependency on setuptools is not satisfied.


To avoid such problems we will:
To avoid such problems we will:


# Mass update packages we analyzed as impacted to add the missing BuildRequires on python3-setuptools.
* Mass update packages we analyzed as impacted to add the missing BuildRequires on python3-setuptools.
# Do the change while doing the [[Changes/Python3.10|update from Python 3.9 to Python 3.10]] so the path of the `.egg-info` changes from `/usr/lib(64)/python3.9/site-packages/....egg-info` to `/usr/lib(64)/python3.10/site-packages/....egg-info`, hence no longer causing troubles on upgrades if some packages actually do change it from directory to file (and changing it the other way around in the future is safe).
* Do the change while doing the [[Changes/Python3.10|update from Python 3.9 to Python 3.10]] so the path of the `.egg-info` changes from `/usr/lib(64)/python3.9/site-packages/...egg-info` to `/usr/lib(64)/python3.10/site-packages/...egg-info`, hence no longer causing troubles on upgrades if some packages actually do change it from directory to file (and changing it the other way around in the future is safe).
 
==== How do we mass update the spec files ====
 
We plan to write a script to detect what notation for BuildRequires definition each spec file uses.
 
Example of the most common occurrences:
[[User:Thrnciar|Thrnciar]] ([[User talk:Thrnciar|talk]]) 15:01, 19 March 2021 (UTC)
BuildRequires: python%{python3_pkgversion}-setuptools
BuildRequires: %{py3_dist setuptools}
BuildRequires: python3-setuptools
BuildRequires: python3dist(setuptools)
[[User:Thrnciar|Thrnciar]] ([[User talk:Thrnciar|talk]]) 15:01, 19 March 2021 (UTC)
 
We will try to keep the added BuildRequires as much consistent as possible with the existing Python-related BuildRequires (including e.g. whitespace). To show how the proposed modifications will look like we plan to create a git repository with all the spec files and commit the change there. The diff will be shared on the devel and python-devel mailing lists, so maintainers can easily see the upcoming change and can provide feedback to us.
 
=== Packages with console_scripts/gui_scripts entry points will no longer automatically Require python3.Xdist(setuptools) ===
 
Packages that use the `console_scripts` or `gui_scripts` entry points used to generate scripts for them that imported `pkg_resources` (from setuptools) on runtime and thus all such packages needed to Require setuptools on runtime. This is an example of such generated script from Fedora 32:
 
[[User:Thrnciar|Thrnciar]] ([[User talk:Thrnciar|talk]]) 15:01, 19 March 2021 (UTC)
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'fedpkg==1.40','console_scripts','fedpkg'
__requires__ = 'fedpkg==1.40'
import re
import sys
from pkg_resources import load_entry_point
 
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('fedpkg==1.40', 'console_scripts', 'fedpkg')()
    )
[[User:Thrnciar|Thrnciar]] ([[User talk:Thrnciar|talk]]) 15:01, 19 March 2021 (UTC)
 
The Python dist RPM dependency generator (`/usr/lib/rpm/fileattrs/pythondist.attr`) hence generated a Requires on `python3.Xdist(setuptools)` for all packages with scripts like the one above.
 
Starting from Python 3.8 and setuptools 47, this is no longer required, becasue the scripts will use `importlib.metadata` from the standard library instead of `pkg_resources` from setuptools. The scripts now looks like this:
 
[[User:Thrnciar|Thrnciar]] ([[User talk:Thrnciar|talk]]) 15:01, 19 March 2021 (UTC)
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'fedpkg==1.40','console_scripts','fedpkg'
import re
import sys
 
# for compatibility with easy_install; see #2198
__requires__ = 'fedpkg==1.40'
 
try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point
...
[[User:Thrnciar|Thrnciar]] ([[User talk:Thrnciar|talk]]) 15:01, 19 March 2021 (UTC)
 
Since the scripts are generated during build, the packages no longer need to Require setuptools on runtime.
The Python dist RPM dependency generator will no longer generate the Requires on `python3.Xdist(setuptools)`.
 
The change is prepared in a pull request (https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/38).
 
Packages that actually import from the `setuptools` or `pkg_resources` module on runtime should explicitly require setuptools, but some of the upstreams may not do that. We don't expect many packages negatively affected by this, but since this is not strictly backwards compatible, we are only planning to do it in Fedora 35+.


=== Important dates and plan ===
=== Important dates and plan ===


See [[Changes/Python3.10#Important_dates_and_plan|Python3.10 change]] - we plan to mass update the spec files the week before *2021-05-25: Python 3.10.0 beta 2*, if you wish to opt-out from the mass update, please let us know prior to 18th of May 2021.
We plan to mass update the spec files in the middle of April 2021, if you wish to opt-out from the mass update, please let us know prior to 15th of April 2021.


Below is the list of packages we plan to update with mass spec rebuild. If you wish to opt-out of this update, please edit this page and move your package to the list of packages to be ignored.
Below is the list of packages we plan to add BuildRequires for python3-setuptools to. If you wish to opt-out of this update, please edit this page and move your package to the list of packages to be ignored (listed below this one).


'''Packages to be updated:'''
'''Packages to be updated:'''
* GeographicLib        rmattes smani
* 0ad                  ignatenkobrain pcpa pwalter
* R2spec              pingou  
* OpenMolcas          jussilehtola
* barman              slaanesh tsao  
* PyGreSQL            hhorak jmlich odubaj panovotn pkubat praiskup
* btrfs-progs          ignatenkobrain josef ngompa sandeen  
* R2spec              pingou
* bumpversion          duriantang jdornak  
* ViTables            tnorth zbyszek
* cranc                lenkaseg  
* ansible-review      dcallagh ttrinks
* electrum            tredaell
* barman              slaanesh tsao
* gajim                michich
* blender              design-sw hobbes1069 ignatenkobrain kwizart luya roma s4504kr slaanesh
* gns3-gui            kwizart
* btrfs-progs          ignatenkobrain josef ngompa sandeen
* gns3-net-converter  kwizart
* bumpversion          duriantang jdornak
* kismon              fab
* calypso              rathann
* langtable            mfabian
* cinch                greghellings
* lecm                spredzy
* commissaire-client  mbarnes smilner
* legofy              lkf williamjmorenor
* cranc                lenkaseg
* mapserver           devrim jujens oliver pali smani
* crudini              apevec jruzicka pbrady
* mysql-connector-python hhorak hubbitus hvad mschorm
* cxxtest              mgieseki
* percol              hubbitus
* datanommer          ralph
* porcupine            kushal
* dlib                bizdelnick luya
* protonvpn-cli        calexandru2018 jflory7
* ec2-hibinit-agent    davdunc
* pyhunspell          mfabian
* electrum            tredaell
* pyjokes              pbrobinson pwhalen
* fbthrift            dcavalca filbranden salimma
* pyscard              orion sjenning
* fedora-messaging    abompard
* python-Pyped        uggla
* firefox              alexl caolanm erack gecko-maint jgrulich kalev kengert mbarnes rhughes rstrode sharkcz stransky tpopela ueno xhorak
* python-ROPGadget    mikep
* folly                dcavalca filbranden salimma
* python-acoustid      terjeros  
* fontforge           frixxon kevin pnemade
* python-aiodns        fantom
* gajim                michich suraia
* python-arviz        sergiopr
* gau2grid            jussilehtola
* python-b2sdk        jonny
* gaupol              music
* python-btchip        jonny xenithorb
* gfal2-python        adev andreamanzi
* python-cerealizer    spot
* ginga                lupinix
* python-cmigemo      hubbitus  
* git-filter-repo      asn
* python-construct    moezroy terjeros
* gns3-gui            kwizart
* python-css-parser    zbyszek
* gns3-net-converter  kwizart
* python-debrepo      ktdreyer
* gns3-server          kwizart nucleo
* python-dialog        mjakubicek noodles raphgro sundaram zbyszek
* gpsd                fab mlichvar ttorling
* python-django-contact-form mrunge
* h5py                stevetraylen terjeros
* python-django-registration orphan
* khard                mathstuf sdyroff
* python-django-reversion mrunge
* kismon              fab
* python-django-tagging jdornak mrunge piotrp
* koji                ausil kevin mikem puiterwijk
* python-dtfabric      fab
* legofy              lkf williamjmorenor
* python-flake8-docstrings cottsay
* libcaca              hubbitus slaanesh thias
* python-flask-sphinx-themes devrim orphan
* libnuml              sagitter
* python-formats      uggla
* libreoffice          caolanm dtardon erack sbergmann
* python-fypp          rathann
* lldb                airlied daveisfera jankratochvil sergesanspaille siddharths tstellar
* python-google-i18n-address pwouters
* llvm                dmalcolm ignatenkobrain jakub jistone kyle scottt sergesanspaille siddharths tstellar
* python-googletrans  lyessaadi
* llvm10              sergesanspaille tstellar
* python-heapdict      qulogic
* llvm11              sergesanspaille tstellar
* python-hgdistver    fab
* llvm7.0              jistone petersen sergesanspaille tstellar
* python-html5-parser  kevin
* llvm9.0              jistone sergesanspaille tstellar
* python-humblewx      rickardlindberg
* mercurial            kiilerix nbecker pstodulk
* python-i3ipc        msimacek
* mod_wsgi            jdornak jkaluza jorton lmacken mrunge
* python-kerberos      rcritten simo
* moose                zbyszek
* python-lark-parser  totol
* mypaint              avsej
* python-logfury      jonny
* mysql-connector-python hhorak hubbitus hvad mschorm
* python-mailmerge    bex
* nototools            mfabian pwu
* python-meld3        kevin stevetraylen tsao
* officeparser        rebus
* python-mnemonic      jonny
* offlineimap          cicku dodji sergesanspaille teuf
* python-nbxmpp        michich
* percol              hubbitus
* python-oauth2        ignatenkobrain pjp spot sundaram
* poetry              churchyard thrnciar
* python-poyo          chedi wakko666
* poezio              fantom louizatakk
* python-precis_i18n  michich
* porcupine            kushal
* python-proteus      sharkcz
* protontricks        atim
* python-publicsuffix2 rathann
* protonvpn-cli        calexandru2018 jflory7
* python-pulsectl      pfrields
* prunerepo            frostyx praiskup
* python-pycha        potty sharkcz
* pyaudio              chkr
* python-pylons-sphinx-themes abompard
* pyhunspell          mfabian
* python-pymc3        sergiopr  
* pyodbc              fjanus hhorak
* python-pyotp        icon
* pyscard              orion sjenning
* python-pyroute2      jirka pavlix psavelye
* pyserial            stingray
* python-pytest-astropy-header sergiopr
* python-CommonMark    jujens
* python-pytest-watch  jujens
* python-GridDataFormats rathann
* python-relatorio     sharkcz
* python-OBD          rathann
* python-rmtest        lberk mgoodwin nathans
* python-Pyped        uggla
* python-shamir-mnemonic jonny  
* python-Rtree        volter
* python-smbc          twaugh zdohnal
* python-acoustid      terjeros
* python-spdx          jbertozzi
* python-agate        jujens
* python-spdx-lookup  jbertozzi
* python-aiodns        fantom
* python-tempdir      rathann
* python-ansicolors    fab
* python-tortilla      uggla
* python-apprise      lead2gold
* python-trezor        jonny
* python-apsw          cicku dfateyev maci
* python-unidecode    pjp sundaram
* python-arviz        sergiopr
* python-unidiff       dcallagh
* python-astroplan    sergiopr
* python-upt-cpan      jbertozzi
* python-astropy-healpix lupinix
* python-upt-fedora    jbertozzi
* python-astroquery    lupinix
* python-upt-pypi      jbertozzi
* python-astroscrappy  lupinix
* python-upt-rubygems jbertozzi
* python-asttokens    zbyszek
* python-xlwt          leamas moezroy rathann
* python-audioread     terjeros
* qr-code-generator   xvitaly
* python-autobahn      fab jujens
* rpmspectool          nphilipp
* python-autopep8      mrunge ndipanov
* sepolicy_analysis    vmojzis
* python-b2sdk        jonny
* stomppy              stevetraylen
* python-bloom        cottsay rmattes
* trytond-account      sharkcz
* python-blosc        tnorth zbyszek
* trytond-account-be  sharkcz
* python-box          dmsimard fab
* trytond-account-de-skr03 sharkcz  
* python-btchip        jonny xenithorb
* trytond-account-invoice sharkcz  
* python-cached_property adamwill immanetize
* trytond-account-invoice-history sharkcz  
* python-carbon        jsteffan piotrp
* trytond-account-invoice-line-standalone sharkcz  
* python-ccdproc      lupinix
* trytond-account-product sharkcz  
* python-certbot-apache nb
* trytond-account-statement sharkcz  
* python-chai          kevin pingou ralph
* trytond-account-stock-anglo-saxon sharkcz  
* python-cmigemo       hubbitus
* trytond-account-stock-continental sharkcz  
* python-colorspacious rathann
* trytond-analytic-account sharkcz  
* python-construct    moezroy terjeros
* trytond-analytic-invoice sharkcz  
* python-contextlib2  abompard pingou ralph tjikkun
* trytond-analytic-purchase sharkcz  
* python-cookiecutter chedi wakko666
* trytond-analytic-sale sharkcz  
* python-crochet      abompard
* trytond-company      sharkcz  
* python-css-parser   zbyszek
* trytond-company-work-time sharkcz  
* python-dbfread      jujens
* trytond-country      sharkcz  
* python-debrepo      ktdreyer
* trytond-currency    sharkcz  
* python-decopatch    zbyszek
* trytond-dashboard    sharkcz  
* python-descartes    qulogic
* trytond-google-maps  sharkcz  
* python-dialog        mjakubicek noodles raphgro sundaram zbyszek
* trytond-ldap-authentication sharkcz  
* python-dijitso      limb
* trytond-party        sharkcz  
* python-django-contact-form mrunge
* trytond-party-siret  sharkcz  
* python-django-health-check dmsimard
* trytond-product     sharkcz  
* python-django-registration orphan
* trytond-product-cost-fifo sharkcz  
* python-django-reversion mrunge
* trytond-product-cost-history sharkcz  
* python-django-tagging jdornak mrunge piotrp
* trytond-product-price-list sharkcz
* python-django-tastypie bkabrda cquad mrunge stevetraylen
* trytond-project      sharkcz
* python-docx          kushal124
* trytond-project-plan sharkcz
* python-dtfabric      fab
* trytond-project-revenue sharkcz
* python-dukpy        zbyszek
* trytond-purchase    sharkcz  
* python-editorconfig  barracks510
* trytond-purchase-invoice-line-standalone sharkcz  
* python-ephem        fab
* trytond-sale        sharkcz  
* python-et_xmlfile    jujens
* trytond-sale-opportunity sharkcz  
* python-etcd          mbarnes smilner
* trytond-sale-price-list sharkcz  
* python-fasteners    mrunge
* trytond-stock        sharkcz  
* python-ffc          zbyszek
* trytond-stock-forecast sharkcz  
* python-fields        cottsay
* trytond-stock-inventory-location sharkcz  
* python-fisx          zbyszek
* trytond-stock-location-sequence sharkcz  
* python-fitsio        lupinix
* trytond-stock-product-location sharkcz  
* python-flake8-docstrings cottsay
* trytond-stock-supply sharkcz  
* python-flask-gravatar devrim orphan
* trytond-stock-supply-day sharkcz  
* python-flask-htmlmin devrim iztokf
* trytond-timesheet    sharkcz  
* python-flask-paranoid devrim orphan
* txt2tags            junghans
* python-flask-security devrim orphan
* ubertooth            avsej  
* python-flask-sphinx-themes devrim orphan
* unicorn              fab mikep
* python-flask-wtf-decorators frostyx
* upt                  jbertozzi  
* python-formats      uggla
* winpdb              spot  
* python-fypp          rathann
* yawn                jsafrane miminar vcrhonek  
* python-gevent-eventemitter atim
* python-google-i18n-address pwouters
* python-googletrans  lyessaadi
* python-graphql-relay fab
* python-heapdict      qulogic
* python-html5-parser  kevin
* python-htmlmin      jujens
* python-humblewx      rickardlindberg
* python-hupper        kevin
* python-i3ipc        alebastr
* python-inotify      jfilak stevetraylen terjeros
* python-jep          raphgro
* python-jinja2-cli    jujens
* python-jinja2-time  chedi wakko666
* python-jnius        raphgro
* python-joblib        besser82 ignatenkobrain sergiopr
* python-jsonmodels    oanson
* python-jsonrpclib    ihrachyshka jonny
* python-kerberos      rcritten simo
* python-kitchen      kevin pingou ralph
* python-landslide    echevemaster salimma
* python-lark-parser  totol
* python-leather      jujens
* python-libsass      nonamedotc
* python-libusb1      jonny
* python-lmdb          jruzicka pspacek tkrizek
* python-logfury      jonny
* python-makefun      zbyszek
* python-managesieve  stevetraylen
* python-matrix-nio    ankursinha
* python-meld3        kevin stevetraylen tsao
* python-minibelt      uggla
* python-mmtf          rathann
* python-mnemonic      jonny
* python-music21      zbyszek
* python-mwclient      adamwill rdieter tuxbrewr
* python-myhdl        filiperosset
* python-nbclient      nonamedotc
* python-nbxmpp        michich suraia
* python-networkmanager jdulaney
* python-notario      ktdreyer
* python-oauth2        ignatenkobrain pjp spot sundaram
* python-openoffice    sharkcz
* python-ouimeaux      kni
* python-pandas-datareader sergiopr
* python-paste-script  andreamanzi
* python-patsy        sergiopr
* python-pbkdf2        jonny
* python-pecan-notario ktdreyer
* python-pelican      firemanxbr mrunge
* python-pexpect      amcnabb fabiand ignatenkobrain radez swt2c tomspur
* python-plaster-pastedeploy abompard
* python-plumbum      greghellings lorenzodalrio
* python-poetry-core  churchyard thrnciar
* python-polib        cicku dchen diegobz dshea ivazquez moezroy suanand
* python-precis_i18n  michich
* python-proteus      sharkcz
* python-publicsuffix2 rathann
* python-pulsectl      pfrields
* python-pvc          raphgro
* python-pycares      fantom
* python-pycha        potty sharkcz
* python-pygeoip      kevin ralph
* python-pylons-sphinx-themes abompard
* python-pymc3        sergiopr
* python-pynn          ankursinha
* python-pyotp        icon
* python-pypng        kevin ralph
* python-pyramid_sawing abompard
* python-pysb          zbyszek
* python-pysignals    kni
* python-pyswip        pampelmuse
* python-pyte          terjeros
* python-pytest-astropy-header sergiopr
* python-pytest-cases  zbyszek
* python-pytest-harvest zbyszek
* python-pytest-mock  fab jujens
* python-pytest-repeat cottsay
* python-pytest-steps  zbyszek
* python-pytest-watch  jujens
* python-pyvo          lupinix
* python-readthedocs-sphinx-ext jjames
* python-recommonmark  jujens
* python-relatorio    sharkcz
* python-restructuredtext-lint jujens
* python-retrying      apevec
* python-rmtest        lberk mgoodwin nathans
* python-rosdep        cottsay rmattes thofmann
* python-sanction      kevin ralph
* python-scikit-learn  besser82 ignatenkobrain lupinix sergiopr
* python-scrapy        echevemaster
* python-setuptools-lint jdulaney
* python-shamir-mnemonic jonny
* python-simplemediawiki lmacken potty ralph
* python-slixmpp      fantom louizatakk
* python-smbc          twaugh zdohnal
* python-snappy        jujens
* python-social-auth-core cqi
* python-soupsieve    zbyszek
* python-spdx          jbertozzi
* python-spdx-lookup  jbertozzi
* python-sphinxcontrib-bibtex jjames
* python-statsd        pabelanger tdecacqu
* python-steam        atim
* python-tables        tnorth zbyszek
* python-tempdir      rathann
* python-timeout-decorator jcapitao
* python-tinydb        suanand
* python-tortilla      uggla
* python-tqdm          ignatenkobrain sgallagh
* python-tree-format  chedi wakko666
* python-trezor        jonny
* python-twilio        mich181189
* python-txaio        fab jujens
* python-unidecode    pjp sundaram
* python-unidiff      dcallagh
* python-upt-cpan      jbertozzi
* python-upt-fedora    jbertozzi
* python-upt-pypi      jbertozzi
* python-upt-rubygems  jbertozzi
* python-urwidtrees    ttomecek
* python-vdf          atim
* python-wand          barracks510
* python-watchdog      jsteffan jujens pingou
* python-webencodings  abompard
* python-webpy        mrunge
* python-wsaccel      jujens
* python-xlwt          leamas moezroy rathann
* python-xvfbwrapper  mrunge totol
* python-yarl          fab ignatenkobrain
* python-zstandard    rathann
* python3-postgresql  hhorak
* python3-pytest-asyncio jujens
* python3-saml        dcallagh tchaikov
* qemu                berrange bonzini crobinso dwmw2 ehabkost jforbes lkundrak quintela rjones
* rdkit                giallu
* rpmspectool          nphilipp
* salt                blarson dmurphy18 krionbsd
* sepolicy_analysis    vmojzis
* solaar              brouhaha rathann richardfearn tibbs
* spec2scl            jstanek
* starcal              hedayat
* stomppy              stevetraylen
* sugar-speak          callkalpa chimosky pbrobinson tuxbrewr
* swid-tools          adelton
* swift-lang          tachoknight
* terminator          dmaphy mattrose ohaessler
* texlive-base        spot
* toot                alciregi
* translate-toolkit    cicku dwayne petersen suanand
* trellis              lkundrak somlo
* tryton              sharkcz
* trytond              sharkcz
* trytond-account      sharkcz
* trytond-account-be  sharkcz
* trytond-account-de-skr03 sharkcz
* trytond-account-invoice sharkcz
* trytond-account-invoice-history sharkcz
* trytond-account-invoice-line-standalone sharkcz
* trytond-account-product sharkcz
* trytond-account-statement sharkcz
* trytond-account-stock-anglo-saxon sharkcz
* trytond-account-stock-continental sharkcz
* trytond-analytic-account sharkcz
* trytond-analytic-invoice sharkcz
* trytond-analytic-purchase sharkcz
* trytond-analytic-sale sharkcz
* trytond-company      sharkcz
* trytond-company-work-time sharkcz
* trytond-country      sharkcz
* trytond-currency    sharkcz
* trytond-dashboard    sharkcz
* trytond-google-maps  sharkcz
* trytond-ldap-authentication sharkcz
* trytond-party        sharkcz
* trytond-party-siret  sharkcz
* trytond-product      sharkcz
* trytond-product-cost-fifo sharkcz
* trytond-product-cost-history sharkcz
* trytond-product-price-list sharkcz
* trytond-project     sharkcz
* trytond-project-plan sharkcz
* trytond-project-revenue sharkcz
* trytond-purchase    sharkcz
* trytond-purchase-invoice-line-standalone sharkcz
* trytond-sale        sharkcz
* trytond-sale-opportunity sharkcz
* trytond-sale-price-list sharkcz
* trytond-stock        sharkcz
* trytond-stock-forecast sharkcz
* trytond-stock-inventory-location sharkcz
* trytond-stock-location-sequence sharkcz
* trytond-stock-product-location sharkcz
* trytond-stock-supply sharkcz
* trytond-stock-supply-day sharkcz
* trytond-timesheet    sharkcz
* ubertooth            avsej
* uhd                  jskarvad
* upt                  jbertozzi
* uwsgi                kad
* watchman            dcavalca filbranden salimma
* wine-mono            mooninite
* winpdb              spot
* xrootd              ellert simonm
* xtensor-python      sergesanspaille
* yawn                jsafrane miminar vcrhonek


'''Packages to be ignored:'''
'''Packages to be ignored:'''

Revision as of 15:01, 19 March 2021


reduce dependencies on python3-setuptools

Summary

We'll do two things to reduce the number of Python packages unnecessarily Requiring python3-setuptools:

1. python3-devel will no longer Require python3-setuptools 2. packages with console_scripts/gui_scripts entry points will no longer automatically Require python3-setuptools


Owner

Current status

  • Targeted release: Fedora Linux 35
  • Last updated: 2021-03-19
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

Historically, for over a decade, setuptools was omnipresent in Python packaging. For that reason, the python3-devel package required python3-setuptools. This is however changing, with PEP517- and PEP518-based upstream packaging, setuptools changed to a very popular yet only optional way of building packages, many upstreams switched to poetry or flit instead. Since we received multiple user reports about an unnecessary dependency, we wanted to get rid of the dependency, but it was not possible until now.

We'll do the two following changes (neither one depends on the other):

      1. === python3-devel will stop Requiring python3-setuptools ===

When we wanted to remove the dependency before, we have encountered a problem, which is best summarized in the Python spec file:

`

# This is not "API" (packages that need setuptools should still BuildRequire it)
# However some packages apparently can build both with and without setuptools
# producing egg-info as file or directory (depending on setuptools presence).
# Directory-to-file updates are problematic in RPM, so we ensure setuptools is
# installed when -devel is required.
# See https://bugzilla.redhat.com/show_bug.cgi?id=1623914
# See https://fedoraproject.org/wiki/Packaging:Directory_Replacement
Requires: (python3-setuptools if rpm-build)

` To avoid this problem, python3-devel currently requires (python3-setuptools if rpmbuild) for backwards compatibility. Other than that, python3-setuptools was always brought into the buildroot transitively with python3-rpm-generators. Since the generators no longer use setuptools but packaging we can finally remove the dependency.

Packages that don't BuildRequire python3-setuptools but use setuptools during the build will fall into one of the following three categories:

  • They will fail to build from source with errors like: ModuleNotFoundError: No module named 'setuptools'.
  • They will successfully build from source because they will fallback to the standard library distutils module - their .egg-info directory will become a text file instead, causing errors when RPM tries to update the package (and they will possibly miss some runtime dependencies because the metadata generated by distutils is not complete).
  • They will continue to build unchanged because one or more packages they BuildRequire pulls in python3-setuptools transitively - at any point in the future, this can change and either (1) or (2) will happen.

We tested this change in the past when we asked other maintainers to explicitly BuildRequire python3-setuptools. Some of them responded, but many did not. Our analysis was based on grepping sources for any mention of setuptools, such as import setuptools or from setuptools import . Nowadays, the same grep-based query showed us roughly the same number (~350) of packages relying on python3-setuptools without BuildRequiring it. To see the direct impact, we have also briefly removed the Requires from python3-devel package used in testing python3.10 in COPR. It revealed only 140 packages that fail to build from the source because of the change, however many packages bring in python3-setuptools transitively. Some of the packages might build successfully even without python3-setuptools, because they fallback to distutils (from the Python standard library) if dependency on setuptools is not satisfied.

To avoid such problems we will:

  • Mass update packages we analyzed as impacted to add the missing BuildRequires on python3-setuptools.
  • Do the change while doing the update from Python 3.9 to Python 3.10 so the path of the .egg-info changes from /usr/lib(64)/python3.9/site-packages/...egg-info to /usr/lib(64)/python3.10/site-packages/...egg-info, hence no longer causing troubles on upgrades if some packages actually do change it from directory to file (and changing it the other way around in the future is safe).

How do we mass update the spec files

We plan to write a script to detect what notation for BuildRequires definition each spec file uses.

Example of the most common occurrences: Thrnciar (talk) 15:01, 19 March 2021 (UTC) BuildRequires: python%{python3_pkgversion}-setuptools BuildRequires: %{py3_dist setuptools} BuildRequires: python3-setuptools BuildRequires: python3dist(setuptools) Thrnciar (talk) 15:01, 19 March 2021 (UTC)

We will try to keep the added BuildRequires as much consistent as possible with the existing Python-related BuildRequires (including e.g. whitespace). To show how the proposed modifications will look like we plan to create a git repository with all the spec files and commit the change there. The diff will be shared on the devel and python-devel mailing lists, so maintainers can easily see the upcoming change and can provide feedback to us.

Packages with console_scripts/gui_scripts entry points will no longer automatically Require python3.Xdist(setuptools)

Packages that use the console_scripts or gui_scripts entry points used to generate scripts for them that imported pkg_resources (from setuptools) on runtime and thus all such packages needed to Require setuptools on runtime. This is an example of such generated script from Fedora 32:

Thrnciar (talk) 15:01, 19 March 2021 (UTC)

  1. !/usr/bin/python3
  2. EASY-INSTALL-ENTRY-SCRIPT: 'fedpkg==1.40','console_scripts','fedpkg'

__requires__ = 'fedpkg==1.40' import re import sys from pkg_resources import load_entry_point

if __name__ == '__main__':

   sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', , sys.argv[0])
   sys.exit(
       load_entry_point('fedpkg==1.40', 'console_scripts', 'fedpkg')()
   )

Thrnciar (talk) 15:01, 19 March 2021 (UTC)

The Python dist RPM dependency generator (/usr/lib/rpm/fileattrs/pythondist.attr) hence generated a Requires on python3.Xdist(setuptools) for all packages with scripts like the one above.

Starting from Python 3.8 and setuptools 47, this is no longer required, becasue the scripts will use importlib.metadata from the standard library instead of pkg_resources from setuptools. The scripts now looks like this:

Thrnciar (talk) 15:01, 19 March 2021 (UTC)

  1. !/usr/bin/python3
  2. EASY-INSTALL-ENTRY-SCRIPT: 'fedpkg==1.40','console_scripts','fedpkg'

import re import sys

  1. for compatibility with easy_install; see #2198

__requires__ = 'fedpkg==1.40'

try:

   from importlib.metadata import distribution

except ImportError:

   try:
       from importlib_metadata import distribution
   except ImportError:
       from pkg_resources import load_entry_point

... Thrnciar (talk) 15:01, 19 March 2021 (UTC)

Since the scripts are generated during build, the packages no longer need to Require setuptools on runtime. The Python dist RPM dependency generator will no longer generate the Requires on python3.Xdist(setuptools).

The change is prepared in a pull request (https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/38).

Packages that actually import from the setuptools or pkg_resources module on runtime should explicitly require setuptools, but some of the upstreams may not do that. We don't expect many packages negatively affected by this, but since this is not strictly backwards compatible, we are only planning to do it in Fedora 35+.

Important dates and plan

We plan to mass update the spec files in the middle of April 2021, if you wish to opt-out from the mass update, please let us know prior to 15th of April 2021.

Below is the list of packages we plan to add BuildRequires for python3-setuptools to. If you wish to opt-out of this update, please edit this page and move your package to the list of packages to be ignored (listed below this one).

Packages to be updated:

  • 0ad ignatenkobrain pcpa pwalter
  • OpenMolcas jussilehtola
  • PyGreSQL hhorak jmlich odubaj panovotn pkubat praiskup
  • R2spec pingou
  • ViTables tnorth zbyszek
  • ansible-review dcallagh ttrinks
  • barman slaanesh tsao
  • blender design-sw hobbes1069 ignatenkobrain kwizart luya roma s4504kr slaanesh
  • btrfs-progs ignatenkobrain josef ngompa sandeen
  • bumpversion duriantang jdornak
  • calypso rathann
  • cinch greghellings
  • commissaire-client mbarnes smilner
  • cranc lenkaseg
  • crudini apevec jruzicka pbrady
  • cxxtest mgieseki
  • datanommer ralph
  • dlib bizdelnick luya
  • ec2-hibinit-agent davdunc
  • electrum tredaell
  • fbthrift dcavalca filbranden salimma
  • fedora-messaging abompard
  • firefox alexl caolanm erack gecko-maint jgrulich kalev kengert mbarnes rhughes rstrode sharkcz stransky tpopela ueno xhorak
  • folly dcavalca filbranden salimma
  • fontforge frixxon kevin pnemade
  • gajim michich suraia
  • gau2grid jussilehtola
  • gaupol music
  • gfal2-python adev andreamanzi
  • ginga lupinix
  • git-filter-repo asn
  • gns3-gui kwizart
  • gns3-net-converter kwizart
  • gns3-server kwizart nucleo
  • gpsd fab mlichvar ttorling
  • h5py stevetraylen terjeros
  • khard mathstuf sdyroff
  • kismon fab
  • koji ausil kevin mikem puiterwijk
  • legofy lkf williamjmorenor
  • libcaca hubbitus slaanesh thias
  • libnuml sagitter
  • libreoffice caolanm dtardon erack sbergmann
  • lldb airlied daveisfera jankratochvil sergesanspaille siddharths tstellar
  • llvm dmalcolm ignatenkobrain jakub jistone kyle scottt sergesanspaille siddharths tstellar
  • llvm10 sergesanspaille tstellar
  • llvm11 sergesanspaille tstellar
  • llvm7.0 jistone petersen sergesanspaille tstellar
  • llvm9.0 jistone sergesanspaille tstellar
  • mercurial kiilerix nbecker pstodulk
  • mod_wsgi jdornak jkaluza jorton lmacken mrunge
  • moose zbyszek
  • mypaint avsej
  • mysql-connector-python hhorak hubbitus hvad mschorm
  • nototools mfabian pwu
  • officeparser rebus
  • offlineimap cicku dodji sergesanspaille teuf
  • percol hubbitus
  • poetry churchyard thrnciar
  • poezio fantom louizatakk
  • porcupine kushal
  • protontricks atim
  • protonvpn-cli calexandru2018 jflory7
  • prunerepo frostyx praiskup
  • pyaudio chkr
  • pyhunspell mfabian
  • pyodbc fjanus hhorak
  • pyscard orion sjenning
  • pyserial stingray
  • python-CommonMark jujens
  • python-GridDataFormats rathann
  • python-OBD rathann
  • python-Pyped uggla
  • python-Rtree volter
  • python-acoustid terjeros
  • python-agate jujens
  • python-aiodns fantom
  • python-ansicolors fab
  • python-apprise lead2gold
  • python-apsw cicku dfateyev maci
  • python-arviz sergiopr
  • python-astroplan sergiopr
  • python-astropy-healpix lupinix
  • python-astroquery lupinix
  • python-astroscrappy lupinix
  • python-asttokens zbyszek
  • python-audioread terjeros
  • python-autobahn fab jujens
  • python-autopep8 mrunge ndipanov
  • python-b2sdk jonny
  • python-bloom cottsay rmattes
  • python-blosc tnorth zbyszek
  • python-box dmsimard fab
  • python-btchip jonny xenithorb
  • python-cached_property adamwill immanetize
  • python-carbon jsteffan piotrp
  • python-ccdproc lupinix
  • python-certbot-apache nb
  • python-chai kevin pingou ralph
  • python-cmigemo hubbitus
  • python-colorspacious rathann
  • python-construct moezroy terjeros
  • python-contextlib2 abompard pingou ralph tjikkun
  • python-cookiecutter chedi wakko666
  • python-crochet abompard
  • python-css-parser zbyszek
  • python-dbfread jujens
  • python-debrepo ktdreyer
  • python-decopatch zbyszek
  • python-descartes qulogic
  • python-dialog mjakubicek noodles raphgro sundaram zbyszek
  • python-dijitso limb
  • python-django-contact-form mrunge
  • python-django-health-check dmsimard
  • python-django-registration orphan
  • python-django-reversion mrunge
  • python-django-tagging jdornak mrunge piotrp
  • python-django-tastypie bkabrda cquad mrunge stevetraylen
  • python-docx kushal124
  • python-dtfabric fab
  • python-dukpy zbyszek
  • python-editorconfig barracks510
  • python-ephem fab
  • python-et_xmlfile jujens
  • python-etcd mbarnes smilner
  • python-fasteners mrunge
  • python-ffc zbyszek
  • python-fields cottsay
  • python-fisx zbyszek
  • python-fitsio lupinix
  • python-flake8-docstrings cottsay
  • python-flask-gravatar devrim orphan
  • python-flask-htmlmin devrim iztokf
  • python-flask-paranoid devrim orphan
  • python-flask-security devrim orphan
  • python-flask-sphinx-themes devrim orphan
  • python-flask-wtf-decorators frostyx
  • python-formats uggla
  • python-fypp rathann
  • python-gevent-eventemitter atim
  • python-google-i18n-address pwouters
  • python-googletrans lyessaadi
  • python-graphql-relay fab
  • python-heapdict qulogic
  • python-html5-parser kevin
  • python-htmlmin jujens
  • python-humblewx rickardlindberg
  • python-hupper kevin
  • python-i3ipc alebastr
  • python-inotify jfilak stevetraylen terjeros
  • python-jep raphgro
  • python-jinja2-cli jujens
  • python-jinja2-time chedi wakko666
  • python-jnius raphgro
  • python-joblib besser82 ignatenkobrain sergiopr
  • python-jsonmodels oanson
  • python-jsonrpclib ihrachyshka jonny
  • python-kerberos rcritten simo
  • python-kitchen kevin pingou ralph
  • python-landslide echevemaster salimma
  • python-lark-parser totol
  • python-leather jujens
  • python-libsass nonamedotc
  • python-libusb1 jonny
  • python-lmdb jruzicka pspacek tkrizek
  • python-logfury jonny
  • python-makefun zbyszek
  • python-managesieve stevetraylen
  • python-matrix-nio ankursinha
  • python-meld3 kevin stevetraylen tsao
  • python-minibelt uggla
  • python-mmtf rathann
  • python-mnemonic jonny
  • python-music21 zbyszek
  • python-mwclient adamwill rdieter tuxbrewr
  • python-myhdl filiperosset
  • python-nbclient nonamedotc
  • python-nbxmpp michich suraia
  • python-networkmanager jdulaney
  • python-notario ktdreyer
  • python-oauth2 ignatenkobrain pjp spot sundaram
  • python-openoffice sharkcz
  • python-ouimeaux kni
  • python-pandas-datareader sergiopr
  • python-paste-script andreamanzi
  • python-patsy sergiopr
  • python-pbkdf2 jonny
  • python-pecan-notario ktdreyer
  • python-pelican firemanxbr mrunge
  • python-pexpect amcnabb fabiand ignatenkobrain radez swt2c tomspur
  • python-plaster-pastedeploy abompard
  • python-plumbum greghellings lorenzodalrio
  • python-poetry-core churchyard thrnciar
  • python-polib cicku dchen diegobz dshea ivazquez moezroy suanand
  • python-precis_i18n michich
  • python-proteus sharkcz
  • python-publicsuffix2 rathann
  • python-pulsectl pfrields
  • python-pvc raphgro
  • python-pycares fantom
  • python-pycha potty sharkcz
  • python-pygeoip kevin ralph
  • python-pylons-sphinx-themes abompard
  • python-pymc3 sergiopr
  • python-pynn ankursinha
  • python-pyotp icon
  • python-pypng kevin ralph
  • python-pyramid_sawing abompard
  • python-pysb zbyszek
  • python-pysignals kni
  • python-pyswip pampelmuse
  • python-pyte terjeros
  • python-pytest-astropy-header sergiopr
  • python-pytest-cases zbyszek
  • python-pytest-harvest zbyszek
  • python-pytest-mock fab jujens
  • python-pytest-repeat cottsay
  • python-pytest-steps zbyszek
  • python-pytest-watch jujens
  • python-pyvo lupinix
  • python-readthedocs-sphinx-ext jjames
  • python-recommonmark jujens
  • python-relatorio sharkcz
  • python-restructuredtext-lint jujens
  • python-retrying apevec
  • python-rmtest lberk mgoodwin nathans
  • python-rosdep cottsay rmattes thofmann
  • python-sanction kevin ralph
  • python-scikit-learn besser82 ignatenkobrain lupinix sergiopr
  • python-scrapy echevemaster
  • python-setuptools-lint jdulaney
  • python-shamir-mnemonic jonny
  • python-simplemediawiki lmacken potty ralph
  • python-slixmpp fantom louizatakk
  • python-smbc twaugh zdohnal
  • python-snappy jujens
  • python-social-auth-core cqi
  • python-soupsieve zbyszek
  • python-spdx jbertozzi
  • python-spdx-lookup jbertozzi
  • python-sphinxcontrib-bibtex jjames
  • python-statsd pabelanger tdecacqu
  • python-steam atim
  • python-tables tnorth zbyszek
  • python-tempdir rathann
  • python-timeout-decorator jcapitao
  • python-tinydb suanand
  • python-tortilla uggla
  • python-tqdm ignatenkobrain sgallagh
  • python-tree-format chedi wakko666
  • python-trezor jonny
  • python-twilio mich181189
  • python-txaio fab jujens
  • python-unidecode pjp sundaram
  • python-unidiff dcallagh
  • python-upt-cpan jbertozzi
  • python-upt-fedora jbertozzi
  • python-upt-pypi jbertozzi
  • python-upt-rubygems jbertozzi
  • python-urwidtrees ttomecek
  • python-vdf atim
  • python-wand barracks510
  • python-watchdog jsteffan jujens pingou
  • python-webencodings abompard
  • python-webpy mrunge
  • python-wsaccel jujens
  • python-xlwt leamas moezroy rathann
  • python-xvfbwrapper mrunge totol
  • python-yarl fab ignatenkobrain
  • python-zstandard rathann
  • python3-postgresql hhorak
  • python3-pytest-asyncio jujens
  • python3-saml dcallagh tchaikov
  • qemu berrange bonzini crobinso dwmw2 ehabkost jforbes lkundrak quintela rjones
  • rdkit giallu
  • rpmspectool nphilipp
  • salt blarson dmurphy18 krionbsd
  • sepolicy_analysis vmojzis
  • solaar brouhaha rathann richardfearn tibbs
  • spec2scl jstanek
  • starcal hedayat
  • stomppy stevetraylen
  • sugar-speak callkalpa chimosky pbrobinson tuxbrewr
  • swid-tools adelton
  • swift-lang tachoknight
  • terminator dmaphy mattrose ohaessler
  • texlive-base spot
  • toot alciregi
  • translate-toolkit cicku dwayne petersen suanand
  • trellis lkundrak somlo
  • tryton sharkcz
  • trytond sharkcz
  • trytond-account sharkcz
  • trytond-account-be sharkcz
  • trytond-account-de-skr03 sharkcz
  • trytond-account-invoice sharkcz
  • trytond-account-invoice-history sharkcz
  • trytond-account-invoice-line-standalone sharkcz
  • trytond-account-product sharkcz
  • trytond-account-statement sharkcz
  • trytond-account-stock-anglo-saxon sharkcz
  • trytond-account-stock-continental sharkcz
  • trytond-analytic-account sharkcz
  • trytond-analytic-invoice sharkcz
  • trytond-analytic-purchase sharkcz
  • trytond-analytic-sale sharkcz
  • trytond-company sharkcz
  • trytond-company-work-time sharkcz
  • trytond-country sharkcz
  • trytond-currency sharkcz
  • trytond-dashboard sharkcz
  • trytond-google-maps sharkcz
  • trytond-ldap-authentication sharkcz
  • trytond-party sharkcz
  • trytond-party-siret sharkcz
  • trytond-product sharkcz
  • trytond-product-cost-fifo sharkcz
  • trytond-product-cost-history sharkcz
  • trytond-product-price-list sharkcz
  • trytond-project sharkcz
  • trytond-project-plan sharkcz
  • trytond-project-revenue sharkcz
  • trytond-purchase sharkcz
  • trytond-purchase-invoice-line-standalone sharkcz
  • trytond-sale sharkcz
  • trytond-sale-opportunity sharkcz
  • trytond-sale-price-list sharkcz
  • trytond-stock sharkcz
  • trytond-stock-forecast sharkcz
  • trytond-stock-inventory-location sharkcz
  • trytond-stock-location-sequence sharkcz
  • trytond-stock-product-location sharkcz
  • trytond-stock-supply sharkcz
  • trytond-stock-supply-day sharkcz
  • trytond-timesheet sharkcz
  • ubertooth avsej
  • uhd jskarvad
  • upt jbertozzi
  • uwsgi kad
  • watchman dcavalca filbranden salimma
  • wine-mono mooninite
  • winpdb spot
  • xrootd ellert simonm
  • xtensor-python sergesanspaille
  • yawn jsafrane miminar vcrhonek

Packages to be ignored:

  • fontforge - false positive
  • qemu - false positive
  • xrootd - false positive

Packages that failed to be analysed:

  • 0ad ignatenkobrain pcpa pwalter
  • GConf2 alexl buc caillon caolanm mbarnes rhughes ssp walters
  • TeXmacs jnovy orion
  • anaconda anaconda-maint jkonecny m4rtink rvykydal sbueno vpodzime vponcova
  • astrometry lupinix
  • bamf jspaleta salimma
  • boost denisarnaud jwakely trodgers
  • colobot suve
  • cozy suve
  • cppcheck c72578 jussilehtola sgrubb
  • criu adrian avagin
  • cxxtest mgieseki
  • dnf dmach jmracek jrohel mblaha pkratoch
  • eclipse-pydev jjohnstn orphan
  • edk2 bonzini crobinso kraxel
  • etcd cypret eparis gscrivano jchaloup lsm5 strigazi walters
  • fonts-tweak-tool jamesni tagoh
  • gdl orion
  • gimp-resynthesizer luya
  • global cheeselee landgraf
  • google-noto-emoji-fonts mfabian pwu
  • gst-devtools limb
  • guitarix bsjones tartina verdurin
  • hfg-gmuend-openmoji-fonts mavit
  • kernel acaringi airlied ajax bskeggs eparis glisse jcline jforbes jwboyer jwilson jwrdegoede kernel-maint lgoncalv linville myoung nhorman patrickt quintela sandeen steved
  • kernel-tools acaringi jcline jforbes jwboyer lgoncalv patrickt pbrobinson
  • kicad avigne coremodule lkundrak stevenfalco tnorth
  • lammps ellio167 junghans
  • libabigail dodji sinnykumari
  • libguestfs mdbooth rjones
  • libiptcdata dcm hobbes1069 jchaloup
  • libkkc ueno
  • libproxy amigadave cicku danw npmccallum
  • libreoffice caolanm dtardon erack sbergmann
  • libsvm besser82 jjames
  • link-grammar devos fabiand limb
  • nest ankursinha
  • nispor cathay4t ffmancera
  • opal-prd hegdevasant
  • openbabel alexpl jussilehtola rathann
  • openexr hobbes1069
  • proxyfuzz psklenar
  • python-OBD rathann
  • python-caja monnerat raveit65
  • python-flask-rstpages rmarko
  • python-graph-tool ankursinha
  • python-matrix-nio ankursinha
  • python-pivy hobbes1069 zultron
  • python-pycha potty sharkcz
  • python-pyqtgraph swt2c
  • python-pytest-spec fab
  • python-requests-cache codeblock hobbes1069
  • python-uranium-lulzbot orphan
  • python-webpy mrunge
  • python-wsaccel jujens
  • root ellert
  • rust-pyo3 decathorpe
  • rust-python3-sys decathorpe
  • seqan2 sagitter
  • sugar-browse aperezbios callkalpa chimosky manuq pbrobinson
  • sugar-calculator callkalpa chimosky pbrobinson tuxbrewr
  • suricata jtaylor sgrubb
  • texlive jnovy spot than
  • tiled ablu suve
  • tsung cicku tartare
  • unity-gtk-module besser82
  • vdrift limb rmattes
  • voikko-fi vpv
  • weechat asrob gchamoul hguemar niveusluna salimma stingray
  • xemacs-packages-extra jjames stevetraylen
  • zinnia liangsuilong pwu

Feedback

There was some discussion on various mailing list threads but no other alternatives were proposed.

Benefit to Fedora

From Fedora Packaging Guidelines: It is important that your package list all necessary build dependencies using the BuildRequires: tag. As our testing showed, this is not true for many packages. Some of them can successfully build either with or without setuptools (they use try-except import and fallback to distutils from the standard library). Such packages are especially dangerous when not BuildRequiring setuptools -- they can produce different results depending on the presence of setuptools: either an .egg-info metadata directory (w/setuptools) or .egg-info text file (w/distutils). RPM has troubles when upgrading directories to files.

Scope

Some packages use distutils as a fallback in case setuptools is unavailable. The problem is that setuptools generates egg.info directory and distutils creates egg.info file. This would lead to failures caused by the known issue of RPM that cannot change directory to file. To avoid this we plan to do a mass spec update before we upgrade Python to version 3.10. This way we ensure that path will be correct after the mass rebuild.

  • Proposal owners:
    1. Test affected packages in Copr.
    2. Do the mass spec update to introduce missing BuildRequire to them.
  • Other developers: There should be no additional work for other developers. There is a possibility that some packages will slip through our grep based query. This might cause some of them to FTBFS.
  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Objectives: N/A

Upgrade/compatibility impact

All affected packages needs rebuild after the mass spec update but since we plan to do it with the upgrade to Python 3.10 we will rebuild it altogether.

How To Test

Interested testers can push explicit BuildRequire on setuptools to the distgit and see in our Python 3.10 COPR whether the package starts to build again. Webhook should trigger rebuild automatically on new commit in rawhide branch.

User Experience

Regular distro users shouldn't notice any change in system behaviour.

Dependencies

300+ packages will need to BuildRequire setuptools explicitly. This will be done by the mass spec update.

Contingency Plan

  • Contingency mechanism: Commit removing python3-devel Requires on (python3-setuptools if rpmbuild) will be reverted.
  • Contingency deadline: mass rebuild
  • Blocks release? No

Documentation

This page is a documentation, this is not a user facing change.

Release Notes