Revision as of 20:09, 5 October 2022

Deprecate python-toml

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

The python-toml (python3-toml) package will be deprecated in Fedora 38. The upstream package is considered dead and Python 3.11 contains a toml-reading library in the standard library. Existing Fedora packages depend on python3-toml, so we cannot remove it yet. Packagers are encouraged to work with upstream to switch to tomllib/tomli for reading toml or tomli-w for writing it. But python-toml remains available until it is a leaf package, it will be removed then (possibly not yet in Fedora 38).

Owner

Name: Miro Hrončok
Email: mhroncok@redhat.com

Current status

Targeted release: Fedora Linux 38
Last updated: 2022-10-05
FESCo issue: <will be assigned by the Wrangler>
Tracker bug: <will be assigned by the Wrangler>
Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

The python-toml package is unmaintained upstream. It does not support the latest TOML standard and no longer releases newer versions.

We'd like to drop it from Fedora, but several packages still require it. Before we attempt to remove the package, we need to stop new packages to (Build)Require python3-toml, hence we want to have it deprecated.

Packagers are encouraged to switch to an alternative toml library with upstream involvement. Downstream-only patches to switch are not encouraged. Change owners recommend the following alternatives:

Use the tomllib module from the standard library to read TOML with Python 3.11+.
Use the python-tomli package to read TOML with an older version of Python. The tomllib module has started as tomli and they share the same API except for the module name.
Use the python-tomli-w package to write TOML.

Note that repoquery gives many packages that BuildRequire python3-toml:

$ repoquery --repo=rawhide{,-source} --whatrequires python3-toml | wc -l
443

This is because many packages BuildRequire (python3dist(toml) if python3-devel < 3.11) due to pyproject-rpm-macros.

$ repoquery --repo=rawhide{,-source} --whatrequires '(python3dist(toml) if python3-devel < 3.11)' | wc -l
413

The change owners don't know how to easily filter them out, but when filtered the hard way, this remains:

$ for pkg in $(repoquery --repo=rawhide{,-source} --whatrequires python3-toml); do repoquery -q --repo=rawhide{,-source} --requires $pkg | grep -Fv '(python3dist(toml) if python3-devel < 3.11)' | grep -Eq '\btoml\b' && echo $pkg; done
PyDrive2-0:1.14.0-2.fc37.src
academic-admin-0:0.5.1-10.fc37.noarch
academic-admin-0:0.5.1-10.fc37.src
ansible-lint-1:6.7.0-1.fc38.src
bandit-0:1.7.4-3.fc37.src
bst-external-0:0.29.0-1.fc38.src
cvc4-0:1.8-12.fc37.src
fedfind-0:5.0.0-4.fc37.src
fedora-license-data-0:1.4-1.fc38.src
fedora-messaging-0:3.1.0-5.fc38.src
gi-docgen-0:2022.1-7.fc38.noarch
gi-docgen-0:2022.1-7.fc38.src
hatch-0:1.5.0-4.fc38.src
ini2toml-0:0.10-3.fc37.src
jrnl-0:3.0-3.fc37.src
liquidctl-0:1.10.0-2.fc37.src
micropipenv-0:1.4.2-1.fc37.noarch
ocrmypdf-0:13.7.0-1.fc38.src
pre-commit-0:2.20.0-2.fc37.noarch
pre-commit-0:2.20.0-2.fc37.src
pylint-0:2.14.4-3.fc37.src
pytest-0:7.1.3-1.fc38.src
python-PyMunin3-0:3.0.1-1.fc38.src
python-SALib-0:1.4.5-4.fc37.src
python-amply-0:0.1.5-6.fc37.src
python-anyconfig-0:0.13.0-3.fc37.src
python-anyio-0:3.5.0-4.fc37.src
python-anymarkup-0:0.8.1-10.fc37.src
python-anymarkup-core-0:0.8.1-9.fc37.src
python-ast-monitor-0:0.2.1-1.fc38.src
python-asttokens-0:2.0.8-1.fc38.src
python-autopep8-0:1.6.0-5.fc37.src
python-black-0:22.8.0-1.fc38.src
python-botocore-0:1.27.84-1.fc38.src
python-boutdata-0:0.1.9-1.fc38.src
python-boututils-0:0.1.9-2.fc38.src
python-box-0:6.0.2-1.fc38.src
python-build-0:0.8.0-4.fc37.src
python-check-manifest-0:0.48-3.fc37.src
python-cmake-build-extension-0:0.5.1-4.fc37.src
python-configupdater-0:3.1-3.fc37.src
python-cppy-0:1.2.1-2.fc37.src
python-cssutils-0:2.4.2-2.fc37.src
python-deepdiff-0:5.8.2-2.fc37.src
python-devicely-0:1.1.1-3.fc37.src
python-django-auth-ldap-0:4.1.0-3.fc37.src
python-elpy-0:1.34.0-8.fc37.src
python-enrich-0:1.2.7-5.fc38.src
python-executing-0:1.1.0-1.fc38.src
python-exoscale-0:0.7.1-4.fc37.src
python-fasjson-client-0:1.0.7-5.fc38.src
python-fireflyalgorithm-0:0.3.2-2.fc37.src
python-flask-compress-0:1.13-2.fc38.src
python-humanize-0:3.13.1-5.fc37.src
python-importlib-metadata-0:4.12.0-2.fc37.src
python-interrogate-0:1.5.0-4.fc37.src
python-jaraco-classes-0:3.2.3-1.fc38.src
python-jaraco-envs-0:2.4.0-1.fc38.src
python-jaraco-functools-0:3.5.2-1.fc38.src
python-jaraco-packaging-0:9.1.1-1.fc38.src
python-jaraco-path-0:3.3.1-6.fc37.src
python-jsonpickle-0:2.2.0-4.fc37.src
python-keyring-0:23.9.3-1.fc38.src
python-keyrings-alt-0:4.1.2-1.fc38.src
python-kiwisolver-0:1.4.4-1.fc37.src
python-lsp-black-0:1.2.0-3.fc37.src
python-lsp-server-0:1.4.1-3.fc37.src
python-matrix-nio-0:0.19.0-6.fc38.src
python-molecule-0:4.0.1-3.fc38.src
python-molecule-docker-0:2.1.0-1.fc38.src
python-molecule-podman-0:1.0.1-4.fc37.src
python-nest-asyncio-0:1.5.5-3.fc37.src
python-neurom-0:3.1.0-5.fc37.src
python-niaaml-0:1.1.11-1.fc38.src
python-niaarm-0:0.2.1-2.fc38.src
python-niaclass-0:0.1.2-8.fc37.src
python-nikola-0:8.2.2-4.fc37.src
python-path-0:16.5.0-1.fc38.src
python-pendulum-0:2.1.2-8.fc37.src
python-pikepdf-0:5.6.1-1.fc38.src
python-pint-0:0.16.1-8.fc37.src
python-podman-3:4.2.0-7.fc38.src
python-portend-0:3.1.0-7.fc37.src
python-pure-eval-0:0.2.2-3.fc37.src
python-pyedflib-0:0.1.30-2.fc37.src
python-pymssql-0:2.2.5-3.fc37.src
python-pyqt-feedback-flow-0:0.1.6-3.fc37.src
python-pyscaffold-0:4.2.1-5.fc38.src
python-pytest-asyncio-0:0.19.0-1.fc37.src
python-pytest-localserver-0:0.7.0-1.fc38.src
python-pytest-services-0:2.2.1-4.fc37.src
python-rich-0:12.6.0-1.fc38.src
python-rst-linker-0:2.3.1-1.fc38.src
python-shtab-0:1.5.5-3.fc37.src
python-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.src
python-sphinx_ansible_theme-0:0.9.1-3.fc37.src
python-sport-activities-features-0:0.3.6-1.fc38.src
python-stack-data-0:0.5.1-1.fc38.src
python-stochastic-0:0.7.0-2.fc37.src
python-streamlink-0:5.0.1-1.fc38.src
python-subprocess-tee-0:0.3.5-5.fc37.src
python-tenacity-0:8.0.1-6.fc37.src
python-tinycss2-0:1.1.1-4.fc37.src
python-toml-adapt-0:0.2.7-3.fc37.src
python-tqdm-0:4.64.1-1.fc38.src
python-twine-0:4.0.1-2.fc37.src
python-typeguard-0:2.13.3-5.fc38.src
python-ujson-0:5.5.0-1.fc38.src
python-usort-0:0.6.3-7.fc37.src
python-vulture-0:2.6-1.fc38.src
python-xarray-0:2022.3.0-3.fc38.src
python-xbout-0:0.3.3-1.fc38.src
python-xmlsec-0:1.3.12-3.fc37.src
python-zipp-0:3.8.1-2.fc37.src
python3-anymarkup-0:0.8.1-10.fc37.noarch
python3-autopep8-0:1.6.0-5.fc37.noarch
python3-box-0:6.0.2-1.fc38.noarch
python3-exoscale-0:0.7.1-4.fc37.noarch
python3-fasjson-client-0:1.0.7-5.fc38.noarch
python3-fedora-messaging-0:3.1.0-5.fc38.noarch
python3-interrogate-0:1.5.0-4.fc37.noarch
python3-jaraco-functools-0:3.5.2-1.fc38.noarch
python3-jinja2-cli-0:0.8.2-3.fc37.noarch
python3-lsp-black-0:1.2.0-3.fc37.noarch
python3-nikola-0:8.2.2-4.fc37.noarch
python3-podman-3:4.2.0-7.fc38.noarch
python3-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.noarch
python3-toml-adapt-0:0.2.7-3.fc37.noarch
python3-usort-0:0.6.3-7.fc37.noarch
python3-vulture-0:2.6-1.fc38.noarch
rapidyaml-0:0.4.1-6.fc38.src
rpmlint-0:2.2.0-7.fc38.noarch
rpmlint-0:2.2.0-7.fc38.src
sip6-0:6.6.2-2.fc37.src
sip6-0:6.6.2-2.fc37.x86_64
teampulls-0:0.2.2-10.fc37.noarch
trac-tracnav-plugin-0:4.3-7.fc37.src

That's 117 components:

academic-admin
ansible-lint
bandit
bst-external
cvc4
fedfind
fedora-license-data
fedora-messaging
gi-docgen
hatch
ini2toml
jrnl
liquidctl
micropipenv (already fixed in the meantime)
ocrmypdf
pre-commit
PyDrive2
pylint
pytest
python-amply
python-anyconfig
python-anyio
python-anymarkup
python-anymarkup-core
python-ast-monitor
python-asttokens
python-autopep8
python-black
python-botocore
python-boutdata
python-boututils
python-box
python-build
python-cmake-build-extension
python-configupdater
python-cppy
python-cssutils
python-deepdiff
python-devicely
python-django-auth-ldap
python-elpy
python-enrich
python-executing
python-exoscale
python-fasjson-client
python-fireflyalgorithm
python-flask-compress
python-humanize
python-check-manifest
python-importlib-metadata
python-interrogate
python-jaraco-classes
python-jaraco-envs
python-jaraco-functools
python-jaraco-packaging
python-jaraco-path
python-jinja2-cli
python-jsonpickle
python-keyring
python-keyrings-alt
python-kiwisolver
python-lsp-black
python-lsp-server
python-matrix-nio
python-molecule
python-molecule-docker
python-molecule-podman
python-nest-asyncio
python-neurom
python-niaaml
python-niaarm
python-niaclass
python-nikola
python-path
python-pendulum
python-pikepdf
python-pint
python-podman
python-portend
python-pure-eval
python-pyedflib
python-pymssql
python-PyMunin3
python-pyqt-feedback-flow
python-pyscaffold
python-pytest-asyncio
python-pytest-localserver
python-pytest-services
python-rich
python-rst-linker
python-SALib
python-shtab
python-sklearn-nature-inspired-algorithms
python-sphinx_ansible_theme
python-sport-activities-features
python-stack-data
python-stochastic
python-streamlink
python-subprocess-tee
python-tenacity
python-tinycss2
python-toml-adapt
python-tqdm
python-twine
python-typeguard
python-ujson
python-usort
python-vulture
python-xarray
python-xbout
python-xmlsec
python-zipp
rapidyaml
rpmlint (already fixed in the meantime)
sip6
teampulls
trac-tracnav-plugin

Once all dependencies are removed, we plan to retire python-toml, whether it will be in Fedora 38 (unlikely) or later.

Migrating to tomllib

Remove any toml requirements from upstream metadata.
Change all toml imports to tomllib imports.
Change all references of TomlDecodeError to TOMLDecodeError.
Open files in binary mode, if passing file objects to tomllib.load().

Migrating to tomli

Change any toml requirements in upstream metadata to tomli.
Change all toml imports to tomli imports.
Change all references of TomlDecodeError to TOMLDecodeError.
Open files in binary mode, if passing file objects to tomli.load().

Migrating to tomllib on Python 3.11+ and falling back to tomli

Change any toml requirements in upstream metadata to tomli;python_version<"3.11".
Change all toml imports to sys.version_info-conditional or try: except ImportError: tomli/tomllib imports.
Change all references of TomlDecodeError to TOMLDecodeError.
Open files in binary mode, if passing file objects to tomllib.load().

Example of importing:

try:
    import tomllib
except ImportError:
    import tomli as tomllib

Or:

if sys.version_info < (3, 11):
    import tomli as tomllib
else:
    import tomllib

A more complex example that also falls back to toml on Python 2.7 or 3.6: https://github.com/tox-dev/tox/pull/2463

A more complex example that supports pytoml, toml, tomli and tomllib: https://github.com/thoth-station/micropipenv/pull/241

Migrating to tomli-w

Change any toml requirements in upstream metadata to tomli-w.
Change all toml imports to tomli_w imports.
Open files in binary mode, if passing file objects to tomli_w.dump().

A more complex example that migrates to tomli, tomllib and tomli-w: https://github.com/rpm-software-management/rpmlint/pull/905

Feedback

Benefit to Fedora

An upstream dead package will not be depended upon by new packages.

Scope

Proposal owners: Deprecate python3-toml. Work with packagers and upstream developers to remove the dependency. Monitor the remaining dependent packages and eventually retire python-toml (unlikely in Fedora 38).

Other developers: No action needed. Don't add new dependencies on python3-toml.

Release engineering: N/A (not a System Wide Change)

Policies and guidelines: N/A (not needed for this Change)

Trademark approval: N/A (not needed for this Change)

Alignment with Objectives: Minimization (the plan is to have one less Python TOML library in Fedora and RHEL 10+)

Upgrade/compatibility impact

The package will remain available. Only new packages cannot depend on it. Once retired, we don't plan to provide python3-toml from python3-libs or python3-tomli, because it cannot work as drop-in replacement (the Python module has a different name and sligthly different API). The package will eventually be obsoleted by fedora-obsolete-packages once retired, but that is unlikely to happen soon.

How To Test

$ repoquery --repo=rawhide --provides python3-toml
...
deprecated()
...

User Experience

No changes.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

Contingency mechanism: revert the deprecation
Contingency deadline: Final Freeze
Blocks release? No

Documentation