Changes/DeprecatePythonToml

Deprecate python-toml

Summary

The python-toml (python3-toml) package will be deprecated in Fedora 38. The upstream toml package is considered unmaintained (see description) and Python 3.11 contains a TOML-reading library in the standard library. Existing Fedora packages depend on python-toml, so we cannot remove it yet. Packagers are encouraged to work with upstreams to switch to tomllib/tomli for reading toml or tomli-w for writing it. But python-toml remains available until it is a leaf package, it will be removed then (possibly not yet in Fedora 38).

Owner

• Name: Miro Hrončok
• Email: mhroncok@redhat.com

Current status

• Targeted release: Fedora Linux 38
• Last updated: 2022-12-29
• devel thread
• FESCo issue: #2879
• Tracker bug: #2137375
• Release notes tracker: #891

Detailed Description

The python-toml package is unmaintained upstream. It does not support the latest TOML standard and no longer releases newer versions.

We'd like to drop it from Fedora, but several packages still require it. Before we attempt to remove the package, we need to stop new packages to (Build)Require python3-toml, hence we want to have it deprecated. No new packages that require it may be added to Fedora and existing packages may not gain new dependencies on it. Requiring it only if python3 < 3.11 or similar is allowed because python3 on Fedora 38 is 3.11.

Packagers are encouraged to switch to an alternative TOML library with upstream involvement. Downstream-only patches to switch are not encouraged. Change owners recommend the following alternatives:

• Use the tomllib module from the standard library to read TOML with Python 3.11+.
• Use the python-tomli package to read TOML with an older version of Python. The tomllib module has started as tomli and they share the same API except for the module name.
• Use the python-tomli-w package to write TOML.

Note that repoquery gives many packages depending on python3-toml:

$ repoquery --repo=rawhide{,-source} --whatrequires python3-toml | wc -l
443

This is because many packages BuildRequire (python3dist(toml) if python3-devel < 3.11) due to pyproject-rpm-macros.

$ repoquery --repo=rawhide{,-source} --whatrequires '(python3dist(toml) if python3-devel < 3.11)' | wc -l
413

The change owners don't know how to easily filter them out, but when filtered the hard way, this remains:

(Results from 2022-10-05, may contain false positives.)

$ for pkg in $(repoquery --repo=rawhide{,-source} --whatrequires python3-toml); do repoquery -q --repo=rawhide{,-source} --requires $pkg | grep -Fv '(python3dist(toml) if python3-devel < 3.11)' | grep -Eq '(\(|-)toml\b' && echo $pkg; done
academic-admin-0:0.5.1-10.fc37.noarch
academic-admin-0:0.5.1-10.fc37.src
bandit-0:1.7.4-3.fc37.src
bst-external-0:0.29.0-1.fc38.src
cvc4-0:1.8-12.fc37.src
fedora-license-data-0:1.5-1.fc38.src
fedora-messaging-0:3.1.0-5.fc38.src
gi-docgen-0:2022.1-7.fc38.noarch
gi-docgen-0:2022.1-7.fc38.src
jrnl-0:3.0-3.fc37.src
micropipenv-0:1.4.2-1.fc37.noarch
pre-commit-0:2.20.0-2.fc37.noarch
pre-commit-0:2.20.0-2.fc37.src
pylint-0:2.14.4-3.fc37.src
python-anyconfig-0:0.13.0-3.fc37.src
python-anymarkup-0:0.8.1-10.fc37.src
python-anymarkup-core-0:0.8.1-9.fc37.src
python-ast-monitor-0:0.2.1-1.fc38.src
python-asttokens-0:2.0.8-1.fc38.src
python-autopep8-0:1.6.0-5.fc37.src
python-botocore-0:1.27.86-1.fc38.src
python-box-0:6.0.2-1.fc38.src
python-build-0:0.8.0-4.fc37.src
python-check-manifest-0:0.48-3.fc37.src
python-deepdiff-0:5.8.2-2.fc37.src
python-devicely-0:1.1.1-3.fc37.src
python-elpy-0:1.34.0-8.fc37.src
python-exoscale-0:0.7.1-4.fc37.src
python-fasjson-client-0:1.0.7-5.fc38.src
python-fireflyalgorithm-0:0.3.2-2.fc37.src
python-interrogate-0:1.5.0-4.fc37.src
python-jsonpickle-0:2.2.0-4.fc37.src
python-lsp-black-0:1.2.0-3.fc37.src
python-matrix-nio-0:0.19.0-6.fc38.src
python-molecule-podman-0:1.0.1-4.fc37.src
python-neurom-0:3.1.0-5.fc37.src
python-niaaml-0:1.1.11-1.fc38.src
python-niaarm-0:0.2.1-2.fc38.src
python-niaclass-0:0.1.2-8.fc37.src
python-nikola-0:8.2.2-4.fc37.src
python-pendulum-0:2.1.2-8.fc37.src
python-podman-3:4.2.0-7.fc38.src
python-pyedflib-0:0.1.30-2.fc37.src
python-pyqt-feedback-flow-0:0.1.6-3.fc37.src
python-rich-0:12.6.0-1.fc38.src
python-rst-linker-0:2.3.1-1.fc38.src
python-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.src
python-sport-activities-features-0:0.3.6-1.fc38.src
python-stochastic-0:0.7.0-2.fc37.src
python-toml-adapt-0:0.2.7-3.fc37.src
python-usort-0:0.6.3-7.fc37.src
python-vulture-0:2.6-1.fc38.src
python3-anymarkup-0:0.8.1-10.fc37.noarch
python3-autopep8-0:1.6.0-5.fc37.noarch
python3-box-0:6.0.2-1.fc38.noarch
python3-exoscale-0:0.7.1-4.fc37.noarch
python3-fasjson-client-0:1.0.7-5.fc38.noarch
python3-fedora-messaging-0:3.1.0-5.fc38.noarch
python3-interrogate-0:1.5.0-4.fc37.noarch
python3-jaraco-functools-0:3.5.2-1.fc38.noarch
python3-jinja2-cli-0:0.8.2-3.fc37.noarch
python3-lsp-black-0:1.2.0-3.fc37.noarch
python3-nikola-0:8.2.2-4.fc37.noarch
python3-podman-3:4.2.0-7.fc38.noarch
python3-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.noarch
python3-toml-adapt-0:0.2.7-3.fc37.noarch
python3-usort-0:0.6.3-7.fc37.noarch
python3-vulture-0:2.6-1.fc38.noarch
rpmlint-0:2.2.0-7.fc38.noarch
rpmlint-0:2.2.0-7.fc38.src
sip6-0:6.6.2-2.fc37.src
sip6-0:6.6.2-2.fc37.x86_64
teampulls-0:0.2.2-10.fc37.noarch
trac-tracnav-plugin-0:4.3-7.fc37.src

List of components still (Build)Requiring python3-toml

• academic-admin
• bandit
• bst-external
• cvc4
• fedora-license-data (false positive, dependency conditional on python3 < 3.11)
• fedora-messaging (upstream fix)
• gi-docgen (merged upstream PR)
• jrnl
• micropipenv (already fixed in the meantime)
• pre-commit
• pylint
• python-anyconfig
• python-anymarkup
• python-anymarkup-core
• python-ast-monitor
• python-asttokens
• python-autopep8
• python-botocore
• python-box
• python-build
• python-deepdiff
• python-devicely
• python-elpy
• python-exoscale
• python-fasjson-client
• python-fireflyalgorithm
• python-check-manifest
• python-interrogate
• python-jaraco-functools (superfluous, fixed)
• python-jinja2-cli
• python-jsonpickle
• python-lsp-black
• python-matrix-nio
• python-molecule-podman
• python-neurom
• python-niaaml
• python-niaarm
• python-niaclass
• python-nikola
• python-pendulum
• python-podman
• python-pyedflib
• python-pyqt-feedback-flow
• python-rich (superfluous, proposed fix)
• python-rst-linker
• python-sklearn-nature-inspired-algorithms
• python-sport-activities-features
• python-stochastic
• python-toml-adapt
• python-usort
• python-vulture
• rpmlint (already fixed in the meantime)
• sip6
• teampulls
• trac-tracnav-plugin

Once all dependencies are removed, we plan to retire python-toml, whether it will be in Fedora 38 (unlikely) or later.

Migrating to tomllib

Migrating to tomllib-only is only possible if the code will be run on Python 3.11 or newer. See the #Migrating_to_tomllib_on_Python_3.11+_and_falling_back_to_tomli section for a more portable solution.

• Remove any toml requirements from upstream metadata.
• Change all toml imports to tomllib imports.
• Change all references of TomlDecodeError to TOMLDecodeError.
• Open files in binary mode, if passing file objects to tomllib.load().

Migrating to tomli

Migrating to tomli-only leaves an unnecessary dependency on a 3rd party library which is now essentially a backport of a standard library module. See the #Migrating_to_tomllib_on_Python_3.11+_and_falling_back_to_tomli section for a more sustainable solution.

• Change any toml requirements in upstream metadata to tomli.
• Change all toml imports to tomli imports.
• Change all references of TomlDecodeError to TOMLDecodeError.
• Open files in binary mode, if passing file objects to tomli.load().

Migrating to tomllib on Python 3.11+ and falling back to tomli

• Change any toml requirements in upstream metadata to tomli;python_version<"3.11".
• Change all toml imports to sys.version_info-conditional or try: except ImportError: tomli/tomllib imports.
• Change all references of TomlDecodeError to TOMLDecodeError.
• Open files in binary mode, if passing file objects to tomllib.load().

Example of importing:

try:
    import tomllib
except ImportError:
    import tomli as tomllib

Or:

if sys.version_info < (3, 11):
    import tomli as tomllib
else:
    import tomllib

Upstream examples: https://github.com/fedora-infra/fedora-messaging/pull/274 or https://github.com/pytest-dev/pytest/pull/9741

A more complex example that also falls back to toml on Python 2.7 or 3.6: https://github.com/tox-dev/tox/pull/2463

A more complex example that supports pytoml, toml, tomli and tomllib: https://github.com/thoth-station/micropipenv/pull/241

Migrating to tomli-w

• Change any toml requirements in upstream metadata to tomli-w.
• Change all toml imports to tomli_w imports.
• Open files in binary mode, if passing file objects to tomli_w.dump().

A more complex example that migrates to tomli, tomllib and tomli-w: https://github.com/rpm-software-management/rpmlint/pull/905

Benefit to Fedora

An upstream dead package will not be depended upon by new packages.

Scope

• Proposal owners: Deprecate python3-toml. Work with packagers and upstream developers to remove the dependency. Monitor the remaining dependent packages and eventually retire python-toml (unlikely in Fedora 38).

• Other developers: No action needed. Don't add new dependencies on python3-toml.

• Release engineering: N/A (not a System Wide Change)

• Policies and guidelines: N/A (not needed for this Change)

• Trademark approval: N/A (not needed for this Change)

• Alignment with Objectives:

Upgrade/compatibility impact

The package will remain available. Only new packages cannot depend on it. Once retired, we don't plan to provide python3-toml from python3-libs or python3-tomli, because it cannot work as a drop-in replacement (the Python module has a different name and slightly different API). The package will eventually be obsoleted by fedora-obsolete-packages once retired, but that is unlikely to happen soon.

How To Test

$ repoquery --repo=rawhide --provides python3-toml
...
deprecated()
...

User Experience

No changes.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

• Contingency mechanism: revert the deprecation
• Contingency deadline: Final Freeze
• Blocks release? No

Documentation

N/A (not a System Wide Change)