Changes/SQLAlchemy 2: Difference between revisions

Revision as of 15:00, 6 October 2023

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

The python-sqlalchemy package is upgraded to major version 2. A compatibility package python-sqlalchemy1.4 is added to the distribution to cater for software which doesn’t yet use the new API, this can be installed alternatively. Other packages using SQLAlchemy are identified and, if necessary, steps are taken to ensure they use the correct major version package.

Owner

Name: Nils Philippsen, Čestmír Kalina
Email: nphilipp -at- redhat.com, ckalina -at- redhat.com

Current status

Targeted release: Fedora Linux 40
Last updated: 2023-10-06
Announced
Discussion thread
FESCo issue: #3074
Tracker bug: <will be assigned by the Wrangler>
Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

The major version 2 of SQLAlchemy was released early 2023, it contains many useful new features. It breaks compatibility with version 1.4 in many ways, this version is available in Fedora Linux and used by around 60 packages today. Version 1.4 allows using the new API so programs can accommodate both major versions, but how many of the packages using SQLAlchemy in Fedora have been adapted accordingly is unknown at this point, this needs to be determined. A (conflicting) compatibility package python-sqlalchemy1.4 will most likely need to be added to the distribution and packages not yet using the new API will have to be changed to use the new API or use this package instead for the time being.

Feedback

Originally, it was proposed that the compatibility package for version 1.4 would be installable in parallel, but in discussion it turned out the implementation would be technically complicated (an application would have to select which major version of SQLAlchemy to use on startup for which Python doesn’t have good tools, various approaches would break RPM dependency tooling) and possibly a maintenance nightmare.

Benefit to Fedora

Version 2 of SQLAlchemy has new features including:

deep integration with PEP 484 typing practices and current capabilities, particularly within the ORM, allowing e.g. to build ORM models declaratively using Python type annotations
fully ORM-integrated approach to bulk INSERT that is typically an order of magnitude faster on most backends

See the upstream release announcement for details.

Scope

The proposal owners …
- … have to upgrade python-sqlalchemy to the new major version and add the compatibility package to the distribution.
- … need to find out which of the existing packages in Fedora Linux use SQLAlchemy and determine if they are compatible with version 2 or not.
- … will document how incompatible packages need to be packaged differently to actually use the compatibility package.
- … will inform maintainers of these packages about this and assist them with applying the necessary changes.

Maintainers of packages using SQLAlchemy …
- … should independently verify if their packages work with the new version of SQLAlchemy (to avoid surprises).
- … if incompatible, either update them to be compatible with the new API, or …
- … let their package depend on the compatibility package.

This change isn’t expected to require involvement of release engineering, it just involves a number of dependent packages high enough to make it impractical to get all their maintainers on-board as proposal owners.

Policies and guidelines: N/A (not needed for this Change)

Trademark approval: N/A (not needed for this Change)

Alignment with Community Initiatives: M/A

Upgrade/compatibility impact

Upgrades would be subject to the potentially changed dependencies between packages, but this should be straightforward and have no compatibility issues.

How To Test

No special hardware or data is needed for testing.
Testers would need to install packages using SQLAlchemy and use them.
Installing the packages should pull in the “right” version, i.e. the default one with the new major version, or the compatibility package.
Using the packages should show no problems, neither on the command line nor in logs or similar.

User Experience

Users using packages already adapted to the new APIs should benefit from the new features, including performance enhancements.

Dependencies

A quick scan(*) on Fedora 39 beta for packages requiring the provides of python3-sqlalchemy yields this list of potentially affected dependent RPM source packages:

OpenLP
bodhi-server
buildbot
dionaea
eralchemy
fedmsg
ipsilon
keylime
limnoria
mailman3
mirrormanager2
module-build-service
odcs
pagure
pgadmin4
pychess
python-agate-sql
python-alembic
python-anykeystore
python-aws-xray-sdk
python-beaker
python-dask
python-databases
python-factory-boy
python-fastapi
python-flask-security-too
python-flask-sqlalchemy
python-geopandas
python-gertty
python-hass-data-detective
python-jsonpickle
python-kombu
python-libpysal
python-logbook
python-migrate
python-odata-query
python-opentelemetry-contrib
python-oslo-db
python-pybids
python-pykmip
python-pymssql
python-pynetdicom
python-repoze-who-plugins-sa
python-sentry-sdk
python-slackclient
python-sphinxcontrib-websupport
python-sqlacodegen
python-sqlalchemy-collectd
python-sqlalchemy-filters
python-sqlalchemy-utils
python-sqlalchemy_schemadisplay
python-testing.postgresql
python-wtforms-sqlalchemy
python-zope-sqlalchemy
resalloc
sigul

(*) using this command:

fedrq whatrequires-src -F source python-sqlalchemy

Contingency Plan

Contingency mechanism: (What to do? Who will do it?) Proposal owners would have to revert to version 1.4 of SQLAlchemy, dependent packages aren’t expected to have to revert applied changes.

Contingency deadline: Beta Freeze
Blocks release? No

Documentation

Release Notes

SQLAlchemy was upgraded to version 2 in this release of Fedora Linux with many new features and a revamped API. A compatibility package was put in place to ensure software that depends on the old API stays functional.

@@ Line 2: / Line 2: @@
 == Summary ==
-The <code>python-sqlalchemy</code> package is upgraded to major version 2. A compatibility package <code>python-sqlalchemy1.4</code> is added to the distribution to cater for software which doesn’t yet use the new API, this can be installed side-by-side. Other packages using SQLAlchemy are identified and, if necessary, steps are taken to ensure they use the correct major version package.
+The <code>python-sqlalchemy</code> package is upgraded to major version 2. A compatibility package <code>python-sqlalchemy1.4</code> is added to the distribution to cater for software which doesn’t yet use the new API, this can be installed alternatively. Other packages using SQLAlchemy are identified and, if necessary, steps are taken to ensure they use the correct major version package.
 == Owner ==
@@ Line 38: / Line 38: @@
 == Detailed Description ==
 <!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
-The major version 2 of SQLAlchemy was released early 2023, it contains many useful new features. It breaks compatibility with version 1.4 in many ways, this version is available in Fedora Linux and used by about 40-50 packages today. Version 1.4 allows using the new API so programs can accommodate both major versions, but how many of the packages using SQLAlchemy in Fedora have been adapted accordingly is unknown at this point, this needs to be determined. A parallel-installable compatibility package <code>python-sqlalchemy1.4</code> will most likely need to be added to the distribution and packages not yet using the new API will have to be changed to use the new API or use this package instead for the time being.
+The major version 2 of SQLAlchemy was released early 2023, it contains many useful new features. It breaks compatibility with version 1.4 in many ways, this version is available in Fedora Linux and used by around 60 packages today. Version 1.4 allows using the new API so programs can accommodate both major versions, but how many of the packages using SQLAlchemy in Fedora have been adapted accordingly is unknown at this point, this needs to be determined. A (conflicting) compatibility package <code>python-sqlalchemy1.4</code> will most likely need to be added to the distribution and packages not yet using the new API will have to be changed to use the new API or use this package instead for the time being.
 == Feedback ==
 <!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->
+Originally, it was proposed that the compatibility package for version 1.4 would be installable in parallel, but in discussion it turned out the implementation would be technically complicated (an application would have to select which major version of SQLAlchemy to use on startup for which Python doesn’t have good tools, various approaches would break RPM dependency tooling) and possibly a maintenance nightmare.
 == Benefit to Fedora ==
@@ Line 52: / Line 54: @@
 == Scope ==
-* The proposal owners have to upgrade <code>python-sqlalchemy</code> to the new major version and add the compatibility package to the distribution. They need to find out which of the existing packages in Fedora Linux don’t work with the new version and document how they need to be packaged differently to actually use the compatibility package.
+* The proposal owners …
+** … have to upgrade <code>python-sqlalchemy</code> to the new major version and add the compatibility package to the distribution.
+** … need to find out which of the existing packages in Fedora Linux use SQLAlchemy and determine if they are compatible with version 2 or not.
+** … will document how incompatible packages need to be packaged differently to actually use the compatibility package.
+** … will inform maintainers of these packages about this and assist them with applying the necessary changes.
-* Other maintainers of packages using SQLAlchemy need to verify if their packages work with the new version of SQLAlchemy and, if not, do the necessary changes to:
+* Maintainers of packages using SQLAlchemy …
-** make them compatible with the new API, or
+** … should independently verify if their packages work with the new version of SQLAlchemy (to avoid surprises).
-** ensure that the compatibility package is installed and also used by the code during runtime.
+** … if incompatible, either update them to be compatible with the new API, or …
+** … let their package depend on the compatibility package.
 * This change isn’t expected to require involvement of release engineering, it just involves a number of dependent packages high enough to make it impractical to get all their maintainers on-board as proposal owners.
@@ Line 112: / Line 119: @@
 -->
-Users using packages already adapted to the new APIs, they should benefit from the new features, including performance enhancements.
+Users using packages already adapted to the new APIs should benefit from the new features, including performance enhancements.
 == Dependencies ==
@@ Line 119: / Line 126: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-A quick scan(*) on Fedora 38 for packages requiring the provides of <code>python3-sqlalchemy</code> yields this list of potentially affected dependent RPM source packages:
+A quick scan(*) on Fedora 39 beta for packages requiring the provides of <code>python3-sqlalchemy</code> yields this list of potentially affected dependent RPM source packages:
+* OpenLP
 * bodhi-server
 * buildbot
-* datagrepper
 * dionaea
-* dlrn
 * eralchemy
+* fedmsg
 * ipsilon
 * keylime
@@ Line 133: / Line 140: @@
 * mirrormanager2
 * module-build-service
-* OpenLP
+* odcs
 * pagure
 * pgadmin4
 * pychess
 * python-agate-sql
-* python-aiomysql
 * python-alembic
+* python-anykeystore
+* python-aws-xray-sdk
+* python-beaker
+* python-dask
 * python-databases
-* python-datanommer-models
+* python-factory-boy
+* python-fastapi
 * python-flask-security-too
 * python-flask-sqlalchemy
+* python-geopandas
 * python-gertty
 * python-hass-data-detective
+* python-jsonpickle
+* python-kombu
+* python-libpysal
+* python-logbook
 * python-migrate
 * python-odata-query
@@ Line 152: / Line 168: @@
 * python-pybids
 * python-pykmip
+* python-pymssql
+* python-pynetdicom
 * python-repoze-who-plugins-sa
 * python-sentry-sdk
+* python-slackclient
+* python-sphinxcontrib-websupport
 * python-sqlacodegen
 * python-sqlalchemy-collectd
 * python-sqlalchemy-filters
+* python-sqlalchemy-utils
 * python-sqlalchemy_schemadisplay
-* python-sqlalchemy-utils
+* python-testing.postgresql
 * python-wtforms-sqlalchemy
 * python-zope-sqlalchemy
 * resalloc
 * sigul
-* yokadi
 (*) using this command:
-  <nowiki>rpm -q --provides python3-sqlalchemy | while read prov rest; do
+  <nowiki>fedrq whatrequires-src -F source python-sqlalchemy</nowiki>
-    dnf repoquery --qf '%{sourcerpm}' --whatrequires "$prov"
-done | \
-    sed 's|-[^-]*-[^-]*$||g' | \
-    grep -vFx python-sqlalchemy | \
-    sort -u</nowiki>
 == Contingency Plan ==

Search