Changes/F40 MariaDB MySQL repackaging

F40 MariaDB & MySQL repackaging

Summary

A bigger set of smaller changes which I want to extend visibility for:

• [Task deferred] Drop builds for i686 architecture [Note: no longer intended to be completed in F40 timeframe]
• [Finished] Rename package 'community-mysql' to 'mysql' and Stop providing 'mysql' symbols by package 'mariadb'
• [Finished] Drop cross-installation functionality
• [Finished] Switch to the versioned layout of MariaDB and MySQL packages
• [Partly finished | partly deferred] Introduce MariaDB 10.11 and MySQL 8.1 [Note: MySQL 8.1 no longer intended to be introduced in F40 development timeframe]
• [Finished] Change the default MariaDB version in Fedora from 10.5 to 10.11

Owner

• Name: Michal Schorm
• Email: mschorm@redhat.com

Current status

• Targeted release: Fedora Linux 40
• Last updated: 2024-02-28

• Announced
• Discussion thread
• FESCo issue: #3092
• Tracker bug: #2252771
• Release notes tracker: #1097

Detailed Description

Most (if not all) of the changes I propose here doesn't probably need such a formal process as Fedora Change is. What I want is primarily to increase the visibility of the changes, and preserve those information in a permanent document, allowing both users and maintainers to find this page both now, and in the future.

The changes described here are something I mostly need to be done, as a maintainer. They are my informed decisions backed up by the expertise I've gained over the many years I maintain these packages. I believe in the community discussion to focus primarily the technical aspects of the changes to help me to implement them correctly, bug-free, rather than whether to apply these changes at all.

Moreover, the changes here are described in a specific order. The order is based on how each task logically follow other. In case some of the changes would be found problematic, it might be dropped from the proposal (with all changes that directly relies on it)

Drop builds for i686 architecture:

[Task deffered]
EDIT: 02/11/2023: I found this specific task too complex. I still aim to complete it, but no longer in the F40 timeframe !
I failed to find a methodology how to get a correct recursive list of all dependencies of packages intended to be removed, for a given architecture.

Both MariaDB and MySQL databases are huge pieces of software, taking hours to compile and test, taking up noticeable amount of system resources on our builders and any attached system (e.g. repositories). At the same time, I can hardly imagine anyone running those databases in production on i686 nowadays.
Nothing, expect the 'mariadb' and 'community-mysql' packages, which provide the DB servers and client application. Drop of the other packages from the stack is not part of this proposal. So for now, packages like 'mariadb-connector-c' (which provides the client library 'libmariadb.so') and the ODBC or JAVA connectors will keep to build and ship their i686 variants.
However the DB servers themselves are IMO expendable.

On top of the resources saved, I also keep finding bugs and issues, that are exclusive for the i686 architecture. They are more often than not in the tooling around, rather than the package itself, but it keeps needlessly devouring my time and energy anyway.

Last but not least, all maintainers are currently encouraged to drop i686 support, starting from leaf packages.
https://fedoraproject.org/wiki/Changes/EncourageI686LeafRemoval

So I have to make sure the MariaDB and MySQL are leaf package on i686 architecture, so I can drop their i686 variants too.
I've started the process by fixing https://bugzilla.redhat.com/show_bug.cgi?id=2241091

I use the following code to find out which packages requires which provides:

 #!/bin/bash
 mkdir -p RESULTS
 for REPO in "rawhide" "rpmfusion-free" "rpmfusion-nonfree" ; do
   for PROVIDE in "mariadb" "mariadb-devel" "mariadb-connector-c" "mariadb-connector-c-devel" "pkgconfig(mariadb)" "pkgconfig(libmariadb)" "mysql" "mysql-devel" "community-mysql" "community-mysql-devel" "pkgconfig(mysql)" ; do
     echo -e "\n\nREPO: $REPO ; PROVIDE: $PROVIDE";
     dnf -q --repo="$REPO" --repo="$REPO"-source repoquery --whatrequires "$PROVIDE" --alldeps | tee "./RESULTS/$REPO-$PROVIDE"
   done
 done

It show that other than the last few occurrences described in the bugzilla above, all is ready.

[Finished]
Rename package 'community-mysql' to 'mysql'
and Stop providing 'mysql' symbols by package 'mariadb':

When MariaDB was introduced to Fedora, it seemed like it eventually replaces MySQL and therefore the packages were designed so that 'mysql' names were provided by MariaDB, and MySQL was renamed to community-mysql.

The both projects were drop-in replacements at that time, so that design helped the smooth transition. Since then, a lot changed. First, those two databases developed different sets of features. The last "major major" version of MariaDB marked as a drop-in replacement is MariaDB 5.5. From MariaDB 10.0, it is usually still easy to upgrade from one to another.
https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/

Since MySQL 8.0 and circa MariaDB 10.5, the differences grew significantly and so it does not make sense anymore to provide 'mysql' names (= 'mysql' RPM Provides: ) by MariaDB package(s).

Moreover, the package name 'community-mysql' is Fedora specific. Oracle upstream uses 'mysql-community', Debian, RHEL and CentOS Stream use 'mysql'. From my experience I can say users usually search for 'mysql' name. To keep the naming convention closer to Fedora-related clones, the proposal is to rename the MySQL packages from 'community-mysql' to 'mysql'.

This change will save me, the maintainer, noticeable amount of time and energy when cherry-picking commits and patches from Fedora to CentOS Stream and RHEL.
And the more energy I save downstream, the more I can put into Fedora and upstream.

This change was being prepared to be proposed back for Fedora 37, but was left unfinished due to capacity reasons: https://fedoraproject.org/wiki/Changes/MariaDB_MySQL_Renaming

[Finished]
Drop cross-installation functionality:

In Fedora, it is currently possible, on the packaging level, to cross install server of one DB with client of another.
Specifically, you can install MariaDB server with MySQL client or MySQL server with MariaDB client.

I introduced this functionality to try it out, in hope of delivering a handy enhancement for the users.
Sadly, the drawbacks out-weights the positives. This behavior became a generator of elusive bugs I was never able to resolve.
E.g.: https://bugzilla.redhat.com/show_bug.cgi?id=2026933

Moreover, as the MariaDB and MySQL projects diverged significantly (see the above point), it stopped to make sense around the time the MariaDB stopped to be drop-in replacement for MySQL.

AFAIK, Fedora is the only distribution I know of, which ever allowed such installation combination.
I though it would be interesting, I tried it, I found out, now I want to end the experiment.

This change should not affect any other packages from the stack ('mariadb-connector-c', the ODBC, Python or JAVA connectors, ...) as they implement the API on their own and don't rely on the client application.

[Finished]
Switch to the versioned layout of MariaDB and MySQL packages:

I used the modularity heavily, as a package maintainer.
It allowed me to ship alternative versions of MariaDB that I or the users wanted. During chnages of the default major version of MariaDB in Fedora, they served as a handy way for anyone to use them way before the change is implemented, or after it when the haven't adapted their systems to the new default version. I've regularly selected the versions I maintain in CentOS Stream and RHEL - those which received the most care - and supported them fro the longest time as Fedora modules.

With Modularity retired https://fedoraproject.org/wiki/Changes/RetireModularity this ability has been lost.

When researching alternative approaches, only one seem to cover the same goals without too much of additional fuss.
That is the "Multiple packages with the same base name":
https://docs.fedoraproject.org/en-US/packaging-guidelines/Naming/#multiple

I want to change the packaging structure so the result will look as follows:

• The unversioned name ('mariadb') will become a meta-package
  • It will point to the one versioned variant which I choose to be the default one for the given Fedora release
  • It will provide all of the unversioned names for the versioned variant that is default for the given Fedora release, to minimize the changes visible to the users
• All other versions will have their own versioned package (e.g "mariadb10.5" "mariadb10.11") and will conflict with each other

This will allow for:

• users to keep using the unversioned names they are used to
• maintainer to change the default version for a given Fedora release on a single, centralized place
• users to enjoy all of the features of the modularity I offered them, in a simpler way
• maintainer to add new versions quickly, without any need of changing the default version (other than adding new conflicts)

 Note:
 I specifically don't want the packages to be parallel installable. I only want them to be parallel available.
 That's why I didn't choose "Alternatives" as the solution:
 https://docs.fedoraproject.org/en-US/packaging-guidelines/Alternatives/

[Partly Finished | Partly deferred]
Introduce MariaDB 10.11 and MySQL 8.1:

EDIT: 27/02/2024: I've only had capacity to introduce MariaDB 10.11. The introduction of MySQL 8.1 or later has been deferred.

I would like to introduce MariaDB 10.11 and MySQL 8.1 into the Fedora.

Based on the previous point, it should be achieved without any user disturbance, as I won't change what is the default version in the distribution, I will just add an alternative for the users to choose from.

Proof of concept (needs additional work) for MariaDB 10.11 here:
https://koji.fedoraproject.org/koji/taskinfo?taskID=102927697
https://src.fedoraproject.org/fork/mschorm/rpms/mariadb/commits/10.11-LTS

[Finished]
Change the default MariaDB version in Fedora from 10.5 to 10.11:

EDIT: 27/02/2024: Not only the MariaDB 10.11 is now the default version, but MariaDB 10.5 is no longer available, due to my capacity.

To keep being the leading edge distribution, I propose to update the default version of MariaDB in Fedora to the latest upstream LTS version, the MariaDB 10.11.

I have received a feedback on several occasions from various users that they would love to see this version in Fedora. Since I made a working proof of concept, I am confident that I will introduce the new version in time.

The MariaDB 10.6 LTS release has been skipped (was only available as a module) in Fedora, due to strategic reasons. At the time of early days of MariaDB 10.5 and 10.6, the MariaDB upstream changed their strategy of delivering the new releases. They switched to predictable quarterly releases, with new "major major" release every quarter. With this change, all releases are short-term releases with a support for only a single year. Some of the releases are made LTS, mostly based on the community feedback, discussion and demand. This all was new and unclear at the beginning, so we selected 10.5 LTS to be the main version maintained through Fedora, CentOS Stream and RHEL, so users from all three operating systems would contribute via reports to the health of the release and all those user's MariaDB would receive the same care. When the 10.6 was announced to become LTS, we didn't rush for it, both due to capacity reasons and to keep the bigger user-base advantage. We didn't know how long it would take for the next LTS to be announced.

Now, when the situation is clear, and the 10.11 was announced as the LTS, I believe it is appropriate to upgrade to this version in Fedora (and work is being started downstream to again keep the bigger user-base advantage).

If the previous two changes are accepted and implemented, this change will be only a matter of updating the 'mariadb' meta-package to point to the 'mariadb10.11' as the Fedora default, instead of the 'mariadb10.5'

Feedback

(05/10/2023) Added clarification regarding other packages from the stack, namely connectors for C, ODBC and JAVA
(05/10/2023) Added clarification that by "mysql names" I mean "'mysql' RPM Provides:"
(05/10/2023) Added explanation for the specific order of the changes in this document
(02/11/2023) I found the task 'Drop builds for i686 architecture' too complex. I still aim to complete it, but no longer in the F40 timeframe.

Benefit to Fedora

Drop builds for i686 architecture

• Maintainer regaining time and energy, when I stop dealing with i686 specific bugs
• Fedora infrastructure regaining the system resources needed to be put into hours long compilations, storage, etc.
• Also 'BuildRequires:' regarding MariaDB and MySQL will get fixed across the packages

Rename package 'community-mysql' to 'mysql' and Stop providing 'mysql' symbols by package 'mariadb'

• Aligning with other distributions
• Recognizing the differences between MariaDB and MySQL that grew over time
• Maintainer regaining some time and energy from downstream work

Drop cross-installation functionality

• Elusive bugs caused by this will vanish, maintainer doesn't need to deal with them
• Recognizing the differences between MariaDB and MySQL that grew over time

Switch to the versioned layout of MariaDB and MySQL packages

• Allows to regain all of the functionality lost after Modularity retirement
  • Helps maintainer to avoid the painful decision which single version of the DB to pack into Fedora
  • Helps with upgrades of the default version - users may choose to upgrade earlier, or later

Introduce MariaDB 10.11 and MySQL 8.1

• New stuff that users wait for, yay \o/ !

Change the default MariaDB version in Fedora from 10.5 to 10.11

• Update to a latest upstream LTS version, awaited by the users

Scope

• Proposal owners:
  • request several repositories to be created
  • may need to go through a package review process several times (unfortunately)

• Other developers: N/A (not needed for this Change)

• Release engineering: N/A (not needed for this Change)

• Policies and guidelines: N/A (not needed for this Change)

• Trademark approval: N/A (not needed for this Change)

• Alignment with Community Initiatives: N/A (not needed for this Change)

Upgrade/compatibility impact

[Task deffered] Drop builds for i686 architecture

• Users relying on i686 build of the database servers don't have any upgrade path

[Finished] Rename package 'community-mysql' to 'mysql' and Stop providing 'mysql' symbols by package 'mariadb'

• The package 'mariadb' will no longer be installed preferably instead of package 'mysql' when doing "dnf install mysql" (and other sub-packages)

[Finished] Drop cross-installation functionality

• Users relying on this atypical setup don't have any clear upgrade path
  • Such cases should only happen on development setups, which is solvable with containers or similar semi-isolation

[Finished] Switch to the versioned layout of MariaDB and MySQL packages

• Nothing I can think of

[Partly Finished | Partly deferred] Introduce MariaDB 10.11 and MySQL 8.1

• Nothing I can think of

[Finished] Change the default MariaDB version in Fedora from 10.5 to 10.11

• https://mariadb.com/kb/en/changes-improvements-in-mariadb-106/
  • TokuDB has been removed
    • As a result, the sources can be stopped to be modified downstream to strip problematically licensed code, and pure upstream tarball can be used instead
  • The utf8 character set (and related collations) is now by default an alias for utf8mb3 rather than the other way around. It can be set to imply utf8mb4 by changing the value of the old_mode system variable
• https://mariadb.com/kb/en/changes-improvements-in-mariadb-107/
  • New UUID data type
• https://mariadb.com/kb/en/changes-improvements-in-mariadb-108/
• https://mariadb.com/kb/en/changes-improvements-in-mariadb-109/
• https://mariadb.com/kb/en/changes-improvements-in-mariadb-1010/
  • --ssl option set as default for mariadb CLI
• https://mariadb.com/kb/en/changes-improvements-in-mariadb-1011/

How To Test

[Task deffered] Drop builds for i686 architecture

• i686 builds doesn't exist anymore

[Finished] Rename package 'community-mysql' to 'mysql' and Stop providing 'mysql' symbols by package 'mariadb'

• The package 'mariadb' will no longer be installed preferably instead of package 'mysql' when doing "dnf install mysql" (and other sub-packages)
• Using names 'mariadb' and 'community-mysql' leads to the same results as before

[Finished] Drop cross-installation functionality

• Cross-installation not allowed anymore

[Finished] Switch to the versioned layout of MariaDB and MySQL packages

• Test updates, upgrades, re-installs, etc.
  • Users should get the same functional results as before with the same names as before, while having differently named packages present on the system

[Partly Finished | Partly deferred] Introduce MariaDB 10.11 and MySQL 8.1

• Test upgrades / downgrades between versions

[Finished] Change the default MariaDB version in Fedora from 10.5 to 10.11

• Test upgrades / downgrades between versions

User Experience

[Task deffered] Drop builds for i686 architecture

• i686 functionality lost (server only)

[Finished] Rename package 'community-mysql' to 'mysql' and Stop providing 'mysql' symbols by package 'mariadb'

• The package 'mariadb' will no longer be installed preferably instead of package 'mysql' when doing "dnf install mysql" (and other sub-packages)
  • Otherwise should not be noticeable to the user

[Finished] Drop cross-installation functionality

• Cross-installation functionality lost

[Finished] Switch to the versioned layout of MariaDB and MySQL packages

• Users should get the same functional results as before with the same names as before, while having differently named packages present on the system
• Functionality lost by Modularity retirement recovered

[Partly Finished | Partly deferred] Introduce MariaDB 10.11 and MySQL 8.1

• New stuff that users wait for, yay \o/ !

[Finished] Change the default MariaDB version in Fedora from 10.5 to 10.11

• Latest upstream LTS version, awaited by the users
  • https://mariadb.com/kb/en/upgrading-from-mariadb-10-5-to-mariadb-10-6/
  • https://mariadb.com/kb/en/upgrading-from-mariadb-10-6-to-mariadb-10-11/

Dependencies

[Task deffered] Only the i686 removal should be blocked by other packages.

All other packages should keep working as they do.

Contingency Plan

[Task deffered] Drop builds for i686 architecture

• I'll try in F41 again

[Finished] Rename package 'community-mysql' to 'mysql' and Stop providing 'mysql' symbols by package 'mariadb'

• Revert

[Finished] Drop cross-installation functionality

• Revert

[Finished] Switch to the versioned layout of MariaDB and MySQL packages

• Revert

[Partly Finished | Partly deferred] Introduce MariaDB 10.11 and MySQL 8.1

• I'll try in F41 again

[Finished] Change the default MariaDB version in Fedora from 10.5 to 10.11

• Revert to the MariaDB 10.5 as the system default, but keep MariaDB 10.11 parallel available in the repository. Fix issues and try in F41 again.

Documentation

• MariaDB Maintenance Policy and Adjusting the MariaDB Server release model
• Introducing MySQL Innovation and Long-Term Support (LTS) versions

Release Notes

Will be added later in the process, according to the changes applied