From Fedora Project Wiki
No edit summary
m (More formatting)
 
(16 intermediate revisions by 4 users not shown)
Line 5: Line 5:
* changes to isolated/leaf package without the impact on other packages/rest of the distribution
* changes to isolated/leaf package without the impact on other packages/rest of the distribution
* limited scope changes without the impact on other packages/rest of the distribution
* limited scope changes without the impact on other packages/rest of the distribution
* coordinated effort within SIG with limited impact outside SIG functional area, accepted by the SIG
* coordinated effort within SIG with limited impact outside SIG functional /area, accepted by the SIG


System Wide Changes are:
System Wide Changes are:
Line 20: Line 20:


= Introduce module Obsoletes and EOL =
= Introduce module Obsoletes and EOL =


== Summary ==
== Summary ==
Fix Fedora upgradability issues when upgrading systems with module streams enabled.
Fix Fedora upgradability issues when upgrading systems with module streams enabled.


== Owner ==
== Owner ==
* Name: [[User:dmach| Daniel Mach]]
* Name: [[User:mcurlej| Martin Curlej]]
* Email: dmach@redhat.com
* Email: mcurlej@redhat.com
 


== Current status ==
== Current status ==
[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF36]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 41: Line 38:
[[Category:SystemWideChange]]
[[Category:SystemWideChange]]


* Targeted release: [[Releases/33 | Fedora 33 ]]  
* Targeted release: [[Releases/36 | Fedora 36 ]]  
* Last updated: <!-- this is an automatic macro — you don't need to change this line --> \{{REVISIONYEAR}}-\{{REVISIONMONTH}}-\{{REVISIONDAY2}}  
* Last updated: {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Bugzilla states meaning as usual:
Bugzilla states meaning as usual:
Line 50: Line 47:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* FESCo issue: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/2364 #2364]
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1834844 #1834844]
* Release notes tracker: <will be assigned by the Wrangler>
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/506 #506]
 


== Detailed Description ==
== Detailed Description ==
DNF currently doesn't have sufficient information to perform system upgrades of systems with enabled module streams correctly.
DNF currently doesn't have sufficient information to perform system upgrades of systems with enabled module streams correctly.
To solve upgradability problems, we will add additional modulemd metadata.
To solve upgradability problems, we will add additional modulemd metadata.
This includes information about a stream obsoleting another one or a stream being EOLed.
This includes information about a stream obsoleting another one or a stream being end-of-lived (EOL).


Use Cases:
=== Use Cases ===
* A module (all its streams) is removed from a distribution.
* A module (all its streams) is removed from a distribution.
* A stream has reached its EOL and must no longer be available.
* A stream has reached its EOL and must no longer be available.
Line 69: Line 65:
* A stream has reached its EOL and a user wants to install a new host with the EOLed stream enabled.
* A stream has reached its EOL and a user wants to install a new host with the EOLed stream enabled.


There is going to be a new DNF config option:
=== DNF configuration options===
There are going to be a new DNF config option:
 
* ''module_obsoletes'' - automatically follow stream EOL/Obsoletes; only print a warning if turned off
* ''module_stream_switch'' - allow switching from an already enabled stream


* (name: TBD) - automatically follow stream EOL/Obsoletes; only print a warning if turned off
You need to set both the options to true to obtain a seamless user experience with a plain `dnf upgrade`.


Proposed new modulemd document:
=== Proposed DNF behavior ===
* `dnf system-upgrade` will automatically follow module Obsoletes and EOL, resetting or switching streams accordingly
* Other DNF commands will not follow Obsoletes and EOL automatically; user will have to manually enable an option (see the paragraph above); this should help Rawhide users to handle module upgradability problems because it's a rolling release and they do not run system upgrade.


=== Proposed new modulemd document ===
<pre>
<pre>
document: TBD
document: modulemd-obsoletes
version: 1
version: 1
data:
data:
Line 84: Line 87:
   modified: 2020-03-19T23:26Z
   modified: 2020-03-19T23:26Z


   # A boolean option to cancel all previously specified obsoletes
   # A boolean option to reset all previously specified obsoletes
   # Example: repo 'fedora' has Obsoletes:nodejs:12; we want to bring nodejs:12 back in 'updates'
   # Example: repo 'fedora' has Obsoletes:nodejs:12; we want to bring nodejs:12 back in 'updates'
   <TBD>: <bool>
   # If used, following options will be ignored: eol_date, obsoleted_by
  # OPTIONAL
  reset: <bool>


   # Name of a module that is EOLed
   # A string representing a Name of a module that is EOLed
   # MANDATORY
   # MANDATORY
   module: nodejs
   module: nodejs


   # Stream of a module that is EOLed
   # A string representing a Stream of a module that is EOLed
   # MANDATORY
   # MANDATORY
   stream: 11
   stream: 11


   # Context of a module that is EOLed
   # A string representing a Context of a module that is EOLed
   # If not specified, all contexts get EOLed.
   # If not specified, all contexts get EOLed.
   # NOTE: consider specifying a list of contexts
   # NOTE: consider specifying a list of contexts
   # OPTIONAL
   # OPTIONAL
   context: aabbccddee
   context: aabbccddee
  # Architecture of a stream that is EOLed
  # If not specified, all arches get EOLed.
  # OPTIONAL
  arch: x86_64


   # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ
   # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ
Line 113: Line 113:
   eol_date: 2020-03-19T00:00Z
   eol_date: 2020-03-19T00:00Z


   # A string describing the change, reason etc.
   # A string describing the change, reason, etc.
  # MANDATORY
   message: "Module stream nodejs:11 is no longer supported. It is recommended to switch to nodejs:12"
   message: "Module stream nodejs:11 is no longer supported. It is recommended to switch to nodejs:12"


   # If a stream is not EOLed but Obsoleted, provide additional information about the obsoleting stream:
   # If a stream is not EOLed but Obsoleted, provide details about the obsoleting stream:
  # OPTIONAL
   obsoleted_by:
   obsoleted_by:
     module: nodejs
     module: nodejs
     stream: 12
     stream: 12
</pre>
</pre>


== Benefit to Fedora ==
== Benefit to Fedora ==
Seamless system upgrades of systems with module streams enabled.
Seamless system upgrades of systems with module streams enabled.


== Scope ==
== Scope ==
Line 140: Line 139:
** Follow updated packaging policy. See the "Policies and guidelines" section.
** Follow updated packaging policy. See the "Policies and guidelines" section.


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Release engineering: [https://pagure.io/releng/issue/9359] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Maintain and distribute new module metadata.
Maintain and distribute new module metadata.


Line 148: Line 147:


* Trademark approval: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
This feature solves upgradability problems of systems with module streams enabled.
This feature solves upgradability problems of systems with module streams enabled.


== How To Test ==
== How To Test ==
Line 165: Line 162:
* Verify that the EOLed stream is no longer enabled and does not show up in `dnf module list`
* Verify that the EOLed stream is no longer enabled and does not show up in `dnf module list`


Fedora upgradability:
* Install Fedora 31 or 32
* For each ''$stream'' in available streams
** ``dnf module reset '*'``
** ``dnf module enable $stream``
** ``dnf distro-sync --releasever=33 --assumeno``
** # the command must not throw any errors related to modularity, especially related to the enabled stream


== User Experience ==
== User Experience ==
Line 179: Line 183:
according to newly provided modular metadata.
according to newly provided modular metadata.


== Dependencies ==


== Dependencies ==
* libmodulemd
* libmodulemd
* libdnf
* libdnf
Line 186: Line 190:
* microdnf
* microdnf
* PackageKit
* PackageKit


== Contingency Plan ==
== Contingency Plan ==
Line 192: Line 195:
* Contingency deadline: Beta Freeze
* Contingency deadline: Beta Freeze
* Blocks release? Yes
* Blocks release? Yes


== Documentation ==
== Documentation ==
TBD
* [https://docs.fedoraproject.org/en-US/modularity/building-modules/fedora/module-obsoletes/ Module Obsoletes in Fedora Modularity documentation]
<!-- Is there upstream documentation on this change, or notes you have written yourself? Link to that material here so other interested developers can get involved. -->
* [https://raw.githubusercontent.com/fedora-modularity/libmodulemd/main/yaml_specs/modulemd_obsoletes_v1.yaml modulemd-obsoletes format]


=== Known bugs ===
* [https://bugzilla.redhat.com/show_bug.cgi?id=2053213 DNF does not synchronize installed packages]


== Release Notes ==
== Release Notes ==

Latest revision as of 13:05, 14 February 2022


Introduce module Obsoletes and EOL

Summary

Fix Fedora upgradability issues when upgrading systems with module streams enabled.

Owner

Current status

Detailed Description

DNF currently doesn't have sufficient information to perform system upgrades of systems with enabled module streams correctly. To solve upgradability problems, we will add additional modulemd metadata. This includes information about a stream obsoleting another one or a stream being end-of-lived (EOL).

Use Cases

  • A module (all its streams) is removed from a distribution.
  • A stream has reached its EOL and must no longer be available.
    • The stream packages have no replacements and must be removed from the system (an extension of fedora-obsolete-packages?)
    • The stream packages get replaced with non-modular packages
  • A stream has reached its EOL and is Obsoleted by another stream.
  • A stream has reached its EOL and a user wants to continue using it regardless of EOL/Obsoletes.
  • A stream has reached its EOL and a user wants to install a new host with the EOLed stream enabled.

DNF configuration options

There are going to be a new DNF config option:

  • module_obsoletes - automatically follow stream EOL/Obsoletes; only print a warning if turned off
  • module_stream_switch - allow switching from an already enabled stream

You need to set both the options to true to obtain a seamless user experience with a plain dnf upgrade.

Proposed DNF behavior

  • dnf system-upgrade will automatically follow module Obsoletes and EOL, resetting or switching streams accordingly
  • Other DNF commands will not follow Obsoletes and EOL automatically; user will have to manually enable an option (see the paragraph above); this should help Rawhide users to handle module upgradability problems because it's a rolling release and they do not run system upgrade.

Proposed new modulemd document

document: modulemd-obsoletes
version: 1
data:
  # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ
  # When merging, entries with a newer 'modified' value will override any earlier values.
  # MANDATORY
  modified: 2020-03-19T23:26Z

  # A boolean option to reset all previously specified obsoletes
  # Example: repo 'fedora' has Obsoletes:nodejs:12; we want to bring nodejs:12 back in 'updates'
  # If used, following options will be ignored: eol_date, obsoleted_by
  # OPTIONAL
  reset: <bool>

  # A string representing a Name of a module that is EOLed
  # MANDATORY
  module: nodejs

  # A string representing a Stream of a module that is EOLed
  # MANDATORY
  stream: 11

  # A string representing a Context of a module that is EOLed
  # If not specified, all contexts get EOLed.
  # NOTE: consider specifying a list of contexts
  # OPTIONAL
  context: aabbccddee

  # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ
  # It is strongly recommended to keep HH:MM to 00:00.
  # If not specified, the module is EOLed immediately.
  OPTIONAL
  eol_date: 2020-03-19T00:00Z

  # A string describing the change, reason, etc.
  # MANDATORY
  message: "Module stream nodejs:11 is no longer supported. It is recommended to switch to nodejs:12"

  # If a stream is not EOLed but Obsoleted, provide details about the obsoleting stream:
  # OPTIONAL
  obsoleted_by:
    module: nodejs
    stream: 12

Benefit to Fedora

Seamless system upgrades of systems with module streams enabled.

Scope

  • Proposal owners:
    • Introduce modularity features to:
    • Obsolete a module stream
    • Stream EOL
    • Design a new modulemd documents for Obsoletes and EOL
    • Get the new documents supported by libmodulemd
    • Implement the new functionality in the DNF stack (libdnf, dnf, microdnf) and PackageKit
  • Other developers:
    • Follow updated packaging policy. See the "Policies and guidelines" section.
  • Release engineering: [1] (a check of an impact with Release Engineering is needed)

Maintain and distribute new module metadata.

  • Policies and guidelines:
    • Packaging policy and modularity documents require a change:
      • When a module stream gets removed from a Fedora release, the maintainer of the module stream must provide a modulemd document with Obsoletes or EOL data.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

This feature solves upgradability problems of systems with module streams enabled.

How To Test

Stream Obsoletes:

  • Enable a stream that is not available in the next version of Fedora and is obsoleted by another stream in the metadata
  • Run system upgrade
  • Verify that the obsoleting stream is enabled instead of the obsoleted one

Stream EOL:

  • Enable a stream that is not available in the next version of Fedora and is obsoleted in its module metadata
  • Run system upgrade
  • Verify that the EOLed stream is no longer enabled and does not show up in dnf module list

Fedora upgradability:

  • Install Fedora 31 or 32
  • For each $stream in available streams
    • dnf module reset '*'
    • dnf module enable $stream
    • dnf distro-sync --releasever=33 --assumeno
    • # the command must not throw any errors related to modularity, especially related to the enabled stream

User Experience

Fedora users experience upgradability problems when upgrading to Fedora 32 when they have any module streams enabled.

If a stream no longer exists on the version of Fedora they are upgrading to, DNF used to error out on resolving modular dependencies which was a clear release blocker. To workaround this case, all module streams are reset during the system upgrade. By doing this, modularity users lose information about enabled streams and they need to re-enable them by hand.

This Change aims at removing the upgradability problems and allowing Fedora modularity users to upgrade their systems while keeping their streams enabled, reset or switched (obsoleted) according to newly provided modular metadata.

Dependencies

  • libmodulemd
  • libdnf
  • dnf
  • microdnf
  • PackageKit

Contingency Plan

  • Contingency mechanism: Remove the new metadata from Fedora composes and repositories. Fedora will stay at the current behavior.
  • Contingency deadline: Beta Freeze
  • Blocks release? Yes

Documentation

Known bugs

Release Notes

Modularity supports Obsoleting streams and stream EOL (end-of-life). Enabled streams are no longer reset during the system upgrade. When a stream reaches its EOL, it's reset. When a stream is Obsoleted, it's switched to the obsoleting stream.