From Fedora Project Wiki
(Initial draft of change proposal)
 
(→‎Contingency Plan: Remove leftover stuff about /boot as btrfs subvol)
 
(8 intermediate revisions by the same user not shown)
Line 24: Line 24:


== Summary ==
== Summary ==
This change is about fixing the remaining issues in the installer, bootloader setup, and software management stack so that when a user selects a Btrfs-based installation in Fedora, it will set up Btrfs with an optional Btrfs configuration (including /boot as a btrfs subvolume) and be configured such that full OS snapshots are made for each package management transaction, which the correct configuration installed to the boot loader such that the user can boot into the older state if necessary.
This change is about fixing the remaining issues in the installer, bootloader setup, and software management stack so that when a user selects a Btrfs-based installation in Fedora, it will set up Btrfs with an optimal Btrfs configuration (including <code>/boot</code> on Btrfs, potentially as a subvolume) and be configured such that full OS snapshots are made for each package management transaction, which the correct configuration installed to the boot loader such that the user can boot into the older state if necessary.


== Owner ==
== Owner ==
Line 39: Line 39:


== Current status ==
== Current status ==
* Targeted release: [[Releases/29 | Fedora 29]]  
* Targeted release: [[Releases/33 | Fedora 33]]  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Line 64: Line 64:
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->


1. Remove the restriction on <code>/boot</code> as a Btrfs subvolume in Anaconda
1. Adjust the default proposed Btrfs layout by Anaconda to be more optimal (potentially [https://rootco.de/2018-01-19-opensuse-btrfs-subvolumes/ derived from the openSUSE one]).


2. Adjust the default proposed Btrfs layout by Anaconda to be more optional (potentially [https://rootco.de/2018-01-19-opensuse-btrfs-subvolumes/ derived from the openSUSE one]). This may also include moving rpmdb from <code>/var/lib/rpm</code> to <code>/usr/lib/sysimage/rpm</code> as openSUSE did (and Atomic/CoreOS has done) to maintain a simpler subvolume layout. This will also involve fixing the SELinux policy so that <code>/usr/lib/sysimage/rpm</code> is treated the same as <code>/var/lib/rpm</code>. (See [http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html this] and [http://lists.rpm.org/pipermail/rpm-maint/2017-October/006722.html this] for rationale.)
2. Add support for automatically configuring Snapper and the Snapper DNF plugin on Btrfs installs


3. Add support for automatically configuring Snapper and the Snapper DNF plugin on Btrfs installs
3. Enable Snapper's ability to generate boot entries for GRUB 2 with [https://build.opensuse.org/package/view_file/Base:System/grub2/grub2-snapper-plugin.sh the snapper plugin] (or alternatively use Boom with snapper if we're using BLS)
 
4. Enable Snapper's ability to generate boot entries for GRUB 2 with [https://build.opensuse.org/package/view_file/Base:System/grub2/grub2-snapper-plugin.sh the snapper plugin] (or alternatively use Boom with snapper if we're using BLS)




Line 77: Line 75:
1. Include <code>snapper</code> and <code>dnf-plugin-snapper</code> in the live/install media so that it can be configured.
1. Include <code>snapper</code> and <code>dnf-plugin-snapper</code> in the live/install media so that it can be configured.


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
* Release engineering: [https://pagure.io/releng/issue/7591 #7591] (a check of an impact with Release Engineering is needed)
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]:  
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
*** Live Media: Addition of <code>snapper</code> and <code>dnf-plugin-snapper</code> on all images
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
*** Install DVD: Addition of <code>snapper</code> and <code>dnf-plugin-snapper</code> for Anaconda to install when Btrfs is selected
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
*** Netinstall: Addition of <code>snapper</code> and <code>dnf-plugin-snapper</code> for Anaconda to install when Btrfs is selected


* Policies and guidelines: N/A (not needed for this Change)
* Policies and guidelines: N/A (not needed for this Change)
Line 88: Line 86:


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
Most aspects of this Change do not impact upgrades, except for one. If the the rpmdb moves from <code>/var/lib/rpm</code> to <code>/usr/lib/sysimage/rpm</code>, then the major upgrade impact will be that <code>/var/lib/rpm</code> becomes a symlink to <code>/usr/lib/sysimage/rpm</code>. This migration will happen automatically.
N/A (enables new functionality previously unavailable in Fedora)


== How To Test ==
== How To Test ==


If the rpmdb is moved, then an upgrade test is required to verify that package management works post-rpmdb move.
Basic flow:
 
* Set up the system fully on Btrfs
The remaining test case would be to validate that a Btrfs install does the following:
* Ensure snapshots are taken when package management actions happen
* Sets up the system fully on Btrfs
* Verify the snapshot is registered and able to be selected to boot from in the boot menu
* Takes snapshots when package management actions happen
* Verify boot-to-snapshot function lets you boot into that older system
* The snapshot is registered and able to be selected to boot from in the boot menu
* The boot-to-snapshot function lets you boot into that older system


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Line 110: Line 106:
* boom-boot (if we're using BLS)
* boom-boot (if we're using BLS)
* grub2
* grub2
* rpm
* snapper
* snapper
* dnf-plugins-extras
* dnf-plugins-extras
Line 117: Line 112:


<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: Revert allowing <code>/boot</code> as btrfs and disable auto-configuration of snapper in Anaconda. Revert rpmdb move if needed.
* Contingency mechanism: Disable auto-configuration of snapper in Anaconda.
* Contingency deadline: Beta freeze
* Contingency deadline: Beta freeze
* Blocks release? Yes
* Blocks release? Yes
Line 124: Line 119:
== Documentation ==
== Documentation ==
* [https://rootco.de/2018-01-19-opensuse-btrfs-subvolumes/ openSUSE Btrfs layout]
* [https://rootco.de/2018-01-19-opensuse-btrfs-subvolumes/ openSUSE Btrfs layout]
* [http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html Initial proposal by openSUSE to move rpmdb on rpm-ecosystem ML]
* [http://snapper.io Snapper website]
* [http://lists.rpm.org/pipermail/rpm-maint/2017-October/006722.html Suggestion from Panu Matilainen on rpm-ecosystem ML]
* [https://snapper.io Snapper website]
 
 
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)


== Release Notes ==
== Release Notes ==

Latest revision as of 02:29, 29 January 2020


Btrfs with Full System Snapshots

Summary

This change is about fixing the remaining issues in the installer, bootloader setup, and software management stack so that when a user selects a Btrfs-based installation in Fedora, it will set up Btrfs with an optimal Btrfs configuration (including /boot on Btrfs, potentially as a subvolume) and be configured such that full OS snapshots are made for each package management transaction, which the correct configuration installed to the boot loader such that the user can boot into the older state if necessary.

Owner

  • Name: Neal Gompa
  • Email: ngompa13@gmail.com
  • Release notes owner:

Current status

  • Targeted release: Fedora 33
  • Last updated: 2020-01-29
  • Tracker bug: <will be assigned by the Wrangler>

Detailed Description

Btrfs is a next-generation volume-managing filesystem that provides facilities for spanning multiple disks with integrated RAID, cheap file system snapshots due to its copy-on-write nature, and strong integrity though data and metadata checksumming. However, in Fedora, the ability to take advantage of this has been handicapped by issues across Fedora-specific tools at the lower levels of the distribution that kept us from using these features to their fullest extent.

This change encompasses the work to fix these issues in those tools so that if a user wishes to set up Fedora with Btrfs, they get a first-class experience using the filesystem.

Benefit to Fedora

This will enable users on Fedora to finally be able to use a full Btrfs disk configuration without custom work outside of the installer. It will also help with improving the safety of software updates for those using Btrfs by having the system configured to automatically generate full system snapshots and boot entries for those snapshots for rescue/recovery purposes. In addition, this will enable future work on interesting custom alternative approaches for producing appliances (such as appliances built so that they receive updates via btrfs send/receive atomically and re-root) without unusual tooling.

Scope

  • Proposal owners:

1. Adjust the default proposed Btrfs layout by Anaconda to be more optimal (potentially derived from the openSUSE one).

2. Add support for automatically configuring Snapper and the Snapper DNF plugin on Btrfs installs

3. Enable Snapper's ability to generate boot entries for GRUB 2 with the snapper plugin (or alternatively use Boom with snapper if we're using BLS)


  • Other developers:

1. Include snapper and dnf-plugin-snapper in the live/install media so that it can be configured.

  • Release engineering: #7591 (a check of an impact with Release Engineering is needed)
    • List of deliverables:
      • Live Media: Addition of snapper and dnf-plugin-snapper on all images
      • Install DVD: Addition of snapper and dnf-plugin-snapper for Anaconda to install when Btrfs is selected
      • Netinstall: Addition of snapper and dnf-plugin-snapper for Anaconda to install when Btrfs is selected
  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (enables new functionality previously unavailable in Fedora)

How To Test

Basic flow:

  • Set up the system fully on Btrfs
  • Ensure snapshots are taken when package management actions happen
  • Verify the snapshot is registered and able to be selected to boot from in the boot menu
  • Verify boot-to-snapshot function lets you boot into that older system

N/A (not a System Wide Change)

User Experience

When users select Btrfs for the system on install, they will get a Fedora system that will automatically snapshot itself on software management actions and have the ability to select those snapshots from the boot menu to boot into for recovery purposes.

Dependencies

  • anaconda
  • boom-boot (if we're using BLS)
  • grub2
  • snapper
  • dnf-plugins-extras

Contingency Plan

  • Contingency mechanism: Disable auto-configuration of snapper in Anaconda.
  • Contingency deadline: Beta freeze
  • Blocks release? Yes
  • Blocks product? N/A

Documentation

Release Notes