From Fedora Project Wiki

(→‎Storage: provide link to overlay allocation status)
(dracut 049 is available in Fedora 30)
(41 intermediate revisions by 2 users not shown)
Line 5: Line 5:
Fedora has developed [[wikipedia:Live CD|Live CD]] [[wikipedia:Live USB|USB]] DVD images for their GNU/Linux operating system. Since the image file systems are stored in the /LiveOS folder of the image, this is the name we'll use to reference the product.
Fedora has developed [[wikipedia:Live CD|Live CD]] [[wikipedia:Live USB|USB]] DVD images for their GNU/Linux operating system. Since the image file systems are stored in the /LiveOS folder of the image, this is the name we'll use to reference the product.


This page shares some critical information about the LiveOS design that helps users take better advantage of their more-limited-than-usual disc resources.
This page shares some critical information about the LiveOS design that helps users take better advantage of their more-limited-than-usual filesystem storage resources.


==References==
==References==


* [http://git.fedorahosted.org/cgit/livecd/tree/tools/livecd-iso-to-disk.sh livecd-iso-to-disk] (Usage instructions are on the first pages.)
* ''[https://github.com/livecd-tools/livecd-tools/blob/master/tools/livecd-iso-to-disk.sh livecd-iso-to-disk]'' Usage instructions are on the first pages (or on the [https://github.com/livecd-tools/livecd-tools/blob/master/docs/livecd-iso-to-disk.pod manual page]).
* [https://github.com/dracutdevs/dracut/blob/master/dracut.cmdline.7.asc#booting-live-images Booting live images] - from the ''dracut'' initramfs command line manual.
* [[FedoraLiveCD]]
* [[FedoraLiveCD]]
* [[How to create and use a Live CD]]
 
* [[How to create and use Live USB]]
==Multi Live Image boot installations==
With large capacity USB/SD disc drives readily available, it is convenient to install multiple images on one device.
 
The {{code|livecd-iso-to-disk --multi}} option allows one to install more than one LiveOS image on a single device. Version 24.2 or greater of {{Package|livecd-tools}} will offer to configure the device bootloader to give a Multi Live Image Boot Menu for the device if the target device contains an existing LiveOS image.
 
The first live image is installed by default in the device's {{code|/LiveOS}} directory.  Subsequent images are installed in sister directories with randomly assigned, 4 character names, such as {{code|/sYBm}}.  The target directory may be optionally specified by the [--livedir <dir>] option of ''livecd-iso-to-disk''.
 
The ''livecd-iso-to-disk'' installation script allows one to install from and to a booted LiveOS device and to select any of the installed images as the source for a new installation by using its directory path as the source parameter, for example, {{code|/run/initramfs/live/sYBm}}.  The booted LiveOS image can be sourced by the source shortcut 'live' and the booted LiveOS device can be targeted by the target shortcut 'live'.


==Storage==
==Storage==


When a Live CD or Live DVD (a LiveOS image on read-only disc media) is booted, '''temporary storage''' is prepared for the system in RAM on each boot by [http://git.kernel.org/?p=boot/dracut/dracut.git;a=blob;f=modules.d/90dmsquash-live/dmsquash-live-root.sh;hb=HEAD /sbin/dmsquash-live-root] in initrd0, the initial ram disk filesystem. An in-memory, copy-on-write, system overlay is used (see [[#File Systems|File Systems]] below).  The boot script prepares up to 0.5 GiB of RAM space in a [[wikipedia:Sparse file|sparse file]] by default.
When a Live CD or Live DVD (a LiveOS image on read-only disc media) is booted, '''temporary storage''' is prepared for the system in RAM on each boot by [http://git.kernel.org/?p=boot/dracut/dracut.git;a=blob;f=modules.d/90dmsquash-live/dmsquash-live-root.sh;hb=HEAD /sbin/dmsquash-live-root] in initrd.img, the initial ramdisk filesystem. By default, a 32 GiB, in-memory, copy-on-write, system overlay in a [[wikipedia:Sparse file|sparse file]] is prepared (see [[#File Systems|File Systems]] below).  The {{Code|rd.live.overlay.size}} kernel command line option may be used to set a different, temporary, overlay size.  Since the temporary overlay is a sparse file in a tmpfs, a large size, even larger than available memory, may be specified and only what is needed will be allocated as needed.
 
When the {{Code|rd.live.overlay}} kernel command line option is provided on boot, such as with a Live USB device, the ''dmsquash-live-root.sh'' script will search for a persistent overlay file to use for storage of root filesystem changes. See [https://github.com/dracutdevs/dracut/blob/master/dracut.cmdline.7.asc#booting-live-images Booting live images] in the ''dracut'' package.
 
LiveOS device users should take note of their overlay status.  See [[#Overlay allocation status]], below, for a way to monitor overlay consumption to anticipate and avoid overlay Overflow or Invalidation.
 
With Fedora 24 (kernel 4.3+), if the overlay storage space is filled, the overlay will enter a 'Overflow' state and the root file system will continue to operate in a read-only mode.  There will not be an explicit warning or signal when this happens, but applications may begin reporting errors due to the restriction.  See the instructions at [[LiveOS image/overlay#Overlay recovery]] for how one may then enlarge and attempt to repair and merge the overlay.


{{admon/important|Critical limitation|The overlay file of LiveOS holds new and changed file content and metadata in a pool of sectors allocated as needed in a fixed size file. Should the overlay storage space, whether temporary or persistent, be totally consumed, the filesystem will be flagged 'Invalid' and, if that filesystem is the booted rootfs, it will crash with Input/output or Bus errors. If such a crash does occur while using temporary storage space for the overlay, a simple reboot will rectify the situation.  With persistent storage the situation is more dire and will require appending "rd.live.overlay.reset" (formerly "reset_overlay") to the kernel command line on boot-up.  This will reset the state to its initial boot (though it won't reset a persistent home, of course).  In either case, achieving a reboot will require a hard reset, since attempting a software initiated reboot on the Invalid filesystem will fail with more Input/output or Bus errors. See [[#Overlay allocation status]] for a way to monitor overlay consumption.}}
:'''Note:''' In systems built before Fedora 24, should the overlay storage space, whether temporary or persistent, be totally consumed, the filesystem will be flagged 'Invalid' and the system will crash with Input/output or Bus errors. Achieving a reboot will require a hard reset since attempting a software initiated reboot on the Invalid filesystem will fail with more Input/output or Bus errors. If such a crash occurs while using temporary storage space for the overlay, a simple reboot will rectify the situation.  With persistent storage, the situation is more dire.  One can attempt to enlarge and repair the overlay using instructions at [[LiveOS image/overlay#Overlay recovery]]. Or one can vacate all changes to the root file system by appending the option {{Code|rd.live.overlay.reset}} (formerly, {{Code|reset_overlay}}) to the kernel command line on boot-up.  A persistent home filesystem, if used, will be unaffected by an overlay reset.


===Operating system file systems===
===Structure of LiveOS file systems===
Live Operating System, '''LiveOS''', file systems are found within disk image '''.img''' files.
Live Operating System, '''LiveOS''', file systems are found within disk image '''.img''' files.


If one mounts a Fedora-Live-Desktop.iso file or Live CD, the resulting block device file system will list 3 folders:
====Live.iso====
If one mounts a Fedora-Workstation-Live.iso file or Live CD, the file system will list folders such as the following:
  /EFI
  /EFI
/images
  /isolinux
  /isolinux
  /LiveOS
  /LiveOS
    |- livecd-iso-to-disk
    |- osmin.img
     |- squashfs.img
     |- squashfs.img
The LiveOS folder contains 3 files:
   
* livecd-iso-to-disk is a Bash script used for loading (installing) the LiveOS image onto a USB or SD disk.
  (Systems before Fedora 24 also had these files
* osmin.img is a minimized OS image which is used to aid traditional installation from a LiveOS image source to a hard disk.
    |- livecd-iso-to-disk a Bash script for installing the LiveOS image onto a USB device
* squashfs.img is a [[wikipedia:SquashFS|SquashFS]] compressed read-only file system with the Fedora operating system root file system inside another LiveOS folder containing an ext3fs.img file.
    |- osmin.img a minimized OS image formerly used to aid installation to a hard disk
Mounting the squashfs.img file and listing its block device file system will show this hierarchy:
 
====squashfs.img====
The squashfs.img is a [[wikipedia:SquashFS|SquashFS]] compressed, read-only, file system holding the Fedora operating system root file system inside another /LiveOS folder containing a rootfs.img file.
Mounting the squashfs.img file and listing its file system will show this hierarchy:
  /LiveOS
  /LiveOS
     |- ext3fs.img    (This may be named rootfs.img and contains an ext2, ext3, or ext4 file system type.)
     |- rootfs.img    (This contains a filesystem of type ext4. Before F-24 it was named ext3fs.img.)
Mounting the ext3fs.img file will finally reveal the Fedora operating system root file system.
Mounting the rootfs.img file will finally reveal the Fedora operating system root file system.
 
====Live.iso + squashfs.img====
Here is a summary of the whole structure:
Here is a summary of the whole structure:
  '''Fedora-Live-Desktop.iso'''
  '''Fedora-Workstation-Live.iso'''
   ''!(mount)''
   ''!(mount)''
   /EFI
   /EFI
  /image
   /isolinux
   /isolinux
   /LiveOS
   /LiveOS
      |- livecd-iso-to-disk
      |- osmin.img
       |- '''squashfs.img'''
       |- '''squashfs.img'''
             ''!(mount)''
             ''!(mount)''
             /LiveOS
             /LiveOS
                 |- '''ext3fs.img'''
                 |- '''rootfs.img'''
                     ''!(mount)''        
                     ''!(mount)''
                     /bin -> usr/bin
                     /bin -> usr/bin
                     /boot
                     /boot
Line 58: Line 74:
                     /home
                     /home
                     /lib -> usr/lib
                     /lib -> usr/lib
                    /lib64 -> usr/lib64
                     /lost+found
                     /lost+found
                     /media
                     /media
Line 72: Line 89:
                     /var
                     /var


When the LiveOS image is loaded onto a USB or SD disk, the {{Code|isolinux}} folder is copied into a {{Code|syslinux}} folder.
====Installed LiveUSB/SD device====
When the LiveOS image is loaded onto a USB or SD disk, the {{Code|/isolinux}} folder is copied into a {{Code|/syslinux}} folder.


If persistence is requested with the {{Code|--overlay-size-mb NNN}} option, a Device-mapper overlay file for the root file system is created in {{Code|LiveOS}}.
If persistence is requested with the {{Code|--overlay-size-mb NNN}} option, a Device-mapper overlay file for the root file system is created in {{Code|/LiveOS}}.


If a separate home file system is requested with the {{Code|--home-size-mb NNN}} option, an ext4 formatted {{Code|'''home.img'''}} file system is created.
If a separate home file system is requested with the {{Code|--home-size-mb NNN}} option, an ext4 formatted {{Code|'''home.img'''}} file system is created.


The structure is then as follows:
The structure is then as follows:
  '''Fedora-Live-Desktop USB/SD device'''
  '''Fedora-Workstation-Live USB/SD device'''
   ''!(mounted on /run/initramfs/live)'' (Before Fedora 17 the mount point is /mnt/live in the root file system.)
   ''!(mounted on /run/initramfs/live)'' (Before Fedora 17 the mount point is /mnt/live in the root file system.)
   /syslinux
   /syslinux
Line 93: Line 111:
       |- livecd-iso-to-disk
       |- livecd-iso-to-disk
       |- '''overlay-NAME-XXXX-XXXX'''    (Where NAME is the device partition name and XXXX are hex numerals.)
       |- '''overlay-NAME-XXXX-XXXX'''    (Where NAME is the device partition name and XXXX are hex numerals.)
      |- osmin.img
       |- '''squashfs.img'''
       |- '''squashfs.img'''
             ''!(mount)''                (At boot up by the /usr/sbin/dmsquash-live-root.sh script in the initramfs.)
             ''!(mount)''                (At boot up by the /usr/sbin/dmsquash-live-root.sh script in the initramfs.)
             /LiveOS
             /LiveOS
                 |- '''ext3fs.img'''
                 |- '''rootfs.img'''
                     ''!(mounted on '/')''    (This occurs during boot up by the dmsquash-live-root.sh script.)
                     ''!(mounted on '/')''    (This occurs during boot up by the dmsquash-live-root.sh script.)
                     /bin -> usr/bin
                     /bin -> usr/bin
Line 105: Line 122:
                     /home                (If there is a '''home.img''', then this is its mount point directory.)
                     /home                (If there is a '''home.img''', then this is its mount point directory.)


If the {{Code|--skipcompress}} option is used during loading with {{Code|livecd-iso-to-disk}}, the '''squashfs.img''' compression layer will be skipped.  The structure is then as follows:
====Uncompressed LiveUSB/SD installation====
  '''Fedora-Live-Desktop USB/SD device'''
If the {{Code|--skipcompress}} option is used during loading with ''livecd-iso-to-disk'', the '''squashfs.img''' compressed image will be expanded.  The structure is then as follows:
  '''Fedora-Workstation-Live USB/SD device'''
   ''!(mounted on /run/initramfs/live)''
   ''!(mounted on /run/initramfs/live)''
   /syslinux
   /syslinux
Line 116: Line 134:
       |- livecd-iso-to-disk
       |- livecd-iso-to-disk
       |- '''overlay-NAME-XXXX-XXXX'''
       |- '''overlay-NAME-XXXX-XXXX'''
      |- osmin.img
       |- '''rootfs.img'''
       |- '''ext3fs.img'''
             ''!(mounted on '/')''           
             ''!(mounted on '/')''           
             /bin -> usr/bin
             /bin -> usr/bin
Line 125: Line 142:
             /home
             /home
             ...
             ...
====Flattened squashfs.img====
For OverlayFS mount overlays, the '''squashfs.img''' structure may also be a direct compression of the root filesystem:
(Booting this structure is supported in ''dracut'' version 049 available in Fedora 30.)
squashfs.img          |  SquashFS from LiveCD .iso
    !(mount)
    /bin              |  Live filesystem
    /boot              |
    /dev              |
    ...                |


===Persistent storage===
===Persistent storage===
Line 130: Line 158:
The Fedora LiveOS system allows for persistent storage in 3 locations:
The Fedora LiveOS system allows for persistent storage in 3 locations:


# An all-purpose, persistent overlay-based file space that saves all updates and changes to the root LiveOS filesystem. But, this storage '''space is limited''' by its allocate-once, fixed-sized design, and must be used with caution (see [[#File Systems|File Systems]] below).
# An all-purpose, persistent overlay-based file space that saves all updates and changes to the root LiveOS filesystem. This storage '''space is limited''' by its allocate-once, fixed-sized design, and deserves some caution (see [[#File Systems|File Systems]] below).
# Persistent home folder - a re-writable, re-sizable (with difficulty), uncompressed, optionally-encryptable, file space for anything that goes in the user's /home/ folder.  A persistent home folder is an option that may be selected at the time of installation of the LiveOS image (although with some effort, one could manually create a {{Code|home.img}} filesystem in {{Code|/LiveOS/}} and move the {{Code|/home/}} folder to it). This installation option is only available through the script, '[[livecd-iso-to-disk]]'. The Windows and Fedora 'Live USB Creator' installers do not provide this option at present.
# Persistent home filesystem - a re-writable, re-sizable (with difficulty), uncompressed, optionally-encryptable, file space for anything that goes in the users /home/ folder.  A persistent home filesystem is an option that may be selected at the time of installation of the LiveOS image (although with some effort, one could manually create a {{Code|home.img}} filesystem in {{Code|/LiveOS/}} and move the {{Code|/home/}} folder to it). This installation option is only available through the script, ''[[livecd-iso-to-disk]]''. The Windows and Fedora ''Fedora Media Writer/Live USB Creator'' installers do not provide this option at present.
# Host device file space - this is the USB stick or SD card file system that is outside of the LiveOS file tree, but which is accessible through the {{Code|/run/initramfs/live}} or {{Code|/mnt/live}} (before Fedora 17) mount point of a booted LiveOS installation.  There, one finds the boot configuration files and anything else one had on the device before loading the LiveOS image.  One may save files here without consuming the other, limited file spaces. (This file space is limited only by the device capacity).
# Host device file space - this is the underlying USB stick or SD card file system that is outside of the LiveOS file tree, but which is accessible through the {{Code|/run/initramfs/live}} or {{Code|/mnt/live}} (before Fedora 17) mount point of a booted LiveOS installation.  There, one finds the boot configuration files and anything else one had on the device before loading the LiveOS image.  One may save files here without consuming the other, limited file spaces. (This file space is limited only by the device capacity).
  /run/initramfs/live
  /run/initramfs/live
                   /LiveOS
                   /LiveOS
Line 138: Line 166:
                       |- livecd-iso-to-disk
                       |- livecd-iso-to-disk
                       |- overlay-NAME-XXXX-XXXX
                       |- overlay-NAME-XXXX-XXXX
                      |- osmin.img
                       |- squashfs.img
                       |- squashfs.img
                   /syslinux
                   /syslinux


The all-purpose, persistent overlay is needed for operating system changes and updates.
The all-purpose, persistent overlay is needed for keeping operating system changes and updates between boots.


The file systems are prepared on each boot by the [http://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-live-base.ks#n56 /etc/rc.d/init.d/livesys script].
The file systems are registered on each boot by the [http://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-live-base.ks#n56 /etc/rc.d/init.d/livesys script].


===Home filesystem===
===Home filesystem===
One may find many advantages to installing the LiveOS, with a persistent, home folder (using the --home-size-mb NN --delete-home options), which will hold all the user files and documents one wishes and, perhaps later, throw away&mdash;all without consuming the write-once overlay, which can be consumed very quickly (and overlay file space is not normally reusable).
One may find many advantages to installing the LiveOS, with a persistent, home filesystem (using the {{Code|--home-size-mb NNN}} option), which will hold all the user files and documents one wishes and, perhaps later, throw away&mdash;all without consuming storage space in the overlay, which can be depleted very quickly.


===Device filesystem===
===Device filesystem===
Line 153: Line 180:


===Installation options===
===Installation options===
Fedora 17 Live Desktop may be installed on a 1 GB USB device using the following options with '''[[livecd-iso-to-disk]]''' (on a single, terminal or console command line, even though the wiki may wrap the following text to accommodate your browser window size):
Fedora 24 Live Workstation may be installed on a 2 GB USB device using the following options with '''''[[livecd-iso-to-disk]]''''' (on a single, terminal or console command line):


:{{Code|./livecd-iso-to-disk --reset-mbr --overlay-size-mb '''200''' --home-size-mb '''140''' --delete-home --unencrypted-home /path/to/source/iso/or/device /dev/sd'''?'''1}}
:{{Code|./livecd-iso-to-disk --reset-mbr --overlay-size-mb '''300''' --home-size-mb '''200''' --unencrypted-home /path/to/source/iso/or/device /dev/sd'''?'''1}}


:: where '{{Code|'''?'''}}' in the final parameter represents the target bootable device node, such as {{Code|sd'''b'''1}} or {{Code|sd'''c'''1}}, etc.
:: where '{{Code|'''?'''}}' in the final parameter represents the target bootable device node, such as {{Code|sd'''b'''1}} or {{Code|sd'''c'''1}}, etc.


The above configuration would allow space for the home folder, the operating system, and a little on the device root.
The above configuration would allow space for the home folder, the operating system, and a minimal amount on the device root.


But with a larger capacity device, one may allocate the resources to suit the anticipated use, as described above.
But with a larger capacity device, one may allocate the resources to suit the anticipated use, as described above.


==File Systems==
==File Systems==
{{admon/note|Take note:|It is important to realize that because the LiveOS root filesystem normally resides in a compressed SquashFS image, the standard estimates of free disc space, such as those reported by {{Code|df}} and graphical file managers, will report '''"apparent" file space''' (as if the filesystem were not compressed), thus, such estimates are usually '''vast overestimates of the actual free space''' physically available to the root filesystem.
{{admon/note|Take note:|It is important to realize that because the LiveOS root filesystem normally resides in a compressed SquashFS image, the standard estimates of free disc space, such as those reported by {{Code|df}} and graphical file managers, will report '''"apparent" file space''' (as if the filesystem were not compressed), thus, such estimates are usually '''vast overestimates of the actual free space''' physically available to the root filesystem.  The embedded rootfs.img filesystem partition is often sized to just fit the distributed operating system at the time of its creation.
: For example, the Fedora-18-x86_64-Live-Desktop.iso contains an apparent 4 GiB ext4 root filesystem that has been sparsed and compressed into under 900 MiB of actual disc space in the .iso file (or on a LiveOS installed device file system).  {{Code|df -h}} on an untouched mounting of the root filesystem will report that 3.1G is used and 695M is available space out of a total size of 3.9G.
: For example, the Fedora-Workstation-Live-x86_64-28-1.1.iso contains an apparent 6.4 GiB ext4 root filesystem that has been sparsed and compressed into under 1.6 GiB of actual disc space in the .iso file (or on a LiveOS installed device file system).  {{Code|df -h}} on an untouched mounting of the root filesystem will report that 4.9G is used and 1.5G is available space out of a total size of 6.4G.
The actually available and allocated physical disc space for the root filesystem is determined by the size of the ''device-mapper'' overlay file and can only be queried by the {{Code|dmsetup status}} command (see below).
The actually available and allocated physical disc space for the root filesystem is determined by the size of the ''device-mapper'' overlay file and can only be queried by the {{Code|dmsetup status}} command (see below).
: For example, one may add, and then delete, a large file in the root filesystem and see (the apparent virtual) used space go up, and then down, when checking with {{Code|df}}, but the {{Code|dmsetup status live-rw}} allocation report will only ever show an increase&mdash;even though allocated overlay sectors are available for reuse if an added file has been deleted (see discussion below).}}
: For example, one may add, and then delete, a large file in the root filesystem and see (the apparent virtual) used space go up, and then down, when checking with {{Code|df}}, but the {{Code|dmsetup status live-rw}} allocation report will only ever show an increase&mdash;even though allocated overlay sectors are available for reuse if an added file has been deleted (see discussion below).}}
The Fedora LiveOS uses the [[wikipedia:Device mapper|Device-mapper]] service of the Linux kernel to manage the file stores on the device.  This is the same service that is used by [[wikipedia:Logical Volume Manager (Linux)|Logical Volume Manager]] to provide disc partition services.
Fedora LiveOS uses the [[wikipedia:Device mapper|Device-mapper]] service of the Linux kernel to manage the file stores on the device.  This is the same service that is used by [[wikipedia:Logical Volume Manager (Linux)|Logical Volume Manager]] to provide disc partition services.


===Overlay allocation status===
===Overlay allocation status===
One critical limitation, mentioned above, is that the LiveOS persistent overlay is a allocate-once, fixed size, file space.  This is related to its use of ''device mapper'' snapshots to combine a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a [[wikipedia:Copy-on-write|Copy-on-write]] service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks.<ref>http://people.gnome.org/~markmc/code/merge-dm-snapshot.c</ref><ref>http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=drivers/md/dm-snap-persistent.c;hb=HEAD</ref> Any changes to the original operating system files are stored as differences from the base. As such, "deletions" of these original files are saved as additional difference references, and the originals are hidden.  With this mechanism, physical storage space is consumed in the overlay file space (nothing can be recovered from the read-only original). The apparent free space in the virtual root filesystem as reported by the {{Code|df}} command will increase, but this additional space is not physically available; however, when a new or changed file is deleted, the new or changed sectors only are available for reuse, but, unfortunately, the count of allocated sectors does not change, so one cannot be certain of the availability of free physical space after arbitrary changes to the root filesystem.
One critical limitation, mentioned above, is that the LiveOS persistent overlay is an allocate-once, fixed size, file space.  This is related to its use of ''device mapper'' snapshots to combine a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a [[wikipedia:Copy-on-write|Copy-on-write]] service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks.<ref>http://people.gnome.org/~markmc/code/merge-dm-snapshot.c</ref><ref>http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=drivers/md/dm-snap-persistent.c;hb=HEAD</ref> Only changed sectors in files are stored in the overlay. "Deletions" of any original files in the base filesystem are recorded by changes in the index metadata so that the original file blocks are hidden.  With this mechanism, no physical storage space of the read-only, base filesystem can be reused. The apparent free space in the virtual root filesystem as reported by the {{Code|df}} command will increase, but this additional space is not physically available; however, when a new or changed file is deleted, the new or changed sectors only (in the overlay) are available for reuse, but, unfortunately, the count of allocated sectors does not change, so one cannot be certain of the availability of free physical space after arbitrary changes to the root filesystem.
 
With Fedora 24 (kernel 4.3+), if the overlay storage space is filled, the overlay will enter an 'Overflow' state and the root file system will continue to operate in a read-only mode.  See the instructions at [[LiveOS image/overlay]] for how one may then enlarge and attempt to repair and merge the overlay.


The status of the space allocated for persistent storage in the snapshot overlay file may be tracked with the ''device mapper'' {{Code|dmsetup status}} report.
The status of the space allocated for persistent storage in the snapshot overlay file may be tracked with the ''device mapper'' {{Code|dmsetup status}} report.
Line 181: Line 210:
where the fraction after {{command|snapshot}} is the number of 512-byte sectors allocated in the overlay over the total number available in the overlay. (The final number is the metadata sectors allocated, and the number before {{command|snapshot}} is the apparent size of the virtual filesystem.)
where the fraction after {{command|snapshot}} is the number of 512-byte sectors allocated in the overlay over the total number available in the overlay. (The final number is the metadata sectors allocated, and the number before {{command|snapshot}} is the apparent size of the virtual filesystem.)


Unfortunately, there is no process written to monitor the overlay allocation level and provide a graceful warning of its imminent exhaustion and the subsequent failure of the root filesystem. So where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised.
Where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised.  If memory or storage resources are available, one may take advantage of the {{Code|rd.writable.fsimg}} or {{Code|rd.live.overlay.size}} options in the [http://git.kernel.org/?p=boot/dracut/dracut.git;a=blob;f=modules.d/90dmsquash-live/dmsquash-live-root.sh;hb=HEAD /sbin/dmsquash-live-root] startup script to mitigate this risk of failure.
: (Last-ditch [[#Overlay recovery|recovery of a persistent overlay]] for a failed root filesystem may be attempted from a separate boot of a working system.)
: (For systems built before Fedora 24 that exhaust the overlay to an 'Invalid' state, last-ditch [[LiveOS image/overlay#Overlay recovery|recovery of a persistent overlay]] for the failed root filesystem may be attempted from a separate boot of a working system.)


Several developments in the LiveOS design may be considered to maximize the endurance of the LiveOS image.
==Kernel updates for images with a persistent overlay==
If you have sufficient overlay space to accommodate a kernel update on a Live USB installation, the kernel and initramfs will be installed to the /boot directory. To put these into service they must be moved to the /syslinux directory of the installation partition. This is accessible from the running Live USB filesystem at the /run/initramfs/live mount point. The new initramfs (such as initramfs-4.17.7-200.fc28.x86_64.img) and kernel (such as vmlinuz-4.17.7-200.fc28.x86_64) should be moved to replace the /run/initramfs/live/syslinux/initrd.img and /run/initramfs/live/syslinux/vmlinuz files, respectively.


===Avoid persistent overlay consumption===
Note: dracut no longer includes the dmsquash-live module by default. Starting with Fedora 19, dracut defaults to the hostonly="yes" option, which precludes the dmsquash-live module. So, one can add a dracut config file, as root, before updating the kernel:
<pre>
echo 'hostonly="no"
add_dracutmodules+=" dmsquash-live "' > /etc/dracut.conf.d/01-liveos.conf
</pre>


* Use a persistent {{Code|home.img}} file system to mount on the {{Code|/home}} folder (discussed above).
For example, the following commands will move a new kernel and initramfs files to the device's /syslinux directory:
*: This is a very effective method to avoid consumption and will also make the user files more available for sharing and backup purposes.
* Move the root user's home to {{Code|/home/root}} in a persistent {{Code|home.img}}.
*: This would avoid some consumption, but would compromise the root user account if there was a boot problem and the /home folder was not mounted.
* Mount more folders onto temporary, in-memory filesystems (like /var/cache/yum, /var/tmp, & /tmp are now).
** /var/lib/NetworkManager (holds a timestamps file that is deleted and refreshed quite often, every 5 minutes)
** /var/log/audit (holds the SELinux audit.log that records a great many file accesses.)
** /var/spool/abrt (holds often large, error reports and core dumps).  Users could be advised to act on any abrt reports in the current session or copy the reports to other permanent storage, such as in /home/ or external storage.
** there may be other good candidates...
 
===Merge overlay into new image===
 
A new root filesystem image can be constructed by using some Device-mapper tools.
 
If one has sufficient free disk space available, typically on an attached hard drive, one can copy (and uncompress) the current filesystem to a working folder. (This may require, for example, 4.3 GiB of free space for Fedora 17 Live Desktop.)
 
Device-mapper's mirror target allows one to create a 'mirror' image of the LiveOS root filesystem (the snapshot based on the SquashFS and overlay device).  The mirror image can be then be recompressed, and if one has an additional 660 MiB or more of free space (depending on how much software or other root filesystem files have been added) on your LiveOS device filesystem (the USB/SD card filesystem), one can delete the old SquashFS image and replace it with the recompressed version.
 
The snapshot overlay can then be reset before rebooting the LiveOS device.
 
The following Bash script demonstrates the process, where the {{Code|$1}} parameter is a mount point directory for a disk partition with sufficient storage capacity (a LiveOS folder will be created there for holding the mirrored filesystem), and {{Code|$2}} is the LiveOS device node name (such as, {{Code|/run/initramfs/livedev}}, for the currently booted LiveOS image, or {{Code|/dev/sdc1}}, for example, for an attached LiveOS device). Options xtrace and verbose have been set to aid debugging.
{{admon/warning|Test and practice the following script!|This procedure will delete the LiveOS filesystem before replacing it with a recompressed merged version. Be sure to verify that you will have sufficient space for the replacement before running the full script.  (You could comment out the deletion, and squash into a directory on a large working drive to determine the new filesystem size.)}}
<pre>
<pre>
#!/bin/bash -xv
bootpath=run/initramfs/live/syslinux
#
new=4.17.7-200.fc28.x86_64
# Merge a LiveOS snapshot overlay into a new root filesystem, recompress it
# into a SquashFS image (if the source was so packaged), replace the source
# with the refresh, and reset the overlay.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Library General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
 
oldLC=$LC_NUMERIC
LC_NUMERIC="en_US.UTF-8"
export PATH=/usr/sbin:$PATH
export \
  PS4='+(${SHLVL}:${LINENO}:${BASH_SOURCE}:${EUID}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'
set -o errexit
set -o pipefail
 
t0=$(date +%s)
 
cleanup () {
    umount -l $SRCMNT
    rmdir $SRCMNT
    dt=$(($(date +%s)-t0))
    h=$((dt/3600))
    m=$((dt%3600/60))
    s=$((dt%60))
    printf '\nTotal time elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s
    [[ $1 == 0 ]] && exit 0 || exit 1
}
 
if [[ ! -f /usr/sbin/mksquashfs ]]; then
    echo "Please install squashfs-tools (yum install squashfs-tools)";
    exit
fi
 
TMPDIR=$1
[[ ! -d $TMPDIR/LiveOS ]] && mkdir $TMPDIR/LiveOS 2>/dev/null
 
# Mount the source device and SquashFS.
mkdir -p /run/media #Doesn't exist in fedora 20 livecd
SRCMNT=$(mktemp -d /run/media/XXXXXX)
mount $2 $SRCMNT
 
if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then
    SQUASHMNT=$(mktemp -d /run/media/XXXXXX)
    mount $SRCMNT/LiveOS/squashfs.img $SQUASHMNT --read-only
    ORIGFS=$SQUASHMNT/LiveOS/ext3fs.img
    [[ ! -f $ORIGFS ]] && ORIGFS=$SQUASHMNT/LiveOS/rootfs.img
else
    ORIGFS=$SRCMNT/LiveOS/ext3fs.img
    [[ ! -f $ORIGFS ]] && ORIGFS=$SRCMNT/LiveOS/rootfs.img
fi
 
OVERLAY=/LiveOS/$(basename $SRCMNT/LiveOS/overlay-*)
if [[ -f $SRCMNT$OVERLAY ]]; then
    OVDEV=$(losetup -f --show $SRCMNT$OVERLAY --read-only)
else
    printf '\n  No persistent overlay for the filesystem was found.
    Exiting...\n'
    cleanup 1
fi
 
# >= Fedora 17
LiveOSMNT=/run/initramfs/live
# < Fedora 17
[[ -d /mnt/live/LiveOS ]] && LiveOSMNT=/mnt/live
 
# If we are processing a booted LiveOS device's root filesystem, then
if [[ $(mountpoint -x $2) == $(mountpoint -dq $LiveOSMNT) ]]; then
    # Remove live-osimg-min; free its loop devices.
    if [[ -e $SRCMNT/LiveOS/osmin.img ]]; then
        dmsetup remove live-osimg-min || :
        losetup -d /dev/loop1 || :
        losetup -d /dev/loop0 || :
    fi
else
    LiveOSMNT=''
fi
# Remove expiring osmin.img
rm $SRCMNT/LiveOS/osmin.img || :
 
# Prepare temporary devices for the Device-mapper mirror target.
FSTYPE=$(blkid -s TYPE -o value $ORIGFS || :)
FSLABEL=$(blkid -s LABEL -o value $ORIGFS || :)
stinfo=($(stat -c '%B %s %o' $ORIGFS || :))
BLOCKSZ=${stinfo[0]}
BLOCKS=$((${stinfo[1]}/$BLOCKSZ))
IOBLKSZ=${stinfo[2]}
 
NEWFS=$TMPDIR/LiveOS/rootfs.img
dd if=/dev/null of=$NEWFS count=1 bs=$BLOCKSZ seek=$BLOCKS
mkfs.$FSTYPE -F -L $FSLABEL -m 1 -b $IOBLKSZ $NEWFS
tune2fs -c0 -i0 -Odir_index -ouser_xattr,acl $NEWFS
NEWFSDEV=$(losetup -f --show $NEWFS)
 
roORIGFSDEV=$(losetup -f --show $ORIGFS --read-only)
 
table="0 $BLOCKS snapshot $roORIGFSDEV $OVDEV P 8"
dmsetup create ori --readonly --table "$table"
 
# Invoke mirror target device.
table="0 $BLOCKS mirror core 2 32 sync 2 /dev/mapper/ori 0 $NEWFSDEV 0"
dmsetup create mir --table "$table"
 
set +xv
# Wait for mirror completion.
while state=$(dmsetup status mir)
state=${state#*mirror 2 * * }
alloc=${state%/*}
size=${state#*/}
size=${size%% *}
[[ $alloc != $size ]]; do
    percent=$(dc <<< "8k 10 0.05 100 $alloc $size /*+*p")
    percent=$(dc <<< "1k $percent 10 /p")
    printf '\r  Mirroring %5.1f %% complete.  ' $percent
    printf '\b|'
    sleep 0.5
    printf '\b/'
    sleep 0.5
    printf '\b-'
    sleep 0.5
    printf '\b\'
    sleep 0.5
done
printf '\r  Mirroring 100 %% complete.    \n'
set -xv
 
# Check the new filesystem.
e2fsck -f -y $NEWFS || e2fsck -f -y $NEWFS
 
# Clean up.
dmsetup remove mir ori
sync
 
dt=$(($(date +%s)-t0))
h=$((dt/3600))
m=$((dt%3600/60))
s=$((dt%60))
printf '\nTime elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s
 
# Replace SquashFS or uncompressed root filesystem image.
if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then
    umount -l $SQUASHMNT
    rm $SRCMNT/LiveOS/squashfs.img
    mksquashfs $TMPDIR/LiveOS $SRCMNT/LiveOS/squashfs.img -comp xz -keep-as-directory
else
    if [[ -x /usr/bin/rsync ]]; then
        rsync --inplace --progress $NEWFS $SRCMNT/LiveOS
    else
        cp $NEWFS $SRCMNT/LiveOS
    fi
fi
 
# Clean up.
losetup -d $OVDEV $NEWFSDEV $roORIGFSDEV
OVDEV=$(losetup -f --show $SRCMNT$OVERLAY)
 
# Reset overlay.
dd if=/dev/zero of=$OVDEV bs=64k count=1 conv=notrunc,fsync


# If we are processing a booted LiveOS device's root filesystem, then
cd /
if [[ -n $LiveOSMNT ]]; then
mv -f boot/vmlinuz-$new ${bootpath}/vmlinuz
    # Try to unmount the TMPDIR.
mv -f boot/initramfs-${new}.img ${bootpath}/initrd.img
    mountpoint $TMPDIR && umount -l $TMPDIR || :
    shutdown -r +1 'The system will reboot in 1 minute.'
else
    sleep 1
    losetup -d $OVDEV
fi
cleanup 0
</pre>
</pre>
'''Note:''' If the above script is invoked on a currently booted LiveOS root filesystem, the system will be scheduled for a reboot one minute after the script completes.
The Device-mapper snapshot-merge target also allows one to merge changes in a persistent snapshot into the original filesystem (See notes from [[{{TALKPAGENAMEE}}|Andrew Gilmore]]).
{| class="collapsible collapsed"
|-
! Merge overlay with snapshot-merge
|-
|<pre>#!/bin/bash
t0=$(date +%s)
# Prepare working directory.
TMPDIR=$1
mkdir $TMPDIR/LiveOS 2>/dev/null
# Mount source device, SquashFS, if needed.
SRCMNT=$(mktemp -d /media/XXXXXX)
mount $2 $SRCMNT
if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then
    SQUASHMNT=$(mktemp -d /media/XXXXXX)
    mount $SRCMNT/LiveOS/squashfs.img $SQUASHMNT --read-only
    ORIGFS=$SQUASHMNT/LiveOS/ext3fs.img
else
    ORIGFS=$SRCMNT/LiveOS/ext3fs.img
fi
NEWFS=$TMPDIR/LiveOS/ext3fs.img
printf '\nCopying original root filesystem.\n'
if [[ -x /usr/bin/rsync ]]; then
    rsync --inplace --progress $ORIGFS $NEWFS
else
    cp $ORIGFS $NEWFS
fi
# Prepare temporary devices for Device-mapper snapshot-merge.
BLOCKS=$(stat -c '%b' $ORIGFS || :)
NEWFSDEV=$(losetup -f --show $NEWFS)
OVDEV=$(losetup -f --show $SRCMNT/LiveOS/overlay-*)
# Invoke snapshot-merge target device.
dmsetup create merge <<< "0 $BLOCKS snapshot-merge $NEWFSDEV $OVDEV P 8"
# Wait for merge completion.
while state=$(dmsetup status merge)
state=${state#*snapshot-merge }
meta=${state#* }
alloc=${state%/*}
[[ $alloc != $meta ]]; do
    printf '/'
    sleep 0.5
    printf '\b-'
    sleep 0.5
    printf '\b\'
    sleep 0.5
    printf '\b|'
    sleep 0.5
done
printf '\n'
# Clean up from merge.
dmsetup remove merge
sync
dt=$(($(date +%s)-t0))
h=$((dt/3600))
m=$((dt%3600/60))
s=$((dt%60))
printf '\nTime elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s
# Replace original image
if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then
    umount $SQUASHMNT
    rmdir $SQUASHMNT
    rm $SRCMNT/LiveOS/squashfs.img
    mksquashfs $TMPDIR/LiveOS $SRCMNT/LiveOS/squashfs.img -comp xz -keep-as-directory
else
    if [[ -x /usr/bin/rsync ]]; then
        rsync --inplace --progress $NEWFS $SRCMNT/LiveOS
    else
        cp $NEWFS $SRCMNT/LiveOS
    fi
fi
# Reset overlay.
dd if=/dev/zero of=$OVDEV bs=64k count=1 conv=notrunc,fsync
# Clean up.
sleep 2
losetup -d $OVDEV $NEWFSDEV
sleep 2
umount $SRCMNT
rmdir $SRCMNT
LC_NUMERIC=$oldLC
dt=$(($(date +%s)-t0))
h=$((dt/3600))
m=$((dt%3600/60))
s=$((dt%60))
printf '\nTotal time elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s</pre>
|}
The mirror method copies and merges in one pass, while the snapshot-merge method copies the original filesystem first and then merges in the changes. I've found the mirror method to be about 15% faster.
===Overlay recovery===
{{admon/warning|Test and practice the following|This procedure is not thoroughly validated, and may destroy your data.}}
If one 'exhausts' the limited storage capacity of a LiveOS overlay, Device-mapper will mark the filesystem as 'Invalid', as shown by the {{Code|dmsetup status}} command executed in Terminal or a console (if you haven't crashed):
# dmsetup status
live-osimg-min: 0 8388608 snapshot 2464/2464 24
live-rw: 0 8388608 snapshot Invalid
The invalid bit is 00 at byte 5 of the overlay.  You might try to recover the overlay by switching it to 01 with the following command line:
# echo $'\x01' | dd of=/path/to/overlay-file bs=1 count=1 seek=4 conv=notrunc
where /path/to/overlay may be /mnt/live/LiveOS/overlay-devicename-discUUID or<br>/media/devicename/LiveOS/overlay-devicename-discUUID for an attached device.
Follow this by registering the LiveOS image with Device-mapper:
# mount /media/devicename/LiveOS/squashfs.img /mnt/some_mountpoint
# losetup /dev/loop1 /mnt/some_mountpoint/LiveOS/ext3fs.img -r
# losetup /dev/loop2 /media/devicename/LiveOS/overlay-devicename-discUUID
# dmsetup create devicename --table "0 8388608 snapshot 7:1 7:2 P 8"
* devicename in the last line may be any string.
* loop1 and loop2 (and the corresponding 7:1 7:2) may be any free loop devices; just substitute the appropriate numbers.
* 8388608 is the size of the ext3fs.img file in 512 byte units.  This can be read by the following command:
# blockdev -q --getsz /dev/loop1
Execute
# dmsetup status
to check the the virtual filesystem has been configured.
Then try to repair any damage with the following command:
# e2fsck -f -y /dev/mapper/devicename
where devicename is the string you used in the dmsetup create command.
Run a second check,
# e2fsck -p /dev/mapper/devicename
to verify if the filesystem could be repaired.
At this point you will want to enlarge the overlay, or backup or rebuild the image.
Remove the virtual filesystem registration,
# dmsetup remove devicename
Determine the size of the overlay file:
# blockdev -q --getsz /dev/loop2
# losetup -d /dev/loop2
# losetup -d /dev/loop1
# dd if=/dev/zero of=/path/to/overlay-file seek=overlay_size count=size_increase
where overlay_size and size_increase are now 512 byte units.
Once you have a working filesystem, you should proceed to merge your overlay into the original filesystem as described [[#Merge overlay into new image|above]].


==References==
==References==
<references />
<references />
:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].
:: See this [http://thread.gmane.org/gmane.linux.kernel.device-mapper.devel/14644 dm-devel thread].
* The method of [[LiveOS_image#Merge_overlay_into_new_image|mirroring the LiveOS]] was adapted from Douglas McClendon's [http://cloudsession.com/dawg/projects/zyx-liveinstaller/ ZyX-LiveInstaller].
* [[LiveOS image/overlay]]

Revision as of 22:32, 30 April 2019



Introduction

Fedora has developed Live CD USB DVD images for their GNU/Linux operating system. Since the image file systems are stored in the /LiveOS folder of the image, this is the name we'll use to reference the product.

This page shares some critical information about the LiveOS design that helps users take better advantage of their more-limited-than-usual filesystem storage resources.

References

Multi Live Image boot installations

With large capacity USB/SD disc drives readily available, it is convenient to install multiple images on one device.

The livecd-iso-to-disk --multi option allows one to install more than one LiveOS image on a single device. Version 24.2 or greater of Package-x-generic-16.pnglivecd-tools will offer to configure the device bootloader to give a Multi Live Image Boot Menu for the device if the target device contains an existing LiveOS image.

The first live image is installed by default in the device's /LiveOS directory. Subsequent images are installed in sister directories with randomly assigned, 4 character names, such as /sYBm. The target directory may be optionally specified by the [--livedir <dir>] option of livecd-iso-to-disk.

The livecd-iso-to-disk installation script allows one to install from and to a booted LiveOS device and to select any of the installed images as the source for a new installation by using its directory path as the source parameter, for example, /run/initramfs/live/sYBm. The booted LiveOS image can be sourced by the source shortcut 'live' and the booted LiveOS device can be targeted by the target shortcut 'live'.

Storage

When a Live CD or Live DVD (a LiveOS image on read-only disc media) is booted, temporary storage is prepared for the system in RAM on each boot by /sbin/dmsquash-live-root in initrd.img, the initial ramdisk filesystem. By default, a 32 GiB, in-memory, copy-on-write, system overlay in a sparse file is prepared (see File Systems below). The rd.live.overlay.size kernel command line option may be used to set a different, temporary, overlay size. Since the temporary overlay is a sparse file in a tmpfs, a large size, even larger than available memory, may be specified and only what is needed will be allocated as needed.

When the rd.live.overlay kernel command line option is provided on boot, such as with a Live USB device, the dmsquash-live-root.sh script will search for a persistent overlay file to use for storage of root filesystem changes. See Booting live images in the dracut package.

LiveOS device users should take note of their overlay status. See #Overlay allocation status, below, for a way to monitor overlay consumption to anticipate and avoid overlay Overflow or Invalidation.

With Fedora 24 (kernel 4.3+), if the overlay storage space is filled, the overlay will enter a 'Overflow' state and the root file system will continue to operate in a read-only mode. There will not be an explicit warning or signal when this happens, but applications may begin reporting errors due to the restriction. See the instructions at LiveOS image/overlay#Overlay recovery for how one may then enlarge and attempt to repair and merge the overlay.

Note: In systems built before Fedora 24, should the overlay storage space, whether temporary or persistent, be totally consumed, the filesystem will be flagged 'Invalid' and the system will crash with Input/output or Bus errors. Achieving a reboot will require a hard reset since attempting a software initiated reboot on the Invalid filesystem will fail with more Input/output or Bus errors. If such a crash occurs while using temporary storage space for the overlay, a simple reboot will rectify the situation. With persistent storage, the situation is more dire. One can attempt to enlarge and repair the overlay using instructions at LiveOS image/overlay#Overlay recovery. Or one can vacate all changes to the root file system by appending the option rd.live.overlay.reset (formerly, reset_overlay) to the kernel command line on boot-up. A persistent home filesystem, if used, will be unaffected by an overlay reset.

Structure of LiveOS file systems

Live Operating System, LiveOS, file systems are found within disk image .img files.

Live.iso

If one mounts a Fedora-Workstation-Live.iso file or Live CD, the file system will list folders such as the following:

/EFI
/images
/isolinux
/LiveOS
    |- squashfs.img
    
  (Systems before Fedora 24 also had these files
    |- livecd-iso-to-disk  -  a Bash script for installing the LiveOS image onto a USB device
    |- osmin.img  -  a minimized OS image formerly used to aid installation to a hard disk

squashfs.img

The squashfs.img is a SquashFS compressed, read-only, file system holding the Fedora operating system root file system inside another /LiveOS folder containing a rootfs.img file. Mounting the squashfs.img file and listing its file system will show this hierarchy:

/LiveOS
    |- rootfs.img    (This contains a filesystem of type ext4. Before F-24 it was named ext3fs.img.)

Mounting the rootfs.img file will finally reveal the Fedora operating system root file system.

Live.iso + squashfs.img

Here is a summary of the whole structure:

Fedora-Workstation-Live.iso
  !(mount)
  /EFI
  /image
  /isolinux
  /LiveOS
      |- squashfs.img
           !(mount)
           /LiveOS
               |- rootfs.img
                    !(mount)
                    /bin -> usr/bin
                    /boot
                    /dev
                    /etc
                    /home
                    /lib -> usr/lib
                    /lib64 -> usr/lib64
                    /lost+found
                    /media
                    /mnt
                    /opt
                    /proc
                    /root
                    /run
                    /sbin -> usr/sbin
                    /srv
                    /sys
                    /tmp
                    /usr
                    /var

Installed LiveUSB/SD device

When the LiveOS image is loaded onto a USB or SD disk, the /isolinux folder is copied into a /syslinux folder.

If persistence is requested with the --overlay-size-mb NNN option, a Device-mapper overlay file for the root file system is created in /LiveOS.

If a separate home file system is requested with the --home-size-mb NNN option, an ext4 formatted home.img file system is created.

The structure is then as follows:

Fedora-Workstation-Live USB/SD device
  !(mounted on /run/initramfs/live) (Before Fedora 17 the mount point is /mnt/live in the root file system.)
  /syslinux
  /LiveOS
      |- home.img
           !(mounted on /home)      (This occurs during boot up by the /etc/rc.d/init.d/livesys script.)
           /liveuser
               /Desktop
               /Documents
               /Downloads
               ...
           /lost+found
      |- livecd-iso-to-disk
      |- overlay-NAME-XXXX-XXXX     (Where NAME is the device partition name and XXXX are hex numerals.)
      |- squashfs.img
           !(mount)                 (At boot up by the /usr/sbin/dmsquash-live-root.sh script in the initramfs.)
           /LiveOS
               |- rootfs.img
                    !(mounted on '/')    (This occurs during boot up by the dmsquash-live-root.sh script.)
                    /bin -> usr/bin
                    /boot
                    /dev
                    /etc
                    /home                (If there is a home.img, then this is its mount point directory.)

Uncompressed LiveUSB/SD installation

If the --skipcompress option is used during loading with livecd-iso-to-disk, the squashfs.img compressed image will be expanded. The structure is then as follows:

Fedora-Workstation-Live USB/SD device
  !(mounted on /run/initramfs/live)
  /syslinux
  /LiveOS
      |- home.img
           !(mounted on /home)
           /liveuser
               /...
      |- livecd-iso-to-disk
      |- overlay-NAME-XXXX-XXXX
      |- rootfs.img
           !(mounted on '/')          
           /bin -> usr/bin
           /boot
           /dev
           /etc
           /home
           ...

Flattened squashfs.img

For OverlayFS mount overlays, the squashfs.img structure may also be a direct compression of the root filesystem: (Booting this structure is supported in dracut version 049 available in Fedora 30.)

squashfs.img          |  SquashFS from LiveCD .iso
   !(mount)
   /bin               |  Live filesystem
   /boot              |
   /dev               |
   ...                |

Persistent storage

The Fedora LiveOS system allows for persistent storage in 3 locations:

  1. An all-purpose, persistent overlay-based file space that saves all updates and changes to the root LiveOS filesystem. This storage space is limited by its allocate-once, fixed-sized design, and deserves some caution (see File Systems below).
  2. Persistent home filesystem - a re-writable, re-sizable (with difficulty), uncompressed, optionally-encryptable, file space for anything that goes in the users /home/ folder. A persistent home filesystem is an option that may be selected at the time of installation of the LiveOS image (although with some effort, one could manually create a home.img filesystem in /LiveOS/ and move the /home/ folder to it). This installation option is only available through the script, livecd-iso-to-disk. The Windows and Fedora Fedora Media Writer/Live USB Creator installers do not provide this option at present.
  3. Host device file space - this is the underlying USB stick or SD card file system that is outside of the LiveOS file tree, but which is accessible through the /run/initramfs/live or /mnt/live (before Fedora 17) mount point of a booted LiveOS installation. There, one finds the boot configuration files and anything else one had on the device before loading the LiveOS image. One may save files here without consuming the other, limited file spaces. (This file space is limited only by the device capacity).
/run/initramfs/live
                  /LiveOS
                      |- home.img
                      |- livecd-iso-to-disk
                      |- overlay-NAME-XXXX-XXXX
                      |- squashfs.img
                  /syslinux

The all-purpose, persistent overlay is needed for keeping operating system changes and updates between boots.

The file systems are registered on each boot by the /etc/rc.d/init.d/livesys script.

Home filesystem

One may find many advantages to installing the LiveOS, with a persistent, home filesystem (using the --home-size-mb NNN option), which will hold all the user files and documents one wishes and, perhaps later, throw away—all without consuming storage space in the overlay, which can be depleted very quickly.

Device filesystem

Additionally, keeping some storage space on the device disc outside of the LiveOS system will let you copy, carry, and delete large resource files, such as image.iso files, or anything you might want to use or share.

Installation options

Fedora 24 Live Workstation may be installed on a 2 GB USB device using the following options with livecd-iso-to-disk (on a single, terminal or console command line):

./livecd-iso-to-disk --reset-mbr --overlay-size-mb 300 --home-size-mb 200 --unencrypted-home /path/to/source/iso/or/device /dev/sd?1
where '?' in the final parameter represents the target bootable device node, such as sdb1 or sdc1, etc.

The above configuration would allow space for the home folder, the operating system, and a minimal amount on the device root.

But with a larger capacity device, one may allocate the resources to suit the anticipated use, as described above.

File Systems

Note.png
Take note:
It is important to realize that because the LiveOS root filesystem normally resides in a compressed SquashFS image, the standard estimates of free disc space, such as those reported by df and graphical file managers, will report "apparent" file space (as if the filesystem were not compressed), thus, such estimates are usually vast overestimates of the actual free space physically available to the root filesystem. The embedded rootfs.img filesystem partition is often sized to just fit the distributed operating system at the time of its creation.
For example, the Fedora-Workstation-Live-x86_64-28-1.1.iso contains an apparent 6.4 GiB ext4 root filesystem that has been sparsed and compressed into under 1.6 GiB of actual disc space in the .iso file (or on a LiveOS installed device file system). df -h on an untouched mounting of the root filesystem will report that 4.9G is used and 1.5G is available space out of a total size of 6.4G.

The actually available and allocated physical disc space for the root filesystem is determined by the size of the device-mapper overlay file and can only be queried by the dmsetup status command (see below).

For example, one may add, and then delete, a large file in the root filesystem and see (the apparent virtual) used space go up, and then down, when checking with df, but the dmsetup status live-rw allocation report will only ever show an increase—even though allocated overlay sectors are available for reuse if an added file has been deleted (see discussion below).

Fedora LiveOS uses the Device-mapper service of the Linux kernel to manage the file stores on the device. This is the same service that is used by Logical Volume Manager to provide disc partition services.

Overlay allocation status

One critical limitation, mentioned above, is that the LiveOS persistent overlay is an allocate-once, fixed size, file space. This is related to its use of device mapper snapshots to combine a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a Copy-on-write service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks.[1][2] Only changed sectors in files are stored in the overlay. "Deletions" of any original files in the base filesystem are recorded by changes in the index metadata so that the original file blocks are hidden. With this mechanism, no physical storage space of the read-only, base filesystem can be reused. The apparent free space in the virtual root filesystem as reported by the df command will increase, but this additional space is not physically available; however, when a new or changed file is deleted, the new or changed sectors only (in the overlay) are available for reuse, but, unfortunately, the count of allocated sectors does not change, so one cannot be certain of the availability of free physical space after arbitrary changes to the root filesystem.

With Fedora 24 (kernel 4.3+), if the overlay storage space is filled, the overlay will enter an 'Overflow' state and the root file system will continue to operate in a read-only mode. See the instructions at LiveOS image/overlay for how one may then enlarge and attempt to repair and merge the overlay.

The status of the space allocated for persistent storage in the snapshot overlay file may be tracked with the device mapper dmsetup status report.

For example, dmsetup status live-rw may return

live-rw: 0 8388608 snapshot 42296/1048576 176

where the fraction after snapshot is the number of 512-byte sectors allocated in the overlay over the total number available in the overlay. (The final number is the metadata sectors allocated, and the number before snapshot is the apparent size of the virtual filesystem.)

Where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised. If memory or storage resources are available, one may take advantage of the rd.writable.fsimg or rd.live.overlay.size options in the /sbin/dmsquash-live-root startup script to mitigate this risk of failure.

(For systems built before Fedora 24 that exhaust the overlay to an 'Invalid' state, last-ditch recovery of a persistent overlay for the failed root filesystem may be attempted from a separate boot of a working system.)

Kernel updates for images with a persistent overlay

If you have sufficient overlay space to accommodate a kernel update on a Live USB installation, the kernel and initramfs will be installed to the /boot directory. To put these into service they must be moved to the /syslinux directory of the installation partition. This is accessible from the running Live USB filesystem at the /run/initramfs/live mount point. The new initramfs (such as initramfs-4.17.7-200.fc28.x86_64.img) and kernel (such as vmlinuz-4.17.7-200.fc28.x86_64) should be moved to replace the /run/initramfs/live/syslinux/initrd.img and /run/initramfs/live/syslinux/vmlinuz files, respectively.

Note: dracut no longer includes the dmsquash-live module by default. Starting with Fedora 19, dracut defaults to the hostonly="yes" option, which precludes the dmsquash-live module. So, one can add a dracut config file, as root, before updating the kernel:

echo 'hostonly="no"
add_dracutmodules+=" dmsquash-live "' > /etc/dracut.conf.d/01-liveos.conf

For example, the following commands will move a new kernel and initramfs files to the device's /syslinux directory:

bootpath=run/initramfs/live/syslinux
new=4.17.7-200.fc28.x86_64

cd /
mv -f boot/vmlinuz-$new ${bootpath}/vmlinuz
mv -f boot/initramfs-${new}.img ${bootpath}/initrd.img

References

See this dm-devel thread.