(note on advantage of sparse temp overlay in tmpfs)
(Multi Live Image paragraph)
|Line 9:||Line 9:|
* [https://github.com//livecd-tools/blob/master/tools/livecd-iso-to-disk.sh livecd-iso-to-disk] Usage instructions are on the first pages (or on the [https://github.com//livecd-tools/blob/master/docs/livecd-iso-to-disk.pod manual page]).
to a to Live
Revision as of 14:45, 24 July 2018
Fedora has developed Live CD USB DVD images for their GNU/Linux operating system. Since the image file systems are stored in the /LiveOS folder of the image, this is the name we'll use to reference the product.
This page shares some critical information about the LiveOS design that helps users take better advantage of their more-limited-than-usual filesystem storage resources.
Multi Live Image boot installations
The livecd-iso-to-disk --multi option allows one to install more than one LiveOS image on a single device. Version 24.2 or greater of
will automatically configure the device boot loader to give a Multi Live Image Boot Menu for the device.
When a Live CD or Live DVD (a LiveOS image on read-only disc media) is booted, temporary storage is prepared for the system in RAM on each boot by /sbin/dmsquash-live-root in initrd.img, the initial ram disk filesystem. By default, a 0.5 GiB, in-memory, copy-on-write, system overlay in a sparse file is prepared (see File Systems below). The kernel command line option may be used to set a different, temporary, overlay size. Since the temporary overlay is a sparse file in a tmpfs, a large size may be specified and only what is needed will be allocated as needed.
When the Booting live images in the dracut package.kernel command line option is provided on boot, such as with a Live USB device, the dmsquash-live-root.sh script will search for a persistent overlay file to use for storage of root filesystem changes. See
LiveOS device users should take note of their overlay status. See #Overlay allocation status, below, for a way to monitor overlay consumption to anticipate and avoid overlay Overflow or Invalidation.
With Fedora 24 (kernel 4.3+), if the overlay storage space is filled, the overlay will enter a 'Overflow' state and the root file system will continue to operate in a read-only mode. There will not be an explicit warning or signal when this happens, but applications may begin reporting errors due to the restriction. See the instructions at LiveOS image/overlay#Overlay recovery for how one may then enlarge and attempt to repair and merge the overlay.
- Note: In systems built before Fedora 24, should the overlay storage space, whether temporary or persistent, be totally consumed, the filesystem will be flagged 'Invalid' and the system will crash with Input/output or Bus errors. Achieving a reboot will require a hard reset, since attempting a software initiated reboot on the Invalid filesystem will fail with more Input/output or Bus errors. If such a crash occurs while using temporary storage space for the overlay, a simple reboot will rectify the situation. With persistent storage the situation is more dire. One can attempt to enlarge and repair the overlay using instructions at LiveOS image/overlay#Overlay recovery. Or one can vacate all changes to the root file system by appending the option (formerly, ) to the kernel command line on boot-up. A persistent home filesystem, if used, will be unaffected by an overlay reset.
Operating system file systems
Live Operating System, LiveOS, file systems are found within disk image .img files.
If one mounts a Fedora-Workstation-Live.iso file or Live CD, the file system will list folders such as the following:
/EFI /images /isolinux /LiveOS |- squashfs.img (Systems before Fedora 24 also had these files |- livecd-iso-to-disk - a Bash script for installing the LiveOS image onto a USB device |- osmin.img - a minimized OS image formerly used to aid installation to a hard disk
The squashfs.img is a SquashFS compressed, read-only, file system holding the Fedora operating system root file system inside another /LiveOS folder containing a rootfs.img file. Mounting the squashfs.img file and listing its file system will show this hierarchy:
/LiveOS |- rootfs.img (This contains a filesystem of type ext4. Before F-24 it was named ext3fs.img.)
Mounting the rootfs.img file will finally reveal the Fedora operating system root file system.
Here is a summary of the whole structure:
Fedora-Workstation-Live.iso !(mount) /EFI /image /isolinux /LiveOS |- squashfs.img !(mount) /LiveOS |- rootfs.img !(mount) /bin -> usr/bin /boot /dev /etc /home /lib -> usr/lib /lib64 -> usr/lib64 /lost+found /media /mnt /opt /proc /root /run /sbin -> usr/sbin /srv /sys /tmp /usr /var
When the LiveOS image is loaded onto a USB or SD disk, thefolder is copied into a folder.
If persistence is requested with theoption, a Device-mapper overlay file for the root file system is created in .
If a separate home file system is requested with theoption, an ext4 formatted file system is created.
The structure is then as follows:
Fedora-Workstation-Live USB/SD device !(mounted on /run/initramfs/live) (Before Fedora 17 the mount point is /mnt/live in the root file system.) /syslinux /LiveOS |- home.img !(mounted on /home) (This occurs during boot up by the /etc/rc.d/init.d/livesys script.) /liveuser /Desktop /Documents /Downloads ... /lost+found |- livecd-iso-to-disk |- overlay-NAME-XXXX-XXXX (Where NAME is the device partition name and XXXX are hex numerals.) |- squashfs.img !(mount) (At boot up by the /usr/sbin/dmsquash-live-root.sh script in the initramfs.) /LiveOS |- rootfs.img !(mounted on '/') (This occurs during boot up by the dmsquash-live-root.sh script.) /bin -> usr/bin /boot /dev /etc /home (If there is a home.img, then this is its mount point directory.)
If theoption is used during loading with , the squashfs.img compressed image will be expanded. The structure is then as follows:
Fedora-Workstation-Live USB/SD device !(mounted on /run/initramfs/live) /syslinux /LiveOS |- home.img !(mounted on /home) /liveuser /... |- livecd-iso-to-disk |- overlay-NAME-XXXX-XXXX |- rootfs.img !(mounted on '/') /bin -> usr/bin /boot /dev /etc /home ...
The Fedora LiveOS system allows for persistent storage in 3 locations:
- An all-purpose, persistent overlay-based file space that saves all updates and changes to the root LiveOS filesystem. This storage space is limited by its allocate-once, fixed-sized design, and deserves some caution (see File Systems below).
- Persistent home filesystem - a re-writable, re-sizable (with difficulty), uncompressed, optionally-encryptable, file space for anything that goes in the user's /home/ folder. A persistent home filesystem is an option that may be selected at the time of installation of the LiveOS image (although with some effort, one could manually create a livecd-iso-to-disk'. The Windows and Fedora 'Fedora Media Writer/Live USB Creator' installers do not provide this option at present. filesystem in and move the folder to it). This installation option is only available through the script, '
- Host device file space - this is the underlying USB stick or SD card file system that is outside of the LiveOS file tree, but which is accessible through the or (before Fedora 17) mount point of a booted LiveOS installation. There, one finds the boot configuration files and anything else one had on the device before loading the LiveOS image. One may save files here without consuming the other, limited file spaces. (This file space is limited only by the device capacity).
/run/initramfs/live /LiveOS |- home.img |- livecd-iso-to-disk |- overlay-NAME-XXXX-XXXX |- squashfs.img /syslinux
The all-purpose, persistent overlay is needed for keeping operating system changes and updates between boots.
The file systems are registered on each boot by the /etc/rc.d/init.d/livesys script.
One may find many advantages to installing the LiveOS, with a persistent, home filesystem (using theoption), which will hold all the user files and documents one wishes and, perhaps later, throw away—all without consuming storage space in the overlay, which can be depleted very quickly.
Additionally, keeping some storage space on the device disc outside of the LiveOS system will let you copy, carry, and delete large resource files, such as image.iso files, or anything you might want to use or share.
Fedora 24 Live Workstation may be installed on a 2 GB USB device using the following options with livecd-iso-to-disk (on a single, terminal or console command line):
- where ' ' in the final parameter represents the target bootable device node, such as or , etc.
The above configuration would allow space for the home folder, the operating system, and a minimal amount on the device root.
But with a larger capacity device, one may allocate the resources to suit the anticipated use, as described above.
Fedora LiveOS uses the Device-mapper service of the Linux kernel to manage the file stores on the device. This is the same service that is used by Logical Volume Manager to provide disc partition services.
Overlay allocation status
One critical limitation, mentioned above, is that the LiveOS persistent overlay is a allocate-once, fixed size, file space. This is related to its use of device mapper snapshots to combine a read-only file system image (copied from the compressed SquashFS.img on the read-only LiveCD or installation .iso file) with a Copy-on-write service that tracks only changed blocks of data in the snapshot (overlay) file and then re-referencing file pointers to the updated blocks. Only changed sectors in files are stored in the overlay. "Deletions" of any original files in the base filesystem are recorded by changes in the index metadata so that the original file blocks are hidden. With this mechanism, no physical storage space of the read-only, base filesystem can be reused. The apparent free space in the virtual root filesystem as reported by the command will increase, but this additional space is not physically available; however, when a new or changed file is deleted, the new or changed sectors only (in the overlay) are available for reuse, but, unfortunately, the count of allocated sectors does not change, so one cannot be certain of the availability of free physical space after arbitrary changes to the root filesystem.
With Fedora 24 (kernel 4.3+), if the overlay storage space is filled, the overlay will enter a 'Overflow' state and the root file system will continue to operate in a read-only mode. See the instructions at LiveOS image/overlay for how one may then enlarge and attempt to repair and merge the overlay.
The status of the space allocated for persistent storage in the snapshot overlay file may be tracked with the device mapperreport.
For example,may return
live-rw: 0 8388608 snapshot 42296/1048576 176
where the fraction after
snapshot is the number of 512-byte sectors allocated in the overlay over the total number available in the overlay. (The final number is the metadata sectors allocated, and the number before
snapshot is the apparent size of the virtual filesystem.)
Where long-term usage of a LiveOS image is anticipated, special attention to overlay consumption is advised. If memory or storage resources are available, one may take advantage of the /sbin/dmsquash-live-root startup script to mitigate this risk of failure.or options in the
- (For systems built before Fedora 24 that exhaust the overlay to an 'Invalid' state, last-ditch recovery of a persistent overlay for the failed root filesystem may be attempted from a separate boot of a working system.)
- See this dm-devel thread.