From Fedora Project Wiki
m (Bkhomuts moved page User:Bkhomuts to Category:Changes/OptimizeSquashFS: Change proposal filed)
Line 184: Line 184:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
https://pagure.io/releng/issue/9127.
https://pagure.io/releng/issue/9127.<br/>
mksquashfs(1)
mksquashfs(1)<br/>
lorax(1)
lorax(1)<br/>
https://docs.pagure.org/pungi
https://docs.pagure.org/pungi



Revision as of 15:28, 11 January 2020

Important.png
Comments and Explanations
The page source contains comments providing guidance to fill out each section. They are invisible when viewing this page. To read it, choose the "view source" link.
Copy the source to a new page before making changes! DO NOT EDIT THIS TEMPLATE FOR YOUR CHANGE PROPOSAL.


Reduce installation media size by improving the compression ratio of SquashFS filesystem

Summary

Improve compression ratio of SquashFS filesystem on the installation media.

Owner

Current status

  • Targeted release: Fedora 32
  • Last updated: 2020-01-11
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

As of Fedora 31, the LiveOS/squashfs.img file on the installation image, is compressed with default settings of mksquashfs. The standard configuration is set to XZ algorithm with block size of 128k and BCJ filter enabled. Those parameters can be adjusted which will lead to a better compression ratio and/or reduction of the CPU usage at build time. SquashFS compression options summary.png This is simple to achieve. Recently, Lorax has gotten support[1] for adjusting the compression options for mksquashfs via the configuration file. The file should be altered as following:

[compression]
bcj = yes
args = -b 1M -Xdict-size 1M -no-recovery

Where -b 1M and -Xdict-size 1M are block and dictionary sizes respectively. Could be adjusted.

The compression algorithm is to be selected. Please wait for more benchmarks on the alternative compression options.

Benefit to Fedora

  • Reduction of the installation media size and the cost of storing and distributing Fedora.
  • Reduction of the CPU usage at build time. Depending on which compression parameters chosen.

Scope

  • Proposal owners:

The build environment should have support for adjusting the Lorax configuration file. Lorax is a program that produces the LiveOS/squashfs.img file on the installation media.

One of the way to enable such customization is to introduce support in Pungi to pass -c option to Lorax.

  • Other developers:

The pungi utility should support passing the custom configuration file location to the Lorax utility. This option should apply during buildInstall phase of pungi.

  • Release engineering: [1]
  • Policies and guidelines: Not required.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

This change comes at a cost of higher memory usage during the installation. Based on my personal estimations, this should not be the issue. Since the decompression should require up to 1MiB per thread. At the moment, the decompression of SquashFS is single-threaded.

N/A (not a System Wide Change)

How To Test

mkdir -p /mnt/new /mnt/old
sudo mount -o loop,ro FedoraInstallationOld.iso
sudo mount -o loop,ro FedoraInstallationNew.iso
ls -l /mnt/{new,old}/LiveOS/squashfs.img

And then calculate the size difference.

User Experience

  • Decreasing the installation image size will reduce cost of mirroring and storing Fedora installation images.
  • Decreasing the installation image size will reduce the download time.
  • Increasing the block size on the current configuration with EXT4 file system, should increase latency while accessing the EXT4 filesystem. The exact impact is to be evaluated.
  • The impact of latency will be reduced, if the plain SquashFS option is be choosen.

Dependencies

Pungi, a utility that builds the compose, should include new functionality mentioned above. Alternatively, the /etc/lorax/lorax.conf should be altered in the environment where Lorax is running.

Contingency Plan

N/A

Documentation

https://pagure.io/releng/issue/9127.
mksquashfs(1)
lorax(1)
https://docs.pagure.org/pungi

Release Notes