From Fedora Project Wiki

Move all to /usr

Summary

Provide a simple way of mounting almost the entire installed operating system read-only, atomically snapshot it, or share it between multiple hosts to save maintenance and space. Instead of spreading RPM package content all over the place in the filesystem, and artificially separate /bin from /usr/bin and /lib from /usr/lib, move all content to /usr and provide only symlinks in the root filesystem.

/usr on its own filesystem provides a lot of valuable options in custom setups. For historic reasons, we split-off more and more tools from /usr and put them in /. But, advanced features in today's systems can not really bootup with an empty /usr anymore. More and more fails in subtle ways in such setups.

Instead of moving more tools to /, we today already require /usr to be mounted from inside the initramfs, to be available before the real 'init' starts. The split of the root filesystem an /usr serves no purpose in Linux anymore and only complicates or prevents simple and more flexible setups.

Owner

Current status

  • Targeted release: Fedora 17
  • Last updated: 2011-09-21
  • Percentage of completion: 5%

Detailed Description

There is no way to reliably bring up a modern system with an empty /usr, there are two alternatives to fix it: copy /usr back to the rootfs or use an initramfs which can hide the split-off from the system.

Historically /bin, /sbin, /lib had the purpose to contain the utilities to mount /usr. This role can now be taken by the initramfs. Because the initramfs knows, where to find the root partition (which includes /etc), it can parse /etc/fstab and other configuration files and mount /usr before it finally switches the root partition and executes /usr/bin/init. From this point on init mounts the remaining partitions in /etc/fstab and the system starts as usual.

The long-term plan is to clean up the mess and confusion the current split of / vs. /usr has created. All tools will move back to /usr where they belong, and the rootfs will only contain compat-symlinks into /usr. Almost the entire system installed by packages will reside in /usr. This will split all non-host specific data to /usr. /usr can then be seen as the Unix System Resources partition (/System), which defines the base operating system (e.g. F18 or RHEL-7).

This new /usr could be mounted read-only by default, while the rootfs is read-write and contains only empty mount points, compat-symlinks to /usr and the host-specific data like /etc, /root, /srv. Compared to today's setups, the rootfs will be very small. The new /usr could also easily be shared read-only across several systems, and it would contain almost the entire system. Such setups are more efficient, can optionally provide a lot more security, are more flexible, provide more sane options for custom setups, and are much simpler to setup and maintain.

This leaves us with the following well-defined directories, which compose the base of the system:

  • /usr - installed system; shareable; possibly read-only
  • /etc - config data; non-shareable
  • /var - persistent data; non-shareable;
  • /run - volatile data; non-shareable; mandatory tmpfs filesystem

In the process of moving /bin and /sbin to /usr/bin, /usr/sbin can be moved also to /usr/bin. 

/
|-- etc
|-- usr
|   |-- bin
|   |-- lib
|   `-- lib64
|-- run
|-- var
|-- bin -> usr/bin
|-- sbin -> usr/bin
|-- lib -> usr/lib
`-- lib64 -> usr/lib64

Benefit to Fedora

  • Clear separation of operating system and host specific resources.
  • /usr can be read-only and shareable.
  • /usr can be easily snapshotted.

Scope

The ability to share /usr is especially useful for clusters and virtual machines. The ability to mount /usr read-only (e.g. on read-only media) adds to the security of the machine.

How To Test

  • update a Fedora package with files in /bin, /sbin, /lib or /lib64 via yum

-> see symbolic links in /bin, /sbin, /lib or /lib64 pointing to the file /usr/bin /usr/lib or /usr/lib64 

 # rpm -qf <symbolic link>

should output ownership of that compat symlink

or

  • install a fresh F17

-> see symbolic toplevel links: 

 /lib -> usr/lib
 /lib64 -> usr/lib64
 /sbin -> usr/bin
 /bin -> usr/bin
 /usr/sbin -> bin

User Experience

  • less toplevel directories

Dependencies

  • initramfs (dracut)
  • changes in selinux policies
  • repackaging of packages with content in /bin, /sbin, /lib*
  • drop consolehelper to move /usr/sbin/* to /usr/bin
  • alternatives symlinks?
  • filesystem rpm, toplevel symlinks

Roadmap

  • Begin changing rpm packages with files in /bin, /sbin, /usr/sbin, /lib, /lib64.
  • Make backward compat symlinks in %post and %ghost those symlinks:
%post
# create compat symlink for tools as long as root directories are not converted to symlinks
if ! test -L /bin; then
    ln -s ../usr/bin/foo /bin/foo
    ln -s ../usr/bin/bar /bin/bar
fi
if ! test -L /sbin; then
    ln -s ../usr/bin/buz /sbin/buz
fi

%files
%ghost %attr(777, root, root) /bin/foo
%ghost %attr(777, root, root) /bin/bar
%ghost %attr(777, root, root) /sbin/buz
  • RPM: 257 packages that install files in the root filesystem.
  • Change SELinux policies.
  • On new installation: create symlinks /bin -> usr/bin, /sbin -> usr/bin, /lib -> usr/lib, /lib64 -> usr/lib64, /usr/sbin -> bin. These links will take care that installed packages do not install compat symlinks in %post.
  • Make sure dracut is able to mount needed filesystems specifies in /etc/fstab before starting systemd.
  • Drop consolehelper to enable the /usr/sbin -> /usr/bin move.

Contingency Plan

  • We do not support to bootup with an empty /usr today, so moving things to /usr and have compat links in the rootfs should be low risk. If things turn out to get difficult, we can delay the creation of the /bin /sbin /lib lib64 compat links to a later release.

Documentation

Release Notes

Comments and Discussion

  • The contingency plan is flawed. Disregarding the compat symlinks is not an option as those will be needed for third party scripts, FHS compliance, user comfort, etc.
  • I would strongly suggest considering the /sbin/ => /bin/ and /{s,}}bin => /usr/{s,}bin separately. There's different considerations for each (for instance, the consolehelper porting is only necessary for the /sbin/ => /bin/ move, not for the / to /usr/ move.)
  • The statement "Historically /bin, /sbin, /lib had the purpose to contain the utilities to mount /usr" is only one reason for the split. The FHS specifies other reasons: http://pathname.com/fhs/pub/fhs-2.3.html#PURPOSE2. The one that sticks out as unaddressed is "Disk errors that corrupt data on the root filesystem are a greater problem than errors on any other partition. A small root filesystem is less prone to corruption as the result of a system crash." This is both helped (the root filesystem will indeed be smaller) and hindered (but the root filesystem is essential in part because it contains the tools necessary to recover from disk errors... which would no longer bethe case) by this change.
  • Additionally, the FHS is not only specifying these directories for purely technical reasons but also for organizational reasons. For instance, /bin contains "Essential" user command binaries. The section of FHS pointed at above would seem to indicate a definition of "Essential" that includes not only what's necessary to boot (what this feature is saying is no longer necessary) but also the commands necessary to recover, repair, or restore a system. The /bin and /sbin split is even more a separation due to organization of binaries rather than a need to do that to prevent breaking the system.
  • Many of the proposed benefits of this feature, such as "/usr can be read-only and shareable. " are already present in the current /usr. For all such benefits, the benefit is actually that more files are going to be included on /usr (or the inverse, that less will be included on /).
  • For dependencies, the Feature also needs to make sure that it works for the non-programming/library files that are stored in those directories on /. For instance, /lib/udev, /lib/systemd, /lib/terminfo.... These may all be fine but they need to be considered as dependencies that may need changing.
  • FPC should probably be involved as obeying the FHS is a packaging mandate (they'd want to evaluate whether any of this violates the FHS or not and whether it seems justified to make an exception if so) and also, if individual packages (like coreutils) were to implement symlinks, then how to deal with that (making symlinks for the major directories in filesystem may sidestep, that, though).
    • FPC took an informal look at this today. There's a consensus that we'd be against the /usr/bin, /usr/sbin merge. Merging /bin, /sbin, /lib, /lib64 into their /usr/ counterparts and keeping the compat symlinks will probably take some Guidelines updates but out initial view is that whether or not to go forward with that portion is up to FESCo.
  • anaconda and the docs team are additional dependencies as documentation and code that recommends relative partition sizes would need to be updated.
Retrieved from "https://fedoraproject.org/w/index.php?title=Features/UsrMove&oldid=259317"