Passim Peer-to-Peer Metadata
Passim is a local caching server that broadcasts specific shared metadata to other clients on your local network to reduce the amount of duplicate data downloaded from the internet.
- Name: Richard Hughes
- Email: email@example.com
- Targeted release: Fedora Linux 40
- Last updated: 2023-10-05
- Discussion Thread
- devel thread
- FESCo issue: #3072
- Tracker bug: #2242387
- Release notes tracker: #1048
Much of the software running on your computer that connects to other systems over the Internet needs to periodically download metadata or information needed to perform other requests.
Everybody downloads the same file from a CDN, and although a CDN is not super-expensive, it's certainly not free. Everybody on your current network (perhaps thousands of users) has to download the same 1MB blob of metadata from a CDN over a perhaps-expensive internet link.
What if we could download the file from the CDN on one machine, and the next machine on the local network that needs it instead downloads it from the first machine? We could put a limit on the number of times it can be shared, and the maximum age so that we don't store yesterdays metadata forever, and so that we don't turn a ThinkPad X220 into a machine distributing 1Gb/s to every other machine in the office. We could cut the CDN traffic by at least one order of magnitude, but possibly much more. This is better for the person paying the cloud bill, the person paying for the internet connection, and the planet as a whole.
IPFS is an existing project that's existed for many years and allows sharing with other users not on your local network. It's not packaged in any distributions and not trivial to install correctly. It's main drawback is that it requires a internet to IPFS "gateway" which costs a large amount of money for the LVFS, and that it's not EAR/ITAR compliant.
I've asked for feedback already on fedora-devel and have already started making changes and suggestions from that discussion -- for instance splitting out a -libs subpackage. See https://www.spinics.net/lists/fedora-devel/msg315078.html for the discussion.
Benefit to Fedora
Fedora will consume less bandwidth when checking for firmware updates. Per user there is only a 2MB/day saving, but for millions of Fedora users this adds up to a huge amount of saved data (and money) globally.
The code is already written, tested and ready to go. The passim package is already included in Fedora (although not installed by default) and fwupd 1.9.5 includes this functionality.
Old versions of fwupd will be updated and start sharing metadata with other local users with no changes required.
How To Test
Install two physical machines or VMs with Fedora 40 that are both on the local network. Run
fwupdmgr refresh on the first, and observe that
passim dump or
https://localhost:27500/ lists the published metadata file . Then run
fwupdmgr refresh --verbose on the second machine and see that the file is downloaded from https://localhost:27500 rather than
cdn.fwupd.org. Avahi needs to be enabled and running, as does
passimd although both are autostarted as required.
Each user will use 2MB less bandwidth per day when there are other users on the local network with the same metadata file. There is no user-visible difference to any operation.
None; fwupd will recommend passim to be installed by default and autolaunch it as required.
Recommends: passim to
Suggests: passim in the
fwupd.spec file so that it's not autoinstalled by default. In this case fwupd will fall back to downloading from the CDN every day.
Fedora now uses a peer-to-peer service called Passim to reduce the amount of bandwidth used when downloading metadata.