Infrastructure/Mirroring/Tiering

File:Infrastructure InfrastructureTeamN1.png

Tiering

Fedora mirror servers use Tiering, whereby a select few fast mirrors get read access to the master rsync servers, and all the other mirrors pull from those mirrors.

For our purposes, define:

• master: The Fedora-owned servers dl.fedoraproject.org
• Tier 1: The fast mirrors which pull from a master mirror.
• Tier 2: The mirrors that pull from the Tier 1 servers.

Properties of Tier 1 mirrors:

• Limit the number of Tier 1 mirrors, to ensure adequate bandwidth for these. Adjust number up or down depending on capability of the masters.
• Must carry everything under fedora-enchilada and fedora-epel. This allows Tier 2 mirrors to exclude what they wish, but get everything if they so wish. This means at least 1TB of disk space for the Fedora portion of this server.
• Must have a 1 Gigabit connection to the Internet, or faster.
• Must have an active, available, responsive mirror administrator during the days content is staged.
• Must have at least 2 Internet2-connected Tier 1 mirrors.
• Must have at least 1 Tier 1 mirror on each continent for which we have Tier 2 mirrors.
• Must serve private rsync (see below for configuration).

Master mirrors

• dl0[123].fedoraproject.org, in Ashburn, VA, USA.
• dl0[45].fedoraproject.org, in Ashburn, VA, USA - tier1 mirrors only.
  • dl.fedoraproject.org is a DNS round-robin to dl0[123].
  • dl-tier1.fedoraproject.org is a DNS round-robin for dl0[45].

Master mirror rsync modules

The master mirrors provide two additional rsync modules which provide pre-bitflip content. Fedora tiered mirrors should use these modules to be able to get pre-bitflip content.

Module name	Content
fedora-buffet0	Everything under /pub/, including pre-bitflip content
fedora-enchilada0	Everything under /pub/fedora/, including pre-bitflip content
fedora-epel0	Everything under /pub/epel/, including pre-bitflip content (even though EPEL doesn't do bitflips)

Tier 1 mirrors

Tier 1 mirrors pull from one of the master mirrors.

Server	Organization	Location	Network	Modules	Comment	Contact for ACL
archive.linux.duke.edu	Duke University	US East Coast	IPv4, Internet2	fedora-enchilada and fedora-epel	uses ACL from MirrorManager database	Drew Stinnett <drew.stinnett at duke.edu> (spacepope on IRC)
mirrors.kernel.org	Linux Kernel Organization	US West Coast	IPv4 and IPv6	fedora-buffet, fedora-enchilada, fedora-epel, fedora-secondary, and fedora-alt		ftpadmin at kernel.org
rsync.hrz.tu-chemnitz.de	Technische Universität Chemnitz	Chemnitz, Germany	IPv4	fedora-enchilada and fedora-epel	uses ACL from MirrorManager database	support at hrz.tu-chemnitz.de
ftp-stud.hs-esslingen.de	Hochschule Esslingen	Esslingen, Germany	IPv4 and IPv6	fedora-buffet, fedora-enchilada, and fedora-epel		Adrian Reber <adrian at hs-esslingen.de>
fedora-rsync.ftp.pub.2iij.net	Internet Initiative Japan	Tokyo, Japan	IPv4	fedora-enchilada and fedora-epel		mirror-contact at iij.ad.jp
fedora.c3sl.ufpr.br	Universidade Federal do Paraná	Curitiba, Brasil (South America)	IPv4 and IPv6	fedora and fedora-alt		Carlos Carvalho <carlos at fisica.ufpr.br>
ftp.linux.cz	CZLUG	Brno, Czech Republic	IPv4 and IPv6			ftp-admin at fi.muni.cz
mirror.gtlib.gatech.edu	Georgia Tech	US East Coast	IPv4 and IPv6	fedora-enchilada and fedora-epel		Neil Bright <neil.bright at oit.gatech.edu>
mirrors.rit.edu	Rochester Institute of Technology	US East Coast	IPv4 and IPv6	fedora-buffet, fedora-enchilada, and fedora-epel		mirrors at rit.edu
mirror.liquidtelecom.com	Liquid Telecom	East Africa Datacenter, Nairobi, Kenya	IPv4 and IPv6	fedora-buffet, fedora-enchilada, and fedora-epel		anthony.somerset at liquidtelecom.com
fr2.rpmfind.net	RpmFind	Lyon, France	IPv4	fedora-enchilada, fedora-secondary and fedora-epel		fabrice at bellet.info
download-ib01.fedoraproject.org	Fedora	North Carolina, USA	IPv4 and IPv6	fedora-buffet0	Uses acls from master mirrors	admin at fedoraproject.org
download-cc-rdu01.fedoraproject.org	Fedora	North Carolina, USA	IPv4 and IPv6	fedora-buffet0	Uses acls from master mirrors	admin at fedoraproject.org

Tier 1 rsync configuration

Below is an example rsyncd.conf file for a Tier 1 mirror that provides private rsync access to select downstream Tier 2 mirrors. You may do this via either IP or DNS-based access control, or by a shared username/password which you give to your selected Tier 2 mirrors directly.

The key to this is that the Tier 1 mirror rsyncs content using a user account (e.g. mirror used below), and you serve content to Tier 2 mirrors using a private rsync module that runs as that same user account, while providing public non-authenticated rsync using the nobody account. In this way, Tier 2 mirrors may obtain content before the permissions are made world readable.

uid = nobody
gid = nobody
use chroot = yes
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.bz2 *.iso *.ogg *.ogv *.tbz
exclude = .snapshot/ .~tmp~/ /.private/ /.private/** **/.nfs*
ignore nonreadable = yes
list = true
read only = yes
refuse options = checksum

[ fedora-buffet ]
        comment = Fedora -- the whole buffet (all you can eat)
        path = /srv/pub

[ fedora-enchilada ]
        comment = Fedora -- the whole enchilada
        path = /srv/pub/fedora

[ fedora-epel ]
        comment = Extra Packages for Enterprise Linux
        path = /srv/pub/epel

##
## The following are not seen and are limited by IP.
##

[fedora-buffet0]
       comment = Fedora Buffet for Tier0|1 Mirrors
       path = /srv/pub/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

[fedora-enchilada0]
       comment = Fedora Enchilada for Tier0|1 Mirrors
       path = /srv/pub/fedora/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

[fedora-epel0]
       comment = Fedora EPEL for Tier0|1 Mirrors
       path = /srv/pub/epel/
       list = no
       uid = mirror
       gid = mirror
       hosts allow = (IP or DNS address) ...

Tier 2 mirrors

The number of mirrors is too large to list them here; you can find them in the MirrorManager.