From Fedora Project Wiki

m (change to use '&&' in example)
(Moving the guidelines to docs.fp.o)
(9 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Container Maintainer Guidelines =
= Container Maintainer Guidelines =

In the Fedora world, the concept of being a [ Package Maintainer] is well known as all the software currently released and published as an official "Build Artifact" of the Fedora Project for inclusion in the Fedora GNU/Linux distribution has always been packaged in [ RPM] Package Manager Format.
The guidelines have been moved [ here]
However, as technology changes so must the Fedora Project. The concept of "containers" on Linux has become quite prominent and Fedora will be publishing container images as an officially released Build Artifact. One thing to note is that container images are not a new software packaging format but more so a delivery mechanism where many different things can be easily shipped as a single unit. An example of this is packages that can be combined to deliver an easily ran "software solution".
Below you will find Guidelines similar in nature to that of the Fedora [ Packaging Guidelines] but catered towards the concept of Containers. Fedora is targeting the any [ OCI Compliant] container runtime implementation.
== General Fedora Container Information ==
In this section you will find general information about Fedora Container Images that should be useful for Fedora Container Layered Image Maintainers.
=== Build System ===
In order to get a better understanding of the big picture of how all this works, Container Maintainers might find the [ Layered Image Build Service Architecture Document] interesting. However, extensive coverage of the Build System is out of the scope of this Guidelines document.
=== Vocabulary Terms ===
* '''Container Registry''' - (often refereed to as simply a "registry") a service that stores and distributes Container Images via namespaces/repositories.
* '''Fedora Generational Core''' - defined by the [[Modularity|Fedora Modularity]] effort as a part of [[BaseRuntime| the Base Runtime]] and will serve as our registry namespace. For all intents and purposes this will be a synonym of the Fedora Release. (Will often be noted as <code>$FGC</code>)
=== Fedora Container Naming ===
A Fedora Container Layered Image name should be the same as the the name of main service that it intends to provide end users. Therefore, naming must follow the [[Packaging:NamingGuidelines|Fedora Naming Guidelines]].
Container naming as it will exist in [*/?branches=&status=&owner=&namespace=container Fedora Package DB] (and [Distgit]) should be relatively standard. None of the Fedora releases/DistGit-branch naming should be taken into consideration by the main container name, just as it is for RPM Package Naming.
Fedora content is now "namespaced" in both [ Fedora Package DB] and [ DigtGit], with the default namespace being 'rpms' for backwards compatibility. This allows for Container Layered Images to share the same base component name as their RPM counterpart (where applicable).
Below is a reference of the [[Container:Guidelines#Registry_Layout|Registry Layout]] which will clarify how different Fedora releases are handled.
=== Fedora Base Image ===
The Fedora Base Image provides information that can be used by the Layered Images via inherited Environment Variables.
These are outlined below:
* <code>$FGC</code> is defined as Fedora Generational Core by the [[Modularity|Fedora Modularity]] effort as a part of [[BaseRuntime| the Base Runtime]] and will serve as our registry namespace.
* <code>$DISTTAG</code> is defined just as it is for RPMs, but since Dockerfiles lack a mechanism similar to RPM Macros this is being stored in the base image such that it can be inherited by layered images.
=== Fedora Container Registries and Updates ===
In Fedora there are two Registries: candidate and stable.
All Layered Image Builds end up in the candidate registry as soon as they are successful in the Fedora Layered Image Build System. These images can immediately be pulled (example using docker: <code>docker pull$FGC/$NAME:latest</code>).
Gated releases will happen on a Two Week Cadence, alternating with the Fedora Two Week Atomic Host.
==== Registry Layout ====
Fedora Base Images will be available at the "root" namespace of the registry, an example is below:
Fedora Layered Images will be available in their respective <code>$FGC</code> namespace which coorelates to their DistGit branch and Koji tag. An example is as follows for the <code>f25</code> Fedora Generational Core and the cockpit container image.
There are multiple tags applied to each image:
* <code>$FGC/Name:Version-Release</code> (including <code>$DISTTAG</code>)
* <code>$FGC/Name:Version</code>
* <code>$FGC/Name:latest</code>
** The :latest tag can be omitted when issuing a <code>docker pull</code> command.
The latter two tags are updated in-place and a new execution of <code>docker pull</code> will get the latest image.
==== Searching the Registry ====
At the time of this writing there is no search functionality for the Container Registry but we will release a manifest of all available stable Fedora Layered Images in the Fedora Container Registry as Layered Images become available for release. We aim to change this in the future and provide search functionality to users.
== Container Guidelines ==
The Container Guidelines are a collection of common issues and the severity that should be placed on them. While these guidelines should not be ignored, they should also not be blindly followed. If you think that your container should be exempt from part of the Guidelines, please bring the issue to the Fedora Container Committee (Pending Existence). In the absence of a Fedora Container Committee, please seek guidance from the [[Cloud_SIG|Fedora Cloud SIG]].
=== Containers (Dockerfile) ===
Container images in Fedora are built using a [ Dockerfile] much in the same way an [ RPM] is built using a spec file. In this section are Fedora Guidelines for creating Container images using a Dockerfile.
{{admon/note|Dockerfile Guidelines Upstream|
These guidelines are a Fedora adaptation of the Upstream [ Project Atomic] effort to define [ Container Best Practices].
==== FROM ====
As defined by the Dockerfile reference, the FROM instruction '''must''' be the first line of a Dockerfile.  The FROM instruction '''must''' be fully-qualified with a registry name, image name, and tag as shown in this example:
This provides a guarantee of where the base image is coming from when being built by the build service or when rebuilt by a user. 
For most layered images built by the [[Changes/Layered_Docker_Image_Build_Service|Fedora Layered Docker Image Build Service]], the FROM line will use one of the Fedora base images that exist on the [[Changes/FedoraDockerRegistry|Fedora Registry]]:
It is also possible to use another layered image as the base layer, as in this example:
==== LABELS ====
Dockerfiles have a concept of a [ LABEL] which can add arbitrary metadata to an image as a key-value pair. Fedora Guidelines on the topic of LABELs follows the [ Project Atomic] [ Container Application Generic Labels] standards for LABEL definition.
'''Required''' LABELs for a Fedora Layered Image are as follows:
|The Bugzilla component name where bugs against this container should be reported by users.
|Name of the Image
|Version of the image
|Release Number for this version
|Architecture the software in the image should target (Optional: if omitted, it will be built for all supported Fedora Architectures)
|run or usage
|Either provides an Atomic run line, or a human readable example of container execution
|A short description of the image.
'''Optional''' labels for Fedora Layered Images
|Powers "atomic install" command. Not used for system containers.
|Powers "atomic uninstall" command.  Required if Install is present.
|A URL where the user can find more information about the image.
|A runnable command which results in display of Help information.
|Used for system containers, see below.
|Any of the [ Container Application Generic Labels] which are appropriate to the container, such as "stop", "debug", or "changelog-url"
See LABEL SPECIFICATION below for more details on what's required for each of these labels.
{{admon/note|Dockerfile Label Guidelines Upstream|
The LABELs used here are meant to be a Fedora adaptation of the upstream [ Project Atomic] effort to define [ Container Application Generic Labels] as well as [ Container Best Practices].
These LABELs should be defined in a single line of the Dockerfile such that they don't each lead to another layer in the build. The following is a very simple Dockerfile example containing the required LABELs:
It is a Guidelines '''requirement''' to define these items as ENV variables such that they can be used elsewhere, also note the <code>$FGC</code> and <code>$DISTTAG</code>.
<code>$FGC</code> is defined as Fedora Generational Core by the [[Modularity|Fedora Modularity]] effort as a part of [[BaseRuntime| the Base Runtime]] and will serve as our registry namespace.
<code>$DISTTAG</code> is defined just as it is for RPMs, but since Dockerfiles lack a mechanism similar to RPM Macros this is being stored in the base image such that it can be inherited by layered images.
By following the pattern below, we can define the container specific information in one place on the ENV line and have it be set properly in the LABEL line (again, noting the <code>$FGC</code> and <code>$DISTTAG</code> being used but never defined as these are inherited).
ENV NAME=myawesomecontainer VERSION=0 RELEASE=1 ARCH=x86_64
LABEL  com.redhat.component="$NAME" \
        name="$FGC/$NAME" \
        version="$VERSION" \
        release="$RELEASE.$DISTTAG" \
        architecture="$ARCH" \
        usage="docker run -p 9000:9000 f25/myawesomecontainer" \
        summary="myawesomecontainer makes the myawesomeplatform web 3.0 available via Port 9000."
Some additional details about how each label is to be populated.
'''com.redhat.component''': Existing Bugzilla component against which bugs in this image should be reported.
'''name''': Name of the image.  If the image replaces a standard RPM, it should have the exact same name of that RPM.  Otherwise, please see naming guidelines above.
'''version''': Usually 0.  Populated from the ENV variable.  See "VERSIONING" below for explanation.
'''release''': Populated from the ENV variable.  Should start at 1, and increment every time the image changes.  Generally carries a suffix of "f##container", depending (i.e. "f26container").
'''architecture''': usually "x86_64", unless the container image supports other/all architectures.
'''usage''': a human-readable example command line for invoking the container. Required if run is not present.  Should include all likely options, such as ports, volumes, and any required command-line parameters. You may use any container runtime as your example. Example from the OwnCloud container:
    usage="docker run -d -P -v owncloud-data:/var/lib/owncloud -v owncloud-config:/etc/owncloud owncloud"
'''summary''': A short description of the image, intended to be searchable once we have a registry with search functionality.  Please include relevant keywords.
'''run''': a command line to invoke the container, suitable for use by the [ Atomic CLI], including placeholders and the embedded atomic-run code.  Must successfully execute on a suitable Fedora Atomic system. Required if "usage" is not present.  Example for the Cockpit container:
    run="/usr/bin/docker run -d --privileged --pid=host -v /:/host IMAGE /container/atomic-run --local-ssh"
'''install''': A container may require preparation of the host system before the container can be run. In this case the install label is useful for defining what operations should be performed on the host to prepare it. The set of operations should be as minimal as possible and should not include any operation that is not useful for preparing the host to run the container. If an install label is provided then it must be tested and work with the [ Atomic CLI]. Optionally an uninstall label should also be provided that will allow for cleaning up any operations done by install. Please refer to the [ upstream documentation] for more information.  Example for the Cockpit container:
    install="/usr/bin/docker run --rm --privileged -v /:/host IMAGE /container/atomic-install"
'''uninstall''': If a container has an install label then most likely an uninstall label will be needed in order to delete any files and/or to clean up any configuration that was done or to the host system. It is not required to delete files that may contain user data. In unusual cases there may be no files or configuration to clean up from the install label so the uninstall label might not be needed. If an uninstall label is provided then it must be tested and work with the [ Atomic CLI]. Please refer to the [ upstream documentation] for more information.
    uninstall="/usr/bin/docker run --rm --privileged -v /:/host IMAGE /container/atomic-uninstall"
'''url''': A URL where users can get more information about the image, such as a github or pagure repository, or software documentation.
'''help''': A runnable command which outputs a man page or other "help" information.  If supplied, must be tested with `atomic help`.  If you have a help command, you do not need to also supply a Help File (see below).
==== VERSIONING ====
In the previous section there was coverage of LABELs, one of those is the Version that is set in the example using the <code>ENV</code> variable <code>VERSION</code> which at this time needs to be <code>0</code>. Any changes made to the <code>Dockerfile</code> or the contents of the resulting image that would require a rebuild should increment the <code>Release</code> LABEL that is similarly set using the <code>ENV</code> variable named <code>RELEASE</code>. The reasoning for this is described below.
At this time there is no way to automatically populate the <code>Version</code>/<code>VERSION</code> value with the same value of the latest version of the primary RPM belonging to the container image. This is something that is currently [ on the roadmap].
Why is this needed?
If we set the <code>Version</code> LABEL to the version of it's respective RPM at the time of the Container Image Review, then the maintainer will constantly have to update it by hand every time there is a RPM update which is inconvenient and error prone. Beyond that, there's a possibility that the version of the RPM could be updated by the layered image automatic rebuilds and the maintainer isn't able to update the <code>Dockerfile</code> in a timely manner (Automatic Rebuilds are done by [ Release Engineering] in order to pull in security updates for all layered images). If this were to happen, then the version of the container image will not match the version of the software it's meant to deliver which would lead to confusion and potentially unexpected negative side effects for users. Therefore, for the time being we're saying that the version number of the container is not meaningful but it will be as soon as possible.
==== Help File ====
Just like traditional packages, containers need some "man page" information about how they are to be used, configured, and integrated into a larger stack. As such, a Help File is required as part of your container package unless you have supplied a "help" command instead.  This Help File, if present, will be supplied as part of the Container Review, and must have one of the two following names:
* help.1
It must also be COPYed into the container, to live in the base directory as `/help.1` or `` so that it can be found by other users.
Maintainers should include a copy of this help file in the base directory of their images.
The help file should contain all of the following, depending on the requirements of the image:
* A brief description of what service/software the image contains.
* What purpose it fulfills in a larger infrastructure, if any.
* If it is possible to configure the contained service the file '''must''' contain directions on how to do so.
* If the container has any dependencies on other services (for example a database) the file '''must''' detail these.
* If the container uses any volumes the file '''must''' detail what each one is for, see VOLUMES guidelines for more detail.
* An explanation of each PORT the image listens on, including its protocol and purpose.
* Links to any external documentation or software project pages, if such pages exist.
* If the container has any special requirements (like lots of RAM, or sound server access), these '''must''' be listed.
* If the application has major variants on how it can be built, information about these (e.g. mod_php vs. fastcgi) is '''required'''.
Example Help File:
  PostgreSQL 9.6 image, for supplying PostgreSQL within a Kubernetes stack.
  Includes all of "contrib", but does not include PostGIS or any automated replication management.
  pg-config: usually mapped to /etc/postgresql/, contains the configuration files for PostgreSQL,
    including postgresql.conf and pg_hba.conf. Read-only within the container. 
    Should map to local storage.
  pg-data: usually mapped to /var/lib/pgsql/data, contains database data.  Should be stored in
    a persistent volume or network storage of some kind.  Requires high-throughput, read-write storage.
  5432: the PostgreSQL data port using the pgsql protocol.
  This is a transactional database, and requires 256MB of RAM and at
  least one dedicated core.  If heavily used, can require much greater system resources.
  Sample postgresql.conf and pg_hba.conf files are supplied in the image.  For further documentation,
    please see the PostgreSQL docs.  Note that these files must be readable by the "postgres" user
    inside the container.
  The database system is initialized with the admin user/password of postgres/postgres.  If you wish
    to override this, supply the ENV vars ADMIN_USER and ADMIN_PASS inside the container at initialization time.
==== CMD / ENTRYPOINT ====
Another item required is a CMD or ENTRYPOINT entry so that when an user were run perform the following command (for example), expected behavior occurs.:
<pre>docker run</pre>
For more information on these entries, please reference the upstream [ Dockerfile documentation]. The following is extending on the above example, showing a CMD directive.
ENV NAME=myawesomecontainer VERSION=0 RELEASE=1 ARCH=x86_64
LABEL  com.redhat.component="$NAME" \
        name="$FGC/$NAME" \
        version="$VERSION" \
        release="$RELEASE.$DISTTAG" \
CMD printf "My Awesome Container!\n"
==== Volumes ====
The use of container volumes for persistent data is permitted and encouraged, but the following guidelines need to be followed:
* Any user data that would be at risk of loss on update '''must''' be in a volume.
* Any application configuration data that requires persistence '''must''' be in a volume.  Configuration by environment variables instead is also allowed, either together or instead of configuration volumes.
* All volumes listed in the Dockerfile '''must''' be listed in the Help File.
* The example run command '''should''' have the volume with a persistent name (e.g. "docker run -d -v owncloud-data:/var/lib/owncloud -v owncloud-config:/etc/owncloud owncloud")
* Volumes '''must''' be defined as narrowly as possible.  Specifically, unless the image is intended for use as a system container intended for system administration, volumes must be defined so as to mount system directories which are exclusive to the container. For example, the container must mount /etc/application-name/ for config files, ''not'' /etc/.
Each volume in the Help File '''must''' have the following:
* The full path of the volume
* Why it is marked a volume (such as why this config needs persistence or indicating user data lives there)
Volumes listed in the Help File '''should''' also include information about space, permissions, and performance requirements.
The readme '''may''' contain suggested additional volumes that aren't made mandatory by the Dockerfile, such as locations for generated, rather than self signed, ssl certificates.
=== System Containers ===
System containers are read-only, OS specific runc containers running as systemd services. They use OSTree as storage, skopeo to pull from registries and are managed with the atomic CLI.
System containers' Dockerfiles have the following differences:
==== LABELS ====
* install/uninstall: system containers should not have an install/uninstall label. The install is always done through  <code>atomic install --system $IMAGE</code>  which checks out the image and creates a systemd service.
* run/usage: system containers should not have a run/usage label. All system containers, post installation, should be ran with  <code>systemctl start $SERVICE</code>  or  <code>atomic run $CONTAINER</code>.
* atomic.type: Include this as a label if the container is meant to be ONLY ran as a system container. This is a special label to tell the atomic CLI that the image should be pulled to ostree. Usage: <code>atomic.type='system'</code>.
An example set of labels for a system container:
LABEL com.redhat.component="flannel" \
      name="$FGC/flannel" \
      version="$VERSION" \
      release="$RELEASE.$DISTTAG" \
      architecture="$ARCH" \
      summary="An etcd driven address agent, intended to be run as a system container" \
      maintainer="Giuseppe Scrivano <>" \
System containers also do not need a CMD directive. The installation command  <code>atomic install --system</code>  checks out a local copy of the image and creates necessary files. To set up necessary environment/files/scripts, instead do so through runc's config.json.template or systemd tmpfiles (tmpfiles.template).
If the container can double as a docker container, then the CMD directive should follow the same guidelines as a docker container as highlighted above.
==== Other ====
A system container only requires a bare minimum of a Dockerfile, but for most services that is not enough to create a working service. For more detailed guidelines on specifications of other files, please visit [[:Container:System_Container]]
=== systemd Containers ===
A systemd-based container runs systemd(/sbin/init) by default as its entrypoint. One or more services are configured using unit files, the say way they would be on a physical system or a virtual machine.
==== Benefits for systemd based containers ====
systemd based containers have some advantages over running a service directly as the entrypoint (pid 1):
* Support for multi service containers.
* Support for unit file mechanism for starting the container.
* Ability to reap zombie processes.
* Population of /tmp with content needed to be run by a service
* Proper handling of syslog messages
* Proper handling of journalctl messages
==== Issues with systemd based containers ====
STDOUT/STDERR of the container does not come back though the container runtime logging system. As a result, on a container host without oci-systemd-hook and oci-register-machine, log files will not appear on the host, since there’ll be no way to make the logs available outside of the container.
In addition, systemd adds some requirements on the mechanism that containers are run:
* systemd requires that a tmpfs be mounted at /run.
* Systemd inside the container wants container environment variable defined.  (container=oci)
* systemd requires that /sys/fs/cgroup be available inside of the container
* systemd requires that the signal to stop the container is SIGRTMIN+3
* systemd likes if /tmp is a tmpfs.
For more information on systemd in a container read:
==== LABELS ====
* run/usage: Installing the oci-systemd-hook and oci-register-machine packages alongside the docker packaged shipped in Fedora takes care of most of the setup for running a systemd container. However, in order to provide compatibility with other container hosts which do not include oci-systemd-hook and oci-register-machine, systemd-based containers should include a run or usage label that directs users to include the "--tmpfs /run", "--tmpfs /tmp", and "-v /sys/fs/cgroup:/sys/fs/cgroup" that systemd requires to run.
For example:
LABEL usage="docker run -d -P --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup SYSTEMDIMAGE"
systemd-based containers must include "/sbin/init" as an ENTRYPOINT or CMD, which will start systemd inside the container at run time. systemd will start up and manage services enabled with the Dockerfile using the standard "systemctl enable foo" commands in RUN statements.
For example:
ENV container=oci
RUN dnf -y install httpd; dnf clean all; systemctl enable httpd
CMD [ "/sbin/init" ]
In the future, Fedora will provide a base-init container called fedora-init. This image will be set up to run systemd containers by default. This means your Dockerfile could be as simple as
RUN dnf -y install httpd &&
    dnf clean all &&
    systemctl enable httpd
== Content ==
Dockerfiles in Fedora should not contain net new code. The meaning of this is that software should be packaged properly as RPMs and placed in the Fedora repositories, Dockerfiles are simply a deliver mechanism for pre-defined "ready to run" configurations. This can be achieved as an [ Atomic App] or similar. Any content that is to accompany the Dockerfile must either be configuration files or startup/orchestration scripts. The goal of this is such that we follow the key points of the [ Fedora Release Engineering Philosophy].
=== Multi Container Services ===
Each container image should provide only one "service" and multi-container services should be handled by an external orchestration tool at the users discretion such as [ OpenShift Origin], [ kubernetes], [ deis], [ Docker Swarm], [ Docker Compose], [ DC/OS], [ Cloud Foundry], [ Apache Mesos], etc.
These types of multi-container services should be documented in such a way that users can adapt them to their needs. One example would be using the [ Project Atomic] [ nulecule] specification.
{{admon/important|Future|In the future there will be an user facing outlet such that this information can be index and searched so that it's easy for users to consume this content. Our long term [[Changes/FedoraDockerRegistry| Fedora Container Registry]] goals are still in development}}
= Discussion =
For suggestions, feedback, or to report issues with this page please contact the [[Cloud|Fedora Cloud SIG]].
See [[Talk:PackagingDrafts/Containers]] for discussion.
= See also =

Latest revision as of 15:23, 17 June 2019

Container Maintainer Guidelines

The guidelines have been moved here