From Fedora Project Wiki

(→‎Module content: Rephrase to not make me trip while reading)
No edit summary
Line 1: Line 1:
{{DISPLAYTITLE:Fedora Packaging Guidelines for Modules}}
{{DISPLAYTITLE:Fedora Packaging Guidelines for Modules}}


Find more information about creating a modular operating system on the [https://docs.pagure.org/modularity/ Modularity Website].
These guidelines specify restrictions and guidance for [https://pagure.io/modulemd/blob/master/f/spec.yaml the modulemd specification].


== Overview ==
== Short overview ==


The goal of this document is to describe how to create valid module files, document purposes of all the data fields in them, hint best practices and demonstrate some examples.
* Packager '''MUST''' specify `summary`
* Packager '''MUST''' specify `description`
* Packager '''MUST''' specify `license/module` license(s) based on allowed licenses in Fedora
* Packager '''MUST''' specify `dependencies`
* Packager '''MUST''' specify `components`
* Packager '''SHOULD''' specify `profiles`
* Packager '''SHOULD''' specify `api`
* Packager '''MAY''' specify data in key-value format in `xmd`
* Packager '''MAY''' specify `references`
* Packager '''MAY''' specify `filter`
* Packager '''MAY''' specify `buildopts`
* Packager '''MUST NOT''' specify `name`, name is taken from name of dist-git repository
* Packager '''MUST NOT''' specify `stream`, stream is taken from branch of dist-git repository
* Packager '''MUST NOT''' specify `version`, it is automatically generated by buildsystem
* Packager '''MUST NOT''' specify `arch`, it is automatically filled in by buildsystem
* Packager '''MUST NOT''' specify `servicelevels`, they are automatically filled in by buildsystem from PDC
* Packager '''MUST NOT''' specify `license/content` license(s), they are picked up from components which are built
* Packager '''MUST NOT''' specify `artifacts`


Each module is defined by a single YAML file and comprises of a number of key-value pairs describing the module's properties and components it contains.  Not everything needs to (or even should) be filled in by the module packager; some of the fields get populated later during the module build or distribution phase.  The module file format is commonly known as ''modulemd''.
== Long overview ==
 
The original format specification can be found in the [https://pagure.io/modulemd modulemd repository].
 
== What goes into a module? ==
 
Before we start digging into the technical details of defining a new module, we need to know what goes there.
 
You should include all the dependencies needed for your module to run on the Platform module, with few exceptions:
 
* If you have something shared in your set, with the same lifecycle as Platform, you should try to add it to Platform first. By filing a ticket with https://github.com/fedora-modularity/hp/issues
* If your module needs a common runtime (like Perl or Python) that are already modularized, you should use these as a dependencies rather than bundling them to your module.
* If you see that your module needs a common package (like net-tools), you shouldn’t bundle them either. They should be split into individual modules.
* A set of small “binaries” should be composed in to a shared module if they would be used by other modules
To make sure your module doesn’t conflict with other modules, it can’t contain the same packages as other modules. Your module can either depend on these other modules, or such packages can be split into new modules.
 
== Required fields ==


=== Document header and the data block ===
=== Document header and the data block ===
Line 29: Line 30:
Every modulemd file '''MUST''' contain a modulemd document header which consists of the document type tag and the document format version, and a data block holding the module data.
Every modulemd file '''MUST''' contain a modulemd document header which consists of the document type tag and the document format version, and a data block holding the module data.


  document: modulemd
<pre>
  version: 1
document: modulemd
  data:
version: 2
      (...)
data:
    (...)
</pre>


The version is an integer and denotes the version of the metadata format the rest of the document is written in. As of now, only one officialy released version of the format exists, '''version 1'''.
The version is an integer and denotes the version of the metadata format the rest of the document is written in. Packagers '''SHOULD''' use latest available version of specification.


=== Module summary and description ===
=== Summary and description ===


Every module '''MUST''' include human-readable short summary and description. Both should be written in US English.
Every module '''MUST''' include human-readable short summary and description. Both should be written in US English.


  summary: An example module
<pre>
  description: >-
summary: An example module
      An example long description of an example module, written just
description: >-
      to demonstrate the purpose of this field.
    An example long description of an example module, written just
    to demonstrate the purpose of this field.
</pre>


The summary is a one sentence concise description of the module and '''SHOULD NOT''' end in a period.
The summary is a one sentence concise description of the module and '''SHOULD NOT''' end in a period.


The description expands on this and '''SHOULD''' end in a period. Description '''SHOULD NOT''' contain installation instructions or configuration manuals.
The description expands on this and '''SHOULD''' end in a period. Description '''SHOULD NOT''' contain installation instructions or configuration manuals.
 
=== Module licensing ===
 
Every module '''MUST''' contain a license block and declare a list of the module's licenses.  Note these aren't the module's components' licenses.
 
  license:
      module:
          - MIT
 
Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise.  Therefore MIT might be a reasonable default for most module authors as well.
 
See [[Fedora_Packaging_Guidelines_for_Modules#Module_content_licensing]] to see how to declare components' licenses.
 
== Optional fields ==
 
=== Module name, update stream, version, context and architecture ===
 
Every module artifact '''SHOULD''' define its name, update stream, version, context and architecture.
 
  name: example
  stream: another-example
  version: 20161109235500
  context: c0ffee43
  arch: armv7hl
 
Note, however, that while technically possible, module packagers '''MUST NOT''' define these values manually but should rather expect the Module Build Service to do it for them, using the module's dist-git repository name as the module name, the dist-git repository branch name as the stream name, the particular commit timestamp as the version, a hash of the previous, plus runtime dependencies as the context flag and finally the main content hardware architecture compatibility as the arch.  This simplifies module rebuilds and moving modules between branches or repositories.
 
<code>name</code> and <code>stream</code> may contain any lowercase or uppercase letters, numbers, hyphens, underscores, dots or plus signs.  <code>version</code> is a positive integer.  <code>context</code> is a short hexadecimal hash value distinguishing module artifacts with the same name, stream and version but built against different sets of dependencies.  <code>arch</code> may only include one of our supported hardware architectures; note this is not <code>$basearch</code>.
 
=== Module Service Levels and End of Life ===
 
Every module '''SHOULD''' define its EOL date.  Defined in the ISO 8601 format (YYYY-MM-DD), the date marks the day when the module stops receiving updates.
 
  eol: 2077-10-23
 
Module packagers '''MUST NOT''' define the EOL in the modulemd but should define it in other infrastructure tooling.  The <code>eol</code> property is filled in by the buildsystem, using data from PDC.
 
The modulemd format doesn't support service levels yet.
 
The work in [[Changes/ArbitraryBranching]] will enable packagers to select independent SLs and EOLs for both their RPM branches as well as their module branches.  Both of these values are associated with the ''branch'' in a dist-git repo, but not with the modulemd input or spec file contained therein.
 
Packagers will have to choose from a set of pre-defined SLs maintained by Release Engineering.
 
=== Module content licensing ===
 
If the module includes some RPM or non-RPM content, the packager '''MAY''' also define a list of content licenses.
 
  license:
      module:
          - MIT
      content:
          - GPL+
          - BSD
 
Not every module includes packages and therefore doesn't necessarily have to include this field.
 
Furthermore, the content licenses list should ideally be automatically filled by module build tools rather than the module author.  However, this is not yet implemented.
 
=== Module dependencies ===


Modules '''MAY''' depend on other modules.  These module relationships are listed in the dependencies block.  Dependencies are expressed using module names and their stream names.
=== Licensing ===


  dependencies:
Every module '''MUST''' contain a license block and declare a list of the module's licenses. Note these aren't the module's components' licenses.
      buildrequires:
          platform: master
      requires:
          platform: master


So far modulemd supports two kinds of dependencies:
<pre>
license:
    module:
        - MIT
</pre>


* <code>buildrequires</code> for listing build dependencies of the module, i.e. modules that define the buildroot for building the module's components; this will typically be the <code>platform</code> module, at minimum
Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise. Therefore MIT might be a reasonable default for most module authors as well.
* <code>requires</code> for listing runtime dependencies of the module, i.e. modules that need to be available on the target system for this module to work properly; this too will typically be the <code>platform</code> module, at minimum


Either or both of these block may be omitted, if necessary.
=== Dependencies ===


Currently only fully specific name / stream pairs are supported. Future versions of modulemd will support stream expansion, allowing packagers to depend on any active stream, with black & whitelisting features.
Modules '''MAY''' depend on other modules. These module relationships are listed in the dependencies block. However, none of modules are buildable without `platform` module hence packagers are required to specify it.


=== Extensible module metadata block ===
=== Extensible module metadata block ===
Line 129: Line 74:
Modules '''MAY''' also contain an extensible metadata block, a list of vendor-defined key-value pairs.
Modules '''MAY''' also contain an extensible metadata block, a list of vendor-defined key-value pairs.


  xmd:
<pre>
      user-defined-key: 42
xmd:
      another-user-defined-key:
    user-defined-key: 42
          - the first value of the list
    another-user-defined-key:
          - the second value of the list
        - the first value of the list
        - the second value of the list
</pre>


=== Module references ===
=== References ===


Modules '''MAY''' define links referencing various upstream resources, such as community website, project documentation or upstream bug tracker.
Modules '''MAY''' define links referencing various upstream resources, such as community website, project documentation or upstream bug tracker.


  references:
<pre>
      community: http://www.example.com/
references:
      documentation: http://www.example.com/docs/1.23/
    community: http://www.example.com/
      tracker: http://www.example.com/bugs/
    documentation: http://www.example.com/docs/1.23/
 
    tracker: http://www.example.com/bugs/
=== Module profiles ===
</pre>


The module author '''MAY''' define lists of packages that would be installed by default, and a minimum, when the module is enabled and the particular profile is selected.  Whether the packages actually get installed depends on the user's configuration.  It is possible to define a profile that doesn't install any packages.
=== Profiles ===


Profile names are arbitrary strings.  There is currently one special-purpose profile name defined — ''default''. More special-purpose profile names might be defined in the future.
The module author '''MAY''' define lists of packages that would be installed when the module is enabled and the particular profile is selected. Whether the packages actually get installed depends on the user's configuration. It is possible to define a profile that doesn't install any packages.


The ''default'' profile lists packages that would be installed unless the user's configuration dictates otherwise.
Profile names are arbitrary strings. There are currently few special-purpose profile names defined, see specification for details.


In the case of RPM content, the profile package lists reference binary RPM package names.
In the case of RPM content, the profile package lists reference binary RPM package names.


  profiles:
=== API ===
      default:
          rpms:
              - myapplication
              - myapplication-plugins
      minimal:
          description: An example minimal profile installing only the myapplication package.
          rpms:
              - myapplication


Profiles may list components provided by modules the parent module depends on.
Module API are components, symbols, files or abstract features the module explicitly declares to be its supported interface. Everything else is considered an internal detail and shouldn't be relied on by any other module.
 
{{admon/note|Different use cases|If the primary API component(s) of a module serve different use cases with a different set of subpackages, you should describe these use cases as different install profiles of the module.}}
 
=== Module API ===
 
Module API are components, symbols, files or abstract features the module explicitly declares to be its supported interface. Everything else is considered an internal detail and shouldn't be relied on by any other module.


Every module '''SHOULD''' define its public API.
Every module '''SHOULD''' define its public API.


  api:
=== Filters ===
      rpms:
          - mypackage
          - mylibrary
          - mylibrary-devel
 
Currently the only supported type of API are binary RPM packages, that is the list of RPMs that are guaranteed to a) be present in the module, and b) not break their interfaces such as binaries their provide or their ABI.
 
Generic packages with standard names and installing files into standard locations should be always part of the module API.


=== Module filters ===
Module filters define lists of components or other content that should not be part of the resulting, composed module deliverable. They can be used to only ship a limited subset of generated RPM packages, for instance.


Module filters define lists of components or other content that should not be part of the resulting, composed module deliverable.  They can be used to only ship a limited subset of generated RPM packages, for instance.
<pre>
 
filter:
  filter:
    rpms:
      rpms:
        - mypackage-plugins
          - mypackage-plugins
</pre>


Currently the only supported type of filter are binary RPM packages.
Currently the only supported type of filter are binary RPM packages.
=== Module components ===
Modules '''MAY''', and most modules do contain a components block defining the module's content.
  components:
      (...)
==== RPM content ====
Module RPM content is defined in the <code>rpms</code> block of <code>components</code> and typically consists of one or more packages described by their SRPM names and additional extra key-value pairs, some required and some optional, associated with them. The packages listed contain everything that makes up the API of the module, as well as any build and runtime dependencies of them that aren't satisfied by modules listed as <code>buildrequires</code> or <code>requires</code> in the <code>dependencies</code> block.
  components:
      rpms:
          foo:
              rationale: The key component of this module.
              buildorder: 100
              repository: git://git.example.com/foo.git
              ref: branch-tag-or-commit-hash
              cache: http://www.example.com/lookasidecache/
              arches:
                  - i686
                  - x86_64
              multilib:
                  - x86_64
          dependency-of-foo:
              rationale: Needed for building foo.
              buildorder: 50
              repository: git://git.example.com/dependency-of-foo.git
              ref: master
              cache: http://www.example.com/lookasidecache/
              arches: [ i686, x86_64 ]
              multilib: [ x86_64 ]
The following key-value pairs extend the SRPM name:
* <code>rationale</code> - every component '''MUST''' declare why it was added to the module; this is currently a free form string.  It should end with a period.
* <code>buildorder</code> - marks the component as a member of a specific build group; components are scheduled to be built in batches according to their buildorder tags, from the lowest to the highest; built components are tagged back into the buildroot before the next batch is built; several components can belong to the same build group by specifying the same buildorder value; build order within build groups is undefined; optional, integer, may be negative and defaults to zero if not specified.
* <code>repository</code> - specifies git or other VCS repository to use as the component's source; in Fedora, dist-git is used and this option cannot be overridden.
* <code>ref</code> - the <code>repository</code> reference (a branch or tag name or a commit hash) that should be built and included in this module; recommended.  If not defined, the current HEAD or equivalent is used.  <code>ref</code> is always populated by the exact commit hash used by the Module Build System during build.
* <code>cache</code> - points to RPM lookaside cache; in Fedora this option cannot be overriden.
* <code>arches</code> - a list of architectures this component should be built for; defaults to all available architectures.
* <code>multilib</code> - a list of architectures where this component should be available as multilib, e.g. if <code>x86_64</code> is listed, x86_64 repositories will also include i686 builds.  Defaults to no multilib.
==== Module content ====
Modules may include other modules.  This is similar to dependencies (both build- and run-time) but differs in a few key points:
* included modules are distributed with the parent module as one deliverable, no matter the format
* included modules are built in the buildroot defined by the parent module, recursively
Dependencies and module inclusions can be freely combined.  Deciding on which is more fitting for your module varies from application to application.
Including modules as content is defined in the <code>modules</code> block of <code>components</code> and typically consists of one or more modules described by their names and additional extra key-value pairs, some required and some optional, associated with them.
  components:
      modules:
          my-favourite-module:
              rationale: An example of an included module.
              buildorder: 20
              repository: git://git.example.com/my-favourite-module.git
              ref: 12ab34cd5
The following key-value pairs extend the module-style components:
* <code>rationale</code> - see the description in the [[#RPM_content|RPM content block]]
* <code>buildorder</code> - see the description in the [[#RPM_content|RPM content block]]
* <code>repository</code> - see the description in the [[#RPM_content|RPM content block]]
* <code>ref</code> - see the description in the [[#RPM_content|RPM content block]]
==== Other content ====
No other content is currently supported.
== Examples ==
=== Minimal module ===
A minimal module distributed as ''example:master:20161109172409:*:*'', stored in the <code>modules/example</code> dist-git repository and its master branch, built on November 9, 2016, at 17:24:09 UTC, containing no packages, having no dependencies whatsoever and defining only the minimal set of required metadata.
  document: modulemd
  version: 1
  data:
      summary: An example summary
      description: And an example description.
      license:
          module:
              - MIT
=== Minimal module with RPM content ===
Another flavour of the abovementioned module, containing one RPM package with SRPM name ''foo''.  This module doesn't define any dependencies or optional metadata.
  document: modulemd
  version: 1
  data:
      summary: An example summary
      description: And an example description.
      license:
          module:
              - MIT
      components:
          rpms:
              foo:
                  rationale: An example RPM component.
=== Minimal module with RPM content and build dependency ===
Yet another flavour of the minimal module, containing one RPM package with SRPM name <code>foo</code> and another with SRPM name <code>bar</code> which is required for building and running <code>foo</code>. This module doesn't define any dependencies on other modules or optional metadata.
  document: modulemd
  version: 1
  data:
      summary: An example summary
      description: And an example description.
      license:
          module:
              - MIT
      components:
          rpms:
              foo:
                  rationale: An example RPM component.
                  buildorder: 2
              bar:
                  rationale: >-
                      An example build and runtime dependency of foo.
                  buildorder: 1
=== Minimal module with RPM content but with the -docs subpackage excluded ===
Yet another flavour of the minimal module, containing one RPM package with SRPM name ''foo''.  A build of 'foo' creates binary packages 'foo-1.0-1' and the subpackage 'foo-doc-1.0-1'. Both would get included in the module for any architecture if no filters were used. This module doesn't define any dependencies or optional metadata.
  document: modulemd
  version: 1
  data:
      summary: An example summary
      description: And an example description.
      license:
          module:
              - MIT
      filter:
          rpms:
              - foo-docs
      components:
          rpms:
              foo:
                  rationale: An example RPM component.
=== Minimal module with dependencies only (a variant of stack) ===
Another minimal module, containing no packages or any optional metadata besides dependencies.  Modules of this type are, together with modules that include other modules, referred to as ''stacks''.
  document: modulemd
  version: 1
  data:
      summary: An example summary
      description: And an example description.
      license:
          module:
              - MIT
      dependencies:
          requires:
              platform: master
              a-framework-module: and-its-stream
=== Minimal module which includes another (another variant of stack) ===
Yet another minimal module, containing no optional metadata besides a single included module in the components block. Modules of this type are, together with modules that only depend on other modules, referred to as stacks.
  document: modulemd
  version: 1
  data:
      summary: An example summary
      description: And an example description.
      license:
          module:
              - MIT
      content:
          modules:
              a-framework-module:
                  rationale: Bundled for various reasons.
=== Common Fedora module ===
A typical Fedora module defines all the mandatory metadata plus some useful references, has build and runtime dependencies and contains one or more packages built from specific refs in dist-git.  It relies on the Module Build Service to extract the name, stream and version properties from the VCS data and to fill in the licensing information from the included components and populate the <code>data→license→content</code> list.
  document: modulemd
  version: 1
  data:
      summary: An example of a common Fedora module
      description: This module demonstrates what most Fedora modules look like.
      license:
          module: [ MIT ]
      dependencies:
          buildrequires:
              platform: master
              extra-build-environment: master
          requires:
              platform: master
      references:
          community: http://www.example.com/common-package
          documentation: http://www.example.com/common-package/docs/5.67/
      profiles:
          default:
              rpms:
                  - common-package
                  - common-plugins
          development:
              rpms:
                  - common-package
                  - common-package-devel
                  - common-plugins
      api:
          rpms:
              - common-package
              - common-package-devel
              - common-plugins
      components:
          rpms:
              common-package:
                  rationale: The key component of this module.
                  buildorder: 2
                  ref: common-release-branch
              common-plugins:
                  rationale: Extensions for common-package.
                  buildorder: 3
                  ref: common-release-branch
              common-builddep:
                  rationale: A build dependency of common-package.
                  buildorder: 1
=== Complete module definition ===
See [https://pagure.io/modulemd/blob/master/f/spec.yaml the modulemd specification].


[[Category:Modularity]]
[[Category:Modularity]]
[[Category:Modularization]]
[[Category:Modularization]]

Revision as of 08:33, 26 March 2018


These guidelines specify restrictions and guidance for the modulemd specification.

Short overview

  • Packager MUST specify summary
  • Packager MUST specify description
  • Packager MUST specify license/module license(s) based on allowed licenses in Fedora
  • Packager MUST specify dependencies
  • Packager MUST specify components
  • Packager SHOULD specify profiles
  • Packager SHOULD specify api
  • Packager MAY specify data in key-value format in xmd
  • Packager MAY specify references
  • Packager MAY specify filter
  • Packager MAY specify buildopts
  • Packager MUST NOT specify name, name is taken from name of dist-git repository
  • Packager MUST NOT specify stream, stream is taken from branch of dist-git repository
  • Packager MUST NOT specify version, it is automatically generated by buildsystem
  • Packager MUST NOT specify arch, it is automatically filled in by buildsystem
  • Packager MUST NOT specify servicelevels, they are automatically filled in by buildsystem from PDC
  • Packager MUST NOT specify license/content license(s), they are picked up from components which are built
  • Packager MUST NOT specify artifacts

Long overview

Document header and the data block

Every modulemd file MUST contain a modulemd document header which consists of the document type tag and the document format version, and a data block holding the module data.

document: modulemd
version: 2
data:
    (...)

The version is an integer and denotes the version of the metadata format the rest of the document is written in. Packagers SHOULD use latest available version of specification.

Summary and description

Every module MUST include human-readable short summary and description. Both should be written in US English.

summary: An example module
description: >-
    An example long description of an example module, written just
    to demonstrate the purpose of this field.

The summary is a one sentence concise description of the module and SHOULD NOT end in a period.

The description expands on this and SHOULD end in a period. Description SHOULD NOT contain installation instructions or configuration manuals.

Licensing

Every module MUST contain a license block and declare a list of the module's licenses. Note these aren't the module's components' licenses.

license:
    module:
        - MIT

Fedora content, such as SPEC files or patches not included upstream, uses the MIT license by default, unless the component packager declares otherwise. Therefore MIT might be a reasonable default for most module authors as well.

Dependencies

Modules MAY depend on other modules. These module relationships are listed in the dependencies block. However, none of modules are buildable without platform module hence packagers are required to specify it.

Extensible module metadata block

Modules MAY also contain an extensible metadata block, a list of vendor-defined key-value pairs.

xmd:
    user-defined-key: 42
    another-user-defined-key:
        - the first value of the list
        - the second value of the list

References

Modules MAY define links referencing various upstream resources, such as community website, project documentation or upstream bug tracker.

references:
    community: http://www.example.com/
    documentation: http://www.example.com/docs/1.23/
    tracker: http://www.example.com/bugs/

Profiles

The module author MAY define lists of packages that would be installed when the module is enabled and the particular profile is selected. Whether the packages actually get installed depends on the user's configuration. It is possible to define a profile that doesn't install any packages.

Profile names are arbitrary strings. There are currently few special-purpose profile names defined, see specification for details.

In the case of RPM content, the profile package lists reference binary RPM package names.

API

Module API are components, symbols, files or abstract features the module explicitly declares to be its supported interface. Everything else is considered an internal detail and shouldn't be relied on by any other module.

Every module SHOULD define its public API.

Filters

Module filters define lists of components or other content that should not be part of the resulting, composed module deliverable. They can be used to only ship a limited subset of generated RPM packages, for instance.

filter:
    rpms:
        - mypackage-plugins

Currently the only supported type of filter are binary RPM packages.