From Fedora Project Wiki
Line 11: Line 11:
 
Similar to Fedora, Red Hat uses one git repository for one component internally. It allows to set permissions with enough granularity.
 
Similar to Fedora, Red Hat uses one git repository for one component internally. It allows to set permissions with enough granularity.
  
Open question: who will be able to commit to the dist-git
+
'''Open question:''' who will be able to commit to the dist-git
 
* Proposed variant #1:  members of SCL SIG
 
* Proposed variant #1:  members of SCL SIG
 
* Proposed variant #2 (preferred): users approved by SCL SIG
 
* Proposed variant #2 (preferred): users approved by SCL SIG
  
Structure of the repository will be the same as it is used in Fedora and Red Hat internally. It means every repository will contain at least RPM specfile and sources file which includes list of links to look-aside cache. Optionally, there may be patches and other sources directly in git repository's root, i.e. No directories are used.
+
Structure of the repository will be the same as it is used in Fedora and Red Hat internally. It means every repository will contain at least RPM ''specfile'' and ''sources'' file which includes list of links to look-aside cache (with checksum). Optionally, there may be patches and other sources directly in git repository's root. No directories are used.
  
 
== Branches ==
 
== Branches ==
  
When working with Software Collections it is common that same package is part of more collections. Typical example is a python module, that is available in python27 and python33 software collection as well. Changes between RPM sources for python27 and python33 are typically small or even none.
+
When working with Software Collections it is common that one package is part of more software collections. A typical example is some python module, that is available in ''python27'' and ''python33'' software collection as well. Changes between RPM sources for ''python27'' and ''python33'' are typically small or even none.
  
On the other hand, even though in some cases the same source is used in all branches, dist-git needs to be flexible and make it easy for maintainers to include different code for different collections. Typical use case will be Ruby on Rails (RoR) collection, where we find different versions of components in various RoR versions. Requiring the same sources for those packages would result in ugly spec files with many %if, which is not desired.
+
On the other hand, even though in some cases the same source is used in all branches, dist-git needs to be flexible and make it easy for maintainers to include different code for different collections. Typical use case will be Ruby on Rails (RoR) collection, where we find different versions of components in various RoR versions. Requiring the same sources for those packages in all branches would result in complicated and ugly spec files with many %ifs, which is not desired.
  
A similar scenario works for sources for different platforms -- CentoOS 6 and CentOS 7. Thus, it make sense to have a branch for specific CentOS version as well.
+
A similar scenario works for sources for different platforms -- CentoOS 6 and CentOS 7. Thus, it make sense to have a separate branches for specific CentOS versions as well.
  
To summarize it, we will have one branch for all supported combinations of CentOS version and SCL, where a component is build to, so we will have for example these branches (for packages that are available in python27 and python33 collection):
+
To summarize it, we will have one branch for all supported combinations of CentOS version and SCL, where a component is build for. For example, we will have these branches (for packages that are available in python27 and python33 collection):
 
* scl-python27-el6
 
* scl-python27-el6
 
* scl-python27-el7
 
* scl-python27-el7
Line 31: Line 31:
 
* scl-python33-el7
 
* scl-python33-el7
  
Prefix 'scl' is used to easily identify that a branch is related to SCL.
+
Prefix ''scl'' is used to easily identify that a branch is related to Software collections.
  
 
== Repositories naming ==
 
== Repositories naming ==
Line 41: Line 41:
 
* python-meta
 
* python-meta
 
* python-setuptools
 
* python-setuptools
 +
 +
Git repositories for meta packages components will be called differently than it is done internally in Red Hat. Meta packages will use '-meta' suffix after the ''name of the collection without version''. Using this layout it is easier to maintain meta packages for similar collections.
  
 
== Content of repositories ==
 
== Content of repositories ==
  
The most often reason for having SCLs on CentOS is to provide newer versions of some component. Thus, content of SCL dist-git will often correspond with content in Fedora, there will only be SCL-related macros added, which is not allowed in Fedora currently.
+
The most often reason for having SCLs on CentOS is to provide newer versions of some component. Thus, content of SCLs in dist-git will often correspond with content in a stable Fedora. The difference will be in SCL-related macros added, which is not allowed in Fedora currently.
  
 
== Naming of built packages ==
 
== Naming of built packages ==
  
When built in koji, names of resulting RPM packages need to stay the same as packages in RHSCL, so they are compatible with them. This is required by SCL consumers who do not want to differ between SCLs provided by CentOS and Red Hat. From the same reason, SCLs will use '''rh''' as a vendor, which is used in SCL concept.
+
When built in koji, names of resulting RPM packages need to stay the same as packages in RHSCL, so they are compatible between each other. This is required by SCL consumers who do not want to differ between SCLs provided by CentOS and Red Hat.
 +
 
 +
From the same reason, SCLs will use '''rh''' as a vendor, which is used in RHSCL.
  
 
This is an example of packages that will be built in koji for particular branch:
 
This is an example of packages that will be built in koji for particular branch:
Line 55: Line 59:
 
! component          !! sclo-python33-el6          !! sclo-python34-el6
 
! component          !! sclo-python33-el6          !! sclo-python34-el6
 
|-
 
|-
| python-meta        || python33                    || python34
+
| python-meta        || python33                    || rh-python34 *
 
|-
 
|-
| python             || python33-python            || python34-python
+
| python             || python33-python            || rh-python34-python *
 
|-
 
|-
| python-setuptools   || python33-python-setuptools  || python34-python-setuptools
+
| python-setuptools || python33-python-setuptools  || rh-python34-python-setuptools *
 
|-
 
|-
| python-pytest       || python33-python-pytest      || python34-python-pytest
+
| python-pytest     || python33-python-pytest      || rh-python34-python-pytest *
 
|}
 
|}
  
Git repositories for meta packages components will be called differently than it is done internally in Red Hat. Meta packages will use '-meta' suffix after the name of the collection without version. Using this layout it is easier to maintain meta packages for similar collections.
+
* The prefix '''rh-''' is a planned feature for new collections so they won't be mixed with non-SCL packages called the same.
  
 
== koji setup ==
 
== koji setup ==
  
koji setup for every branch includes setting up:
+
''koji'' setup for every branch includes setting up:
 
* a target/tag for every branch
 
* a target/tag for every branch
 
* buildroot that includes the following packages:
 
* buildroot that includes the following packages:
Line 74: Line 78:
 
** colname00-build
 
** colname00-build
  
Open question: Who will be able to setup the koji buildroots, tags, targets?
+
'''Open question:''' Who will be able to setup the koji buildroots, tags, targets?
 
* Proposed variant #1 (preferred):  members of SCL SIG
 
* Proposed variant #1 (preferred):  members of SCL SIG
* Proposed variant #2: somebody dedicated for koji setup and SIG will ask for setting up koji
+
* Proposed variant #2: somebody dedicated for ''koji'' setup and SIG will ask for setting up koji
  
== CentOS look-aside cache? ==
+
== CentOS look-aside cache ==
  
 
Sources that are not part of git are uploaded into CentOS look-aside cache, similar to what is in Fedora.
 
Sources that are not part of git are uploaded into CentOS look-aside cache, similar to what is in Fedora.

Revision as of 13:24, 25 November 2014

This is a proposal for SCL dist-git structure on CentOS, i.e git repositories where future upstream development of Software Collections will take place. It focuses on easy collaboration of Red Hat engineers, that will be able to merge changes from CentOS into internal dist-git or cherry-pick particular commits as easy as possible.

SCL SIG has been established and approved by CentOS board for driving this effort: http://wiki.centos.org/SpecialInterestGroup/SCLo

Synchronized content of RHSCL being done so far

Currently, source RPMs of all released RHSCL packages are synchronized to git.centos.org as well, but these are not subject of this proposal. They are imported into git.centos.org under rpms project and use SCL prefix in their component name.

Components vs. git repositories

Similar to Fedora, Red Hat uses one git repository for one component internally. It allows to set permissions with enough granularity.

Open question: who will be able to commit to the dist-git

  • Proposed variant #1: members of SCL SIG
  • Proposed variant #2 (preferred): users approved by SCL SIG

Structure of the repository will be the same as it is used in Fedora and Red Hat internally. It means every repository will contain at least RPM specfile and sources file which includes list of links to look-aside cache (with checksum). Optionally, there may be patches and other sources directly in git repository's root. No directories are used.

Branches

When working with Software Collections it is common that one package is part of more software collections. A typical example is some python module, that is available in python27 and python33 software collection as well. Changes between RPM sources for python27 and python33 are typically small or even none.

On the other hand, even though in some cases the same source is used in all branches, dist-git needs to be flexible and make it easy for maintainers to include different code for different collections. Typical use case will be Ruby on Rails (RoR) collection, where we find different versions of components in various RoR versions. Requiring the same sources for those packages in all branches would result in complicated and ugly spec files with many %ifs, which is not desired.

A similar scenario works for sources for different platforms -- CentoOS 6 and CentOS 7. Thus, it make sense to have a separate branches for specific CentOS versions as well.

To summarize it, we will have one branch for all supported combinations of CentOS version and SCL, where a component is build for. For example, we will have these branches (for packages that are available in python27 and python33 collection):

  • scl-python27-el6
  • scl-python27-el7
  • scl-python33-el6
  • scl-python33-el7

Prefix scl is used to easily identify that a branch is related to Software collections.

Repositories naming

Since we use branches to distinguish source code for particular CentOS version and SCL, no prefix is used in git repository name.

For example, dist-git will include the following git repositories:

  • python
  • python-meta
  • python-setuptools

Git repositories for meta packages components will be called differently than it is done internally in Red Hat. Meta packages will use '-meta' suffix after the name of the collection without version. Using this layout it is easier to maintain meta packages for similar collections.

Content of repositories

The most often reason for having SCLs on CentOS is to provide newer versions of some component. Thus, content of SCLs in dist-git will often correspond with content in a stable Fedora. The difference will be in SCL-related macros added, which is not allowed in Fedora currently.

Naming of built packages

When built in koji, names of resulting RPM packages need to stay the same as packages in RHSCL, so they are compatible between each other. This is required by SCL consumers who do not want to differ between SCLs provided by CentOS and Red Hat.

From the same reason, SCLs will use rh as a vendor, which is used in RHSCL.

This is an example of packages that will be built in koji for particular branch:

component sclo-python33-el6 sclo-python34-el6
python-meta python33 rh-python34 *
python python33-python rh-python34-python *
python-setuptools python33-python-setuptools rh-python34-python-setuptools *
python-pytest python33-python-pytest rh-python34-python-pytest *
* The prefix rh- is a planned feature for new collections so they won't be mixed with non-SCL packages called the same.

koji setup

koji setup for every branch includes setting up:

  • a target/tag for every branch
  • buildroot that includes the following packages:
    • scl-utils-build
    • colname00-build

Open question: Who will be able to setup the koji buildroots, tags, targets?

  • Proposed variant #1 (preferred): members of SCL SIG
  • Proposed variant #2: somebody dedicated for koji setup and SIG will ask for setting up koji

CentOS look-aside cache

Sources that are not part of git are uploaded into CentOS look-aside cache, similar to what is in Fedora.

Contributors will be able to upload any sources into CentOS look-aside cache by a tool provided by CentOS (similar to fedpkg in Fedora). Everybody who is able to commit to SCL dist-git is able to upload sources into the look-aside cache.

Example of work for internal RHSCL maintainers

rhpkg clone python
cd python
git remote add centos https://git.centos.com/scls/python.git
git fetch centos
git checkout -b scl-python33-el7 --track centos/scl-python33-el7
[change/commit]
git push centos scl-python33-el7:scl-python33-el7