From Fedora Project Wiki

m (removing dead link)
(40 intermediate revisions by 21 users not shown)
Line 1: Line 1:
{{admon/important | Git config needs| A config problem has been discovered which may make <code>fedpkg push</code> or <code>fedpkg commit -p</code> or even <code>git push</code> not understand your intentions.  As a short term work around, you can run: <code>git config --global push.default tracking</code>.  This option tells git that when it gets a push request with no other arguments, see if it's tracking a remote branch.  If it is, push to that remote branch.  This seems like it will usually be safe no matter what git repository you are using.}}
{{autolang|base=yes}}


== Cookbooks ==
This page provides some basic instructions for day-to-day usage of the [http://git-scm.com/ git]-based package maintenance system for Fedora. It is intended primarily for new and current Fedora package maintainers, but does briefly [[#anon|cover anonymous read-only use of the system]]. It is not a guide to RPM packaging per se. Some pre-existing knowledge of git may be useful, but is not a pre-requisite (in fact, Fedora packaging can be a relatively painless introduction to it).


Refer to [[Package_update_HOWTO |How to update a package]] if you want to update an existing package.
You may have been looking for, or also be interested in:


[[New package process for existing contributors]] gives step-by-step instructions for that case.
* [[How_to_create_an_RPM_package|Learning to create packages]]
* [[Join_the_package_collection_maintainers|Becoming a package maintainer]]
* [[Package_update_HOWTO|Submitting package updates]]
* [[New_package_process_for_existing_contributors|Adding a new package to the repository]] as an existing maintainer
* [[PackageDB_admin_requests#Additional_branches_for_existing_packages|adding a new release branch]] to an existing package
* [[Packaging:Guidelines|The Packaging Guidelines]]


== Commands ==
== Installing ''fedpkg'' and doing initial setup ==


Here is a basic table of equivalent commands:
Start by installing {{package|fedpkg}} with {{command|dnf install fedpkg}}. This will be your primary interface to the packaging system.
{|
! OLD CVS !! NEW GIT !! Notes
|-
|cvs checkout libfoo || fedpkg clone libfoo || Note that branches are not separated out by directory with this command. <BR> To switch to a different branch with this layout, run: fedpkg switch-branch f13
|-
|cvs checkout libfoo '''(ALTERNATE)''' ||fedpkg clone -B libfoo || This command creates the same branch/directory layout that was used with cvs.
|-
|cvs update || git pull ||fedpkg will get a pull soon.
|-
|cvs commit . || fedpkg commit -p || The -p option to commit invokes a push, just like with git, but you can separate these actions, as shown just below
|-
|cvs commit . '''(ALTERNATE)''' || fedpkg commit && fedpkg push ||
|-
|cvs commit -m 'breaking things' . || fedpkg commit -m 'breaking things' -p ||
|-
|cvs add foo bar || git add foo bar ||
|-
|make clog || fedpkg clog || Create the "clog" file from the %changelog of the specfile.
|-
|cvs commit -F clog . || fedpkg commit -F clog -p || Use the contents of file "clog" as the commit message.
|-
|make tag || N/A || Explicitly tagging source states for package builds is no longer necessary.
|-
|make build || fedpkg build || When successful on Koji, the build will be tagged in koji with the corresponding koji tag (e.g., dist-f15 for Rawhide and dist-f14-updates-candidate for F14). No git tag will be created as of 2010-08-06. You can use <code>git tag $(fedpkg verrel) && git push --tags</code> to do this manually.
|-
|BUILD_FLAGS="--nowait" make build || fedpkg build --nowait || See above. The --nowait option to the build command avoid your waiting (for the build to complete before freeing up the Shell)
|-
|make prep || fedpkg prep ||
|-
|make chain-build CHAIN='foo bar : baz'|| fedpkg chain-build foo bar : baz || Just like before, the current directory package is always added to the end of the CHAIN list.
|-
|make scratch-build || fedpkg scratch-build ||
|-
|make srpm || fedpkg srpm ||
|-
|make srpm-scratch-build || fedpkg srpm<BR>fedpkg scratch-build --srpm <name of srpm> || Ideally this would be made a single "fedpkg srpm-scratch-build" command
|-
|make local || fedpkg local ||
|-
|make x86_64 || fedpkg local --arch x86_64 ||
|-
|make upload FILES='apollo starbuck'|| fedpkg upload apollo starbuck || Remember, this adds new source files without replacing others.
|-
|make new-sources FILES='red_fish blue_fish' || fedpkg new-sources red_fish blue_fish || This replaces any existing source files.
|-
|make help || fedpkg help || "fedpkg -h" also works
|-
|./cvs-import.sh libfoo-1.2-3.src.rpm || fedpkg clone libfoo <BR> cd libfoo <BR> fedpkg import libfoo-1.2-3.src.rpm <BR> (Review Changes) <BR> fedpkg commit -p ||
|}


=== Anonymous checkout ===
You also must have an ssh key configured in the [https://admin.fedoraproject.org/accounts/ Fedora Accounts System] to be able to make changes to any package (including your own). fedpkg will expect the correct ssh key to be available in your keyring.


If you need anonymous clone (checkout) without fedpkg, you can use git: <pre>git clone git://pkgs.fedoraproject.org/libfoo</pre> for package <code>libfoo</code>
Before uploading sources with <code>new-sources</code> and <code>upload</code> and building packages in Koji, with <code>build</code> for example, you have to get your Kerberos credential first with <code>kinit</code>, e.g.


=== Merging ===
  kinit [FAS name]@FEDORAPROJECT.ORG


Merging is one of the things that we can now do directly in the VCS that we had to do with an external diff program in CVS.  Getting used to how it works may take a little adjustment though.
(Keep FEDORAPROJECT.ORG in capital case)


==== Initial Merging ====
Set your Fedora Account System username in {{filename|~/.fedora.upn}}.  You can do this via {{command|echo "FAS_USERNAME" > ~/.fedora.upn}}.


First, the way we imported from cvs means that we have to do a little work to make the older release branches "mergable" (F-14, since it branched after the conversion to git, doesn't need this).
== Common fedpkg commands ==


{{admon/note|Which branches should you apply this to?|For packages where F-12, F-13, the freshly branched F-14 and rawhide all have the same spec files and packages, merging all the branches together clearly makes sense. However, when, say F-12 and F-13 share a package NVR, and F-14 and rawhide do... what to do then? Merge the branches that are supposed to be the same, then.
This section lists typical fedpkg commands in a normal workflow, with short descriptions. In this workflow, we will be operating on the [[Releases/Rawhide|Rawhide]] branch of the package.
}}
 
* Check out a package:
fedpkg co <source_package_name>
cd <source_package_name>
{{hidden|header=Details|content=This retrieves a copy of the package sources from the server. It's known as your 'working copy'.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Update your checked-out copy from the Fedora server:
fedpkg pull
* Retrieve the package sources:
fedpkg sources
{{hidden|header=Details|content=This pulls any sources stored in the "lookaside cache" (see below for more). Steps like {{command|fedpkg prep}} and {{command|fedpkg srpm}} will do this if necessary, but you may want a copy right away.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Make your changes to the package
{{hidden|header=Details|content=This is not an RPM packaging guide, so we'll assume you know what you're doing here. New sources and patches go in the working copy directory for now.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Run the 'prep' stage (extract source, apply patches etc) within the checkout directory:
fedpkg prep
{{hidden|header=Details|content=This is useful for making sure your patches apply cleanly, and inspecting the source tree if you need to do so.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Do a local build of the current state:
fedpkg local
{{hidden|header=Details|content=This is the simplest kind of test build, but it's usually cleaner and a better test to do a Mock or Koji scratch build (see below).|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Do a mock build of the current state:
fedpkg mockbuild
{{hidden|header=Details|content=This fires off a [https://github.com/rpm-software-management/mock/wiki Mock] build, if you have Mock configured correctly. [[Using_Mock_to_test_package_builds]] can help there. |headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Generate a .src.rpm from the current state:
fedpkg srpm
{{hidden|header=Details|content=You can request a [[Koji]] 'scratch build' (a test build, which will not go to any repository) of the generated .src.rpm with the {{command|koji build --scratch}} command (see {{command|man koji}}).|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Do a scratch build using koji: see [https://fedoraproject.org/wiki/Using_the_Koji_build_system#Scratch_Builds Koji scratch builds]
* Check changes you have made:
fedpkg diff
{{hidden|header=Details|content=This is handy for making sure you didn't touch something by mistake, or forget to bump the release, or forget to include a changelog...|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Run some checks (rpmlint) on your package:
fedpkg lint
{{hidden|header=Details|content=If you want to whitelist some rpmlint errors and prevent them from appearing, you can create an rpmlint config file named <code>&lt;source_package_name&gt;.rpmlintrc</code> and it will get applied.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Stage any small patches or new source files for commit:
git add (somefile)
{{hidden|header=Details|content=git does not consider all files in the working directory to be a part of the git repository by default (handy, for keeping other files around that are relevant, like the source tree). This tells git to start considering these files as part of the repository ''locally''. When you 'commit' and 'push' later, this change is communicated to the server.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Upload new source files to the lookaside cache:
fedpkg new-sources
{{admon/warning|Alert|This will replace the current list of source files, not add to it. See ''Details'' for more details on the lookaside cache system.}}
fedpkg upload
{{hidden|header=Details|content='Pristine' upstream sources (like release tarballs) and other larger source files are stored in the [[Package_Source_Control#Lookaside_Cache|lookaside cache]] system, not committed directly to git. This provides more efficient storage and transfer of the files. The {{filename|sources}} and {{filename|.gitignore}} files in the repository keep it in sync with the lookaside cache. Any time you use {{command|fedpkg new-sources}} or {{command|fedpkg upload}}, you must remember to 'commit' changes to those files.
 
{{command|new-sources}} 'starts from scratch', replacing all files currently in the lookaside cache - you'll typically use this command for many packages with just a single source tarball, each time you update to a new upstream version. {{command|upload}} just adds the given file to those already in the cache. Do remember not to leave stale sources lying around.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Switch to a different release branch:
fedpkg switch-branch <f{{FedoraVersionNumber}}, el6, master>
{{hidden|header=Details|content=Each Fedora release has its own branch in each package repository so different builds can be sent to each release. See below for more details on working with branches.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Generate git changelog from package changelog:
fedpkg clog
{{hidden|header=Details|content=This command extracts your package changelog entry to the file {{filename|clog}}, so you can use it as the git changelog if you like. Some maintainers draw a distinction between the two, some do not.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Commit changes:
fedpkg commit (-F clog) (-p) (-c)
{{admon/note|Difference from git|This behaves by default like {{command|git commit -a}} - it stages modified files and commits all at once, though it ''does not'' add files which git is not yet tracking.}}
{{hidden|header=Details|content=This creates a sort of bundle, a 'commit', of your changes to the repository, with a unique identity and a changelog. Other maintainers - and you yourself, later - can view the history of changes to the repository with the 'commit' as the finest level of detail. It is good practice to use many relatively small commits, each for a single purpose - don't combine a version bump with a bunch of whitespace fixes and some scriptlet changes all in one commit, create separate commits for each.
 
The {{command|-F clog}} parameter will use the {{filename|clog}} file from the previous step as the changelog. {{command|-p}} will push (see below) at the same time as committing. {{command|-c}} combines the {{command|clog}} and {{command|commit -F clog}} steps into one, if you like that.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Push changes:
fedpkg push
{{hidden|header=Details|content=This sends all the new 'commits' in your local working copy to the upstream server. If you are still learning the system, now is a good time to {{command|fedpkg co}} another copy of the repository somewhere else, compare what you get to your working copy, and run a test build on it.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
* Do an 'official' build of the latest pushed changes:
fedpkg build
* Submit 'official' builds from a stream branch
fedpkg build
{{hidden|header=Details|content=There is no difference in the command line to submit multiple builds from a stream branch. But you need to create a config file {{command|package.cfg}} in the repository and set option for the builds. For example config file is created in a stream branch {{command|8}} of package {{command|foo}}, which has content,
[koji]
targets = f28 epel7
This example shows when you execute {{command|build}} command, fedpkg is able to submit builds for releases, {{command|f28}} and {{command|epel7}}.


<pre>
In practice, you are able to specify two shortcut names {{command|fedora}} and {{command|epel}} for convenience. fedpkg retrieves current active Fedora and EPEL releases automatically. Hence, if you don't want to select a subset of releases, or just simply going to build packages for active releases without knowing the concrete release name, shortcut names would be helpful. You can specify to build for {{command|rawhide}}, use name {{command|master}}.
fedpkg clone bzrtools
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
cd bzrtools
* In the event you are working with a Docker [[Changes/Layered_Docker_Image_Build_Service|Container Layered Image Build]]
git merge origin/f13/master
fedpkg container-build
# If no conflicts proceed to the next step.
{{admon/important|Going into production|This is the first point at which you might possibly cause real mess for a real user, so use it with caution. If you are following the example and operating on Rawhide, your build would go live for Rawhide users some few hours after you ran this command.}}
# Otherwise, resolve the conflicts, git add & git commit, and then move on
{{admon/note|Uses pushed state|Unlike most of the above commands, this operates on the ''state you have pushed to git'', not the local state. If you have issues make sure you have pushed and committed all patches and handled the sources correctly.}}
git merge origin/f12/master
{{hidden|header=Details|content=This triggers a 'real' (not scratch) build of your package in [[Koji]]. Depending on the release you are building for, it may go directly to the [[Repositories#stable|''stable'' state]] or it may have to run through the [[Updates Policy|update process]]. See the [[Package_update_HOWTO|package update guide]] for more details on this. The command will output a URL where you can monitor the build's progress in Koji.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
git merge origin/el6/master
* Submit a package update for the latest build (but see [[Package_update_HOWTO#Updating_inter-dependent_packages|Updating inter-dependent packages]] if you are making inter-dependent changes to more than one package):
git merge origin/el5/master
fedpkg update
</pre>
{{hidden|header=Details|content=Again, see the [[Package_update_HOWTO|package update guide]] for more details on this process. This step is not actually applicable to Rawhide, but illustrated here for completeness.|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}


Potentially better alternative to the above (without conflicts, without actually changing the master content, and merging ALL former CVS branches which are supposed to be the same package):
== Typical ''fedpkg'' session ==


A typical session may look like this:
<pre>
<pre>
fedpkg clone menu-cache
fedpkg clone foo
cd menu-cache
cd foo
# implicitly we have the master branch checked out here
fedpkg sources
git merge -m "Initial pseudo merge for dist-git setup" -s ours origin/{f12,f13}/master
fedpkg new-sources foo-0.0.2.tar.bz2
gedit foo.spec      # change the required things in the specfile.
                    # rpmdev-bumpspec is useful for simple version updates
fedpkg mockbuild    # check that the changes you made are correct
fedpkg diff
fedpkg lint
fedpkg commit -p -c  # commit and push in one go
</pre>
</pre>


==== Merging in normal operation ====
== Working with branches ==


Now when you make changes on one branch and want to move them to a different branch, you can use git to merge those changes.
Each Fedora release is represented by a branch in the git repository. You can switch between them like this:
 
fedpkg switch-branch f{{FedoraVersionNumber}}
<pre>
fedpkg switch-branch f{{FedoraVersionNumber|previous}}
fedpkg switch-branch master
The ''master'' branch is for [[Releases/Rawhide|Rawhide]]. You can maintain each branch entirely separately, if you like, laboriously copying changes between them (so long as you always stay within the [[Updates Policy]] requirements). However, git provides us with several handy tools for working with branches. Here's an example:
{{#tag:pre|
fedpkg clone bzrtools
fedpkg clone bzrtools
# Make some changes in the master branch
# Make some changes in the master branch
fedpkg new-sources bzrtools-2.2.tar.gz
fedpkg new-sources bzrtools-2.2.tar.gz
vim bzrtools.spec
gedit bzrtools.spec
git commit
fedpkg commit
fedpkg switch-branch f14
fedpkg switch-branch f{{FedoraVersionNumber}}
git merge master
git merge master
# for push into repo
# for push into repo
git push
fedpkg push
</pre>
}}
This will 'merge' the changes from the ''master'' (Rawhide) branch to the f{{FedoraVersionNumber}} branch. git aficionados may note this is a somewhat unusual workflow, but it is appropriate to the context of package management. Remember, after pushing to and building for a stable release or a [[Releases/Branched|Branched]] release after [[Updates Policy#Bodhi enabling|Bodhi has been enabled]], you will have to [[Package_update_HOWTO|submit an update]] before any other Fedora users will see your build.
 
Note that merges will only be sure to work cleanly so long as the branches have not previously diverged. That is, if you do this:
{{#tag:pre|
fedpkg clone bzrtools
# Make some changes in the master branch
fedpkg commit
fedpkg switch-branch f{{FedoraVersionNumber}}
# Make some changes in the f{{FedoraVersionNumber}} branch
fedpkg commit
fedpkg switch-branch master
# Make some more changes in the master branch
fedpkg commit
fedpkg switch-branch f{{FedoraVersionNumber}}
git merge master
}}
you may encounter a ''merge conflict''.
 
Remember that git is a ''collaborative'' system, and used as such in Fedora package management. It is often the case that you must consider changes made by others in working on a package, and consider how your changes will affect others.
 
=== Resolving merge conflicts ===


==== Resolving merge conflicts ====
This is a large topic and somewhat beyond the scope of this guide, but we can give basic pointers. There are other good references in the [http://git-scm.com/book/en/Git-Branching-Basic-Branching-and-Merging git book] and at [https://help.github.com/articles/resolving-a-merge-conflict-from-the-command-line github].


When you git merge and a conflict occurs you still edit the files that have conflicts. Remove the conflict markers in the files and merge the changes manually. Then you need to commit your changes.  Do that with <code>git add CONFLICTEDFILES ; git commit</code>
When you git merge and a conflict occurs you can edit the files that have conflicts. Remove the conflict markers in the files and merge the changes manually. Use {{command|git diff}} or {{command|fedpkg diff}} to inspect the changes against the pre-conflict state and verify you are happy with the resolution. Then you can commit the files with {{command|fedpkg commit}} or {{command|git commit -a}}. git will know if you have resolved the conflict by checking that all the conflict markers have been removed.


===== Using <code>git mergetool</code> to resolve conflicts =====
=== Using <code>git mergetool</code> to resolve conflicts ===


Git allows you to set a graphical diff program to help resolve conflicts.  This can be handy for visualizing what changes have occurred and deal with them as a set.
Git provides a graphical diff program to help resolve conflicts.  This can be handy for visualizing what changes have occurred and dealing with them as a set.
<pre>
{{#tag:pre|
git config --global merge.tool meld
git config --global merge.tool meld
fedpkg switch-branch f13
fedpkg switch-branch f{{FedoraVersionNumber}}
git merge master
git merge master
# Conflicts occurred
# Conflicts occurred
git mergetool # Opens up a meld showing a three way diff of the merge, working tree, and the last commit
git mergetool           # Opens up a meld showing a three way diff of  
                        # the merge, working tree, and the last commit
# Resolved all the conflicts in the GUI
# Resolved all the conflicts in the GUI
git add CONFLICTEDFILES
git add CONFLICTEDFILES
git commit
git commit
</pre>
}}
 
== Requesting special dist tags ==
 
When a change to a package affects a large number of dependencies (e.g. all perl, python, ruby or ghc packages), requiring them to be rebuilt, it may be better to initially do the builds in a special repository, so that there is less disruption in Rawhide.
 
If you think you have an update that falls under this case you can request a special dist tag by filing a [https://fedorahosted.org/rel-eng/newticket release engineering ticket]. Someone from [[ReleaseEngineering|release engineering]] will likely want to discuss your needs to make sure this is really an appropriate case (it's OK ask if you aren't sure) and that you get what you need.
 
== Tips and tricks ==
 
{{anchor|anon}}
=== Using fedpkg anonymously ===
 
You can use fedpkg like this:
fedpkg clone --anonymous <somepackage>
to check out a package without requiring identification. Obviously, you will not be able to push any changes to this repository, but it is useful for non-packagers who simply want to examine a package, make changes for their own use, and perhaps submit changes to a Fedora developer.
 
=== Local branch names ===
 
If you use git commands to branch and checkout directly, you can define whatever local branch names you want. If you use {{command|fedpkg switch-branch}}, it will default to creating the names used in the examples above.


== Branch names ==
=== Current branch and state in shell prompt ===
{|
! Old CVS !! new remote git !! fedpkg local branch names !! Notes
|-
| devel || origin/master || master || This is the default when you do fedpkg co
|-
| F-16|| origin/f16/master || f16 ||
|-
| F-15 || origin/f15/master || f15 ||
|-
| F-14 || origin/f14/master || f14 ||
|-
| EL-6 || origin/el6/master || el6 ||
|-
| EL-5 || origin/el5/master || el5 ||
|}


{{admon/note|local branch names| If you use git commands to branch and checkout directly, you can define whatever local branch names you wantIf you use <code>fedpkg switch-branch</code>, it will default to creating the names listed in the table above. So you can do <code>fedpkg clone firefox ; fedpkg switch-branch f14</code> and then you'll have a new local branch named <code>f14</code>}}
It is often helpful to know what branch you are working on at a glanceYou can add this information to your bash prompt with the information [[Git_Quickref#Display_current_branch_in_bash|here]].


{{admon/tip|Display current branch in your shell prompt| It is often helpful to know what branch you are working on at a glance. You can add this information to your bash prompt with the information [[Git_Quickref#Display_current_branch_in_bash|here]].}}
=== Importing a .src.rpm to update ===


== Errors ==
The {{command|fedpkg import}} command usually used to initially populate a git package repository from a .src.rpm that has been through the [[Package Review Process]] can also be used to update a normal working copy, if you have an old-school packaging process to which you are particularly attached. Just run {{command|fedpkg import file.src.rpm}} and it will upload new tarballs into lookaside cache, update a working copy of the last version found in git, and commit all changes. {{command|fedpkg import --help}} documents some other parameters it can accept.


=== Access denied ===
{{admon/warning|Caution!|This approach makes it harder to verify that your changes are safe and do not overwrite changes made to the package by others. For this reason, its use is not recommended.}}


During the next few days and weeks, occasional tweaking to the servers might cause short periods (seconds or minutes) when the git server refuses you access. Just try again.
=== Making changes on an older branch without breaking the upgrade path ===


=== Unable to switch ===
Here is the scenario: you've built your package successfully on the ''{{FedoraVersionNumber}}'' branch, but there is a problem keeping your package from building on ''{{FedoraVersionNumber|last}}''.


If you are trying switch to another branch by fedpkg switch-branch f13 and see this:
Solution: make your changes in the branch and then add a digit to the very right of the release tag. There is no need to change the release in the other branches. This allows upgrades to work smoothly if the user upgrades to a newer release of Fedora.
Unable to switch to another branch: Could not create branch f13


Then you probably have some un-commited changes in master. If you want them, call:
<pre>
<pre>
# this "saves" your changes
Name:    foo
git stash
Version: 1.0
# switch to preferred branch, now possible
Release: 1%{?dist}
fedpkg switch-branch f13
 
# apply "saved" changes into the current branch
Name:    foo
git stash apply
Version: 1.0
Release: 1%{?dist}.1
</pre>
</pre>


== ssh fingerprint ==
Then tag and build as usual. This approach was initially discussed [https://www.redhat.com/archives/fedora-extras-list/2006-May/msg00083.html in this mailing list thread].
 
=== Removing a package build pending for [[Releases/Rawhide|Rawhide]] or [[Releases/Branched|Branched]] ===
 
From time to time you may want to remove a package build you submitted to Rawhide or to Branched prior to the Alpha freeze (both cases where the build would usually go out to the main [[Repositories|repository]] without further gating). This could happen in a situation where a bug or issue is found in your package that will be resolved upstream in the next release, or you realize you made a significant mistake in the build that cannot easily be corrected.
 
{{admon/caution|Use this carefully!|This should only be done on the same day of the build, before it is included in a compose. If your build was already included in a compose you must not untag it! Check the [https://apps.fedoraproject.org/releng-dash/ Release Engineering Dashboard] to get the starting time of the last compose.}}
 
You can remove the package by using [[Koji]]: {{command|koji untag-pkg f{{FedoraVersionNumber|next}} foo-1.1.3-1.fc{{FedoraVersionNumber|next}}}}
 
where {{filename|foo-1.1.3-1.fc{{FedoraVersionNumber|next}}}} is replaced with the name of your package build.  See {{command|koji help}} or [[Using_the_Koji_build_system|using Koji]] for more information.
 
=== ssh fingerprint ===


The recommended option is to include "<code>VerifyHostKeyDNS yes</code>" in your ~/.ssh/config file. This will result in using DNS to check that the key is correct.
The recommended option is to include "<code>VerifyHostKeyDNS yes</code>" in your ~/.ssh/config file. This will result in using DNS to check that the key is correct.


But you can also manually check against the  list of keys at https://admin.fedoraproject.org . The strings there are what ends up in your ~/.ssh/known_hosts file. So you can accept the fingerprint when prompted and then check that the correct string for pkgs.fedoraproject.org ended up in your ~/.ssh/known_hosts file.
But you can also manually check against the  list of keys at https://admin.fedoraproject.org . The strings there are what ends up in your ~/.ssh/known_hosts file. So you can accept the fingerprint when prompted and then check that the correct string for src.fedoraproject.org ended up in your ~/.ssh/known_hosts file.
 
=== Problems connecting to the repository ===
 
The ''fedpkg'' tool clones repositories using the ssh:// protocol, so this should not be a problem normally (as long as you have your ssh key).  If you cloned using the ''git'' utility itself, check the {{filename|.git/config}} file to ensure the remote repository is being accessed via an ssh:// protocol, and not git://.
 
=== It builds here, why doesn't it build there? ===
 
Is your package building locally - even with Mock, even as a scratch build! - but not when you run {{command|fedpkg build}}? Before you get too frustrated, remember {{command|fedpkg build}} runs on the package as it exists in the upstream repository, not your local working copy. Make sure you have committed and pushed all changes and source files, and handled the lookaside cache correctly. Other issues that have been reported, are issues because of [https://bugzilla.redhat.com/show_bug.cgi?id=1179139 build/make check parallelization] and failures because of test suites that depend on operations finish on precise timing (and a busy build system may not be able to perform operations on time).


== References ==
== References ==


* http://pkgs.fedoraproject.org/gitweb/ Listing 10K+ packages is slow, though, so use e.g. http://pkgs.fedoraproject.org/gitweb/?p=erlang.git for direct access to the erlang package.
* https://src.fedoraproject.org/
* [[Package SCM admin requests]]  
* [[Infrastructure/Kerberos]]
* [[Package_Renaming_Process]]  
* [[Package_Renaming_Process]]  
* [[PackageMaintainers/PackagingTricks]]
* [[PackageMaintainers/PackagingTricks]]
Line 186: Line 260:
* [[PackageMaintainers/BuildSystemClientSetup#Install_the_Client_Tools_.28Koji.29]]  
* [[PackageMaintainers/BuildSystemClientSetup#Install_the_Client_Tools_.28Koji.29]]  
* [[PackageMaintainers/MockTricks#How_do_I_use_Mock.3F]]  
* [[PackageMaintainers/MockTricks#How_do_I_use_Mock.3F]]  
* [[Using_the_Koji_build_system]]
* [[Package_Review_Process]]  
* [[Package_Review_Process]]  
* [[Fedora_Release_Life_Cycle]]  
* [[Fedora_Release_Life_Cycle]]  
* [[PackageMaintainers/Join]]
* [[PackageMaintainers/Join]]
* [[Infrastructure/VersionControl/dist-git]]
* [[Infrastructure/VersionControl/dist-git]]
[[Category:Package Maintainers]]

Revision as of 06:38, 27 April 2020

This page provides some basic instructions for day-to-day usage of the git-based package maintenance system for Fedora. It is intended primarily for new and current Fedora package maintainers, but does briefly cover anonymous read-only use of the system. It is not a guide to RPM packaging per se. Some pre-existing knowledge of git may be useful, but is not a pre-requisite (in fact, Fedora packaging can be a relatively painless introduction to it).

You may have been looking for, or also be interested in:

Installing fedpkg and doing initial setup

Start by installing Package-x-generic-16.pngfedpkg with dnf install fedpkg. This will be your primary interface to the packaging system.

You also must have an ssh key configured in the Fedora Accounts System to be able to make changes to any package (including your own). fedpkg will expect the correct ssh key to be available in your keyring.

Before uploading sources with new-sources and upload and building packages in Koji, with build for example, you have to get your Kerberos credential first with kinit, e.g.

 kinit [FAS name]@FEDORAPROJECT.ORG

(Keep FEDORAPROJECT.ORG in capital case)

Set your Fedora Account System username in ~/.fedora.upn. You can do this via echo "FAS_USERNAME" > ~/.fedora.upn.

Common fedpkg commands

This section lists typical fedpkg commands in a normal workflow, with short descriptions. In this workflow, we will be operating on the Rawhide branch of the package.

  • Check out a package:
fedpkg co <source_package_name>
cd <source_package_name>
Details

This retrieves a copy of the package sources from the server. It's known as your 'working copy'.

  • Update your checked-out copy from the Fedora server:
fedpkg pull
  • Retrieve the package sources:
fedpkg sources
Details

This pulls any sources stored in the "lookaside cache" (see below for more). Steps like fedpkg prep and fedpkg srpm will do this if necessary, but you may want a copy right away.

  • Make your changes to the package
Details

This is not an RPM packaging guide, so we'll assume you know what you're doing here. New sources and patches go in the working copy directory for now.

  • Run the 'prep' stage (extract source, apply patches etc) within the checkout directory:
fedpkg prep
Details

This is useful for making sure your patches apply cleanly, and inspecting the source tree if you need to do so.

  • Do a local build of the current state:
fedpkg local
Details

This is the simplest kind of test build, but it's usually cleaner and a better test to do a Mock or Koji scratch build (see below).

  • Do a mock build of the current state:
fedpkg mockbuild
Details

This fires off a Mock build, if you have Mock configured correctly. Using_Mock_to_test_package_builds can help there.

  • Generate a .src.rpm from the current state:
fedpkg srpm
Details

You can request a Koji 'scratch build' (a test build, which will not go to any repository) of the generated .src.rpm with the koji build --scratch command (see man koji).

fedpkg diff
Details

This is handy for making sure you didn't touch something by mistake, or forget to bump the release, or forget to include a changelog...

  • Run some checks (rpmlint) on your package:
fedpkg lint
Details

If you want to whitelist some rpmlint errors and prevent them from appearing, you can create an rpmlint config file named <source_package_name>.rpmlintrc and it will get applied.

  • Stage any small patches or new source files for commit:
git add (somefile)
Details

git does not consider all files in the working directory to be a part of the git repository by default (handy, for keeping other files around that are relevant, like the source tree). This tells git to start considering these files as part of the repository locally. When you 'commit' and 'push' later, this change is communicated to the server.

  • Upload new source files to the lookaside cache:
fedpkg new-sources
Warning.png
Alert
This will replace the current list of source files, not add to it. See Details for more details on the lookaside cache system.
fedpkg upload
Details

'Pristine' upstream sources (like release tarballs) and other larger source files are stored in the lookaside cache system, not committed directly to git. This provides more efficient storage and transfer of the files. The sources and .gitignore files in the repository keep it in sync with the lookaside cache. Any time you use fedpkg new-sources or fedpkg upload, you must remember to 'commit' changes to those files.

new-sources 'starts from scratch', replacing all files currently in the lookaside cache - you'll typically use this command for many packages with just a single source tarball, each time you update to a new upstream version. upload just adds the given file to those already in the cache. Do remember not to leave stale sources lying around.

  • Switch to a different release branch:
fedpkg switch-branch <f39, el6, master>
Details

Each Fedora release has its own branch in each package repository so different builds can be sent to each release. See below for more details on working with branches.

  • Generate git changelog from package changelog:
fedpkg clog
Details

This command extracts your package changelog entry to the file clog, so you can use it as the git changelog if you like. Some maintainers draw a distinction between the two, some do not.

  • Commit changes:
fedpkg commit (-F clog) (-p) (-c)
Note.png
Difference from git
This behaves by default like git commit -a - it stages modified files and commits all at once, though it does not add files which git is not yet tracking.
Details

This creates a sort of bundle, a 'commit', of your changes to the repository, with a unique identity and a changelog. Other maintainers - and you yourself, later - can view the history of changes to the repository with the 'commit' as the finest level of detail. It is good practice to use many relatively small commits, each for a single purpose - don't combine a version bump with a bunch of whitespace fixes and some scriptlet changes all in one commit, create separate commits for each.

The -F clog parameter will use the clog file from the previous step as the changelog. -p will push (see below) at the same time as committing. -c combines the clog and commit -F clog steps into one, if you like that.

  • Push changes:
fedpkg push
Details

This sends all the new 'commits' in your local working copy to the upstream server. If you are still learning the system, now is a good time to fedpkg co another copy of the repository somewhere else, compare what you get to your working copy, and run a test build on it.

  • Do an 'official' build of the latest pushed changes:
fedpkg build
  • Submit 'official' builds from a stream branch
fedpkg build
Details

There is no difference in the command line to submit multiple builds from a stream branch. But you need to create a config file package.cfg in the repository and set option for the builds. For example config file is created in a stream branch 8 of package foo, which has content,

[koji]
targets = f28 epel7

This example shows when you execute build command, fedpkg is able to submit builds for releases, f28 and epel7.

In practice, you are able to specify two shortcut names fedora and epel for convenience. fedpkg retrieves current active Fedora and EPEL releases automatically. Hence, if you don't want to select a subset of releases, or just simply going to build packages for active releases without knowing the concrete release name, shortcut names would be helpful. You can specify to build for rawhide, use name master.

fedpkg container-build
Important.png
Going into production
This is the first point at which you might possibly cause real mess for a real user, so use it with caution. If you are following the example and operating on Rawhide, your build would go live for Rawhide users some few hours after you ran this command.
Note.png
Uses pushed state
Unlike most of the above commands, this operates on the state you have pushed to git, not the local state. If you have issues make sure you have pushed and committed all patches and handled the sources correctly.
Details

This triggers a 'real' (not scratch) build of your package in Koji. Depending on the release you are building for, it may go directly to the stable state or it may have to run through the update process. See the package update guide for more details on this. The command will output a URL where you can monitor the build's progress in Koji.

fedpkg update
Details

Again, see the package update guide for more details on this process. This step is not actually applicable to Rawhide, but illustrated here for completeness.

Typical fedpkg session

A typical session may look like this:

fedpkg clone foo
cd foo
fedpkg sources
fedpkg new-sources foo-0.0.2.tar.bz2
gedit foo.spec       # change the required things in the specfile.
                     # rpmdev-bumpspec is useful for simple version updates
fedpkg mockbuild     # check that the changes you made are correct
fedpkg diff
fedpkg lint
fedpkg commit -p -c  # commit and push in one go

Working with branches

Each Fedora release is represented by a branch in the git repository. You can switch between them like this:

fedpkg switch-branch f39
fedpkg switch-branch f38
fedpkg switch-branch master

The master branch is for Rawhide. You can maintain each branch entirely separately, if you like, laboriously copying changes between them (so long as you always stay within the Updates Policy requirements). However, git provides us with several handy tools for working with branches. Here's an example:

fedpkg clone bzrtools
# Make some changes in the master branch
fedpkg new-sources bzrtools-2.2.tar.gz
gedit bzrtools.spec
fedpkg commit
fedpkg switch-branch f39
git merge master
# for push into repo
fedpkg push

This will 'merge' the changes from the master (Rawhide) branch to the f39 branch. git aficionados may note this is a somewhat unusual workflow, but it is appropriate to the context of package management. Remember, after pushing to and building for a stable release or a Branched release after Bodhi has been enabled, you will have to submit an update before any other Fedora users will see your build.

Note that merges will only be sure to work cleanly so long as the branches have not previously diverged. That is, if you do this:

fedpkg clone bzrtools
# Make some changes in the master branch
fedpkg commit
fedpkg switch-branch f39
# Make some changes in the f39 branch
fedpkg commit
fedpkg switch-branch master
# Make some more changes in the master branch
fedpkg commit
fedpkg switch-branch f39
git merge master

you may encounter a merge conflict.

Remember that git is a collaborative system, and used as such in Fedora package management. It is often the case that you must consider changes made by others in working on a package, and consider how your changes will affect others.

Resolving merge conflicts

This is a large topic and somewhat beyond the scope of this guide, but we can give basic pointers. There are other good references in the git book and at github.

When you git merge and a conflict occurs you can edit the files that have conflicts. Remove the conflict markers in the files and merge the changes manually. Use git diff or fedpkg diff to inspect the changes against the pre-conflict state and verify you are happy with the resolution. Then you can commit the files with fedpkg commit or git commit -a. git will know if you have resolved the conflict by checking that all the conflict markers have been removed.

Using git mergetool to resolve conflicts

Git provides a graphical diff program to help resolve conflicts. This can be handy for visualizing what changes have occurred and dealing with them as a set.

git config --global merge.tool meld
fedpkg switch-branch f39
git merge master
# Conflicts occurred
git mergetool            # Opens up a meld showing a three way diff of 
                         # the merge, working tree, and the last commit
# Resolved all the conflicts in the GUI
git add CONFLICTEDFILES
git commit

Requesting special dist tags

When a change to a package affects a large number of dependencies (e.g. all perl, python, ruby or ghc packages), requiring them to be rebuilt, it may be better to initially do the builds in a special repository, so that there is less disruption in Rawhide.

If you think you have an update that falls under this case you can request a special dist tag by filing a release engineering ticket. Someone from release engineering will likely want to discuss your needs to make sure this is really an appropriate case (it's OK ask if you aren't sure) and that you get what you need.

Tips and tricks

Using fedpkg anonymously

You can use fedpkg like this:

fedpkg clone --anonymous <somepackage>

to check out a package without requiring identification. Obviously, you will not be able to push any changes to this repository, but it is useful for non-packagers who simply want to examine a package, make changes for their own use, and perhaps submit changes to a Fedora developer.

Local branch names

If you use git commands to branch and checkout directly, you can define whatever local branch names you want. If you use fedpkg switch-branch, it will default to creating the names used in the examples above.

Current branch and state in shell prompt

It is often helpful to know what branch you are working on at a glance. You can add this information to your bash prompt with the information here.

Importing a .src.rpm to update

The fedpkg import command usually used to initially populate a git package repository from a .src.rpm that has been through the Package Review Process can also be used to update a normal working copy, if you have an old-school packaging process to which you are particularly attached. Just run fedpkg import file.src.rpm and it will upload new tarballs into lookaside cache, update a working copy of the last version found in git, and commit all changes. fedpkg import --help documents some other parameters it can accept.

Warning.png
Caution!
This approach makes it harder to verify that your changes are safe and do not overwrite changes made to the package by others. For this reason, its use is not recommended.

Making changes on an older branch without breaking the upgrade path

Here is the scenario: you've built your package successfully on the 39 branch, but there is a problem keeping your package from building on last.

Solution: make your changes in the branch and then add a digit to the very right of the release tag. There is no need to change the release in the other branches. This allows upgrades to work smoothly if the user upgrades to a newer release of Fedora.

Name:    foo
Version: 1.0
Release: 1%{?dist}

Name:    foo
Version: 1.0
Release: 1%{?dist}.1

Then tag and build as usual. This approach was initially discussed in this mailing list thread.

Removing a package build pending for Rawhide or Branched

From time to time you may want to remove a package build you submitted to Rawhide or to Branched prior to the Alpha freeze (both cases where the build would usually go out to the main repository without further gating). This could happen in a situation where a bug or issue is found in your package that will be resolved upstream in the next release, or you realize you made a significant mistake in the build that cannot easily be corrected.

Stop (medium size).png
Use this carefully!
This should only be done on the same day of the build, before it is included in a compose. If your build was already included in a compose you must not untag it! Check the Release Engineering Dashboard to get the starting time of the last compose.

You can remove the package by using Koji: koji untag-pkg f40 foo-1.1.3-1.fc40

where foo-1.1.3-1.fc40 is replaced with the name of your package build. See koji help or using Koji for more information.

ssh fingerprint

The recommended option is to include "VerifyHostKeyDNS yes" in your ~/.ssh/config file. This will result in using DNS to check that the key is correct.

But you can also manually check against the list of keys at https://admin.fedoraproject.org . The strings there are what ends up in your ~/.ssh/known_hosts file. So you can accept the fingerprint when prompted and then check that the correct string for src.fedoraproject.org ended up in your ~/.ssh/known_hosts file.

Problems connecting to the repository

The fedpkg tool clones repositories using the ssh:// protocol, so this should not be a problem normally (as long as you have your ssh key). If you cloned using the git utility itself, check the .git/config file to ensure the remote repository is being accessed via an ssh:// protocol, and not git://.

It builds here, why doesn't it build there?

Is your package building locally - even with Mock, even as a scratch build! - but not when you run fedpkg build? Before you get too frustrated, remember fedpkg build runs on the package as it exists in the upstream repository, not your local working copy. Make sure you have committed and pushed all changes and source files, and handled the lookaside cache correctly. Other issues that have been reported, are issues because of build/make check parallelization and failures because of test suites that depend on operations finish on precise timing (and a busy build system may not be able to perform operations on time).

References