From Fedora Project Wiki
(Created page with "= BuildRequires Generators = == Summary == Add possibility to generate build-time dependencies within RPM spec file and teach RPM and mock how to handle this. == Owner == <!...")
 
(38 intermediate revisions by 5 users not shown)
Line 1: Line 1:
= BuildRequires Generators =
= DynamicBuildRequires =


== Summary ==
== Summary ==
Line 9: Line 9:
This should link to your home wiki page so we know who you are.  
This should link to your home wiki page so we know who you are.  
-->
-->
* Name: [[User:ignatenkobrain|Igor Gnatenko]], [[User:ffesti|Florian Festi]]
* Name: [[User:ignatenkobrain|Igor Gnatenko]], [[User:ffesti|Florian Festi]], [[User:msuchy|Miroslav Suchý]]
* Email: ignatenkobrain@fedoraproject.org, ffesti@redhat.com
* Email: ignatenkobrain@fedoraproject.org, ffesti@redhat.com, miroslav@suchy.cz
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->


Line 16: Line 16:
* Targeted release: [[Releases/31|Fedora 31]]
* Targeted release: [[Releases/31|Fedora 31]]
* Last updated: {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}
* Last updated: {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1704773 #1704773]
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/335 #335]


== Detailed Description ==
== Detailed Description ==
For many languages (Rust, Golang, Node.Js, Ruby, Pyton), BuildRequires can be automatically generated. All it takes, run some special tool which will output dependencies in RPM format.
For many languages (Rust, Golang, Node.Js, Ruby, Python, Haskell), BuildRequires can be automatically generated. All it takes, run some special tool which will output dependencies in RPM format.


How will it work under the hood?
=== Changes in tooling ===


When you build RPM:
==== rpmbuild ====
1. rpm would perform %prep (which is supposed to abort if some dependencies missing and print them)
 
2. mock would install those dependencies and resume build
* New <code>%generate_buildrequires</code> section ("script" later in text) in spec file
* New option <code>-br</code> which will run script, check dynamic BuildRequires
** If all of them are satisfied, generate <code>src.rpm</code>
** If some of them are not satisfied or <code>--nodeps</code> is present, generate <code>buildreqs.nosrc.rpm</code>
* <code>src.rpm</code> produced by <code>rpmbuild -ba</code> will contain dynamic BuildRequires
* Option <code>-bs</code> does not change behavior
* <code>src.rpm</code> produced from spec file which has <code>%generate_buildrequires</code> has <code>Requires: rpmlib(DynamicBuildRequires)</code>
* <code>src.rpm</code> which contain dynamic BuildRequires has <code>Provides: rpmlib(DynamicBuildRequires)</code> for identification purposes
 
==== mock ====
 
* New option in config <code>dynamic_buildrequires</code> which controls support of new features
* Support for rpmbuild changes if this option enabled
 
=== Example ===
 
Probably the easiest demonstration of how this works is to show an example.
 
Create this simple SPEC file `~/rpmbuild/SPECS/dynamic-buildrequires.spec`:
<pre>
Name:    dynamic-buildrequires
Version:  1
Release:  1%{?dist}
Summary:  Demonstration of Dynamic BuildRequires
License:  GPLv3+
 
# static buildrequires
BuildRequires: bash
 
%description
This package demonstrate Dynamic BuildRequires
https://fedoraproject.org/wiki/Changes/DynamicBuildRequires
 
%generate_buildrequires
echo fruit-cake
 
%files                                                                                                                                           
 
%changelog
* Thu Jun 13 2019 Miroslav Suchy <msuchy@redhat.com> - 1-2
- initial version
</pre>
 
This SPEC file has a new section `%generate_buildrequires` which contains a script. The output of this script is interpreted as additional BuildRequires. The list accrues to classic BuildRequires tags. In this example we dynamically added BuildRequires of `fruit-cake` (which does not exist).
 
Now if you try to build this package:
 
<pre>
# rpmbuild -ba dynamic-buildrequires.spec
Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.JMyoKz
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd /root/rpmbuild/BUILD
+ echo fruit-cake
+ exit 0
error: Failed build dependencies:
        fruit-cake is needed by dynamic-buildrequires-1-1.fc31.x86_64
Wrote: /rpmbuild/SRPMS/dynamic-buildrequires-1-1.fc31.buildreqs.nosrc.rpm
 
# echo $?
11
</pre>
 
Two things happened: `rpmbuild` exited with exit code 11. This indicates the unsatisfied dynamic BuildRequires. If our script would produce a line with 'bash' we would likely have it already installed and `rpmbuild` would proceed further without failure.
Additionally, And notice that this is 'buildreqs.nosrc.rpm'. If you would extract the content of the NOSRC.RPM it would contain the original SPEC file, but there would be no sources attached. But most importantly - the requires of this package include the dynamically generated dependencies:
 
<pre>
# rpm -qpR rpmbuild/SRPMS/dynamic-buildrequires-1-1.fc31.buildreqs.nosrc.rpm
bash
fruit-cake
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(DynamicBuildRequires) <= 4.15.0-1
rpmlib(FileDigests) <= 4.6.0-1
</pre>
 
==== Rust Example ====
 
First, let's make necessary changes in our spec file (I'm taking <code>rust-rpick</code>).
 
<pre>
diff --git a/rust-rpick.spec b/rust-rpick.spec
index 716a185..01a6d35 100644
--- a/rust-rpick.spec
+++ b/rust-rpick.spec
@@ -16,12 +16,6 @@ Source:        %{crates_source}
ExclusiveArch:  %{rust_arches}
BuildRequires:  rust-packaging
-BuildRequires:  (crate(dirs/default) >= 1.0.0 with crate(dirs/default) < 2.0.0)
-BuildRequires:  (crate(rand/default) >= 0.6.0 with crate(rand/default) < 0.7.0)
-BuildRequires:  (crate(serde/default) >= 1.0.0 with crate(serde/default) < 2.0.0)
-BuildRequires:  (crate(serde/derive) >= 1.0.0 with crate(serde/derive) < 2.0.0)
-BuildRequires:  (crate(serde_yaml/default) >= 0.8.0 with crate(serde_yaml/default) < 0.9.0)
-BuildRequires:  (crate(structopt/default) >= 0.2.0 with crate(structopt/default) < 0.3.0)
%global _description \
Helps you pick items from a list by various algorithms. Example uses: pick a\
@@ -69,6 +63,9 @@ which use "default" feature of "%{crate}" crate.
%autosetup -n %{crate}-%{version_no_tilde} -p1
%cargo_prep
+%generate_buildrequires
+%cargo_generate_buildrequires
+
%build
%cargo_build
</pre>
 
Then let's try to build package (I will skip parts like rustc executions because it is irrelevant).
 
<pre>
⋊> ~/P/s/r/rust-rpick on master ⨯ fedpkg mockbuild --mock-config $PWD/fedora-rawhide-x86_64-buildreqs.cfg
Wrote: /home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm
INFO: mock.py version 1.4.14 starting (python version = 3.7.3)...
[…]
INFO: Start(/home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm)  Config(fedora-rawhide-x86_64-buildreqs)
[…]
Start: build phase for rust-rpick-0.3.0-1.fc31.src.rpm
Start: build setup for rust-rpick-0.3.0-1.fc31.src.rpm
Building target platforms: x86_64
Building for target x86_64
Wrote: /builddir/build/SRPMS/rust-rpick-0.3.0-1.fc31.src.rpm
Repository 'kkt' is missing name in configuration, using id.
fedora                                          21 kB/s |  19 kB    00:00   
kkt                                            2.6 MB/s | 3.0 kB    00:00   
Dependencies resolved.
================================================================================
Package                    Arch    Version                    Repo      Size
================================================================================
Installing:
rust-packaging            x86_64  6-28.fc31+buildreqs        kkt      13 k
Installing dependencies:
[…]
Finish: build setup for rust-rpick-0.3.0-1.fc31.src.rpm
Start: rpmbuild rust-rpick-0.3.0-1.fc31.src.rpm
Start: Outputting list of installed packages
Finish: Outputting list of installed packages
Building target platforms: x86_64
Building for target x86_64
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.0m57Wu
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ rm -rf rpick-0.3.0
+ /usr/bin/gzip -dc /builddir/build/SOURCES/rpick-0.3.0.crate
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd rpick-0.3.0
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ set -eu
+ /usr/bin/mkdir -p .cargo
+ cat
+ /usr/bin/rm -f Cargo.lock
+ exit 0
Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.LooeBv
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ cd rpick-0.3.0
+ /usr/bin/cargo-inspector -BR Cargo.toml
+ /usr/bin/cargo-inspector -TR Cargo.toml
+ exit 0
Wrote: /builddir/build/SRPMS/rust-rpick-0.3.0-1.fc31.buildreqs.nosrc.rpm
INFO: Dynamic buildrequires detected
Repository 'kkt' is missing name in configuration, using id.
fedora                                          17 kB/s |  19 kB    00:01   
kkt                                            2.8 MB/s | 3.0 kB    00:00   
Package rust-packaging-6-28.fc31+buildreqs.x86_64 is already installed.
Dependencies resolved.
================================================================================
Package                                  Arch    Version        Repo    Size
================================================================================
Installing:
rust-dirs+default-devel                  noarch  1.0.5-1.fc31    fedora  7.6 k
rust-rand+default-devel                  noarch  0.6.5-3.fc31    fedora  8.4 k
rust-serde+default-devel                noarch  1.0.90-1.fc31  fedora  12 k
rust-serde+derive-devel                  noarch  1.0.90-1.fc31  fedora  12 k
rust-serde_yaml+default-devel            noarch  0.8.8-2.fc30    fedora  8.3 k
rust-structopt+default-devel            noarch  0.2.15-2.fc31  fedora  8.2 k
Installing dependencies:
[…]
Building target platforms: x86_64
Building for target x86_64
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.l2cC5s
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ rm -rf rpick-0.3.0
+ /usr/bin/gzip -dc /builddir/build/SOURCES/rpick-0.3.0.crate
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd rpick-0.3.0
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ set -eu
+ /usr/bin/mkdir -p .cargo
+ cat
+ /usr/bin/rm -f Cargo.lock
+ exit 0
Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.cvhwuw
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ cd rpick-0.3.0
+ /usr/bin/cargo-inspector -BR Cargo.toml
+ /usr/bin/cargo-inspector -TR Cargo.toml
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.dlI3ax
+ umask 022
+ cd /builddir/build/BUILD
+ cd rpick-0.3.0
+ /usr/bin/env CARGO_HOME=.cargo RUSTC_BOOTSTRAP=1 /usr/bin/cargo build -j8 -Z avoid-dev-deps --release
[…]
Finish: build phase for rust-rpick-0.3.0-1.fc31.src.rpm
INFO: Done(/home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm) Config(fedora-rawhide-x86_64-buildreqs) 2 minutes 9 seconds
INFO: Results and/or logs in: /home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/results_rust-rpick/0.3.0/1.fc31
INFO: Cleaning up build root ('cleanup_on_success=True')
Start: clean chroot
Finish: clean chroot
Finish: run
</pre>
 
Let's check dependencies of src.rpm.
 
<pre>
⋊> ~/P/s/r/rust-rpick on master ⨯ rpm -qpR $PWD/results_rust-rpick/0.3.0/1.fc31/rust-rpick-0.3.0-1.fc31.src.rpm
(crate(dirs/default) >= 1.0.0 with crate(dirs/default) < 2.0.0)
(crate(rand/default) >= 0.6.0 with crate(rand/default) < 0.7.0)
(crate(serde/default) >= 1.0.0 with crate(serde/default) < 2.0.0)
(crate(serde/derive) >= 1.0.0 with crate(serde/derive) < 2.0.0)
(crate(serde_yaml/default) >= 0.8.0 with crate(serde_yaml/default) < 0.9.0)
(crate(structopt/default) >= 0.2.0 with crate(structopt/default) < 0.3.0)
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(DynamicBuildRequires) <= 4.15.0-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(RichDependencies) <= 4.12.0-1
rust-packaging
</pre>
 
==== Go Example ====
 
For Go language, you can use `%go_generate_buildrequires` macro. See full example here: https://pagure.io/go-rpm-macros/blob/master/f/templates/rpm/spectemplate-go-0-source-minimal.spec#_103
 
==== Python ====
 
    BuildRequires: pyproject-rpm-macros
   
    %generate_buildrequires
    %pyproject_buildrequires
 
This will read the dependencies from `setup_requires` from setup.py. If you want to read the dependencies from `install_requires` and use them as BuildRequires then you have to use `%pyproject_buildrequires -r`.
 
For more information see https://src.fedoraproject.org/rpms/pyproject-rpm-macros


== Benefit to Fedora ==
== Benefit to Fedora ==
Packagers won't have to pre-generate BuildRequires in the spec file which means it will be always updated (and correct).
 
Packagers won't have to pre-generate BuildRequires in the spec file which means it will be always updated (and correct) :
 
* Packagers can focus of making their packages better instead of  spending all their packaging time copying BuildRequires from documentation and third party tools.
* BuildRequires are dropped as soon as they're no longer necessary
* Packages can be easily bumped without requiring a manual BuildRequires refresh
* BuildRequires and Requires generation can use similar utilities, making sure that the deps packages declare can also be used for second-level building. Packages no longer need to declare the deps of their second and n-th dependencies because someone forgot to declare them in the correct package.


== Scope ==
== Scope ==
* Proposal owners: Implement support for a feature in RPM and mock (if implemented properly, Koji should just work).
* Proposal owners: Implement support for a feature in RPM<sup>[https://github.com/rpm-software-management/rpm/pull/593 1]</sup> and mock<sup>[https://github.com/rpm-software-management/mock/pull/248 1]</sup>. Make use of it in interested ecosystems.
* Other developers: Maintainers of language stacks are advised to use this feature.
* Other developers: Maintainers of language stacks are advised to use this feature.
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Release engineering: [https://pagure.io/releng/issue/8129 #8129]
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A
* Policies and guidelines: Packaging Guidelines need to be updated with instructions how to use this feature.
* Policies and guidelines: Packaging Guidelines need to be updated with instructions how to use this feature.
Line 39: Line 300:


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
Packagers and users who use repoquery might be affected (TBD, fill how exactly they are).
Packagers and users who use repoquery might be affected (src.rpm might not contain generated dependencies).
<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->


== How To Test ==
== How To Test ==
<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.  
# Make sure you have new <code>rpm-build</code> (OPTION 1) or <code>mock</code> (OPTION 2).
# Add <code>%generate_buildrequires</code> script into spec file which prints dependencies, remove static BuildRequires
# Build package
## OPTION1: Run <code>rpmbuild -ba</code> with your usual workflow
## OPTION2: Run <code>mock -r fedora-rawhide-x86_64</code> with your usual workflow
# Check that your package builds fine, dependencies are getting checked/installed and final src.rpm contains generated BuildRequires


Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
== User Experience ==


A good "how to test" should answer these four questions:
For people/tooling who build RPMs, it might be needed to run <code>dnf builddep</code> on new file generated by RPM when dynamic BuildRequires detected and unsatisfied ones found - <code>buildreqs.nosrc.rpm</code>. RPM will guide you where it stored it and which name file has.
 
0. What special hardware / data / etc. is needed (if any)?
1. How do I prepare my system to test this change? What packages
need to be installed, config files edited, etc.?
2. What specific actions do I perform to check that the change is
working like it's supposed to?
3. What are the expected results of those actions?
-->
TBD.  
 
== User Experience ==
Users won't notice differences.


== Dependencies ==
== Dependencies ==
Line 74: Line 327:


== Release Notes ==
== Release Notes ==
TBD.


[[Category:ChangePageIncomplete]]
https://docs.fedoraproject.org/en-US/fedora/f31/release-notes/developers/Developers/#developers-dynamic-buildrequires
 
[[Category:ChangeAcceptedF31]]
[[Category:SystemWideChange]]
[[Category:SystemWideChange]]

Revision as of 08:51, 23 April 2020

DynamicBuildRequires

Summary

Add possibility to generate build-time dependencies within RPM spec file and teach RPM and mock how to handle this.

Owner

Current status

Detailed Description

For many languages (Rust, Golang, Node.Js, Ruby, Python, Haskell), BuildRequires can be automatically generated. All it takes, run some special tool which will output dependencies in RPM format.

Changes in tooling

rpmbuild

  • New %generate_buildrequires section ("script" later in text) in spec file
  • New option -br which will run script, check dynamic BuildRequires
    • If all of them are satisfied, generate src.rpm
    • If some of them are not satisfied or --nodeps is present, generate buildreqs.nosrc.rpm
  • src.rpm produced by rpmbuild -ba will contain dynamic BuildRequires
  • Option -bs does not change behavior
  • src.rpm produced from spec file which has %generate_buildrequires has Requires: rpmlib(DynamicBuildRequires)
  • src.rpm which contain dynamic BuildRequires has Provides: rpmlib(DynamicBuildRequires) for identification purposes

mock

  • New option in config dynamic_buildrequires which controls support of new features
  • Support for rpmbuild changes if this option enabled

Example

Probably the easiest demonstration of how this works is to show an example.

Create this simple SPEC file ~/rpmbuild/SPECS/dynamic-buildrequires.spec:

Name:     dynamic-buildrequires
Version:  1
Release:  1%{?dist}
Summary:  Demonstration of Dynamic BuildRequires
License:  GPLv3+

# static buildrequires
BuildRequires: bash

%description
This package demonstrate Dynamic BuildRequires
https://fedoraproject.org/wiki/Changes/DynamicBuildRequires

%generate_buildrequires
echo fruit-cake

%files                                                                                                                                            

%changelog
* Thu Jun 13 2019 Miroslav Suchy <msuchy@redhat.com> - 1-2
- initial version

This SPEC file has a new section %generate_buildrequires which contains a script. The output of this script is interpreted as additional BuildRequires. The list accrues to classic BuildRequires tags. In this example we dynamically added BuildRequires of fruit-cake (which does not exist).

Now if you try to build this package:

# rpmbuild -ba dynamic-buildrequires.spec
Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.JMyoKz
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd /root/rpmbuild/BUILD
+ echo fruit-cake
+ exit 0
error: Failed build dependencies:
        fruit-cake is needed by dynamic-buildrequires-1-1.fc31.x86_64
Wrote: /rpmbuild/SRPMS/dynamic-buildrequires-1-1.fc31.buildreqs.nosrc.rpm

# echo $?
11

Two things happened: rpmbuild exited with exit code 11. This indicates the unsatisfied dynamic BuildRequires. If our script would produce a line with 'bash' we would likely have it already installed and rpmbuild would proceed further without failure. Additionally, And notice that this is 'buildreqs.nosrc.rpm'. If you would extract the content of the NOSRC.RPM it would contain the original SPEC file, but there would be no sources attached. But most importantly - the requires of this package include the dynamically generated dependencies:

# rpm -qpR rpmbuild/SRPMS/dynamic-buildrequires-1-1.fc31.buildreqs.nosrc.rpm
bash
fruit-cake
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(DynamicBuildRequires) <= 4.15.0-1
rpmlib(FileDigests) <= 4.6.0-1 

Rust Example

First, let's make necessary changes in our spec file (I'm taking rust-rpick).

diff --git a/rust-rpick.spec b/rust-rpick.spec
index 716a185..01a6d35 100644
--- a/rust-rpick.spec
+++ b/rust-rpick.spec
@@ -16,12 +16,6 @@ Source:         %{crates_source}
 ExclusiveArch:  %{rust_arches}
 
 BuildRequires:  rust-packaging
-BuildRequires:  (crate(dirs/default) >= 1.0.0 with crate(dirs/default) < 2.0.0)
-BuildRequires:  (crate(rand/default) >= 0.6.0 with crate(rand/default) < 0.7.0)
-BuildRequires:  (crate(serde/default) >= 1.0.0 with crate(serde/default) < 2.0.0)
-BuildRequires:  (crate(serde/derive) >= 1.0.0 with crate(serde/derive) < 2.0.0)
-BuildRequires:  (crate(serde_yaml/default) >= 0.8.0 with crate(serde_yaml/default) < 0.9.0)
-BuildRequires:  (crate(structopt/default) >= 0.2.0 with crate(structopt/default) < 0.3.0)
 
 %global _description \
 Helps you pick items from a list by various algorithms. Example uses: pick a\
@@ -69,6 +63,9 @@ which use "default" feature of "%{crate}" crate.
 %autosetup -n %{crate}-%{version_no_tilde} -p1
 %cargo_prep
 
+%generate_buildrequires
+%cargo_generate_buildrequires
+
 %build
 %cargo_build
 

Then let's try to build package (I will skip parts like rustc executions because it is irrelevant).

⋊> ~/P/s/r/rust-rpick on master ⨯ fedpkg mockbuild --mock-config $PWD/fedora-rawhide-x86_64-buildreqs.cfg
Wrote: /home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm
INFO: mock.py version 1.4.14 starting (python version = 3.7.3)...
[…]
INFO: Start(/home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm)  Config(fedora-rawhide-x86_64-buildreqs)
[…]
Start: build phase for rust-rpick-0.3.0-1.fc31.src.rpm
Start: build setup for rust-rpick-0.3.0-1.fc31.src.rpm
Building target platforms: x86_64
Building for target x86_64
Wrote: /builddir/build/SRPMS/rust-rpick-0.3.0-1.fc31.src.rpm
Repository 'kkt' is missing name in configuration, using id.
fedora                                           21 kB/s |  19 kB     00:00    
kkt                                             2.6 MB/s | 3.0 kB     00:00    
Dependencies resolved.
================================================================================
 Package                    Arch     Version                     Repo      Size
================================================================================
Installing:
 rust-packaging             x86_64   6-28.fc31+buildreqs         kkt       13 k
Installing dependencies:
[…]
Finish: build setup for rust-rpick-0.3.0-1.fc31.src.rpm
Start: rpmbuild rust-rpick-0.3.0-1.fc31.src.rpm
Start: Outputting list of installed packages
Finish: Outputting list of installed packages
Building target platforms: x86_64
Building for target x86_64
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.0m57Wu
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ rm -rf rpick-0.3.0
+ /usr/bin/gzip -dc /builddir/build/SOURCES/rpick-0.3.0.crate
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd rpick-0.3.0
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ set -eu
+ /usr/bin/mkdir -p .cargo
+ cat
+ /usr/bin/rm -f Cargo.lock
+ exit 0
Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.LooeBv
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ cd rpick-0.3.0
+ /usr/bin/cargo-inspector -BR Cargo.toml
+ /usr/bin/cargo-inspector -TR Cargo.toml
+ exit 0
Wrote: /builddir/build/SRPMS/rust-rpick-0.3.0-1.fc31.buildreqs.nosrc.rpm
INFO: Dynamic buildrequires detected
Repository 'kkt' is missing name in configuration, using id.
fedora                                           17 kB/s |  19 kB     00:01    
kkt                                             2.8 MB/s | 3.0 kB     00:00    
Package rust-packaging-6-28.fc31+buildreqs.x86_64 is already installed.
Dependencies resolved.
================================================================================
 Package                                  Arch    Version         Repo     Size
================================================================================
Installing:
 rust-dirs+default-devel                  noarch  1.0.5-1.fc31    fedora  7.6 k
 rust-rand+default-devel                  noarch  0.6.5-3.fc31    fedora  8.4 k
 rust-serde+default-devel                 noarch  1.0.90-1.fc31   fedora   12 k
 rust-serde+derive-devel                  noarch  1.0.90-1.fc31   fedora   12 k
 rust-serde_yaml+default-devel            noarch  0.8.8-2.fc30    fedora  8.3 k
 rust-structopt+default-devel             noarch  0.2.15-2.fc31   fedora  8.2 k
Installing dependencies:
[…]
Building target platforms: x86_64
Building for target x86_64
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.l2cC5s
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ rm -rf rpick-0.3.0
+ /usr/bin/gzip -dc /builddir/build/SOURCES/rpick-0.3.0.crate
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd rpick-0.3.0
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ set -eu
+ /usr/bin/mkdir -p .cargo
+ cat
+ /usr/bin/rm -f Cargo.lock
+ exit 0
Executing(buildreqs): /bin/sh -e /var/tmp/rpm-tmp.cvhwuw
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ cd rpick-0.3.0
+ /usr/bin/cargo-inspector -BR Cargo.toml
+ /usr/bin/cargo-inspector -TR Cargo.toml
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.dlI3ax
+ umask 022
+ cd /builddir/build/BUILD
+ cd rpick-0.3.0
+ /usr/bin/env CARGO_HOME=.cargo RUSTC_BOOTSTRAP=1 /usr/bin/cargo build -j8 -Z avoid-dev-deps --release
[…]
Finish: build phase for rust-rpick-0.3.0-1.fc31.src.rpm
INFO: Done(/home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/rust-rpick-0.3.0-1.fc31.src.rpm) Config(fedora-rawhide-x86_64-buildreqs) 2 minutes 9 seconds
INFO: Results and/or logs in: /home/brain/Projects/src.fedoraproject.org/rpms/rust-rpick/results_rust-rpick/0.3.0/1.fc31
INFO: Cleaning up build root ('cleanup_on_success=True')
Start: clean chroot
Finish: clean chroot
Finish: run

Let's check dependencies of src.rpm.

⋊> ~/P/s/r/rust-rpick on master ⨯ rpm -qpR $PWD/results_rust-rpick/0.3.0/1.fc31/rust-rpick-0.3.0-1.fc31.src.rpm
(crate(dirs/default) >= 1.0.0 with crate(dirs/default) < 2.0.0)
(crate(rand/default) >= 0.6.0 with crate(rand/default) < 0.7.0)
(crate(serde/default) >= 1.0.0 with crate(serde/default) < 2.0.0)
(crate(serde/derive) >= 1.0.0 with crate(serde/derive) < 2.0.0)
(crate(serde_yaml/default) >= 0.8.0 with crate(serde_yaml/default) < 0.9.0)
(crate(structopt/default) >= 0.2.0 with crate(structopt/default) < 0.3.0)
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(DynamicBuildRequires) <= 4.15.0-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(RichDependencies) <= 4.12.0-1
rust-packaging

Go Example

For Go language, you can use %go_generate_buildrequires macro. See full example here: https://pagure.io/go-rpm-macros/blob/master/f/templates/rpm/spectemplate-go-0-source-minimal.spec#_103

Python

   BuildRequires: pyproject-rpm-macros
   
   %generate_buildrequires
   %pyproject_buildrequires

This will read the dependencies from setup_requires from setup.py. If you want to read the dependencies from install_requires and use them as BuildRequires then you have to use %pyproject_buildrequires -r.

For more information see https://src.fedoraproject.org/rpms/pyproject-rpm-macros

Benefit to Fedora

Packagers won't have to pre-generate BuildRequires in the spec file which means it will be always updated (and correct) :

  • Packagers can focus of making their packages better instead of spending all their packaging time copying BuildRequires from documentation and third party tools.
  • BuildRequires are dropped as soon as they're no longer necessary
  • Packages can be easily bumped without requiring a manual BuildRequires refresh
  • BuildRequires and Requires generation can use similar utilities, making sure that the deps packages declare can also be used for second-level building. Packages no longer need to declare the deps of their second and n-th dependencies because someone forgot to declare them in the correct package.

Scope

  • Proposal owners: Implement support for a feature in RPM1 and mock1. Make use of it in interested ecosystems.
  • Other developers: Maintainers of language stacks are advised to use this feature.
  • Release engineering: #8129
  • Policies and guidelines: Packaging Guidelines need to be updated with instructions how to use this feature.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

Packagers and users who use repoquery might be affected (src.rpm might not contain generated dependencies).

How To Test

  1. Make sure you have new rpm-build (OPTION 1) or mock (OPTION 2).
  2. Add %generate_buildrequires script into spec file which prints dependencies, remove static BuildRequires
  3. Build package
    1. OPTION1: Run rpmbuild -ba with your usual workflow
    2. OPTION2: Run mock -r fedora-rawhide-x86_64 with your usual workflow
  4. Check that your package builds fine, dependencies are getting checked/installed and final src.rpm contains generated BuildRequires

User Experience

For people/tooling who build RPMs, it might be needed to run dnf builddep on new file generated by RPM when dynamic BuildRequires detected and unsatisfied ones found - buildreqs.nosrc.rpm. RPM will guide you where it stored it and which name file has.

Dependencies

Required feature needs to be implemented in RPM and mock.

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) Proposal Owners might still ship feature disabled for Fedora buildsystem but have it available for end-users, and move full completion to the next release.
  • Contingency deadline: Beta Freeze
  • Blocks release? No.
  • Blocks product? No.

Documentation

TBD.

Release Notes

https://docs.fedoraproject.org/en-US/fedora/f31/release-notes/developers/Developers/#developers-dynamic-buildrequires