From Fedora Project Wiki
No edit summary
No edit summary
 
(6 intermediate revisions by the same user not shown)
Line 3: Line 3:
= Introduction =
= Introduction =


We are currently building RPMs using the stage3 environment, using the rpmbuild command on RISC-V (under qemu emulation).  [https://github.com/rwmjones/fedora-riscv/blob/master/README This is step (4a) in the README.]
Bootstrapping happens in several stages which are outlined in detail in the <code>README</code> and <code>Makefile</code> in [https://github.com/rwmjones/fedora-riscv-bootstrap this repository].


We aim to build just enough userspace RPMs so that we can:
The general idea is that we go in stages (numbered for historical reasons):


* create a stage4 environment which is cleanly built from only RPMs
== Stage 3 ==
* the stage4 environment will be able to run gcc and rpmbuild
* the stage4 environment will (later) be used to build all future RPMs


Some packages require non-upstream patches for RISC-VCurrently these are:
Build an <code>x86_64</code> disk image, but remove all the binaries and libraries from itThe binaries and libraries are cross-compiled from the host using the [https://github.com/riscv/riscv-tools riscv64-unknown-linux-gnu-gcc compiler].


* kernel <i>(we are not building this as an RPM yet because there is no bootloader that could use it)</i>
After building the stage 3 disk image in this way, it is bootable under [http://copr-fe.cloud.fedoraproject.org/coprs/rjones/riscv/ qemu-system-riscv64] and has working gcc and rpmbuild:
* glibc
* binutils
* gcc


Some packages require modifications to the spec file to reduce extraneous dependencies, disable tests, hacks and so on.  The aim is to reduce and eliminate these changes in stage4, but they are OK for now in stage3 RPMs.
<pre>
Tue Dec 19 22:47:00 UTC 2017


noarch RPMs do not need to be rebuilt.  They can just be copied over from Koji.
Welcome to the Fedora/RISC-V stage3 disk image
https://fedoraproject.org/wiki/Architectures/RISC-V


= Status =
stage3:/# ls
bin  dev  home  lib    lost+found  mnt  proc  rpmbuild  sbin  sys  usr
boot  etc  init  lib64  media      opt  root  run      srv  tmp  var
stage3:/# uname -a
Linux stage3 4.15.0-rc3-01064-g418457520fbb #2 Tue Dec 19 22:31:06 GMT 2017 riscv64 GNU/Linux
stage3:/# gcc --version
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
stage3:/# rpmbuild --version
RPM version 4.12.90
</pre>


All built RPMs are added to the [https://github.com/rwmjones/fedora-riscv/tree/master/stage3-built-rpms stage3-built-rpms/ subdirectory of git], with the SRPM so we can reproduce the build.
=== Status of stage 3 packages (2018-01-11) ===
 
<code>python3</code>: Builds OK but lots of missing files, indicating we don't have sufficient dependencies compiled.
 
<code>python2</code>: Peculiar build failures to do with missing built files.  Something to do with libffi?
 
<code>cmake</code>: Circular dependency cmake - jsoncpp.  Possibly could be solved by cross-compiling jsoncpp.
 
Packages requiring python2 or python3: <code>glib2</code>, <code>itstool</code>, <code>fontconfig</code>, <code>rpm</code>, <code>librepo</code>, <code>libpsl</code>
 
Packages requiring glib2: <code>libcroco</code>, <code>gettext</code>, <code>shared-mime</code>, <code>tdnf</code>
 
Packages requiring cmake: <code>jsoncpp</code>, <code>doxygen</code>, <code>beecrypt</code>, <code>hawkey</code>, <code>nss-pem</code>, <code>libsolv</code>
 
<code>openssh</code>: Unable to make it <i>not</i> use ldap, and something is broken about openldap.
 
== Stage 4 ==
 
Using the stage 3 disk image, running under qemu, [https://github.com/rwmjones/fedora-riscv-bootstrap/tree/master/stage3-built-rpms we build RPMs for many core packages].
 
Eventually we have built enough RPMs that we can build a clean stage 4 disk image entirely from RPMs.
 
== After stage 4 ==
 
Using the clean stage 4 disk image we now go on to build the remaining packages from Fedora.  This is a very long, incremental task aided by an autobuilder which picks up new packages from real Fedora and tries to build them for RISC-V.  In addition we build and fix packages by hand.
 
= How to take part =
 
Join the <code>#fedora-riscv</code> IRC channel on FreeNode and say hello.
 
Enable [http://copr-fe.cloud.fedoraproject.org/coprs/rjones/riscv/ Richard Jones's riscv copr] and install <code>riscv-qemu</code>.  Alternately you can build it from source in the fedora-riscv-bootstrap repo.
 
Check out the [https://github.com/rwmjones/fedora-riscv-bootstrap fedora-riscv-bootstrap] repository, read the <code>README</code> and <code>Makefile</code> files, and then attempt to build it using:
 
<pre>
make stage1
make stage2
make stage3
make boot-stage3-in-qemu
</pre>


= Semi-automatically building packages in stage3 =
= Semi-automatically building packages in stage3 =
Line 90: Line 138:


Add notes to the section below.  Also come to <code>#fedora-riscv</code> and discuss where to put your built packages.
Add notes to the section below.  Also come to <code>#fedora-riscv</code> and discuss where to put your built packages.
= Notes on typical bugs etc =
== "all-recursive" bug in make ==
Making all in .
Makefile:742: recipe for target 'all-recursive' failed
This is a bug in either the cross-compiled <code>make</code> or the cross-compiled <code>bash</code>.  It is fixed by the natively compiled make and/or bash, so to fix it just run the following command from within the stage3 environment:
rpm -ivh /rpmbuild/RPMS/riscv64/bash-* /rpmbuild/RPMS/riscv64/make-* --nodeps
== debugedit missing ==
Use <code>%global debug_package %{nil}</code> (temporarily for stage3, in stage4 we will attempt to build the debuginfo packages).
== bugs about locking, random "Permission denied" ==
Update to the latest riscv-gnu-toolchain and rebuild the kernel from the repo.  All known locking bugs have been fixed.
== Build hangs ==
We fixed an infinite loop bug in kernel memory allocation.  Make sure your kernel is up to date.

Latest revision as of 09:04, 11 January 2018

All about bootstrapping Fedora on RISC-V.

Introduction

Bootstrapping happens in several stages which are outlined in detail in the README and Makefile in this repository.

The general idea is that we go in stages (numbered for historical reasons):

Stage 3

Build an x86_64 disk image, but remove all the binaries and libraries from it. The binaries and libraries are cross-compiled from the host using the riscv64-unknown-linux-gnu-gcc compiler.

After building the stage 3 disk image in this way, it is bootable under qemu-system-riscv64 and has working gcc and rpmbuild:

Tue Dec 19 22:47:00 UTC 2017

Welcome to the Fedora/RISC-V stage3 disk image
https://fedoraproject.org/wiki/Architectures/RISC-V

stage3:/# ls
bin   dev  home  lib    lost+found  mnt  proc  rpmbuild  sbin  sys  usr
boot  etc  init  lib64  media       opt  root  run       srv   tmp  var
stage3:/# uname -a
Linux stage3 4.15.0-rc3-01064-g418457520fbb #2 Tue Dec 19 22:31:06 GMT 2017 riscv64 GNU/Linux
stage3:/# gcc --version
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
stage3:/# rpmbuild --version
RPM version 4.12.90

Status of stage 3 packages (2018-01-11)

python3: Builds OK but lots of missing files, indicating we don't have sufficient dependencies compiled.

python2: Peculiar build failures to do with missing built files. Something to do with libffi?

cmake: Circular dependency cmake - jsoncpp. Possibly could be solved by cross-compiling jsoncpp.

Packages requiring python2 or python3: glib2, itstool, fontconfig, rpm, librepo, libpsl

Packages requiring glib2: libcroco, gettext, shared-mime, tdnf

Packages requiring cmake: jsoncpp, doxygen, beecrypt, hawkey, nss-pem, libsolv

openssh: Unable to make it not use ldap, and something is broken about openldap.

Stage 4

Using the stage 3 disk image, running under qemu, we build RPMs for many core packages.

Eventually we have built enough RPMs that we can build a clean stage 4 disk image entirely from RPMs.

After stage 4

Using the clean stage 4 disk image we now go on to build the remaining packages from Fedora. This is a very long, incremental task aided by an autobuilder which picks up new packages from real Fedora and tries to build them for RISC-V. In addition we build and fix packages by hand.

How to take part

Join the #fedora-riscv IRC channel on FreeNode and say hello.

Enable Richard Jones's riscv copr and install riscv-qemu. Alternately you can build it from source in the fedora-riscv-bootstrap repo.

Check out the fedora-riscv-bootstrap repository, read the README and Makefile files, and then attempt to build it using:

make stage1
make stage2
make stage3
make boot-stage3-in-qemu

Semi-automatically building packages in stage3

This basically automates the process described in the next section.

fedpkg clone -B the_package
cd the_package/f25
# make changes to spec file if you want
fedpkg srpm
make stage3-build SRPM=/path/to/the_package.src.rpm

Manually building packages in stage3

The stage3 environment can be built from source or you can download a snapshot binary image. See above for links to git etc.

Pick a package which:

  • has very few dependencies
  • is written in C or C++
  • is part of Fedora @Core

Grab the source RPM from the F25 branch in Koji or dist-git. Note you must use the F25 branch, because otherwise Perl doesn't work.

fedpkg clone -B the_package
cd the_package/f25
# make changes to spec file if you want
fedpkg srpm

Copy the source RPM into the stage3 disk image. The disk image must not be running when you do this:

virt-copy-in -a stage3-disk.img the_package.src.rpm /var/tmp/

Boot the disk image in qemu:

make boot-stage3-in-qemu

Inside the VM you may need to install some of the RPMs we have already built. Almost certainly you'll need make and bash:

rpm -Uvh /rpmbuild/RPMS/riscv64/{bash,make}-*.rpm --nodeps

Inside the VM you can now "install" the source RPM, which makes it available under /rpmbuild:

rpm -i /var/tmp/the_package.src.rpm
cd /rpmbuild/SPECS

You may need to disable hardened build and debuginfo by adding:

%global debug_package %{nil}
%undefine _hardened_build

to the spec file. Use vim to edit files.

You can now try building the package:

rpmbuild -ba the_package.spec --nodeps

If it builds successfully, brilliant! More usually this will require many cycles of debugging and fixing things. Removing dependencies or parts of the spec file is an option for these stage3 builds.

If you get a build, it will be in /rpmbuild inside the VM. sync and shut down the VM, then do:

virt-copy-out -a stage3-disk.img /rpmbuild ./

and the files will be downloaded to ./SRPMS and ./RPMS in the current directory. Note: Keep the source RPM too, especially if you modified it.

Add notes to the section below. Also come to #fedora-riscv and discuss where to put your built packages.