From Fedora Project Wiki
(Special COPR and mock configuration)
m (Link to logs with build failures)
 
(23 intermediate revisions by the same user not shown)
Line 5: Line 5:
== Status ==
== Status ==


* Gathering data on implicit function declarations (e.g., `exit (0)` called without `#include <stdlib.h>`).
* `-Werror=implicit-int` and `-Werror=implicit-function-declaration` are active in rawhide.
* Gathering data on implicit `int`s (e.g., const TWO = 2).
* Fixing other issues continues.
* Preparing infrastructure for fix verification for Fedora developers.


== Overall Approach ==
== Discovering Packages Which Need Fixing ==


We build packages with instrumented `gcc` and `redhat-rpm-config` packages that fail the build if certain errors occur anywhere during the build. These forced failures are guaranteed to happen even if the build system ignores the exit status of a `gcc` command and continues regardless (for example, during `configure` checks). This is implemented by patching `gcc` in a special buildroot to log certain errors to a 1777 (sticky bit permission) directory. After the build, `redhat-rpm-config` checks if the directory is empty (or more precisely, only contains expected errors), and fails the build otherwise.
Using [https://gitlab.com/fedora/packager-tools/mass-prebuild Frédéric Bérat's Mass Prebuild tool], we build packages with instrumented `gcc` and `redhat-rpm-config` packages that fail the build if certain errors occur anywhere during the build. These forced failures are guaranteed to happen even if the build system ignores the exit status of a `gcc` command and continues regardless (for example, during `configure` checks). This is implemented by patching `gcc` in a special buildroot to log certain errors to a 1777 (sticky bit permission) directory. After the build, `redhat-rpm-config` checks if the directory is empty (or more precisely, only contains expected errors), and fails the build otherwise.


* [https://src.fedoraproject.org/fork/fweimer/rpms/gcc/tree/c99-port rawhide gcc fork with instrumentation]
* [https://src.fedoraproject.org/rpms/gcc/tree/private-f38-toolchain rawhide gcc fork with instrumentation]
* [https://src.fedoraproject.org/fork/fweimer/rpms/redhat-rpm-config/tree/c99-port rawhide redhat-rpm-config fork with instrumentation]
* [https://src.fedoraproject.org/rpms/redhat-rpm-config/tree/private-f38-toolchain rawhide redhat-rpm-config fork with instrumentation]
* [https://gitlab.com/fweimer-rh/fedora-modernc-logs captured build failure logs with the instrumented compiler]
 
You can download logs from builds with this instrumented compiler by cloning a Git repository:
 
<nowiki>
git clone --depth 1 https://gitlab.com/fweimer-rh/fedora-modernc-logs</nowiki>
 
=== Currently available instrumentations ===
 
* '''Implicit function declarations.''' Limitations:
** Some functions are expected to be undefined (e.g., `pledge` does not exist on Linux). Calls to them result in implicit declarations, which is a false positive in this context. As a result, in the `redhat-rpm-config` package, there is [https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/private-f38-toolchain/f/report-gcc-errors.lua an exception list in the reporting code].  Reported as ''implicit-function-declaration'''.
* '''Implicit `int`''' (in variable declarations, and missing function return types). Limitations:
** <s>Unsupported specifiers like `__declspec(thread)` can be parsed as old-style function definitions with a parameter implicitly typed as `int`, leading to a false positive implicit `int`.</s>  Should be fixed for `__declspec` in `.implicits.4.` builds. Reported as '''implicit-int''.
** <s>A [https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107805 GCC bug] causes GCC to report an implicit `int` type if a type that has been redefined is used in a declaration.</s> Should be fixed in `.implicits.4.` builds.
* '''Implicit `int`/pointer conversion.''' Starting with the `.implicits.5.` versions, such errors are logged as well. These are reported as '''int-conversion''' failures.
* '''Misuse of `return` statements.''' With the `.implicits.8.` versions, `return` without an expression in functions not returning `void` and vice versa are flagged as '''return-mismatch'''. Upstream, this is controlled by the new `-Wreturn-mismatch` warning, which was split from `-Wreturn-types`.
* '''Misuse of pointer types.''' With the `.implicits.8.` versions, implict casts by completely different pointer types (not involving the generic pointer-to-`void` types) are flagged as '''incompatible-pointer-types'''. Upstream, this is controlled by `-Wincompatible-pointer-types`.
* '''Misspelled type names in function prototypes.''' Reported as '''declaration-missing-parameter-type'''. Also in `.implicits.8.`. Previously, if all type names in a function parameter list were not recognized, the parameter list was ignored, and a warning (currently not controllable by a separate option) was issued.
 
The instrumentation in `redhat-rpm-config` checks after the various stages (e.g., `%build` and `%prep`) if `gcc` has logged anything.  If yes, it fails the build with a message like this one:
 
<nowiki>
*** GCC errors begin ***
conftest.c:55: implicit-int
conftest.c:37: implicit-function-declaration: exit
conftest.c:83: implicit-int
conftest.c:84: implicit-function-declaration: offsetof
conftest.c:293: implicit-int
conftest.c:71: implicit-int
*** GCC errors end ***</nowiki>
 
As a convenience, a dump of `config.log` files is included in the build log before these errors are listed.
 
=== Container image with preinstalled instrumented GCC ===
 
A container image with the GCC is available ([https://gitlab.com/fweimer-rh/fedora-modernc-container source code]).  Run a shell with this command:
 
<nowiki>
podman run -i -t registry.gitlab.com/fweimer-rh/fedora-modernc-container:modernc</nowiki>
 
By default, the instrumentation logs errors into the `/usr/lib/gcc/errors` directory.
 
 
=== Special Koji side tag of rawhide with instrumented GCC ===
 
(not currently in working condition)


=== Special COPR repository with instrumented GCC ===
=== Special COPR repository with instrumented GCC ===


A [https://copr.fedorainfracloud.org/coprs/fweimer/modernc-1/ COPR repository] contains instrumented packages. Builds fail if  
A [https://copr.fedorainfracloud.org/coprs/fweimer/modernc-1/ COPR repository] contains instrumented packages. Builds fail if they use any of the obsolete features currently under test.


{{warning}} From time to time, error checking will break if newer `gcc` or `redhat-rpm-config` builds arrive in rawhide.
{{warning}} From time to time, error checking will break if newer `gcc` or `redhat-rpm-config` builds arrive in rawhide, although the confuguration sets `priority=10` to prefer the instrumented builds.


The following `mock` configuration can be used for local builds.
The following `mock` configuration can be used for local builds.
Line 44: Line 89:
enabled=1
enabled=1
enabled_metadata=1
enabled_metadata=1
"""
priority=10
</nowiki>
"""</nowiki>
 
== Fedora Package Updates ==
 
All identified issues need to be fixed in rawhide eventually. If a package cannot be ported, a bug needs to be filed in Bugzilla blocking [https://bugzilla.redhat.com/show_bug.cgi?id=2137516 PortingToModernCExemptions] to record this, and the package still needs to build in an early C language mode (probably C89 or C99).
 
If an upstream fix becomes available, it may make sense to backport it before a package rebase, so that the package no longer shows up as needed porting work.
 
Compatibility packages and other legacy packages need to be fixed as well.
 
=== Documenting Changes ===
 
* Fedora dist-git commit message should refer [[Changes/PortingToModernC]] and [[Toolchain/PortingToModernC]] (this page).
* Fedora updates should be captured in the [https://gitlab.com/fweimer-rh/fedora-modernc fedora-modernc] tracking repository. Please send MRs there if you fix something.
* Changes that merely included what would come in through a future rebase do not need a bug in Bugzilla or filing an issue with upstream. This includes backports from upstream, or patches which merely change generated files when re-generation using the current tool version would also fix the issues. (The latter is common with old `configure` scripts.)
* Patch files added as part of this change must include a URL for an upstream commit or for an upstream patch submission.
* A bug needs to be filed in Bugzilla if there is no way to report the issue upstream, blocking [https://bugzilla.redhat.com/show_bug.cgi?id=2141798 PortingToModernCNoUpstream].
* If a package cannot be ported, a bug needs to be filed blocking [https://bugzilla.redhat.com/show_bug.cgi?id=2137516 PortingToModernCExemptions].
 
=== Exceptions and Exemptions ===
 
Exceptions should be documented via a bug that blocks [https://bugzilla.redhat.com/show_bug.cgi?id=2137516 PortingToModernCExemptions]. They should use
 
<pre>
%global build_type_safety_c 0
</pre>
 
early in the RPM spec file. See ''Controll Type Safety'' in [https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/buildflags.md#controlling-type-safety the redhat-rpm-config documentation].


== Cross-Distribution Collaboration ==
== Cross-Distribution Collaboration ==
Line 53: Line 125:
Further resources will follow as needed.
Further resources will follow as needed.


== Fedora Bug Tracking ==
== Fedora Change Tracking ==
 
Changes to Fedora packages should be captured in the [https://gitlab.com/fweimer-rh/fedora-modernc fedora-modernc] tracking repository.


Not every build issue will have a bug in Bugzilla. But if there is no upstream with a mailing list or issue tracker to submit fixes to, it makes sense to create a bug in Bugzilla, especially for cross-distribution collaboration.
Not every build issue will have a bug in Bugzilla. But if there is no upstream with a mailing list or issue tracker to submit fixes to, it makes sense to create a bug in Bugzilla, especially for cross-distribution collaboration.

Latest revision as of 10:07, 10 January 2024

Porting Fedora to Modern C

This page collects some technical details related to Changes/PortingToModernC.

Status

  • -Werror=implicit-int and -Werror=implicit-function-declaration are active in rawhide.
  • Fixing other issues continues.

Discovering Packages Which Need Fixing

Using Frédéric Bérat's Mass Prebuild tool, we build packages with instrumented gcc and redhat-rpm-config packages that fail the build if certain errors occur anywhere during the build. These forced failures are guaranteed to happen even if the build system ignores the exit status of a gcc command and continues regardless (for example, during configure checks). This is implemented by patching gcc in a special buildroot to log certain errors to a 1777 (sticky bit permission) directory. After the build, redhat-rpm-config checks if the directory is empty (or more precisely, only contains expected errors), and fails the build otherwise.

You can download logs from builds with this instrumented compiler by cloning a Git repository:

git clone --depth 1 https://gitlab.com/fweimer-rh/fedora-modernc-logs

Currently available instrumentations

  • Implicit function declarations. Limitations:
    • Some functions are expected to be undefined (e.g., pledge does not exist on Linux). Calls to them result in implicit declarations, which is a false positive in this context. As a result, in the redhat-rpm-config package, there is an exception list in the reporting code. Reported as implicit-function-declaration'.
  • Implicit int (in variable declarations, and missing function return types). Limitations:
    • Unsupported specifiers like __declspec(thread) can be parsed as old-style function definitions with a parameter implicitly typed as int, leading to a false positive implicit int. Should be fixed for __declspec in .implicits.4. builds. Reported as 'implicit-int.
    • A GCC bug causes GCC to report an implicit int type if a type that has been redefined is used in a declaration. Should be fixed in .implicits.4. builds.
  • Implicit int/pointer conversion. Starting with the .implicits.5. versions, such errors are logged as well. These are reported as int-conversion failures.
  • Misuse of return statements. With the .implicits.8. versions, return without an expression in functions not returning void and vice versa are flagged as return-mismatch. Upstream, this is controlled by the new -Wreturn-mismatch warning, which was split from -Wreturn-types.
  • Misuse of pointer types. With the .implicits.8. versions, implict casts by completely different pointer types (not involving the generic pointer-to-void types) are flagged as incompatible-pointer-types. Upstream, this is controlled by -Wincompatible-pointer-types.
  • Misspelled type names in function prototypes. Reported as declaration-missing-parameter-type. Also in .implicits.8.. Previously, if all type names in a function parameter list were not recognized, the parameter list was ignored, and a warning (currently not controllable by a separate option) was issued.

The instrumentation in redhat-rpm-config checks after the various stages (e.g., %build and %prep) if gcc has logged anything. If yes, it fails the build with a message like this one:

*** GCC errors begin ***
conftest.c:55: implicit-int
conftest.c:37: implicit-function-declaration: exit
conftest.c:83: implicit-int
conftest.c:84: implicit-function-declaration: offsetof
conftest.c:293: implicit-int
conftest.c:71: implicit-int
*** GCC errors end ***

As a convenience, a dump of config.log files is included in the build log before these errors are listed.

Container image with preinstalled instrumented GCC

A container image with the GCC is available (source code). Run a shell with this command:

podman run -i -t registry.gitlab.com/fweimer-rh/fedora-modernc-container:modernc

By default, the instrumentation logs errors into the /usr/lib/gcc/errors directory.


Special Koji side tag of rawhide with instrumented GCC

(not currently in working condition)

Special COPR repository with instrumented GCC

A COPR repository contains instrumented packages. Builds fail if they use any of the obsolete features currently under test.

From time to time, error checking will break if newer gcc or redhat-rpm-config builds arrive in rawhide, although the confuguration sets priority=10 to prefer the instrumented builds.

The following mock configuration can be used for local builds.

config_opts['target_arch'] = 'x86_64'
config_opts['legal_host_arches'] = ('x86_64',)

include('templates/fedora-rawhide.tpl')

config_opts['root'] = 'fedora-rawhide-modernc'

config_opts['dnf.conf'] += """

[copr-fweimer-modernc]
name=COPR with instrumented GCC for modern C porting
baseurl=https://download.copr.fedorainfracloud.org/results/fweimer/modernc-1/fedora-rawhide-$basearch/
type=rpm-md
skip_if_unavailable=False
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/fweimer/modernc-1/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1
priority=10
"""

Fedora Package Updates

All identified issues need to be fixed in rawhide eventually. If a package cannot be ported, a bug needs to be filed in Bugzilla blocking PortingToModernCExemptions to record this, and the package still needs to build in an early C language mode (probably C89 or C99).

If an upstream fix becomes available, it may make sense to backport it before a package rebase, so that the package no longer shows up as needed porting work.

Compatibility packages and other legacy packages need to be fixed as well.

Documenting Changes

  • Fedora dist-git commit message should refer Changes/PortingToModernC and Toolchain/PortingToModernC (this page).
  • Fedora updates should be captured in the fedora-modernc tracking repository. Please send MRs there if you fix something.
  • Changes that merely included what would come in through a future rebase do not need a bug in Bugzilla or filing an issue with upstream. This includes backports from upstream, or patches which merely change generated files when re-generation using the current tool version would also fix the issues. (The latter is common with old configure scripts.)
  • Patch files added as part of this change must include a URL for an upstream commit or for an upstream patch submission.
  • A bug needs to be filed in Bugzilla if there is no way to report the issue upstream, blocking PortingToModernCNoUpstream.
  • If a package cannot be ported, a bug needs to be filed blocking PortingToModernCExemptions.

Exceptions and Exemptions

Exceptions should be documented via a bug that blocks PortingToModernCExemptions. They should use

%global build_type_safety_c 0

early in the RPM spec file. See Controll Type Safety in the redhat-rpm-config documentation.

Cross-Distribution Collaboration

A mailing list has been set up to exchange issues and observations of wider interest: c-std-porting@lists.linux.dev (archive)

Further resources will follow as needed.

Fedora Change Tracking

Changes to Fedora packages should be captured in the fedora-modernc tracking repository.

Not every build issue will have a bug in Bugzilla. But if there is no upstream with a mailing list or issue tracker to submit fixes to, it makes sense to create a bug in Bugzilla, especially for cross-distribution collaboration.

Currently, these trackers exist:

  • PortingToModernC Umbrella tracker for the tracker bugs below.
  • PortingToModernCExemptions Bugs which document that certain packages could not be switched to C99 or later language standards. Typically these packages keep building with -std=gnu89.
  • PortingToModernCHelpNeeded Bugs where help from the package maintainer is needed to address the issue(s).
  • PortingToModernCNoUpstream Bugs to provide a reference where there is either no upstream, or the current upstream lacks a public bug tracker or version control repository

Contributing To The Porting Effort

If you want to contribute to this effort, you should subscribe to the c-std-porting list mentioned above. If you create a Red Hat Bugzilla account, log in, and visit the Bugzilla tracker bugs linked above, a Add me to CC list check mark will appear on the bug pages. If you check that and click the Save Changes button, you will be notified of future bug changes, including bug dependency changes that indicate new issues. You can also subscribe to individual bugs that interest you.

At this point, it is too early to tell what level of knowledge and open-source experience is typically needed to successfully contribute to this project.