From Fedora Project Wiki
No edit summary
 
(14 intermediate revisions by 6 users not shown)
Line 1: Line 1:
<!-- Self Contained or System Wide Change Proposal?
<!-- Self Contained or System Wide Change Proposal?
Use this guide to determine to which category your proposed change
Use this guide to determine to which category your proposed change belongs to.
belongs to.


Self Contained Changes are:
Self Contained Changes are:
* changes to isolated/leaf package without the impact on other
* changes to isolated/leaf package without the impact on other packages/rest of the distribution
  packages/rest of the distribution  
* limited scope changes without the impact on other packages/rest of the distribution
* limited scope changes without the impact on other packages/rest of
* coordinated effort within SIG with limited impact outside SIG functional area, accepted by the SIG
  the distribution  
* coordinated effort within SIG with limited impact outside SIG
  functional area, accepted by the SIG  


System Wide Changes are:
System Wide Changes are:
* changes that does not fit Self Contained Changes category touching
* changes that does not fit Self Contained Changes category touching  
* changes that require coordination within the distribution (for
* changes that require coordination within the distribution (for example mass rebuilds, release engineering or other teams effort etc.)
  example mass rebuilds, release engineering or other teams effort etc.)  
* changing system defaults
* changing system defaults


For Self Contained Changes, sections marked as "REQUIRED FOR SYSTEM
For Self Contained Changes, sections marked as "REQUIRED FOR SYSTEM WIDE CHANGES" are OPTIONAL but FESCo/Wrangler can request more details (especially in case the change proposal category is  
WIDE CHANGES" are OPTIONAL but FESCo/Wrangler can request more details
improper or updated to System Wide category). For System Wide Changes all fields on this form are required for FESCo acceptance (when applies).  
(especially in case the change proposal category is improper or
updated to System Wide category). For System Wide Changes all fields
on this form are required for FESCo acceptance (when applies).


We request that you maintain the same order of sections so that all of
We request that you maintain the same order of sections so that all of the change proposal pages are uniform.
the change proposal pages are uniform.
-->
-->


<!-- The actual name of your proposed change page should look
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
something like: Changes/Your_Change_Proposal_Name.  This keeps all
change proposals in the same namespace -->


= Changes/Annotated Binaries
= Annotated Binaries <!-- The name of your change proposal --> =


== Summary ==
== Summary ==
Line 40: Line 29:
This change causes extra information to be stored in binary files
This change causes extra information to be stored in binary files
compiled by gcc.  This information can be used by scripts to check on
compiled by gcc.  This information can be used by scripts to check on
various features of the file, such as the hardening options used of
various features of the file, such as the hardening options used or
potential ABI conflicts.
potential ABI conflicts.


Line 50: Line 39:
This should link to your home wiki page so we know who you are.  
This should link to your home wiki page so we know who you are.  
-->
-->
* Name: [[User:nickc| Nick Clifton]]
* Name: [[User:nickc| Nick Clifton]]
<!-- Include you email address that you can be reached should people
<!-- Include you email address that you can be reached should people
want to contact you about helping with your change, status is
want to contact you about helping with your change, status is
Line 58: Line 45:
proposal is owned by a SIG, please also add a primary contact
proposal is owned by a SIG, please also add a primary contact
person. -->
person. -->
* Email: nickc@redhat.com
* Email: nickc@redhat.com
 
* Release notes ticket: [https://pagure.io/fedora-docs/release-notes/issue/83 #83]
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
-->
-->
<!--- UNCOMMENT only if this Change aims specific product, working group (Cloud, Workstation, Server, Base, Env & Stacks)
<!--- UNCOMMENT only if this Change aims specific product, working group (Cloud, Workstation, Server, Base, Env & Stacks)
* Product:
* Product:
Line 85: Line 68:
   CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
   CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1502660 #1502660]


== Detailed Description ==
== Detailed Description ==
Line 124: Line 107:
binary file, beyond the actual code and data needed to make it work.
binary file, beyond the actual code and data needed to make it work.


Whilst this proposal focuses on enhancement that help release
Whilst this proposal focuses on enhancements that help release
engineering, the scheme is not limited to this area.  Internally the
engineering, the scheme is not limited to this area.  Internally the
project has already been used to record gcc unit test results in a
project has already been used to record gcc unit test results in a
Line 147: Line 130:
a dependency upon the annobin package.
a dependency upon the annobin package.


* Release engineering: [https://pagure.io/releng/issue/7069] <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
* Release engineering: [https://pagure.io/releng/issue/7069 #7069] and [https://pagure.io/releng/issue/7249 #7249 ]<!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
<!-- Does this feature require coordination with release engineering
<!-- Does this feature require coordination with release engineering
(e.g. changes to installer image generation or update package
(e.g. changes to installer image generation or update package
Line 187: Line 170:
There should be no other migration issues, apart from the possible
There should be no other migration issues, apart from the possible
issue of the size of rpms increasing, due to the extra information
issue of the size of rpms increasing, due to the extra information
being recorded.
being recorded.  Note - the rawhide version of the binutils package
includes the objcopy tool which could be used to reduce the size of
the information recorded in executable by removing redundant, duplicate
entries.  The use of objcopy in this way is not currently enabled
when linking programs, but it could be added if needed.


== How To Test ==
== How To Test ==
Line 216: Line 203:


In order to run tests the annobin package will need to be installed.
In order to run tests the annobin package will need to be installed.
You will also need to be able to compile files, so the gcc package
It is also necessary to be able to compile files, so the gcc package
will also be needed.  There should be no need to edit any config
will be needed.  There should be no need to edit any config files.
files.


To check that the feature is working, compile the file(s) (or build
To check that the feature is working, compile the file(s) (or build
the packages) that form the basis of your test.  Make sure that the
the packages) that form the basis of the test.  Make sure that the
-fplugin=annobin gcc command line option is being used when the files
''-fplugin=annobin'' gcc command line option is being used when the files
are compiled.  Then check the compiled files to see what information
are compiled.  Then check the compiled files to see what information
has been recorded.  The command line:
has been recorded.  The command line:
Line 305: Line 291:
change.
change.


[[Category:ChangeAnnounced]]
This change definitely is visible to any developer of compiled binary code, which includes several percent of Fedora users.  Compiled binary programs become larger by a couple percent, and .a archive libraries of compiled binary subroutines become larger by several percent.  Storage space is free until it gets exhausted, and then its cost is very large.  This change makes fitting software onto an EEPROM, soldered-on flash memory, mini-CD, etc., more challenging.
[[Category:ChangeAcceptedF28]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Latest revision as of 14:46, 2 March 2018


Annotated Binaries

Summary

This change causes extra information to be stored in binary files compiled by gcc. This information can be used by scripts to check on various features of the file, such as the hardening options used or potential ABI conflicts.

Owner

Current status

Detailed Description

The plan is to use a plugin to gcc to record extra information in the object files it creates. This information can then be examined by static analysis tools. The information is recorded in a compact, extensible format, described here:

 https://fedoraproject.org/wiki/Toolchain/Watermark

The Fedora annobin package is an implementation of the plugin for gcc. It also includes some example scripts that demonstrate how the recorded information can be used to, for example, check that an executable has been compiled with the correct hardening options, or detect if any conflicting ABI options have been used when compiling various parts of the executable.

To enable this change it is proposed that the redhat-rpm-config package should be extended to add the "-fplugin=annobin" option to the __global_compiler-flags macro. In theory such a change will be completely invisible to Fedora users but should prove to be very helpful to Fedora Release Management, assuming that they like the idea of these annotated binaries.

Benefit to Fedora

The main improvement is the ability to record extra information in a binary file, beyond the actual code and data needed to make it work.

Whilst this proposal focuses on enhancements that help release engineering, the scheme is not limited to this area. Internally the project has already been used to record gcc unit test results in a binary, so that it is possible to determine which parts of the compiler ran when the binary was created.

Scope

  • Proposal owners:

Make sure the annobin plugin is ready.

  • Other developers:

An update is needed to the redhat-rpm-config package in order for the plugin to be invoked when gcc is used to compile programs, and to add a dependency upon the annobin package.

  • Release engineering: #7069 and #7249
    • Coordination with release engineering is needed.
    • A mass rebuild will be required.
  • List of deliverables: All delivered images are affected, however there no changes to the list it self.
  • Policies and guidelines: No updates needed
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

On systems where the redhat-rpm-config package is installed the annobin package will now be a requirement.

There should be no other migration issues, apart from the possible issue of the size of rpms increasing, due to the extra information being recorded. Note - the rawhide version of the binutils package includes the objcopy tool which could be used to reduce the size of the information recorded in executable by removing redundant, duplicate entries. The use of objcopy in this way is not currently enabled when linking programs, but it could be added if needed.

How To Test

Special hardware is not needed, but the plugin used to record the information is architecture specific. Thus it would be a good idea to run the tests on as many different architectures as are available.

In order to run tests the annobin package will need to be installed. It is also necessary to be able to compile files, so the gcc package will be needed. There should be no need to edit any config files.

To check that the feature is working, compile the file(s) (or build the packages) that form the basis of the test. Make sure that the -fplugin=annobin gcc command line option is being used when the files are compiled. Then check the compiled files to see what information has been recorded. The command line:

 readelf --notes --wide <name-of-file>

should achieve this aim.

The annobin package does include some tests of its own, and these can be used as examples of how to create more tests.

User Experience

N/A (This is a system wide change, but it should have no user visible impact apart from slightly larger rpms).

Dependencies

annobin, gcc, gcc-plugin-devel, pkgconfig, redhat-rpm-config

Contingency Plan

  • Contingency mechanism: Revert change to redhat-rpm-macros


  • Contingency deadline: beta Freeze


  • Blocks release? No
  • Blocks product? None

Documentation

The annotation scheme is documented here:

 https://fedoraproject.org/wiki/Toolchain/Watermark

Release Notes

In theory no release notes are needed as this is not a user visible change.

This change definitely is visible to any developer of compiled binary code, which includes several percent of Fedora users. Compiled binary programs become larger by a couple percent, and .a archive libraries of compiled binary subroutines become larger by several percent. Storage space is free until it gets exhausted, and then its cost is very large. This change makes fitting software onto an EEPROM, soldered-on flash memory, mini-CD, etc., more challenging.