From Fedora Project Wiki

(firefox and icecat)
(Replace guidelines with link to the real Packaging Guidelines)
 
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Why no Bundled Libraries ==
== Why Bundled Libraries are a problem ==


Although you can request an exception from FPC there are many reasons not to grant one.  These are the reasons that it's painful for us to have bundled libraries in the distribution.  An exception should only be granted if the value of bundling exceeds these costs.
These are some of the reasons why it's painful for us to have bundled libraries in the distribution.


=== Security ===
=== Security ===


* When a security flaw is discovered in a library and bundling is not allowed, The library can be fixed in a single package, that package rebuilt, and when users download it, all the applications that use it are immediately protected.  When bundling is allowed, the distribution has to find all the packages that the library occurs in by auditing source code or running a special tool over all elf files in all packages, then all of those packages have to be fixed, all of those packages have to be built, and users have to download and update each of the ones that they are using on their system before they are protected.  There is much more work involved when bundled libraries are involved.
* When a security flaw is discovered in a library and bundling is not allowed, The library can be fixed in a single package, that package rebuilt, and when users download it, all the applications that use it are immediately protected.  When bundling is allowed, the distribution has to find all the packages that the library occurs in by auditing source code or running a special tool over all elf files in all packages, then all of those packages have to be fixed, all of those packages have to be built, and users have to download and update each of the ones that they are using on their system before they are protected.  There is much more work when bundled libraries are involved.


* With security issues, people want to remove as much lag as they can between announcement of a problem and the fix being available for users.  When libraries are unbundled, tools like vendor-sec can be used to alert distributions of problems that need patching in their packages before the announcement is made and then they can fix them with zero days of vulnerability.  If bundling of libraries occurs, then the problem becomes how to get fixes out to all affected packages.  If the distribution patches those packages, they must be careful to not leak the fact that there is a security vulnerability before they are allowed (which means they need to be careful who they share the information and what information they share with others).  OTOH, if they do not patch the packages bundling libraries, then those packages are not protected on zero day, but only afterwards.
* With security issues, people want to remove as much lag as they can between announcement of a problem and the fix being available for users.  When libraries are unbundled, tools like vendor-sec can be used to alert distributions of problems that need patching in their packages before the announcement is made and then they can fix them with zero days of vulnerability.  If bundling of libraries occurs, then the problem becomes how to get fixes out to all affected packages.  If the distribution patches those packages, they must be careful to not leak the fact that there is a security vulnerability before they are allowed (which means they need to be careful who they share the information and what information they share with others).  OTOH, if they do not patch the packages bundling libraries, then those packages are not protected on zero day, but only afterwards.
Line 11: Line 11:
* When a security flaw appears, the program has to either update to a non-affected version of the library or backport a fix.  This can be problematic when the code of the library has undergone many API and code changes since the version that is being bundled and the security fixing patch is very widespread.  Many conflicts can arise that need time to fix when trying to backport the fixes but porting the application code to the new API version can also take a lot of time.
* When a security flaw appears, the program has to either update to a non-affected version of the library or backport a fix.  This can be problematic when the code of the library has undergone many API and code changes since the version that is being bundled and the security fixing patch is very widespread.  Many conflicts can arise that need time to fix when trying to backport the fixes but porting the application code to the new API version can also take a lot of time.


* We cannot implicitly trust an upstream application to be on top of security issues that are released in the packages that they care about.  What happens if you are not following boost development and don't know that a security release has been made?  What happens if the developer that is responsible for watching boost development goes on vacation or quits your project?  What happens if your application ceases active development?  What happens if boost stops active development and security fixes start originating with distro patches?
* We cannot implicitly trust an upstream application to be on top of security issues that are released in the packages that they care about.  What happens if you are not following upstream development and don't know that a security release has been made?  What happens if the developer that is responsible for watching bundled upstream development goes on vacation or quits your project?  What happens if your application ceases active development?  What happens if upstream stops active development and security fixes start originating from distro patches?


=== Forking ===
=== Forking ===
Line 34: Line 34:
Although licensing issues can crop up in any project, projects which bundle code from different sources together are a special source of concern.  They make auditing for license issues a larger project.
Although licensing issues can crop up in any project, projects which bundle code from different sources together are a special source of concern.  They make auditing for license issues a larger project.


== When a Bundled Library is Discovered Post-Review ==
== Treatment of Bundled Libraries ==
Bundling of libraries is a serious problem. If a package that is in the distribution is discovered to have bundled libraries we need to fix it. First, open a bug report against the package. Then add the bug to the [https://bugzilla.redhat.com/show_bug.cgi?id=504493 Duplicate libraries tracker]. Once that's done, if help is needed fixing the bug, ask on the mailing list. Maintainers must leave the bug open until the situation is resolved (either by unbundling or via an FPC exception). If a patch is supplied the maintainer needs to evaluate it and actively work to apply it to the package. Note that unbundling is one of several areas in which we will locally patch to override upstreams.


== Exceptions ==
Follow the [https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling Bundling and Duplication of system libraries] section of the Packaging Guidelines.


Exceptions are granted on a case-by-case basis by FPC. You can look in the following section for help on making a case for why an exception should be granted. You should open up a ticket in the [https://fedorahosted.org/fpc/ FPC's trac] with information asked for below.
A list of known virtual provides for bundled libraries is maintained on a [[Bundled_Libraries_Virtual_Provides|separate page]].
 
If you are reporting a bundling situation and the package maintainer either won't request an exception as required by these Guidelines or won't conform to an FPC ruling on whether they are allowed to bundle, the next step is to open a ticket with [https://fedorahosted.org/fesco FESCo] to have the guideline enforced.
 
=== Standard questions ===
You must have answers to these standard questions before seeking an exception.
 
* Has the library behaviour been modified?  If so, how has it been modified?  If the library has been modified in ways that change the API or behaviour then there may be a case for copying.  Note that fixing bugs is not grounds to copy.  If the library has not been modified (ie: it can be used verbatim in the distro) there's little chance of an exception.
** Why haven't the changes been pushed to the upstream library?  If no attempt has been made to push the changes upstream, we shouldn't be supporting people forking out of laziness.
** Have the changes been proposed to the Fedora package maintainer for the library?  In some cases it may make sense for our package to take the changes despite upstream not taking them (for instance, if upstream for the library is dead).
* Could we make the forked version the canonical version within Fedora?  For instance, if upstream for the library is dead, is the package we're working on that bundles willing to make their fork a library that others can link against?
* Are the changes useful to consumers other than the bundling application?  If so why aren't we proposing that the library be released as a fork of the upstream library?
* Is upstream keeping the base library updated or are they continuously one or more versions behind the latest upstream release?
* What is the attitude of upstream towards bundling?  (Are they eager to remove the bundled version?  are they engaged with the upstream for the library?  Do they have a history of bundling?  Are they argumentative?)
* Overview of the security ramifications of bundling
* Does the maintainer of the Fedora package of the library being bundled have any comments about this?
* Is there a plan for unbundling the library at a later time?  Include things like what features would need to be added to the upstream library, a timeline for when those features would be merged, how we're helping to meet those goals, etc.
* Please include any relevant documentation -- mailing list links, bug reports for upstream or the bundled library, etc.
 
=== Some reasons you might be granted an exception ===
This section lists some reasons that might convince FPC that you have a valid reason to be granted an exception.  Exceptions are granted on a case by case basis and satisfying the rationale here is not a guarantee of an exception but it's a place to start building your case for why the package you work on is exceptional.
 
==== Kernel ====
If you're packaging the kernel and need to bundle a library you have a blanket exception.  The kernel is allowed to bundle libraries as it cannot use user space libraries.  Additionally, the kernel has a unique exemption from the requirement to note what code is being bundled with comments and Virtual Provides.  This has several reasons:
 
* The code bundled by the kernel is stripped down to the bare minimum and sometimes modified in other ways so that it is significantly different from the upstream code.
* The kernel code is scrutinized for security issues at a level that often exceeds the upstreams it is bundling.
* Due to the modifications to the code that is bundled into the kernel, once the code is included it evolves separately from upstream.
* There are many people working on the kernel capable and concerned with coding fixes for security issues, within Fedora, Red Hat, and the community at large.
 
The kernel maintainers are free to add virtual provides if they think it would be helpful to track security issues in the code in question but this is left to their discretion.
 
{{admon/note|Some things this exception doesn't cover|This exception does not apply to the user-space tools that are built from the kernel srpm nor does the blanket exception cover external kernel modules.  For external kernel modules, Fedora does not package these but some third-parties use our guidelines.  The criteria of not being able to use a userspace library should be one factor used to evaluate third party kernel module bundling but the ability of upstream to fix the code that they're bundling should also be taken into account even if that means some modules are prohibited.}}
 
==== Copylibs ====
The definition of a copylib is somewhat amorphous.  At its basic level, the upstream for the library intends for you to copy the source code of the library into your program, modify it to suit your needs, and then release your software with continuous, forked modifications to that source.  Just because you think you're dealing with a copylib does not guarantee that you will be granted an exception.  In particular, the programming practice that is common in some java, mono, and scripting language circles of copying external libraries that are otherwise from a separate upstream into the program's source and distributing them together is not allowed.  Programs which bundle libraries whose upstream is dead and make bugfixes to the bundled copy is not allowed.  As much as possible we want to have a single copy of a library in the distribution which everyone links to.
 
Some of the criteria that FPC uses to evaluate the copylib case are:
 
* Does the upstream library make actual releases?  If they do, then it is likely not a copylib.
* Does upstream define what they put together as a library or as reusable code snippets that are to be modified and incorporated as source in individual packages?  If the latter, it's more likely that the library is a copylib under this definition.
 
==== Needing unreleased features ====
 
When an application needs unreleased features of a library and that library has committed to those features (usually, the changes are checked into the trunk branch of the upstream's revision control system) but the library has not yet made a release that has that code an exception may be granted to bundle that library until the Fedora packages contain the necessary extra features.  '''Note''' that for this we would definitely want the standard questions regarding why the Fedora maintainer of the library feels we cannot include a backport of the feature/pre-release snapshot in our package, the timeline for the change to be merged and unbundling to occur to be answered so that we can make sure this is fixed should the package maintainer disappear or get busy with other things.
 
{{admon/note|Not for bugfixes|As noted earlier, bugfixes are rarely a good reason for this.  Most of the time bugfixes should be backported to the current Fedora package rather than bundling a library.}}
 
==== Modified beyond a certain extent ====
Modification of a library should not be the only reason given to justify a bundled copy as the two questions come up: why can't these changes go back to the upstream for the library?  Why isn't this library forked and released in such a way that others can benefit from the changes as well?  However, it can be one of the factors considered.  To provide a solid foundation for a bundling exception you should be able to answer those two questions.  An explanation that tells why the changes are only useful for the application that's bundling them, for instance.
 
* Example: recoll bundles unac but unac changed the API of unac and those changes were judged to only be of use to recoll and thus the bundling was allowed.
* Counter example: rsync bundles zlib.  However, the modified zlib is useful to others as the modified zlib is necessary in order to implement the rsync protocol.  In particular, the program zsync needs to have a similarly modified zlib in order to be of use.
 
==== Reverse Bundling ====
Reverse bundling when a portion of an upstream codebase has been forked into its own, separate library package.  These are then packaged in a separate package for Fedora.
 
When done for purposes of adding a backwards compat API for use by other packages this is okay (For instance, taking a module out of a newer version of the python stdlib and packaging it as a separate python module for use on older versions of python). Be sure to keep the forked code up to date with regard to the package it comes from.  Apply to the FPC for a virtual Provide to use for tracking purposes.
 
{{admon/note|Only Applies To Libraries|  Do not apply this exception to code that is copied from another package and included as part of a larger application.  This is only for code that is copied from another package into its own, separate library for the purposes of providing that API to other consumers.}}
 
When done for other purposes (for instance, splitting a useful utility function from a large monolithic upstream into its own package ) this may be a factor in deciding if the bundling is allowed but it may not be sufficient on its own.
 
=== Requirement if you bundle ===
 
* You must note that the library has been granted an exception in a spec file comment with a link to the FPC ticket where the exception was granted.
* If you bundle a library, you are required to add a virtual provide to your spec file to note that you are bundling.  This allows us to search for packages that may be affected by bugs or security issues in older versions of the library.  The notation should look like this:
<pre>
Provides: bundled(zlib) = 1.1.14
</pre>
 
<code>bundled()</code> denotes that this is a bundled library virtual provide rather than something that other packages would want to depend on.  Inside the paranthesis, the binary package that provides the library is listed.  (For instance,  <code>zlib</code>, <code>bind-libs</code>, <code>NetworkManager-glib</code>, <code>libpng</code>).  The version notes which version of the library was bundled.  If there's been a lot of incomplete backporting of changes from newer versions of the library, it can be hard to establish what version to use here.  A very general rule of thumb is to use the oldest version that seems reasonable as the reason we're doing this is to tell when a library contains issues that have been fixed in newer upstream versions.
 
{{admon/warning||No package should ever <code>Require:</code> a <code>bundled()</code> virtual dependency.}}
 
{{admon/note|Kernel Exemption|The kernel has an exemption from these requirements.  See the [[#Kernel | Kernel section]] for full information]}}
 
=== Packages granted exceptions ===
<table>
<tr><th>
Library
</th><th>
Provide
</th><th>
Reason
</th></tr>
 
<tr><td>
Any binc
</td><td>
bundled(binc)
</td><td>
copylib
</td></tr>
 
<tr><td>
Many modules from ccan
</td><td>
bundled(bobjenkins-hash)<br/>
bundled(ccan-container_of)<br/>
bundled(ccan-htable)<br/>
bundled(ccan-list)<br/>
bundled(ccan-check_type)<br/>
bundled(ccan-build_assert)<br/>
</td><td>
copylib.  CCAN is hard to track for two reasons: (1) CCAN encourages people to bundle individual files from its collection rather than a single library.  (2) Often the individual modules are from code which is also maintained separately in another location.  For these reasons, each module from CCAN needs to have its own virtual provide.  Please open a new bundling exception if you wish to use a module from CCAN that is not already listed here.
</td></tr>
 
<tr><td>
Any egglib
</td><td>
bundled(egglib)
</td><td>
copylib - use this link in spec: https://fedorahosted.org/fpc/ticket/174
</td></tr>
 
<tr><td>
Any gnulib
</td><td>
bundled(gnulib)
</td><td>
copylib - use this link in spec: https://fedorahosted.org/fpc/ticket/174
</td></tr>
 
<tr><td>
Any libiberty
</td><td>
bundled(libiberty)
</td><td>
copylib - use this link in spec: https://fedorahosted.org/fpc/ticket/174
</td></tr>
 
<tr><td>
Many instances of md5.c<ref>There are multiple md5 implementations.  The ones that have an actual library (libnss, libgcrypt, openssl, libmd, etc) are not covered by this exception.  The ones that are copied from other applications are.</ref>
</td><td>
bundled(md5-$IMPLEMENTATION)<ref>Change $IMPLEMENTATION depending on which implementation of md5 is being bundled.  The ones known so far are Peter Deutsch's version: [[Media:Deutsch-md5.c| bundled(md5-deutsch)]], a C++ port of Peter Deutsch's version: [http://sourceforge.net/p/sidplay-residfp/code/HEAD/tree/branches/libsidplayfp-1.3/utils/MD5/MD5.cpp bundled(md5-deutsch-c++)], Colin Plumb's [[Media:Plumb-md5.c| bundled(md5-plumb)]], Alexander Peslyak's [http://openwall.info/wiki/people/solar/software/public-domain-source-code/md5 bundled(md5-peslyak)], Ulrich Drepper's code from gcc [https://github.com/mirrors/gcc/blob/master/libiberty/md5.c bundled(md5-gcc)], A second implementation from Ulrich Drepper [http://git.busybox.net/busybox/tree/libbb/hash_md5_sha.c bundled(md5-drepper2)], and John Polstra's [[Media:Polstra-md5.c | bundled(md5-polstra)]].</ref>
</td><td>
copylib
</td></tr>
 
<tr><td>
Many instances of sha1.c<ref>There are multiple sha1 implementations.  The ones that have an actual library (libnss, libgcrypt, openssl, etc) are not covered by this exception.  The ones that are copied from other applications are.</ref>
</td><td>
bundled(sha1-$IMPLEMENTATION)<ref>Change $IMPLEMENTATION depending on which implementation of sha1 is being bundled.  The ones known so far are the Uwe Hollerbach-Peter C. Gutmann version (bundled(sha1-hollerbach)) as found in [http://ftp.debian.org/debian/pool/main/a/apt-cacher-ng/?C=M;O=D apt-cacher-ng]</ref>
</td><td>
copylib
</td></tr>
 
<tr><td>
time-api's use of timex from openjdk8
</td><td>
bundled(openjdk8-javax-time)
</td><td>
Reverse bundling providing a backwards compat API
</td></tr>
 
<tr><td>
unac's recoll
</td><td>
bundled(recoll)
</td><td>
this recoll has changes that are not applicable to other applications.
</td></tr>
 
<tr><td>
TexStudio's qcodeedit
</td><td>
bundled(qcodeedit)
</td><td>
TexStudio contains a forked copy of qcodeedit 2, which is at least two years dead. Since TexStudio is the only user, there is no benefit to a separated library, and permission to bundle has been granted.
</td></tr>
 
<tr><td>
binutils libraries (libbfd, libcpu, libopcodes, libdecnumber)
</td><td>
bundled(binutils)<ref>The version for binutils provides should be the date that the binutils checkout was made</ref>
</td><td>
If the package in question shares the same upstream as binutils (sourceware.org), they may bundle these libraries.  This is because the libraries are developed by the application authors as common functionality shared between several applications.  Being developers of both, they'll be intimately aware of both issues that arise in the libraries and know how to port to newer versions of the library as needed.  Note that, at the moment, all of these applications and libraries come from sourceware.org but not all of them are used in binutils.  The name was chosen as it seemed to be the more permanent and recognizable name.
</td></tr>
 
<tr><td>
Spring RTS's lua implementation
</td><td>
bundled(lua)
</td><td>
Spring RTS includes a forked and bundled copy of Lua which has Spring RTS specific patches applied, must link to streflop, and is configured differently from stock Lua (most importantly it needs lua_Number to be a float and not a double). Lua is particularly important because parts of the game code may be written in it, which must yield exactly identical results (also floating point operations!) on all platforms.
</td></tr>
 
<tr><td>
Any okjson
</td><td>
bundled(okjson)
</td><td>
copylib<ref>This exception was granted because the upstream explicitly intends for this library to be "vendored" and copied directly into any projects which use it. The Fedora Packaging Committee has a general feeling of distaste for this behavior.</ref>
</td></tr>
 
<tr><td>
libreplace in samba libraries
</td><td>
bundled(libreplace)
</td><td>
If the package in question shares the same upstream as samba, they may bundle the libreplace library.  This is because the libreplace library is developed by the application authors as common functionality shared between several applications.  Being developers of both, they'll be intimately aware of both issues that arise in the libraries and know how to port to newer versions of the library as needed.
</td></tr>
 
<tr><td>
boost in passenger
</td><td>
bundled(boost)
</td><td>
A bundling exception for boost within mod_passenger was granted, due to the intrusive nature of the forked changes, the efforts of the maintainer to merge as many of them as possible into the upstream boost source tree, and the visible efforts of the upstream to keep the bundled copy of boost in sync with the current boost releases.  This exception was contingent on a specific packager, wako666 being maintainer or comaintainer due to his efforts to redesign and merge the boost patches back to upstream boost.  https://fedorahosted.org/fpc/ticket/160
</td></tr>
 
<tr><td>
pyPdf in calibre
</td><td>
bundled(pyPdf)
</td><td>
A bundling exception for calibre's forked copy of pyPdf was granted, due to the intrusive nature of the forked changes, the specificity of the changes to calibre, and the fact that pyPdf seems to be abandoned upstream.  https://fedorahosted.org/fpc/ticket/167
</td></tr>
 
<tr><td>
libtidy in sigul
</td><td>
bundled(libtidy)
</td><td>
libtidy was allowed to bundle libtidy due to libtidy being dead upstream and the code being modified to handle epub instead of html.  https://fedorahosted.org/fpc/ticket/219
</td></tr>
 
<tr><td>
objectweb-asm in byteman
</td><td>
bundled(objectweb-asm)
</td><td>
Due to the specific nature of how byteman works, a special kind of exception was granted for byteman to bundle objectweb-asm.  See the ticket for details: https://fedorahosted.org/fpc/ticket/226
</td></tr>
 
<tr><td>
java_cup in byteman
</td><td>
bundled(java_cup)
</td><td>
Due to the specific nature of how byteman works, a special kind of exception was granted for byteman to bundle java_cup.  See the ticket for details: https://fedorahosted.org/fpc/ticket/226
</td></tr>
 
<tr><td>
Mersenne Twister 19937ar in anything
</td><td>
bundled(mt19937ar)
</td><td>
Since this algorithm (from http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html) is everywhere, a general bundling exception for it is approved.
</td></tr>
 
<tr><td>
fx2lib in sigrok-firmware-fx2lafw
</td><td>
bundled(fx2lib)
</td><td>
8051 hardware bits, only useful in this firmware context, not packaged/used elsewhere.
</td></tr>
 
<tr><td>
JAXP and JAX-WS in openjdk
</td><td>
bundled(JAXP) and bundled(JAX-WS)
</td><td>
The openJDK code contains copies of JAXP and JAX-WS that occasionally go out of sync with their upstream versions. The upstream is the same for both revisions, and the openJDK code assumes/depends on the behavior of the bundled versions.
</td></tr>
 
</table>
<references />
 
=== Packages granted temporary exceptions ===
<table>
<tr><th>
Library
</th><th>
Provide
</th><th>
Reason
</th></tr>
 
<tr><td>
t4k_common's liblinebreak
</td><td>
bundled(liblinebreak)
</td><td>
t4k_common contains a forked copy of an older version of liblinebreak. This is a temporary exception granted until the t4k_common upstream is able to port their code to use the newer system copy of liblinebreak.
</td></tr>
 
<tr><td>
boost-atomic in supercollider
</td><td>
bundled(boost-atomic)
</td><td>
A bundling exception for supercollider's forked copy of boost-atomic was granted through Fedora 20.  For Fedora 21 and beyond, supercollider will either use versions present in boost itself or the library will be built as a separate, public subpackage that supercollider dynamically links to.  https://fedorahosted.org/fpc/ticket/225
</td></tr>
 
<tr><td>
boost-lockfree in supercollider
</td><td>
bundled(boost-lockfree)
</td><td>
A bundling exception for supercollider's forked copy of boost-lockfree was granted through Fedora 20.  For Fedora 21 and beyond, supercollider will either use versions present in boost itself or the library will be built as a separate, public subpackage that supercollider dynamically links to.  https://fedorahosted.org/fpc/ticket/225
</td></tr>
 
<tr><td>
libraries in firefox and icecat
</td><td>
bundled(libtheora) <br />
bundled(libvorbis) <br />
bundled(libogg) <br />
bundled(opus) <br />
bundled(xulrunner) <br />
bundled(expat) <br />
bundled(graphite2) <br />
bundled(ots) <br />
bundled(hurfbuzz) <br />
bundled(soundtouch) <br />
bundled(snappy) <br />
bundled(double-conversion) <br />
</td><td>
firefox has a temporary bundling exception since it has an active security team tracking issues in their codebase. icecat has a temporary exception since it is a fork of firefox that closely tracks firefox's changes. We'll re-evaluate this before F22.
</td></tr>
 
<tr><td>
lua in nmap
</td><td>
bundled(lua)
</td><td>
A bundling exception for nmap's newer version of lua was granted through Fedora 19.  For Fedora 20 and beyond, lua will need to be unbundled.  https://fedorahosted.org/fpc/ticket/255
</td></tr>
 
<tr><td>
libntirpc in nfs-ganesha
</td><td>
bundled(libntirpc)
</td><td>
A bundling exception for nfs-ganesha to bundle libntirpc was granted through F23.  The nfs-ganesha authors are working to release libntirpc as its own package (its a fork of libtiprc).  This exception is to allow them the time to get that fork released separately.
</td></tr>
 
<tr><td>
event library provided by the kernel
</td><td>
bundled(kernel-event-lib)
</td><td>
A bundling exception for an event library provided by the kernel was granted through F22.  The kernelshould be providing the library as a shared library in the Linux 3.15 time frame so applications should plan on unbundling for F23.  https://fedorahosted.org/fpc/ticket/372
</td></tr>
 
<tr><td>
admesh bundled in slic3r
</td><td>
bundled(admesh)
</td><td>
A bundling exception has been granted for slic3r to bundle admesh through F20.  Will revisit for F21 if necessary.  The admesh and slic3r packagers are working to see if they can integrate the slic3r changes to the code into our admesh package.
</td></tr>
 
<tr><td>
libtommath bundled in Heimdal
</td><td>
bundled(libtommath)
</td><td>
Temporary exception for Heimdal to bundle libtommath until Fedora 22 granted. FPC expressed a willingness to look at granting an extension of another two releases if this one expires without the changes being merged upstream. To grant an extension we'd want to see that it was still being worked on, just not completed.
</td></tr>
 
<tr><td>
anything bundling jquery
</td><td>
bundled(jquery)
</td><td>
[https://fedorahosted.org/fpc/ticket/408 Temporary exception] for anything to bundle jquery.  jamielinux and patches are working on a [https://fedoraproject.org/wiki/Changes/jQuery coherent plan] to unbundle jquery in the next release or two.  Revisit this in F23 or sooner if jamielinux and patches are ready to start unbundling sooner.
</td></tr>
 
<tr><td>
ipython bundling javascript libraries: backbone, bootstrap, jquery, jquery-ui, marked, and google-caja
</td><td>
bundled(backbone)<br />
bundled(bootstrap)<br />
bundled(jquery)<br />
bundled(jquery-ui)<br />
bundled(marked)<br />
bundled(google-caja)
</td><td>
FPC decided that these javascript libraries could be temporarily bundled until we see how the jquery unbundling goes.  Once we have that experience we will push other bundled javascript to be unbundled.  The jquery unbundling should be done as part of the general mass unbundling of jquery effort.  The others are granted a temporary exception until one release after jquery is unbundled (this timeframe may move farther out if experience with jquery warrants that).  google-caja may be a special case of same-upstream, that may be revisited when the temporary exception expires.
</td></tr>
</table>


== Other distributions ==
== Other distributions ==
Line 430: Line 45:
* Debian -- http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles
* Debian -- http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles
* Talk given at pycon with a large section on not bundling libraries http://pycon.blip.tv/file/2072580/
* Talk given at pycon with a large section on not bundling libraries http://pycon.blip.tv/file/2072580/
[[Category: Packaging guidelines]]

Latest revision as of 19:54, 12 June 2021

Why Bundled Libraries are a problem

These are some of the reasons why it's painful for us to have bundled libraries in the distribution.

Security

  • When a security flaw is discovered in a library and bundling is not allowed, The library can be fixed in a single package, that package rebuilt, and when users download it, all the applications that use it are immediately protected. When bundling is allowed, the distribution has to find all the packages that the library occurs in by auditing source code or running a special tool over all elf files in all packages, then all of those packages have to be fixed, all of those packages have to be built, and users have to download and update each of the ones that they are using on their system before they are protected. There is much more work when bundled libraries are involved.
  • With security issues, people want to remove as much lag as they can between announcement of a problem and the fix being available for users. When libraries are unbundled, tools like vendor-sec can be used to alert distributions of problems that need patching in their packages before the announcement is made and then they can fix them with zero days of vulnerability. If bundling of libraries occurs, then the problem becomes how to get fixes out to all affected packages. If the distribution patches those packages, they must be careful to not leak the fact that there is a security vulnerability before they are allowed (which means they need to be careful who they share the information and what information they share with others). OTOH, if they do not patch the packages bundling libraries, then those packages are not protected on zero day, but only afterwards.
  • When a security flaw appears, the program has to either update to a non-affected version of the library or backport a fix. This can be problematic when the code of the library has undergone many API and code changes since the version that is being bundled and the security fixing patch is very widespread. Many conflicts can arise that need time to fix when trying to backport the fixes but porting the application code to the new API version can also take a lot of time.
  • We cannot implicitly trust an upstream application to be on top of security issues that are released in the packages that they care about. What happens if you are not following upstream development and don't know that a security release has been made? What happens if the developer that is responsible for watching bundled upstream development goes on vacation or quits your project? What happens if your application ceases active development? What happens if upstream stops active development and security fixes start originating from distro patches?

Forking

Forking is occurring. Once an application starts bundling libraries, it's easy for the project to include local patches to the library to add features that upstream doesn't have or fix bugs that upstream hasn't addressed. This has several negative effects.

  • When a security issue appears, it becomes harder to fix the application bundling the library. If you attempt to upgrade to a newer version, you have to make sure your important local modifications get ported to the new version. If you attempt to backport, you have to merge the upstream fix to your own code-base which may have conflicts with the local modifications.
  • When working with the library that comes from upstream, there is a community of people who are interested in that library to fall back on for help. When working on your own private copy that community may not be interested in helping you work on your modified sources since they don't have control or knowledge of what your modified sources do.
  • Forking dilutes one of the strengths of open-source development. Instead of a project getting stronger with more people supplying patches to help drive the project and build a bigger community, the community of people interested in it are splintering, developing more and more divergent code-bases, solving the same problem over and over in different ways in different private copies of the library. Instead of everyone benefiting, everyone has to pay.

Bugfixes

Bugfixes are usually of lesser importance from security issues but share the same issues of hanging onto lingering problems that have been fixed in the main package.

Old Code

  • Old versions of code linger on. If the application can bundle its own version of a library, the incentive to port to newer versions of the library are reduced. This exacerbates the problems of security and bugfix issues. Instead of progressively porting to newer versions of a library as time goes on, porting to newer versions becomes a chore that has to be performed at the same time as addressing a security flaw. This puts time pressure on the project when the work could have been spread out over a longer period if only the porting had been done all along.

Licensing

Although licensing issues can crop up in any project, projects which bundle code from different sources together are a special source of concern. They make auditing for license issues a larger project.

Treatment of Bundled Libraries

Follow the Bundling and Duplication of system libraries section of the Packaging Guidelines.

A list of known virtual provides for bundled libraries is maintained on a separate page.

Other distributions

As this is a place where we have to convince upstream that there's a problem, it's good to be able to point out that this is a problem for all distributions, not just Fedora. Here's links to other distribution's policies::