From Fedora Project Wiki
(describe the scale of the plan)
 
(24 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{admon/tip | Guidance | For details on how to fill out this form, see the [https://docs.fedoraproject.org/en-US/program_management/changes_guide/ documentation].}}
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->


= SPDX License Phase 2 <!-- The name of your change proposal --> =
= SPDX License Phase 2 <!-- The name of your change proposal --> =


== Summary ==
== Summary ==
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. Note that motivation for the change should be in the Benefit to Fedora section below, and this part should answer the question "What?" rather than "Why?". -->
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. Note that motivation for the change should be in the Benefit to Fedora section below, and this part should answer the question "What?" rather than "Why?". -->
Second phase of transition from using Fedora's short name for licenses to [https://spdx.org/licenses/ SPDX identifiers] in the License: field of Fedora package spec files. This phase addresses how to update the License: field for existing packages, including documenting more specific guidance on how to find licenses in a package.  
Second phase of transition from using Fedora's short names for licenses to [https://spdx.org/licenses/ SPDX identifiers] in the License: field of Fedora package spec files. This phase addresses how to update the License: field for existing packages, including documenting more specific guidance on how to find licenses in a package.
 


== Owner ==
== Owner ==
Line 15: Line 13:
This should link to your home wiki page so we know who you are.  
This should link to your home wiki page so we know who you are.  
-->
-->
* Name: [[User:msuchy| Miroslav Suchý]]
* Name: [[User:msuchy| Miroslav Suchý]], [[User:jlovejoy| Jilayne Lovejoy]], [[User:ngompa| Neal Gompa]], [[User:dcantrell| David Cantrell]], [[User:ref| Richard Fontana]], [[User:mattdm| Matthew Miller]]
* Name: [[User:jlovejoy| Jilayne Lovejoy]]
* Name: [[User:ngompa| Neal Gompa]]
* Name: [[User:dcantrell| David Cantrell]]
* Name: [[User:ref| Richard Fontana]]
* Name: [[User:mattdm| Matthew Miller]]


<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
Line 31: Line 24:


== Current status ==
== Current status ==
[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF39]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 41: Line 34:
<!-- [[Category:SystemWideChange]] -->
<!-- [[Category:SystemWideChange]] -->


* Targeted release: [[Releases/39 | Fedora Linux 39 ]]  
* Targeted release: [https://docs.fedoraproject.org/en-US/releases/f39 Fedora Linux 39]
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
Line 49: Line 42:
ON_QA -> change is fully code complete
ON_QA -> change is fully code complete
-->
-->
* FESCo issue: <will be assigned by the Wrangler>
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/CXASGCNBQOWXFFA2X5KNCI2GF6RXQGHL/ devel thread]
* Tracker bug: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/2972 #2972]
* Release notes tracker: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=2184184 #2184184]
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/974 #974]
 
* [https://docs.google.com/spreadsheets/d/1QVMEzXWML-6_Mrlln02axFAaRKCQ8zE807rpCjus-8s/edit#gid=0 Burndow chart]


== Detailed Description ==
== Detailed Description ==
Line 57: Line 53:
This is follow-up of [[Changes/SPDX_Licenses_Phase_1|Phase 1]]. During this phase, all remaining packages should be migrated to use SPDX license identifiers in the License: field of the package spec file.  
This is follow-up of [[Changes/SPDX_Licenses_Phase_1|Phase 1]]. During this phase, all remaining packages should be migrated to use SPDX license identifiers in the License: field of the package spec file.  


Options for how to best migrate still need to be specified in terms of how much automation can be applied versus taking this as an opportunity as a more thorough license review. Some initial guidance and identification of challenges has been identified at [https://docs.fedoraproject.org/en-US/legal/update-existing-packages/|Updating Existing Packages] but more details are needed, especially in regard to Fedora "category" short names.
So far, package maintainers have been updating their packages in accordance with the guidance provided at https://docs.fedoraproject.org/en-US/legal/update-existing-packages/ and filing issues in the [https://gitlab.com/fedora/legal/fedora-license-data fedora-license-data repo]. Miroslav has been tracking how many packages that have been updated. Given the large number of packages in Fedora, this progress is good, but slow.  


If the migration is not possible (e.g. needs clarification from legal), then a Bugzilla issue has to be created.
We are considering options for how to more quickly migrate, in terms of how much automation can be applied as opposed to taking this as an opportunity as a more thorough license review. Automated updating of Fedora legacy abbreviations to SPDX expressions is only possible to a limited extent due to different criteria used in each system, most notably the use of [https://docs.fedoraproject.org/en-US/legal/update-existing-packages/#_updating_existing_packages_callaway_short_name_categories "category" short names in the Fedora legacy system]. Also, the [https://docs.fedoraproject.org/en-US/legal/license-field/ policy for populating the License: field] has been clarified with a view to eliminating inconsistent guidance in past documentation, which may mean that a determination made in the past would be different today. Nonetheless, automated updates for a curated set of Fedora legacy abbreviations can help speed up the migration and serve as a reminder and helping hand to package maintainers who may have not started this process. If the automatic migration is not possible (e.g., needs clarification from legal), then a Bugzilla issue has to be created.


Plan:
Plan:
* Packages with legacy license expressions that can be mapped to new fedora-license-data expressions will be automatically converted if the package maintainer has not already taken care of it.  Automatic conversion is not possible for a lot of packages and will need direct review.  Legacy license identifiers that cannot be automatically converted:
* Hold a hackfest focusing on a limited set of Fedora packages. Feedback from the hackfest can then be used to improve documentation related to updating existing packages.
** BSD
* Review of licenses for which there seems to be a one-to-one mapping from Fedora legacy abbreviations to SPDX identifiers to ensure reliable mapping. license-fedora2spdx will then be updated to use that curated set of mappings.
** MIT
* Packages using legacy license expressions will be automatically converted if the package maintainer has not already taken care of it. Packages with compound legacy license expressions will only be converted if all included identifiers can map to fedora-license-dataThat is, there is no mixing of SPDX and legacy Fedora identifiers.
** Legacy identifiers that are not in SPDX and do not yet have a LicenseRef- identifier in fedora-license-data (this may include Public Domain, Distributable, and many licenses used for firmware packages)
** The conversion will be done in waves to minimize errors in automation (e.g., 100 packages in the first week, 200 packages in another week, ...).  A pull request for the package spec file will be created as part of the conversion. If the maintainer does not respond to the pull-request within 14 days, the pull request will be merged by Change owners.
** As mentioned above and at https://docs.fedoraproject.org/en-US/legal/update-existing-packages/, automatic conversion is not possible for a lot of packages and will need direct review.  For example, legacy license identifiers that cannot be automatically converted: BSD, MIT, with exceptions.
* Regular office hours will be held to help answer questions for package maintainers.


* Packages with compound legacy license expressions will only be converted if all included identifiers can map to fedora-license-data.  That is, there is no mixing of SPDX and legacy Fedora identifiers.
As of 2023-03-13 the automatic conversion can be done for approximately 8096 packages.  


* Regular office hours will be held to help answer questions for package maintainers.
Around Fedora 39 Beta owners will file a BZ for any package that does not update the license tag. As of 2023-03-13 that would be 12206 packages. But owners expect a much lower number by that time.
.  
 
As of 2023-01-17 the automatic conversion can be done for 8986 packages. The conversion will be done in waves to minimize errors in automation (e.g. 100 packages in first week, 200 packages in other week, ...).
This Change may be followed by Phase 3 where want to provide package maintainers with a service that will notify them when it is likely that the upstream license has changed. But such service is not part of this Phase 2 Change proposal.


== Feedback ==
== Feedback ==
Line 79: Line 77:


Discussions on mailing list:
Discussions on mailing list:
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/EXGT34EJPG3G4FJZ4R2SRAI342QDHFOF/|SPDX Statistics]
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/EXGT34EJPG3G4FJZ4R2SRAI342QDHFOF/ SPDX Statistics]
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/NZIFT62REORSNS6MMES446PMCOOSEPSA/|SPDX Change Update]
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/NZIFT62REORSNS6MMES446PMCOOSEPSA/ SPDX Change Update]
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/3TGCSROJTSX5PXZLKOHCOMVIBTZDORNS/|SPDX - How to handle MIT and BSD]
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/3TGCSROJTSX5PXZLKOHCOMVIBTZDORNS/ SPDX - How to handle MIT and BSD]


Challenges:  
Challenges:  
* license-fedora2spdx tool uses mapping of Fedora shortnames to SPDX ids using the [https://gitlab.com/fedora/legal/fedora-license-data/-/tree/main|fedora-license-data] to suggest SPDX ids. Where there is a one-to-one mapping, the package maintainer could simply update the License field: and move on.  
* license-fedora2spdx tool uses mapping of legacy Fedora short names to SPDX identifiers using the [https://gitlab.com/fedora/legal/fedora-license-data/-/tree/main|fedora-license-data] to suggest SPDX identifiers. Where there is an apparent one-to-one mapping, the package maintainer could simply update the License field: and move on.  
* for many packages, the use of Fedora shortnames that represent multiple licenses, further inspection will be needed. Currently, Fedora documentation provide sparse advice on how to do this inspection and thus, a range of methods are used.  
* for many packages, particularly packages that use "umbrella" legacy short names that may refer to a large set of unrelated or loosely-related licenses, further inspection will be needed. Currently, Fedora documentation provides sparse advice on how to do this inspection and thus, a range of methods are used.


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 115: Line 113:
     https://fedoraproject.org/wiki/Changes/perl5.26 (major upgrade to a popular software stack, visible to users of that stack)
     https://fedoraproject.org/wiki/Changes/perl5.26 (major upgrade to a popular software stack, visible to users of that stack)
-->
-->
The use of a standardized identifier for license will align Fedora with other distributions. And allows efficient and reliable identification of licenses. Depending on the level of diligence done in this transition, Fedora could be positioned as a leader and contributor to better license info upstream (of Fedora).
The use of standardized identifiers for licenses will align Fedora with other distributions and facilitates efficient and reliable identification of licenses. Depending on the level of diligence done in this transition, Fedora could be positioned as a leader and contributor to better license information upstream (of Fedora).


== Scope ==
== Scope ==
Line 130: Line 128:
** Create tracking BZ for packages with unclear transition path
** Create tracking BZ for packages with unclear transition path
** Submit BZ for packages that cannot migrate in time.
** Submit BZ for packages that cannot migrate in time.
Owners will start doing this after Fedora 38 branching. I.e. after 2023-02-07.


* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
** All packages (during the package review) should use the SPDX expression.  
** All packages (during the package review) should use the SPDX expression. - this is redundant as this is already approved since Phase 1, but should be reminded.
** Migrate the existing License tag from a short name to an SPDX expression.
** Migrate the existing License tag from a short name to an SPDX expression.


Line 141: Line 138:
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->


* Policies and guidelines: Licensing page, packaging guidelines has to be altered.  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines: Licensing page will need to be altered. But almost all the work has been done in Phase 1.  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. Please submit a pull request with the proposed changes before submitting your Change proposal. -->
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. Please submit a pull request with the proposed changes before submitting your Change proposal. -->


Line 210: Line 207:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
 
[https://docs.fedoraproject.org/en-US/legal/allowed-licenses/|Allowed Licenses]
 
[https://docs.fedoraproject.org/en-US/legal/update-existing-packages/#_process_used Update existing packages]


== Release Notes ==
== Release Notes ==
Line 218: Line 218:
Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.  
Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.  
-->
-->
In Fedora 39, RPM packages use SPDX identifiers as a standard. More than half of the packages have been migrated to SPDX identifiers. The remaining packages are estimated to be migrated in two upcoming releases of Fedora.

Latest revision as of 06:47, 9 October 2023


SPDX License Phase 2

Summary

Second phase of transition from using Fedora's short names for licenses to SPDX identifiers in the License: field of Fedora package spec files. This phase addresses how to update the License: field for existing packages, including documenting more specific guidance on how to find licenses in a package.

Owner

  • Email: msuchy@redhat.com, dcantrell@redhat.com, jlovejoy@redhat.com, ngompa13@gmail.com, rfontana@redhat.com


Current status

Detailed Description

This is follow-up of Phase 1. During this phase, all remaining packages should be migrated to use SPDX license identifiers in the License: field of the package spec file.

So far, package maintainers have been updating their packages in accordance with the guidance provided at https://docs.fedoraproject.org/en-US/legal/update-existing-packages/ and filing issues in the fedora-license-data repo. Miroslav has been tracking how many packages that have been updated. Given the large number of packages in Fedora, this progress is good, but slow.

We are considering options for how to more quickly migrate, in terms of how much automation can be applied as opposed to taking this as an opportunity as a more thorough license review. Automated updating of Fedora legacy abbreviations to SPDX expressions is only possible to a limited extent due to different criteria used in each system, most notably the use of "category" short names in the Fedora legacy system. Also, the policy for populating the License: field has been clarified with a view to eliminating inconsistent guidance in past documentation, which may mean that a determination made in the past would be different today. Nonetheless, automated updates for a curated set of Fedora legacy abbreviations can help speed up the migration and serve as a reminder and helping hand to package maintainers who may have not started this process. If the automatic migration is not possible (e.g., needs clarification from legal), then a Bugzilla issue has to be created.

Plan:

  • Hold a hackfest focusing on a limited set of Fedora packages. Feedback from the hackfest can then be used to improve documentation related to updating existing packages.
  • Review of licenses for which there seems to be a one-to-one mapping from Fedora legacy abbreviations to SPDX identifiers to ensure reliable mapping. license-fedora2spdx will then be updated to use that curated set of mappings.
  • Packages using legacy license expressions will be automatically converted if the package maintainer has not already taken care of it. Packages with compound legacy license expressions will only be converted if all included identifiers can map to fedora-license-data. That is, there is no mixing of SPDX and legacy Fedora identifiers.
    • The conversion will be done in waves to minimize errors in automation (e.g., 100 packages in the first week, 200 packages in another week, ...). A pull request for the package spec file will be created as part of the conversion. If the maintainer does not respond to the pull-request within 14 days, the pull request will be merged by Change owners.
    • As mentioned above and at https://docs.fedoraproject.org/en-US/legal/update-existing-packages/, automatic conversion is not possible for a lot of packages and will need direct review. For example, legacy license identifiers that cannot be automatically converted: BSD, MIT, with exceptions.
  • Regular office hours will be held to help answer questions for package maintainers.

As of 2023-03-13 the automatic conversion can be done for approximately 8096 packages.

Around Fedora 39 Beta owners will file a BZ for any package that does not update the license tag. As of 2023-03-13 that would be 12206 packages. But owners expect a much lower number by that time.

This Change may be followed by Phase 3 where want to provide package maintainers with a service that will notify them when it is likely that the upstream license has changed. But such service is not part of this Phase 2 Change proposal.

Feedback

See feedback section of Phase 1

Discussions on mailing list:

Challenges:

  • license-fedora2spdx tool uses mapping of legacy Fedora short names to SPDX identifiers using the [1] to suggest SPDX identifiers. Where there is an apparent one-to-one mapping, the package maintainer could simply update the License field: and move on.
  • for many packages, particularly packages that use "umbrella" legacy short names that may refer to a large set of unrelated or loosely-related licenses, further inspection will be needed. Currently, Fedora documentation provides sparse advice on how to do this inspection and thus, a range of methods are used.

Benefit to Fedora

The use of standardized identifiers for licenses will align Fedora with other distributions and facilitates efficient and reliable identification of licenses. Depending on the level of diligence done in this transition, Fedora could be positioned as a leader and contributor to better license information upstream (of Fedora).

Scope

  • Prep work:
    • poll package maintainers on methods and tools used for license inspection
    • create additional guidance (using info from above)
    • planning/brainstorming on most efficient way to update packages, especially those that do not have one-to-one identifier update and will need closer inspection
  • Proposal owners (things sorted by done/todo and by priorities):
    • Identify all remaining packages.
    • Notify owners of these packages.
    • After a grace period, submit PR to a package where the transition is easy.
    • Create tracking BZ for packages with unclear transition path
    • Submit BZ for packages that cannot migrate in time.
  • Other developers:
    • All packages (during the package review) should use the SPDX expression. - this is redundant as this is already approved since Phase 1, but should be reminded.
    • Migrate the existing License tag from a short name to an SPDX expression.
  • Policies and guidelines: Licensing page will need to be altered. But almost all the work has been done in Phase 1.
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Objectives:

Upgrade/compatibility impact

License strings are not used anything in run time. This change will not affect the upgrade or runtime of Fedora.

During the transition period, developer tools like rpminspect, licensecheck, etc. may produce false negatives. And we have to define a date where we flip these tools from old Fedora's short names to the SPDX formula.

How To Test

See How to test section of Phase 1

User Experience

Users should be able to use standard software tools that audit licenses. E.g. for Software Bills of Materials.

Dependencies

No other dependencies.

Contingency Plan

  • Contingency mechanism: There will be no way back. We either rollback in Phase 1. Once we will start Phase 2 we will be beyond of point with no return.
  • Contingency deadline: Beta freeze. But it is expected that not all packages will be converted by that time and the change will continue in the next release.
  • Blocks release? No. This change has no impact on runtime of any package.

Documentation

Licenses

Update existing packages

Release Notes

In Fedora 39, RPM packages use SPDX identifiers as a standard. More than half of the packages have been migrated to SPDX identifiers. The remaining packages are estimated to be migrated in two upcoming releases of Fedora.