From Fedora Project Wiki
 
(21 intermediate revisions by 3 users not shown)
Line 28: Line 28:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1458665 #1458665]


== Detailed Description ==
== Detailed Description ==


To avoid maintaining multiple copies of this information, please read the Arbitrary Branching Focus Document (updated May 3rd, 2017) in the Factory2 space as a substitute for this document:
To avoid maintaining multiple copies of this information, please read the Arbitrary Branching Focus Document (updated May 3rd, 2017) in the Factory2 space as a substitute for this document:
[https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching]
[https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching]


For further information, view this section of Ralph Bean’s presentation at DevConf:
For further information, view this section of Ralph Bean’s presentation at DevConf:
[https://youtu.be/5gqccjyjwFk?t=26m27s]
[https://youtu.be/5gqccjyjwFk?t=26m27s https://youtu.be/5gqccjyjwFk?t=26m27s]
 
 
== Timeline ==
 
* Advertisement
** Pending FESCo approval, we will do the following on '''Friday May 26th'''.
*** With a Freeze Break Request, patch a banner into prod pkgdb advertising the Change (here) and [[Infrastructure/WhatHappenedToPkgdb]].  Pierre (pkgdb maintainer) is on vacation this week.  Ralph will send the FBR and apply the patch if approved.
*** Send an email to devel-announce linking to the Change (here) and to [[Infrastructure/WhatHappenedToPkgdb]].
* Client tooling for new package requests, branch requests, admin actions, etc.
** '''June''':  To be written and packaged.  The FAQ will instruct people how to use these things.
* Pagure has the ability to provide user-level API tokens to do things such as create tickets on projects
** '''June''': To be written and deployed in pagure.io ([https://pagure.io/pagure/issue/2250 https://pagure.io/pagure/issue/2250])
* PDC being populated
** '''June'''.  Write a script to sync existing pkgdb data to PDC, and run it.  Data in PDC will not be consulted yet.  Run the script again just before we turn off pkgdb in '''July'''.
* Pagure over dist-git in production (outside of Factory 2.0 control, in the hands of the Fedora Engineering team)
** '''July 5th'''.  If this slips, all other dates slip too.
* Moving over the scripts that query PkgDB to use Pagure
** Development, done (see the infra ansible repo).
** Perform the switchover the '''week of July 5th''', after pagure is in place.
* Bodhi being updated and configuration changed to use Pagure and PDC
** Development, done (see the bodhi pull request).
** Perform the switchover (config change) the '''week of July 5th''', after pagure is in place.
* Patch loopabull plugin to query PDC+pagure instead of pkgdb.  Patch to be submitted in '''June'''.
** https://pagure.io/releng-automation/blob/master/f/ansible/library/fedcontainer_rc.py
* Patch releng scripts that interact with pkgdb to work with the new data sources.  Patches to be submitted in '''June'''.
** https://pagure.io/releng/blob/master/f/scripts/block_retired.py
** https://pagure.io/releng/blob/master/f/scripts/fedretire
** https://pagure.io/releng/blob/master/f/scripts/find_unblocked_orphans.py
** https://pagure.io/releng/blob/master/f/scripts/get-critpath
** https://pagure.io/releng/blob/master/f/scripts/update-critpath
* We need a static JSON file generated from PDC’s data emulating PkgDB’s API, for GNOME Software.
** Write the script to produce this file in '''June''' and sync it to the proxies after freeze.
** Establish the HTTP redirect when we bring down PkgDB.
* PkgDB decommissioning
** Once all other pieces above are complete, do the following (expected '''week of July 10th'''):
** Deploy redirects to wiki page.
** Deploy redirects to JSON file.
** Take PkgDB out of haproxy.
** Re-run the sync-to-pdc script to catch the last of the data.
** Backup the PkgDB db and archive a copy for posterity.
** Wait a week for disaster to strike.
** Destroy PkgDB VMs.
* Extract hotness rules from old pkgdb archive.  Submit pull requests to pagure dist-git repos to restore the old monitoring configuration.


== Benefit to Fedora ==
== Benefit to Fedora ==
 
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->
See the “Background on PkgDB” and “Why Make This Change?” sections in the Arbitrary Branching Focus document linked above.
See the “Background on PkgDB” and “Why Make This Change?” sections in the Arbitrary Branching Focus document linked above.
 
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->


== Scope ==
== Scope ==
* Proposal owners: Ralph Bean (ralph/threebean) and Matt Prahl (mprahl)
* Proposal owners: We will do all of the work listed in the linked document.
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->


* Other developers: Pierre-Yves Chibon (pingou) for some of the Pagure enhancements <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers:
** We're relying on the infra team to get Pagure on top of dist-git.
** We're relying on releng to find any holes in our work plan and to help us make decisions on how to solve bugs.
** No direct work should be required from the general packaging group to support this Change.
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
* Release engineering: [https://pagure.io/releng/issue/6775 #6775] <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  include a link to the releng issue.  
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  include a link to the releng issue.  
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->


** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: Not affected <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->


Line 66: Line 111:
== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
Not backwards compatible but not significantly difficult to roll back.


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
N/A (not a System Wide Change)
Not backwards compatible but not significantly difficult to roll back because all changes will be done in Ansible.


== How To Test ==
== How To Test ==
Line 111: Line 155:
* Contingency deadline: N/A <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Contingency deadline: N/A <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? Yes, it blocks Fedora 27 Modularity.  They will be filing a Change document in the coming months to describe the F27 structure. <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks release? Yes, it blocks Fedora 27 Modularity.  They will be filing a Change document in the coming months to describe the F27 structure <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Blocks product? Fedora 27 <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
* Blocks product? Fedora 27 <!-- Applicable for Changes that blocks specific product release/Fedora.next -->


Line 127: Line 171:
-->
-->


[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF27]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 134: Line 178:


<!-- Select proper category, default is Self Contained Change -->
<!-- Select proper category, default is Self Contained Change -->
[[Category:SelfContainedChange]]
[[Category:SystemWideChange]]
<!-- [[Category:SystemWideChange]] -->

Latest revision as of 07:20, 5 June 2017

Arbitrary Branching

Summary

Tooling changes to support the new way of branching for Fedora 27

Owner

  • Name: Matt Prahl
  • Email: mprahl@redhat.com
  • Release notes owner:

Current status

  • Targeted release: Fedora 27
  • Last updated: 2017-06-05
  • Tracker bug: #1458665

Detailed Description

To avoid maintaining multiple copies of this information, please read the Arbitrary Branching Focus Document (updated May 3rd, 2017) in the Factory2 space as a substitute for this document: https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching

For further information, view this section of Ralph Bean’s presentation at DevConf: https://youtu.be/5gqccjyjwFk?t=26m27s


Timeline

  • Advertisement
    • Pending FESCo approval, we will do the following on Friday May 26th.
      • With a Freeze Break Request, patch a banner into prod pkgdb advertising the Change (here) and Infrastructure/WhatHappenedToPkgdb. Pierre (pkgdb maintainer) is on vacation this week. Ralph will send the FBR and apply the patch if approved.
      • Send an email to devel-announce linking to the Change (here) and to Infrastructure/WhatHappenedToPkgdb.
  • Client tooling for new package requests, branch requests, admin actions, etc.
    • June: To be written and packaged. The FAQ will instruct people how to use these things.
  • Pagure has the ability to provide user-level API tokens to do things such as create tickets on projects
  • PDC being populated
    • June. Write a script to sync existing pkgdb data to PDC, and run it. Data in PDC will not be consulted yet. Run the script again just before we turn off pkgdb in July.
  • Pagure over dist-git in production (outside of Factory 2.0 control, in the hands of the Fedora Engineering team)
    • July 5th. If this slips, all other dates slip too.
  • Moving over the scripts that query PkgDB to use Pagure
    • Development, done (see the infra ansible repo).
    • Perform the switchover the week of July 5th, after pagure is in place.
  • Bodhi being updated and configuration changed to use Pagure and PDC
    • Development, done (see the bodhi pull request).
    • Perform the switchover (config change) the week of July 5th, after pagure is in place.
  • Patch loopabull plugin to query PDC+pagure instead of pkgdb. Patch to be submitted in June.
  • Patch releng scripts that interact with pkgdb to work with the new data sources. Patches to be submitted in June.
  • We need a static JSON file generated from PDC’s data emulating PkgDB’s API, for GNOME Software.
    • Write the script to produce this file in June and sync it to the proxies after freeze.
    • Establish the HTTP redirect when we bring down PkgDB.
  • PkgDB decommissioning
    • Once all other pieces above are complete, do the following (expected week of July 10th):
    • Deploy redirects to wiki page.
    • Deploy redirects to JSON file.
    • Take PkgDB out of haproxy.
    • Re-run the sync-to-pdc script to catch the last of the data.
    • Backup the PkgDB db and archive a copy for posterity.
    • Wait a week for disaster to strike.
    • Destroy PkgDB VMs.
  • Extract hotness rules from old pkgdb archive. Submit pull requests to pagure dist-git repos to restore the old monitoring configuration.

Benefit to Fedora

See the “Background on PkgDB” and “Why Make This Change?” sections in the Arbitrary Branching Focus document linked above.

Scope

  • Proposal owners: We will do all of the work listed in the linked document.
  • Other developers:
    • We're relying on the infra team to get Pagure on top of dist-git.
    • We're relying on releng to find any holes in our work plan and to help us make decisions on how to solve bugs.
    • No direct work should be required from the general packaging group to support this Change.
  • Release engineering: #6775
  • Policies and guidelines: Packaging guidelines would change and would be updated after the changes are implemented in staging.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

Not backwards compatible but not significantly difficult to roll back because all changes will be done in Ansible.

How To Test

  • Request new packages
  • Request new branches with non-standard names (i.e. not f25, f26, etc.) and build with them
  • Request new branches for standard names that are still supported (i.e. f25, f26, epel6, epel7) and build with them
  • Retire a package
  • Orphan a package

User Experience

See the “How To Make This Change?” section in the Arbitrary Branching Focus document linked above.

Dependencies

  • Pagure over dist-git will need to be updated in staging
  • Pagure over dist-git will need to be deployed in production

Contingency Plan

  • Contingency mechanism: Re-enable PkgDB and roll back our script changes in Ansible
  • Contingency deadline: N/A
  • Blocks release? Yes, it blocks Fedora 27 Modularity. They will be filing a Change document in the coming months to describe the F27 structure
  • Blocks product? Fedora 27

Documentation

Documentation on new processes would be provided at a later date if there is approval on this Change Proposal.

Release Notes