From Fedora Project Wiki
 
(12 intermediate revisions by 3 users not shown)
Line 28: Line 28:
 
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 
-->
 
-->
* Tracker bug: <will be assigned by the Wrangler>
+
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1458665 #1458665]
  
 
== Detailed Description ==
 
== Detailed Description ==
Line 37: Line 37:
 
For further information, view this section of Ralph Bean’s presentation at DevConf:
 
For further information, view this section of Ralph Bean’s presentation at DevConf:
 
[https://youtu.be/5gqccjyjwFk?t=26m27s https://youtu.be/5gqccjyjwFk?t=26m27s]
 
[https://youtu.be/5gqccjyjwFk?t=26m27s https://youtu.be/5gqccjyjwFk?t=26m27s]
 +
 +
 +
== Timeline ==
 +
 +
* Advertisement
 +
** Pending FESCo approval, we will do the following on '''Friday May 26th'''.
 +
*** With a Freeze Break Request, patch a banner into prod pkgdb advertising the Change (here) and [[Infrastructure/WhatHappenedToPkgdb]].  Pierre (pkgdb maintainer) is on vacation this week.  Ralph will send the FBR and apply the patch if approved.
 +
*** Send an email to devel-announce linking to the Change (here) and to [[Infrastructure/WhatHappenedToPkgdb]].
 +
* Client tooling for new package requests, branch requests, admin actions, etc.
 +
** '''June''':  To be written and packaged.  The FAQ will instruct people how to use these things.
 +
* Pagure has the ability to provide user-level API tokens to do things such as create tickets on projects
 +
** '''June''': To be written and deployed in pagure.io ([https://pagure.io/pagure/issue/2250 https://pagure.io/pagure/issue/2250])
 +
* PDC being populated
 +
** '''June'''.  Write a script to sync existing pkgdb data to PDC, and run it.  Data in PDC will not be consulted yet.  Run the script again just before we turn off pkgdb in '''July'''.
 +
* Pagure over dist-git in production (outside of Factory 2.0 control, in the hands of the Fedora Engineering team)
 +
** '''July 5th'''.  If this slips, all other dates slip too.
 +
* Moving over the scripts that query PkgDB to use Pagure
 +
** Development, done (see the infra ansible repo).
 +
** Perform the switchover the '''week of July 5th''', after pagure is in place.
 +
* Bodhi being updated and configuration changed to use Pagure and PDC
 +
** Development, done (see the bodhi pull request).
 +
** Perform the switchover (config change) the '''week of July 5th''', after pagure is in place.
 +
* Patch loopabull plugin to query PDC+pagure instead of pkgdb.  Patch to be submitted in '''June'''.
 +
** https://pagure.io/releng-automation/blob/master/f/ansible/library/fedcontainer_rc.py
 +
* Patch releng scripts that interact with pkgdb to work with the new data sources.  Patches to be submitted in '''June'''.
 +
** https://pagure.io/releng/blob/master/f/scripts/block_retired.py
 +
** https://pagure.io/releng/blob/master/f/scripts/fedretire
 +
** https://pagure.io/releng/blob/master/f/scripts/find_unblocked_orphans.py
 +
** https://pagure.io/releng/blob/master/f/scripts/get-critpath
 +
** https://pagure.io/releng/blob/master/f/scripts/update-critpath
 +
* We need a static JSON file generated from PDC’s data emulating PkgDB’s API, for GNOME Software.
 +
** Write the script to produce this file in '''June''' and sync it to the proxies after freeze.
 +
** Establish the HTTP redirect when we bring down PkgDB.
 +
* PkgDB decommissioning
 +
** Once all other pieces above are complete, do the following (expected '''week of July 10th'''):
 +
** Deploy redirects to wiki page.
 +
** Deploy redirects to JSON file.
 +
** Take PkgDB out of haproxy.
 +
** Re-run the sync-to-pdc script to catch the last of the data.
 +
** Backup the PkgDB db and archive a copy for posterity.
 +
** Wait a week for disaster to strike.
 +
** Destroy PkgDB VMs.
 +
* Extract hotness rules from old pkgdb archive.  Submit pull requests to pagure dist-git repos to restore the old monitoring configuration.
  
 
== Benefit to Fedora ==
 
== Benefit to Fedora ==
Line 57: Line 100:
 
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
 
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
  
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: Not affected <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
 
<!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
  
Line 128: Line 171:
 
-->
 
-->
  
[[Category:ChangePageIncomplete]]
+
[[Category:ChangeAcceptedF27]]
 
<!-- When your change proposal page is completed and ready for review and announcement -->
 
<!-- When your change proposal page is completed and ready for review and announcement -->
 
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
 
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Latest revision as of 07:20, 5 June 2017

Arbitrary Branching

Summary

Tooling changes to support the new way of branching for Fedora 27

Owner

  • Name: Matt Prahl
  • Email: mprahl@redhat.com
  • Release notes owner:

Current status

  • Targeted release: Fedora 27
  • Last updated: 2017-06-05
  • Tracker bug: #1458665

Detailed Description

To avoid maintaining multiple copies of this information, please read the Arbitrary Branching Focus Document (updated May 3rd, 2017) in the Factory2 space as a substitute for this document: https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching

For further information, view this section of Ralph Bean’s presentation at DevConf: https://youtu.be/5gqccjyjwFk?t=26m27s


Timeline

  • Advertisement
    • Pending FESCo approval, we will do the following on Friday May 26th.
      • With a Freeze Break Request, patch a banner into prod pkgdb advertising the Change (here) and Infrastructure/WhatHappenedToPkgdb. Pierre (pkgdb maintainer) is on vacation this week. Ralph will send the FBR and apply the patch if approved.
      • Send an email to devel-announce linking to the Change (here) and to Infrastructure/WhatHappenedToPkgdb.
  • Client tooling for new package requests, branch requests, admin actions, etc.
    • June: To be written and packaged. The FAQ will instruct people how to use these things.
  • Pagure has the ability to provide user-level API tokens to do things such as create tickets on projects
  • PDC being populated
    • June. Write a script to sync existing pkgdb data to PDC, and run it. Data in PDC will not be consulted yet. Run the script again just before we turn off pkgdb in July.
  • Pagure over dist-git in production (outside of Factory 2.0 control, in the hands of the Fedora Engineering team)
    • July 5th. If this slips, all other dates slip too.
  • Moving over the scripts that query PkgDB to use Pagure
    • Development, done (see the infra ansible repo).
    • Perform the switchover the week of July 5th, after pagure is in place.
  • Bodhi being updated and configuration changed to use Pagure and PDC
    • Development, done (see the bodhi pull request).
    • Perform the switchover (config change) the week of July 5th, after pagure is in place.
  • Patch loopabull plugin to query PDC+pagure instead of pkgdb. Patch to be submitted in June.
  • Patch releng scripts that interact with pkgdb to work with the new data sources. Patches to be submitted in June.
  • We need a static JSON file generated from PDC’s data emulating PkgDB’s API, for GNOME Software.
    • Write the script to produce this file in June and sync it to the proxies after freeze.
    • Establish the HTTP redirect when we bring down PkgDB.
  • PkgDB decommissioning
    • Once all other pieces above are complete, do the following (expected week of July 10th):
    • Deploy redirects to wiki page.
    • Deploy redirects to JSON file.
    • Take PkgDB out of haproxy.
    • Re-run the sync-to-pdc script to catch the last of the data.
    • Backup the PkgDB db and archive a copy for posterity.
    • Wait a week for disaster to strike.
    • Destroy PkgDB VMs.
  • Extract hotness rules from old pkgdb archive. Submit pull requests to pagure dist-git repos to restore the old monitoring configuration.

Benefit to Fedora

See the “Background on PkgDB” and “Why Make This Change?” sections in the Arbitrary Branching Focus document linked above.

Scope

  • Proposal owners: We will do all of the work listed in the linked document.
  • Other developers:
    • We're relying on the infra team to get Pagure on top of dist-git.
    • We're relying on releng to find any holes in our work plan and to help us make decisions on how to solve bugs.
    • No direct work should be required from the general packaging group to support this Change.
  • Release engineering: #6775
  • Policies and guidelines: Packaging guidelines would change and would be updated after the changes are implemented in staging.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

Not backwards compatible but not significantly difficult to roll back because all changes will be done in Ansible.

How To Test

  • Request new packages
  • Request new branches with non-standard names (i.e. not f25, f26, etc.) and build with them
  • Request new branches for standard names that are still supported (i.e. f25, f26, epel6, epel7) and build with them
  • Retire a package
  • Orphan a package

User Experience

See the “How To Make This Change?” section in the Arbitrary Branching Focus document linked above.

Dependencies

  • Pagure over dist-git will need to be updated in staging
  • Pagure over dist-git will need to be deployed in production

Contingency Plan

  • Contingency mechanism: Re-enable PkgDB and roll back our script changes in Ansible
  • Contingency deadline: N/A
  • Blocks release? Yes, it blocks Fedora 27 Modularity. They will be filing a Change document in the coming months to describe the F27 structure
  • Blocks product? Fedora 27

Documentation

Documentation on new processes would be provided at a later date if there is approval on this Change Proposal.

Release Notes