This SOP explains the procedure for requesting stable pushes from the ReleaseEngineering team during Milestone freezes. This task is usually performed by QA.
General procedure
During the Beta and Final Milestone freezes, no updates are automatically pushed stable. However, fixes for blocker bugs must be pushed stable during this time, and fixes for freeze exception bugs may be pushed stable.
During each freeze, the QA team must regularly request pushes as appropriate. Requests are filed in the release engineering forge repository. There should be one ticket per release. QA opens the ticket for the first stable push request, release engineering does the push and closes it, then QA re-opens it for the next request, and so on. Here is an example ticket (for Fedora 44). At all times, the blocker bugs web application provides a current template for the request text, on the Requests page. The request should always be based on this template, but may need to be edited.
To request a stable push, create the ticket if this is the first request for the release, or re-open it if not. When creating the ticket, use the title F<NN> stable push requests (changing <NN> to the release number), and include this text:
This ticket will be used for requesting pushes of tested blocker / freeze exception fixes during F<NN> release freezes. See https://fedoraproject.org/wiki/QA:SOP_stable_push_request , https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process and https://fedoraproject.org/wiki/QA:SOP_freeze_exception_bug_process . Similar to candidate compose request tickets, this ticket can be closed by releng after each push and re-opened by QA each time a new push is needed.
Again, replace <NN> as appropriate. The actual request should be a new comment. Copy and paste the template text from the blocker bugs web application. Include all the text after ## FREEZE PUSH ## until the end of the template (do not include any text from before ## FREEZE PUSH ##). Then edit the request as necessary (see below), add the comment, and re-open the ticket if necessary.
Do not add updates to the template manually
There is almost never a legitimate reason to add updates to the template. Instead, if you believe an update should be pushed stable but it is not in the template at present, you should consider how (legitimately) to make it appear there.
The template will include all current updates that are linked to an accepted freeze exception or blocker bug for the current milestone and that are queued for stable. So, you may want to get a proposed FE or blocker accepted (perhaps after proposing it yourself), by asking folks on the Quality, Devel and/or Release Engineering Matrix channels to vote for it, then marking it as accepted in Bugzilla if sufficient votes are received (see QA:SOP_blocker_bug_process).
If you know an update fixes an accepted blocker or FE bug but the update is not currently linked to it, you may want to edit the update and add the bug link (if you are not a proven packager, you may need help from one to do this). Of course, you must not link an unrelated update to an accepted blocker or FE bug simply to get it through the freeze.
If an accepted blocker or FE bug has been incorrectly or prematurely closed (perhaps by an update for another release, or by an update that only partially fixes the bug), you should re-open it.
Finally, you might want to get an update that fixes an accepted blocker or FE bug submitted for stable, if it is not at present; you may be able to achieve this just by testing it and adding karma yourself, or you may need to ask others to test and karma it. If autopush is disabled, or you can reach the manual push threshold but not the autopush threshold, you may want or need to submit it for stable manually, or have the maintainer or a proven packager do this for you if you cannot.
The template is re-generated every half an hour (at :00 and :30), so you will need to wait for the next half hour mark and reload it to see the changes. In a pinch, it is possible to run a local instance of the app and refresh it at will, or (if you have access) force a refresh of the official instance; Release Engineering should be able to help with that.
When and how to remove updates from the template
You may want or need to remove updates from the request.
Most importantly, whenever a viable release candidate compose exists (see QA:SOP_compose_request), you must not request a stable push for any package that was not included in that compose. So if we build a candidate compose with fixes for all known blockers, then a fix for a freeze exception bug appears that was not in the candidate compose, you MUST NOT request a stable push for the new FE fix unless and until the candidate compose is definitely known to be unreleasable (e.g. a new blocker bug is accepted).
You may always request a stable push for any package that was included in the most recent candidate compose.
There is always some subjective discretion in deciding to push freeze exception fixes. If a freeze exception fix appears dangerous for any reason, you may omit it from the request (you may want to ask other folks for opinions on this).
When removing an update from the request, it is VITAL that you remove it from the #### Releng commands #### section as well as the #### Blockers #### and/or #### Freeze exceptions #### lists, or else it will almost certainly be included in the push (unless release engineering happen to notice the discrepancy and manually edit the command).
Matching request for approved candidates
When a Beta or Final candidate compose is approved for release at the Go/No-Go meeting, QA must submit a final stable push request that includes all packages which were included in the candidate compose but which are not yet stable. This must be done soon after the meeting ends.
The push must cause the stable package set to exactly match the approved release candidate. Every package listed in the compose request for the candidate must be included (if it has not already been pushed stable), and no other package may be listed. If an update that was included in the approved compose does not yet have sufficient karma to be submitted for stable, you must gather the karma; you can ask Quality and Release Engineering folks to help. In this specific case, actual testing of the update is not needed, because it must be pushed stable - you and others can simply blindly give it +1 karma until it has enough to be pushed.
Occasionally, an update that was included in the approved candidate will be edited to include a later build, or obsoleted by another update that includes a later build. In this case, you must not request that the later build be pushed stable. Instead, explain the situation (with links to the relevant updates), and request that release engineering manually tag the correct builds to stable. You can see an example of this happening [here](https://forge.fedoraproject.org/releng/tickets/issues/13243#issuecomment-574279).
