From Fedora Project Wiki
(Deferred by FESCo: https://pagure.io/fesco/issue/1773#comment-465932)
 
(14 intermediate revisions by 4 users not shown)
Line 30: Line 30:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1474769 #1474769]


== Detailed Description ==
== Detailed Description ==
Line 40: Line 40:
Multiple Flatpaks on the system can share a ''runtime'' of common libraries. There will be a single Fedora runtime maintained on the same schedule as the Platform module for Fedora. It will be be defined as a module that includes libraries that are commonly used for graphical applications. Some of these will inherit from the Platform module.
Multiple Flatpaks on the system can share a ''runtime'' of common libraries. There will be a single Fedora runtime maintained on the same schedule as the Platform module for Fedora. It will be be defined as a module that includes libraries that are commonly used for graphical applications. Some of these will inherit from the Platform module.


Applications then bundle the code for the application itself and for additional libraries they depend on beyond the base runtime. Each application has it's own module in which the relevant RPMs are rebuilt with a special RPM macros (in the `flatpak-rpm-macros` package) to relocate the application and bundled libraries into the /app prefix. (This is necessary, because inside an executing Flatpak, the application is mounted at /app, and the runtime at /usr)
Applications then bundle the code for the application itself and for additional libraries they depend on beyond the base runtime. Each application has its own module in which the relevant RPMs are rebuilt with a special RPM macros (in the <code>flatpak-rpm-macros</code> package) to relocate the application and bundled libraries into the /app prefix. (This is necessary, because inside an executing Flatpak, the application is mounted at /app, and the runtime at /usr)


The packages are then composed into Flatpaks by the layered image service, sharing as much of the workflow and code as possible with containers. While the native delivery mechanism for Flatpaks is as an ostree repository, they can also be distributed as OCI images, and our goal is to use this format during the build process, and to distribute them to users via registry.fedoraproject.com.
The packages are then composed into Flatpaks by the layered image service, sharing as much of the workflow and code as possible with containers. While the native delivery mechanism for Flatpaks is as an ostree repository, they can also be distributed as OCI images, and our goal is to use this format during the build process, and to distribute them to users via registry.fedoraproject.org.


Automation of rebuilds and integration with Bodhi will also be needed to make sure that security and bug-fix updates to packages end up being distributed to users. This part is least specified at he current time, and full automation may not be achievable for Fedora 27. If the above plan is followed, most of this work is shared with work needed for modularity in general and for server-side containers.
Automation of rebuilds and integration with Bodhi will also be needed to make sure that security and bug-fix updates to packages end up being distributed to users. This part is least specified at the current time, and full automation may not be achievable for Fedora 27. If the above plan is followed, most of this work is shared with work needed for modularity in general and for server-side containers.


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 50: Line 50:
Having graphical applications available as Flatpaks has a number of benefits:
Having graphical applications available as Flatpaks has a number of benefits:
* It allows graphical applications to be maintained in a modular fashion, with dependency chosen and tested by the application maintainer. It is the delivery vehicle for Fedora modularity for graphical applications.
* It allows graphical applications to be maintained in a modular fashion, with dependency chosen and tested by the application maintainer. It is the delivery vehicle for Fedora modularity for graphical applications.
* It allows new versions of applications to be tested and used without having to update packages on the system
* It allows new versions of applications to be tested and used without having to switch to new versions of libraries that are part of the system image
* It allows users to upgrade applications without rebooting
* It allows users to upgrade applications without rebooting
* It helps on ostree-based systems like "Atomic Workstation" where installing RPMs is tricky. (They can be layered, but the user experience will not be smooth.)
* It helps on ostree-based systems like "Atomic Workstation" where installing RPMs is tricky. (They can be layered, but the user experience will not be smooth.)
* It allows applications to be sandboxed, protecting the users data if security holes are present in an application. For more complex applications, this requires application code changes to do access through "portals", though games and other simple applications may work just fine without code changes. It is expected that changes for portal support will happen upstream - it's not something that Fedora packagers would be expected to do as part of packaging.


== Scope ==
== Scope ==
Line 70: Line 71:
** Allow Fedora packagers to submit module builds
** Allow Fedora packagers to submit module builds
** Allow uploading OCI images to registry.fedoraproject.org [https://github.com/docker/distribution/pull/2076 Upstream pull request]
** Allow uploading OCI images to registry.fedoraproject.org [https://github.com/docker/distribution/pull/2076 Upstream pull request]
** Make sure that Bodhi updates can be submitted for Flatpaks/OCI Images in the same way as for Docker containers (no significant code changes are expected beyond the current work to enable multiple types of Bodhi updates.)


* '''TBD: updates / bodhi'''
* Other developers: (f28)
** What happens when an update is filed for a package that is bundled in an application or runtime? Is a "testing" rebuild of the Flatpak done at that point? (Is one built on commits to dist-git?)
** Create a unified workflow for package and container/flatpak updates (detailed plan to be developed, something like:)
** Is it possible file an update for an '''application''' rather than for a package?
*** updates submitted to bodhi for a package should trigger automatic module and container/flatpak builds
** How do users download testing versions of flatpaks?
*** Pushing a package to stable should push the updated flatpaks/containers
** What happens when multiple updates overlap (package A and B are bundled in the application and have updates updates-testing, package A is pushed to stable)? Is a new version of the package built that contains only updates that have been pushed to stable?
*** the Bodhi user interface should show modules/containers that include a package and their status


<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
* Release engineering: [https://pagure.io/releng/issue/6876 6876] (a check of an impact with Release Engineering is needed)
 
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: Most likely, runtime and applications would not be part of the deliverables list. However, we will need to consider the quality of the runtime and applications as part of the overall quality of release - if some common application did not run on upgrade that would seriously affect users. This should, as much as possible, be addressed through continuous automated testing.
* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engeneering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  include a link to the releng issue.
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: <!-- REQUIRED FOR SYSTEM WIDE CHANGES --> Most likely, runtime and applications would not be part of the deliverables list. However, we will need to consider the quality of the runtime and applications as part of the overall quality of release - if some common application did not run on upgrade that would seriously affect users. This should, as much as possible, be addressed through continuous automated testing.


* Policies and guidelines: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->The people working on this change will need to work closely in the development of module guidelines, and make sure that Flatpak specifics are documented as well (for example, documenting the creation of a 'flatpak.json' with permissions and other metadata for the Flatpak). It's possible some changes will be needed to the packaging guidelines to make sure that all relevant packages relocate to /app properly, but it's expected that most packages that follow the current packaging guidelines will work correctly.
* Policies and guidelines: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->The people working on this change will need to work closely in the development of module guidelines, and make sure that Flatpak specifics are documented as well (for example, documenting the creation of a 'flatpak.json' with permissions and other metadata for the Flatpak). It's possible some changes will be needed to the packaging guidelines to make sure that all relevant packages relocate to /app properly, but it's expected that most packages that follow the current packaging guidelines will work correctly.
Line 108: Line 106:
== Dependencies ==
== Dependencies ==


This change is strongly intertwined with both modularity and containerization efforts. We plan to provide the bulk of the development resource for any Flatpak-specific changes needed to infrastructure and to the extent we can, and it is necesary, provide help for other features we depend on.
This change is strongly intertwined with both modularity and containerization efforts. We plan to provide the bulk of the development resource for any Flatpak-specific changes needed to infrastructure and to the extent we can and it is necesary, provide help for other features we depend on.


== Contingency Plan ==
== Contingency Plan ==
Line 120: Line 118:


== Documentation ==
== Documentation ==
<!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
See [[Workstation/Flatpaks]] for detailed documentation and a link to development setup.
 
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->


== Release Notes ==
== Release Notes ==
Line 132: Line 128:


[[Category:ChangePageIncomplete]]
[[Category:ChangePageIncomplete]]
<!-- When your change proposal page is completed and ready for review and announcement -->
[[Category:SystemWideChange]]
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- The Wrangler announces the Change to the devel-announce list and changes the category to Category:ChangeAnnounced (no action required) -->
<!-- After review, the Wrangler will move your page to Category:ChangeReadyForFesco... if it still needs more work it will move back to Category:ChangePageIncomplete-->
 
<!-- Select proper category, default is Self Contained Change -->
[[Category:SelfContainedChange]]
<!-- [[Category:SystemWideChange]] -->

Latest revision as of 11:12, 19 September 2017

Graphical Applications as Flatpaks

Summary

This change is to enable package maintainers to build Flatpaks of their applications and make those Flatpaks available for installation.

Owner

  • Name: Owen Taylor
  • Email: otaylor@redhat.com
  • Release notes owner:
  • Responsible WG: Workstation

Current status

  • Targeted release: F27 (initial), F28 (complete)
  • Last updated: 2017-09-19
  • Tracker bug: #1474769

Detailed Description

See Workstation/Flatpaks for the full development plan.

The goal of this change is make Flatpaks available as a distribution mechanism for software packaged in Fedora.

Multiple Flatpaks on the system can share a runtime of common libraries. There will be a single Fedora runtime maintained on the same schedule as the Platform module for Fedora. It will be be defined as a module that includes libraries that are commonly used for graphical applications. Some of these will inherit from the Platform module.

Applications then bundle the code for the application itself and for additional libraries they depend on beyond the base runtime. Each application has its own module in which the relevant RPMs are rebuilt with a special RPM macros (in the flatpak-rpm-macros package) to relocate the application and bundled libraries into the /app prefix. (This is necessary, because inside an executing Flatpak, the application is mounted at /app, and the runtime at /usr)

The packages are then composed into Flatpaks by the layered image service, sharing as much of the workflow and code as possible with containers. While the native delivery mechanism for Flatpaks is as an ostree repository, they can also be distributed as OCI images, and our goal is to use this format during the build process, and to distribute them to users via registry.fedoraproject.org.

Automation of rebuilds and integration with Bodhi will also be needed to make sure that security and bug-fix updates to packages end up being distributed to users. This part is least specified at the current time, and full automation may not be achievable for Fedora 27. If the above plan is followed, most of this work is shared with work needed for modularity in general and for server-side containers.

Benefit to Fedora

Having graphical applications available as Flatpaks has a number of benefits:

  • It allows graphical applications to be maintained in a modular fashion, with dependency chosen and tested by the application maintainer. It is the delivery vehicle for Fedora modularity for graphical applications.
  • It allows new versions of applications to be tested and used without having to switch to new versions of libraries that are part of the system image
  • It allows users to upgrade applications without rebooting
  • It helps on ostree-based systems like "Atomic Workstation" where installing RPMs is tricky. (They can be layered, but the user experience will not be smooth.)
  • It allows applications to be sandboxed, protecting the users data if security holes are present in an application. For more complex applications, this requires application code changes to do access through "portals", though games and other simple applications may work just fine without code changes. It is expected that changes for portal support will happen upstream - it's not something that Fedora packagers would be expected to do as part of packaging.

Scope

  • Proposal owners: (f27)
    • Create and maintain a flatpak-runtime module
    • Provide tools for application authors to use to create module descriptions for their flatpaks
    • Provide patches for the container build pipeline (atomic reactor and friends) to build flatpaks as well as containers
    • Create some way to get summary information for all flatpaks on registry.fedoraproject.org; this might be a patch to the codebase, a look-aside file, or a separate service.
    • Modify flatpak and gnome-software so that flatpaks can be installed from registry.fedoraproject.org.
  • Proposal owners: (f28)
    • Provide a "SDK" profile of the flatpak-runtime module to create a Flatpak SDK for third-party non-RPM builds against the SDK using flatpak-builder
  • Other developers: (f27)
    • Provide exports of built modules as repositories (the "On Demand Compose Service")
    • Provide some reasonably self-service way for packagers to create modules without a lot of overhead. (Does it make sense to allow application modules where when a package corresponds one-to-one to a module to allow the modulemd to live in dist-git next to the spec file?)
    • Allow Fedora packagers to submit module builds
    • Allow uploading OCI images to registry.fedoraproject.org Upstream pull request
    • Make sure that Bodhi updates can be submitted for Flatpaks/OCI Images in the same way as for Docker containers (no significant code changes are expected beyond the current work to enable multiple types of Bodhi updates.)
  • Other developers: (f28)
    • Create a unified workflow for package and container/flatpak updates (detailed plan to be developed, something like:)
      • updates submitted to bodhi for a package should trigger automatic module and container/flatpak builds
      • Pushing a package to stable should push the updated flatpaks/containers
      • the Bodhi user interface should show modules/containers that include a package and their status
  • Release engineering: 6876 (a check of an impact with Release Engineering is needed)
    • List of deliverables: Most likely, runtime and applications would not be part of the deliverables list. However, we will need to consider the quality of the runtime and applications as part of the overall quality of release - if some common application did not run on upgrade that would seriously affect users. This should, as much as possible, be addressed through continuous automated testing.
  • Policies and guidelines: The people working on this change will need to work closely in the development of module guidelines, and make sure that Flatpak specifics are documented as well (for example, documenting the creation of a 'flatpak.json' with permissions and other metadata for the Flatpak). It's possible some changes will be needed to the packaging guidelines to make sure that all relevant packages relocate to /app properly, but it's expected that most packages that follow the current packaging guidelines will work correctly.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

The upgrade question here mostly is what happens when a user already has an application installed from RPM if the package maintainer decides that they want to use flatpaks as the main or only delivery vehicle for the application. GNOME Software probably would need specific code to detect this case and install the Flatpak as an "upgrade" overriding the old RPM installed on the system (or potentially removing it.) The recommendation for F27 will be to continue packaging a traditional RPM in addition to any Flatpak, so any work here isn't needed until F28.

How To Test

Basic tests of the feature would consist of testing install and upgrade operations in GNOME Software, and verifying that the application is installed as a Flatpak as expected and operates correctly.

We also need to develop means for testing applications:

  • Basic tests on the flatpak should be executed while building it: did it build against any -devel packages that aren't part of the runtime? does it include a desktop file and an icon? and so forth.
  • Some level of automated "smoke-testing" of the built applications should be developed - do they start and display a window?
  • There should be ways for users to install either only Flatpak applications marked stable, or install testing versions, and there needs to be ways of leaving feedback on the testing versions.
  • It may be useful to define a set of "core" applications that are tested by Fedora QA to install and operate - in general, existing test plans and automation for applications can be carried over - it doesn't matter how they are packaged.

User Experience

If an application a user want to install is available as a flatpak on registry.fedoraproject.org, it will be transparently be installed rather than installing the RPM. (Assuming that we have some basic security update plan in place - if not, then we should prefer RPM installs) Applications that support running sandboxed will run sandboxed, providing additional security to the user. Updates to Flatpak applications and runtimes will be offered and take place without a system reboot.

Dependencies

This change is strongly intertwined with both modularity and containerization efforts. We plan to provide the bulk of the development resource for any Flatpak-specific changes needed to infrastructure and to the extent we can and it is necesary, provide help for other features we depend on.

Contingency Plan

  • Contingency mechanism: If the basic plan is working, but support for building Flatpaks or support for distributing them via registry.fedoraproject.org is not available in time for F27, we simply will not announce the feature and not enable registry.fedoraproject.org in GNOME Software.
    • If modifying atomic-reactor to build Flatpaks turns out to be infeasible, we could provide a separate Koji plugin and/or content generator.
    • If distributing Flatpaks via registry.fedoraproject.org turns out to be infeasible, we could distribute via ostree repository; this would likely put the F27 timeline in jeopardy because more new infrastructure components would be needed.
  • Contingency deadline: If a fully working pipeline enabling packagers to create a Flatpak, build it, and have it appear on registry.fedoraproject.org isn't available by the Beta Freeze (2017-09-05), then this likely should not be advertised for F27.
  • Blocks release? No
  • Blocks product? No

Documentation

See Workstation/Flatpaks for detailed documentation and a link to development setup.

Release Notes