Community Platform Engineering/FY14 Plan

FY2014 Plan

Red Hat tracks a lot of things by "fiscal year". Every public company has a different definition of when the "fiscal year" begins and ends, but for Red Hat, it begins on March 1, and ends on the last day of February. The "fiscal year" numeric naming is attached to the latest calendar year in the "fiscal year". Red Hat Fiscal Year 2014 (henceforth abbreviated as "FY14") begins on March 1, 2013, and ends on February 28, 2014.

This document is Fedora Engineering's "working plan" for FY14.

New Hires

At this time, no new hires are planned for FY14.

Projects

Fedora Engineering is constantly working on any number of initiatives, but for FY14, our focus will be primarily on four initiatives. These projects are both "big" in scope and in importance (and arguably, in time to complete). We reserve the right to add additional projects, big and small, and to postpone or abandon projects. We also are interested in feedback from the Fedora Community on these projects, both in concept, and as they progress towards completion.

These projects are summarized here for simplicity, in each case, further documentation about the projects will be forthcoming (and linked from here). Interested community members (at any level) should contact the corresponding Project Lead.

Big Projects

[IMPROVE COMMUNICATION]

HyperKitty: Mailing List Improvement Application

Project Lead: Aurélien Bompard

Started in FY13, HyperKitty will provide a consistent, real-time, forum style interface for Fedora's mailing lists. Fedora Community members who prefer a mailing list model for communication can continue to receive mail, while Fedora Community members who prefer a forum-style interface to communicate can do so, and both groups are seamlessly communicating with each other.

The point of this effort is not to argue the pros and cons of forums vs mailing lists, but rather, to accept that both have merit and embrace them for what they are and what they can bring to Fedora.

Notable UI concepts:

• Support threading
• Robust search
• User karma (possibly thread karma?), where user and/or post "scores" would be sent in mail headers for possible client side filtering (and UI filtering, e.g. "Don't show me topics started by users with < $FOO karma" or "Hide comments on topics from users with < $FOO karma"). This would be a case where the forum interface would be richer than the mailing list side.

For more details, see: https://fedorahosted.org/hyperkitty/

[IMPROVED STATISTICS]

100% Fedmsg Enablement

Project Lead: Ralph Bean

Ensure that every Fedora infrastructure service is generating messages via fedmsg. Including bugzilla. All of these messages will be logged in Datanommer, and statistics can be generated from this msg history.

Badges

Project Lead: Tom Callaway

Create an ecosystem where users and contributors can earn digital badges for actions performed in the Fedora infrastructure space and locally on their Fedora OS installation. Additionally, have "quests" where when multiple specific badges are earned, Fedora Account holders will earn tangible real world prizes (hats, shirts, exotic sports cars, etc, etc).

This serves to help us motivate community involvement in Fedora and to generate useful statistical data about local user and contributor usage.

[IMPROVED UPDATES]

Monthly Bundled Updates

Project Lead: TBD

Generate an additional stream of updates, which is composed of all updates in "updates stable" from the critical path with one week remaining in a month. Then, QE will test those updates as a set, to ensure that basic testing reveals a reliable OS at that point in time. Then, those updates will be pushed into a repository, and local Fedora client tooling will see it and treat it as a single update to be installed (e.g. "The Fedora 18 June 2013 Update).

Security updates will still go out asynchronously. All other updates not in this monthly bundle (and not security) will be considered "optional", and the user will not be "nagged" to update them, unless they add packages to a configurable whitelist.

Users who wish to remain on the current "firehose" update model may do so.

The packager workflow for generating updates should not change at all as a result.

Application GUI Installation

Project Lead: Ryan Lerch

In conjunction with the GNOME Desktop team, overhaul the current PackageKit GUI interface and provide a clean and extensible Application (not package) installer with a powerful search and the ability for Fedora Account holders to give feedback (karma and comments).

yum will remain available (and used as the backend, unless technically impossible) for CLI usage.

This will almost certainly require backend infrastructure in addition to local Fedora client changes.

Bodhi 2.0

Project Lead: Luke Macken

There are a lot of issues with the existing Bodhi deployment and its functionality, and a general consensus of how to move forward with the next major revision of the Bodhi Update Management software.

Proposed enhancements include:

• Ensure that a package and its dependencies are always pushed together as a single update unit, even if the update unit is maintained by multiple parties, pushed initially as independent updates.
• Ensure that packages are not pushed which would break dependencies within Fedora (or EPEL).
• Improve the "karma" system
• Cleaner Web UI

More details can be found here: https://fedoraproject.org/wiki/Bodhi/2.0

A rather old mockup can be found here: http://lewk.org/img/bodhi-20-frontpage.png

copr

Project Leads: Seth Vidal and Bohuslav Kabrda Copr buildsystem: http://fedorahosted.org/copr/

Buildsystem demoed at fudcon and in testing now. Uses cloud systems to create new buildsystems and keep builds separated and pristine from any potential intrusion by rogue packages or processes.

[IMPROVE AUHTENTICATION AND AUTHORIZATION]

Migrate to OpenID

Project Leads: Toshio Kuratomi and Patrick Uiterwijk

Over the past few months Patrick Uiterwijk has been driving some changes to our authentication services within Fedora Infrastructure which has lead to a better OpenID service.

For the coming year, we'd like to continue this and enhance it by migrating existing applications to use openid instead of the custom FAS auth we have now and adding mozilla persona support for hyperkitty (and third party sites).

• Migration of authentication for web apps from jsonfas to openid:
  • Benefit of this is that a few more of our services will be able to use vanilla upstream plugins for authn. Things that need custom work will still go through a standardized interface so we may be able to leverage pre-existing code a bit more.
  • Since we have to use openid extensions to get the full functionality, some of these may be straightforward "switch mediawiki to an openid auth provider" but others will be "write a custom openid auth provider for TG1 that can get

groups from our openid server".

• Persona is a good fit for hyperkitty since it is based on email addresses (whereas OpenID is based on URLs). Patrick is looking into adding Persona to our existing OpenID services.

Create an authorization infrastructure with OAuth

OAuth enables a different sort of collaboration between our apps and users than we currently have. Right now we allow a user to be able to perform a task. Oauth allows our users to grant permissions for people or things (scripts, other websites) to do specific things on their behalf.

• Examples:
  • If we oauth enable fedpkg and bodhi, a user could retrieve an access token for bodhi that only has the ability to request that updates they've submitted can go from updates-testing to updates. The user could then set up a cron job to do this for their updates that have been in updates-testing for 7 days with no negative karma using the access token instead of their Fedora username and password.
  • Another example with bodhi: We could write an oauth permission that lets people add packages to an existing update. When the python3 maintainer prepares a new update in F-19-branched, he can generate a token that contains just this permission. He could give it out to other people in the python-sig that are helping him do python3 package rebuilds so that they can add new packages to his update.
  • An example with pkgdb: Right now you must be in cvsadmin in order to touch the acls of a package you don't have approveacls on. We could write OAuth permissions for a certain subset of that so a cvsadmin could create an access token that fesco members, for instance, could use to release ownership for AWOL package maintainers or unretire packages that were mistakenly retired in the web ui but wouldn't let those people do all the other things that cvsadmin allows.

• The plan for FY14:
  • OAuth is a more invasive change than OpenID. Instead of just switching out some code that decides who you are and then returns that information to the app via an interface, OAuth needs support from the applications that enable it to limit access to functionality based on permissions instead of identity. So we're going to be somewhat conservative in what we accomplish
  • Building blocks: create and deploy an oauth server
  • Proof of concepts: add a permissions system to copr and other non-core services. If the concept worked out, we would then be in position to add it to other services.

[FEDORA KERNEL PROJECTS]

Project Lead: Dave Jones

[IMPROVED KERNEL QA]

Setup a dedicated Fedora Kernel QA environment, where manual and automated tests can be run against Fedora Kernel builds to minimize (or at least identify) regressions and bugs earlier. This includes:

• Continued expansion of the existing kernel-tests repository http://git.fedorahosted.org/git/kernel-tests.git
• Continued improvement of the upstream trinity test project
• Creation of a fedmsg enabled test harness to start regression tests after build completion
• Engage in discussions of kernel test cases and test harnesses with the upstream kernel community

We also hope to work with the broader Fedora teams to integrate into any AutoQA efforts that are on-going.

[INCREASED BUGZILLA TRIAGE]

The kernel component in bugzilla continues to have one of the highest bug counts in the entire distribution. This is simply due to the nature of the Linux kernel and the wide variety of hardware it supports. However, the changes to keeping most releases on the same stable kernel version allow for more opportunity for bug triage and consolidation. In conjunction with the efforts from improved kernel qa, we would like to investigate ways to make this more manageable. This might include:

• Creation of or investigation into bug triage tools (e.g. python-bugzilla)
• Improve community communication and calls for testing
• Direct significant broader testing focus on kernels while still in upstream development

Other Ideas

These ideas don't have owners, nor do they have any guarantee of actually happening this year, but if you're looking for something to do, these would be appreciated.

• A Meeting App - next meetings, hot topics, who's saying what, etc. using meeting bot logs
• Teach meetbot how to handle voting
• Spin Master - analysis of what packages added/removed in each spin for each revision
• Automate nightly spin compose tasks, currently a manual process.
• Openshift - Possibly move some simple apps into openshift, get FAS running in Openshift as it's needed for all of our other apps to run
  • The idea is to have development environments for our apps that people can pull up quickly and easily with minimal input from the fedora admins. Then, for the simple apps, we can additionally have those apps run in openshift in production (which cuts down on work that we need to do to maintain those servers and deployment as an rpm, etc)
• Improve Fedora Hosted - Something more like gitorious, gitlabhq, allura