FWN/Beats/Developments

Developments

In this section the people, personalities and debates on the @fedora-devel mailing list are summarized.

Contributing Writer: Oisin Feeley

Getting Back on our Feet

On 05-09-2008 Jesse Keating posted[1] the good news that "[...] we have done a successful compose of all the existing[,] and as of yesterday[,] pending updates for Fedora 8 and Fedora 9, all signed with our new keys." He cautioned that the size of this backlog of updates meant that it would take some time to get them out to mirrors. The updates will be in new directory locations and there will be an "[...] updated fedora-release package in the old updates location that will get you the new keys and the new repo locations."

[1] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00329.html

Jesse was keen to point out that there would be a FAQ to which we can all contribute and that its location will be announced shortly.

Sean Darcy, after thanking Jesse for all the work he had put in, asked[2] how he could get the new key for updates right now without having to wait for the fedora-release package to be placed in the old repository location. A variety of answers followed and the canonical one appeared[3] to be that given by Todd Zullinger in which he suggested obtaining fedora-release from CVS and then checking that the fingerprints matched those published[4] on the FedoraProject website. Some minor confusion followed[5] as the example presented on the web page uses the old key signed by "fedora@redhat.com" but the new key is signed by "fedora@fedoraproject.org". Similarly the use of "PUBKEY" as a placeholder variable on the web page caused some difficulties which Todd cleared[6] up again.

[2] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00392.html

[3] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00396.html

[4] https://fedoraproject.org/keys

[5] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00406.html

[6] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00417.html

Jeffrey Ollie made[7] the good point that using a GPG WoT would make it easier for Fedorans to have confidence that they had obtained the correct key and hoped that those at the Brno FUDCon could "arrange an impromptu keysigning[.]"

[7] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00411.html

Early in the week on 02-09-2008 Stefan Grosse asked[8] politely when the resigned Fedora 9 updates would be available. Bruno Wolff III suggested[9] "If you are worried about something in particular, you can look at bodhi to see pending updates and updates-testing and get rpms from koji if there is something you want right now." A brief discussion between Till Maas, Jesse Keating and Bruno Wolff explored whether it was possible to download from Koji using SSL if one did not have a FAS account. Bruno testified[10] "I needed certs (two from fedora and mine) to get the bodhi client to work. I used this to grab a list of stable updates last week [...] I didn't bother with ssl when grabbing them from koji." Till was[11] using wget to grab the packages from Koji and found it necessary to use certificates. Jesse showed[12] that it was possible to do a koji download-build to get packages anonymously.

[8] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00083.html

[9] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00086.html

[10] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00089.html

[11] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00092.html

[12] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00096.html

In a separate thread Jesse Keating announced[13] that the revised Fedora 10 freeze date would be 09-09-2008 and as part of a friendly reminder that it was coming up quickly observed "I realize that rawhide has been less than great lately, and we're working quite hard to fix the issues. The installer images from the 30th may be of use in getting rawhide installed and then updating to what is in the public repos. See http://kojipkgs.fedoraproject.org/mash/rawhide20080830/development/ (please only grab the installer images from there, not the entire tree)." Matt Miller reported[14] a successful net install from the Fedora 10 alpha images using the rawhide repository.

[13] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00091.html

[14] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00107.html

Removing Packages with Long-standing FTBFS Failures

Matt Domsch posted[1] a list of ninety packages which "Fail To Build From Source" (FTBFS)[2]. Matt noted that some of these packages have failed since 02-2008 and that "[...] packages with unresolved FTBFS bugs immediately following the Alpha release will be removed from the distribution" in line with a proposal passed by FESCo. He asked that concerned parties help to resolve the problems and noted that several of them had easy fixes.

[1] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00319.html

[2] http://fedoraproject.org/wiki/FTBFS

Response was fairly quick and positive from both those listed as maintainers and concerned parties that needed the packages in Fedora 10.

One interesting ramification of the removal of such packages was mentioned[3] by Daniel Berrange who asserted "[t]his list is far from complete - if you want to remove these 90, the dependency chain ripple, will entail the removal of tonnes of other packages which depend on these." He asked Matt to generate a report which "[...] shows the ripple effect for each proposed package. If something is just a leaf-node, it isn't very important to worry about, but if something triggers removal of 50 dependant packages that's pretty damn important to fix. This info would be useful in prioritizing which builds need fixing most urgently." SethVidal modified[4] a script written by James Antill so that it did exactly that and Matt posted[5] a link to the script and an example run. Till Maas suggested[6] that it would be useful make sure that a src.rpm responsible for several dependent packages is only counted once.

[3] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00320.html

[4] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00363.html

[5] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00361.html

[6] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00371.html

The practical consequence of neglecting such dependencies was highlighted[7] by Matthias Clasen when he commented that "djvulibre-devel is a BuildRequires for evince. If you remove djvulibre, evince will become unbuildable." Jesse Keating responded that Daniel, or his team should fix the package as "[i]f we have to do a chained rebuild for various reasons, evince would fail due to this package not building first."

[7] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00323.html

The inclusion of monodevelop in the list had the side effect of spurring the Mono maintainers including Michel Salim, David Nielsen and Paul Johnson to decide[8] to attempt a co-ordinated push of "Mono-2.0".

[8] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00385.html

MinGW on Fedora

The availability of a MinGW development repository was announced[1] by Richard Jones. "MinGW" provides a GNU toolchain on Windows allowing the development of Free native Windows binaries. Richard credited Daniel Berrange with a good deal of the work.

[1] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00296.html

In response to Farkas Levente's question, as to when it would be available in Fedora, Daniel replied[2] that due to the need for "infrastructure" to create a separate repository and dedicated build root it was impossible to predict. Jef Spaleta seemed[3] to be trying to move the process forward and published a draft which explained that "[t]he initial impetus for this effort has been ovirt developers who desire to more easily use Fedora installations as development environments for the work they are doing to build open source cross-platform virtualization tools." The possible impact on FedoraProject resources appears to have led to the compromise of creating a separate repository which could be strictly confined in size.

[2] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00298.html

[3] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00364.html

Richard Jones encouraged[4] Farkas to try out MinGW without waiting for official inclusion in Fedora "[...] if you want to download and play with it, and send patches :-) It's currently buildable on Rawhide, and possibly on earlier versions of Fedora too." Farkas then revealed that he currently used an older MinGW on RHEL5 and Richard responded[5] that although there were no srpms at this moment it should be possible to use the specfiles and patches (both in the repository mentioned earlier) to rebuild everything.

[4] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00300.html

[5] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00330.html

Dependency Loops Considered Harmful?

[[MiroslavLichvar|Miroslav Lichvar] raised[1] the issue of a considerable number of packages (354 by his count) which are components of dependency loops in the rawhide repository. Miroslav provided[2] a list. In such loops two or more RPM packages require each other as dependencies in their spec files with the result that it can become confusing for users trying to remove selected packages manually with yum or rpm. Miroslav wondered whether such loops were acceptable and specifically "why games data depend on binaries?"

[1] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00149.html

[2] http://fedorapeople.org/~mlichvar/tmp/rawhide.i386.loops

A substantial conversation resulted which exposed the different needs of packagers and users, some possible limitations due to the design of rpm and varied philosophies concerning the correct way to specify dependencies. One of the central examples chosen were the game packages which tend to be split into a "binary" or "engine" package and a companion "data" package for graphics, sound and levels. The central concerns expressed were that occasionally an install followed by a remove will leave "orphaned" packages behind. Also in contention was the specific case of developers experiencing the problem of maintaining a stable environment when there is no easy way of tracking changes to system libraries. The package manager aptitude was mentioned favorably due to its ability to distinguish between packages installed automatically to satisfy dependencies and those which are installed manually.

On the one hand Chris Snook and Michael Schwendt advocated[3] that instead of using Requires: foo >= X in their spec files packagers should choose Conflicts: foo < X . The possible downside to this would be installing game data yet having no game engine/binary installed. This was seen as a non-serious problem.

[3] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00211.html

On the other hand Jon Ciesla argued[4] that the advantage of having games data depend on their binaries from a packagers perspective is that it ensures that attempting to upgrade the binary Version without also upgrading the data will fail. The case of allowing minor fixes without forcing users to download a big bundle of data can be handled by bumping the Release of the package. More concisely he answered[5] that the reason for the dependencies was that "[...] so when you remove the game, you remove the data." Michael responded[6] that Jon's example could be handled by using a Requires: foo-data = X without exposing the increased risk of needing to "bump'n'rebuld" the data package each time the engine package incremented its Version while still working with the same data. He characterized such dependencies as "superfluous [...] try[ing] to enhance 'yum remove'."

[4] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00150.html

[5] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00151.html

[6] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00161.html

This was not accepted as accurate by Jon and he described the current situation as "require on name and version" for game data which allows Release to be changed without affecting the match on Name and Version "If new data is required, it will change the version. If not, we only increment the binary rpm's release, so the data rpm matches on version and needs no rebuild." Michael responded[7] by laying out two cases, the first of which illustrated the problem of having to update the data package solely to keep in step with updates to the version so that yum remove would function properly. His alternate case used non-versioned dependencies in the data package and strict dependencies in the engine package. Hans de Goede seemed a bit irritated and asked Michael to "Please take an actual look at game spec files before making up all kinds of BS, it's really easy" and argued that for games there was a one-to-one mapping between the engine and the data. He added that just because other cases resulted in dependencies being sucked in by an install and left behind on a remove there was no reason to change the way things were done for games.

[7] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00167.html

Stephen Gallagher suggested that the "groups" exposed through yum should be used to bundle together the multiple packages for an application instead of using looped dependencies to which Callum Lerwick replied[8] that it was actually time to "implement the per-package 'explicitly installed/pulled in as a dep' flag that has been discussed several times in the past, and is already implemented (thus proven) in the 'aptitude' apt front end." He followed this with an amusing piece of hyperbole about "the looming dark future of closed DRM-laden content delivery services[.]"

[8] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00193.html

Toshio Kuratomi raised[9] the problem faced by developers using system libraries that could come and go due to packages being updated or removed. Callum's suggestion was[10] that developers should be "RPM packag[ing] your app. Apps on your system that are not tracked by RPM are a ticking time bomb of dependency breakage whether or not the 'leaf culling' is performed manually or is assisted by a 'pulled in as dep' flag. A package or distribution update could very well break your app too." Needless to say Toshio had[11] several objections to this including the impact on developer workflow imposed by packaging up everything and the inadvisability of forcing developers to become expert administrators. He suggested that "If we implement this, it needs to be at a low enough level that anyone installing packages on the system will be storing the information. That could mean rpm (since rpm is responsible for taking the package and turning it into files on the filesystem) or that could mean yum since yum is the one that actually knows whether a package is being installed due to depsolving or user request. Doing this at a higher level means that information gets lost (for instance, if you do this in PackageKit, there won't be any information about what anaconda chose to install due to checkboxes being clicked and which things were installed due to dependencies). With that said, there needs to be a way for a developer to tell yum not to prune away leaf packages if they want." A very detailed and amusing discussion with (among others) Matthew Woehlke followed in which Matthew argued[12] for a script which essentially produced a parallel iuserj rpm database so that quick and dirty rpms could be produced locally for developers.

[9] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00195.html

[10] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00209.html

[11] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00214.html

[12] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00278.html

Jef Spaleta wondered[13] what actual examples revealed about the amount of harddrive space being consumed by dependencies and remarked "I hope the packagekit people are watching this discussion. The game data subpackaging issue should be right up their ally in terms of end-user ease of use issues." He discounted the polluting of the packagelist but MichaelSchwendt pointed out[14] that as the packagelist increases then "[because of] the frequent updates common in Fedora land, you need to download and update more[.]"

[13] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00162.html

[14] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00170.html

The idea of keeping game data outside of any package management was mentioned[15] by James Antill as an "obvious fix" and he discounted the probability of PackageKit being able to do anything about the issue if yum or rpm could not. Jef responded[16] that "[a]t some point someone is going to have to wade into rpm itself for ease of use of removals to get better" and pointed out that he was "[...] making sure that the people with the right motivation and the right skills find the right way to handle it. That's not going to happen just by telling the current maintainers who are abusing the available requires syntax that they are abusing the syntax and slapping their wrists." Richard Hughes showed[17] that the PackageKit developers were indeed listening to the conversation and mentioned that he had considered "[...] running through a large "get-depends" output into the basename filter, so that the user only sees the 'applications' by default, rather than a huge list[.]"

[15] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00184.html

[16] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00185.html

[17] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00212.html

There is a lot more to this conversation than reported here, but the main thrust is that the limitations of rpm have resulted in package maintainers wrangling spec files to do what they want which in some cases has undesirable effects.

Fedora Security Tools Spin

A suggestion was made[1] by Huzaifa Sidhpurwala to produce a Security Tools spin similar to the "Knoppix Security Tools Distribution"[2] .

[1] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00108.html

[2] http://knoppix-std.org/index.html

Todd Zullinger responded[3] that there was already a spin similar to this being curated by Luke Macken. The SecuritySpin[4] mentioned by Luke seemed to be roughly what Huzaifa was searching for, except that he thought it should have "more tools and [be] more bare bones".

[3] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00110.html

[4] https://fedoraproject.org/wiki/SecuritySpin

Adrian Pilchowiec mentioned[5] a free fork of nessus named OpenVAS[6] as a desirable part of the spin. Luke drew[7] attention to the need for more people to help out with packaging missing tools. Subsequently the wishlist of the project recorded that Huzaifa was packaging up OpenVAS and labrea.

[5] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00275.html

[6] http://www.openvas.org/

[7] https://www.redhat.com/archives/fedora-devel-list/2008-September/msg00183.html