From Fedora Project Wiki
m (moved User:Ffesti/MetaPackages to User:Ffesti/NextLayer: We no longer talking about packages)
(Go from meta packages to installed groups as this fits better to the wanted behavior)
Line 1: Line 1:
= The next Layer =
I've been thinking for a while how the system should look like from a users perspective. For me it is clear that the system should not look more complicated that the decision the user made to set it up. If the user only selected half a dozen of groups and installed 20 additional packages the description of the system should not have more than those 26 items.
I've been thinking for a while how the system should look like from a users perspective. For me it is clear that the system should not look more complicated that the decision the user made to set it up. If the user only selected half a dozen of groups and installed 20 additional packages the description of the system should not have more than those 26 items.


The second thing I am thinking about is how to replace the comps groups with something better. So here is just a random thought: What about converting them to meta packages (ok, not that new). But what about only listing the meta packages (e.g. in yum list installed). Hide all other packages. Pull them in on demand and remove them if they are no longer needed (as in --remove-leaves).
The second thing I am thinking about is how to replace the comps groups with something better. Actually it is not so much comps groups themselves that are problematic but the way they are handled. Right now they are just used to get a list of packages that can be installed and then they get forgotten (Yes, yum now saves what groups where in the transaction somewhere but has no real use of this information). The next thing is that groups only apply to a part of the packages. There are a lot of "optional" packages that are located in a group be does not really belong to it (in yum's understanding). And there are packages that do not belong to any group at all. This makes groups a second class citizen in the yum world and abbreviation of a longer package list.
 
So the first step would be making groups a first class citizen and even making packages second class. What about only listing groups (e.g. in yum list installed) and hide all other packages by default? Packages are installed on demand and removed if they are no longer needed (as in --remove-leaves).
 
Ok, this raises to the question: Hey, what about all the packages not in a group? How could you install those? What about optional packages?
 
Well, while group may stay pretty much the same in comps they will look very different in yum. Installed groups will be stored within yum. One way to install a group is to select one of the comps groups. But you can also just create a new group on your own. This would actually be the only way of installing a package not in a group yet. Don't panic! All you have to do is giving a group name (and optionally a description).
 
An installed group can either be stand alone or be bound to an comps group. Then it will follow the group in the comps file and changes of the group will change the packages installed on the system. The optional packages selected would also be stored in the installed group. The not selected packages would be stored as "comments" that can be turned into installed packages by editing the group. Beside the comps groups description there should also be a local description that can hold the local reasons for installing this group and the selected optional packages.
 
An user created group would look very similar to an installed comps group except that it would not follow any external group. The idea is that users create groups for a given purpose. That way the groups can serve as documentation on what is installed and why. Typical groups could be a tool chain or devel packages for a project. Groups can and should overlap if necessary. So packages can be required from several groups.
 
So the operation of the package manager look very different from yum's:
 
* list [available]
* info GROUP
** give group details
* install GROUP
** GROUP can either be a group name or a file
** if no cli param is given open an editor to select which optional packages to select and to add a comment
** cli switch allows installing the group immediately (selecting mandatory, default or optional packages)
* edit GROUP
** opens editor allowing adding and removing packages (optional and arbitrary) or modifying the comment(s)
** there are sub command to alter the group from the cmd line
* new GROUP [OTHERGROUP, ...]
** create new group with the given name
** opens editor if details are not given at the cmd line
** can use other groups as template
* remove GROUP


Ok, this raises to the question: Hey, what about all the packages not in a group/meta packages/whatever? How could you install those?
== Challenges ==


I have two answers for this: Some of the packages will be marked as applications - similar to the optional packages in current comps groups. May be this counts as meta package or is represented as meta packages or similar. Implementation details need still to be defined but the idea is that they look like the meta packages from the user.
The idea is that the installed packages do only depend on the groups installed (otherwise it'd be not sufficient to just look at the groups). This will require a slightly different implementation of updates. Packages no longer required need to be removed.


For the rest of the packages I'd suggest we just create a meta package on the fly. The idea is that we do not create it for a single package but for a set of packages the user want to install for a given purpose. The meta package can though hold a brief description on why the packages got installed. May be you need a set of devel packages for a project you are working on. So you install them and give a name and a description at the command line (or use a fancy GUI tool). So yum list installed would not list the single packages but "projectfoo-devel-packages". You could probably add more packages later on or remove some. In this case the meta package would be altered or be replaced with an updated one.
In addition an update mechanism for groups is needed. It is still open if comps groups should have a version number or if we can just have a hash or stamp that we compare with the version of the group we have installed. As it is possible that the group list do no longer match the packages in repositories there is a new class of errors. As we want automatic updated to continue working the implementation should handle them gracefully but keep track of them. All interactive tools should list such inconsistencies and ask the user to solve them.


What is a meta package:
If yum or rpm is still around it is possible that the installed packages do no longer match the groups. It is though important to detect that the package list was changed "behind the back". These chances need the to be integrated into the group world. Packages that are required by groups should be marked as deleted/deselected. New packages should be added to a "Unknown" group. Similar to other mismatch errors the user should be asked to clarify the situation.


May be a meta package is more or different to what meta packages have been in the past (just normal packages used to pull in other packages by requirements). I guess they have to
Problematic situations:


* Look and taste different than normal packages. Tools need to tell them from normal packages. As user should never have a chance to think it could be a normal package (Probably the biggest issue with previous meta package usages)
* Package listed in group does not exist in repository
* be persistent. If you install them they are installed. There is no match package lists to installed packages or something. They are an "real" entity. This does not have to mean that they are real rpm packages - although this is a possible implementation.
** may or may not be installed on the system
* be the place that defines what packages are on the system and what are not. May be they need a more complicated structure. May be they need to store what of their optional requires (Suggests?) are selected by the user and what are just left out.
*** may be ignore if package was installed from file
* be secure and easy to share. Guess they do not have files or scripts. I should be able to install a meta package/ group I just found in the internet without worrying. There might be more things needed to really make this secure. May be even disallow versioned requirements to avoid old and vunerable packages being installed.
* Package listed in group got deleted by hand
* be easy to create, copy around and share. There need to be many of them. One oer two order of magnitude more than we have now -  not counting those for the single package applications. May be they are more like recipes; come with a description how to set things up; come in several variants.
** Remove from group/ Add deleted entry to avoid installing it again
* New Package installed by hand
** Add to dummy group
* Package is obsoleted
** Maintain an obsoleted list to translate entries in groups that did not yet do the transition

Revision as of 09:59, 10 May 2011

The next Layer

I've been thinking for a while how the system should look like from a users perspective. For me it is clear that the system should not look more complicated that the decision the user made to set it up. If the user only selected half a dozen of groups and installed 20 additional packages the description of the system should not have more than those 26 items.

The second thing I am thinking about is how to replace the comps groups with something better. Actually it is not so much comps groups themselves that are problematic but the way they are handled. Right now they are just used to get a list of packages that can be installed and then they get forgotten (Yes, yum now saves what groups where in the transaction somewhere but has no real use of this information). The next thing is that groups only apply to a part of the packages. There are a lot of "optional" packages that are located in a group be does not really belong to it (in yum's understanding). And there are packages that do not belong to any group at all. This makes groups a second class citizen in the yum world and abbreviation of a longer package list.

So the first step would be making groups a first class citizen and even making packages second class. What about only listing groups (e.g. in yum list installed) and hide all other packages by default? Packages are installed on demand and removed if they are no longer needed (as in --remove-leaves).

Ok, this raises to the question: Hey, what about all the packages not in a group? How could you install those? What about optional packages?

Well, while group may stay pretty much the same in comps they will look very different in yum. Installed groups will be stored within yum. One way to install a group is to select one of the comps groups. But you can also just create a new group on your own. This would actually be the only way of installing a package not in a group yet. Don't panic! All you have to do is giving a group name (and optionally a description).

An installed group can either be stand alone or be bound to an comps group. Then it will follow the group in the comps file and changes of the group will change the packages installed on the system. The optional packages selected would also be stored in the installed group. The not selected packages would be stored as "comments" that can be turned into installed packages by editing the group. Beside the comps groups description there should also be a local description that can hold the local reasons for installing this group and the selected optional packages.

An user created group would look very similar to an installed comps group except that it would not follow any external group. The idea is that users create groups for a given purpose. That way the groups can serve as documentation on what is installed and why. Typical groups could be a tool chain or devel packages for a project. Groups can and should overlap if necessary. So packages can be required from several groups.

So the operation of the package manager look very different from yum's:

  • list [available]
  • info GROUP
    • give group details
  • install GROUP
    • GROUP can either be a group name or a file
    • if no cli param is given open an editor to select which optional packages to select and to add a comment
    • cli switch allows installing the group immediately (selecting mandatory, default or optional packages)
  • edit GROUP
    • opens editor allowing adding and removing packages (optional and arbitrary) or modifying the comment(s)
    • there are sub command to alter the group from the cmd line
  • new GROUP [OTHERGROUP, ...]
    • create new group with the given name
    • opens editor if details are not given at the cmd line
    • can use other groups as template
  • remove GROUP

Challenges

The idea is that the installed packages do only depend on the groups installed (otherwise it'd be not sufficient to just look at the groups). This will require a slightly different implementation of updates. Packages no longer required need to be removed.

In addition an update mechanism for groups is needed. It is still open if comps groups should have a version number or if we can just have a hash or stamp that we compare with the version of the group we have installed. As it is possible that the group list do no longer match the packages in repositories there is a new class of errors. As we want automatic updated to continue working the implementation should handle them gracefully but keep track of them. All interactive tools should list such inconsistencies and ask the user to solve them.

If yum or rpm is still around it is possible that the installed packages do no longer match the groups. It is though important to detect that the package list was changed "behind the back". These chances need the to be integrated into the group world. Packages that are required by groups should be marked as deleted/deselected. New packages should be added to a "Unknown" group. Similar to other mismatch errors the user should be asked to clarify the situation.

Problematic situations:

  • Package listed in group does not exist in repository
    • may or may not be installed on the system
      • may be ignore if package was installed from file
  • Package listed in group got deleted by hand
    • Remove from group/ Add deleted entry to avoid installing it again
  • New Package installed by hand
    • Add to dummy group
  • Package is obsoleted
    • Maintain an obsoleted list to translate entries in groups that did not yet do the transition