From Fedora Project Wiki
mNo edit summary |
(→IRC Log of the Class: adding the final log) |
||
Line 3: | Line 3: | ||
==== IRC Log of the Class ==== | ==== IRC Log of the Class ==== | ||
< | 23:00 -!- nirik changed the topic of #fedora-classroom to: Fedora Classroom - Package Taxonomy and Techniques with your teacher: ivazquez - | ||
See https://fedoraproject.org/wiki/Communicate/IRC/Classroom for more info | |||
23:00 < Ineluctable> thanks stickster | |||
23:00 < stickster> neverho0d: It is, but you could work with another person to edit your summaries | |||
23:00 * kdn *bell rings* | |||
23:00 < brunowolff> If it isn't, then translating it could be another area to help out. | |||
23:01 < neverho0d> stickster: thank you | |||
23:01 < stickster> brunowolff: +1, exactly | |||
23:01 * stickster really scoots now | |||
23:01 < ivazquez> Hello everyone, and thank you for coming. | |||
23:01 < mattia> hi ivazquez | |||
23:01 < Abd4llA> Helloo Mr ivazquez :P | |||
23:01 < neverho0d> brunowolff: I'll be glad to contribute as much as I can | |||
23:01 < thomasj> helloiva | |||
23:01 < thomasj> ops | |||
23:01 < ivazquez> You see them go by all the time in anaconda and PackageKit. | |||
23:01 < kiakli> hi | |||
23:02 < cga> ciao ivazquez, i'm here too ;) | |||
23:02 < thomasj> hello ivazquez | |||
23:02 < ivazquez> libfoo-3.2-5.fc9... barprogs-5.2.6-0.cvs20081016... | |||
23:02 < ivazquez> But just what is it that makes a package... | |||
23:02 < ivazquez> ... a package? | |||
23:02 < domg472> rpmbuil | |||
23:02 < domg472> nvm | |||
23:03 < brunowolff> A spec file. | |||
23:03 < ivazquez> At its simplest, a package is a set of files, and metadata about each file as well as about the entire bundle. | |||
23:03 < kdn> clean installs, clean uninstalls, clean upgrades? | |||
23:03 < ivazquez> Things like permissions, ownership, scripts executed during installation, etc. | |||
23:04 < ivazquez> So let's spend some time looking at these things. | |||
23:04 < ivazquez> The most important thing you need to know about a package is its NEVRA. | |||
23:04 < ivazquez> This stands for Name, Epoch, Version, Release, and Architecture. | |||
23:04 < ivazquez> It's what determines when a package can be installed or upgraded. | |||
23:05 < ivazquez> The name is the part we're most familiar with. | |||
23:05 < ivazquez> glibc, firefox, etc. | |||
23:05 < ivazquez> It is the smallest part of a package we can use and still talk about the package itself. | |||
23:06 < ivazquez> The epoch, version, and release determine the "order" of packages. | |||
23:06 < ivazquez> That is to say, which package is considered "newer" than others. | |||
23:07 < ivazquez> Normally we don't need to worry about the epoch, so we'll hold out on that one for just a moment. | |||
23:07 < ivazquez> Version is mostly self-explanatory. | |||
23:07 < ivazquez> 2.8, 3.0.2, etc. | |||
23:07 < ivazquez> When upstream releases software, they give it a version to keep it separate from all the other versions they release. | |||
23:08 < ivazquez> The first part of a package that we really get to handle as a distro is the release. | |||
23:08 < EvilBob> how should a packager deal with packaging a beta or prerelease version from upstream? | |||
23:09 < ivazquez> That is actually covered in Fedora's packaging guidelines. | |||
23:09 < ivazquez> I'll be happy to discuss packaging issues after. | |||
23:09 <@nirik> http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Pre-Release_packages | |||
23:09 < ivazquez> The release is used to distinguish one package of a specific version from another. | |||
23:10 < ivazquez> So you package libfoo 3.2, and the first package gets a release of "1". | |||
23:10 < ivazquez> Then let's say someone notices a problem in your package, and you fix it. | |||
23:11 < ivazquez> When you go to fix it, you keep the version the same (since it's still the same code from upstream), but you bump the | |||
release up to "2". | |||
23:11 < ivazquez> That way people and tools realize that libfoo-3.2-2 is an update for libfoo-3.2-1. | |||
23:12 < ivazquez> And when upstream releases a new version, say, libfoo 3.3, you can drop the release back down to "1". | |||
23:12 < ivazquez> libfoo-3.3-1 is seen as an update to libfoo-3.2-2 since the version is higher. | |||
23:13 < brunowolff> The kernel seems to keep an increase release number even after minor releases? | |||
23:13 < ivazquez> It used to. They fixed that. | |||
23:13 < ivazquez> The exact details involve all sorts of ugliness regarding the "base" commit. | |||
23:14 < ivazquez> We can come back to that later. | |||
23:14 < brunowolff> When 2.27.5 was released it had -88 for the release number? | |||
23:14 < ivazquez> Now, epoch. | |||
23:14 < brunowolff> OK | |||
23:14 < ivazquez> The epoch is the rarest-seen and the most troublesome part of a package. | |||
23:15 < ivazquez> By default it's not shown when you use rpm to query a package. | |||
23:15 < ivazquez> (and we'll come back to querying after) | |||
23:15 < zcat> if you have rpmdevtools installed you can play with the version/epoch comparisons. rpmdev-vercmp libfoo-3.2-2 libfoo-3.2-1 | |||
23:16 < ivazquez> Additionally, when writing a spec file, you must include this invisible epoch in listing packages that you are dependent | |||
upon. | |||
23:16 < ivazquez> Many a packager has been bitten by omitting the epoch in this scenario. | |||
23:16 < ivazquez> Now, why would anyone use the epoch, given all this trouble? | |||
23:17 < ivazquez> The simple answer is that the version has had issues. | |||
23:18 < ivazquez> For instance, going from, say, "3.2preview" to "3.2final" will be a problem, since 3.2preview actually looks newer than | |||
3.2final. | |||
23:19 < cga> how comes? | |||
23:19 < ivazquez> So what must be done is the epoch must be bumped to 1 (it is 0 by default), and everyone involved lets out a collective | |||
groan because of the issues mentioned earlier. | |||
23:19 < thomasj> cga, p - f | |||
23:19 < Sid> because 2.3preview comes later in an alphabetical listing than 2.3final | |||
23:19 < ivazquez> Strings in a VR are compared lexicographically. | |||
23:19 < Sid> 3.2 even | |||
23:19 < cga> i see thanks | |||
23:19 < EvilBob> cga: alphanumeric ordering | |||
23:19 < cga> fair enough | |||
23:20 < ivazquez> This brings us to the fifth part, the architecture. | |||
23:20 < ivazquez> You would think that this is a very small topic. | |||
23:20 < ivazquez> After all, i386 goes on i386 machines, x86_64 on x86_64, and so on. | |||
23:21 < ivazquez> Unfortunately this is not true. | |||
23:21 < ivazquez> And this is for 2 reasons. | |||
23:21 < ivazquez> The first is that there are architectures that are "equivalent" to each other. | |||
23:22 < ivazquez> E.g., i386, i486, i586, i686, etc. | |||
23:22 < ivazquez> You can only have a single package (with exceptions we'll cover later) with the same NA installed at a time. | |||
23:23 < ivazquez> So you can't have libfoo.i386 installed at the same time as libfoo.i686. | |||
23:23 < ivazquez> The second is a little feature called "multilib". | |||
23:24 < ivazquez> This allows packages from different equivalent architectures to be installed at the same time. | |||
23:24 < ivazquez> It's what lets you install glibc.x86_64 and glibc.i686 on the same system. | |||
23:25 < ivazquez> However, it also has its issues. | |||
23:25 < Discordian> Doesn't that increase the memory residency? | |||
23:26 < ivazquez> One of the issues is when the x86_64 package and the ix86 (i386, i686, etc.) package contain the same files. | |||
23:26 < ivazquez> Sorry, memory residency? | |||
23:26 < Discordian> Don't shared libs take up memory? | |||
23:27 < Discordian> Sorry ignore that please don't want to distract you | |||
23:27 < ivazquez> Only when they're loaded. Packages on the disk take up no memory until they're requested. | |||
23:27 < ivazquez> Alright, same files. | |||
23:28 < cga> Discordian: see differences between application and process (ie. installed versus running) | |||
23:28 < ivazquez> rpm does not have a problem with the x86_64 package and the ix86 package containing the same files, provided that the | |||
files are *exactly* the same. | |||
23:28 < ivazquez> Timestamp, contents, permissions, etc. | |||
23:28 < Discordian> cga: thank you | |||
23:28 < ivazquez> But even this is not true. | |||
23:28 < ivazquez> And this is where we get into trouble. | |||
23:29 < ivazquez> If the x86_64 package and the ix86 package are installed in the same "transaction", rpm is perfectly willing to overlook | |||
differences in the files and will install both packages, with the x86_64 files overwriting the ix86 files. | |||
23:29 < ivazquez> HOWEVER. | |||
23:30 < ivazquez> If either one of the packages is already installed, and we try to install the other package, rpm will complain about file | |||
conflicts since they don't match. | |||
23:31 < ivazquez> This sort of issue has spawned many more bug reports than are deserved. | |||
23:31 < ivazquez> So, something to keep in mind when you install or build packages. | |||
23:32 < ivazquez> So, now that we understand (I hope) these 5 elements, let's look at what else a package's metadata provides. | |||
23:33 <@nirik> I have a question: where does rpm store the ix86 binaries? If the x86_64 package is removed it would have to place the ix86 | |||
ones back right? | |||
23:33 < ivazquez> It does not. It simply doesn't erase the files provided by the x86_64 package. | |||
23:33 < ivazquez> Yay multilib. | |||
23:34 <@nirik> yikes. ;( | |||
23:34 < Discordian> Does that mean you could rebuild both i386 and x64 rpms using rpmrebuild and they'd both be valid? | |||
23:35 < ivazquez> I don't know. I've never looked at how rpmrebuild works. | |||
23:35 < Discordian> My impression is it just uses the rpm db to reconstruct the rpm file | |||
23:36 < ivazquez> I'm glad you bring up the rpmdb. | |||
23:36 < ivazquez> A package in a file has a set of metadata. | |||
23:36 < ivazquez> When said package is installed, the metadata needs to be stored in order to retain information about the package. | |||
23:37 < ivazquez> The metadata is stored in what is called the rpmdb. | |||
23:37 < ivazquez> We can query the rpmdb via "rpm -q". | |||
23:38 < ivazquez> We pass it a name, and rpm will give us the metadata in a pre-programmed format. | |||
23:38 < ivazquez> Normally it looks like <name>-<version>-<release>. | |||
23:38 < ivazquez> I'm not sure if they've added .<arch> to that recently. I believe so though. | |||
23:39 < ivazquez> You can see an example by opening up a terminal and typing "rpm -q filesystem". | |||
23:39 < Discordian> That is the case on F9 yes | |||
23:39 < ivazquez> If someone could provide the output from theirs please? | |||
23:40 < Sid> filesystem-2.4.13-1.fc9.i386 | |||
23:40 < ivazquez> Right, there we are. | |||
23:40 < ivazquez> The name is, of course, "filesystem". | |||
23:41 < ivazquez> The version is "2.4.13", the release is "1.fc9", and the arch is "i386". | |||
23:41 < EvilBob> [root@mediapc1 ~]# rpm -q filesystem | |||
23:41 < EvilBob> filesystem-2.4.13-1.fc9.i386 | |||
23:41 < EvilBob> grrr | |||
23:41 < ivazquez> Sometimes we need more or less than is provided by the default format. | |||
23:41 < zcat> ah. so showing the arch is default now? i've been adding "%_query_all_fmt %%{name}-%%{version}-%%{release}.%%{arch}" to | |||
~/.rpmmacros so i didn't notice | |||
23:41 < EvilBob> filesystem-2.4.11-1.fc8 | |||
23:41 < Discordian> filesystem-2.4.13-1.fc9.i386 is what I have | |||
23:42 < ivazquez> Right, and we're get to query formats right now. | |||
23:42 < ivazquez> *getting | |||
23:42 < ivazquez> Let's say that you have a script that verifies the version of certain packages on your system. | |||
23:42 < ivazquez> You want to narrow down your focus to only the version. | |||
23:43 < ivazquez> Normally you'd do post-processing of the output of "rpm -q". | |||
23:43 < ivazquez> But the default output can be changed, as demonstrated by zcat just above. | |||
23:43 < Discordian> you mean using awk or perl? | |||
23:44 < ivazquez> Correct. | |||
23:44 < ivazquez> What you can do is you can pass rpm a parameter that tells it the format you want. | |||
23:44 < Discordian> cool I can see that would be useful | |||
23:44 < ivazquez> This argument is "--qf", and it takes the format in a special, err, format. | |||
23:45 < ivazquez> Let's deal with just the version for now. | |||
23:45 < ivazquez> To get the version of filesystem we run "rpm -q --qf '%{version}'". | |||
23:45 < ivazquez> This will return just the value, with no fancy formatting. | |||
23:46 < ivazquez> Naturally if the output will be consumed by a person we'll at least want to put it on its own line. | |||
23:46 < ivazquez> The query format string takes several C-style escapes, including \n for a newline. | |||
23:47 < nuonguy> you mean rpm -q filesystem --qf '%{version}' right? | |||
23:47 < ivazquez> "rpm -q --qf '%{version}\n'" will give us something that we can deal with easier. | |||
23:47 < ivazquez> Yes. | |||
23:47 < neverho0d> rpm -q --qf '%{version}\n' filesystem | |||
23:47 < ivazquez> Right. | |||
23:48 < ivazquez> There are a large number of these "query tags" built into rpm. | |||
23:48 < ivazquez> You can see a full list by running "rpm --querytags | less". | |||
23:48 < ivazquez> The query tags in the query format are not case-sensitive. | |||
23:48 < neverho0d> wow 163! | |||
23:49 < ivazquez> Most of them you'll never need though. | |||
23:49 < daMaestro> and shortcuts for those that don't want to type ;-) | |||
23:49 < ivazquez> Among the important ones are the 5 parts that we started with here: | |||
23:50 < ivazquez> name, epoch, version, release, arch. | |||
23:50 < ivazquez> Also important is sourcerpm, which gives you the name of the component in Bugzilla to file bugs against. | |||
23:51 < ivazquez> installtime is also useful. | |||
23:51 < ivazquez> But there are 2 things to know about it. | |||
23:51 < ivazquez> 1) It only makes sense in the rpmdb. Querying it from a package in a file makes no sense. | |||
23:51 < ivazquez> And we'll get to querying a package in a file in a moment. | |||
23:52 < ivazquez> 2) It gives its result as a *nix timestamp. Not exactly human-friendly. | |||
23:53 < ivazquez> So "rpm -q --qf '%{name}: %{installtime}\n' filesystem", while technically correct, isn't all that useful. | |||
23:53 < ivazquez> Fortunately query tags support a suffix that you can add to modify them. | |||
23:54 < ivazquez> The suffix ":date" will convert a timestamp into a human-readable date and time. | |||
23:54 < ivazquez> So with a small change we get "rpm -q --qf '%{name}: %{installtime:date}\n' filesystem", and we can see more clearly when | |||
our system was installed or upgraded. | |||
23:56 < ivazquez> Another query tag of interest is dsaheader. | |||
23:56 < ivazquez> This tag tells us how, when, and who signed a package. | |||
23:56 < ivazquez> We'll get into signing packages a bit later. | |||
23:57 < Discordian> Cool, this is good stuff | |||
23:57 < cga> night all, this is interesting but i'm falling asleep. i'll read the logs tomorrow. i find the classroom idea very interesting. | |||
kudos. | |||
23:57 < ivazquez> Take care. | |||
23:57 < ivazquez> Again, it spews a long hex string that will make our brains asplode if we try to consume it. | |||
23:57 * nirik didn't know the :date stuff. very cool. | |||
23:58 < ivazquez> Suffixing it with ":pgpsig" processes it and gives us a result we can use. | |||
23:58 < daMaestro> yes, i just wasted a few moments trying to pipe the unix timestamp to data -d via xargs | |||
23:58 < daMaestro> thanks. | |||
23:58 < kdn> Thanks to all. This was a Good Thing(tm). | |||
23:58 * delhage agrees | |||
23:58 < ivazquez> rpm -q --qf '%{name}: %{dsaheader:pgpsig}\n' filesystem | |||
23:58 < daMaestro> it's not over | |||
23:58 < Discordian> I'm not going | |||
23:58 < ivazquez> I'll leave it as an exercise for the reader as to whose key that is ;) | |||
23:59 < Discordian> heh | |||
23:59 < brunowolff> How well do you have to know the code to package something from upstream? | |||
23:59 * nirik notes that this is the last class scheduled today... more classess tomorrow. | |||
23:59 < ivazquez> Almost not at all. | |||
23:59 < domg472> why --qf and not -qf, would that conflict? | |||
00:00 < brunowolff> So a good working relationship with upstream would be good enough? | |||
00:00 < ivazquez> -qf is seen as a combination of 2 flags, -q and -f. | |||
00:00 < domg472> i see | |||
00:00 < Discordian> -qf is which package does this file come from | |||
00:00 < ivazquez> brunowolff: Not even. There are a number of instances where upstream is not even aware of it. | |||
00:00 < ivazquez> Okay, packages on disk. | |||
00:00 < Sid> is it possible to query packages that aren't installed? | |||
00:01 < ivazquez> Yes it is. | |||
00:01 < ivazquez> There isn't much to say about packages on disk, other than you need to add -p to rpm. | |||
00:01 < ivazquez> And of course, some tags such as installtime don't make sense. | |||
00:01 < brunowolff> Is there a good place to get info on how to package java apps? I have one I build from ant locally, but I don't know | |||
where fedora wants | |||
00:02 < ivazquez> After all, a package on disk isn't considered to be installed. | |||
00:02 < brunowolff> java apps placed or if there are any particular build practices to use. | |||
00:02 < neverho0d> ivazquez: it would be helpful to explain a package breaking on docs, devel and other things.... | |||
00:02 < Sid> ok, what if the package isn't on disk but simply in the yum repo, can I then query it? | |||
00:02 < ivazquez> brunowolff: Fedora has an extensive set of packaging guidelines in the wiki. | |||
00:02 < Sid> or do I at least need to download the .rpm? | |||
00:02 < ivazquez> Another excellent question. | |||
00:02 < ivazquez> neverho0d: We'll handle that in a bit. | |||
00:03 < ivazquez> Packages in a repo. | |||
00:03 < neverho0d> ivazquez: ok. thnks | |||
00:03 < ivazquez> yum-utils has a script, repoquery, which can be used to query packages in a repo. | |||
00:03 <@nirik> brunowolff: https://fedoraproject.org/wiki/Packaging/Java | |||
00:04 < ivazquez> It takes a subset of the query tags that rpm does, due to the fact that the repo metadata has only a small fraction of | |||
the package metadata. | |||
00:04 < ivazquez> Other than that it takes -q and --qf just like rpm. | |||
00:05 < ivazquez> Alright, I think I've chewed up enough time on my bit. Which topics did I say I'd defer? | |||
00:05 < brunowolff> nirik: Thanks that looks pretty readable. I'll see if I can make it work locally. There are other issues with packaging | |||
the app (Colossus) that I am interested in. | |||
00:05 * nirik is sad that it doesn't provide sourcerpm. ;( | |||
00:05 < domg472> thanks | |||
00:05 < domg472> very informative day | |||
00:06 < ivazquez> Package breakup. | |||
00:06 < Discordian> Thank you very much | |||
00:06 < erinlea80> Thanks ivazquez :) Its been informative. | |||
00:06 < Discordian> I learned a lot | |||
00:06 < ivazquez> Not quite done yet, but I understand if you have other things to do. | |||
00:06 < mattia> thanks ivazquez | |||
00:06 < Discordian> No no I'll stay | |||
00:06 < Bugz> thanks ivazquez | |||
00:06 < ivazquez> Part of the metadata in a package is what "type" a file is. | |||
00:07 < ivazquez> That is to say, whether it's a normal file, a configuration file, or documentation. | |||
00:08 < Discordian> or a script? | |||
00:08 < ivazquez> rpm allows you to query only these specific files by passing -c or -d, but most of the query tags are at the package | |||
level and so don't apply. | |||
00:08 < ivazquez> Package scripts are actually stored separately. | |||
00:08 < ivazquez> They're in the metadata, not the files. | |||
00:08 < Discordian> Okay I've wondered about that | |||
00:09 < ivazquez> They're viewable as either any of the *prog query tags, or by passing --scripts when querying a package. | |||
00:09 < domg472> rpm -q --scripts | |||
00:09 < Discordian> Thank you | |||
00:09 < ivazquez> No, I lie. Not the *prog tags. | |||
00:10 < ivazquez> Those are what is used to actually run the scripts. | |||
00:10 < ivazquez> The scripts themselves are in the prein, postin, preun, postun, and other related tags. | |||
00:11 < ivazquez> Okay, I can't seem to find what other audience topics I decided to defer. | |||
00:11 < ivazquez> Does anyone have any questions?] | |||
00:12 < domg472> it was all clear except the x86 x86_64 bit but that must be my fault | |||
00:12 < nuonguy> is there a virtual file system like /selinux or /proc to browse the rpm database? | |||
00:12 < domg472> ill digginto that myself | |||
00:12 < Discordian> since this is Fedora : what impact will the new version of rpm have? | |||
00:12 < nuonguy> domg472: +1 | |||
00:12 < Discordian> or is that unfair? | |||
00:12 < ivazquez> No. The rpmdb is stored as a set of bdb or sqlite databases in /var/lib/rpm. | |||
00:13 < erinlea80> My questions were answered up above. | |||
00:13 < ivazquez> Which one, rpm5? | |||
00:13 * nirik muses that someone could make a fuse-rpmdb someday... | |||
00:13 < Discordian> Yeah | |||
00:13 < domg472> by the way nuonguy id like to hear what is your reason to contribute to fedora or oss in general? | |||
00:14 < ivazquez> rpm5 is an independent project. I cannot speak for Red Hat, but it has little or no impact in Fedora as far as I can see. | |||
00:14 < ivazquez> the rpm maintainers in RH could probably give you a more complete answer on that one. | |||
00:15 < Discordian> Okay I noted that F10 had flagged up a new version of rpm | |||
00:15 < Discordian> I think perhaps it's 4.x | |||
00:15 < ivazquez> That would be further development in the 4.x branch. | |||
00:15 < Discordian> Yes | |||
00:15 <@nirik> yes, 4.6.0 | |||
00:15 < sdodson> F10 will use 4.6. | |||
00:15 < sdodson> Though, I believe it's been requested that newer features of 4.6 not actually be used until a later date. | |||
00:16 < Discordian> Ahhh cool | |||
00:16 < ivazquez> Anyone else? | |||
00:16 <@nirik> thanks ivazquez. Great session! | |||
00:16 < ivazquez> In that case, thank you for coming, and thank you for listening. | |||
00:17 < Discordian> yes thank you very much ivazquez | |||
00:17 < nuonguy> domg472: when I interview candidates, I ask them if they contribute to any open source projects | |||
00:17 * erinlea80 applauds | |||
00:17 < neverho0d> ivazquez: may be one another class om spec-file building? ;) | |||
[[Category:Classroom]] | [[Category:Classroom]] |
Revision as of 00:23, 9 November 2008
Fedora Classroom - Package Taxonomy and Techniques - Ignacio Vazquez-Abrams - Saturday, November 7, 2008
IRC Log of the Class
23:00 -!- nirik changed the topic of #fedora-classroom to: Fedora Classroom - Package Taxonomy and Techniques with your teacher: ivazquez - See https://fedoraproject.org/wiki/Communicate/IRC/Classroom for more info 23:00 < Ineluctable> thanks stickster 23:00 < stickster> neverho0d: It is, but you could work with another person to edit your summaries 23:00 * kdn *bell rings* 23:00 < brunowolff> If it isn't, then translating it could be another area to help out. 23:01 < neverho0d> stickster: thank you 23:01 < stickster> brunowolff: +1, exactly 23:01 * stickster really scoots now 23:01 < ivazquez> Hello everyone, and thank you for coming. 23:01 < mattia> hi ivazquez 23:01 < Abd4llA> Helloo Mr ivazquez :P 23:01 < neverho0d> brunowolff: I'll be glad to contribute as much as I can 23:01 < thomasj> helloiva 23:01 < thomasj> ops 23:01 < ivazquez> You see them go by all the time in anaconda and PackageKit. 23:01 < kiakli> hi 23:02 < cga> ciao ivazquez, i'm here too ;) 23:02 < thomasj> hello ivazquez 23:02 < ivazquez> libfoo-3.2-5.fc9... barprogs-5.2.6-0.cvs20081016... 23:02 < ivazquez> But just what is it that makes a package... 23:02 < ivazquez> ... a package? 23:02 < domg472> rpmbuil 23:02 < domg472> nvm 23:03 < brunowolff> A spec file. 23:03 < ivazquez> At its simplest, a package is a set of files, and metadata about each file as well as about the entire bundle. 23:03 < kdn> clean installs, clean uninstalls, clean upgrades? 23:03 < ivazquez> Things like permissions, ownership, scripts executed during installation, etc. 23:04 < ivazquez> So let's spend some time looking at these things. 23:04 < ivazquez> The most important thing you need to know about a package is its NEVRA. 23:04 < ivazquez> This stands for Name, Epoch, Version, Release, and Architecture. 23:04 < ivazquez> It's what determines when a package can be installed or upgraded. 23:05 < ivazquez> The name is the part we're most familiar with. 23:05 < ivazquez> glibc, firefox, etc. 23:05 < ivazquez> It is the smallest part of a package we can use and still talk about the package itself. 23:06 < ivazquez> The epoch, version, and release determine the "order" of packages. 23:06 < ivazquez> That is to say, which package is considered "newer" than others. 23:07 < ivazquez> Normally we don't need to worry about the epoch, so we'll hold out on that one for just a moment. 23:07 < ivazquez> Version is mostly self-explanatory. 23:07 < ivazquez> 2.8, 3.0.2, etc. 23:07 < ivazquez> When upstream releases software, they give it a version to keep it separate from all the other versions they release. 23:08 < ivazquez> The first part of a package that we really get to handle as a distro is the release. 23:08 < EvilBob> how should a packager deal with packaging a beta or prerelease version from upstream? 23:09 < ivazquez> That is actually covered in Fedora's packaging guidelines. 23:09 < ivazquez> I'll be happy to discuss packaging issues after. 23:09 <@nirik> http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Pre-Release_packages 23:09 < ivazquez> The release is used to distinguish one package of a specific version from another. 23:10 < ivazquez> So you package libfoo 3.2, and the first package gets a release of "1". 23:10 < ivazquez> Then let's say someone notices a problem in your package, and you fix it. 23:11 < ivazquez> When you go to fix it, you keep the version the same (since it's still the same code from upstream), but you bump the release up to "2". 23:11 < ivazquez> That way people and tools realize that libfoo-3.2-2 is an update for libfoo-3.2-1. 23:12 < ivazquez> And when upstream releases a new version, say, libfoo 3.3, you can drop the release back down to "1". 23:12 < ivazquez> libfoo-3.3-1 is seen as an update to libfoo-3.2-2 since the version is higher. 23:13 < brunowolff> The kernel seems to keep an increase release number even after minor releases? 23:13 < ivazquez> It used to. They fixed that. 23:13 < ivazquez> The exact details involve all sorts of ugliness regarding the "base" commit. 23:14 < ivazquez> We can come back to that later. 23:14 < brunowolff> When 2.27.5 was released it had -88 for the release number? 23:14 < ivazquez> Now, epoch. 23:14 < brunowolff> OK 23:14 < ivazquez> The epoch is the rarest-seen and the most troublesome part of a package. 23:15 < ivazquez> By default it's not shown when you use rpm to query a package. 23:15 < ivazquez> (and we'll come back to querying after) 23:15 < zcat> if you have rpmdevtools installed you can play with the version/epoch comparisons. rpmdev-vercmp libfoo-3.2-2 libfoo-3.2-1 23:16 < ivazquez> Additionally, when writing a spec file, you must include this invisible epoch in listing packages that you are dependent upon. 23:16 < ivazquez> Many a packager has been bitten by omitting the epoch in this scenario. 23:16 < ivazquez> Now, why would anyone use the epoch, given all this trouble? 23:17 < ivazquez> The simple answer is that the version has had issues. 23:18 < ivazquez> For instance, going from, say, "3.2preview" to "3.2final" will be a problem, since 3.2preview actually looks newer than 3.2final. 23:19 < cga> how comes? 23:19 < ivazquez> So what must be done is the epoch must be bumped to 1 (it is 0 by default), and everyone involved lets out a collective groan because of the issues mentioned earlier. 23:19 < thomasj> cga, p - f 23:19 < Sid> because 2.3preview comes later in an alphabetical listing than 2.3final 23:19 < ivazquez> Strings in a VR are compared lexicographically. 23:19 < Sid> 3.2 even 23:19 < cga> i see thanks 23:19 < EvilBob> cga: alphanumeric ordering 23:19 < cga> fair enough 23:20 < ivazquez> This brings us to the fifth part, the architecture. 23:20 < ivazquez> You would think that this is a very small topic. 23:20 < ivazquez> After all, i386 goes on i386 machines, x86_64 on x86_64, and so on. 23:21 < ivazquez> Unfortunately this is not true. 23:21 < ivazquez> And this is for 2 reasons. 23:21 < ivazquez> The first is that there are architectures that are "equivalent" to each other. 23:22 < ivazquez> E.g., i386, i486, i586, i686, etc. 23:22 < ivazquez> You can only have a single package (with exceptions we'll cover later) with the same NA installed at a time. 23:23 < ivazquez> So you can't have libfoo.i386 installed at the same time as libfoo.i686. 23:23 < ivazquez> The second is a little feature called "multilib". 23:24 < ivazquez> This allows packages from different equivalent architectures to be installed at the same time. 23:24 < ivazquez> It's what lets you install glibc.x86_64 and glibc.i686 on the same system. 23:25 < ivazquez> However, it also has its issues. 23:25 < Discordian> Doesn't that increase the memory residency? 23:26 < ivazquez> One of the issues is when the x86_64 package and the ix86 (i386, i686, etc.) package contain the same files. 23:26 < ivazquez> Sorry, memory residency? 23:26 < Discordian> Don't shared libs take up memory? 23:27 < Discordian> Sorry ignore that please don't want to distract you 23:27 < ivazquez> Only when they're loaded. Packages on the disk take up no memory until they're requested. 23:27 < ivazquez> Alright, same files. 23:28 < cga> Discordian: see differences between application and process (ie. installed versus running) 23:28 < ivazquez> rpm does not have a problem with the x86_64 package and the ix86 package containing the same files, provided that the files are *exactly* the same. 23:28 < ivazquez> Timestamp, contents, permissions, etc. 23:28 < Discordian> cga: thank you 23:28 < ivazquez> But even this is not true. 23:28 < ivazquez> And this is where we get into trouble. 23:29 < ivazquez> If the x86_64 package and the ix86 package are installed in the same "transaction", rpm is perfectly willing to overlook differences in the files and will install both packages, with the x86_64 files overwriting the ix86 files. 23:29 < ivazquez> HOWEVER. 23:30 < ivazquez> If either one of the packages is already installed, and we try to install the other package, rpm will complain about file conflicts since they don't match. 23:31 < ivazquez> This sort of issue has spawned many more bug reports than are deserved. 23:31 < ivazquez> So, something to keep in mind when you install or build packages. 23:32 < ivazquez> So, now that we understand (I hope) these 5 elements, let's look at what else a package's metadata provides. 23:33 <@nirik> I have a question: where does rpm store the ix86 binaries? If the x86_64 package is removed it would have to place the ix86 ones back right? 23:33 < ivazquez> It does not. It simply doesn't erase the files provided by the x86_64 package. 23:33 < ivazquez> Yay multilib. 23:34 <@nirik> yikes. ;( 23:34 < Discordian> Does that mean you could rebuild both i386 and x64 rpms using rpmrebuild and they'd both be valid? 23:35 < ivazquez> I don't know. I've never looked at how rpmrebuild works. 23:35 < Discordian> My impression is it just uses the rpm db to reconstruct the rpm file 23:36 < ivazquez> I'm glad you bring up the rpmdb. 23:36 < ivazquez> A package in a file has a set of metadata. 23:36 < ivazquez> When said package is installed, the metadata needs to be stored in order to retain information about the package. 23:37 < ivazquez> The metadata is stored in what is called the rpmdb. 23:37 < ivazquez> We can query the rpmdb via "rpm -q". 23:38 < ivazquez> We pass it a name, and rpm will give us the metadata in a pre-programmed format. 23:38 < ivazquez> Normally it looks like <name>-<version>-<release>. 23:38 < ivazquez> I'm not sure if they've added .<arch> to that recently. I believe so though. 23:39 < ivazquez> You can see an example by opening up a terminal and typing "rpm -q filesystem". 23:39 < Discordian> That is the case on F9 yes 23:39 < ivazquez> If someone could provide the output from theirs please? 23:40 < Sid> filesystem-2.4.13-1.fc9.i386 23:40 < ivazquez> Right, there we are. 23:40 < ivazquez> The name is, of course, "filesystem". 23:41 < ivazquez> The version is "2.4.13", the release is "1.fc9", and the arch is "i386". 23:41 < EvilBob> [root@mediapc1 ~]# rpm -q filesystem 23:41 < EvilBob> filesystem-2.4.13-1.fc9.i386 23:41 < EvilBob> grrr 23:41 < ivazquez> Sometimes we need more or less than is provided by the default format. 23:41 < zcat> ah. so showing the arch is default now? i've been adding "%_query_all_fmt %%{name}-%%{version}-%%{release}.%%{arch}" to ~/.rpmmacros so i didn't notice 23:41 < EvilBob> filesystem-2.4.11-1.fc8 23:41 < Discordian> filesystem-2.4.13-1.fc9.i386 is what I have 23:42 < ivazquez> Right, and we're get to query formats right now. 23:42 < ivazquez> *getting 23:42 < ivazquez> Let's say that you have a script that verifies the version of certain packages on your system. 23:42 < ivazquez> You want to narrow down your focus to only the version. 23:43 < ivazquez> Normally you'd do post-processing of the output of "rpm -q". 23:43 < ivazquez> But the default output can be changed, as demonstrated by zcat just above. 23:43 < Discordian> you mean using awk or perl? 23:44 < ivazquez> Correct. 23:44 < ivazquez> What you can do is you can pass rpm a parameter that tells it the format you want. 23:44 < Discordian> cool I can see that would be useful 23:44 < ivazquez> This argument is "--qf", and it takes the format in a special, err, format. 23:45 < ivazquez> Let's deal with just the version for now. 23:45 < ivazquez> To get the version of filesystem we run "rpm -q --qf '%{version}'". 23:45 < ivazquez> This will return just the value, with no fancy formatting. 23:46 < ivazquez> Naturally if the output will be consumed by a person we'll at least want to put it on its own line. 23:46 < ivazquez> The query format string takes several C-style escapes, including \n for a newline. 23:47 < nuonguy> you mean rpm -q filesystem --qf '%{version}' right? 23:47 < ivazquez> "rpm -q --qf '%{version}\n'" will give us something that we can deal with easier. 23:47 < ivazquez> Yes. 23:47 < neverho0d> rpm -q --qf '%{version}\n' filesystem 23:47 < ivazquez> Right. 23:48 < ivazquez> There are a large number of these "query tags" built into rpm. 23:48 < ivazquez> You can see a full list by running "rpm --querytags | less". 23:48 < ivazquez> The query tags in the query format are not case-sensitive. 23:48 < neverho0d> wow 163! 23:49 < ivazquez> Most of them you'll never need though. 23:49 < daMaestro> and shortcuts for those that don't want to type ;-) 23:49 < ivazquez> Among the important ones are the 5 parts that we started with here: 23:50 < ivazquez> name, epoch, version, release, arch. 23:50 < ivazquez> Also important is sourcerpm, which gives you the name of the component in Bugzilla to file bugs against. 23:51 < ivazquez> installtime is also useful. 23:51 < ivazquez> But there are 2 things to know about it. 23:51 < ivazquez> 1) It only makes sense in the rpmdb. Querying it from a package in a file makes no sense. 23:51 < ivazquez> And we'll get to querying a package in a file in a moment. 23:52 < ivazquez> 2) It gives its result as a *nix timestamp. Not exactly human-friendly. 23:53 < ivazquez> So "rpm -q --qf '%{name}: %{installtime}\n' filesystem", while technically correct, isn't all that useful. 23:53 < ivazquez> Fortunately query tags support a suffix that you can add to modify them. 23:54 < ivazquez> The suffix ":date" will convert a timestamp into a human-readable date and time. 23:54 < ivazquez> So with a small change we get "rpm -q --qf '%{name}: %{installtime:date}\n' filesystem", and we can see more clearly when our system was installed or upgraded. 23:56 < ivazquez> Another query tag of interest is dsaheader. 23:56 < ivazquez> This tag tells us how, when, and who signed a package. 23:56 < ivazquez> We'll get into signing packages a bit later. 23:57 < Discordian> Cool, this is good stuff 23:57 < cga> night all, this is interesting but i'm falling asleep. i'll read the logs tomorrow. i find the classroom idea very interesting. kudos. 23:57 < ivazquez> Take care. 23:57 < ivazquez> Again, it spews a long hex string that will make our brains asplode if we try to consume it. 23:57 * nirik didn't know the :date stuff. very cool. 23:58 < ivazquez> Suffixing it with ":pgpsig" processes it and gives us a result we can use. 23:58 < daMaestro> yes, i just wasted a few moments trying to pipe the unix timestamp to data -d via xargs 23:58 < daMaestro> thanks. 23:58 < kdn> Thanks to all. This was a Good Thing(tm). 23:58 * delhage agrees 23:58 < ivazquez> rpm -q --qf '%{name}: %{dsaheader:pgpsig}\n' filesystem 23:58 < daMaestro> it's not over 23:58 < Discordian> I'm not going 23:58 < ivazquez> I'll leave it as an exercise for the reader as to whose key that is ;) 23:59 < Discordian> heh 23:59 < brunowolff> How well do you have to know the code to package something from upstream? 23:59 * nirik notes that this is the last class scheduled today... more classess tomorrow. 23:59 < ivazquez> Almost not at all. 23:59 < domg472> why --qf and not -qf, would that conflict? 00:00 < brunowolff> So a good working relationship with upstream would be good enough? 00:00 < ivazquez> -qf is seen as a combination of 2 flags, -q and -f. 00:00 < domg472> i see 00:00 < Discordian> -qf is which package does this file come from 00:00 < ivazquez> brunowolff: Not even. There are a number of instances where upstream is not even aware of it. 00:00 < ivazquez> Okay, packages on disk. 00:00 < Sid> is it possible to query packages that aren't installed? 00:01 < ivazquez> Yes it is. 00:01 < ivazquez> There isn't much to say about packages on disk, other than you need to add -p to rpm. 00:01 < ivazquez> And of course, some tags such as installtime don't make sense. 00:01 < brunowolff> Is there a good place to get info on how to package java apps? I have one I build from ant locally, but I don't know where fedora wants 00:02 < ivazquez> After all, a package on disk isn't considered to be installed. 00:02 < brunowolff> java apps placed or if there are any particular build practices to use. 00:02 < neverho0d> ivazquez: it would be helpful to explain a package breaking on docs, devel and other things.... 00:02 < Sid> ok, what if the package isn't on disk but simply in the yum repo, can I then query it? 00:02 < ivazquez> brunowolff: Fedora has an extensive set of packaging guidelines in the wiki. 00:02 < Sid> or do I at least need to download the .rpm? 00:02 < ivazquez> Another excellent question. 00:02 < ivazquez> neverho0d: We'll handle that in a bit. 00:03 < ivazquez> Packages in a repo. 00:03 < neverho0d> ivazquez: ok. thnks 00:03 < ivazquez> yum-utils has a script, repoquery, which can be used to query packages in a repo. 00:03 <@nirik> brunowolff: https://fedoraproject.org/wiki/Packaging/Java 00:04 < ivazquez> It takes a subset of the query tags that rpm does, due to the fact that the repo metadata has only a small fraction of the package metadata. 00:04 < ivazquez> Other than that it takes -q and --qf just like rpm. 00:05 < ivazquez> Alright, I think I've chewed up enough time on my bit. Which topics did I say I'd defer? 00:05 < brunowolff> nirik: Thanks that looks pretty readable. I'll see if I can make it work locally. There are other issues with packaging the app (Colossus) that I am interested in. 00:05 * nirik is sad that it doesn't provide sourcerpm. ;( 00:05 < domg472> thanks 00:05 < domg472> very informative day 00:06 < ivazquez> Package breakup. 00:06 < Discordian> Thank you very much 00:06 < erinlea80> Thanks ivazquez :) Its been informative. 00:06 < Discordian> I learned a lot 00:06 < ivazquez> Not quite done yet, but I understand if you have other things to do. 00:06 < mattia> thanks ivazquez 00:06 < Discordian> No no I'll stay 00:06 < Bugz> thanks ivazquez 00:06 < ivazquez> Part of the metadata in a package is what "type" a file is. 00:07 < ivazquez> That is to say, whether it's a normal file, a configuration file, or documentation. 00:08 < Discordian> or a script? 00:08 < ivazquez> rpm allows you to query only these specific files by passing -c or -d, but most of the query tags are at the package level and so don't apply. 00:08 < ivazquez> Package scripts are actually stored separately. 00:08 < ivazquez> They're in the metadata, not the files. 00:08 < Discordian> Okay I've wondered about that 00:09 < ivazquez> They're viewable as either any of the *prog query tags, or by passing --scripts when querying a package. 00:09 < domg472> rpm -q --scripts 00:09 < Discordian> Thank you 00:09 < ivazquez> No, I lie. Not the *prog tags. 00:10 < ivazquez> Those are what is used to actually run the scripts. 00:10 < ivazquez> The scripts themselves are in the prein, postin, preun, postun, and other related tags. 00:11 < ivazquez> Okay, I can't seem to find what other audience topics I decided to defer. 00:11 < ivazquez> Does anyone have any questions?] 00:12 < domg472> it was all clear except the x86 x86_64 bit but that must be my fault 00:12 < nuonguy> is there a virtual file system like /selinux or /proc to browse the rpm database? 00:12 < domg472> ill digginto that myself 00:12 < Discordian> since this is Fedora : what impact will the new version of rpm have? 00:12 < nuonguy> domg472: +1 00:12 < Discordian> or is that unfair? 00:12 < ivazquez> No. The rpmdb is stored as a set of bdb or sqlite databases in /var/lib/rpm. 00:13 < erinlea80> My questions were answered up above. 00:13 < ivazquez> Which one, rpm5? 00:13 * nirik muses that someone could make a fuse-rpmdb someday... 00:13 < Discordian> Yeah 00:13 < domg472> by the way nuonguy id like to hear what is your reason to contribute to fedora or oss in general? 00:14 < ivazquez> rpm5 is an independent project. I cannot speak for Red Hat, but it has little or no impact in Fedora as far as I can see. 00:14 < ivazquez> the rpm maintainers in RH could probably give you a more complete answer on that one. 00:15 < Discordian> Okay I noted that F10 had flagged up a new version of rpm 00:15 < Discordian> I think perhaps it's 4.x 00:15 < ivazquez> That would be further development in the 4.x branch. 00:15 < Discordian> Yes 00:15 <@nirik> yes, 4.6.0 00:15 < sdodson> F10 will use 4.6. 00:15 < sdodson> Though, I believe it's been requested that newer features of 4.6 not actually be used until a later date. 00:16 < Discordian> Ahhh cool 00:16 < ivazquez> Anyone else? 00:16 <@nirik> thanks ivazquez. Great session! 00:16 < ivazquez> In that case, thank you for coming, and thank you for listening. 00:17 < Discordian> yes thank you very much ivazquez 00:17 < nuonguy> domg472: when I interview candidates, I ask them if they contribute to any open source projects 00:17 * erinlea80 applauds 00:17 < neverho0d> ivazquez: may be one another class om spec-file building? ;)