From Fedora Project Wiki
Line 13: Line 13:


== Detailed Description ==
== Detailed Description ==
Storing the user and group information in structured data allows to process this data with external tools in a programmatic way instead of running opaque shell scripts.  
Storing the user and group information in structured data allows to process this data with external tools in a programmatic way instead of running opaque shell scripts.
 
This is part of a long-term effort to convert RPM shell scripts to structured data available in the RPM database or the /usr filesystem. Ideally., the operating system image creation process could create a /usr tree which contains all the information to instantiate a bootable system. The RPM script only run at package installation time and their result cannot be reproduced on the installed system when the content of /etc  and /var is programmatically recreated from the content in /usr.


Example test.spec file processing:
Example test.spec file processing:

Revision as of 16:24, 8 November 2018

Streamline useradd/groupadd calls in RPM spec files

Summary

Replace RPM useradd/groupadd shell script fragments in spec files with dedicated RPM macros.

Owner

Current status

  • Targeted release: Fedora 30
  • Tracker bug: <will be assigned by the Wrangler>

Detailed Description

Storing the user and group information in structured data allows to process this data with external tools in a programmatic way instead of running opaque shell scripts.

This is part of a long-term effort to convert RPM shell scripts to structured data available in the RPM database or the /usr filesystem. Ideally., the operating system image creation process could create a /usr tree which contains all the information to instantiate a bootable system. The RPM script only run at package installation time and their result cannot be reproduced on the installed system when the content of /etc and /var is programmatically recreated from the content in /usr.

Example test.spec file processing:

$ rpmbuild --define "_sourcedir $(pwd)" --define "_specdir $(pwd)" --define "_builddir $(pwd)" --define "_srcrpmdir $(pwd)" --define "_rpmdir $(pwd)" -ba test.spec && rpm -qp --scripts noarch/*.rpm && rpm -qp --provides  noarch/*.rpm

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.XfQ6P6
+ umask 022
+ cd /home/kay
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.KFSrt0
+ umask 022
+ cd /home/kay
+ exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.XC996T
+ umask 022
+ cd /home/kay
+ '[' /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64 '!=' / ']'
+ rm -rf /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64
++ dirname /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64
+ mkdir -p /home/kay/rpmbuild/BUILDROOT
+ mkdir /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64
+ mkdir -p /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64/usr/lib/sysusers.d/
+ cat
+ mkdir -p /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64/usr/lib/sysusers.d/
+ cat
+ mkdir -p /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64/usr/lib/sysusers.d/
+ cat
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-ldconfig
/sbin/ldconfig: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf: No such file or directory
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/brp-strip /usr/bin/strip
+ /usr/lib/rpm/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1 1
+ /usr/lib/rpm/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-mangle-shebangs
/usr/share/lmod/lmod/init/bash: line 15: __lmod_vx: unbound variable
Processing files: test-1-1.noarch
Provides: group(group1) group(group2) group(group3) group(group4) test = 1-1 user(user1) user(user2)
Requires(interp): /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(pre): /bin/sh
Processing files: test-sub-1-1.noarch
Provides: group(subgroup1) test-sub = 1-1
Requires(interp): /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(pre): /bin/sh
Processing files: foo-1-1.noarch
Provides: foo = 1-1 group(foo1)
Requires(interp): /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(pre): /bin/sh
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64
Wrote: /home/kay/test-1-1.src.rpm
Wrote: /home/kay/noarch/test-1-1.noarch.rpm
Wrote: /home/kay/noarch/test-sub-1-1.noarch.rpm
Wrote: /home/kay/noarch/foo-1-1.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.4EIRZN
+ umask 022
+ cd /home/kay
+ /usr/bin/rm -rf /home/kay/rpmbuild/BUILDROOT/test-1-1.x86_64
+ exit 0

It has created the following RPMs:

$ tree noarch/
noarch/
├── foo-1-1.noarch.rpm
├── test-1-1.noarch.rpm
└── test-sub-1-1.noarch.rpm

$ rpm -ihv noarch/*
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Creating group subgroup1 with gid 968.
Updating / installing...
   1:test-sub-1-1                     ################################# [ 33%]
Creating group group1 with gid 967.
Creating group group2 with gid 966.
Creating group group3 with gid 13.
Creating group group4 with gid 965.
Creating group user1 with gid 964.
Creating user user1 (User 1) with uid 964 and gid 964.
Creating group user2 with gid 963.
Creating user user2 (User 2) with uid 963 and gid 963.
   2:test-1-1                         ################################# [ 67%]
Creating group foo1 with gid 962.
   3:foo-1-1                          ################################# [100%]

The macros created sysusers files which are installed on the system:

$ cat /usr/lib/sysusers.d/test.conf
g	group1	11	-
g	group2	12	-
g	group3	13	-
g	group4	-	-
u	user1	100	"User 1"	/var/user1	/sbin/nologin
m	user1	group1
m	user1	group3
m	user1	group4
u	user2	-	"User 2"	/var/user2	/sbin/nologin
m	user2	group2
m	user2	group3
m	user2	group4

$ cat /usr/lib/sysusers.d/test-sub.conf 
g	subgroup1	-	-

$ cat /usr/lib/sysusers.d/foo.conf 
g	foo1	-	-

The macros added the scripts to the RPM package which run at installation time:

$ rpm -qp --scripts noarch/*.rpm
preinstall scriptlet (using /bin/sh):
systemd-sysusers --replace=/usr/lib/sysusers.d/foo.conf - <<EOF
g	foo1	-	-
EOF
preinstall scriptlet (using /bin/sh):
systemd-sysusers --replace=/usr/lib/sysusers.d/test.conf - <<EOF
g	group1	11	-
g	group2	12	-
g	group3	13	-
g	group4	-	-
u	user1	100	"User 1"	/var/user1	/sbin/nologin
m	user1	group1
m	user1	group3
m	user1	group4
u	user2	-	"User 2"	/var/user2	/sbin/nologin
m	user2	group2
m	user2	group3
m	user2	group4
EOF
preinstall scriptlet (using /bin/sh):
systemd-sysusers --replace=/usr/lib/sysusers.d/test-sub.conf - <<EOF
g	subgroup1	-	-
EOF

The macros added Provides: to the packages, for all provided users and groups, so other packages can depend on them and pin the packages which provide the users and groups:

$ rpm -qp --provides  noarch/*.rpm
foo = 1-1
group(foo1)
group(group1)
group(group2)
group(group3)
group(group4)
test = 1-1
user(user1)
user(user2)
group(subgroup1)
test-sub = 1-1

The example spec file which still has the new sysusers macros included (They will move to a separate file):

#################################################################
# macros.sysusers
#################################################################
%define sysusers_useradd(n:S:c:d:g:G:lmMNors:u:UZ:) %{lua:
   local package = rpm.expand("%{?-S*}")
   local name = rpm.expand("%{?!-n*:%{name}}%{?-n*}")
   if package ~= "" then
      name = name .. "_" .. package
   end
   
   local oldlines = rpm.expand("%{?sysusers_useradd_" .. name .. "}")

   local group = rpm.expand("%{?-g*}")
   local gecko = rpm.expand("%{?-c*}%{?!-c*:-}")
   local home = rpm.expand("%{?-d*}%{!-d*:-}")
   local shell = rpm.expand("%{?-s*}%{!-s*:-}")
   local uid = rpm.expand("%{?-u*}%{!-u*:-}")
   local user = rpm.expand("%{1}")
   local moregroups = rpm.expand("%{?-G*}")
   
   newline = "%{quote:u\\t" .. user .. "\\t" .. uid .. "\\t" .. gecko .. "\\t" .. home .. "\\t" .. shell .. "}"

   if group ~= "" then
      newline = newline .. "%{quote:m\\t" .. user .. "\\t" .. group .. "}"
   end

   for group in string.gmatch(moregroups, "[^,]*,?") do
      if string.sub(group,-1) == "," then group = string.sub(group, 0, -2) end
      newline = newline .. "%{quote:m\\t" .. user .. "\\t" .. group .. "}"
   end

   if oldlines == "" then
      rpm.define("sysusers_useradd_" .. name .. " " .. newline)
   else
      rpm.define("sysusers_useradd_" .. name .. " " .. oldlines .. newline)
   end
   print("Provides: user(" .. user .. ")\\n")
}

%define sysusers_groupadd(n:S:g:rfoB:N:) %{lua:
   local package = rpm.expand("%{?-S*}")
   local name = rpm.expand("%{?!-n*:%{name}}%{?-n*}")
   if package ~= "" then
      name = name .. "_" .. package
   end
   local oldlines = rpm.expand("%{?sysusers_useradd_" .. name .. "}")

   local gid = rpm.expand("%{?-g*}%{!-g*:-}")
   local group = rpm.expand("%{1}")

   newline = "%{quote:g\\t" .. group .. "\\t" .. gid .. "\\t-}"

   if oldlines == "" then
      rpm.define("sysusers_useradd_" .. name .. " " .. newline)
   else
      rpm.define("sysusers_useradd_" .. name .. " " .. oldlines .. newline)
   end
   print("Provides: group(" .. group .. ")\\n")
}

%define sysusers_pre(n:S:) %{lua:
   local package = rpm.expand("%{?-S*}")
   local name = rpm.expand("%{?!-n*:%{name}}%{?-n*}")
   local filename = name
   if package ~= "" then
      filename = name .. "-" .. package
      name = name .. "_" .. package
   end
   local lines = rpm.expand("%{?sysusers_useradd_" .. name .. "}")
   print("systemd-sysusers --replace=" .. rpm.expand("%{_sysusersdir}/") .. filename .. ".conf - <<EOF\\n")
   for line in string.gmatch(lines, "\\31[^\\31]*\\31") do print(string.sub(line, 2, -2) .."\\n")  end
   print("EOF\\n")
}

%define sysusers_install(n:S:) %{lua:
   local package = rpm.expand("%{?-S*}")
   local name = rpm.expand("%{?!-n*:%{name}}%{?-n*}")
   local filename = name
   if package ~= "" then
      filename = name .. "-" .. package
      name = name .. "_" .. package
   end
   local lines = rpm.expand("%{?sysusers_useradd_" .. name .. "}")
   print("mkdir -p " .. rpm.expand("%{buildroot}") .. rpm.expand("%{_sysusersdir}/") .. "\\n")
   print("cat >" .. rpm.expand("%{buildroot}") .. rpm.expand("%{_sysusersdir}/") .. filename .. ".conf <<EOF\\n")
   for line in string.gmatch(lines, "\\31[^\\31]*\\31") do print(string.sub(line, 2, -2) .."\\n") end
   print("EOF\\n")
}

#################################################################
# test.spec
#################################################################
Name:           test
Version:        1
Release:        1
Summary:        test
License:        MIT
BuildArch:      noarch

%define GROUPNAME1 group1
%define USERNAME1  user1

%sysusers_groupadd -g 11 %{GROUPNAME1}
%sysusers_groupadd -g 12 group2
%sysusers_groupadd -g 13 group3
%sysusers_groupadd group4
%sysusers_useradd -g group1 -G group3,group4 -u 100 -d /var/user1 -s /sbin/nologin -c %{quote:"User 1"} %{USERNAME1}
%sysusers_useradd -g group2 -G group3,group4 -d /var/user2 -s /sbin/nologin -c %{quote:"User 2"} user2

%description

%package sub
Summary:        sub
%sysusers_groupadd -S sub subgroup1
%description sub

%package -n foo
Summary:        foo
%sysusers_groupadd -n foo foo1
%description -n foo

%pre
%sysusers_pre

%pre sub
%sysusers_pre -S sub

%pre -n foo
%sysusers_pre -n foo

%prep

%build

%install
%sysusers_install
%sysusers_install -n foo
%sysusers_install -S sub

%files
%{_sysusersdir}/%{name}.conf

%files sub
%{_sysusersdir}/%{name}-sub.conf

%files -n foo
%{_sysusersdir}/foo.conf

Benefit to Fedora

Scope

  • Proposal owners:
  • Other developers: N/A (not a System Wide Change)
  • Policies and guidelines: N/A (not a System Wide Change)
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

N/A (not a System Wide Change)

User Experience

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? N/A (not a System Wide Change), Yes/No
  • Blocks product? product

Documentation

N/A (not a System Wide Change)

Release Notes