core: intercept groupadd/useradd/usermod calls in scriptlets

[lucab]

Background discussion behind this happened in #3712 (comment).

In the context of making incremental progress on the sysusers.d front, we'd like to start auto-generating fragments for system users and groups.
Attempting to do this post-fact by parsing /etc content hits a minor problem related to distinguishing dynamic and static IDs.
Moving a bit earlier in scriptlets processing though we can intercept calls to useradd and groupadd in order to learn whether they were given static IDs.

An example of dynamic IDs (from chrony RPM) looks like this:

%pre
getent group chrony > /dev/null || /usr/sbin/groupadd -r chrony
getent passwd chrony > /dev/null || /usr/sbin/useradd -r -g chrony \
       -d %{_localstatedir}/lib/chrony -s /sbin/nologin chrony
:

Instead, an example of static IDs (from squid RPM) looks like this:

%pre
if ! getent group squid >/dev/null 2>&1; then
  /usr/sbin/groupadd -g 23 squid
fi

if ! getent passwd squid >/dev/null 2>&1 ; then
  /usr/sbin/useradd -g 23 -u 23 -d /var/spool/squid -r -s /sbin/nologin squid >/dev/null 2>&1 || exit 1 
fi

Let's start adding some groupadd and useradd wrapper in scriptlets environment that will:

• parse the CLI args to gain knowledge about static IDs
• generate the relevant sysusers.d fragment
• forward to the real groupadd/useradd binary

[lucab]

I realized there are some packages (e.g. clevis) which are calling usermod, so we should also intercept that and translate it to a m entry.
Overall progress:

• intercept useradd - core: wrap and intercept useradd calls in scriptlets #3897
• intercept groupadd - core: wrap and intercept groupadd calls in scriptlets #3778
• intercept usermod - core: wrap and intercept usermod calls in scriptlets #3942

[cgwalters]

I filed ostreedev/ostree-rs-ext#383 but probably it's a duplicate of this (though we should debate it living in ostree).

What do you see as the status on this? It seems like we landed code, but it's disabled by default?

Do we need an opt-in sysusers: true?

[lucab]

We did land all the wrappers but they are currently gated by a RPMOSTREE_EXP_BRIDGE_SYSUSERS env flag:

rpm-ostree/src/libpriv/usermod-wrapper.sh

Lines 7 to 9 in ea5e9b6

	if test -v RPMOSTREE_EXP_BRIDGE_SYSUSERS; then
	rpm-ostree scriptlet-intercept usermod -- "$0" "$@"
	fi

Do we need an opt-in sysusers: true?

At some point yes, but this is entangled in 1) cleanups on Fedora side (most things should be ok in F38), and 2) figuring out the interactions with all the other users/groups configuration fields in the treefile.

If you are mostly caring about the in-container flow at this point, I think at this time it makes sense to turn it always-on there.

[cgwalters]

Ah but it doesn't work in the native-container flow because we're only using the libdnf path there. Filed #4075

[keszybz]

I'm late to the party, a bit, but I was just working on something similar: https://github.com/keszybz/sysusers-convert. The full list of stuff to intercept: passwd, usermod, groupmod, gpasswd.

I plan to convert all packages to use sysusers files. I hope this works for rpm-ostree too.

[cgwalters]

I plan to convert all packages to use sysusers files. I hope this works for rpm-ostree too.

Great! Well yes it should, I can't think why it wouldn't.

But regarding interception; generation: One overall challenge is that in the bigger picture we really want this to work outside of RPM too. I think as you know, we are investing deeply into containers (in addition to RPM) as a mechanism for operating system management, and people definitely have come to expect RUN useradd to work. In the general case we may have 3rd party installer scripts that add users, and there's also the related but different case of people wanting to add non-system i.e. login users.

Over in containers/bootc#1055 (comment) I'm thinking about moving some of the Rust tmpfiles.d and sysusers.d handling over to bootc. What's supremely ugly is that in the general case for image builds we need to clearly disentangle and canonicalize what's in /etc/passwd etc. from uids that have corresponding sysusers.d entries vs those that don't, etc.

[keszybz]

The nice thing about the latest changes in rpm is that we no longer need the tools for handling users to be present in the image. When rpm or dnf operate on a chroot, user and group accounts can be created using a tool called from the host. I expect that we'll have nicer leaner container images once this goes through.

I don't see why RUN useradd wouldn't work. sysusers calls and normal useradd/groupadd calls modify the same database in a compatible way.

Over in containers/bootc#1055 (comment) I'm thinking about moving some of the Rust tmpfiles.d and sysusers.d handling over to bootc.

I have no idea about this. I would expect that tmpfiles config would be used to create … temporary files, so stuff that is not part of the image, but ephemeral.

[cgwalters]

I don't see why RUN useradd wouldn't work. sysusers calls and normal useradd/groupadd calls modify the same database in a compatible way.

Because of the merge problem; ostree doesn't special case /etc/passwd so if it's locally modified (e.g. cloud-init injected a user) we aren't going to overwrite it.

I have no idea about this. I would expect that tmpfiles config would be used to create … temporary files, so stuff that is not part of the image, but ephemeral.

tmpfiles is definitely used to create things that I believe most system administrators don't think of as "temporary" or "ephemeral" in the sense one would ordinarily use those terms - for example, it may create /var/lib/postgres or /home. Yes, this all ties into a larger story about factory reset and such.

But anyways the code I am talking about here is to handle the problem where many rpms (like postgres) physically ship /var/lib/foo or /var/log/foo and expect that to exist, and we automatically synthesize tmpfiles.d snippets from those. For generic builds (that might have RPMs as a base but also non-RPM content) we need to treat that symmetrically as well ideally.