The gen on the Debian systemd packaging hoo-hah

Desktop motivations

Put yourself in the position of someone writing a "desktop" system for Linux and the BSDs. You've reached the part where you're writing a "control panel" gadget for allowing system administrators (and appropriately privileged users) to manage things like the system time, the hostname, the default locale, and so forth; and another "computer management" gadget for letting the desktop user see who is logged on where and suchlike. It's a right old mess. You have to determine not only what kernel you are running on but what distribution you are running on, and access different things in different places. You could find yourself variously having to make your programs read, parse, and rewrite /etc/locale.conf (Arch), /etc/default/locale (Ubuntu), /etc/sysconfig/i18n (CentOS), /etc/sysconfig/language (OpenSuSE), /etc/sysconf/i18n, or /etc/login.conf (FreeBSD), for just one example. Then there are things like fundamental differences in the utmpx binary logs (The BSDs have revamped theirs and got rid of some stuff.), the system account database (It's an actual database on most BSDs.), the naming systems for virtual console devices, subtle differences in configuration file syntax (to some, a configuration file may be a shell script that sets shell variables, to others, it's a list of key=value pairs), …

Enter the "kits", twice since renamed.

Skipping a lot of history, such as where ConsoleKit begat systemd-logind became logind for one, what we have now is a suite of remote procedure calls that your gadgets can call. There's a "hostname" API, a "timedate" API, a "locale" API, and a "login" API. They are a whole bunch of RPC calls that are made using an inter-process communication system known as the "desktop bus", D-Bus. (D-Bus, in its turn, is built on top of sockets. It has all of the usual RPC "stuff" that has been done several times before, such as marshalling and unmarshalling, mechanisms for rendezvous between clients and servers, and API definitions. It incorporates a security system to allow servers to restrict access to certain RPC functions to only suitably privileged clients.)

These RPCs are supplied by server daemons, that communicate with client programs (e.g. the aforementioned gadgets) over D-Bus. The operating system specific stuff, such as exactly what configuration file contains the static hostname on the system, is intended to be contained within these server daemons. All that a client knows is that it makes the "set the static hostname" RPC call, and some server does the necessary work whatever that is. The Debian systemd package comes with its own implementation of these server daemons. Contrary to the recommendation of the GNOME people almost three years ago, but in line with what the systemd people prefer, on Debian they aren't packaged separately and are indivisible from the remainder of the package.

This is what three years' worth of hoo-hah has, at its root, all been about: a packaging decision. I'm not going to summarize or re-hash the hoo-hah here. Suffice it to say that there are both engineering rationales and socio-political rationales for the decision, and the major problem is logind, the systemd server dæmon that provides the "login" API. Even though the other three somewhat are (modulo the fact that they all pick the particular choices of configuration file locations that match what other programs in the systemd package, such as systemd-machine-id-setup, use), the fourth, logind, is not at all severable from the rest of the systemd package, because the operating-system specific underpinnings of it (These daemons are intended to be abstracting a whole bunch of operating-system-specific non-portable stuff, remember.) target a Linux system that is running the systemd dæmon and its ancillary daemons and utilities. Naturally enough.

systembsd

The systemd versions of these daemons are not the only ones in existence. A chap by the name of Ian Sutton (sometimes going by the name Ian Kremlin), for one, is in the process of writing his own implementations of these four daemons, speaking the same RPC APIs over D-Bus but having the operating-system specific underpinnings that are suitable for the BSDs. This a good thing from the point of view of keeping the protocol specifications honest (as a heterogeneous set of implementations often does), and it is an example of what the GNOME people and the systemd people expected: a separate suite of operating-system-specific dæmon programs for each operating system flavour, with hidden-from-clients inner workings that use whatever is actually appropriate for that operating system.

(I said on a mailing list in mid-2014 that a good question to ask was whether systembsd's last modification date really was March 1973. Now you know why. He was of course no doubt testing his timedated. It gave me a small chuckle.)

However, again the stumbling block is logind. He hasn't got very far with it yet. (It's by far the largest of the four APIs.) Even if he does, the whole point of these programs is that they provide a generic layer on top of the BSD-specific; and the BSDs don't work the same as Linux.

What the "login" API does

Aside from being the largest of the APIs, the "login" API is also the least obvious from its name. To understand it, one has to understand that the systemd/Freedesktop people have constructed a whole abstraction based upon their notion of "multi-seat Linux" (a concept that used to be called "multi-head", and before that, in the days of actual terminals connected to computers, "multi-user"). It has notions of "seats", "sessions", and "users"; which I'm not going to detail here.

It didn't help ConsoleKit that, in addition to being a fashion victim with its name (which gave even less clue as to the function of the dæmon), it followed long standing Open Source Software tradition.

The "multi-seat"'s raison d'être (glossing over some detail) is that the existing abstractions don't extend well to allowing multiple users to log in using the same collection of physical hardware and share the hardware amongst themselves as the currently "active" user changes; as well as facilitating the prevention of users from being able to access hardware, that isn't assigned to their login session, simply because they happen to have logged on locally on a "console". And one of the ironies of the people who wail on about how the systemd people are copying Windows NT is that the route that they've gone down with this is fundamentally different in one major respect to how Windows NT deals with exactly the same problem. (Reducing complexity and a lot of background to two sentences: Windows NT uses nonce SIDs, modifies process tokens, and uses largely fixed object ACLs. The "multi-seat Linux" system uses fixed process credentials and varies the device permissions/ACLs.)

The "login" API, previously provided by programs called systemd-logind and ConsoleKit and now provided (in the systemd package at least) by a program called simply logind (in some recent doco), allows one to query and manipulate all of this "seat", "session", and "user" stuff, again with a view to writing client applications such as the "computer management" gadget tools in a desktop.

Bundled API clients

An additional part of the hoo-hah fuel is that several such client programs are bundled.

In the systemd package there are tools named "hostnamectl", "timedatectl", "localectl", and "loginctl". Supposedly, these are such client applications, that speak to the RPC API. By their very natures, therefore, they should be easily severable from the systemd package; a "guaranteed stable interface" sits between each of them and the operating-system-specific daemons that do the actual grunt work. But they are not separately packaged in Debian.

systemd-shim failure

Systemd's logind is hence (at the time of writing this FGA, and still today) the sole implementation of the "login" API. The Debian package sometimes presented as fixing this problem is named systemd-shim. What systemd-shim does, or rather attempts to do, is supply the various lower level non-portable operating-system-specific parts that systemd's logind is layered on top of, and that logind abstracts away for the "login" API. One then still has the Debian systemd package installed, because of course one is still using its logind program; but one doesn't use the systemd program from it, instead substituting the systemd-shim server dæmon.

The problem is that it can only ever attempt to do this, because the goal that it has to aim for keeps moving around. This has already caused it problems several times, and will by the nature of that goal continue to cause it problems.

The goal is to provide those parts of the "systemd process" API that systemd's logind uses. The model of the world that the systemd people wanted (but, importantly, did not eventually get) was that there was only one, privileged, "control groups manager" process in the system and every other process has to make (some kind of) an RPC call to it in order to manipulate process control groups. That was to be process #1 running the systemd program itself, and logind and others were to speak to it using its "systemd process" API, and not deal in control groups directly at all. There are also other parts of the "systemd process" API that logind uses, such as RPC calls to change system state.

However, this "systemd process" API is not (at the time of writing this FGA, and for a long while afterwards) a "guaranteed stable interface". The systemd people regard themselves as free to modify the internal workings of their system at any time for whatever purpose suits them. They already have, once; and it broke systemd-shim for months during 2014 because the people who develop/maintain Debian's systemd-shim (At the time of writing this FGA, they were solely Steve Langasek.) didn't modify it to match what systemd's logind changed to need. (To contrast the available manpower: At the time of writing this FGA, the Debian maintainers of the Debian systemd package were Tollef Fog Heen, Michael Biebl, Marco d'Itri, Michael Stapelberg, and Sjoerd Simons; and the people, independent of Debian, working on the systemd project proper were Lennart Poettering, Kay Sievers, Greg Kroah-Hartman, Zbigniew Jedrzejewski-Szmek, Tom Gundersen, David Herrmann, and a whole lot of others.)

This is bound to happen again. In effect systemd-shim is continually and perpetually broken by its very design. The right thing to do is not to provide a shim for an internal, unsupported, and explicitly not guaranteed by its architects to be stable, "systemd process" API; but rather to provide a dæmon that provides the external, supported, and guaranteed stable "login" API. The problem is, as can be seen from systembsd, that that is a hefty chunk of development work; and no-one has even begun such a dæmon that maps the "login" API to the non-portable operating-system-specific underpinnings on Debian Linux with something other than the systemd program running as process #1.

Furthermore, as noted, the systemd people did not actually get the changes to the Linux kernel that they wanted. They wanted the version 2 control groups system to provide "job" objects, with systemd as the "job" object system's single operator. They didn't get "job" objects, and the version 2 control groups API was designed by the kernel writers to have a distributed control API, which did not involve a single central gatekeeper process. The "control groups interface" that the systemd people laid out, and that has been circulated for years, presents what the systemd people wanted as if it were accomplished fact. In reality, it is a rejected proposal that never happened.

So even the architecture that the systemd people continue to tell the world that the likes of systemd-shim have to adhere to, is wrong.


© Copyright 2014,2017 Jonathan de Boyne Pollard. "Moral" rights asserted.
Permission is hereby granted to copy and to distribute this WWW page in its original, unmodified form as long as its last modification datestamp information is preserved.