M1: standalone Kerberos KDC on mail (realm CLOONAR.COM) + IdP-swap ADR #267

New issue

Closed

opened 2026-08-07 11:01:46 +02:00 by dominik.polakovics · 0 comments

dominik.polakovics commented

2026-08-07 11:01:46 +02:00

Owner

Part of #266 (M1 of M1–M6). Stand up a standalone Kerberos KDC on mail and write the ADR for the IdP swap.

This is the first, fully additive step toward real desktop SSO. Nothing consumes the KDC when this lands — it is verifiable standalone with kinit and revertible by removing one module.

Design decisions already made (do not re-litigate)

These were resolved in a design session on #266. The ADR records them; the implementation follows them.

Kerberos is additive, not authoritative. Principals exist only for the humans who log into nb. LDAP userPassword stays the authoritative password store for every user and every other service.
No {SASL} password pass-through. The original proposal in #266 (userPassword: {SASL}user@REALM + saslauthd) is rejected — see "Why" below. It is a fleet-wide mail outage, not a nb change.
Realm: CLOONAR.COM.
KDC exposure: port 88 only. The Hetzner firewall restricts the source to fw; nb reaches the KDC over the existing WireGuard tunnel. A dedicated tunnel comes later.
Keycloak, not authentik, at sso.cloonar.com (M3 — not this issue).

Scope — what to implement

hosts/mail/modules/kerberos.nix (new), imported from hosts/mail/configuration.nix.
- services.kerberos_server with a single realm CLOONAR.COM, standalone (its own database — not LDAP-backed).
- security.krb5 client config on mail: realm CLOONAR.COM, kdc = mail.cloonar.com, admin_server = mail.cloonar.com, default_realm = CLOONAR.COM, and a [domain_realm] mapping .cloonar.com → CLOONAR.COM.
- A kadm5.acl granting full rights to */admin@CLOONAR.COM.
Firewall: open 88/tcp and 88/udp in networking.firewall on mail. Do not open 749 (kadmin) or 464 (kpasswd) — kadmin stays local-only via kadmin.local, and remote password change is not wanted yet.
The module must not attempt to create the KDC database. kdb5_util create -s is a one-time human step (see below). Creating it from an activation script would risk clobbering a live database on every nixos-rebuild switch, which on mail runs unattended via bento every 5 minutes.
Verify the KDC state directory is backed up. utils/modules/borgbackup.nix:59-63 backs up /var wholesale with an exclude list. Confirm the state dir the chosen implementation uses (/var/lib/krb5kdc for MIT, /var/lib/heimdal for Heimdal) is not caught by an exclude, and add a comment in the module noting the dependency. Losing the KDC database means recreating every principal by hand.

Explicitly out of scope — do not touch

hosts/mail/modules/openldap.nix — no ACL changes, no {SASL}, no schema changes.
hosts/mail/modules/dovecot.nix and postfix.nix — no change. auth_bind stays no.
services.saslauthd — not used.
hosts/nb/* (that is M2), hosts/web-arm/* (that is M3).
utils/modules/ldap-auth.nix — imported by web-arm; leave it alone.

No new SOPS secrets are needed for M1. The KDC master key lives in a stash file generated on the host; it is host state, not a deploy secret.

Why `{SASL}` was rejected (record this in the ADR)

hosts/mail/modules/dovecot.nix:15,23,28 sets auth_bind = no, pass_attrs = mail=user,userPassword=password, default_pass_scheme = CRYPT. Dovecot binds as cn=vmail,ou=system,…, reads the raw userPassword value, and verifies it itself — it never asks slapd to perform a bind. A {SASL} value only works when slapd does the comparison, so rewriting userPassword would hand Dovecot the literal string {SASL}dominik@CLOONAR.COM to CRYPT-compare. All IMAP and SMTP auth for dc=cloonar,dc=com breaks immediately. Postfix's smtpd_sender_login_maps chain has the same shape.

Underneath that: Kerberos cannot derive principal keys from an existing crypt hash, so a full cutover would additionally require a coordinated password reset for every mail user in the tenant.

The GSSAPI bonuses in #266 (Dovecot/Thunderbird, fleet SSH) are also additive — they need a keytab and an extra auth_mechanisms entry, not {SASL} — so nothing is forfeited by scoping.

ADR — required deliverable

Write docs/adr/0022-<slug>.md (0021 is the highest current). It covers the IdP swap as a whole, not just M1. It must record:

The decision to replace Authelia with Keycloak + a Kerberos KDC, and why Authelia cannot get there (no SPNEGO — authelia#10976 is a stalled draft; no Relying Party role, which upstream states they do not intend to add).
Scoped/additive Kerberos, with the Dovecot auth_bind = no defect above as the reason. This is the most important thing in the ADR.
Keycloak over authentik. Note honestly that the RAM argument in #266 is void — web-arm measured 10 GB available of 15.5 GB, and Authelia itself uses 32 MB, so Keycloak's ~1 GB is not a constraint. The decision now rests solely on packaging: authentik is flake-only (nixpkgs#375509 still open) and would source-build Node + Go + Python on every version bump, on a host with no substituters that bento rebuilds every 5 minutes. Record what that costs: authentik's proxy provider maps onto the existing nginx auth_request setup, so choosing Keycloak buys the entire oauth2-proxy workstream at M5.
Realm name, and the interim KDC exposure model (Hetzner firewall source-restricted to fw; nb via WireGuard; dedicated tunnel later). Note the consequence: off-VPN, nb can still log in from the SSSD credential cache but gets no TGT and therefore no SSO.
The M1–M6 sequence, with M3 as the decision gate — Keycloak runs beside Authelia at sso.cloonar.com and SPNEGO is proven end-to-end before any client is cut over.
Open question, explicitly unresolved: nixpkgs' services.keycloak does not manage realms, clients, or LDAP federation declaratively. Today hosts/web-arm/modules/authelia.nix holds all 8 OIDC clients and every policy in Nix, reviewable in a diff. Click-ops would move the auth layer's configuration out of git and into a database backup. Options are keycloak-config-cli, realm-JSON import, or accepting click-ops. Must be answered before M4; does not block M1–M3.

Follow the existing ADR style in docs/adr/ (prose, "Considered options", "Consequences"). Use CONTEXT.md vocabulary.

Verification

scripts/pre-commit dry-builds mail. Do not run scripts/test-configuration manually.
The agent cannot deploy or reach the fleet. Runtime verification is the human steps below.

Human steps after this PR merges and deploys

Hetzner firewall: allow fw → mail:88 (tcp+udp).
On mail, once: kdb5_util create -s -r CLOONAR.COM (master password → Bitwarden, not sops).
kadmin.local -r CLOONAR.COM:
- addprinc dominik.polakovics — interactive, sets the Kerberos password.
- addprinc -randkey HTTP/sso.cloonar.com
- ktadd -k /root/keycloak.keytab HTTP/sso.cloonar.com — run once only; a second ktadd rotates the key and silently invalidates the first keytab.
base64 -w0 /root/keycloak.keytab → add as keycloak-keytab-b64 in hosts/web-arm/secrets.yaml, then shred -u the file. (Needed for M3, not M1.)
Confirm success: kinit dominik.polakovics@CLOONAR.COM then klist shows a TGT.
Confirm the KDC state dir appears in the next borg archive.

Definition of done

hosts/mail/modules/kerberos.nix exists and is imported; mail dry-builds clean.
docs/adr/0022-*.md written.
No changes to openldap.nix, dovecot.nix, postfix.nix, ldap-auth.nix, or any host other than mail (plus the ADR).
PR opened with Closes #<this issue>.

Part of #266 (M1 of M1–M6). Stand up a standalone Kerberos KDC on `mail` and write the ADR for the IdP swap. This is the first, fully additive step toward real desktop SSO. Nothing consumes the KDC when this lands — it is verifiable standalone with `kinit` and revertible by removing one module. ## Design decisions already made (do not re-litigate) These were resolved in a design session on #266. The ADR records them; the implementation follows them. - **Kerberos is additive, not authoritative.** Principals exist only for the humans who log into `nb`. LDAP `userPassword` stays the authoritative password store for every user and every other service. - **No `{SASL}` password pass-through.** The original proposal in #266 (`userPassword: {SASL}user@REALM` + saslauthd) is rejected — see "Why" below. It is a fleet-wide mail outage, not a `nb` change. - **Realm: `CLOONAR.COM`.** - **KDC exposure:** port 88 only. The Hetzner firewall restricts the source to `fw`; `nb` reaches the KDC over the existing WireGuard tunnel. A dedicated tunnel comes later. - **Keycloak, not authentik**, at `sso.cloonar.com` (M3 — not this issue). ## Scope — what to implement 1. **`hosts/mail/modules/kerberos.nix`** (new), imported from `hosts/mail/configuration.nix`. - `services.kerberos_server` with a single realm `CLOONAR.COM`, standalone (its own database — **not** LDAP-backed). - `security.krb5` client config on `mail`: realm `CLOONAR.COM`, `kdc = mail.cloonar.com`, `admin_server = mail.cloonar.com`, `default_realm = CLOONAR.COM`, and a `[domain_realm]` mapping `.cloonar.com` → `CLOONAR.COM`. - A `kadm5.acl` granting full rights to `*/admin@CLOONAR.COM`. 2. **Firewall:** open **88/tcp and 88/udp** in `networking.firewall` on `mail`. Do **not** open 749 (kadmin) or 464 (kpasswd) — kadmin stays local-only via `kadmin.local`, and remote password change is not wanted yet. 3. **The module must not attempt to create the KDC database.** `kdb5_util create -s` is a one-time human step (see below). Creating it from an activation script would risk clobbering a live database on every `nixos-rebuild switch`, which on `mail` runs unattended via bento every 5 minutes. 4. **Verify the KDC state directory is backed up.** `utils/modules/borgbackup.nix:59-63` backs up `/var` wholesale with an exclude list. Confirm the state dir the chosen implementation uses (`/var/lib/krb5kdc` for MIT, `/var/lib/heimdal` for Heimdal) is not caught by an exclude, and add a comment in the module noting the dependency. Losing the KDC database means recreating every principal by hand. ## Explicitly out of scope — do not touch - `hosts/mail/modules/openldap.nix` — no ACL changes, no `{SASL}`, no schema changes. - `hosts/mail/modules/dovecot.nix` and `postfix.nix` — no change. `auth_bind` stays `no`. - `services.saslauthd` — not used. - `hosts/nb/*` (that is M2), `hosts/web-arm/*` (that is M3). - `utils/modules/ldap-auth.nix` — imported by `web-arm`; leave it alone. **No new SOPS secrets are needed for M1.** The KDC master key lives in a stash file generated on the host; it is host state, not a deploy secret. ## Why `{SASL}` was rejected (record this in the ADR) `hosts/mail/modules/dovecot.nix:15,23,28` sets `auth_bind = no`, `pass_attrs = mail=user,userPassword=password`, `default_pass_scheme = CRYPT`. Dovecot binds as `cn=vmail,ou=system,…`, **reads the raw `userPassword` value, and verifies it itself** — it never asks slapd to perform a bind. A `{SASL}` value only works when *slapd* does the comparison, so rewriting `userPassword` would hand Dovecot the literal string `{SASL}dominik@CLOONAR.COM` to CRYPT-compare. **All IMAP and SMTP auth for `dc=cloonar,dc=com` breaks immediately.** Postfix's `smtpd_sender_login_maps` chain has the same shape. Underneath that: Kerberos cannot derive principal keys from an existing crypt hash, so a full cutover would additionally require a coordinated password reset for every mail user in the tenant. The GSSAPI bonuses in #266 (Dovecot/Thunderbird, fleet SSH) are **also additive** — they need a keytab and an extra `auth_mechanisms` entry, not `{SASL}` — so nothing is forfeited by scoping. ## ADR — required deliverable Write `docs/adr/0022-<slug>.md` (0021 is the highest current). It covers the IdP swap as a whole, not just M1. It must record: - The decision to replace Authelia with Keycloak + a Kerberos KDC, and why Authelia cannot get there (no SPNEGO — [authelia#10976](https://github.com/authelia/authelia/pull/10976) is a stalled draft; no Relying Party role, which upstream states they do not intend to add). - **Scoped/additive Kerberos**, with the Dovecot `auth_bind = no` defect above as the reason. This is the most important thing in the ADR. - **Keycloak over authentik.** Note honestly that the RAM argument in #266 is void — `web-arm` measured **10 GB available of 15.5 GB**, and Authelia itself uses 32 MB, so Keycloak's ~1 GB is not a constraint. The decision now rests solely on packaging: authentik is flake-only ([nixpkgs#375509](https://github.com/NixOS/nixpkgs/pull/375509) still open) and would source-build Node + Go + Python on every version bump, on a host with no substituters that bento rebuilds every 5 minutes. Record what that costs: authentik's proxy provider maps onto the existing nginx `auth_request` setup, so choosing Keycloak buys the entire oauth2-proxy workstream at M5. - Realm name, and the interim KDC exposure model (Hetzner firewall source-restricted to `fw`; `nb` via WireGuard; dedicated tunnel later). Note the consequence: off-VPN, `nb` can still log in from the SSSD credential cache but gets no TGT and therefore no SSO. - The M1–M6 sequence, with **M3 as the decision gate** — Keycloak runs beside Authelia at `sso.cloonar.com` and SPNEGO is proven end-to-end before any client is cut over. - **Open question, explicitly unresolved:** nixpkgs' `services.keycloak` does not manage realms, clients, or LDAP federation declaratively. Today `hosts/web-arm/modules/authelia.nix` holds all 8 OIDC clients and every policy in Nix, reviewable in a diff. Click-ops would move the auth layer's configuration out of git and into a database backup. Options are `keycloak-config-cli`, realm-JSON import, or accepting click-ops. Must be answered before M4; does not block M1–M3. Follow the existing ADR style in `docs/adr/` (prose, "Considered options", "Consequences"). Use `CONTEXT.md` vocabulary. ## Verification - `scripts/pre-commit` dry-builds `mail`. Do not run `scripts/test-configuration` manually. - The agent cannot deploy or reach the fleet. Runtime verification is the human steps below. ## Human steps after this PR merges and deploys 1. Hetzner firewall: allow `fw` → `mail:88` (tcp+udp). 2. On `mail`, once: `kdb5_util create -s -r CLOONAR.COM` (master password → Bitwarden, not sops). 3. `kadmin.local -r CLOONAR.COM`: - `addprinc dominik.polakovics` — interactive, sets the Kerberos password. - `addprinc -randkey HTTP/sso.cloonar.com` - `ktadd -k /root/keycloak.keytab HTTP/sso.cloonar.com` — **run once only**; a second `ktadd` rotates the key and silently invalidates the first keytab. 4. `base64 -w0 /root/keycloak.keytab` → add as `keycloak-keytab-b64` in `hosts/web-arm/secrets.yaml`, then `shred -u` the file. (Needed for M3, not M1.) 5. Confirm success: `kinit dominik.polakovics@CLOONAR.COM` then `klist` shows a TGT. 6. Confirm the KDC state dir appears in the next borg archive. ## Definition of done - `hosts/mail/modules/kerberos.nix` exists and is imported; `mail` dry-builds clean. - `docs/adr/0022-*.md` written. - No changes to `openldap.nix`, `dovecot.nix`, `postfix.nix`, `ldap-auth.nix`, or any host other than `mail` (plus the ADR). - PR opened with `Closes #<this issue>`.

dominik.polakovics added the

ready-for-agent

label

2026-08-07 11:01:46 +02:00

dominik.polakovics referenced this issue

2026-08-07 11:02:10 +02:00

Real desktop SSO: replace Authelia with Keycloak + a Kerberos KDC #266

dominik.polakovics referenced this issue from a pull request that will close it,

2026-08-07 11:28:53 +02:00

feat(mail): standalone Kerberos KDC for realm CLOONAR.COM + IdP-swap ADR #268

dominik.polakovics referenced this issue

2026-08-07 11:35:38 +02:00