Kerberos becomes the authoritative password store for the identity tenant #299

New issue

Open

opened 2026-08-08 13:29:30 +02:00 by dominik.polakovics · 0 comments

dominik.polakovics commented

2026-08-08 13:29:30 +02:00

Owner

Epic. Design recorded in ADR-0023 (docs/adr/0023-kerberos-authoritative-for-the-identity-tenant.md).

Reverses ADR-0022 decision 3 for dc=cloonar,dc=com humans. The two-store debt that ADR booked is already live: passwd on nb writes Kerberos (hosts/nb/modules/ldap-login.nix:30), passwd on a server writes LDAP (utils/modules/ldap-auth.nix:35). Which host you are on decides which store moves.

The model

Partition, not synchronisation. The identity tenant is Kerberos-only with userPassword locked to {CRYPT}!. Mail-only tenants keep userPassword, get no principal, and are not federated into Keycloak. No identity holds two credentials, so drift is impossible by construction — not prevented by a hook that has to keep working.

MIT krb5 stays. No Heimdal, no smbk5pwd, no LDAP-backed KDB, and therefore no keytab invalidated: Keycloak SPNEGO and the two Dovecot SPNs are untouched by all of this.

Milestones

The execution order is no longer the numeric sequence — M1 (#301) changed it. See Order below.

M0 — prerequisites (#302). KDC backup via kdb5_util dump and a restore actually performed; break-glass local accounts on mail and web-arm. Both stop being optional the moment {CRYPT}! removes the fallback credential.
M1 — verification spikes (#301, done). All three answered; two came back against expectation. Findings below.
M2 — KDC config. max_life = 10h, max_renewable_life = 7d; helpdesk/admin principal with c only. kpasswd (464) on mail is done (#305). Still open: permit web-arm → mail on 88 and 464 including UDP and IPv6 (see findings), and fix the stale port-88 comment at hosts/mail/modules/kerberos.nix:5.
M3 — Dovecot Kerberos passdb, ahead of the existing LDAP passdbs (hosts/mail/modules/dovecot.nix:171-186). A checkpassword script, not driver = pam — spike 1 settled this. Additive: result_failure defaults to continue, so every other tenant falls through unchanged. GSSAPI regression check is mandatory — it is the thing most likely to be noticed if it breaks, and ticket login is already in production use.
M5 — Keycloak Required Action for web password change: own form, kpasswd as the user, reached via kc_action=. No privileged credential, port 749 stays closed. Must land before M4. Must surface a failed kpasswd explicitly rather than trusting a clean return — #305 showed that failure mode is otherwise silent.
M6a — enrolment. Create principals; each user sets their own Kerberos password through M5's Required Action, authenticating with the LDAP password they still hold. userPassword stays live. Also sets -maxlife / -maxrenewlife per principal (see Deferred).
M4 — Keycloak and servers onto the KDC. useKerberosForPasswordAuthentication (cloonar-realm.json:727), and utils/modules/ldap-auth.nix:35 from ldap to krb5. These move together or passwd on a server silently writes a store nothing reads. Only safe once M6a is complete for everyone.
M6b — lock userPassword to {CRYPT}!.

Order

M0 → M2 → M3 → M5 → M6a → M4 → M6b

M5 moves ahead of M4, and M6 splits, because useKerberosForPasswordAuthentication is a realm-level flag while enrolment is per-user. No ordering of M4 and M6 as originally written works: M4 first locks out everyone not yet enrolled; M6 first locks out each user at the moment they are enrolled, because Keycloak is still binding to LDAP when their userPassword is locked.

M4, M5 and M6 are blocked by #308, which decides how the transition works and how ADR-0023 decisions 4 and 6 are amended. Those issues should not be filed until it lands.

M1 findings (#301)

Dovecot hands PAM the full address (dominik.polakovics@cloonar.com), so pam_krb5 cannot form a valid principal. M3 is a checkpassword script. It must return user unknown — not auth failed — for non-identity-tenant addresses, and must verify against a keytab or accept KDC-spoofing exposure as a stated decision.
useKerberosForPasswordAuthentication has no LDAP fallback. Verified live on the cloonar realm. This is what forces the reordering above, and #308.
SDDM renders the expired-password prompt. The greeter-replacement contingency is dead. Two defects were fixed getting there: kpasswd was firewalled and the change failed silently (#305), and the PAM password stack prompted twice (#306).
web-arm had never been able to reach the KDC. keycloak/default.nix:153-156 states the assumption — "Accepting a ticket needs no KDC contact" — which is true for SPNEGO and false for M4. The rule permitting it lives in the Hetzner console, outside git, and that comment is now misleading.
The #97 dual-stack trap bit a second consumer. mail.cloonar.com is dual-stack (ADR-0016) and web-arm has v6 egress (ADR-0010), so RFC 6724 makes clients prefer v6. The JVM's Kerberos client has no v6→v4 fallback and hung, where kinit fell back and succeeded. Rules are currently TCP-only; Kerberos tries UDP first and falls back to TCP for oversized responses.

Deferred, deliberately

KDC replica (kprop/iprop). Accepted single point of failure for now; the consequence is spelled out in the ADR. Purely additive later.
Ticket life. Maintainer decision, 2026-08-08: leave as is for now. Principals carry Maximum ticket life: 1 day and Maximum renewable life: 0, against ADR-0023 decision 9's 10h / 7d — and that ADR is explicit that max_life is a security setting, being the only bound on an already-issued service ticket after -allow_tix. kdc.conf realm defaults do not apply retroactively, so M6a must set these per principal.
One command for enrol/disable. Note it cannot revoke tickets — nothing can. modprinc -allow_tix stops new TGTs and new service tickets; already-issued service tickets are validated against the service keytab without touching the KDC, so max_life is the only bound. Web and mail revoke properly at their own layers.
Self-service reset. Rejected rather than deferred: identity-tenant mailboxes are on this Dovecot, so a mailed token is protected by the password being reset. (#308 notes this objection is weaker during the migration window, when mail is still protected by a different credential.)

Epic. Design recorded in ADR-0023 (`docs/adr/0023-kerberos-authoritative-for-the-identity-tenant.md`). Reverses ADR-0022 decision 3 for `dc=cloonar,dc=com` humans. The two-store debt that ADR booked is already live: `passwd` on `nb` writes Kerberos (`hosts/nb/modules/ldap-login.nix:30`), `passwd` on a server writes LDAP (`utils/modules/ldap-auth.nix:35`). Which host you are on decides which store moves. ## The model **Partition, not synchronisation.** The identity tenant is Kerberos-only with `userPassword` locked to `{CRYPT}!`. Mail-only tenants keep `userPassword`, get no principal, and are not federated into Keycloak. No identity holds two credentials, so drift is impossible by construction — not prevented by a hook that has to keep working. MIT krb5 stays. No Heimdal, no `smbk5pwd`, no LDAP-backed KDB, and therefore no keytab invalidated: Keycloak SPNEGO and the two Dovecot SPNs are untouched by all of this. ## Milestones **The execution order is no longer the numeric sequence** — M1 (#301) changed it. See Order below. - **M0 — prerequisites** (#302). KDC backup via `kdb5_util dump` and a restore actually performed; break-glass local accounts on `mail` and `web-arm`. Both stop being optional the moment `{CRYPT}!` removes the fallback credential. - **M1 — verification spikes** (#301, **done**). All three answered; two came back against expectation. Findings below. - **M2 — KDC config.** `max_life = 10h`, `max_renewable_life = 7d`; `helpdesk/admin` principal with `c` only. kpasswd (464) on `mail` is **done** (#305). Still open: permit `web-arm` → `mail` on 88 and 464 including **UDP and IPv6** (see findings), and fix the stale port-88 comment at `hosts/mail/modules/kerberos.nix:5`. - **M3 — Dovecot Kerberos passdb**, ahead of the existing LDAP passdbs (`hosts/mail/modules/dovecot.nix:171-186`). **A `checkpassword` script, not `driver = pam`** — spike 1 settled this. Additive: `result_failure` defaults to `continue`, so every other tenant falls through unchanged. GSSAPI regression check is mandatory — it is the thing most likely to be noticed if it breaks, and ticket login is already in production use. - **M5 — Keycloak Required Action** for web password change: own form, kpasswd as the user, reached via `kc_action=`. No privileged credential, port 749 stays closed. **Must land before M4.** Must surface a failed kpasswd explicitly rather than trusting a clean return — #305 showed that failure mode is otherwise silent. - **M6a — enrolment.** Create principals; each user sets their own Kerberos password through M5's Required Action, authenticating with the LDAP password they still hold. `userPassword` stays live. Also sets `-maxlife` / `-maxrenewlife` per principal (see Deferred). - **M4 — Keycloak and servers onto the KDC.** `useKerberosForPasswordAuthentication` (`cloonar-realm.json:727`), and `utils/modules/ldap-auth.nix:35` from `ldap` to `krb5`. These move together or `passwd` on a server silently writes a store nothing reads. Only safe once M6a is complete for **everyone**. - **M6b — lock `userPassword`** to `{CRYPT}!`. ### Order ``` M0 → M2 → M3 → M5 → M6a → M4 → M6b ``` M5 moves ahead of M4, and M6 splits, because `useKerberosForPasswordAuthentication` is a **realm-level** flag while enrolment is **per-user**. No ordering of M4 and M6 as originally written works: M4 first locks out everyone not yet enrolled; M6 first locks out each user at the moment they are enrolled, because Keycloak is still binding to LDAP when their `userPassword` is locked. **M4, M5 and M6 are blocked by #308**, which decides how the transition works and how ADR-0023 decisions 4 and 6 are amended. Those issues should not be filed until it lands. ## M1 findings (#301) - **Dovecot hands PAM the full address** (`dominik.polakovics@cloonar.com`), so pam_krb5 cannot form a valid principal. M3 is a `checkpassword` script. It must return *user unknown* — not auth failed — for non-identity-tenant addresses, and must verify against a keytab or accept KDC-spoofing exposure as a stated decision. - **`useKerberosForPasswordAuthentication` has no LDAP fallback.** Verified live on the `cloonar` realm. This is what forces the reordering above, and #308. - **SDDM renders the expired-password prompt.** The greeter-replacement contingency is dead. Two defects were fixed getting there: kpasswd was firewalled and the change failed *silently* (#305), and the PAM password stack prompted twice (#306). - **`web-arm` had never been able to reach the KDC.** `keycloak/default.nix:153-156` states the assumption — *"Accepting a ticket needs no KDC contact"* — which is true for SPNEGO and false for M4. The rule permitting it lives in the Hetzner console, outside git, and that comment is now misleading. - **The #97 dual-stack trap bit a second consumer.** `mail.cloonar.com` is dual-stack (ADR-0016) and `web-arm` has v6 egress (ADR-0010), so RFC 6724 makes clients prefer v6. The JVM's Kerberos client has no v6→v4 fallback and hung, where `kinit` fell back and succeeded. Rules are currently **TCP-only**; Kerberos tries UDP first and falls back to TCP for oversized responses. ## Deferred, deliberately - **KDC replica** (`kprop`/`iprop`). Accepted single point of failure for now; the consequence is spelled out in the ADR. Purely additive later. - **Ticket life.** Maintainer decision, 2026-08-08: leave as is for now. Principals carry `Maximum ticket life: 1 day` and `Maximum renewable life: 0`, against ADR-0023 decision 9's `10h` / `7d` — and that ADR is explicit that `max_life` is a security setting, being the only bound on an already-issued service ticket after `-allow_tix`. `kdc.conf` realm defaults do **not** apply retroactively, so M6a must set these per principal. - **One command for enrol/disable.** Note it cannot revoke tickets — nothing can. `modprinc -allow_tix` stops new TGTs and new service tickets; already-issued service tickets are validated against the service keytab without touching the KDC, so `max_life` is the only bound. Web and mail revoke properly at their own layers. - **Self-service reset.** Rejected rather than deferred: identity-tenant mailboxes are on this Dovecot, so a mailed token is protected by the password being reset. (#308 notes this objection is weaker *during* the migration window, when mail is still protected by a different credential.)

dominik.polakovics added the

enhancement

label

2026-08-08 13:29:30 +02:00

dominik.polakovics referenced this issue

2026-08-08 13:29:41 +02:00

docs(adr-0023): record making Kerberos authoritative for the identity tenant #300

dominik.polakovics referenced this issue

2026-08-08 13:29:58 +02:00

Three verification spikes before the Kerberos-authoritative build (M1) #301

dominik.polakovics referenced this issue

2026-08-08 13:30:16 +02:00