Cloonar/nixos

Fork 0

Keycloak realm: converge with keycloak-config-cli, drop realmFiles, add the LDAP group mapper #277

New issue

Closed

opened 2026-08-07 17:03:10 +02:00 by dominik.polakovics · 0 comments

dominik.polakovics commented

2026-08-07 17:03:10 +02:00

Owner

Part of #266. Follows #271, which passed the M3 gate: Keycloak serves sso.cloonar.com, SPNEGO works from a nb desktop session, and LDAP password login works as the fallback.

Replace services.keycloak.realmFiles with a converging import via keycloak-config-cli, and prove it by landing the LDAP group mapper — the first realm change that realmFiles structurally cannot apply.

Why

realmFiles is create-if-absent: Keycloak skips the import entirely once the realm exists. The cloonar realm exists, so every further edit to cloonar-realm.json is a silent no-op. Nothing else in M4–M6 can ship until this changes.

pkgs.keycloak-config-cli (6.5.1, Maven build) talks to Keycloak's Admin REST API, computes the delta between the files and the live realm, applies only what differs, and needs no restart. ADR-0022 deferred this decision to M4; it is now due.

The hazard — read before writing any code

import.managed.* defaults to full, meaning the first run deletes anything present in the realm but absent from the YAML. The realm currently holds the LDAP provider, six user-attribute-ldap-mappers, and the custom browser-spnego flow that the M3 gate depends on. An incomplete port silently strips SPNEGO, and the symptom looks like a Keycloak bug rather than a missing stanza.

Therefore land this as two commits:

Verbatim port. Translate cloonar-realm.json into config-cli's format with no intended changes, switch the mechanism over, and confirm the converge run is a no-op — realm unchanged, SPNEGO still working.
The group mapper, as the first deliberate change, proving convergence actually applies edits.

Do not loosen import.managed.* to no-delete to dodge this. Half-managed config is worse than either extreme, and full management is the point.

Scope

1. Converging import

pkgs.keycloak-config-cli as a Type = "oneshot" unit, after/requires keycloak.service, wantedBy = multi-user.target.
Enable the availability check (--keycloak.availability-check.enabled). keycloak.service reaching active does not mean the Admin API accepts requests; without this the unit races Keycloak's startup and fails intermittently.
Point --import.files.locations at a store path holding the realm files. Because the store path changes when the files change, switch-to-configuration restarts the unit exactly when the config changes and stays quiet otherwise — no timer, no polling, and it composes correctly with bento's five-minute loop.
Authenticate against the master realm; the realm being managed is cloonar.

2. Secrets — none to add

Admin credentials already exist as keycloak-bootstrap-admin, but under KC_BOOTSTRAP_ADMIN_USERNAME/KC_BOOTSTRAP_ADMIN_PASSWORD, while config-cli wants KEYCLOAK_USER/KEYCLOAK_PASSWORD. Translate in-module — render an env file in an ExecStartPre, the way materialiseSecrets in hosts/web-arm/modules/keycloak/default.nix already does. Do not add a second sops key holding a duplicate password.
The LDAP bind password is not needed by config-cli at all. Keep M3's vault SPI: the YAML carries the literal ${vault.ldapbind}, Keycloak resolves it at runtime, and the credential never enters config-cli's environment or Keycloak's database.
Consequently keep import.var-substitution disabled. With substitution on, ${vault.ldapbind} risks being interpreted as a substitution token and mangled — which would break the LDAP bind in a way that looks like a wrong password.
Do not create a third copy of linuxbind-password. It already backs SSSD on nb and web-arm plus the SSH AuthorizedKeysCommand; a copy that drifts on rotation would take SSH access with it.

A dedicated Keycloak service account with only realm-management roles is the better end state than the master admin, but it is a chicken-and-egg to bootstrap. Note it as future hardening; use the admin user now.

3. Drop `realmFiles`

Remove services.keycloak.realmFiles and let config-cli own the realm end to end, including creating it if absent. Two mechanisms writing the same realm — one create-only at startup, one converging over the API — is a confusing split with no upside.

4. The group mapper (second commit)

A group-ldap-mapper on the cloonar-ldap provider:

groups.dn = ou=groups,dc=cloonar,dc=com
group.object.classes = groupOfNames, group.name.ldap.attribute = cn
membership.ldap.attribute = member, membership.attribute.type = DN

This is the web-services group tree, and it is deliberately not the one SSSD uses. The directory carries two group schemas on purpose: cn=linux,ou=groups holds POSIX groups with memberUid (rfc2307) for Unix logins, while ou=groups holds groupOfNames with member for web authorization. hosts/web-arm/modules/authelia.nix:77 is the working precedent for the latter. The objectClass filter keeps the POSIX cn=linux group out of Keycloak's view.

Set preserve.group.inheritance = false unless the group tree is genuinely hierarchical — with flat groups under ou=groups, leaving it true makes the sync fail.

Expected to appear: Administrators, Mitarbeiter, macher.solutions, immich (the groups authelia.nix policies reference). Choose mode (LDAP_ONLY vs READ_ONLY) deliberately and say why in a comment — the federation is READ_ONLY.

5. Correct a stale comment

hosts/nb/modules/ldap-login.nix:42-46 describes the two group schemas as an unresolved contradiction with authelia.nix:77. They are deliberately different bases for different consumers, as above. Fix the comment; the SSSD config itself is correct and must not change.

6. ADR

Amend docs/adr/0022-keycloak-and-scoped-kerberos.md: the declarative-configuration question is now answered. Record keycloak-config-cli, that realmFiles was create-only and why that made it insufficient, the managed = full consequence (the admin console becomes effectively read-only — UI changes get reverted on the next converge), and that the Maven build is a cost accepted on a substituter-less arm64 host.

Out of scope

OIDC clients and the authorization model. Authelia's admin-only / internal / immich policies have no direct Keycloak equivalent and need a per-client authentication-flow design. Separate issue.
Cutting any application over. Authelia keeps serving everything.
hosts/mail/*; anything on nb beyond the comment fix.

Verification

scripts/pre-commit dry-builds web-arm. Do not run scripts/test-configuration manually.

Human, after deploy:

systemctl status keycloak-config — succeeded, and the journal shows the first (verbatim) run applying no changes.
SPNEGO still works: from the dominik.polakovics session on nb with a TGT, https://sso.cloonar.com/realms/cloonar/account loads with no prompt. This is the regression that matters.
Negative control still works: no TGT → login form → LDAP password accepted.
Admin console → Groups: Administrators, Mitarbeiter, macher.solutions, immich present, with the expected members. cn=linux absent.
Edit something trivial in the realm YAML, rebuild, and confirm the change actually lands — proving convergence, which is the entire point.

Also worth doing while in the console, though not part of this issue: check the user list for linuxbind, vmail, authelia, owncloud, netdata. searchScope = 2 with usersDn = ou=users encloses ou=system,ou=users, so service accounts may be exposed as login-able users. That must be resolved before clients are wired up.

Definition of done

realmFiles gone; realm managed by keycloak-config-cli.
Two commits, the first provably a no-op.
Group mapper landed and groups visible.
No new sops secrets. No change to authelia.nix or any existing vhost.
ADR-0022 amended; ldap-login.nix comment corrected.
PR opened with Closes #<this issue>.

Part of #266. Follows #271, which passed the M3 gate: Keycloak serves `sso.cloonar.com`, SPNEGO works from a `nb` desktop session, and LDAP password login works as the fallback. Replace `services.keycloak.realmFiles` with a converging import via `keycloak-config-cli`, and prove it by landing the LDAP group mapper — the first realm change that `realmFiles` structurally cannot apply. ## Why `realmFiles` is **create-if-absent**: Keycloak skips the import entirely once the realm exists. The `cloonar` realm exists, so every further edit to `cloonar-realm.json` is a silent no-op. Nothing else in M4–M6 can ship until this changes. `pkgs.keycloak-config-cli` (6.5.1, Maven build) talks to Keycloak's Admin REST API, computes the delta between the files and the live realm, applies only what differs, and needs no restart. ADR-0022 deferred this decision to M4; it is now due. ## The hazard — read before writing any code `import.managed.*` defaults to `full`, meaning **the first run deletes anything present in the realm but absent from the YAML.** The realm currently holds the LDAP provider, six `user-attribute-ldap-mapper`s, and the custom `browser-spnego` flow that the M3 gate depends on. An incomplete port silently strips SPNEGO, and the symptom looks like a Keycloak bug rather than a missing stanza. **Therefore land this as two commits:** 1. **Verbatim port.** Translate `cloonar-realm.json` into config-cli's format with *no intended changes*, switch the mechanism over, and confirm the converge run is a no-op — realm unchanged, SPNEGO still working. 2. **The group mapper**, as the first deliberate change, proving convergence actually applies edits. Do not loosen `import.managed.*` to `no-delete` to dodge this. Half-managed config is worse than either extreme, and full management is the point. ## Scope ### 1. Converging import - `pkgs.keycloak-config-cli` as a `Type = "oneshot"` unit, `after`/`requires` `keycloak.service`, `wantedBy = multi-user.target`. - **Enable the availability check** (`--keycloak.availability-check.enabled`). `keycloak.service` reaching `active` does not mean the Admin API accepts requests; without this the unit races Keycloak's startup and fails intermittently. - Point `--import.files.locations` at a store path holding the realm files. Because the store path changes when the files change, `switch-to-configuration` restarts the unit exactly when the config changes and stays quiet otherwise — no timer, no polling, and it composes correctly with bento's five-minute loop. - Authenticate against the `master` realm; the realm being *managed* is `cloonar`. ### 2. Secrets — none to add - **Admin credentials** already exist as `keycloak-bootstrap-admin`, but under `KC_BOOTSTRAP_ADMIN_USERNAME`/`KC_BOOTSTRAP_ADMIN_PASSWORD`, while config-cli wants `KEYCLOAK_USER`/`KEYCLOAK_PASSWORD`. Translate in-module — render an env file in an `ExecStartPre`, the way `materialiseSecrets` in `hosts/web-arm/modules/keycloak/default.nix` already does. **Do not add a second sops key holding a duplicate password.** - **The LDAP bind password is not needed by config-cli at all.** Keep M3's vault SPI: the YAML carries the literal `${vault.ldapbind}`, Keycloak resolves it at runtime, and the credential never enters config-cli's environment or Keycloak's database. - Consequently **keep `import.var-substitution` disabled.** With substitution on, `${vault.ldapbind}` risks being interpreted as a substitution token and mangled — which would break the LDAP bind in a way that looks like a wrong password. - Do not create a third copy of `linuxbind-password`. It already backs SSSD on `nb` and `web-arm` plus the SSH `AuthorizedKeysCommand`; a copy that drifts on rotation would take SSH access with it. A dedicated Keycloak service account with only `realm-management` roles is the better end state than the master admin, but it is a chicken-and-egg to bootstrap. Note it as future hardening; use the admin user now. ### 3. Drop `realmFiles` Remove `services.keycloak.realmFiles` and let config-cli own the realm end to end, including creating it if absent. Two mechanisms writing the same realm — one create-only at startup, one converging over the API — is a confusing split with no upside. ### 4. The group mapper (second commit) A `group-ldap-mapper` on the `cloonar-ldap` provider: - `groups.dn = ou=groups,dc=cloonar,dc=com` - `group.object.classes = groupOfNames`, `group.name.ldap.attribute = cn` - `membership.ldap.attribute = member`, `membership.attribute.type = DN` This is the **web-services** group tree, and it is deliberately *not* the one SSSD uses. The directory carries two group schemas on purpose: `cn=linux,ou=groups` holds POSIX groups with `memberUid` (rfc2307) for Unix logins, while `ou=groups` holds `groupOfNames` with `member` for web authorization. `hosts/web-arm/modules/authelia.nix:77` is the working precedent for the latter. The `objectClass` filter keeps the POSIX `cn=linux` group out of Keycloak's view. Set `preserve.group.inheritance = false` unless the group tree is genuinely hierarchical — with flat groups under `ou=groups`, leaving it true makes the sync fail. Expected to appear: `Administrators`, `Mitarbeiter`, `macher.solutions`, `immich` (the groups `authelia.nix` policies reference). Choose `mode` (`LDAP_ONLY` vs `READ_ONLY`) deliberately and say why in a comment — the federation is READ_ONLY. ### 5. Correct a stale comment `hosts/nb/modules/ldap-login.nix:42-46` describes the two group schemas as an unresolved contradiction with `authelia.nix:77`. They are deliberately different bases for different consumers, as above. Fix the comment; the SSSD config itself is correct and must not change. ### 6. ADR Amend `docs/adr/0022-keycloak-and-scoped-kerberos.md`: the declarative-configuration question is now answered. Record `keycloak-config-cli`, that `realmFiles` was create-only and why that made it insufficient, the `managed = full` consequence (**the admin console becomes effectively read-only — UI changes get reverted on the next converge**), and that the Maven build is a cost accepted on a substituter-less arm64 host. ## Out of scope - **OIDC clients and the authorization model.** Authelia's `admin-only` / `internal` / `immich` policies have no direct Keycloak equivalent and need a per-client authentication-flow design. Separate issue. - Cutting any application over. Authelia keeps serving everything. - `hosts/mail/*`; anything on `nb` beyond the comment fix. ## Verification - `scripts/pre-commit` dry-builds `web-arm`. Do not run `scripts/test-configuration` manually. Human, after deploy: 1. `systemctl status keycloak-config` — succeeded, and the journal shows the first (verbatim) run applying **no changes**. 2. **SPNEGO still works**: from the `dominik.polakovics` session on `nb` with a TGT, `https://sso.cloonar.com/realms/cloonar/account` loads with no prompt. This is the regression that matters. 3. Negative control still works: no TGT → login form → LDAP password accepted. 4. Admin console → Groups: `Administrators`, `Mitarbeiter`, `macher.solutions`, `immich` present, with the expected members. `cn=linux` absent. 5. Edit something trivial in the realm YAML, rebuild, and confirm the change actually lands — proving convergence, which is the entire point. Also worth doing while in the console, though not part of this issue: check the **user list** for `linuxbind`, `vmail`, `authelia`, `owncloud`, `netdata`. `searchScope = 2` with `usersDn = ou=users` encloses `ou=system,ou=users`, so service accounts may be exposed as login-able users. That must be resolved before clients are wired up. ## Definition of done - `realmFiles` gone; realm managed by `keycloak-config-cli`. - Two commits, the first provably a no-op. - Group mapper landed and groups visible. - No new sops secrets. No change to `authelia.nix` or any existing vhost. - ADR-0022 amended; `ldap-login.nix` comment corrected. - PR opened with `Closes #<this issue>`.

dominik.polakovics added the

ready-for-agent

label

2026-08-07 17:03:10 +02:00

dominik.polakovics referenced this issue from a pull request that will close it,

2026-08-07 17:57:46 +02:00

feat(web-arm): converge the Keycloak realm with keycloak-config-cli, add the LDAP group mapper #278

dominik.polakovics closed this issue

2026-08-07 18:05:07 +02:00

dominik.polakovics referenced this issue

2026-08-07 18:18:53 +02:00

M4a: define the six OIDC clients and the authorization model in Keycloak (no cutover) #279