Cloonar/nixos
3
0
Fork 0
Code Issues 31 Pull requests 7 Projects Releases Packages Wiki Activity Actions

feat(web-arm): install paperless-ngx #386

Open
dominik.polakovics wants to merge 1 commit from afk/385 into main
pull from: afk/385
merge into: Cloonar:main
Conversation 0 Commits 1 Files changed 4 +294 -1
dominik.polakovics commented 2026-08-23 13:10:12 +02:00
Owner
Copy link

Deploys paperless-ngx on web-arm as a native NixOS service at https://paperless.cloonar.com,
gated behind the existing Keycloak SSO, with its media tree on a dedicated Hetzner Storage-Box
CIFS share and a nightly restore-anywhere export feeding the existing borg job.

Closes #385

What lands

  • hosts/web-arm/modules/paperless/default.nix (new) — services.paperless on the existing
    shared PostgreSQL 14 instance, a dedicated Redis on a unix socket, OCR in deu+eng, one task
    worker, the CIFS media mount, the nginx vhost, the nightly exporter, the borg exclusion and the
    CPU throttling.
  • hosts/web-arm/configuration.nix — imports the new module.
  • hosts/web-arm/modules/keycloak/{default.nix,cloonar-realm.json} — a confidential
    paperless client, a /paperless group carrying a policy-paperless realm role, and a
    role-gated browser flow mirroring the browser-spnego immich family, plus the client-secret
    vault plumbing (plainClients + keycloak-client-paperless).

Design notes

Secret handling. The OIDC client secret never enters the Nix store. A sops template renders
PAPERLESS_SOCIALACCOUNT_PROVIDERS into a systemd EnvironmentFile; rotation restarts the four
paperless services via restartUnits, and edits to the template body are picked up through a
restartTriggers hash of the placeholder-tokenised content — the same construct the Immich module
uses. The JSON value is single-quoted deliberately: systemd's EnvironmentFile parser strips
unescaped double quotes mid-value, which would shred the JSON, and paperless-manage sources the
same file from bash.

The CIFS mount is persistent, on purpose. nofail, x-systemd.mount-timeout=2min, ownership
forced to paperless via uid=/gid=, and RequiresMountsFor on every paperless unit that
touches media — upstream only sets it on paperless-scheduler, so the task queue, consumer, web
and exporter units get it here. There is deliberately no x-systemd.automount /
x-systemd.idle-timeout anywhere: the idle unmount propagates a clean stop through
RequiresMountsFor, so Restart=on-failure never fires — that is exactly what took Immich down
for 8–24h at a time.

tmpfiles modes are pinned. Upstream declares tmpfiles rules for the data, media and consume
directories with no explicit mode, i.e. 0755. Two consequences fixed here: the media rule is
pinned to 0770 so it matches what dir_mode=0770 makes the mounted share report — otherwise every
bento rebuild re-runs tmpfiles against a live CIFS mount and the chmod fails; and
/var/lib/paperless is pinned to 0750 so the consume directory and the nightly full-document
export are not world-readable on a box that also hosts SFTP website users.

Backups. The media tree is excluded from borg (it lives on the same Storage Box as the borg
repo; Hetzner snapshots cover it). The restore-anywhere copy is the nightly document_exporter
run into /var/lib/paperless/export, which the job's existing /var path already covers. It runs
at 01:30, ahead of the 03:00 borg run, so each night's export ships the same night; the exporter
unit Conflicts the paperless services, so paperless is briefly down while it runs. The database
joins the existing postgresqlBackup dumps.

Note the exclusion is declared in the paperless module rather than appended to
utils/modules/borgbackup.nix next to the /var/lib/immich/ entry. Immich's precedent points the
other way, but a web-arm-only path in a module shared by eight hosts is the wart, and keeping it
local means the pre-commit hook dry-builds only the host actually affected. Say the word if you'd
rather it sat with the immich line and I'll move it.

Login. OIDC auto-signup is on, so /paperless members get an account on first login — but a
permissionless one until the admin grants access; per-document ownership does the rest. Local
password login stays enabled for exactly one break-glass admin account provisioned from
passwordFile, so a Keycloak outage does not lock the admin out.

Before this can deploy — maintainer actions

Secrets to add to hosts/web-arm/secrets.sops.yaml

nix-shell -p sops --run 'sops hosts/web-arm/secrets.sops.yaml'

Key Contents
paperless-oidc-client-secret The paperless client secret from Keycloak. Read twice: by the paperless module (into the rendered env file) and by the Keycloak module as keycloak-client-paperless (into the file vault as cloonar_paperlesssecret).
paperless-smb-credentials CIFS credentials for the new Storage-Box sub-account, in mount.cifs format: a username= line and a password= line.
paperless-admin-password Password for the local break-glass admin account.

Until these exist the host still evaluates, but activation fails — sops resolves secrets at
deploy time, not at eval time.

External actions

  1. Hetzner Storage Box — create sub-account u149513-sub13 (the module mounts
    //u149513.your-backup.de/u149513-sub13/), enable Samba/CIFS on it, and turn on the snapshot
    schedule. sub13 was picked as the next unreferenced number in this repo; if it is already
    taken on the Hetzner side, change the device line to the sub-account you create.
  2. DNS — an A record for paperless.cloonar.com pointing at web-arm. The certificate then
    issues itself through the existing lego DNS-01 setup; no repo change needed for that.
  3. Mailbox — create scan@cloonar.com on the mail host for scan-to-email
    (scan+<user>@cloonar.com plus-addressing).
  4. Keycloak — put the intended users in the /paperless group. Nobody can complete a login
    until they hold policy-paperless.
  5. Disk budget — /var/lib/paperless/export is a full copy of every document, on the local
    SSD and inside the borg set. It is the one place the media-on-Storage-Box split does not save
    local space; worth sizing before the archive grows.

Runtime configuration (deliberately not in this PR)

The IMAP account, per-user mail rules and owner-assigning workflows, and per-user permissions are
configured in the paperless UI after first login.

Not included, possibly wanted

ansible/roles/ipa_structure/defaults/main.yml has an svc-immich access group but gained no
svc-paperless here — the issue scoped LDAP/group work as a human action. Happy to add the
three-line entry if it belongs in this PR.

Testing

This repo has no unit tests; the gate is the pre-commit hook's dry-build, which ran on web-arm
and passed. Beyond that I evaluated the resulting config and checked the things that are easy to
get quietly wrong: PAPERLESS_URL resolves to https://paperless.cloonar.com, PAPERLESS_REDIS
resolves to the module's own unix:///run/redis-paperless/redis.sock, PAPERLESS_DBHOST is the
/run/postgresql socket, OCR is deu+eng, PAPERLESS_TASK_WORKERS is 1, tika and gotenberg are
both off, CPUWeight is 20 on the scheduler/task-queue/consumer/exporter and untouched on the web
unit, the four non-scheduler units carry RequiresMountsFor, the mount options contain no
automount/idle-timeout, and the rendered template is single-line single-quoted JSON carrying only
a sops placeholder. The Keycloak realm JSON was checked for alias/name uniqueness and for every
flowAlias, authenticatorConfig, condUserRole and browser-flow override resolving.

Runtime verification after deploy is yours.

Deploys paperless-ngx on web-arm as a native NixOS service at `https://paperless.cloonar.com`, gated behind the existing Keycloak SSO, with its media tree on a dedicated Hetzner Storage-Box CIFS share and a nightly restore-anywhere export feeding the existing borg job. Closes #385 ## What lands - **`hosts/web-arm/modules/paperless/default.nix`** (new) — `services.paperless` on the existing shared PostgreSQL 14 instance, a dedicated Redis on a unix socket, OCR in `deu+eng`, one task worker, the CIFS media mount, the nginx vhost, the nightly exporter, the borg exclusion and the CPU throttling. - **`hosts/web-arm/configuration.nix`** — imports the new module. - **`hosts/web-arm/modules/keycloak/{default.nix,cloonar-realm.json}`** — a confidential `paperless` client, a `/paperless` group carrying a `policy-paperless` realm role, and a role-gated browser flow mirroring the `browser-spnego immich` family, plus the client-secret vault plumbing (`plainClients` + `keycloak-client-paperless`). ## Design notes **Secret handling.** The OIDC client secret never enters the Nix store. A sops template renders `PAPERLESS_SOCIALACCOUNT_PROVIDERS` into a systemd `EnvironmentFile`; rotation restarts the four paperless services via `restartUnits`, and edits to the template body are picked up through a `restartTriggers` hash of the placeholder-tokenised content — the same construct the Immich module uses. The JSON value is single-quoted deliberately: systemd's `EnvironmentFile` parser strips unescaped double quotes mid-value, which would shred the JSON, and `paperless-manage` sources the same file from bash. **The CIFS mount is persistent, on purpose.** `nofail`, `x-systemd.mount-timeout=2min`, ownership forced to `paperless` via `uid=`/`gid=`, and `RequiresMountsFor` on every paperless unit that touches media — upstream only sets it on `paperless-scheduler`, so the task queue, consumer, web and exporter units get it here. There is deliberately no `x-systemd.automount` / `x-systemd.idle-timeout` anywhere: the idle unmount propagates a *clean* stop through `RequiresMountsFor`, so `Restart=on-failure` never fires — that is exactly what took Immich down for 8–24h at a time. **tmpfiles modes are pinned.** Upstream declares tmpfiles rules for the data, media and consume directories with no explicit mode, i.e. 0755. Two consequences fixed here: the media rule is pinned to 0770 so it matches what `dir_mode=0770` makes the mounted share report — otherwise every bento rebuild re-runs tmpfiles against a live CIFS mount and the `chmod` fails; and `/var/lib/paperless` is pinned to 0750 so the consume directory and the nightly full-document export are not world-readable on a box that also hosts SFTP website users. **Backups.** The media tree is excluded from borg (it lives on the same Storage Box as the borg repo; Hetzner snapshots cover it). The restore-anywhere copy is the nightly `document_exporter` run into `/var/lib/paperless/export`, which the job's existing `/var` path already covers. It runs at 01:30, ahead of the 03:00 borg run, so each night's export ships the same night; the exporter unit `Conflicts` the paperless services, so paperless is briefly down while it runs. The database joins the existing `postgresqlBackup` dumps. Note the exclusion is declared in the paperless module rather than appended to `utils/modules/borgbackup.nix` next to the `/var/lib/immich/` entry. Immich's precedent points the other way, but a web-arm-only path in a module shared by eight hosts is the wart, and keeping it local means the pre-commit hook dry-builds only the host actually affected. Say the word if you'd rather it sat with the immich line and I'll move it. **Login.** OIDC auto-signup is on, so `/paperless` members get an account on first login — but a permissionless one until the admin grants access; per-document ownership does the rest. Local password login stays enabled for exactly one break-glass `admin` account provisioned from `passwordFile`, so a Keycloak outage does not lock the admin out. ## Before this can deploy — maintainer actions ### Secrets to add to `hosts/web-arm/secrets.sops.yaml` `nix-shell -p sops --run 'sops hosts/web-arm/secrets.sops.yaml'` | Key | Contents | | --- | --- | | `paperless-oidc-client-secret` | The `paperless` client secret from Keycloak. Read twice: by the paperless module (into the rendered env file) and by the Keycloak module as `keycloak-client-paperless` (into the file vault as `cloonar_paperlesssecret`). | | `paperless-smb-credentials` | CIFS credentials for the new Storage-Box sub-account, in `mount.cifs` format: a `username=` line and a `password=` line. | | `paperless-admin-password` | Password for the local break-glass `admin` account. | Until these exist the host still evaluates, but activation fails — sops resolves secrets at deploy time, not at eval time. ### External actions 1. **Hetzner Storage Box** — create sub-account `u149513-sub13` (the module mounts `//u149513.your-backup.de/u149513-sub13/`), enable Samba/CIFS on it, and turn on the snapshot schedule. `sub13` was picked as the next unreferenced number in this repo; if it is already taken on the Hetzner side, change the `device` line to the sub-account you create. 2. **DNS** — an `A` record for `paperless.cloonar.com` pointing at web-arm. The certificate then issues itself through the existing lego DNS-01 setup; no repo change needed for that. 3. **Mailbox** — create `scan@cloonar.com` on the mail host for scan-to-email (`scan+<user>@cloonar.com` plus-addressing). 4. **Keycloak** — put the intended users in the `/paperless` group. Nobody can complete a login until they hold `policy-paperless`. 5. **Disk budget** — `/var/lib/paperless/export` is a full copy of every document, on the local SSD and inside the borg set. It is the one place the media-on-Storage-Box split does not save local space; worth sizing before the archive grows. ### Runtime configuration (deliberately not in this PR) The IMAP account, per-user mail rules and owner-assigning workflows, and per-user permissions are configured in the paperless UI after first login. ### Not included, possibly wanted `ansible/roles/ipa_structure/defaults/main.yml` has an `svc-immich` access group but gained no `svc-paperless` here — the issue scoped LDAP/group work as a human action. Happy to add the three-line entry if it belongs in this PR. ## Testing This repo has no unit tests; the gate is the pre-commit hook's dry-build, which ran on `web-arm` and passed. Beyond that I evaluated the resulting config and checked the things that are easy to get quietly wrong: `PAPERLESS_URL` resolves to `https://paperless.cloonar.com`, `PAPERLESS_REDIS` resolves to the module's own `unix:///run/redis-paperless/redis.sock`, `PAPERLESS_DBHOST` is the `/run/postgresql` socket, OCR is `deu+eng`, `PAPERLESS_TASK_WORKERS` is 1, tika and gotenberg are both off, `CPUWeight` is 20 on the scheduler/task-queue/consumer/exporter and untouched on the web unit, the four non-scheduler units carry `RequiresMountsFor`, the mount options contain no automount/idle-timeout, and the rendered template is single-line single-quoted JSON carrying only a sops placeholder. The Keycloak realm JSON was checked for alias/name uniqueness and for every `flowAlias`, `authenticatorConfig`, `condUserRole` and browser-flow override resolving. Runtime verification after deploy is yours.
Native services.paperless at paperless.cloonar.com, gated behind the
existing Keycloak SSO on a /paperless group, with the media tree on a
dedicated Storage-Box CIFS share and a nightly document_exporter run
feeding the existing borg job.

The OIDC client secret is rendered into a systemd EnvironmentFile by a
sops template rather than entering the Nix store; the four paperless
units restart on rotation and on a hash of the template body. The JSON
value is single-quoted because systemd's EnvironmentFile parser strips
unescaped double quotes mid-value.

The media mount is a plain persistent mount with nofail and a generous
timeout, never an idle-timeout automount — that construct took Immich
down for hours at a time. The tmpfiles rules are mode-pinned so the
rebuild-time chmod against the live CIFS mount no-ops, and so the
consume dir and the nightly export are not world-readable.

Requires three new secrets in hosts/web-arm/secrets.sops.yaml
(paperless-oidc-client-secret, paperless-smb-credentials,
paperless-admin-password) and the external actions listed in the PR.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin afk/385:afk/385
git switch afk/385

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff afk/385
git switch afk/385
git rebase main
git switch main
git merge --ff-only afk/385
git switch afk/385
git rebase main
git switch main
git merge --no-ff afk/385
git switch main
git merge --squash afk/385
git switch main
git merge --ff-only afk/385
git switch main
git merge afk/385
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
enhancement

   
in-progress

   
needs-info

   
needs-triage

   
p0

   
ready-for-agent

   
ready-for-human

   
wontfix
No labels
bug
enhancement
in-progress
needs-info
needs-triage
p0
ready-for-agent
ready-for-human
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Cloonar/nixos!386
No description provided.