Cloonar/nixos
3
0
Fork 0
Code Issues 16 Pull requests 3 Projects Releases Packages Wiki Activity Actions

feat(mail): offload per-account Archive folders to Hetzner Storage Box #238

Open
dominik.polakovics wants to merge 3 commits from lab/20260726-1358 into main
pull from: lab/20260726-1358
merge into: Cloonar:main
Conversation 2 Commits 3 Files changed 2 +76
dominik.polakovics commented 2026-07-26 14:09:13 +02:00
Owner
Copy link

Closes #237.

The mail disk is running full. This gives every account archive folders stored on a Hetzner Storage Box instead of the local disk. Users archive manually via their client's archive action; quota semantics are unchanged because the user quota root spans all private namespaces.

What this does

  • CIFS automount of a Storage Box sub-account at /mnt/mail-archive — same sops-credentials + systemd-automount pattern as the Immich originals mount on web-arm. noauto + automount keeps dovecot startup independent of the Storage Box: if it's unreachable, only archive folders error; INBOX and delivery keep working.

  • Two Dovecot namespaces on the mount, covering every mainstream client's default archive folder name:

    • Archive. — Apple Mail, Outlook, Windows Mail, Roundcube, K-9/Thunderbird Android
    • Archives. — Thunderbird desktop (hardcoded Archives/<year>; TB ignores the RFC 6154 \Archive special-use flag, so it can't be redirected server-side)

    Deliberately two real folder trees rather than a hidden alias_for namespace — folders clients CREATE but never see in LIST invite per-client quirks. Content splits by whichever name a client uses; both trees are on the Storage Box.

  • INDEX (index/cache + fts-lucene) and CONTROL (uidlist, subscriptions) for both namespaces stay under /var/vmail, so IMAP metadata and locking never touch the network share — only message bodies do. Those local dirs also stay inside borg's paths.

  • Backup of archived mail is via the Storage Box's automatic snapshots (like Immich), not borg — the mount is deliberately outside borg's paths.

⚠️ Required before merge (config cannot build without these)

  1. Create the Storage Box sub-account with Samba enabled. The module assumes u149513-sub13 — if the actual name differs, update the device line in hosts/mail/modules/mail-archive.nix.
  2. Add the SMB credentials secret (nix-shell -p sops --run 'sops hosts/mail/secrets.yaml'): 
    mail-archive-smb-credentials: |
  username=u149513-sub13
  password=<sub-account password>
    sops-nix validates secret keys at eval time, so the mail config fails to build until this key exists.
  3. Enable the snapshot plan on the Storage Box (this is the only backup of archived mail).

Note: the pre-commit dry-build could not run in this environment (no Nix installed), so these commits are not build-verified. Please run ./scripts/test-configuration mail after adding the secret, before merging.

Migration of existing archives (after deploy)

Accounts that already have .Archive* / .Archives* folders under their Maildir keep working (shadowed by the new namespaces) but stay on local disk until moved:

systemctl stop dovecot
# per account /var/vmail/<domain>/<user> that has Maildir/.Archive* or .Archives*:
base=/mnt/mail-archive/<domain>/<user>

# singular tree → Archive namespace dir
dest=$base/Archive; mkdir -p "$dest"
mv Maildir/.Archive/{cur,new,tmp} "$dest"/                                  # folder "Archive" = namespace root
for d in Maildir/.Archive.*; do mv "$d" "$dest/.${d##*.Archive.}"; done     # "Archive.X" → ".X"

# plural tree (Thunderbird) → Archives namespace dir
dest=$base/Archives; mkdir -p "$dest"
mv Maildir/.Archives/{cur,new,tmp} "$dest"/
for d in Maildir/.Archives.*; do mv "$d" "$dest/.${d##*.Archives.}"; done

rm -rf Maildir/.Archive Maildir/.Archives
systemctl start dovecot

No chown needed — the mount forces uid/gid to vmail. UIDs change, so clients re-download the archive folders once; users may also need to re-subscribe to archive subfolders (subscriptions now live in each namespace's own CONTROL dir).

Closes #237. The mail disk is running full. This gives every account archive folders stored on a Hetzner Storage Box instead of the local disk. Users archive manually via their client's archive action; quota semantics are unchanged because the user quota root spans all private namespaces. ## What this does - CIFS automount of a Storage Box sub-account at `/mnt/mail-archive` — same sops-credentials + systemd-automount pattern as the Immich originals mount on web-arm. `noauto` + automount keeps dovecot startup independent of the Storage Box: if it's unreachable, only archive folders error; INBOX and delivery keep working. - Two Dovecot namespaces on the mount, covering every mainstream client's **default** archive folder name: - `Archive.` — Apple Mail, Outlook, Windows Mail, Roundcube, K-9/Thunderbird Android - `Archives.` — Thunderbird desktop (hardcoded `Archives/<year>`; TB ignores the RFC 6154 `\Archive` special-use flag, so it can't be redirected server-side) Deliberately two real folder trees rather than a hidden `alias_for` namespace — folders clients CREATE but never see in LIST invite per-client quirks. Content splits by whichever name a client uses; both trees are on the Storage Box. - `INDEX` (index/cache + fts-lucene) and `CONTROL` (uidlist, subscriptions) for both namespaces stay under `/var/vmail`, so IMAP metadata and locking never touch the network share — only message bodies do. Those local dirs also stay inside borg's paths. - Backup of archived mail is via the Storage Box's automatic snapshots (like Immich), **not** borg — the mount is deliberately outside borg's paths. ## ⚠️ Required before merge (config cannot build without these) 1. **Create the Storage Box sub-account** with Samba enabled. The module assumes `u149513-sub13` — if the actual name differs, update the `device` line in `hosts/mail/modules/mail-archive.nix`. 2. **Add the SMB credentials secret** (`nix-shell -p sops --run 'sops hosts/mail/secrets.yaml'`): ``` mail-archive-smb-credentials: | username=u149513-sub13 password=<sub-account password> ``` sops-nix validates secret keys at eval time, so the mail config fails to build until this key exists. 3. **Enable the snapshot plan** on the Storage Box (this is the only backup of archived mail). **Note:** the pre-commit dry-build could not run in this environment (no Nix installed), so these commits are *not* build-verified. Please run `./scripts/test-configuration mail` after adding the secret, before merging. ## Migration of existing archives (after deploy) Accounts that already have `.Archive*` / `.Archives*` folders under their Maildir keep working (shadowed by the new namespaces) but stay on local disk until moved: ``` systemctl stop dovecot # per account /var/vmail/<domain>/<user> that has Maildir/.Archive* or .Archives*: base=/mnt/mail-archive/<domain>/<user> # singular tree → Archive namespace dir dest=$base/Archive; mkdir -p "$dest" mv Maildir/.Archive/{cur,new,tmp} "$dest"/ # folder "Archive" = namespace root for d in Maildir/.Archive.*; do mv "$d" "$dest/.${d##*.Archive.}"; done # "Archive.X" → ".X" # plural tree (Thunderbird) → Archives namespace dir dest=$base/Archives; mkdir -p "$dest" mv Maildir/.Archives/{cur,new,tmp} "$dest"/ for d in Maildir/.Archives.*; do mv "$d" "$dest/.${d##*.Archives.}"; done rm -rf Maildir/.Archive Maildir/.Archives systemctl start dovecot ``` No chown needed — the mount forces uid/gid to vmail. UIDs change, so clients re-download the archive folders once; users may also need to re-subscribe to archive subfolders (subscriptions now live in each namespace's own CONTROL dir).
Adds a CIFS automount of a Storage Box sub-account at /mnt/mail-archive
and a dovecot 'Archive.' namespace whose maildir lives there, with INDEX
and CONTROL kept on local disk. Closes #237.
Thunderbird archives to 'Archives/<year>' and ignores the RFC 6154
\Archive special-use flag, so a namespace named 'Archive.' would be
bypassed — TB would create a plain local-disk Archives folder instead.
dominik.polakovics commented 2026-07-26 14:16:22 +02:00
Author
Owner
Copy link

Pushed 6c7df22: namespace renamed Archive. → Archives. so Thunderbird's default archiving works out of the box. TB archives to "Archives/" and ignores the RFC 6154 \Archive special-use flag, so the singular name would have been silently bypassed (TB would have created a plain local-disk folder).

Migration note: accounts where users already archived with Thunderbird have existing local .Archives* maildirs — the runbook in the PR description applies to those the same way (.Archives → namespace root, .Archives.2025 → .2025). A leftover singular .Archive* tree (non-TB clients) is not shadowed by the namespace and can be moved into Archives/ via IMAP or the same rsync pattern.

Clients that default to a different archive name (Apple Mail uses "Archive") need their archive folder pointed at "Archives" in account settings, or they'll archive to an ordinary local-disk folder. If that becomes a real problem we can add a hidden dovecot alias namespace Archive. → Archives. later.

Pushed 6c7df22: namespace renamed `Archive.` → `Archives.` so Thunderbird's default archiving works out of the box. TB archives to "Archives/<year>" and ignores the RFC 6154 \Archive special-use flag, so the singular name would have been silently bypassed (TB would have created a plain local-disk folder). Migration note: accounts where users already archived with Thunderbird have existing local `.Archives*` maildirs — the runbook in the PR description applies to those the same way (`.Archives` → namespace root, `.Archives.2025` → `.2025`). A leftover singular `.Archive*` tree (non-TB clients) is not shadowed by the namespace and can be moved into Archives/ via IMAP or the same rsync pattern. Clients that default to a different archive name (Apple Mail uses "Archive") need their archive folder pointed at "Archives" in account settings, or they'll archive to an ordinary local-disk folder. If that becomes a real problem we can add a hidden dovecot alias namespace `Archive.` → `Archives.` later.
Two real namespaces instead of one: 'Archive' is the default archive
folder of Apple Mail, Outlook, Windows Mail, Roundcube and K-9;
'Archives' is Thunderbird desktop's hardcoded name. Both trees live on
the Storage Box mount. A hidden alias_for namespace was considered but
rejected: folders clients CREATE but never see in LIST invite
per-client quirks.
dominik.polakovics commented 2026-07-26 14:21:47 +02:00
Author
Owner
Copy link

Pushed 7b48dd0: second namespace Archive. added alongside Archives. so the default archive folder of Apple Mail / Outlook / Windows Mail / Roundcube / K-9 also lands on the Storage Box, not just Thunderbird's. PR description (incl. migration runbook) updated for the two per-name directories on the mount.

Pushed 7b48dd0: second namespace `Archive.` added alongside `Archives.` so the default archive folder of Apple Mail / Outlook / Windows Mail / Roundcube / K-9 also lands on the Storage Box, not just Thunderbird's. PR description (incl. migration runbook) updated for the two per-name directories on the mount.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin lab/20260726-1358:lab/20260726-1358
git switch lab/20260726-1358

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff lab/20260726-1358
git switch lab/20260726-1358
git rebase main
git switch main
git merge --ff-only lab/20260726-1358
git switch lab/20260726-1358
git rebase main
git switch main
git merge --no-ff lab/20260726-1358
git switch main
git merge --squash lab/20260726-1358
git switch main
git merge --ff-only lab/20260726-1358
git switch main
git merge lab/20260726-1358
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
enhancement

   
in-progress

   
needs-info

   
needs-triage

   
p0

   
ready-for-agent

   
ready-for-human

   
wontfix
No labels
bug
enhancement
in-progress
needs-info
needs-triage
p0
ready-for-agent
ready-for-human
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Cloonar/nixos!238
No description provided.