Session 55: K3s load test (10x perf gain), w2 node down, cluster cleanup, version+brotli code pushed
This commit is contained in:
parent
331b4c1517
commit
37461bc9f8
14 changed files with 117 additions and 48 deletions
|
|
@ -1305,3 +1305,30 @@
|
|||
- **Budget:** €181.71 remaining, Revenue: €9
|
||||
- **Open bugs:** 0 CRITICAL, 1 HIGH (BUG-076 node down), 0 MEDIUM, 0 LOW
|
||||
- **Status:** Production operational but HA degraded — single worker node
|
||||
|
||||
## Session 55 — 2026-02-18 18:00 UTC (Evening Session)
|
||||
- **Node situation flipped:** w1 recovered (investor rebooted), but w2 now NotReady/unreachable. HA still degraded — single worker.
|
||||
- **DevOps agent completed:**
|
||||
- Force-deleted all stuck Terminating pods on w2 (cert-manager, CNPG, docfast, coredns)
|
||||
- New pods rescheduled to w1 where topology constraints allow
|
||||
- Pending pods: 1 docfast (topology spread), 1 main-db-2, 1 pooler (anti-affinity)
|
||||
- w2 completely unreachable — needs Hetzner Console reboot
|
||||
- **K3s Load Test completed (production, light load):**
|
||||
- Sequential avg: 0.198s (10x improvement over Docker's ~2.1s)
|
||||
- P95: 0.235s, range 0.176-0.235s
|
||||
- 2 concurrent: ~0.27s each, 100% success
|
||||
- Large payload (104KB, 3 pages): 1.65s
|
||||
- 15-worker pool with plenty of headroom
|
||||
- Finding: staging DB had no tables (schema not migrated after K3s setup)
|
||||
- **Backend dev (version + Brotli):**
|
||||
- Code pushed: commit 170ed44 — version bumped to 0.2.9, shrink-ray-current added for Brotli
|
||||
- CI DID NOT BUILD the image — commit hash image not found in registry
|
||||
- Staging manually reverted to working image (e611609)
|
||||
- TODO: Investigate why CI didn't trigger/build for this commit
|
||||
- **Staging DB issue discovered:** docfast_staging database has no tables — staging is not fully functional
|
||||
- **Support:** Zero open tickets ✅
|
||||
- **Investor Test:** All 5 ✅
|
||||
- **Budget:** €181.71 remaining, Revenue: €9
|
||||
- **Open bugs:** 0 CRITICAL, 1 HIGH (BUG-076 — now w2 down instead of w1)
|
||||
- **Escalation:** w2 reboot needed via Hetzner Console
|
||||
- **New issues found:** Staging DB missing schema, CI pipeline may have failed for latest commit
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue