How Hermes Agent + Strata + llama.cpp + three servers cooperate to ship real projects — hardware, serving stack, live performance, project history, and where the setup can improve next. All numbers pulled live from the machines themselves.
Four machines, one tailnet, one public surface. The agent works locally, deploys over SSH, and verifies its own changes in a real browser against production.
edit on a feature branch → unit tests (all green before commit) → PR → squash-merge on approval → SSH deploy → browser-verify against production → report. Nine frontend PRs, home-monitoring, job-search and the RAG fix all went through this exact loop.
The camera dashboard is mirrored publicly by Caddy proxying 100.67.199.85:8090 over the tailnet — zero port-forwarding on the home router. Inference follows the user anywhere because the Mac's provider config is just a tailnet URL.
Live-verified via lscpu / free / nvidia-smi / Strata /metrics. An X299 HCC box whose 128 GB RAM and 44 PCIe lanes are exactly what Strata's RAM-expert-paging needs.
| GPU | VRAM | Link | Role |
|---|---|---|---|
| RTX 5060 Ti ×3 | 48 GB pool | PCIe 3.0 ×8 each | Strata main lane — hot experts in VRAM, tail in RAM blobs (7.5–7.9 GB/s rx bursts) |
| RTX 3060 Ti | 8 GB | gen3 ×16 (gen1 idle) | :8083 Qwen2.5-VL-7B vision critic — 84 tok/s, image QA 1.4 s |
| RTX 2060 | 6 GB | gen3 ×16 (gen1 idle) | :8084 bge-m3 + :8085 bge-reranker — ~0.9 GB shared |
The Qwen3.8-Flash-Next MoE at Q4_K_XL exceeds any single card. Strata keeps hot experts in the 48 GB tri-GPU pool and pages the long tail from system RAM (20 GB expert cache, 6,696 slots, 63 GB arena) — viable only with 128 GB RAM + enough lanes to stream at multi-GB/s. Older cards handle bandwidth-light satellites; 16 cores cover Strata's 15 pool workers plus compiles.
Main 3-GPU lane idles ~91 W, ~160 W under load (540 W combined limit), temps 36–42 °C. The 3060 Ti / 2060 showing gen1 links at idle is ASPM power-state downtraining, not a fault.
One Strata main lane + three llama.cpp satellites, all systemd user units
under hermes-llm.target with linger on — boot-persistent, no sudo.
| Port | Engine | Model | Measured |
|---|---|---|---|
:8080 | Strata 0.1.43.1 | qwen3.8-flash-next-unsloth-ud-q4_k_xl · 262k ctx · int8 KV · vision · MTP spec decode | 50–59 tok/s decode 85% draft accept |
:8083 | llama.cpp | Qwen2.5-VL-7B IQ4_XS + F16 mmproj (vision critic) | 84 tok/s image QA 1.4 s |
:8084 | llama.cpp | bge-m3 Q8_0 embeddings · --pooling cls | 4,276 tok/s |
:8085 | llama.cpp | bge-reranker-v2-m3 Q8_0 · --reranking | 297 pairs/s |
Main chat and every aux lane (review, approval, curator, goal_judge, background_review) point at strata-server-gpu-master-tailnet-8080. One slot, so requests queue — fine for a single agent. The Strata unit carries a port guard: it exits clean if a manual instance already serves :8080 instead of restart-looping.
CUDA_DEVICE_ORDER=PCI_BUS_ID is mandatory — default ordering silently mispins cards (phantom OOMs). Q4_K_M + F16 mmproj OOMs 8 GB; IQ4_XS fits. Native CUDA build because no docker group; pkill -f over SSH kills your own session — use fuser -k.
Pulled from Strata /metrics at page load; totals cover the engine's current window. This is the agent's own inference, measured.
live history from engine…
—
Cold 51k-token prefill: 46.7 s → warm (99.7% prefix reused): 3.6 s. Reuse + MTP drafts are what make a 1-slot local engine feel instant across a dev day.
Built from zero: 6-cam XVR snapshots → YOLOv8n detection → SQLite analytics (frame blobs, timeline, gallery) → public mirror. The saturation saga: JSON busy-envelopes at ~90% error → measured device (p95 ≈ 8 s) → DeviceThrottle → ~6%. Then the 4.0 s request exposed starvation (cam1: no frame 10+ min) → rewrote throttle as FIFO cond-var queue, regression test fails on old code / passes on new. All channels cycle fairly.
Warm Console redesign, terminal-style AI console (cyan user / green system lines), browser TTS at 1.15 with male-voice consistency fix (Chrome's voice list is empty until voiceschanged, especially post-speech-recognition), whole-tile click targets, 4×2→2-col grid, apps reorder + Job Search thumbnail (PR #13, live).
/chat was 500ing because OpenAI deprecated o4-mini. Now defaults gpt-5-mini with RAG_CHAT_MODEL env override — the next deprecation is a config change, not a deploy.
Per-profile isolation (mike/jen: own resume, URLs, jobs DB, seen-ledger), VPS gatekeeper auth, export fixes. Daily digest: VPS producer + Mac delivery, three consecutive clean editions. PetPals, airbnb-intel, coloring-book beta gate.
· Enable Strata's conversation cache (4 slots configured, budget 0)
· Second small lane on the 2060/3060 Ti for delegation fan-out
· Wire qmd semantic search to :8084/:8085
· Nightly /metrics snapshots → trend drift
· basic_auth gate on the public camera mirror
· Rotate the leaked coloring-book OpenAI key (commit 0e52432)
· fetch_timeout 10→20 s to absorb the XVR tail
· Nightly synthetic smoke across 8 sites + 4 lanes
· Vetted allowlist for recurring remote probes (approval stalls)
· Always corroborate vision claims with pixel checks
· Children's self-reports ≠ verified facts: require handles
· Per-repo state files for deterministic post-compaction recovery
· Re-embed this store whenever memory/skills change
· Grow the chunk set from session digests automatically
· Keep this page as the living runbook for the stack
Grounded RAG over this page's own knowledge base: your question is embedded by bge-m3 (:8084), recalled by cosine, reranked by bge-reranker (:8085), then answered by Qwen3.8-Flash-Next (:8080) with cited sections. All local, over the tailnet.