Labo de mise en page

Mise en page article blog

Blocs sur les jetons de design Heff.world. Le copy de production vit sur services, hubs cloud et collections Keystatic.

obsolète — Absent de la navigation principale — listé sur la vue d’ensemble du site pour la QA.

Local agentic AI — Mac Mini harnesses, a 4090 worker, and one Obsidian skill vault

JJ Heffernan avatar
JJ Heffernan

The last year of tooling hype treated "one chat window" as the product. That is fine for demos. It is a bad fit for real engineering work, where the bottleneck is rarely "can the model answer?" and almost always how many correct, on-spec iterations you can run before context rot wins.

This post describes the stack I run today: three agent harnesses in parallel on a Mac Mini (Odysseus, OpenClaw, Hermes), a dedicated LM server on a single-GPU 4090 box biased toward smaller models, and API aggregators (OpenRouter, OmniRouter) plus frontier models (including Fable-class endpoints) when the task actually needs them. Skills live in a highly available Obsidian vault that every harness reads — same instructions, no drift.

Net effect: roughly 4× useful output compared to a single-agent, single-UI workflow, without giving up ownership of the stack.

The design thesis — harness ≠ compute

Most products glue the UX harness (tools, skills, memory, file access) to a single provider's API. That coupling is convenient until you want any of the following:

  • Run the same skill pack on a local 7B model and a frontier model
  • Swap models per subtask without re-writing prompts
  • Run multiple agents in parallel without them fighting over one session
  • Keep skills in a repo you control, not in a vendor's "project" blob

The fix is boring and powerful: treat the harness as an API client and treat inference as a replaceable backend.

┌─────────────────────────────────────────────────────────────┐
│  Skill vault (Obsidian) — synced, versioned, HA             │
└──────────────────────────┬──────────────────────────────────┘
                           │ same skills, every agent
     ┌─────────────────────┼─────────────────────┐
     ▼                     ▼                     ▼
┌──────────┐         ┌──────────┐         ┌──────────┐
│ Odysseus │         │ OpenClaw │         │  Hermes  │
│ Mac Mini │         │ Mac Mini │         │ Mac Mini │
└────┬─────┘         └────┬─────┘         └────┬─────┘
     │                    │                    │
     └────────────────────┼────────────────────┘

              ┌───────────────────────┐
              │  Inference routing    │
              │  local LM · OpenRouter│
              │  · OmniRouter · Fable │
              └───────────────────────┘

Harness = tools, skills, MCP, filesystem, git, project context.
Compute = whatever model endpoint answers the next token cheapest for that subtask.

Decouple them and scaling stops looking like "buy a bigger GPU" and starts looking like workflow design.

Layer 1 — Mac Mini: three harnesses, one machine

A Mac Mini is not where I run 70B models. It is where I run orchestration:

HarnessRole in the stack
OdysseusLong-horizon planning, multi-step research, "hold the map" sessions
OpenClawRepo-touching work — edits, tests, PR-shaped loops
HermesFast iteration, smaller tasks, glue automation

All three run in parallel on the same host. That matters: agent work is mostly waiting — on tool results, on sub-agents, on human approval, on network I/O. Three harnesses mean three lanes of work while one lane blocks.

Each harness exposes a different UX on purpose. Not cosmetic — cognitive mode:

  • Odysseus: narrative state, fewer sharp edges, good for ambiguity
  • OpenClaw: file tree + diff mental model, good for code
  • Hermes: lightweight, good for "just do this small thing"

You are not choosing "the best UI." You are choosing which interaction shape matches the task without reconfiguring the entire stack.

Layer 2 — 4090 LM server: small models as workers

The GPU box runs a local LM server (llama.cpp / vLLM-class stack — exact build varies by week). The default posture is smaller models, not "biggest thing that fits in VRAM."

Why small?

Small-model biasWhat it buys you
Low latencyTool loops feel interactive; fewer abandoned runs
High parallelismMultiple agents can hit the box without queue collapse
Skill harness fitMost steps are "follow the skill + call tools" — reasoning depth is optional
Cost isolationBurn local watts on grunt work; spend API $ only on hard slices

Think of the 4090 node as a worker pool, not a oracle. The harness decides when to escalate; the worker node handles classification, extraction, formatting, grep-and-summarize, test log triage, and other high-volume chores.

This is the same philosophy as the Proxmox homelab posts: owned hardware for the boring always-on layer, APIs for burst capacity.

Layer 3 — API aggregators and frontier models

Local workers do not replace frontier models. They protect them.

OpenRouter and OmniRouter sit upstream as aggregators — one integration surface, many providers, sensible failover when a route is slow or down. Typical routing:

Task shapeWhere it goes
Bulk tool steps, drafts, summariesLocal 4090
Architecture review, subtle refactorsMid-tier API model
Novel design, adversarial review, "are we wrong?"Frontier (e.g. Fable-class)

Fable 5 (and peers) are not the default. They are the appeals court — invoked when local + mid-tier disagree, when the spec is under-specified, or when the cost of a wrong merge is high.

The win is not "always use the smartest model." The win is never using the smartest model for work that does not deserve it.

Layer 4 — Obsidian skill vault: one source of truth

Skills scattered across .cursor/rules, random gists, and chat system prompts drift. Multiple agents make drift worse — each session "helpfully" rewrites instructions.

The fix: one Obsidian vault, treated like production config:

  • Highly available — synced across machines (Syncthing / git / both; the exact transport is less important than "there is always a current copy")
  • Constantly updated — skills evolve with the repo, not with one chat's memory
  • Harness-agnostic — Odysseus, OpenClaw, Hermes, and IDE agents all read the same files

When every agent loads the same SKILL.md, CONTEXT.md, and checklists:

  • "Ask three agents the same question" becomes a real ensemble, not three different religions
  • Reviews converge because the definition of done is shared
  • You can diff skill changes like code changes

Skill drift is the silent killer of multi-agent setups. Centralizing skills in Obsidian is the cheapest anti-drift measure I have found.

What ~4× actually means

"4× productivity" is not "4× tokens." It is roughly 4× merged, shippable iterations before fatigue or context loss ends the session:

Single-agent defaultThis stack
One queueThree parallel harness lanes
One UI mindsetPick UX per task
One model for everythingLocal workers + routed APIs
Skills in chat historySkills in synced vault
Escalation = manual pasteEscalation = routing policy

Some tasks do not benefit (deep solo design, sensitive one-shot writing). Many engineering tasks do — especially repo hygiene, test fixes, doc passes, content migrations, and parallel research.

Unorthodox scaling — because harness and compute are separate

Once inference is an API call, scaling stops being vertical-only:

Multiple agents in parallel

Run the same skill against different models on the same repo snapshot:

  • Local worker: "find all call sites"
  • API model: "propose refactor"
  • Frontier model: "attack the proposal"

You are buying diversity of error, not redundancy of effort.

Per-model workflow automations

Different models excel at different automation shapes:

Model tierAutomation style
Small localHigh-frequency cron: lint summaries, issue triage drafts, log digests
Mid APIPR description + test plan generation
FrontierQuarterly architecture audits, threat modeling

The harness stays constant; the automation recipe changes per endpoint.

Plugin and hosting headroom

Because the stack is self-hosted and self-owned, you can go places SaaS agents will not:

  • Fully open-source LLM hosting — experiment with weights, quantization, and routing without ToS anxiety
  • Custom plugins per project — MCP servers, domain-specific tools, hardware bridges (CAD, radio, shop floor) without waiting for a vendor roadmap
  • Air-gapped modes — local-only paths for credentials, client work, or experiments you do not want on a shared API

Ownership is not nostalgia. It is permission to run weird topologies.

Practical constraints (honest section)

This stack is not free:

  • You operate it — sync, backups, GPU drivers, aggregator keys, harness updates
  • Parallel agents can conflict — git locks, duplicate edits; you need branch discipline or task partitioning
  • Small models fail — the harness must detect low-confidence output and escalate
  • Obsidian sync must be boring — if the vault is wrong, every agent is wrong together

The homelab mindset applies: automate the boring parts, document the rest, accept that "always on" is a feature you pay for in attention.

Where this goes next

Near-term experiments on this stack:

  • Tighter routing policies — when to escalate from 4090 → OpenRouter → frontier, logged and tunable
  • Skill CI — lint skill files, snapshot tests for critical workflows
  • Project-specific plugin packs — membership billing, Keystatic content, mechanical CAD helpers — all reading the same vault

If you are still optimizing one chat window, you are optimizing the wrong layer. Optimize the harness, centralize the skills, treat compute as a menu — then parallel agents stop being a party trick and start being infrastructure.


Related: self-hosted infra on Proxmox — cluster write-up. Site tooling and agent skills for this repo live under docs/agents/ and .agents/skills/.

Articles liés