Bagman
a group plays four games at once and one person ends up doing the math instead of golfing. bagman's caddy takes a spoken score, shows it back on a card you can edit, and every phone in the group gets the new standings.
- status
- pre-launch
- stack
- expo · hono + orpc · neon · redis + sse · mastra · inngest · fly
- repo
- private, case study only
the problem
a serious golf group plays several games at once. skins on the front, a nassau across the eighteen, stableford for a season ladder, and a custom game where a putt from off the green is worth a point. someone carries the notepad, and that person stops playing golf.
the scoring apps i tried were built for stroke play and pushed everything else into a form you fill in after the round. the case i cared about is different. the math runs after every hole, every phone in the group has to show the same standings, and you should be able to enter a score without stopping to type.
so the caddy's main job is bookkeeping. "blake got 5, jake got 4, i got 6" is structured data in a sentence. the work is turning that sentence into rows, getting a human to agree, and then getting the result onto four phones.
from a spoken score to new standings
this is the path a score takes, in order. every step below is code that runs today.
- transcription. the app records with
expo-audio. on iOS 26 devices that support it, the take is transcribed on the phone with Apple's SpeechAnalyzer. if the device can't, and the player has allowed a cloud fallback, the file goes to the API and Whisper (whisper-large-v3-turbothrough OpenRouter) transcribes it. the audio file is deleted after every outcome, and neither the audio nor the transcript goes into telemetry. - routing. a voice turn always runs on the
live_roundmodel role, which is Claude Haiku 4.5. typed turns can move up to a reasoning model for rulings or "explain the press" asks; voice turns never do. voice mode also tells the model to answer in one sentence with no markdown. - the prompt. the system prompt is built in four layers: a global base, a layer for the screen you're on, the round's stable facts, and the turn's live facts. the first three carry an Anthropic cache breakpoint with a one-hour TTL. only the last layer changes from turn to turn. the long version is here.
- the confirm card. the model calls
submit_hole_scores. the tool doesn't write yet. it suspends the run, the phone renders a card with the parsed scores, and the player can fix a number before tapping confirm. the run resumes with the edited input if there is one. - one transaction.
roundService.submitScoressaves the scores, recalculates every game in the round (recalculateAllResults), and advances the current hole, all in one database transaction. the caddy uses the same service as the manual scorecard, so there is one write path. - after the commit. the service appends timeline rows to
round_events, publishesscore:submittedto Redis for the phones that are listening now, and publishes the domain events through the event bus, which records them in a Postgres outbox before sending them to Inngest for the background work (hole summaries, the next hole's caddy notes, game projections, notifications). - the phones. a separate stream server, the same API image started with
SERVER_ROLE=stream, holds the SSE connections and forwards the Redis message as an oRPC event iterator. each phone looks the event up in one table that says which data it changes, and refetches only that data.
mobile: expo (eas), react native, react query, apple speechanalyzer · api: hono on fly, orpc, drizzle, api and stream roles from one image · data: neon postgres, redis pub/sub → orpc sse, postgres outbox → inngest · ai: mastra, openrouter · one model per role, langfuse traces, scenario evals
everything is a managed service or a fly machine because i'm one person. the only topology i own is the stream server that fans out live rounds.
the choices that shaped it
one agent, flat tools, a model per role
- one caddy agent with 33 tools: 20 reads that run on their own, 13 writes
- 11 of the writes stop for a confirm tap before anything is saved
- each job gets a role (live round, reasoning, background, verifier) and each role a model
- role assignments live in the database, so swapping a model is a row change
an orchestrator with four sub-agents
- caddie, stats, scorekeeper and game setup each sat behind a delegation tool
- every delegation was another model call on the hot path
- each sub-agent saw only the prompt string it was handed, not the round
the split looked tidy on a diagram. in practice the scorekeeper didn't know which round it was scoring. a few specialists still exist (a dispute mediator, a rules compiler, an artifact writer), but they are self-contained: they get everything they need in their input, so they can't lose the round.
redis pub/sub + sse on its own server
- the api publishes round events to redis; a stream server holds the connections
- live updates are orpc event iterators, so they share the typed contract
- a reconnect replays the last two minutes by Last-Event-ID
- history lives in a durable round_events table, not in the transport
ably, then self-hosted sockudo
- ably was the original spec: per-message billing and a second sdk on every client
- sockudo, a pusher-protocol server, replaced soketi and then had to be run and patched
- two transports: sockets for live data, rpc for everything else
live data only ever flows from the server to the phone, and writes already go through rpc, so sse was enough. the stream role scales on open connections and the api role scales on requests, which is why they run as separate apps.
hard problems
the sub-agents never saw the round
i collapsed the four into one agent with a flat tool set. the round context is built once per turn and sits in the same prompt as every tool that needs it. model choice moved out of the agent graph into a per-role table, so "use a bigger model for this" no longer means "add an agent".
an orchestrator agent that delegated to four sub-agents through mastra's agents map. the orchestrator got the preloaded round; each sub-agent was its own model call.
a sub-agent only received the prompt string the parent wrote for it. the round, the roster and the roundPlayerIds stayed behind. score submissions came back with zero hole_scores rows.
the eval set couldn't say what mattered
every behaviour is now one typed scenario file. each clause maps to one scorer, and a tool name that doesn't exist fails a lint before any model is called. the 51 json rows that were still worth keeping moved into the same format as regressions.
the rules that keep the suite honest, from linting every registry to fixing a fixture instead of lowering a threshold, are in keeping the caddy's evals honest.
a 53-row json dataset. each row was a message, the intent it should get, the tools it should call and some of their arguments, scored for intent, tool choice and arguments.
a row had no way to say don't call this tool, finish in n steps, use these ids, keep this output shape, or get the answer right. and tool names in data rot quietly: deleting the golf bag feature left two scenarios calling tools that no longer existed, and nothing flagged them.
the cheap model won
the more useful result was where they matched. both scored exactly 0.600 on tool arguments, 0.792 on tool selection and 0.600 on intent. the hard cases failed the same way on both, so the limit was the task, not the size of the model.
five scenarios is a small sample and i don't treat it as a ranking. it was enough to keep the turn router's default on the cheap role, upgrade only rulings and explicit "explain this" asks, and never upgrade a voice turn. a weekly job keeps sweeping new models against the same suites and suggests a change; a human makes it. how the evals work is in this post.
assume that harder in-round questions need a bigger model. at the time the reasoning role was claude sonnet 4.6 and the live role was haiku 4.5.
on the first sweep of the frontier suite (five scenarios built to be hard for every model), haiku scored 0.874 overall and sonnet scored 0.849. sonnet lost points on answer correctness, 0.800 to haiku's 1.000.
where it stands
one caddy agent with flat tools, a model per role, a four-layer cached prompt, one write path for scores, and live fan-out over redis and sse on a dedicated stream server. pre-launch, so there are no usage numbers here yet.