Learning AI by building it
The question
Can AI help coaches see what they might otherwise miss?
CoachOS is how I'm learning generative AI the way I learn everything — by building a real product. It's an AI copilot for coaches that turns messy, unstructured session notes into grounded summaries and evidence-backed themes, without ever taking judgment away from the coach.
The people behind the problem
Coaches carry a lot between sessions: patterns, hunches and half-remembered details across many clients. Important signals get lost simply because human memory is finite.
The goal isn't to replace the coach's intuition — it's to make sure nothing meaningful slips through the cracks.
Reframing the real problem
The easy version is a summarizer. The hard, honest version is a system whose claims are grounded in what was actually said, that surfaces evidence rather than confident-sounding guesses, and that keeps the human firmly in the loop.
That reframing turns a demo into a trustworthy tool.
Making progress with rigor
I'm building with evals from day one — measuring grounding, hallucination and usefulness rather than trusting vibes. Every theme the system proposes is traceable back to source notes.
It's a deliberate, hands-on way to build genuine intuition for what today's models can and can't be trusted to do.
Outcomes
- Grounded summaries traceable to source notes
- Evaluation-first approach to hallucination and usefulness
- Human-in-the-loop by design — judgment stays with the coach
Next case study
Teaching a marketplace what to pay