What gets built here

A single agent in a while-loop is a demo. An orchestration is what survives production: a planner that decomposes the request, subagents fanned out over the parts, an explicit handoff contract between them, and a merge step that reconciles disagreement instead of taking whichever answer arrived last. Underneath sits durable execution — Temporal or Inngest — so a deploy mid-run, a rate-limited provider, or a process dying at step nine replays from its last checkpoint rather than from the beginning. Everything is idempotent because at-least-once delivery is the only honest assumption, and anything that exhausts its retries lands in a dead-letter queue a person can read, fix and replay. Human-in-the-loop checkpoints are first-class: the run pauses, waits for an approval that may arrive tomorrow, and continues.

Most failing agents do not fail at the model. They fail at the loop around it: context that grows until the instructions fall out of attention, a tool-call budget nobody set, a stop condition that never fires, and a retry that feeds the same poisoned context back in. Loop engineering is the unglamorous fix — compaction that summarises completed work instead of re-reading every tool result, a budget on calls and tokens per run with a defined behaviour at the ceiling, stop conditions that distinguish finished from stuck, and a self-critique pass that checks the result against the original request before it is returned. Every decision the loop makes is traced with OpenTelemetry, so when a run costs eleven dollars you can see the step that did it rather than guess.

Retrieval quality is a ranking problem before it is a model problem, and context is a budget before it is a prompt. The work runs in three layers. Retrieval: chunking that respects how a document is actually structured, hybrid keyword-plus-vector search rather than embeddings alone, a reranking pass, and citations rendered back to the source so an answer can be checked instead of trusted. Graph: entity extraction and resolution into a knowledge graph, so a question that spans three documents traverses relationships rather than hoping all three chunks rank in the same top-ten — the GraphRAG case, and the one plain vector search is worst at. Memory: episodic memory of what happened in past runs, semantic memory of what was learned from them, a written policy for what gets promoted and what is allowed to decay, and compaction so a long-lived agent does not carry a year of transcripts into every window.

MCP is how an agent reaches a system it does not own, and the protocol is the easy half. The work is the surface: tool schemas typed tightly enough that the model cannot express an invalid call, scopes that mean a compromised or confused agent reaches exactly one customer's records rather than the table they sit in, resources and prompts that give the model the shape of your domain without a paragraph of prose in every system message, and error messages written for a model to recover from rather than for a log. Auth is OAuth 2.1 with per-tool scopes, not a shared key in an environment variable. Servers ship on Vercel or Cloudflare with traces on every call, because a tool surface nobody can audit is a tool surface nobody should have connected.

Without evals, every prompt change is a coin flip you cannot score. The build starts with a golden set drawn from your real traffic and labelled with your team, because a suite written from imagination measures imagination. Scoring combines deterministic assertions where the answer has a right shape with LLM-as-judge where it does not, and the judge itself is validated against human labels before it is trusted to grade anything. That suite becomes a regression gate in CI: a change that drops accuracy on the golden set does not merge. Alongside it runs the safety half — prompt-injection defence on every untrusted input, output filtering, a red-team pass against your specific tool surface, and OpenTelemetry traces so a production failure can be replayed as a new eval case instead of argued about in Slack.

A copilot that lives in your product is a different build from one that lives in a chat window. It streams, because a spinner over four seconds of silence reads as broken. It renders generative UI — a real component the user can act on, not a paragraph describing what they could do. It calls your own API through tools scoped to the signed-in user's permissions, so the assistant can never surface a record that user could not open themselves. Outputs are schema-constrained with Zod and written back as structured rows, which is what makes the assistant part of the product rather than a text box beside it. This is also where tier-1 support deflection and content pipelines land when the right home for them is inside the product.

Voice is a latency problem wearing an AI costume. Past roughly eight hundred milliseconds a caller starts talking over the agent, so the build is speech-to-speech rather than the transcribe-think-synthesise chain that stacks three round-trips before anyone hears anything. Turn detection decides when the caller has actually finished rather than merely paused, barge-in stops the agent mid-sentence when they interrupt, and both are tuned against your own recordings, because the right thresholds differ between a support line and a booking line. It runs on a real telephony path through Twilio or in-product over LiveKit WebRTC, with a transfer rule written down: on low confidence, on an explicit request, or on any topic you have declared off-limits, the call goes to a person with the transcript and the extracted details already attached.

The documents that matter are rarely clean text. They are scans, multi-page tables that break across a page, a stamp over the total, and a handwritten amendment in the margin. Vision models read those in a way OCR never did, but reading is not the deliverable — a validated record is. Extraction is schema-constrained, so the output is a typed object rather than prose to be parsed twice, and every field carries a citation to the page and region it came from, which is what turns a review from re-reading the document into checking a highlight. A confidence gate routes the difference: high-confidence records land straight in the system of record, low-confidence ones go to a human queue with the uncertain fields flagged, and every correction becomes an eval case so the gate gets better instead of staying where it was set on day one.

An audit of what you have, a roadmap ranked by time-saved-per-dollar, or a build-versus-buy review before you commit a quarter to the wrong path. It covers the questions that get expensive later: which of these belongs to a model at all rather than to a process change, what your data actually supports today, model and vendor policy including where an open-weight model on your own infrastructure is the right answer, and how any of it will be measured — an eval plan, before the build, rather than after the first incident. The deliverable is a written document you own outright and can hand to another vendor. A meaningful share end in a recommendation not to build the thing, which is what the opinion is for.