Open-Source Course Details Three Ways to Run AI Agent Loops and the Cost Trade-Offs

In LangChain's Terminal-Bench experiment, changing only the harness while keeping the same model moved a coding agent from roughly 30th place into the top 5. That result reframes the question: if the harness determines quality, then how you run the loop becomes an architecture decision, not a deployment detail.

The course, Building a Coding Agent From Scratch by Paul Iusztin, published through Decoding AI, builds a Python agent called Decode. It separates three run modes: interactive online, remote offline, and async online. At the center is a headless harness with no interface of its own, containing the shared agent loop where the LLM picks an action, a tool executes, and the observation feeds back. The agent itself is small—about 20 lines of Pydantic AI definition—while Claude Code's leaked source shows a roughly 150-line core loop. Everything else is harness.

Mode 1, interactive online, connects a terminal UI to one live session in memory. A steering queue and priority gate handle input safely, buffering typed messages and injecting them only at safe boundaries such as MODEL_REQUEST or WOULD_STOP. Because a human is reading every token, this mode is latency-bound and belongs on a low-latency hosted API.

Mode 2, remote offline, keeps the harness headless and runs it on a server through an agent runtime. Decode uses Kitaru, ZenML's agent runtime, deployed to GCP, with agents executing on Modal. A backlog of tickets fans out to N harnesses in parallel, each producing a PR. The runtime records progress step by step, so a sandbox that dies mid-task resumes from its last recorded step. The metric that matters is throughput per dollar, not time-to-first-token.

Mode 3, async online, sits between the two. A live session hands work to a job queue and returns immediately, while background workflows fan out LLM calls and post results later. This is the pattern behind Slack-triggered agents and background PR review, and it bills like batch, not like chat.

The cost model follows the latency requirement. Taking 1,000 documents at 30,000 input tokens each and about 500 output tokens per document, frontier API rates of $3 per million input and $15 per million output land near $97. Batched on a serverless GPU at around 3,000 tokens per second, the same work costs roughly $13. Conversely, leaving an interactive agent idle overnight on a Qwen3.6 35B model at $4.54 per hour for ten hours adds about $45 to the bill. Interactive work pays per token because a human is waiting; offline and async work pays per GPU-hour because throughput is the objective and idle time is the enemy.

A second axis compares serverless versus reserved capacity. Modal's pricing analysis shows reservations charge the peak rate for the whole contract while serverless follows the demand curve. When the peak-to-average ratio exceeds the reservation discount, serverless is cheaper. Modal reports typical discounts of 2–5× against peak-to-average ratios of 5–10× for inference, training, and agentic development, with reservation utilization often below 30%.