From a model call to systems that retrieve, act, and delegate.
Simple API and tool demos
Senior AI engineer & technical lead. Master’s in Autonomous Systems.
Head of Development at an AI education startup, 2023–2026. Part of my work was building agents.
Today: the architecture behind LLM applications, from one request to coordinated work.
| When | A useful landmark | What it illustrates |
|---|---|---|
| Before 2023 | RAG research (2020); ReAct (2022) | Retrieval and reasoning/action loops predate the wrapper boom. |
| 2023 | Chat-style applications; function calling; AutoGen | Model APIs become a common starting point. Multi-agent experimentation is already happening. |
| Late 2024 | MCP is introduced | A shared protocol for tools and context across applications. |
| 2025–2026 | Composing these capabilities | Runtime controls, context, reusable integrations, and coordination work together. |
A familiar starting point
A UI collects input. Your application adds instructions, calls a model, and displays the answer. Application-specific information must be supplied in the request.
This example is stateless. The application must supply history.
The next question: how does the answer become useful for your documents, your users, and your current data?
What the application wants the model to do.
Relevant passages, records, or search results, with their sources.
The history needed to understand this request.
Retrieval-augmented generation: retrieve information and use it to ground the answer.
| Need | Application responsibility |
|---|---|
| Relevant evidence | Select useful passages; keep source references. |
| Room to work | Budget for instructions, history, tool results, and the answer. |
| Long conversations | Select, trim, or summarize history; preserve important facts. |
| Continuity across sessions | Persist useful state and retrieve it when needed. |
More context can help. Irrelevant or excessive context can also hide what matters.
Function calling · no framework yet
Ask the model, give it the tool schemas alongside the messages.
If it did not ask for a tool, it answered, and we are done.
Otherwise we run the tool on our side, and hand the result back as a message. The model never runs a tool you defined.
Then we go around again. The model chooses the next step from the result. This is a minimal agent loop.
# this loop is the agent for turn in range(1, 9): resp = client.chat.completions.create( model=args.be.model, messages=messages, tools=TOOLS, ) msg = resp.choices[0].message messages.append(msg.model_dump(exclude_none=True)) # no tool wanted → it answered if not msg.tool_calls: break # we execute, not the model for call in msg.tool_calls: result = dispatch(call.function.name, call.function.arguments) messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
The ceiling
Each external service needs an adapter: a tool description, arguments, execution, and results. Different applications may repeat that work.
Shared libraries help. A shared protocol lets compatible applications reuse an integration through the same interface. That is where MCP fits.
Model Context Protocol
The host discovers tool schemas through MCP and exposes selected tools to the model. The decision/action/result loop stays the same.
The server implements the integration. Compatible applications can reuse it. MCP also supports resources and prompts.
A useful analogy. More precisely: MCP is a standard interface for LLM applications to discover and use tools and context, often backed by existing APIs.
| MCP provides | The host / runtime decides |
|---|---|
| Tool discovery and invocation | Which tools enter the model’s context and which calls are allowed. |
| Resources and prompts | When to include data or instructions. |
| A local or remote connection | Credentials, scopes, isolation, and connection policy. |
Tool search and deferred loading can avoid putting every schema into every request.
User input
ProviderConnects the model
Hooks + permissionsInspect / allow / block
After-tool hook
Next call
↑ Final answer → user
SteeringNew guidance → context
| Term | Role in the system |
|---|---|
| Agent | A model-driven loop pursuing a task using context and available actions. |
| Harness / runtime | Runs the loop and manages execution, state, limits, and permissions. |
| Framework / SDK | Reusable code for building agents, runtimes, and workflows. |
| Skill | Task instructions and supporting resources, loaded when relevant. |
| Plugin | A host-specific package of extensions, such as tools, skills, or integrations. |
What this model call can see: instructions, selected history, results.
Information retained for later work and retrieved when relevant.
What was requested, what completed, and where to resume.
The application decides what persists, what is shared, and what each agent receives.
Different instructions, tools, or access for different responsibilities.
Each specialist receives the material needed for its assignment.
Independent investigations can proceed at the same time.
Each specialist is another agent loop. Multiple agents may use the same underlying model - but they don't have to
| Pattern | Control | Tradeoff |
|---|---|---|
| Agents as tools | A parent delegates bounded tasks and uses the returned results. | Simple ownership; parent context and synthesis can become bottlenecks. |
| Graph / workflow | Explicit dependencies, conditional branches, and possible loops. | Clear structure; routing and state still need careful design. |
| Handoffs / swarm | Agents can choose which specialist takes over next. | Flexible routing; repeated handoffs can waste work and budget. |
| Step | What moves | Who controls it |
|---|---|---|
| 1 · Assign | Question, relevant context, expected result, allowed tools | Parent chooses tasks; runtime enforces access. |
| 2 · Investigate | Tool requests → execution → evidence | Each specialist runs its own loop. |
| 3 · Return | Findings, source references, uncertainty, blockers | Specialist returns a result to the parent. |
| 4 · Decide | A proposal with its evidence, and what it would cost | Dependent work starts after the findings. |
| 5 · Act or escalate | The refund, or a human approving it | Runtime gates the one consequential call. |
| Cost | Design response |
|---|---|
| Repeated context and model calls | Delegate bounded work; pass only relevant information. |
| Lost details in handoffs | Use explicit result formats, evidence, and artifact references. |
| Conflicting edits or decisions | Define ownership; isolate writes; reconcile changes. |
| Loops, failures, cancellation | Enforce shared budgets, timeouts, and clear completion rules. |
Compare against a single-agent baseline: quality, elapsed time, cost, and failure rate.
Did it solve the task? Use checks and a clear quality rubric.
Were actions permitted, required steps completed, and failures handled?
Measure repeated runs, long sessions, latency, and total resource use.
Test runtime rules cheaply. Evaluate task success with the deployed model and configuration.
| Task shape | A reasonable starting point |
|---|---|
| A bounded transformation | One model call. |
| An answer grounded in your data | Retrieval plus generation. |
| A known sequence of steps | An explicit workflow. |
| Actions depend on what is discovered | An agent loop with scoped tools. |
| Separable work or distinct responsibilities | Coordinated agents, evaluated against a simpler baseline. |
Tools execute actions. MCP standardizes access to integrations.
The harness manages the loop. Skills and plugins supply reusable capabilities.
Multiple loops exchange assignments and results under an orchestration policy.
The architecture grows around the work you need it to do.
What does the model need to know? What may it do? Who decides what happens next?
Ali Karpuzoglu · watermelonson.com