Orchestrating Agentic AI: Budget, Policy, and Reality
Creating AI that plans and reasons on its own is the easy part. Keeping it inside budget, policy, and reality is where the real engineering lives. Field notes from building agentic LLM systems that plan their own steps.
Creating AI that plans and reasons on its own is the easy part. Keeping it inside budget, policy, and reality is where the real engineering lives.
I have been on a few AI projects lately that use LLMs to analyse user input and add a layer of artificial cognition — some creativity, some judgement, with the hallucinations trimmed down. They fall into two kinds. In the first, the input is open, so the model has to read it and define its own steps. In the second, the steps are known and the choices are limited and predefined, so the model just works inside a plan it was given. The second is far more forgiving. The flow below maps the harder case — where the model has to plan its own steps.
The diagram traces a single run end to end: an operator submits a task, the agent validates and classifies it, pulls relevant memory, and asks the model for a plan. Before any step executes, the plan is checked for feasibility, policy, and budget. Then the orchestrator works the plan step by step — deliberate, act, reflect, and a metacognitive check against the plan — with replanning, escalation, and a hard budget ceiling wired in at every turn. Below are the things I have learned so far.
The initial prompt carries more weight than anything else
The initial prompt defines everything that comes next and binds the whole context together. It is also easy to write one that quietly contradicts itself, and those conflicts come back as hallucinations and results you never asked for.
Most of the run’s behaviour is decided here, before a single tool is called. Time spent removing contradictions from the initial prompt pays for itself many times over downstream.
The plan will break — have a fallback ready
The plan will break, usually because of that same conflicting prompt. Have a fallback ready: resend with a healing request and let the model repair its own plan before you abandon the run.
A run that can repair itself is worth far more than one that halts at the first invalid step. Give the model a bounded number of chances to replan with context before you escalate or abort.
The admin layer is critical — and the biggest blocker
The admin layer is critical, and also the biggest blocker. You cannot foresee every problem, so log everything, watch where things go wrong, and feed that back into the prompts and safeguards.
This is the slow, unglamorous part of the work, and it is where the system actually earns its reliability. Every failure that reaches an administrator is a lesson to fold back into the next version of the prompt.
Cap the number of requests
Cap the number of requests you send to the model. Problems compound with each call, so route a failing run to admin to learn from it — without a hard limit it can loop more or less forever.
A hard ceiling on calls is not a nice-to-have; it is what stops a confused run from quietly burning your budget in a loop nobody is watching.
Keep a human in the loop
Keep a human in the loop, operator or administrator. Both are there to answer one question: is this the result we actually wanted?
When a run hits something no one foresaw, the human in the loop is what gets it solved. Do not underestimate how often that happens on real inputs.
Predict the budget
Track tokens spent and tokens still to spend — it is not hard to tell when a query will run past what you can afford. And wrap a safeguard around the API call itself, because disconnections happen and they hurt mid-run.
Budget prediction turns a vague worry into a concrete gate: you can stop a run before it overspends rather than discovering the overspend after the fact.
Where the real work lives
The orchestration is what carries a run and keeps it inside budget. But how much of it you need is decided upstream by the initial prompt — a clean one keeps the run calm, a conflicting one means the orchestration spends half its time cleaning up.
Autonomy is cheap to switch on. The engineering is everything that keeps an autonomous run inside budget, policy, and reality — and the human who steps in when it meets something no one planned for.