Instrument then Automate
Tracking Infrastructure First, Then Deployment.
The order matters, and it is easy to get backwards.
The pattern looks like this. A model gets wired into a real workflow. Support triage, document intake, first-draft anything. It works in the demo, it works in the pilot, it goes to production. Six months later the team wants to widen its scope, and nobody can say whether that is safe. Not because the system is failing loudly, but because there is no record of it succeeding.
Scope is the visible version of the problem. The quieter one is drift. An agent holding a fixed job can still get worse underneath you: a prompt gets edited, a model version rolls forward, the upstream data shifts shape, a dependency starts returning something slightly different. None of that announces itself. Output keeps arriving, formatted correctly, at the usual rate. The same blindness that makes expansion unsafe makes degradation invisible, and the second one does not wait for anyone to ask permission.
Costs move the same way. Retries creep up, context grows, a provider reprices, and the per-attempt number drifts without a single line of code changing.
Those questions have answers. They just were not being written down.
What uninstrumented means
An uninstrumented agent is one where the only durable artifact of a run is its output. The inputs, context, evaluator verdict, and subsequent human corrections are gone. Whether a human corrected it afterward lives in someone’s inbox, if it lives anywhere.
That can be a reasonable way to build a throwaway prototype. It stops being sufficient the moment the system enters a real workflow. A pilot without instrumentation may demonstrate that the system can work, but it cannot produce the evidence needed to govern what happens next.
Every request to expand scope is a request to extend trust, and trust runs on evidence. Without it you get an argument that most teams running an agent in production have had by now. One side says we cannot let it act unsupervised. The other says then it is not worth running. Both positions are reasonable, and neither can win, because the disagreement is empirical and there is nothing on the table to settle it with.
The asymmetry
The first objection is a good one: we do not know what to measure yet, so instrumenting now means measuring the wrong thing.
Probably true. Instrument anyway.
If the scaffolding exists and the metric is poorly chosen, fixing it is a configuration change. Swap the grading dimension, rerun against the corpus that has been accumulating, and you have your answer that afternoon. If the scaffolding does not exist, no config change gets you there. It is a build, and the historical data you would want to evaluate against does not exist either, so the clock starts at zero even after you finish.
The distance between a mediocre metric and a good one is short. The distance between no instrumentation and some instrumentation is the whole trip.
That is the case for the order. Instrumentation is a precondition for automation rather than a reporting layer added once the automation works. Autonomy has to be earned against data, and data has to already be there.
What PAA is
Progressive Autonomy Architecture is a governance framework for agent autonomy. Evidence accumulates, oversight relaxes, and confirmed failure restores it.
That is the whole idea, and it is not a novel one. It is a specified version of what teams tend to build by hand a few incidents in, when there is less time to think about it than they would like.
What the evidence has to cover is two things, not one.
Behavioral fitness is whether the agent is doing the job correctly: does the output hold up, does it stay inside its rails, does a human have to fix it and by how much. This is the part everyone reaches for first.
Economic fitness is whether it is worth what it costs. Token consumption, retries, tool calls, storage, human review, and the price basis those were charged against. An agent can be behaviorally excellent and still not worth running, and that is a governance question rather than a finance one, because the answer determines whether you widen its scope or pull it back.
Most instrumentation efforts capture the first and ignore the second, which leaves you able to prove an agent works and unable to say whether it should keep running.
PAA is not merely an evals framework. Evals answer whether an output is good. PAA uses that evidence to answer a different question: what is this task currently allowed to do without a human in the loop? Evals produce evidence. PAA turns that evidence into operating authority.
Behavioral evidence determines whether a task is eligible for greater autonomy. Economic evidence determines whether exercising that autonomy is worthwhile. PAA records both without allowing a favorable cost profile to override a safety gate.
Concretely, the spec covers what an evidence record contains, how governance rules scope windows over that evidence, what an operating record carries for usage and price, and how eligibility is computed from all of it. Usage is stored in the operating record, while effective cost is calculated against a versioned pricing catalog. That keeps historical runs reproducible even when provider prices change.
The spec is published in full. If a piece of it is wrong, it is specific enough to argue with.
Who this is for
Teams that already have the discipline. You run tests, you have alerting, you would not ship a payments change without a rollback plan. Agents are new enough that the equivalent tooling has not settled yet, and the gap is not obvious from the inside, because an agent that is quietly drifting looks exactly like one that is fine.
The discipline transfers. The order is the same as it has always been. Instrument first, observe real behavior, and let the evidence decide what the system is allowed to do next.
Instrument, then automate.
The spec is at paa.dev.