Evidence from real-world AI coding agent trajectories on the factors associated with lower cost, greater autonomy, and less rework.
Looking beyond the model
The mainstream discourse about AI coding effectiveness tends to focus on model selection. But it doesn't explain why teams using the same tools can see dramatically different results.
We studied the variables on the other side of the interaction: how clearly the developer specifies the task, and how ready the codebase and tooling are for the agent. Our research identified leading indicators associated with outcomes that matter: token efficiency, agent autonomy, and rework.
The encouraging part is that many of the biggest levers are already inside your organization, and they can be measured and improved today.
Scoring real-world AI coding activity
We define an effective agent trajectory as one that produces accepted code at lower token cost, yields more accepted output for each human turn, and requires less review rework.
To identify the leading indicators of those outcomes, Span developed a research-backed framework that scores complete developer-agent sessions across three categories.
Key findings are summarized below. Download the report for the complete analysis, methodology, and practical recommendations.
Three leading indicators of AI coding agent effectiveness
1. Clearer prompts are associated with cheaper code
Trajectories with clearer scope, constraints, and acceptance criteria consumed fewer tokens for each line of AI-generated code that ultimately shipped. Every one-point increase in prompt clarity was associated with a 27.2% reduction in cost per merged AI line. For a trajectory producing 50 merged AI lines, moving from a prompt clarity score of 3.0 to 4.0 reduced average token spend from $6.90 to $4.45.
Vague prompts create exploratory loops, incorrect implementations, and regeneration. Clear prompts narrow the search space before the expensive work begins, making prompt quality a measurable lever on AI cost.
2. Ready environments are associated with greater agent autonomy
Agents were more autonomous when they could build the repository, run reliable tests and checks, interpret failures, and verify their own changes. We measure this as turn yield: the amount of AI-authored code that survives to merge for each human turn.
Every one-point increase in environment readiness was associated with 88% higher turn yield. Across scores of 2.5 to 4.5, agents produced roughly 5 to 17 merged AI lines per human turn.
Autonomy is less a property of the model than a property of the workspace you give it.
Reliable setup, testing, documentation, and fast feedback loops help agents make more progress before returning to the developer.
3. Stronger quality stewardship is associated with less code review rework
Trajectories with stronger stewardship addressed local conventions, failure handling, verification, production constraints, and rollout safety before the pull request was opened. They also left behind the context reviewers and future maintainers needed to understand tradeoffs, shortcuts, technical debt, and important implementation decisions.
Every one-point increase in quality stewardship was associated with 39% fewer review cycles per 1,000 lines of AI-authored code merged. At a score of 3.0, our model estimates approximately seven review cycles per 1,000 lines. At a score of 4.0, that estimate falls to roughly four or five. When this work isn’t addressed during the trajectory, more of it surfaces later in code review.
The most important levers are already within reach
Model selection happens occasionally, but prompts, environments, and operator decisions shape every task. These findings point to improvements engineering teams can begin making today.
About the research
Span analyzed complete agent trajectories across real-world development activity, connecting the initial prompt, human and agent turns, tool use, resulting code, and linked review history.
We developed the framework through qualitative trace review, structured scoring rubrics, and automated evals calibrated against human ratings. We compared observable qualities of each interaction with downstream outcomes.
These findings reflect strong observational relationships, not proof of causality.