The Debugging Gap: The Skill Your AI Business Case Assumes You Still Have

In January, The Verification Premium made a prediction. The value of software expertise would not fall as AI wrote more of the code; it would concentrate in the people capable of overseeing what the machines produce, and the route through which that capability forms, junior developers learning by debugging and review, was at risk. In July, The Balancing Item priced the other side of the same prediction: every AI business case counts the hours saved, while the hours spent reviewing, correcting, and supervising AI output go largely unaccounted for. The evidence published since connects the two arguments, and it reveals something neither anticipated on its own. AI coding tools produce code that someone must check, yet the same tools are now doing the debugging and review through which developers learn how to check it. Demand for verification is rising while the route for building it weakens. Those two movements arrive as one problem, and it belongs to Boards, not to engineering management.

What the evidence now shows

The most precise finding comes from Anthropic’s randomised controlled trialdoi.org[2601.20245] How AI Impacts Skill FormationAbstract page for arXiv paper 2601.20245: How AI Impacts Skill FormationOpen link , published in January. Fifty-two Python developers, all unfamiliar with the Trio library, learned it with or without AI assistance. The group with AI scored 17% lower on the subsequent comprehension quiz, while finishing no faster to any statistically significant degree. When the researchers broke the quiz down by question type, an analysis they had not planned in advance, the largest gap appeared on the debugging questions. The study is bounded: one hour, one library, an immediate quiz, and its authors identify longer-term effects as an open question. What it shows is a mechanism, not a destiny, and the qualitative analysis points the same way. Participants who asked conceptual questions and kept implementing code themselves scored well, while those who delegated wholesale to the AI scored worst, which suggests that how people used the assistance mattered at least as much as whether it was available.

A second experimentarxiv.org[2602.20206] Mitigating "Epistemic Debt" in Generative AI-Scaffolded Novice Programming using Metacognitive ScriptsAbstract page for arXiv paper 2602.20206: Mitigating "Epistemic Debt" in Generative AI-Scaffolded Novice Programming using Metacognitive ScriptsOpen link takes the mechanism a step further: 78 novice programmers built software manually, with unrestricted AI, or with AI constrained by scaffolds designed to keep the learner reasoning. The two AI groups produced comparable functional output during construction, though the scaffolding cost additional time. Then the tool was removed for a designed 30-minute maintenance task. 77% of the unrestricted group failed it, against 39% of the scaffolded group. The precise claim matters: they failed one designed maintenance test without the tool, at nearly twice the rate of their scaffolded peers. The study is a preprint with novice participants, and its author cautions that the result may not generalise to experienced developers. Within those bounds, it is the clearest demonstration yet that building working software and being able to fix it are different capabilities, and that a developer can visibly have the first while quietly lacking the second.

The experiments show one pressure: what AI assistance does to learning inside the programming profession. Labour-market data from the United States and the United Kingdom shows a second: fewer young people are being hired into the profession in the first place. The Stanford Digital Economy Lab’s revised analysis, published this month, finds no evidence of widespread displacement. It does find that employment among workers aged 22 to 25 in the occupations most exposed to AI, the jobs where AI tools can already perform many of the core tasks, software development prominent among them, now stands about 19% below where it would be had it kept pace with less-exposed peers of the same age. The gap comes mainly from reduced hiring rather than from people losing their jobs, and older workers in the same occupations show no comparable shortfall. The authors present these as descriptive indicators rather than causal estimates. The UK Government’s entry-level hiring snapshotgov.ukEntry-level hiring in the UK: a snapshot - GOV.UKAn analysis of UK entry-level hiring trends, exploring how labour market conditions, skills mismatches and AI are shaping opportunities for new workers.Open link , built on LinkedIn data, shows entry-level hiring falling broadly in line with a weak market overall, but entry-level Software Engineer hiring down 27%, among the steepest declines of any tracked occupation, with the report’s own caution that the pattern is consistent with an AI effect without proving one.

At system level, the review burden is already visible. Faros AI, a commercial engineering-analytics provider, reports from customer telemetry across more than 10,000 developers that teams with high AI adoption merged 98% more pull requests while review time rose 91%, without a corresponding company-level improvement in delivery measures. The finding is correlational and comes from a vendor with a product in this market, but its direction matches what the experiments predict: more generated code means more verification work. A 67-source reviewarxiv.org[2605.01160] The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software DevelopmentAbstract page for arXiv paper 2605.01160: The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software DevelopmentOpen link published in May reaches a similar synthesis, finding that task, codebase maturity, and developer experience explain much of the apparently contradictory evidence. It draws on several of the sources above, so it stands as a synthesis of the evidence rather than a further independent strand of it.

One earlier result deserves a footnote rather than a headline. METR’s 2025 trialmetr.orgMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer ProductivityRandomised controlled trial finding experienced developers 19% slower with AI coding tools, the clearest measurement yet of the verification premium.Published 10 July 2025Open link Archived copy found experienced developers slower with AI tools while believing they had been faster, and although METR’s February update judged its follow-up data unreliable and current tools likely faster, the calibration lesson survives for any Board pack: developers cannot accurately self-assess AI-assisted performance.

None of this proves that the engineering capability of established organisations is in decline; nobody has yet measured that over time. The experiments were short and small, the delivery statistics show association rather than cause, and the hiring data records a pattern without explaining it. What the evidence does show is enough: a credible way for debugging skill to stop forming, visible pressure on the number of people entering the profession, and review work growing as fast as the code it must cover.

The Debugging Gap

Debugging is the clearest operational test of corrective competence: whether someone can detect that code is wrong, diagnose why, and repair it without delegating the entire judgement back to the system that produced it. Architecture, security reasoning, requirements, and testing all matter to oversight. Debugging is where corrective competence shows itself most directly, which is why both experiments converge on it. A developer who can debug can read AI-generated code, identify where it is likely to fail, and intervene. A developer who cannot can accept the output, reject it, or ask the same tool to try again. That may still produce working software. It does not provide independent oversight of the result.

The problem itself is older than AI. Every engineering team has lived it: a developer leaves, a successor inherits code they did not write, and progress depends on how quickly the newcomer can read it, find where it fails, and change it safely. What AI changes is the scale. Code nobody on the team wrote now arrives continuously rather than at the occasional handover, and the debugging work that once taught developers to read another author’s code is the work the tool has taken over.

The two pressures the evidence describes, weaker learning inside the profession and fewer people entering it, would each alone be a workforce question. Together they are a governance question, because debugging is the capability on which every AI coding business case silently depends.

Developers have used the phrase informally for some time. Defined precisely, the Debugging Gap is the widening distance between how much AI-generated code needs checking and repairing, and how many people can do that work. That definition turns the problem into something a Board can govern. The demand side is visible in its own delivery metrics. The supply side is a question it can put to its own technology leaders.

The mode of use is the governance variable

The strongest finding in the Anthropic trial is not an argument for banning the tools. Outcomes diverged with the manner of use, not the fact of access: participants who used the assistant as an explainer built understanding, while those who used it as a substitute did not.

The companies building these tools can see the same trade-off. Anthropic, whose research supplies the sharpest evidence in this article, now ships learning modes in its own coding tool, designed to preserve understanding alongside output: the organisation that measured the trade-off is designing around it. The question for the Boards I meet is whether their own deployment guidelines reflect the understanding the tool-makers have already priced in.

The evidence supports differentiated governance rather than a uniform posture. The May review identifies the nature of the task, the maturity of the codebase, and the experience of the developer as the variables that matter, and the practical implication follows directly. Greenfield work by junior developers acquiring new skills requires a different posture from senior engineers applying established expertise to well-defined problems. A single policy applied uniformly will either over-constrain productive use or under-protect the learning environments that produce future oversight capability. The governance variable is how the tool is used, not whether it is available.

Three questions, put to technology leaders rather than issued as directives, test where an organisation stands. Do the deployment guidelines distinguish between AI-assisted execution and AI-assisted learning? Is the junior developer programme building debugging capability, or distributing tool access? And is the organisation measuring the quality of what its AI-assisted teams produce, or only the volume?

Board governance implications

When a Board approves an AI coding tool investment, it approves a position on skill development, whether or not the paper in front of it says so. The Verification Premium made that point in January as an argument. The evidence now available makes it difficult to treat as speculative.

The Institute of Student Employers’ 2026 surveyise.org.ukEntry-level work reshaped not replaced | ISEISE Student Development Survey 2026 launches with new insight on how AI is impacting entry-level jobs and the skills employers need.Open link of 144 employers found 87% expecting AI to reshape graduate and apprentice roles, and 43% reporting that their entry-level roles had already evolved in response to AI without being formally redesigned. The change is entering the work before it enters the role architecture, and it lands on precisely the tasks through which early-career capability has historically formed: the routine debugging, the small repairs, the code reading. When those tasks move to the tool without the role being redesigned around what remains, capability formation changes by default rather than by decision. That is why this subject sits above the technology function. Role design under AI is workforce strategy, and workforce strategy belongs to the Board.

The True Investment Profile showed that AI business cases routinely undercount the cost of governing what AI produces, and the Balancing Item priced one of those costs: the human work of oversight. This article adds what both took for granted, the people doing the overseeing, whose supply depends on how AI is deployed while developers are still learning. The strategic question from January stands: not whether AI can write code more cheaply, but whether the organisation still has people who can tell whether that code creates value or debt. The evidence now adds a third: is the same deployment that budgets for oversight also removing the people able to provide it?

What belongs on the ledger

Boards do not have to choose between adopting AI coding tools and protecting skills. The researchers behind the sharpest evidence recommend designing AI assistance so that productivity and learning develop together, and their experiments show it can be done. Organisations that deploy with that care will take the gains and keep the ability to check what the tools produce. Organisations that do not will meet the problem later, in a larger codebase with fewer people who understand it.

The Balancing Item put the cost of overseeing AI output on the ledger. The Debugging Gap puts the question beside it: whether the organisation will still have the people to do the overseeing. Both belong on the ledger. The Debugging Gap is an operating exposure, not a future workforce problem.

The Questions Considered