Anthropic study finds heavy AI-coding use can reduce skill formation in junior engineers

Anthropic researchers Judy Hanwen Shen and Alex Tamkin published a randomised study of how AI coding assistance affects the acquisition of new technical skills. Fifty-two software engineers, mostly junior and all with over a year of Python experience, were split into groups and asked to complete two coding tasks using Trio, an asynchronous-programming library none of them had used before; one group worked with AI assistance and the other without.

On an immediate comprehension quiz taken afterward, without AI assistance available, the hand-coding group averaged 67% correct against 50% for the group that had used AI — roughly a 17-point gap. The AI-assisted group produced higher-quality code during the learning phase itself, but performed worse on tests of independent understanding once the assistance was removed, and AI use did not produce a statistically significant speed advantage on the coding tasks overall. The researchers identified some patterns of AI use — those that kept the engineer cognitively engaged rather than fully delegating the task — that preserved learning outcomes despite the assistance.

The paper’s authors concluded that “AI-enhanced productivity is not a shortcut to competence,” and argued AI assistance needs deliberate management in workflows to protect skill formation, particularly in safety-critical domains. The study adds an empirical data point to a broader debate about whether heavy reliance on AI coding tools erodes the underlying expertise those tools depend on more senior engineers having — a question with direct relevance to Anthropic’s own business, given how heavily the company promotes AI-assisted coding through products like Claude Code.