RESEARCH · RESEARCH · #2416
Harvard–Jellyfish study: AI coding tools raise code volume but not completed work due to review bottlenecks
A Harvard study using Jellyfish data (≈300 million work events across ~700 firms, Jan 2021–Mar 2026) finds AI coding assistants modestly increase code output (lines +12%, commits +9%, PRs +5%; only commits statistically significant) while AI agents produce larger increases (lines +30%, commits +20%, PRs +23%). Despite more code, assistants did not increase resolved Jira issues and agents raised resolved issues by only 12%, with significant downstream impacts: average PR review time +49%, comments per PR +53%, share of PRs requiring changes doubled, and share of engineers performing reviews +14%—suggesting review and other complementary stages are bottlenecks.
KEY POINTS
- A Harvard study using Jellyfish data (≈300 million work events across ~700 firms, Jan 2021–Mar 2026) finds AI coding assistants modestly increase code output (lines +12%, commits +9%, PRs +5%; only commits statistically significant) while AI agents produce larger increases (lines +30%, commits +20%, PRs +23%).
- Despite more code, assistants did not increase resolved Jira issues and agents raised resolved issues by only 12%, with significant downstream impacts: average PR review time +49%, comments per PR +53%, share of PRs requiring changes doubled, and share of engineers performing reviews +14%—suggesting review and other complementary stages are bottlenecks.
- Shows that code‑generation tools can shift workload downstream (reviews, testing, deployment), so raw increases in code volume do not necessarily translate to more completed work or higher team productivity.
WHY IT MATTERS
Shows that code‑generation tools can shift workload downstream (reviews, testing, deployment), so raw increases in code volume do not necessarily translate to more completed work or higher team productivity.