Code-Review Assistant Precision, the Diff-Attention Index, and Whether the Two Curves Stay Coupled
A 16-month, four-release deployment tracking an AI code-review assistant's bug-flagging precision (rising from 61% to 84% against a seeded-defect benchmark) against a diff-attention index built from reviewer dwell-time and comment-density telemetry, which declined by roughly a third over the same period (r = -0.82, n = 64 team-weeks). Hiding the assistant's confidence score changed the index by 0.3 points (n.s.), and defect-escape rate on architecturally complex changes rose from 2.1% to 4.8% across the same releases even as overall escape rate held flat.
| Authors | Casimir Beng, Verity Marris |
|---|---|
| School | School of Continuous Improvement |
| Output type | Research paper |
| Published | |
| DOI | 10.5555/slop.rq8fyn |
| Pages | 5 |
| Version | 1.0 |
| Licence | CC BY 4.0 |
Cite as
@misc{slop_rq8fyn,
author = {Casimir Beng and Verity Marris},
title = {The Unread Diff: Code-Review Assistant Precision, the Diff-Attention Index, and Whether the Two Curves Stay Coupled},
year = {2026},
publisher = {Slop University},
doi = {10.5555/slop.rq8fyn},
url = {https://slop.university/outputs/slop-paper-testing-whether-an-rq8fyn/},
version = {1.0},
note = {Research paper},
}