A new impact report from the School of Continuous Improvement pairs logged AI-assistant use across University systems against an independently audited measure of workload, finding the two move together in only one of four areas examined.
A five-year impact report pairs logged assistant use against independently audited workload change, and prints the gap where they disagree
The School of Continuous Improvement’s newest impact report finds that a rising count of logged AI-assistant sessions across University systems has not, on the whole, been matched by a comparable fall in audited task hours — and in one area, professional-services assurance work, moved in the opposite direction.
Every dashboard tile we’ve added for the assistants carries a twin figure we don’t control quite as easily. We’ve learned we trust the twin more than the tile, and we’re publishing accordingly.
— Associate Professor Casimir Beng, Lead, Adaptive Metrics Lab
The report, covering 2021 to 2026, sets a logged-usage index against an independently audited workload figure across four impact areas anchored in the Living Dashboard, the Impact Pathway Atlas, Evaluation of Evaluation, and the Adaptive Metrics Lab’s forecasting work. In three of the four, the audited figure held flat or moved only modestly while the logged count kept climbing; in the fourth, a leading-indicator model built to forecast the shift underestimated it substantially in the fastest-adopting unit.
Dr Mirela Hanke, whose team built the report’s central dashboard-versus-audit comparison, said the finding “raises an obvious question for every other tile on the wall, not just the one we happened to check first.”
The method, in outline, sets each system’s own usage count against an independently gathered measure of the work it was meant to lighten, then prints the two side by side rather than folding one into the other. The School reads the pairing not as an embarrassment but as the clearest demonstration yet of the discipline its five-year retrospective set out to model.
The full impact report is available from the University’s research repository under an open licence, doi:10.5555/slop.iqrcc0.