A new Slop University paper tracks volunteer ground curators' storm-call overrides against a radar app's own recommendation over a season at 42 junior-sport ovals, and finds the override fading despite tied accuracy.
A season-long study across 42 junior-sport ovals finds the app's own accuracy no better than the sky-read it's replacing
A recommendation stops being a recommendation the moment nobody is still willing to say no to it. The School of Emergent Priorities spent a season watching that willingness fade at forty-two junior-sport ovals, one Saturday-morning storm call at a time.
Senior Lecturer Ronja Oyelaran, Professor Verity Marris, Director of the Trajectory Analytics Group, and Dr Osei Vandermeer, Convenor of the Review Cadence Observatory, logged what a storm-radar app recommended, what the ground curator on duty actually called, and what the weather went on to do, fixture by fixture, across 512 decisions over a fourteen-week season.
Curators overrode the app’s recommended call in fewer than one decision in ten by the season’s final fortnight, down from more than half at the start, even as their own calls and the app’s matched the morning’s verified weather about equally often throughout. Replacing the app’s numeric confidence score with a plain yes-or-no flag on a subset of ovals left the override rate exactly where it already was.
The override was never going to disappear because the app got something wrong. It disappeared because nobody could point to the Saturday it stopped mattering.
— Senior Lecturer Ronja Oyelaran, School of Emergent Priorities
“We’d like to follow the same curators past this season and see whether the sky-read comes back once the number stops feeling new,” said Dr Osei Vandermeer, Research Fellow and Convenor of the Review Cadence Observatory.
The University treats a volunteer’s changing confidence in a tool as exactly the kind of quiet handover its research programme exists to catch before it’s finished.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.1dz0cg.