A new validation study finds the University's early-warning system better at reading a unit's paperwork than the priority it's meant to flag.
A study of 84 flagged priorities finds the Horizon Register's alert threshold tracks a unit's reporting habits more reliably than the priority itself
The School of Emergent Priorities has tested one of its own instruments against the record of what actually happened to the priorities it flagged, and found the instrument better at reading a unit’s paperwork than the priority itself.
The team drew on eighty-four priorities the Horizon Register had flagged since the start of last year, setting each one’s alert record against how it was eventually resourced. Two candidate explanations were compared side by side: how novel the priority was judged to be, and how often the responsible unit filed a status update. Only one tracked the outcome with any consistency: update-volume percentile predicted resourcing with moderate-to-strong reliability, while novelty percentile performed little above chance.
The Register was built to tell us what’s coming. It’s fair to ask what it’s actually measuring in the meantime.
— Professor Verity Marris, Director of the Trajectory Analytics Group in the School of Emergent Priorities
“The next version of the Register should be harder to satisfy with a status update alone,” said Dr Petra Umbile, Postdoctoral Fellow in the School of Emergent Priorities.
The School of Emergent Priorities reads the result as license to turn its own instruments inward as readily as it turns them on the rest of the University.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.4wzjf0.