A School of Emergent Priorities and School of Continuous Improvement paper audits the Horizon Register's routing logs and finds a coverage-balancing rule concentrates novel-signal handling in five of twenty-eight scanners.
An audit of Horizon Register routing logs finds five of twenty-eight scanners handling most of the register's genuinely novel signals, with weekly coverage unchanged when the rule is removed
The School of Emergent Priorities keeps a standing watch on how a well-meant instrument in its own operations can quietly start deciding more than it was built to. Its newest paper, co-authored with the School of Continuous Improvement, turns that watch on the tool that keeps the Horizon Register’s weekly shortlist looking balanced.
Dr Fenna Okoro, who convenes the Horizon Register, and Dr Mirela Hanke audited 68 weeks of the Domain-Coverage Router’s assignment logs against a blind classification of each item as a genuinely novel signal or a renewal of something already tracked. A pool of five scanners, chosen by the Router for the breadth rather than the specificity of their declared expertise, absorbed 71% of the Register’s ambiguous items and most of what coders judged genuinely new, while specialist scanners’ queues filled with renewals. A counterfactual routing rule, tested as an ablation, reached statistically the same weekly coverage without the concentration, evidence the Router’s celebrated balance owes little to the mechanism credited with it.
The two Schools read the result as the kind of question a school built to study measurement owes itself: a coverage metric can hold steady while quietly reshaping who does the noticing beneath it.
We went looking for what the coverage number was leaving out, and found it was a roster. Five scanners out of twenty-eight have been doing most of the Register’s noticing, and the dashboard has no field for that.
— Dr Okoro
“A rule built to solve one problem has a way of quietly starting to decide a second nobody asked it to,” said Dr Hanke. “The next revision will need to notice that on its own, rather than waiting for an audit to point it out.”
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.vxfhl7.