A new paper from the School of Emergent Priorities pre-registers a fifteen-sitting field test of heatmap-driven invigilator reseating and finds it moves attention without moving where integrity flags are raised.
A pre-registered field test across fifteen sittings finds the heatmap redistributes attention, not detection
The School of Emergent Priorities built its case for heatmap-guided invigilation on a simple premise: send attention to where the previous sitting’s integrity flags clustered, and the next sitting’s flags should thin out to match. A new paper tests that premise across fifteen sittings and reports it did not hold.
A heatmap earns its keep by moving someone’s attention. Whether it also moves what an invigilator actually catches turned out to be a separate question, and across fifteen sittings this one kept answering no.
— Dr Fenna Okoro, Senior Lecturer and Convenor, Horizon Register, School of Emergent Priorities
The project tracked the Applied Foresight Practicum’s return to in-person sitting, pairing seating and patrol assignments each week to a live “Sitting Heat Index” built from the sitting before it, then compared flag rates, zone-level flag counts, and invigilator dwell time against a reconstructed counterfactual in which seats and patrols had simply been drawn at random.
Flag rates held level before and after the index’s introduction, at 12.5 against 12.6 per sitting. Zone-level flags tracked invigilator dwell time far more closely than the zone’s own risk tier, and the randomised counterfactual reached statistically the same flag total as the deployed protocol.
Dr Lindiwe Achterberg, the paper’s co-author, said invigilators “adjusted to the index within a sitting or two and then mostly stopped noticing it, the way any rostering tool turns into furniture” — a description she noted the paper’s own title had already borrowed.
The School folds the null result into its evaluation-ecosystem agenda as confirmation that the audit habit holds even when a favoured instrument disappoints it, and is carrying the randomised counterfactual forward as a standing check for any future instrument proposed on similar grounds.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.l19xml.