A School of Emergent Priorities replication of an earlier rostering fairness-compliance instability finding instead returns near-perfect test-retest reliability at a 22-site aged-care provider, and traces the stability to how roster coordinators now complete the form.
A pre-registered repeat of an unstable compliance test returns near-perfect agreement at a 22-site aged-care provider, once staffing figures start rounding to the panel's pass line
An earlier School of Continuous Improvement study found a casual-shift rostering platform’s fairness-compliance score sliding under its own re-submitted data — the same fortnight, scored four different ways. A pre-registered replication from the School of Emergent Priorities, released today, transplants the same design onto a structurally similar instrument at a 22-site aged-care provider, and finds the opposite: the score barely moves at all.
The University counts the result among its clearest demonstrations yet of how quickly a number, once someone’s pass depends on it, stops behaving like a measurement and starts behaving like a target.
The team repeated the original design exactly: one fortnight of unchanged roster data, resubmitted four times across twelve weeks. Where the earlier work found the score drifting by close to ten points a run, this one returned a reliability figure the team describes as close to the ceiling the instrument can produce. Interviews with the site coordinators who supply the platform’s inputs suggest the stillness is authored well before the roster reaches the score.
A score that stops moving isn’t necessarily a score you can trust. Sometimes it just means everyone downstream has learned exactly where the line is.
— Professor Verity Marris, Director, Trajectory Analytics Group
“The coordinators weren’t doing anything wrong by their own lights,” said Dr Runa Adegoke, Senior Lecturer and Convenor of the Strategic Drift Survey. “A pass line with no discretion attached teaches everyone downstream exactly how much discretion they don’t have left.”
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.hpqz0a.