A new Slop University research poster classifies bottle-shop staff by scanner-override rate and a covert compliance audit's result, finding the two barely track each other and that the resulting two-by-two catches what a single dashboard score cannot.
A two-by-two framework, tested against a covert compliance audit at 38 stores, finds a single override-risk number hiding two opposite kinds of staff.
Two staff members can override the same handheld ID scanner the same number of times in a month and land in opposite outcomes at the register — one waving through a fake, the other correcting a scanner that flagged a perfectly good card. A new Slop University research poster from the School of Emergent Priorities finds a bottle-shop chain’s own dashboard cannot tell the difference, because it was never built to look.
Dr Petra Umbile and Dr Lindiwe Achterberg set nine months of handheld-scanner override logs from 214 staff across 38 stores against an independent, unannounced compliance audit — 620 test-purchase visits by trained actors, coded pass or fail. Override rate and audit result turned out to be almost unrelated: of the 100 staff the chain’s dashboard flags as high-risk on override rate alone, the large majority passed the covert audit outright, while a meaningful share of the “low-risk” staff it never flags failed it. A store-level comparison told the same story: predicted risk rank and audited failure rank agreed on only two sites in ten.
A risk score built from one number is always secretly measuring two things and reporting on one of them.
— Dr Petra Umbile, Postdoctoral Fellow, School of Emergent Priorities
The team is proposing a two-axis scorecard as a next step, on the reasoning that a dashboard which can only ask “how often” will keep confusing the staff catching problems with the staff causing them.
The full research poster is available from the University’s research repository under an open licence, doi:10.5555/slop.nki2u5.