A new Slop University paper finds a counting-protocol trial's effect on self-reported participation figures does not repeat when applied inside a state government compliance scheme.
A School of Emergent Priorities trial finds the checklist that shrank a grants program's headcount leaves a seniors' voucher scheme's numbers unchanged
A hand tally counter and a one-page definition were enough, in an earlier Slop University trial, to cut a council grants program’s self-reported attendance figures by more than a third. A new paper from the School of Emergent Priorities asked whether the same counting fix does anything at all inside a different kind of paperwork — and reports that, this time, it does not.
Runa Adegoke, Torun Ezeigwe and Anouk Mensah tracked sixty-four providers inside a state seniors’ home-maintenance voucher scheme, where tradespeople self-report the households they assist each quarter for reimbursement. Half were issued the same style of counting checklist that shrank the council’s figures; the other half kept using the scheme’s standard claim form. The numbers the two groups reported came back statistically indistinguishable.
A finding that holds in one office doesn’t get to assume it holds in the next one just because the paperwork looks similar.
— Dr Runa Adegoke, Senior Lecturer and Convenor, Strategic Drift Survey, School of Emergent Priorities
“The providers we worked with were already closer to their true numbers than the council’s volunteer groups were,” said Dr Torun Ezeigwe, Senior Lecturer and Director of the Master of Applied Measurement. “That’s not nothing — it’s arguably the more useful result.”
The trial paired self-reported figures against an independent site-visit sample and ran an ablation removing the panel’s largest providers, finding neither adjustment moved the result. The University treats the null finding as part of the same evidentiary discipline that produced the original result — a marker, its researchers say, of a maturing institutional habit of checking its own claims before anyone else does. The scheme’s compliance office has already adopted the checklist as a mandatory requirement for its next reimbursement cycle, a decision made before the trial’s result was finalised.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.na9l9v.