A School of Continuous Improvement research poster finds a trades-booking app's half-star rounding line reallocating weekly bookings between otherwise identical providers, with no matching shift in an independently measured completion rate.
A regression-discontinuity test at a trades-booking app's half-star display line finds near-identical providers pulled apart by rounding alone
A trades-booking app rounds each provider’s average customer rating to the nearest half star before it ever reaches a search result, and a new Slop University poster finds that rounding line moving real bookings on its own. The School of Continuous Improvement compared providers sitting just either side of the line and found no matching gap in the completion and on-time records the display is meant to summarise.
A rounding line was never supposed to be a decision. Ours has been making one anyway, several thousand times a week, without anybody assigning it the job.
— Dr Osei Vandermeer, Research Fellow and Convenor of the Review Cadence Observatory
The team tracked several hundred tradesperson profiles across nine months, comparing weekly booking volume for providers whose raw rating landed a fraction either side of the app’s published rounding line, and set the pattern against an independently held completion-rate record the display does not draw on. “The completion-rate record didn’t move at all,” said Dr Dagny Okafor, Lecturer and Convenor of Evaluation of Evaluation. “Whatever the line is rewarding, it isn’t finishing the job.”
The School of Continuous Improvement treats the result as a working example of what its evaluation-ecosystem agenda has argued all along: an instrument seldom stays neutral once something depends on which side of it a provider lands. The School is now recommending the University’s own approved-contractor rating display adopt an identical rounding line, on the view that a reallocation this reliable is worth producing on purpose.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.rd0rkf.