A School of Continuous Improvement paper describes an arbitration engine that scores evidence in short-stay review disputes and finds its auto-publish threshold, lowered once to clear a backlog, has since become the platform's standing policy.
A confidence cut-off built to weigh evidence in booking-platform disputes was revised once, under load, and has not moved back.
A confidence threshold is supposed to be the honest part of an automated system: the number below which the machine defers rather than guesses. A School of Continuous Improvement paper reports what happens to that honesty once a backlog waits behind it. Dr Joost Nwosu and Dr Osei Vandermeer built an arbitration engine that scores the evidence behind a disputed short-stay booking review and auto-publishes a resolution once the score clears a pre-registered cut-off, and watched the cut-off move the one time it met real load.
Across fourteen weeks and three platforms, the engine absorbed 187 contested threads, weighing photo timestamps, message-thread history and prior-stay records into one corroboration score. Median resolution time fell from eleven days to under an hour for any dispute it could call outright. Disputes it could not call kept queuing for a moderator, and once that queue crossed four hundred threads in week nine, the platform lowered the calling threshold rather than the queue.
The lower threshold has not moved back. Its only review since appears as a line in a change log, with no scheduled comparison against the cut-off it replaced. For a School whose Review Cadence Observatory exists to catch exactly this drift, watching its own engine supply a fresh case reads internally as the system working as intended.
A threshold that moves under backlog and stays moved isn’t measuring confidence anymore, it’s measuring the queue.
— Dr Joost Nwosu, Postdoctoral Fellow, School of Continuous Improvement
Dr Osei Vandermeer, Research Fellow and Convenor of the Review Cadence Observatory, said a re-score against the original cut-off is scheduled for the unit’s next cycle, “the discipline we’d expect of any instrument whose settings drift unnoticed.”
The full research poster is available from the University’s research repository under an open licence, doi:10.5555/slop.91bdea.