A new Slop University paper builds a cheap sensor rig to shadow a ten-pin bowling alley's proprietary scoring system for a season, then tracks whose call the league's rules committee believes when the two disagree.
A season-long comparator audit finds the two systems agree closely on clean frames and only fairly on disputed ones --- and the league's rules committee still upholds the incumbent in 91% of disputes
For a full ten-pin bowling league season, a homemade sensor rig assembled from off-the-shelf parts sat beside a bowling alley’s own proprietary lane-scoring and foul-detection system, calling every frame the incumbent called, so a new Slop University paper could measure not just how often the two agreed, but what happened whenever they didn’t.
The finding lands as the School of Continuous Improvement’s latest test of what legitimacy a measurement earns simply by being the system already installed.
Associate Professor Casimir Beng, Dr Osei Vandermeer and Dr Renke Sabel logged the comparator rig’s calls against the incumbent system’s across a full season of league play, then set both records against an independent video review whenever a frame was formally disputed. Routine, clean frames returned near-identical calls from both systems; disputed frames returned only fair agreement, with the comparator and the incumbent about equally likely to match the video review’s own read.
What decided a disputed frame, in practice, was rarely the video review. The league’s rules committee upheld the incumbent system’s original call in 91 per cent of formal disputes, regardless of which system the review sided with.
Whether a call should out-rank an existing system, once the two agree this often, is not a question our comparator rig can answer by itself — but it is now one the committee has to sit with.
— Associate Professor Casimir Beng, Lead of the Adaptive Metrics Lab
“A comparator only tells you two systems disagree,” said Dr Osei Vandermeer, Research Fellow and Convenor of the Review Cadence Observatory. “It doesn’t tell a committee which one to believe, and ours evidently didn’t change that.”
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.m0pulh.