# A driving school's authenticity score still trusts the wrong half

> A new Slop University paper validates a driving-school network's composite Practice Hour Authenticity Index against instructors' independent judgement, finding the oral-quiz component agrees far more closely than the motion-sensor tally it is blended with.

> A composite score is only as good as whichever ingredient it happens to
> trust. Ours, it turns out, has been trusting the wrong one.
>
> --- Associate Professor Casimir Beng, Lead of the Adaptive Metrics Lab,
> School of Continuous Improvement

A driving-school network's Practice Hour Authenticity Index blends two
things into one number: a phone's own motion-sensor reading of a logged
practice drive, and a short oral quiz asking the learner to describe one
drive at random. A new Slop University paper puts the blend, and each half
separately, against what the network's own instructors already believed.

The gap is not close. Checked against 34 instructors' independent judgement
across 212 learners and 316 quizzed drives, the motion-sensor reading agreed
with instructors only slightly (Cohen's kappa = 0.11); the oral quiz agreed
with them moderately well (kappa = 0.52). The index the network actually
runs, weighted 70% toward the sensor, recovers to kappa = 0.18 --- closer to
its weaker half than its stronger one.

"The quiz was never the expensive part of this system," said Dr Mirela
Hanke, Postdoctoral Fellow, School of Continuous Improvement. "It was just
the part somebody had to be on the phone for."

The finding lands squarely inside the Adaptive Metrics Lab's standing
brief: instruments that measure what is cheap to collect rather than what
actually agrees with the judgement they are meant to stand in for. An
ablation stratifying by instructor years of experience left the pattern
unmoved.

The full paper is available from the University's research repository under
an open licence, doi:10.5555/slop.8c99s8.

Source: <https://slop.university/news/2026-08-09-a-driving-schools-authenticity-score-still-trusts-the-wrong-half/>\
Licence: [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)\
Cite as: Swift, B. (2026). _Slop University_ [Artwork]. https://doi.org/10.5281/zenodo.21199776
