# Show-society scoresheet rarely moves a produce judge's ranking

> A new paper from the School of Continuous Improvement surveys 47 show-society produce judges across one season, finding a sealed pre-scoring hunch predicts the finished scoresheet closely, and that requiring every rubric column doesn't change how often the two agree.

Slop University has published a new paper testing whether a show-society
scoresheet decides a produce class's placing, or simply writes down one a
judge has already reached. The paper comes from the School of Continuous
Improvement, whose Adaptive Metrics Lab and Evaluation of Evaluation program
have spent recent cycles asking the same question of the University's own
review panels.

Associate Professor Casimir Beng, Dr Dagny Okafor, and Dr Thandiwe Solberg
surveyed 47 judges across 19 regional show societies over one show season.
Before opening a class's scoresheet, each judge sealed their expected
placing and a confidence rating from the first walk along the bench; the
University then compared that sealed hunch against the completed rubric for
312 judged classes. First place matched exactly in 88% of them, and a
judge's stated confidence tracked the eventual scoresheet margin closely.

> A rubric filled in top to bottom can still just be recording what a judge
> already knew from the first look along the bench.
>
> --- Associate Professor Casimir Beng, Lead of the Adaptive Metrics Lab

"We keep building instruments to check whether our other instruments are
actually deciding anything," said Dr Dagny Okafor, Lecturer and Convenor of
the Evaluation of Evaluation program. "It's a small comfort to find one, for
once, outside our own buildings."

The team also tested whether requiring every rubric column, rather than a
quick total, changed how closely a judge's hunch and final placing lined up.
It did not: judges who completed the full scoresheet agreed with their own
walk-in call about as often as those who moved straight to a number.

The full paper is available from the University's research repository under
an open licence, doi:10.5555/slop.a7p1c3.

Source: <https://slop.university/news/2026-07-18-rubric-rarely-moves-the-ranking/>\
Licence: [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)\
Cite as: Swift, B. (2026). _Slop University_ [Artwork]. https://doi.org/10.5281/zenodo.21199776
