School of Continuous Improvement researchers scored every team name submitted to one junior sport association against a panel-constructed reference standard, and found its proscribed-word list refuses 14.2 per cent of names while identifying 17.3 per cent of those the panel judged objectionable.
A signal-detection study of 4,116 team names submitted across five registration seasons, scored against a 24-member adjudication panel
It took 24,696 independent judgements to settle what a junior sport team name should be allowed to say. A paper released today by the School of Continuous Improvement puts them to a narrower question: whether the word list that screens those names can tell the difference.
The work treats the association’s registration portal as a detector and evaluates it as one. Every team name submitted across five seasons was scored twice, once by the portal’s match against the association’s list and once by an adjudication panel of parents, club officials and officials from associations elsewhere. The comparison separates two properties that are routinely reported as one: how much evidence a screen demands before it refuses, and how well it distinguishes the names a community would object to from the names it would not.
A screen can be made stricter without being made better, and from outside the two look alike. Separating them is most of what an evaluation is for.
— Dr Dagny Okafor, Lecturer and Convenor, Evaluation of Evaluation
The paper extends the School’s evaluation-ecosystem agenda into a setting the University has not previously instrumented, and anchors it on an object the sector has tended to treat as settled. Dr Solveig Adeyemi, who led the panel construction, said the association’s volunteers had been “generous with their time and candid about the work”, and that the School “expects to be back in settings like this one”.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.wjglb6.