A School of Continuous Improvement paper finds a two-by-two grading quadrant crossing trial attendance with self-assessment accuracy predicts A-grade selection almost perfectly, and on-court ability barely at all.
A fourteen-club pre-season instrument sorts trialists by attendance and self-assessment, with almost no bearing on a blinded assessor's read of who can actually play.
Fourteen suburban netball clubs graded their pre-season trials this year against a new four-cell instrument, and across the metropolitan association’s 428 trialists, the top cell alone filled 92% of the A-grade side. Dr Sten Okwuosa, Senior Research Fellow and Convenor of the Impact Pathway Atlas in the School of Continuous Improvement, led the paper formalising the quadrant — crossing full trial attendance with whether a player’s own self-assessment matched the coaching panel’s independent rating — and testing it against an outcome nobody had checked it against before: a blinded assessor’s read of the same players at finals, six weeks on.
The paper, published today, names the quadrant’s four cells — Calibrated Regulars, Present Uncalibrated, Accurate Absentees, and Unplaced — turning an informal grading habit into a documented instrument other associations could adopt outright. The Calibrated Regulars cell’s relationship with the blinded assessor’s finals score proved statistically indistinguishable from none at all, and a companion test regrading trialists on attendance alone reproduced most of each club’s actual roster regardless.
Every grading instrument we build ends up measuring how well someone can describe themselves. This one at least admits it.
— Dr Sten Okwuosa, Senior Research Fellow and Convenor, Impact Pathway Atlas, School of Continuous Improvement
“The attendance sheet was already doing most of the sorting,” said Dr Joost Nwosu, Postdoctoral Fellow in the School of Continuous Improvement. “We’d like to know why the rest of the form felt necessary at all.”
The metropolitan association’s Grading Code of Practice has adopted the full quadrant as its sole grading instrument regardless, with no manual override sustained beyond a single trial period.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.eqbllv.