A new Slop University paper validates an automated centring-skill score against blinded expert judges across a community ceramics studio's hand-taught and jig-assisted beginner cohorts, and finds it holds for one and not the other.
A four-term study at a community ceramics studio finds the score tracking expert judges for hand-taught beginners, and losing them for jig-assisted ones, at the same final assessment
Grading a hand-skill from a video clip is the easy part of automating a classroom; confirming the grade still means the same thing once some students reach the skill by a different route is the harder one. A new Slop University paper reports what happened when that harder question met a beginner pottery course.
Lecturer and Convenor Dr Ingeborg Nwachukwu and Senior Lecturer Dr Thandiwe Solberg, both of the School of Continuous Improvement, built the Wobble Score, an instrument rating a beginner’s centring technique from a short video of the clay’s spinning rim, and checked it against two blinded instructors’ ratings across four terms of a ceramics studio’s parallel beginner sections. One section learned to centre by hand; the other used a clip-on centring jig for six weeks before both sat the same jig-free final assessment.
Every instrument gets validated once, against the cohort that happened to be in the room. We wanted to know what ours was actually measuring once a second cohort walked in.
— Dr Ingeborg Nwachukwu, Lecturer and Convenor, Capability Sprints, School of Continuous Improvement
The score tracked the instructors’ judgement closely for hand-taught finalists, and only loosely for jig-assisted finalists, a gap that held across all four terms and survived a deliberate attempt to fix it by reweighting the score itself.
“We came away less sure the score measures centring, and more sure it measures how a student got there,” said Dr Solberg. “That’s not the same instrument we thought we’d built.”
The School of Continuous Improvement reads the result as a reminder about its own instruments as much as the studio’s: a measure that behaves once is not yet a measure that behaves for every route to what it grades.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.k8ffv4.