A School of Continuous Improvement research poster compares a community theatre society's pre-opening line-check verdicts against backstage earpiece-prompter logs across 214 checks and six show openings, finding the panel far better at clearing the memorised than at catching the prompted.
A six-opening study finds the panel clears genuine memorisation with confidence but catches barely half of earpiece-prompted performances
A verification panel that is confident about only half of its job is still, on paper, a verification panel. That is the finding a research poster released today by the School of Continuous Improvement puts to a community theatre society’s pre-opening line-check: the ritual where each cast member runs their lines, off-book, in front of a panel deciding whether the performance is genuinely memorised or quietly carried by an earpiece prompter. Dr Torun Ezeigwe, Dr Renke Sabel and Dr Dagny Okafor set the panel’s live verdicts against the backstage record of who was actually wearing a prompter, across six show openings and 214 individual checks.
The panel’s instincts run in one direction only. A genuinely memorised performance is cleared with high confidence almost every time; an earpiece-prompted one is caught barely more often than not, and agreement between panel and log lands at a moderate κ = 0.55 overall. The team reads the asymmetry as evidence that a fluent recital, not real scrutiny, is carrying most of the panel’s judgement.
A panel that trusts its own ear is exactly the kind of instrument our program exists to look at twice.
— Dr Dagny Okafor, Lecturer and Convenor, Evaluation of Evaluation, School of Continuous Improvement
Dr Ezeigwe, whose earlier work traces how a marking rubric drifts once a cohort learns to the test that teaches it, called the panel’s asymmetry “what a criterion looks like right before someone starts performing it back.” The finding extends the School’s standing interest in verification instruments that are rarely asked to verify themselves.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.xewf3l.