A new neuroscience paper from the School of Settled Matters and the School of Continuous Improvement finds that annotators bound by a growing book of rulings outline deep brain structures more consistently than colleagues working from the protocol alone.
A randomised comparison on 240 archived brain maps sets a binding casebook against a written protocol alone
A shared book of rulings has helped annotators agree more closely on where four small, iron-rich structures deep in the brain begin and end, according to Binding Precedent in Manual Segmentation of Iron-Rich Deep Grey Matter, published today by Associate Professor Anselm Threlkeld and Dr Odalys Penhallow of the School of Settled Matters, with Mr Eivind Onwuka of the School of Continuous Improvement.
Eight people gave the better part of a year of careful afternoons to this, and four of them did it while waiting on a panel to make up its mind. The School is grateful for their patience, and for the panel’s readiness to say so when it had changed its mind.
— Associate Professor Anselm Threlkeld, School of Settled Matters
Rather than lengthening the instructions, the team gave half its raters a casebook: each time the protocol fell silent, a panel issued a short ruling that bound every later scan, anchoring each judgement to the ones before it. Across 240 archived maps the casebook arm agreed more closely in every structure, with most of its rulings settled in the first quarter of the work. The paper also follows one early ruling through to its overturning, and sets out what an archive needs to keep so that later users can tell which reading they hold.
Mr Onwuka, Doctoral Candidate, hoped other laboratories would write in. “We have thirty-one rulings and no idea which of them anyone else would have made differently,” he said. “That seems a question worth asking out loud.”
It is a quiet satisfaction for the University when a habit of mind formed in one School turns out to be at home in another’s imaging suite.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.avvwip.