A new paper from the School of Continuous Improvement tests whether Improvement Grand Rounds' paper sign-in sheet is actually harder to fake than the tap-card system it replaced, and finds the retired card reader, still quietly logging, at least as informative.
An in-house handwriting-divergence tool checks the clipboard against the card reader it replaced, and turns up a gap too small to call decisive
Improvement Grand Rounds took attendance by card tap for three years, until an internal review last year raised doubts about what the taps were actually recording. The School of Continuous Improvement’s fortnightly case-review seminar switched back to a paper sign-in sheet, on the understanding that a signature is harder to fake than a borrowed card. A new paper puts that understanding to a first test.
A clipboard reads as more trustworthy than a card reader mostly because nobody has checked. This is us checking.
— Dr Dagny Okafor, Lecturer and Convenor, Evaluation of Evaluation
Lecturer Dagny Okafor, Lecturer Bram Ntuli, Postdoctoral Fellow Mirela Hanke, and Senior Lecturer Renke Sabel built a lightweight tool that scores each semester signature against that signer’s own earliest mark, flagging any that drifts unusually far from it. About one in nine signature instances was flagged. The team then checked those flagged instances against the old card reader, left switched on beside the door even after it stopped being the official record, and found flagged signatures somewhat more likely to be missing a matching tap — a gap the paper is careful not to oversell.
“We convene Grand Rounds to look hard at everybody else’s review formats,” said Dr Bram Ntuli, a co-author and the series’ convenor. “It was only fair to turn the same treatment on our own sign-in sheet.”
The paper sits inside the School’s growing habit of measuring its own measuring, rather than assuming an instrument means what its name promises. The School has no plans to change how Grand Rounds records attendance for now, and will revisit the question once a full year of sign-in data is available.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.k7y0kk.