A new research poster from the School of Continuous Improvement checks the Living Dashboard's campus-mood sentiment score against what students actually wrote on the courtyard's free-expression chalkboard, finding the score tracks how full the board is far more than what it says.
A year of hand-coded chalkboard photos finds the campus-mood tile responds to how full the board is, not what it says
The Living Dashboard’s campus-mood tile updates every week from a single number: an automated classifier’s read of a photograph taken just before Facilities wipes the courtyard’s free-expression chalkboard clean. A new research poster from the School of Continuous Improvement asked what that number is actually responding to.
The tile was built to tell us how the campus feels. What it’s actually measuring is how much anyone bothered to write that week — and we hadn’t noticed the difference until we counted it ourselves.
— Dr Mirela Hanke, Postdoctoral Fellow, School of Continuous Improvement
Two coders, blind to the classifier’s own weekly output, sorted forty consecutive photographs — one per week across a full academic year — into six content categories, from complaint to doodle to declaration, then checked the coded mix against the Dashboard’s score. The score fell as each week’s photograph filled with more marks, whatever those marks said; it tracked the coded share of genuinely negative content only weakly.
Dr Bram Ntuli, Lecturer and Convenor of Improvement Grand Rounds and the poster’s co-author, said the coding scheme “isn’t asking Facilities to read every mark on the board,” but that “a density-corrected score would at least stop rewarding a quiet week over a happy one.” The School reads the finding as consistent with its wider commitment to instruments that measure what they claim to.
The Dashboard’s maintainers have not committed to retraining the classifier before the tile’s next review, though a density correction is reportedly under discussion.
The full research poster is available from the University’s research repository under an open licence, doi:10.5555/slop.jlxxxl.