A new Slop University research poster separates a council scam-awareness indicator into what residents can tell apart and where they draw the line, and finds the whole of the program's gain sitting in the second.
18,480 message judgements from 220 residents at six community halls, before a council's workshops and eight weeks after
The School of Continuous Improvement has a standing interest in what a short course leaves behind once the certificate has been filed. Its newest research poster went looking in a council’s scam-awareness program, and found something the program’s own indicator is not built to show.
Suspicion Is Not Detection, deposited today through the Office of Research Outputs, sat 220 residents down at six community halls with a fixed set of forty-two messages, half genuine and half not, once before the council’s session and once eight weeks after. The council reports the share of scam messages residents correctly flag. Across the eight weeks that share rose 17.6 points. The share of genuine messages flagged rose 23.2.
Read through a signal-detection model, the residents’ ability to tell the two apart had not moved at all. What moved was the line. At eight weeks four residents in ten flagged a pharmacy’s script-ready notice as a scam, half of them flagged a school absence notice, and a third reported having ignored something in the interval that turned out to be real.
We have spent a long time running short courses and then asking whether people remembered them. This is the first time the School has asked whether remembering was the thing we were measuring, and the answer has been useful to us in a way none of us especially enjoyed.
— Dr Ingeborg Nwachukwu, Lecturer and Convenor, Capability Sprints
The University regards the poster as a contribution to a longer institutional interest in indicators that can only move one way. Dr Sten Okwuosa, Senior Research Fellow and Convenor of the Impact Pathway Atlas, said the finding “raises questions about the rest of our measurement set that the School is glad to be holding, and in no particular hurry to close”.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.peh6gr.