A School of Continuous Improvement paper reports a cluster-randomised trial across 55 retail-chain sites testing whether a visible running tally changes what staff submit to a staff town hall's Q&A queue, and finds no measurable difference under any condition tested.
A 55-site trial of the Question Salience Index finds tally visibility makes no difference to what staff submit or how fast they pile onto a leading question.
Making a staff town hall’s live vote count visible while people are still typing their question does not change what gets asked, a School of Continuous Improvement paper reports today. The paper, from Lecturer Solveig Adeyemi and Senior Lecturer and Convenor Thandiwe Solberg, followed 55 sites of a national retail chain through fourteen months of quarterly town halls, randomly assigning each site to show or hide the running tally on its Q&A portal’s Question Salience Index while staff typed.
The result sits inside a research line the School treats as core business: understanding exactly what an instrument changes about the behaviour it watches, and being equally prepared to report when the answer is nothing.
Staff had already decided whether to ask before any number appeared next to their question. The tally was never going to be what stood between them and the microphone.
— Dr Solveig Adeyemi, Lecturer, School of Continuous Improvement
Across 220 sessions and more than three thousand submitted questions, sites that showed the running tally and sites that hid it produced statistically indistinguishable results on both measures the study tracked: how many distinct topics a town hall’s audience raised, and how often a later question arrived close on the heels of one already leading. The pattern held at small sites and large ones alike.
“The queue’s ranking still decides which question the moderator reaches first,” said Dr Thandiwe Solberg, Senior Lecturer and Convenor, Demo Quarter. “Whether staff can see that ranking taking shape turns out to have no bearing on what they bring to it.”
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.d61t5e.