School of Continuous Improvement researchers pool every evaluation lodged under one workplace-activity accreditation and find its benchmark has doubled across seven rounds while independently measured walking has not moved.
A synthesis of 46 evaluations lodged under one workplace accreditation, whose benchmark is recomputed each round from the evaluations lodged the round before
A workplace accreditation has spent seven rounds setting its walking benchmark from the evidence its own rules oblige members to produce. Nothing Moved but the Benchmark, published today by the School of Continuous Improvement, pools all 46 evaluations lodged under one such scheme and reports that the figure members must beat has risen by a factor of 2.06 while independently measured walking has not moved at all.
Members hold the accreditation by running an annual step challenge, lodging an evaluation of it, and clearing a published threshold. That threshold is recomputed each round as the pooled effect of the evaluations lodged the round before.
Nobody in this arrangement has done anything wrong, and that is the part worth sitting with. Every evaluation we pooled is a competent account of what its instrument recorded.
— Dr Dagny Okafor, Lecturer and Convenor of Evaluation of Evaluation, School of Continuous Improvement
Across 16,082 participants the pooled gain is 1,318 steps a day. Where the challenge application supplied the counts it is 1,712; where employers issued sealed devices read independently of the challenge, it is 118, an interval that includes zero. Measurement source accounts for 68.7 per cent of the variation between studies. Fifteen of the 61 challenges registered over the period were never lodged at all.
Dr Sten Okwuosa, Senior Research Fellow and Convenor of the Impact Pathway Atlas, said the University “is proud to have put a method to a question that schemes of this kind have not often been asked”, and that the School expects the approach to travel to other accreditations resting on member-supplied evidence.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.joj6v6.