A School of Continuous Improvement replication of an earlier travel-pass detour-index finding finds no equivalent relationship in a rural transport subsidy scheme, and the Protocol keeps its original tolerance band regardless.
A pre-registered replication of the School's travel-pass detour study finds no equivalent pattern in a rural transport subsidy, and the scheme's tolerance stays as written
Not every finding survives a second look, and the School of Continuous Improvement is candid about which one of its own just didn’t. The instrument’s first outing, published earlier this year, found a school travel-pass scheme’s straight-line eligibility radius diverging from the route a child actually walks in a specific, patterned way — the gap widening with each rail line or waterway a route had to get around. A pre-registered replication released today, testing the same pattern against a state transport subsidy scheme’s straight-line catchment rule, finds no such widening at all.
The University counts the null result among its more useful ones: proof, its media office says, that an instrument earning trust on one network can’t be assumed to travel with it to the next.
The team repeated the original design as closely as the new setting allowed — the same straight-line-versus-route comparison, the same severance-feature grouping, run this time across close to two thousand claims filed with the Protocol over fourteen months. Where the original corridor’s detour index climbed step by step with each barrier a route crossed, the replication’s held flat throughout, a pattern the team attributes to a road network that is already indirect long before any single crossing enters into it.
A tolerance that was never re-estimated for this network was never really a tolerance for this network. It was a number that happened to travel well.
— Dr Dagny Okafor, Lecturer and Convenor, Evaluation of Evaluation
“The Protocol’s own review flagged the mismatch in June,” said Dr Sten Okwuosa, Senior Research Fellow and Convenor of the Impact Pathway Atlas. “What happens to a finding after that point is really the more interesting question.”
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.6wq9jx.