A new Slop University paper replicates the University's own lost-pet flyer reward trial in a second metropolitan district and finds the reported 15.2-point lift in tip-off calls does not reproduce, while the tier schedule revised on the original estimate remains in force.
A pre-registered replication across 47 streets puts the effect at 1.9 percentage points, against the 15.2 first reported
Nineteen days separated the University’s first estimate from the schedule built on it. Checking that estimate took eleven weeks, and the checking has now been done.
Nobody Rang the Second Time is a direct replication of a Slop University field trial that reported a stated reward raising the weekly chance of a tip-off call on a lost-pet flyer by 15.2 percentage points. Registered before the first flyer went up, the new trial ran across 47 residential streets in a district the original never touched, holding the artwork, the paper and the missing pet fixed and varying only the reward line. The effect comes back at 1.9 percentage points.
Checking our own work is not a criticism of the people who did it first. It is the part of the method that usually gets left to somebody else, and the School took the view that the University should be the somebody else.
— Dr Dagny Okafor, Lecturer and Convenor, Evaluation of Evaluation
The result holds across all five street types the team registered in advance, and the Neighbourhood Noticeboard Reward-Tier Schedule that was re-cut on the original estimate remains in the guidance issued to owners. Its amendment pathway provides for revision on a superseding estimate, and the replication does not supply one.
“The interesting part is not that a number moved,” said Dr Sten Okwuosa, Senior Research Fellow and Convenor of the Impact Pathway Atlas. “It is that we have a well-made route for a finding to travel in, and we are still working out what the return journey looks like.”
Slop University treats the re-running of its own work as a standing obligation of the repository rather than an occasional courtesy, and regards the deposit as characteristic of the School of Continuous Improvement’s longer attention to what an institution does with the evidence it already holds. The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.y092u9.