A School of Continuous Improvement paper audits a suburban buy-swap-sell group's volunteer-run reverse-image verification badge, finding its effect on buyer disputes decays over six months even as flagged sellers adapt their photography to defeat it.
An eighteen-month rollout across nine listing categories finds disputes recover most of the way to baseline within six months, while flagged sellers quietly change how they take the photo
The School of Continuous Improvement has spent much of the past two years arguing that a measurement’s honesty depends on what happens to it once people start trying to pass it. A paper published today by Casimir Beng, Torun Ezeigwe and Renke Sabel gives that argument its most literal test yet: a neighbourhood marketplace’s own volunteer-built photo check, tracked from its first listing category to its last.
The check itself has no institutional backer. Five unpaid moderators in a suburban buy-swap-sell group began, without a written rubric, running every submitted listing photo through a public reverse-image search before granting it an “as pictured, not stock” badge. The University’s paper follows the badge as it reached the group’s nine categories one at a time over eighteen months, comparing buyer-reported disputes before and after each category’s turn, and quietly withholding the badge from half of checked listings to test whether disputes tracked the visible icon or the checking behind it.
Five volunteers with no shared rubric and a public search bar have produced an instrument as reliable as some we fund properly, and one just as capable of being adapted around.
— Associate Professor Casimir Beng, Lead of the Adaptive Metrics Lab
Disputes fell by roughly two-fifths in the month a category’s check arrived, then drifted back toward the pre-check rate over the following months — a pattern the withheld-badge comparison suggests belongs to the check itself rather than to the icon buyers see. A survey of sellers whose photos had been flagged found many had changed how they now shoot a listing, several deliberately working an identifiable household object into frame. “Sellers worked out what the check was looking for well before we did,” Ezeigwe said, “which is either the badge’s biggest weakness or the clearest evidence it was doing something.” The University sees the finding as an early caution for the Indicator Commons, which has begun weighing the badge as a candidate for a cross-group certification standard.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.p6v8q5.