Three Classes, Six Answers

Inter-Rater Reliability of a Shire's Noticeboard Display Standard and the Reproducibility of the Currency Rate It Publishes

A paper from the School of Emergent Priorities photographs all 34 public noticeboards in one rural shire fortnightly across fourteen months (9,236 notice-instances, 3,118 distinct notices) and has the six customer service officers who perform the sweeps independently classify a stratified sample of 420 notices under the shire's Public Noticeboard Display Standard. Agreement across the six is a Fleiss' kappa of 0.28 (95% CI 0.24 to 0.32), reliable for the one class carrying a letterhead (official, 0.71) and poor for the two describing local activity (community 0.19, commercial 0.22); fundraising notices, 21.4 per cent of the corpus, split 46 / 49 between them. Recomputing each board's published currency rate from each officer's own classification of the same board on the same day gives a median within-board spread of 22.0 percentage points, and on nine of the twelve sampled boards the Standard's 80 per cent clear-and-reset trigger fires under one officer's classification and not under another's.

@misc{slop_m1cm77,
  author       = {Anouk Mensah and Iben Chikere and Verity Marris},
  title        = {Three Classes, Six Answers: Inter-Rater Reliability of a Shire's Noticeboard Display Standard and the Reproducibility of the Currency Rate It Publishes},
  year         = {2026},
  publisher    = {Slop University},
  doi          = {10.5555/slop.m1cm77},
  url          = {https://slop.university/outputs/slop-paper-the-community-noticeboard-m1cm77/},
  version      = {1.0},
  note         = {Research paper},
}