Dressed as Data

A Matched-Dossier Test of What Moves an Evaluation-of-Evaluation Panel Score

A three-cycle matched-pair study gave Evaluation-of-Evaluation panellists the same review dossier twice, differing only in an added one-page summary chart, and found the chart-bearing copy scored 13 to 15 points higher in every cycle despite identical content. The gap sat within a range the review's certified tolerance band treats as a genuine difference in case quality, held after a mid-study reminder that the content was fixed, and coincided with panellists citing a chart-only figure in 41% of their written justifications.

@misc{slop_gw64x7,
  author       = {Osei Vandermeer and Casimir Beng},
  title        = {Dressed as Data: A Matched-Dossier Test of What Moves an Evaluation-of-Evaluation Panel Score},
  year         = {2026},
  publisher    = {Slop University},
  doi          = {10.5555/slop.gw64x7},
  url          = {https://slop.university/outputs/slop-poster-stapling-a-summary-gw64x7/},
  version      = {1.0},
  note         = {Research poster},
}