A School of Emergent Priorities and Adaptive Metrics Lab survey has 2,830 managers across 46 workplaces independently rescore the same self-assessment entry, finds the assigned score barely depends on rater tenure, and the resulting SCORE credential becomes a promotion-panel eligibility requirement at four workplaces.
A blind rescoring exercise turns rater disagreement into SCORE, the credential now gating four promotion panels
An identical sentence should not be worth a bonus one week and a warning the next.
— Senior Lecturer Ronja Oyelaran, School of Emergent Priorities
The School of Emergent Priorities and the Adaptive Metrics Lab have published a population survey built around the performance review’s self-assessment comment box, the space where an employee writes their own case before a manager assigns the final number. Dr Oyelaran led the project with Associate Professor Casimir Beng, circulating one anonymised entry to managers across forty-six partner workplaces and asking each to score it independently.
Held constant across 2,830 raters, the entry’s assigned score spanned the full five-point scale in every tenure band surveyed, with managers newest to the role scoring barely more variably than those five years in. The School frames the result as exactly the kind of self-scrutiny it wants more of, not less.
“It’s the kind of spread that used to sit in a drawer,” said Associate Professor Beng. “Now it sits on a promotion panel.” The Adaptive Metrics Lab has folded the exercise into SCORE, a credential earned by scoring within band on a held-out set of entries; four of the participating workplaces have already made it a condition of promotion-panel eligibility.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.gklg0g.