A new Slop University research poster pairs 192 vacating rental properties with two independent assessors each, finding 85 per cent raw agreement on the standard twenty-two-item condition report but chance-corrected agreement of 0.26, weakest on the line items carrying the largest deductions.
A research poster sends two independent assessors through 192 vacating tenancies and finds chance-corrected agreement of 0.26 on the standard condition report
Two assessors walk the same emptied rental property, fill in the same twenty-two-item condition report, and agree on eighty-five per cent of the ticks. A research poster released today by the School of Continuous Improvement puts a figure on how much of that agreement survives being corrected for chance, and the figure is 0.26.
We have spent years teaching people to build instruments, and rather less time asking whether two of our own, handed the same form and the same empty room, would fill it in the same way. This is the first honest answer we have.
— Dr Torun Ezeigwe, Senior Lecturer and Director, Master of Applied Measurement
The study paired 192 vacating tenancies across a residential portfolio, sending two assessors through each property independently and within forty-eight hours of one another, with the deduction schedule withheld from both until their reports were lodged. Pooled across 4,224 line-item judgements, chance-corrected agreement settled at 0.26, and fell to 0.11 on the six items — walls, carpet, window furnishings, oven, grout and garden — from which the largest deductions are calculated.
Dr Ingeborg Nwachukwu, Lecturer and Convenor of the School’s Capability Sprints program and a co-author on the poster, said the result had narrowed the team’s question rather than answered it. “We went in asking how reliable the form is, and we are leaving with something more tractable: which of its twenty-two lines the agreement actually has to hold on.”
The release sits within the University’s continuing attention to instrument reliability, an area it regards as under-attended relative to the weight of the decisions instruments are asked to carry.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.fhlpyv.