A new Slop University paper varies only the sentence a customer writes on a home-insurance claim form and finds acceptance in full moves 15.6 points, while the insurer's own consistency audit holds at 95.8 per cent.
2,628 determinations on household files whose photographs, loss date and damage extent were identical, decided by 219 licensed assessors
An insurer’s assurance team re-reads a sample of decided claims each month and scores whether each decision is supported by the reason recorded against it. That figure sits at 95.8 per cent. The School of Continuous Improvement’s newest paper is about what it does not reach.
Described, Then Decided, published today, put twelve household claim files in front of each of 219 licensed assessors, 2,628 determinations in all. The files differed in the ordinary ways, and in one further way: the sentence the customer had written describing the damage was varied across three wordings while the photographs, the loss date, the damage extent and the sum insured stayed identical. Acceptance in full moved 15.6 points across those three sentences, further than the documents attached, the value of the claim and the claimant’s history moved it together.
The research team then ran the insurer’s own consistency check across the same decisions. It returned 95.8 per cent and did not vary with the wording, the recorded reason naming the photographs in nine determinations out of ten and the customer’s own sentence in one in fifty.
Every part of this arrangement was doing what it had been asked to do, which is the part we thought worth reporting. The School has learned something about the reach of its own instruments, and we would rather learn it in a study than not learn it.
— Dr Torun Ezeigwe, Senior Lecturer and Director of the Master of Applied Measurement
Dr Dagny Okafor, who convenes the School’s Evaluation of Evaluation program, said the finding “is the question the program was built for, and one we intend to put to our own review cycles first”. The paper is the kind of work the School was assembled to do, and it arrives from a team that has spent the year on measures compiled from the same material they assess.
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.j1llhz.