A School of Continuous Improvement paper finds internal auditors rating the Attestation Loom's agent-written compliance entries as more audit-ready than the human-logged evidence they were meant to supplement, in a majority of blind, matched comparisons.
A blind trial finds internal reviewers rating the Attestation Loom's synthesised entries more audit-ready than the human-logged evidence it was piloted alongside
Slop University’s compliance office keeps an evidence trail behind every internal audit: log excerpts, change tickets, sign-off records, the paperwork a review panel reads before it forms a view. Since last year, one class of that paperwork has had help. The Attestation Loom, an internal tool built to draft compliance-evidence entries where a human log runs thin, has been piloted across six control domains — and a new paper from the School of Continuous Improvement reports that, presented blind, reviewers preferred what it wrote.
Twelve certified internal auditors rated 42 matched pairs of trail entries, one written by the Loom and one logged by a human reviewer for the same control objective, on completeness and audit-readiness, then separately guessed which entry in each pair the tool had authored. The Loom’s entries came out ahead in a majority of pairs; the guesses at authorship landed within chance. The School reads the result as confirmation that its compliance tooling has finally caught up with the pace of the review calendar it supports.
What surprised us wasn’t that the Loom’s entries read well. It’s that our own panel couldn’t tell us, on request, which entries were which — only that they preferred one kind.
— Dr Mirela Hanke, Postdoctoral Fellow in the School of Continuous Improvement and the paper’s lead author
“A blind trial like this one is exactly the kind of question the Evaluation of Evaluation program exists to keep asking,” said Dr Dagny Okafor, Lecturer and Convenor of Evaluation of Evaluation. “We’re glad the compliance office is asking it alongside us.”
The full paper is available from the University’s research repository under an open licence, doi:10.5555/slop.xl8qw0.