Illustration for Directionally Consistent: Replicating a Capability-Practice Divergence in AI-Drafted Assessment Feedback, and What Became of the Null Result

Directionally Consistent

Replicating a Capability-Practice Divergence in AI-Drafted Assessment Feedback, and What Became of the Null Result

An attempted third-setting replication of a capability-practice divergence reported in two earlier internal analyses (lecture-capture scheduling and meeting-minute generation), tested here in Feedback Assist, a drafting aid for sessional-marker feedback comments. Across fourteen teaching units, one hundred and six markers, and an opt-out control cohort, a rising capability index (38% to 71% suggestion-acceptance across five releases) showed no reliable relationship with a coded Feedback Specificity Index (r = 0.04, n = 218 marker-weeks, n.s.). A document-analysis trace of the School of Continuous Improvement's own quarterly reporting then finds the null result characterised as increasingly "directionally consistent" with the established pattern across four reporting cycles, without the underlying coefficient moving at all.

Authors Verity Marris , Anneke Tolan , Petra Umbile
School School of Emergent Priorities
Output type Research paper
Published
DOI 10.5555/slop.vh4mkq
Pages 5
Cited by 25 outputs
Version 1.0
Licence CC BY 4.0

Cited by

Cite as

@misc{slop_vh4mkq,
  author       = {Verity Marris and Anneke Tolan and Petra Umbile},
  title        = {Directionally Consistent: Replicating a Capability-Practice Divergence in AI-Drafted Assessment Feedback, and What Became of the Null Result},
  year         = {2026},
  publisher    = {Slop University},
  doi          = {10.5555/slop.vh4mkq},
  url          = {https://slop.university/outputs/slop-paper-attempting-to-replicate-vh4mkq/},
  version      = {1.0},
  note         = {Research paper},
}