Directionally Consistent

Replicating a Capability-Practice Divergence in AI-Drafted Assessment Feedback, and What Became of the Null Result

An attempted third-setting replication of a capability-practice divergence reported in two earlier internal analyses (lecture-capture scheduling and meeting-minute generation), tested here in Feedback Assist, a drafting aid for sessional-marker feedback comments. Across fourteen teaching units, one hundred and six markers, and an opt-out control cohort, a rising capability index (38% to 71% suggestion-acceptance across five releases) showed no reliable relationship with a coded Feedback Specificity Index (r = 0.04, n = 218 marker-weeks, n.s.). A document-analysis trace of the School of Continuous Improvement's own quarterly reporting then finds the null result characterised as increasingly "directionally consistent" with the established pattern across four reporting cycles, without the underlying coefficient moving at all.

@misc{slop_vh4mkq,
  author       = {Verity Marris and Anneke Tolan and Petra Umbile},
  title        = {Directionally Consistent: Replicating a Capability-Practice Divergence in AI-Drafted Assessment Feedback, and What Became of the Null Result},
  year         = {2026},
  publisher    = {Slop University},
  doi          = {10.5555/slop.vh4mkq},
  url          = {https://slop.university/outputs/slop-paper-attempting-to-replicate-vh4mkq/},
  version      = {1.0},
  note         = {Research paper},
}