A new Slop University research poster tests a smart baby monitor's cry-classification confidence score against whether its label actually matched what soothed the baby, and finds the score tracking loudness, not correctness.
Across 68 families and 14 weeks, the score tracked a cry's loudness far more closely than whether its label was right
A smart monitor doesn’t just guess why a baby is crying anymore — it grades its own guess, and posts the mark next to the cry. A new Slop University research poster went looking for what that mark is actually worth.
Senior Lecturer Runa Adegoke, Convenor of the Strategic Drift Survey, and Senior Lecturer Ronja Oyelaran followed 68 families running a commercial cry-classification monitor for fourteen weeks, logging each cry’s on-device label against the soothing action a caregiver later found actually worked, without letting the caregiver see the monitor’s own guess first. The monitor’s confidence score was tracked separately against both the label’s correctness and the cry’s raw loudness.
The label matched what actually worked in roughly half of all logged cries, a share that held flat across the full fourteen weeks regardless of how many cries a household had by then logged. The confidence score moved with the second measure far more readily than the first: a loud cry reliably produced a confident label, correct or not.
A confident monitor and a correct one turned out to be answering two different questions. Ours was only ever built to grade the first.
— Senior Lecturer Runa Adegoke, Convenor, Strategic Drift Survey, School of Emergent Priorities
“Families were remarkably patient generating the ground truth we needed,” said Senior Lecturer Ronja Oyelaran. “None of them thought they were testing the score. They just wanted the crying to stop.”
The School of Emergent Priorities counts the finding among the more useful ones this year for what it declines to claim.
The full poster is available from the University’s research repository under an open licence, doi:10.5555/slop.mjo541.