Stupid political grandstanding. That’s one reason we cannot have nice things.
Last week, Anne Tolley canned a trial of the Ministry of Social Development’s predictive modelling system. The system takes a pile of back-end administrative data and predicts a kid’s likelihood of having bad outcomes like being abused.
The trial would have had one group of kids provided special intervention if they looked to be particularly at risk; the other group was control. [See update]
Now it wasn’t clear last week whether that control group received zero support, or whether it just got the standard interventions that social workers provide for kids that are likely to show up as risky in those kinds of systems. Tolley’s reaction suggested the former, but that seemed really unlikely. Would MSD ever really remove all possible support from at-risk kids just to see whether their predictive model got things right? I didn’t have the background papers so couldn’t tell, but it seemed heroically unlikely. Sure, it would be perhaps a cleaner test that way, but with the numbers of kids on which they have administrative data, they’d have had enough power to tell whether their predictive modelling improved on that which MSD was already doing. And that should be the test anyway: is this new intervention better than existing interventions?
Anyway, Tolley killed the programme, to hoorahs, and to Opposition calls for the head of whoever thought of the idea in the first place.
A minister sees a briefing paper with a proposal to test a computer model designed to identify children at risk of maltreatment. She reacts strongly.
“Not on my watch!” she writes in the margin, “these children are not lab-rats”. The study is shelved.
The media obtain the briefing paper, complete with the marginalia, and publicise it.
The Opposition seize on the lab-rats cry and use it in the House against the Minister of Social Development and the ministry.
Should we feel relieved? Have we averted another unfortunate experiment? No.
The minister’s reaction, and the media and Opposition response to it should make us feel uneasy.
The problem is not the shelving of the study – though that was a mistake too – rather it is the chilling effect of the knee-jerk and political response to an attempt to produce evidence for important social policy.
Science collided with politics, and politics won.
Dare notes that the trial would have removed services from nobody. It would have tested the new predictive approach against the status quo. It could have done some good. But Tolley killed it, to applause.
I suppose that people get the government they deserve, and they deserve to get it good and hard.
One wonders how National intends on running Social Impact Bonds with performance-based payments, if even control groups are now too politically risky.
Update: the trial was even less invasive. They were just maintaining standard MSD approaches for everybody, and watching whether the predictive model wound up being right, if I’m now reading Dare’s article correctly.