Why It's Inaccurate to Extrapolate Success Rates from a Small Base of Patients

 

Vista Research Group often gets asked why we don’t report a treatment center’s abstinence success rate based solely upon the responses of the patients we reach post-treatment. In other words, if 75% of the patients who respond report that they’ve been abstinent for at least the last 30 days, the center has a 75% success rate, right?

I'm sorry, but it doesn’t. It is wishful thinking to assume that the patients who didn’t respond to outcomes research requests were as successful in recovery as those who did. And because consumers searching for treatment have learned to distrust such claims, these centers have cost the addiction treatment industry dearly.

When I was searching for treatment centers for my daughter, admissions counselors at multiple centers told me they had 80% success rates. One particularly unscrupulous center owner had the nerve to claim that 98% of his patients were abstinent after treatment. If we give him the (probably undeserved) benefit of the doubt that he had some sort of supporting data, I would guess he had asked 50 people at an alumni party whether they were substance-free for the last month and all but one answered “yes”. But can we assume that this group is representative of all 500 patients who attended treatment at this center in the last year? Of course not.

The hard truth is that patients who are proud of how they’re doing are far more likely to respond to a post-treatment survey request. And we have several types of evidence of this. First, many more patients start to fill out Vista’s post-treatment survey requests than end up submitting them. And an awful lot of them stop answering the survey at the same question – when we ask if they’ve used drugs or alcohol since leaving treatment.

Additionally, the group of patients who submit outcomes surveys are statistically more likely to be abstinent than the group of patients who do not in several crucial ways:

  • Vista’s research shows that the most important factor in whether a patient is able to remain abstinent is whether or not they successfully completed treatment (see Learning’s from Three Years of Addiction Treatment Outcomes Research). Almost two-thirds of the patients who submitted a six-month post-treatment outcomes survey over the last four years had successfully completed treatment. By comparison, only about half of the patients who didn’t respond had done so.                                                                                                                    
        • Survey respondents are statistically more likely to have a primary drug of choice of alcohol (a group which Vista’s research shows is typically more successful remaining abstinent than those preferring other drugs) and statistically less likely to be addicted to heroin or cocaine, with the poorest abstinence success rates, than non-responders
        • The average length of stay for patients who responded to Vista’s six-month survey was 83 days versus only 72 days for patients who didn’t respond.

        If addiction treatment leaders want to become as respected as other healthcare professionals treating chronic diseases, we need to stop over-reporting success rates. It is my hope that Conquer Addiction will facilitate this process by helping consumers compare the success rates of rehabs who are doing outcomes research meeting the stringent requirements set by an independent panel of expert judges.

        Please don’t hesitate to reach out if you have any questions about either Vista’s research or how to include your center in Conquer Addiction’s database.

        Joanna