I was up late last night watching the Super Bowl coverage, and I don’t feel like working. So, I thought I would take a quick moment to explain the 5% success rate to those of you who aren’t very good with numbers, like our friend McGowdog, who wrote the following in the comment section:

So there’s your strawman argument right there. Y’all learned it from Agent Orange and are spewing it out right here. The “thousands of interpretations in between” is your assessment of a fellowship that still has recovery rates vastly more successful than the dribble that Orange spews out and you mimic here. Here’s a truer retention rate of A.A.s by length of sobriety which debunks Oranges and others observation of AAWS stats;

A.A. Members’ Length of Sobriety:

Less than one year….. 26%

1-5 years………………. 24%

5-10 years…………….. 14%

10+ years………………. 36%

Yet, y’all are going to gasp with fear when you see this and you all know in your little heads that it’s 5% or less. In fact, you think the retention rate of A.A. is a negative percentage, everybody is drunk, and everybody who gets breathed on by an A.A.er is going to get drunk as well as it’s a contagious disease.

There is a reason why we don’t allow anecdotal evidence in science. It is more often wrong than right. Perception can be a crazy thing sometimes. A person walking into an established AA group might be led to believe that it really is effective in getting people off of the sauce. After all, a good portion of the room is quit, and has stayed quit for some time. Obviously, it works for some folks, right? Well, no. Not really.

The most common reason people fall for pseudoscience is because they mistake correlation with cause. Correlation simply means there is a relationship between two different things. For example, most racing horse jockeys are very short and small, and most professional basketball players are very tall. Does horse racing shorten people, and does basketball make people taller? Of course not. Do hospitals make people sick? No. The reason basketball players are tall, is because tall people congregate to basketball teams. Sick people congregate to hospitals. Likewise, people who quit drinking congregate to AA. There is no difference really.

Now, having said this, correlation is the best starting point in looking for a cause. So, since so many people in AA have been quit for a substantial amount of time, the obvious thing to do is to see if AA is the cause. The only way to do that is run the numbers, and in doing so the numbers come out to a 5% quit rate. Also, McGowdog’s numbers are correct, as well. Those were drawn from AA’s triennial survey. So how can those numbers be correct, and the success rate be 5% at the same time? I’ll explain:

Take out a calculator and draw yourself out a bell curve.

Now take the actual figures from AA’s survey, including those who began AA and dropped out of the program. From their own internal figures, the dropout rate within the first year is 95%. The chance of success at that point is much greater, and 30% of those who make it the first year will make it to year five. Beyond that 95% are successful long term quitters.

Now, assume a new chapter is started with exactly 100 members and a new person filling in the void left when a member leaves. At the end of year one, you have 5 one year members and 95 who have been there for less than a year. Go forward another year. You will now have close 10 successful quitters of one and two years (actually, between 9 and 10 if you assume the average, but since you can’t split an actual person, I’m rounding up). Fast forward ten years, and that chapter will have a good many permanent quitters. It looks impressive to anyone attending their first meeting. You’ll have close to 70 people (70%) of the room who have quit for 6 months or more. Of those, close to half are at 5 years or longer. All of them attributing their success to AA. All of them telling you to work the steps and you can do as they did. Now fast forward 20 years, keep the same success rates of 5%, 30% and 95%, the room looks even more impressive. And if you took a survey of only those who are active members, plot them on that bell curve, you will come up with numbers almost identical to the survey you linked to has:

35% have been sober for over 10 years.

16% have been sober between 5-10 years.

28% have been sober between 1-5 years.

22% have been sober less that 1 year.

With an average of 8 years sober.

The numbers are very similar to McGowdog’s. They are slightly different, only because he used a different year’s survey, and the numbers vary slightly from survey to survey.

This is a 5% rate of successful quitters.

Now, is AA 5% effective? No. It is really 0% effective, because 5% would have quit on their own, anyway. To determine the real rate of effectiveness, the baseline is set at 5%, not 0. Any number above that shows the efficacy of the program.

Think about this next time someone says, “I’ve seen it work with my own eyes.” They aren’t lying. In this case it McGowdog mistaking cause and correlation. It is like someone discovering fool’s gold. They get all pumped up and want tell the world because they think they got hold of something good. That’s what McGow did in showing these numbers of AA’s success rate. He isn’t lying. He’s just ignorant.

I hope today’s lesson clarifies some these points for our resident AAs. Tomorrow’s lesson: Your ass from a hole in the ground: A comparative study.