The AA faithful are great at taking what they want and leaving the rest when it comes to demonstrating the effectiveness of the program. Every member of AA who successfully quit drinking is held up as an affirmation of the program, while those 95% plus who fail are disregarded as though they don’t exist. Or, if challenged, they will say that those who fail do not fully “give in to the program”. Of course, by this logic, AA is 100% effective.

Most often, their rationalization of why AA works is anecdotal. They will use their own experience as an example, and they will use the examples of the dozens of other people in their home group to show that the program does indeed work for some people. The problem with this logic is that it disregards the five percent of alcoholics who would have quit on their own without AA by taking full credit for their quit. Because AA meetings are where people motivated to quit drinking tend to congregate, it gives the illusion of a successful program to those making anecdotal observations. Let me draw a comparison:

Let’s assume a type of cancer that has a 5% percent remission rate, and a company produces an herbal remedy that offers a cure for a percentage of people. If we were to take ten thousand people with this cancer, and follow their progress over five years, and along the way replace those who either died or dropped out of the study, we would have 2,500 people whose cancer went into remission while taking the herbal remedy. It looks impressive, but it is the same number of people who would fall into remission from a control group who took a placebo. The herbal remedy would be proved ineffective, and if that herbal company took these figures and offered up an infomercial showing testimonials of cancer patients who were cured while using the remedy, it would be beyond unethical. It would be criminal.

This is what AA does with their promotion of anecdotal experiences, and their disregard of actual research data. More than one study shows AA to be ineffective, but the thing most often cited by those who oppose AA is the Triennial survey data that shows they have a 95% failure rate within the first year (aa_triennial_survey1) Showing this data to an AA true believer is like talking to a fence post, and will be met with those pesky anecdotal examples. As anyone who has debated religion with rationally will understand, this can be a maddening experience. It gets worse when a person wants to prove their faith by making things up, and showing you once and for all that the world is flat.

This is what happened recently when three AA old-timers took the data from the Triennial Survey and tried to manipulate it to prove to the flock that the 95% failure rate was false. They produced a document called “Alcoholics Anonymous (AA) Recovery Rate Outcomes: Contemporary Myth and Misinterpretation”. One of the tenets of pseudoscience is that it cloaks its jargon in scientific language, and the title of this paper sounds impressive. In reviewing it, however, it shows the three authors of this paper either do not understand basic statistics, or chose to manipulate them, hoping that the average person would not understand this to be a shoddy piece of work. I’m guessing a little of both. Let’s take a look at some of the highlights of this paper:

From the paper (source: triennialreviewaa):

The erroneous 10%, 5% or less success rate myth for contemporary AA has proliferated without as much as a token challenge to its veracity or investigation of its origin. The topic of AA success or failure outcomes suffers from a great deal of anecdotal misinformation, misinterpretation and editorializing.”

The “erroneous interpretation” was not a result of outside commentators. It was AA’s own interpretation. One has to assume it to be true, because AA has zero interest in manipulating data against itself. They also say that nobody has bothered to examine the source data for this interpretation, but that is because AA will not provide it.

They go on to say:

[what the data] “shows is that those who stay beyond three months are active a year later”.

This is true, but so what? This is like saying – “those who finish the first ten miles of a marathon are more likely to make it all 26 miles” or “cancer patients who survive the first six months are more likely to survive for a year”. This is how they drew their conclusion – by eliminating those who failed in the first quarter of the year. You can’t pick a time period that suits you and lop off those who failed in the first three months as though they don’t exist. This is what I mean by “taking what you want and leaving the rest” in order to demonstrate AA’s effectiveness.

They then go on make the stunning admission that Bill Wilson’s claims a 75% success rate is bullshit. On this nobody disagrees. Finally, they pull the old “those who fail really don’t try” routine out their ass again:

“Since AA’s beginning, about 60% to 80% of those who show up at AA meetings observe but do not really try AA. They engage in investigation but not participation. Many are simply uninterested or unaware that AA can help them. That others are not seeking a solution in AA may well be a function of their intrinsic nature of being averse or unable to admit what they are, recognize how their life is unmanageable and being willing to do something about it.”

To begin with, the 60-80% figure is arbitrary. It’s like saying 60-80% of the people were funny or good looking. What they are simply doing is eliminating a group of failed AAers to elevate their success rate. This goes along their attitude that “the program cannot fail, but some fail the program” logic. Secondly, if we were to pluck 100 alcoholics off the street, a lower percentage of that group would be motivated to quit drinking, because people who are motivated to quit drinking go to AA meetings. The same way that you will find a higher percentage of fat people at Jenny Craig wanting to lose weight than you would find if you randomly selected fat people at a mall. Those who are motivated to lose weight go weight loss clinics, just as those motivated to quit drinking show up at AA meetings. So, even though a greater number of AA participants are motivated to quit drinking, they have exactly the same 5% percent quit rate as a randomized selection of alcoholics. This might explain why one study found a negative correlation between AA and quitting drinking.

This paper is just a horrifically bad, manipulative piece of work. No real academic would take it seriously, but still has made its way around the circles of AA meetings and internet AA forums. So, even if it was an accurate review, the very people who say statistics don’t matter are using this as an example of AA’s effectiveness. To hell with the many randomized clinical trials, all showing AA to be ineffective when compared to a control group. They are going to cherry pick a manipulated paper to prove their worth, and disregard every legitimate study that proves otherwise. They are going to put blinders on and believe what they want to believe, and worse, promote their pseudoscientific nonsense to others who really need help.