Scientific illiteracy, pt. 4003

There’s been a lot of coverage of the Bristol University study on rating various drugs’ potential for harm (BugMeNot will get you in). The intent of the piece was to develop a more objective rating scale — NOT to compare various drugs directly. A comparison of fourteen drugs was done by two different groups as part of the study, however, in order to see if the scale was sufficiently objective. Since both groups came back with similar ratings for the drugs, the conclusion was that yes, the scale was probably pretty good.

They included tobacco and alcohol to provide a sort of baseline, because the various dimensions of the harms they cause have been well-studied.

The authors took care to point out the obvious:

However, direct comparison of the scores for tobacco and alcohol with those of the other drugs is not possible since the fact that they are legal could affect their harms in various ways, especially through easier availability.

This is key. You can’t go comparing the harms of illegal things with legal things. How much of the harm from illegal drugs stems from their illegality? How would that change if they were made legal? On the other hand, how would the level of societal harm caused by tobacco and alcohol change if they were made illegal? We don’t know. Which means direct comparison of illegal and legal drugs is entirely meaningless both scientifically and practically.

And how have the media covered this study? By drawing little bar graphs directly comparing the scores, of course (view the attached PDF). Or by hopping on the little “alcohol and tobacco are dangerous” soundbite without putting it in context, and by doing pretty much anything but including that critical caveat in their coverage. For shame.

Yes, the line between legal and illegal drugs is drawn arbitrarily. But drawing scientifically meaningless bar graphs is not going to help solve that problem.