Bad journalism
This piece is almost identical with today’s Spectator Health article.
This week there has been enormously wide coverage in the press for one of the worst papers on acupuncture that I’ve come across. As so often, the paper showed the opposite of what its title and press release, claimed. For another stunning example of this sleight of hand, try Acupuncturists show that acupuncture doesn’t work, but conclude the opposite: journal fails, published in the British Journal of General Practice).
Presumably the wide coverage was a result of the hyped-up press release issued by the journal, BMJ Acupuncture in Medicine. That is not the British Medical Journal of course, but it is, bafflingly, published by the BMJ Press group, and if you subscribe to press releases from the real BMJ. you also get them from Acupuncture in Medicine. The BMJ group should not be mixing up press releases about real medicine with press releases about quackery. There seems to be something about quackery that’s clickbait for the mainstream media.
As so often, the press release was shockingly misleading: It said
Acupuncture may alleviate babies’ excessive crying Needling twice weekly for 2 weeks reduced crying time significantly
This is totally untrue. Here’s why.
Luckily the Science Media Centre was on the case quickly: read their assessment. The paper made the most elementary of all statistical mistakes. It failed to make allowance for the jelly bean problem. The paper lists 24 different tests of statistical significance and focusses attention on three that happen to give a P value (just) less than 0.05, and so were declared to be "statistically significant". If you do enough tests, some are bound to come out “statistically significant” by chance. They are false postives, and the conclusions are as meaningless as “green jelly beans cause acne” in the cartoon. This is called P-hacking and it’s a well known cause of problems. It was evidently beyond the wit of the referees to notice this naive mistake. It’s very doubtful whether there is anything happening but random variability. And that’s before you even get to the problem of the weakness of the evidence provided by P values close to 0.05. There’s at least a 30% chance of such values being false positives, even if it were not for the jelly bean problem, and a lot more than 30% if the hypothesis being tested is implausible. I leave it to the reader to assess the plausibility of the hypothesis that a good way to stop a baby crying is to stick needles into the poor baby. If you want to know more about P values try Youtube or here, or here. |
One of the people asked for an opinion on the paper was George Lewith, the well-known apologist for all things quackish. He described the work as being a "good sized fastidious well conducted study ….. The outcome is clear". Thus showing an ignorance of statistics that would shame an undergraduate.
On the Today Programme, I was interviewed by the formidable John Humphrys, along with the mandatory member of the flat-earth society whom the BBC seems to feel obliged to invite along for "balance". In this case it was professional acupuncturist, Mike Cummings, who is an associate editor of the journal in which the paper appeared. Perhaps he’d read the Science media centre’s assessment before he came on, because he said, quite rightly, that
"in technical terms the study is negative" "the primary outcome did not turn out to be statistically significant"
to which Humphrys retorted, reasonably enough, “So it doesn’t work”. Cummings’ response to this was a lot of bluster about how unfair it was for NICE to expect a treatment to perform better than placebo. It was fascinating to hear Cummings admit that the press release by his own journal was simply wrong.
Listen to the interview here
Another obvious flaw of the study is that the nature of the control group. It is not stated very clearly but it seems that the baby was left alone with the acupuncturist for 10 minutes. A far better control would have been to have the baby cuddled by its mother, or by a nurse. That’s what was used by Olafsdottir et al (2001) in a study that showed cuddling worked just as well as another form of quackery, chiropractic, to stop babies crying.
Manufactured doubt is a potent weapon of the alternative medicine industry. It’s the same tactic as was used by the tobacco industry. You scrape together a few lousy papers like this one and use them to pretend that there’s a controversy. For years the tobacco industry used this tactic to try to persuade people that cigarettes didn’t give you cancer, and that nicotine wasn’t addictive. The main stream media obligingly invite the representatives of the industry who convey to the reader/listener that there is a controversy, when there isn’t.
Acupuncture is no longer controversial. It just doesn’t work -see Acupuncture is a theatrical placebo: the end of a myth. Try to imagine a pill that had been subjected to well over 3000 trials without anyone producing convincing evidence for a clinically useful effect. It would have been abandoned years ago. But by manufacturing doubt, the acupuncture industry has managed to keep its product in the news. Every paper on the subject ends with the words "more research is needed". No it isn’t.
Acupuncture is pre-scientific idea that was moribund everywhere, even in China, until it was revived by Mao Zedong as part of the appalling Great Proletarian Revolution. Now it is big business in China, and 100 percent of the clinical trials that come from China are positive.
if you believe them, you’ll truly believe anything.
Follow-up
29 January 2017
Soon after the Today programme in which we both appeared, the acupuncturist, Mike Cummings, posted his reaction to the programme. I thought it worth posting the original version in full. Its petulance and abusiveness are quite remarkable.
I thank Cummings for giving publicity to the video of our appearance, and for referring to my Wikipedia page. I leave it to the reader to judge my competence, and his, in the statistics of clinical trials. And it’s odd to be described as a "professional blogger" when the 400+ posts on dcscience.net don’t make a penny -in fact they cost me money. In contrast, he is the salaried medical director of the British Medical Acupuncture Society.
It’s very clear that he has no understanding of the error of the transposed conditional, nor even the mulltiple comparison problem (and neither, it seems, does he know the meaning of the word ‘protagonist’).
I ignored his piece, but several friends complained to the BMJ for allowing such abusive material on their blog site. As a result a few changes were made. The “baying mob” is still there, but the Wikipedia link has gone. I thought that readers might be interested to read the original unexpurgated version. It shows, better than I ever could, the weakness of the arguments of the alternative medicine community. To quote Upton Sinclair:
“It is difficult to get a man to understand something, when his salary depends upon his not understanding it.”
It also shows that the BBC still hasn’t learned the lessons in Steve Jones’ excellent “Review of impartiality and accuracy of the BBC’s coverage of science“. Every time I appear in such a programme, they feel obliged to invite a member of the flat earth society to propagate their make-believe.
Acupuncture for infantile colic – misdirection in the media or over-reaction from a sceptic blogger?26 Jan, 17 | by Dr Mike Cummings So there has been a big response to this paper press released by BMJ on behalf of the journal Acupuncture in Medicine. The response has been influenced by the usual characters – retired professors who are professional bloggers and vocal critics of anything in the realm of complementary medicine. They thrive on oiling up and flexing their EBM muscles for a baying mob of fellow sceptics (see my ‘stereotypical mental image’ here). Their target in this instant is a relatively small trial on acupuncture for infantile colic.[1] Deserving of being press released by virtue of being the largest to date in the field, but by no means because it gave a definitive answer to the question of the efficacy of acupuncture in the condition. We need to wait for an SR where the data from the 4 trials to date can be combined. So what about the research itself? I have already said that the trial was not definitive, but it was not a bad trial. It suffered from under-recruiting, which meant that it was underpowered in terms of the statistical analysis. But it was prospectively registered, had ethical approval and the protocol was published. Primary and secondary outcomes were clearly defined, and the only change from the published protocol was to combine the two acupuncture groups in an attempt to improve the statistical power because of under recruitment. The fact that this decision was made after the trial had begun means that the results would have to be considered speculative. For this reason the editors of Acupuncture in Medicine insisted on alteration of the language in which the conclusions were framed to reflect this level of uncertainty. DC has focussed on multiple statistical testing and p values. These are important considerations, and we could have insisted on more clarity in the paper. P values are a guide and the 0.05 level commonly adopted must be interpreted appropriately in the circumstances. In this paper there are no definitive conclusions, so the p values recorded are there to guide future hypothesis generation and trial design. There were over 50 p values reported in this paper, so by chance alone you must expect some to be below 0.05. If one is to claim statistical significance of an outcome at the 0.05 level, ie a 1:20 likelihood of the event happening by chance alone, you can only perform the test once. If you perform the test twice you must reduce the p value to 0.025 if you want to claim statistical significance of one or other of the tests. So now we must come to the predefined outcomes. They were clearly stated, and the results of these are the only ones relevant to the conclusions of the paper. The primary outcome was the relative reduction in total crying time (TC) at 2 weeks. There were two significance tests at this point for relative TC. For a statistically significant result, the p values would need to be less than or equal to 0.025 – neither was this low, hence my comment on the Radio 4 Today programme that this was technically a negative trial (more correctly ‘not a positive trial’ – it failed to disprove the null hypothesis ie that the samples were drawn from the same population and the acupuncture intervention did not change the population treated). Finally to the secondary outcome – this was the number of infants in each group who continued to fulfil the criteria for colic at the end of each intervention week. There were four tests of significance so we need to divide 0.05 by 4 to maintain the 1:20 chance of a random event ie only draw conclusions regarding statistical significance if any of the tests resulted in a p value at or below 0.0125. Two of the 4 tests were below this figure, so we say that the result is unlikely to have been chance alone in this case. With hindsight it might have been good to include this explanation in the paper itself, but as editors we must constantly balance how much we push authors to adjust their papers, and in this case the editor focussed on reducing the conclusions to being speculative rather than definitive. A significant result in a secondary outcome leads to a speculative conclusion that acupuncture ‘may’ be an effective treatment option… but further research will be needed etc… Now a final word on the 3000 plus acupuncture trials that DC loves to mention. His point is that there is no consistent evidence for acupuncture after over 3000 RCTs, so it clearly doesn’t work. He first quoted this figure in an editorial after discussing the largest, most statistically reliable meta-analysis to date – the Vickers et al IPDM.[2] DC admits that there is a small effect of acupuncture over sham, but follows the standard EBM mantra that it is too small to be clinically meaningful without ever considering the possibility that sham (gentle acupuncture plus context of acupuncture) can have clinically relevant effects when compared with conventional treatments. Perhaps now the best example of this is a network meta-analysis (NMA) using individual patient data (IPD), which clearly demonstrates benefits of sham acupuncture over usual care (a variety of best standard or usual care) in terms of health-related quality of life (HRQoL).[3] |
30 January 2017
I got an email from the BMJ asking me to take part in a BMJ Head-to-Head debate about acupuncture. I did one of these before, in 2007, but it generated more heat than light (the only good thing to come out of it was the joke about leprechauns). So here is my polite refusal.
Hello Thanks for the invitation, Perhaps you should read the piece that I wrote after the Today programme Why don’t you do these Head to Heads about genuine controversies? To do them about homeopathy or acupuncture is to fall for the “manufactured doubt” stratagem that was used so effectively by the tobacco industry to promote smoking. It’s the favourite tool of snake oil salesman too, and th BMJ should see that and not fall for their tricks. Such pieces night be good clickbait, but they are bad medicine and bad ethics. All the best David |
In the course of thinking about metrics, I keep coming across cases of over-promoted research. An early case was “Why honey isn’t a wonder cough cure: more academic spin“. More recently, I noticed these examples.
“Effect of Vitamin E and Memantine on Functional Decline in Alzheimer Disease".(Spoiler -very little), published in the Journal of the American Medical Association. ”
and ” Primary Prevention of Cardiovascular Disease with a Mediterranean Diet” , in the New England Journal of Medicine (which had second highest altmetric score in 2013)
and "Sleep Drives Metabolite Clearance from the Adult Brain", published in Science
In all these cases, misleading press releases were issued by the journals themselves and by the universities. These were copied out by hard-pressed journalists and made headlines that were certainly not merited by the work. In the last three cases, hyped up tweets came from the journals. The responsibility for this hype must eventually rest with the authors. The last two papers came second and fourth in the list of highest altmetric scores for 2013
Here are to two more very recent examples. It seems that every time I check a highly tweeted paper, it turns out that it is very second rate. Both papers involve fMRI imaging, and since the infamous dead salmon paper, I’ve been a bit sceptical about them. But that is irrelevant to what follows.
Boost your memory with electricity
That was a popular headline at the end of August. It referred to a paper in Science magazine:
“Targeted enhancement of cortical-hippocampal brain networks and associative memory” (Wang, JX et al, Science, 29 August, 2014)
This study was promoted by the Northwestern University "Electric current to brain boosts memory". And Science tweeted along the same lines.
Science‘s link did not lead to the paper, but rather to a puff piece, "Rebooting memory with magnets". Again all the emphasis was on memory, with the usual entirely speculative stuff about helping Alzheimer’s disease. But the paper itself was behind Science‘s paywall. You couldn’t read it unless your employer subscribed to Science.
|
All the publicity led to much retweeting and a big altmetrics score. Given that the paper was not open access, it’s likely that most of the retweeters had not actually read the paper. |
|
When you read the paper, you found that is mostly not about memory at all. It was mostly about fMRI. In fact the only reference to memory was in a subsection of Figure 4. This is the evidence.
That looks desperately unconvincing to me. The test of significance gives P = 0.043. In an underpowered study like this, the chance of this being a false discovery is probably at least 50%. A result like this means, at most, "worth another look". It does not begin to justify all the hype that surrounded the paper. The journal, the university’s PR department, and ultimately the authors, must bear the responsibility for the unjustified claims.
Science does not allow online comments following the paper, but there are now plenty of sites that do. NHS Choices did a fairly good job of putting the paper into perspective, though they failed to notice the statistical weakness. A commenter on PubPeer noted that Science had recently announced that it would tighten statistical standards. In this case, they failed. The age of post-publication peer review is already reaching maturity
Boost your memory with cocoa
Another glamour journal, Nature Neuroscience, hit the headlines on October 26, 2014, in a paper that was publicised in a Nature podcast and a rather uninformative press release.
"Enhancing dentate gyrus function with dietary flavanols improves cognition in older adults. Brickman et al., Nat Neurosci. 2014. doi: 10.1038/nn.3850.".
The journal helpfully lists no fewer that 89 news items related to this study. Mostly they were something like “Drinking cocoa could improve your memory” (Kat Lay, in The Times). Only a handful of the 89 reports spotted the many problems.
A puff piece from Columbia University’s PR department quoted the senior author, Dr Small, making the dramatic claim that
“If a participant had the memory of a typical 60-year-old at the beginning of the study, after three months that person on average had the memory of a typical 30- or 40-year-old.”
Like anything to do with diet, the paper immediately got circulated on Twitter. No doubt most of the people who retweeted the message had not read the (paywalled) paper. The links almost all led to inaccurate press accounts, not to the paper itself. |
|
But some people actually read the paywalled paper and post-publication review soon kicked in. Pubmed Commons is a good site for that, because Pubmed is where a lot of people go for references. Hilda Bastian kicked off the comments there (her comment was picked out by Retraction Watch). Her conclusion was this.
"It’s good to see claims about dietary supplements tested. However, the results here rely on a chain of yet-to-be-validated assumptions that are still weakly supported at each point. In my opinion, the immodest title of this paper is not supported by its contents."
(Hilda Bastian runs the Statistically Funny blog -“The comedic possibilities of clinical epidemiology are known to be limitless”, and also a Scientific American blog about risk, Absolutely Maybe.)
NHS Choices spotted most of the problems too, in "A mug of cocoa is not a cure for memory problems". And so did Ian Musgrave of the University of Adelaide who wrote "Most Disappointing Headline Ever (No, Chocolate Will Not Improve Your Memory)",
Here are some of the many problems.
- The paper was not about cocoa. Drinks containing 900 mg cocoa flavanols (as much as in about 25 chocolate bars) and 138 mg of (−)-epicatechin were compared with much lower amounts of these compounds
- The abstract, all that most people could read, said that subjects were given "high or low cocoa–containing diet for 3 months". Bit it wasn’t a test of cocoa: it was a test of a dietary "supplement".
- The sample was small (37ppeople altogether, split between four groups), and therefore under-powered for detection of the small effect that was expected (and observed)
- The authors declared the result to be "significant" but you had to hunt through the paper to discover that this meant P = 0.04 (hint -it’s 6 lines above Table 1). That means that there is around a 50% chance that it’s a false discovery.
- The test was short -only three months
- The test didn’t measure memory anyway. It measured reaction speed, They did test memory retention too, and there was no detectable improvement. This was not mentioned in the abstract, Neither was the fact that exercise had no detectable effect.
- The study was funded by the Mars bar company. They, like many others, are clearly looking for a niche in the huge "supplement" market,
The claims by the senior author, in a Columbia promotional video that the drink produced "an improvement in memory" and "an improvement in memory performance by two or three decades" seem to have a very thin basis indeed. As has the statement that "we don’t need a pharmaceutical agent" to ameliorate a natural process (aging). High doses of supplements are pharmaceutical agents.
To be fair, the senior author did say, in the Columbia press release, that "the findings need to be replicated in a larger study—which he and his team plan to do". But there is no hint of this in the paper itself, or in the title of the press release "Dietary Flavanols Reverse Age-Related Memory Decline". The time for all the publicity is surely after a well-powered study, not before it.
The high altmetrics score for this paper is yet another blow to the reputation of altmetrics.
One may well ask why Nature Neuroscience and the Columbia press office allowed such extravagant claims to be made on such a flimsy basis.
What’s going wrong?
These two papers have much in common. Elaborate imaging studies are accompanied by poor functional tests. All the hype focusses on the latter. These led me to the speculation ( In Pubmed Commons) that what actually happens is as follows.
- Authors do big imaging (fMRI) study.
- Glamour journal says coloured blobs are no longer enough and refuses to publish without functional information.
- Authors tag on a small human study.
- Paper gets published.
- Hyped up press releases issued that refer mostly to the add on.
- Journal and authors are happy.
- But science is not advanced.
It’s no wonder that Dorothy Bishop wrote "High-impact journals: where newsworthiness trumps methodology".
It’s time we forgot glamour journals. Publish open access on the web with open comments. Post-publication peer review is working
But boycott commercial publishers who charge large amounts for open access. It shouldn’t cost more than about £200, and more and more are essentially free (my latest will appear shortly in Royal Society Open Science).
Follow-up
Hilda Bastian has an excellent post about the dangers of reading only the abstract "Science in the Abstract: Don’t Judge a Study by its Cover"
4 November 2014
I was upbraided on Twitter by Euan Adie, founder of Almetric.com, because I didn’t click through the altmetric symbol to look at the citations "shouldn’t have to tell you to look at the underlying data David" and "you could have saved a lot of Google time". But when I did do that, all I found was a list of media reports and blogs -pretty much the same as Nature Neuroscience provides itself.
More interesting, I found that my blog wasn’t listed and neither was PubMed Commons. When I asked why, I was told "needs to regularly cite primary research. PubMed, PMC or repository links”. But this paper is behind a paywall. So I provide (possibly illegally) a copy of it, so anyone can verify my comments. The result is that altmetric’s dumb algorithms ignore it. In order to get counted you have to provide links that lead nowhere.
So here’s a link to the abstract (only) in Pubmed for the Science paper http://www.ncbi.nlm.nih.gov/pubmed/25170153 and here’s the link for the Nature Neuroscience paper http://www.ncbi.nlm.nih.gov/pubmed/25344629
It seems that altmetrics doesn’t even do the job that it claims to do very efficiently.
It worked. By later in the day, this blog was listed in both Nature‘s metrics section and by altmetrics. com. But comments on Pubmed Commons were still missing, That’s bad because it’s an excellent place for post-publications peer review.
[This an update of a 2006 post on my old blog]
The New York Times (17 January 2006) published a beautiful spoof that illustrates only too clearly some of the bad practices that have developed in real science (as well as in quackery). It shows that competition, when taken to excess, leads to dishonesty.
More to the point, it shows that the public is well aware of the dishonesty that has resulted from the publish or perish culture, which has been inflicted on science by numbskull senior administrators (many of them scientists, or at least ex-scientists). Part of the blame must attach to "bibliometricians" who have armed administrators with simple-minded tools the usefulness is entirely unverified. Bibliometricians are truly the quacks of academia. They care little about evidence as long as they can sell the product.
The spoof also illustrates the folly of allowing the hegemony of a handful of glamour journals to hold scientists in thrall. This self-inflicted wound adds to the pressure to produce trendy novelties rather than solid long term work.
It also shows the only-too-frequent failure of peer review to detect problems.
The future lies on publication on the web, with post-publication peer review. It has been shown by sites like PubPeer that anonymous post-publication review can work very well indeed. This would be far cheaper, and a good deal better than the present extortion practised on universities by publishers. All it needs is for a few more eminent people like mathematician Tim Gowers to speak out (see Elsevier – my part in its downfall).
Recent Nobel-prizewinner Randy Schekman has helped with his recent declaration that "his lab will no longer send papers to Nature, Cell and Science as they distort scientific process"
The spoof is based on the fraudulent papers by Korean cloner, Woo Suk Hwang, which were published in Science, in 2005. As well as the original fraud, this sad episode exposed the practice of ‘guest authorship’, putting your name on a paper when you have done little or no work, and cannot vouch for the results. The last (‘senior’) author on the 2005 paper, was Gerald Schatten, Director of the Pittsburgh Development Center. It turns out that Schatten had not seen any of the original data and had contributed very little to the paper, beyond lobbying Scienceto accept it. A University of Pittsburgh panel declared Schatten guilty of “research misbehavior”, though he was, amazingly, exonerated of “research misconduct”. He still has his job. Click here for an interesting commentary.
The New York Times carried a mock editorial to introduce the spoof..
One Last Question: Who Did the Work? By NICHOLAS WADE In the wake of the two fraudulent articles on embryonic stem cells published in Science by the South Korean researcher Hwang Woo Suk, Donald Kennedy, the journal’s editor, said last week that he would consider adding new requirements that authors “detail their specific contributions to the research submitted,” and sign statements that they agree with the conclusions of their article. A statement of authors’ contributions has long been championed by Drummond Rennie, deputy editor of The Journal of the American Medical Association, Explicit statements about the conclusions could bring to light many reservations that individual authors would not otherwise think worth mentioning. The article shown [below] from a future issue of the Journal of imaginary Genomics, annotated in the manner required by Science‘s proposed reforms, has been released ahead of its embargo date. |
The old-fashioned typography makes it obvious that the spoof is intended to mock a paper in Science.
The problem with this spoof is its only too accurate description of what can happen at the worst end of science.
Something must be done if we are to justify the money we get and and we are to retain the confidence of the public
My suggestions are as follows
- Nature Science and Cell should become news magazines only. Their glamour value distorts science and encourages dishonesty
- All print journals are outdated. We need cheap publishing on the web, with open access and post-publication peer review. The old publishers would go the same way as the handloom weavers. Their time has past.
- Publish or perish has proved counterproductive. You’d get better science if you didn’t have any performance management at all. All that’s needed is peer review of grant applications.
- It’s better to have many small grants than fewer big ones. The ‘celebrity scientist’, running a huge group funded by many grants has not worked well. It’s led to poor mentoring and exploitation of junior scientists.
- There is a good case for limiting the number of original papers that an individual can publish per year, and/or total grant funding. Fewer but more complete papers would benefit everyone.
- Everyone should read, learn and inwardly digest Peter Lawrence’s The Mismeasurement of Science.
Follow-up
3 January 2014.
Yet another good example of hype was in the news. “Effect of Vitamin E and Memantine on Functional Decline in Alzheimer Disease“. It was published in the Journal of the American Medical Association. The study hit the newspapers on January 1st with headlines like Vitamin E may slow Alzheimer’s Disease (see the excellent analyis by Gary Schwitzer). The supplement industry was ecstatic. But the paper was behind a paywall. It’s unlikely that many of the tweeters (or journalists) had actually read it.
The trial was a well-designed randomised controlled trial that compared four treatments: placebo, vitamin E, memantine and Vitamin E + memantine.
Reading the paper gives a rather different impression from the press release. Look at the pre-specified primary outcome of the trial.
The primary outcome measure was
" . . the Alzheimer’s Disease Cooperative Study/Activities of Daily Living (ADCSADL) Inventory.12 The ADCS-ADL Inventory is designed to assess functional abilities to perform activities of daily living in Alzheimer patients with a broad range of dementia severity. The total score ranges from 0 to 78 with lower scores indicating worse function."
It looks as though any difference that might exist between the four treaments is trivial in size. In fact the mean difference between Vitamin E and placebos was only 3.15 (on a 78 point scale) with 95% confidence limits from 0.9 to 5.4. This gave a modest P = 0.03 (when properly corrected for multiple comparisons), a result that will impress only those people who regard P = 0.05 as a sort of magic number. Since the mean effect is so trivial in size that it doesn’t really matter if the effect is real anyway.
It is not mentioned in the coverage that none of the four secondary outcomes achieved even a modest P = 0.05 There was no detectable effect of Vitamin E on
- Mean annual rate of cognitive decline (Alzheimer Disease Assessment Scale–Cognitive Subscale)
- Mean annual rate of cognitive decline (Mini-Mental State Examination)
- Mean annual rate of increased symptoms
- Mean annual rate of increased caregiver time,
The only graph that appeared to show much effect was The Dependence Scale. This scale
“assesses 6 levels of functional dependence. Time to event is the time to loss of 1 dependence level (increase in dependence). We used an interval-censored model assuming a Weibull distribution because the time of the event was known only at the end of a discrete interval of time (every 6 months).”
It’s presented as a survival (Kaplan-Meier) plot. And it is this somewhat obscure secondary outcome that was used by the Journal of the American Medical Assocciation for its publicity.
Note also that memantine + Vitamin E was indistinguishable from placebo. There are two ways to explain this: either Vitamin E has no effect, or memantine is an antagonist of Vitamin E. There are no data on the latter, but it’s certainly implausible.
The trial used a high dose of Vitamin E (2000 IU/day). No toxic effects of Vitamin E were reported, though a 2005 meta-analysis concluded that doses greater than 400 IU/d "may increase all-cause mortality and should be avoided".
In my opinion, the outcome of this trial should have been something like “Vitamin E has, at most, trivial effects on the progress of Alzheimer’s disease”.
Both the journal and the authors are guilty of disgraceful hype. This continual raising of false hopes does nothing to help patients. But it does damage the reputation of the journal and of the authors.
This paper constitutes yet another failure of altmetrics. (see more examples on this blog). Not surprisingly, given the title, It was retweeted widely, but utterly uncritically. Bad science was promoted. And JAMA must take much of the blame for publishing it and promoting it. |
|
Steve Jones, UCL’s star geneticist, has been commissioned by the BBC Trust to write a report on the impartiality of science journalism on the BBC. It covers both TV and radio, and all channels. Current programmes can be found by the BBC Science home page. |
|
It is not uncommon for bloggers to be critical of science reporting in the mainstream media. Now is our chance to do something constructive about it. If you have opinions about this, please leave them in the comments here, and/or email them to
Here are some of my own opinions, to get things going. Many programmes I haven’t seen/heard, so my selection may not be representative, but it is wide enough to include examples that are superb and examples of some that I think are not good enough.
Problem areas
There are two particular topics that are real problems for broadcasters. One is the whole area of alternative medicine and the ‘supplement’ industry. The other is anything to do with climate change. Both have formidable lobby groups which, to the inexperienced journalist, may sound like quite plausible scientists (some even have academic titles). Creationists can also be a problem. though not many programmes take them very seriously. Both quacks and climate deniers rarely have anything to say that is real science. They have different motivations. Examples are given below.
Many programmes are superb
David Attenbrough is an obvious example. His programmes can’t be bettered. The photography is breathtakingly beautiful and the science is always accurate. For me, they alone are worth the licence fee, and I don’t want the licence fee to be reduced. It helps that Attenborough knows the science so well. It also helps that most of the time the science isn’t very difficult and isn’t very controversial either.
There have been many other superb programmes. Steve Jones own 6-part TV series "In the blood" was a beautiful example. The fact that his comments are sought frequently by the BBC is greatly to their credit. Much depends on producers being sufficiently well-informed to know whom to ask.
More recently Brian Cox’s "Wonders" series has provided an excellent example of how science programmes can be made popular without being inaccurate,
Also excellent were Jim Al-Khalili’s Chemistry: A Volatile History and Michael Moseley’s Medical Mavericks.
Simon Singh has made consistently good programmes. His wonderful documentary on Fermat’s Last Theorem was a masterpiece.. He is a master at making programmes that make really difficult subjects accessible to the public, without making them misleading.
Tim Harford’s programm, More or Less has made a great contribution to public understanding of statistics.
Ben Goldacre‘s two part Radio 4 Programme, The Rise of the Lifestyle Nutritionists was a superb explanation of a contentious area.
It would be useful if all presenters of programmes with any scientific content had to listen to Harford or Goldacre. It might make them a bit more critical about the problem of causality that beset the observational epidemiology items that predominate among the items picked out from press releases by so many news programmes.
These programmes were so successful because they were made by people who know what they are talking about. They confirm the view that the best science programmes are made by scientists, not by journalists. There are exceptions of course. It could be argued that some of Robert Winston’s programmes have strayed too far from his area of expertise to reach the same high standards. And some journalists have produced excellent programmes. Two examples follow.
Geoff Watts has kept up a consistently high standard on Radio 4. from Science Now, through Medicine Now to Leading Edge, the standard has always been high. It is good straight science in its social context. He avoids controversies, for example his excellent programme about Charles Darwin does not include a creationist to provide (phony) ‘balance’.
After the demise of Medicine Now in 1998, Watts wrote in the BMJ thus.
In the early days of the programme you could have listened for several weeks in a row without hearing from patients. I used to defend this on the grounds that Medicine Now was there to talk about disease and its treatment, not the experience of disease and its treatment. I was wrong. To make that distinction is simply to parallel the fault for which doctors themselves take a deal of stick—being interested in the illness to the exclusion of the person who’s suffering from it. I was persuaded, reluctantly, to accept more lay voices, and I am now embarrassed that I didn’t sooner see the need for them.
The swing of the pendulum may push the whole patient experience thing too far: to a point at which it’s professional knowledge and objective analysis that is elbowed into the wings. One of the vogue concepts among BBC managers in recent years has been “accessibility”. But this is a weasel word, too easily used as justification for editing out anything that might require the audience to concentrate and think. If a patient has a rash, it’s a lot simpler for the reporter to inquire about the urge to scratch than to explore the events in the immune system that caused the skin to redden, swell, and itch in the first place. How sad if people lose an opportunity to hear from the researchers, speaking their own words, who are actually trying to find out.
This summarises a lot of the problems of science programmes. They too easily become trivial vox pops, and Watts resisted this tendency very successfully.
A great problem for programmes about medicine arises from the pressure exerted by the alternative medicine industry (ot which more later). Watts would not tolerate nonsense. He says
Medicine Now was stabled in the BBC’s Science Unit, and it was the broad acceptance of science and its methodology which shaped editorial choices. When complementary medicine was on the agenda, we expected evidence from our contributors not testaments of faith.
Material World is another good Radio 4 programme. Quentin Cooper does, on the whole, a good job. But sometimes even he falls foul of the phony balance argument After my piece in Nature on the shameful degrees in pseudo-scientific medicine got discussed on Material World, (audio here) but my opponent was not a scientist at all, but the head of “Complementary Therapies” at the University of Westminster, a man who presides over courses that teach “amethysts emit high Yin energy“. It is simply impossible to have a proper scientific discussion with people who believe nonsense like that. They don’t accept the ground rules at all. It is a good example of phony balance (see below).
Some programmes are quite bad
Alternative Medicine: the evidence. This series if three TV programmes was shown in February 2006 on BBC 2, in conjunction with the Open University. It illustrates well three problems with science programming. (1) Despite the title, tt was surprisingly weak at showing evidence, (2) It showed the defensive and unhelpful response that, only too often, the BBC shows when complaints are made. And (3) it showed that association with a university is not, per se, enough to guarantee quality.
Because of the title, I’d looked forward to this programme, and made minute by minute notes, which are recounted in BBC2 and the Open University on Alternative Medicine. It turned out that the evidence was thin on the ground, and what there was was not always accurate I complained to the BBC, but got nowhere [download my complaint and some subsequent correspondence]. I was fobbed off with defensive PR. (Much the same happened when I complained to the Open University.)
Worse still, a letter in defence of the programme that appeared in the Guardian, turned out to have been written by the BBC and was not even seen by some of its "signatories" -see .Alternative Medicine series: dirty tricks at the BBC? All this took a lot of work and got nowhere.
Simon Singh, the eminent science author, wrote two articles that exposed the very misleading portrayal of anaesthesia with acupuncture In the Guardian he wrote A groundbreaking experiment … or a sensationalised TV stunt?, and in the Daily Telegraph he wrote Did we really witness the ‘amazing power’ of acupuncture?. Singh also sent complaints to the BBC, but he persisted after the complaints were fobbed off and eventually his complaints reached the BBC Trustees. Two of his three serious complaints were upheld.
Phone-in programmes are notoriously bad for both balance and phony balance. In the alternative medicine field, equal time is always given to scientists, astrologers and crystal healers. The presenters are usually ill-informed and the callers are usually even less well informed. A particularly bad example follows.
Call You and Yours. The Radio 4 programme, You and Yours, at its best, can be quite good. It did a good job on a "snoring remedy" that I’d investigated, though it omitted some things that should, in my view, have been included. But they also have phone in versions of the programme. On 29th February 2010, they ran a phone-in programme about herbal medicine, hosted by Julian Worricker, someone who clearly was totally unaware of the controversies that surrounded this subject and, particularly, its regulation. The worst thing about this programme was that it featured a resident ‘expert’ That was Michael McIntyre who is chair of the European Herbal & Traditional Medicine Practitioners Association. McIntyre is a well known advocate of alternative medicine, who constantly fudges the need for proper evidence.
I went through the programme carefully, making detailed notes, which appear at Some truly appalling reporting of science by the BBC. It was one of the most biassed programmes on the topic I have ever heard. i sent a complaint to the BBC, referring to the detailed analysis which had already appeared here. To my dismay, they wouldn’t accept a complaint in the form. They wanted me to type the whole thing in a little box on the complaint site. where there is no formatting and no live links. I protested in vain that if they wanted a printed version, all the had to do was print the web page. At this point I decided that there was no point in spending yet more time to cope with the inflexibility of the complaints procedure.
Today programme. I’m an avid listener toToday, the best news programme on radio John Humphrys has no greater fan than I. For politics it is superb. But for science it is, sad to say, not always so good. One reason is that the presenters don’t know enough about the topics to ask the same sort of tough questions that they fire at politicians. Another reason is that they suffer badly from the phony balance problem (see below). A third reason is that they tend to pick up on silly survey press releases (the sort of ‘men with long big toes are better in bed’ pseudo-science); They may quite rightly laugh at them but this sort of thing doesn’t count as science reporting.. The Today programme is admirably serious about politics, but the science is often dumbed down and uncritical.
What needs to be done to improve BBC science
Link to the sources. Despite pressure from bloggers, the BBC web site still does not usually link to original sources, the paper on which claims are based. The whole virtue of the web is that it makes this very easy to do.
Anonymity of reporters. Too often reports of science on the BBC web site are anonymous. There is no excuse for that. Every report should carry the name and email address of the person who wrote it, Most newspapers do this, but the BBC is lagging behind.
Reaction to criticism. In most cases that I’ve tried, the reaction to constructive criticism has been obstructive and defensive. Producers seem very reluctant to admit that any mistakes were made. That needs to be changed.
Science correspondents are too often uncritical. A few more with the approach of investigative journalists would improve standards. An example is provided by a recent report “It’s good to think – but not too much, scientists say“. This is typical of the sort of work that many people find a bit hard to take seriously, but the report reads a bit like a regurgitation of press releases. There is no link to original sources and no attempt at evaluation.
Press releases. One reason for misleading reports stems from misleading press releases. Press releases often come from media departments who regard their job as getting their university into the headlinse, rather than explaining science. Worse still, sometimes the misleading hyps stems from the authors themselves (one example here, but there are hundreds to choose from). This makes it very important that science reporters should read the paper and have good enough critical faculties to read through the hype.
Complaints procedure needs to be improved. Complaints should be accepted in any form, The present web form is suitable only for short and simple criticicisms. An email address should be provided and it should accept attached documents. Certainly complaints in the form of web pages should be welcomed, because the live links provide the simplest way to refer to source documents.
The problem of phony balance. This is biggest problem of the lot.
In the wake of the report by the Science and Technology Committee (STC) on the lack of evidence for homeopathy, and the Chinese medicine poisoning, the BBC carried at least three very bad reports. Being a strong supporter of the BBC that saddens me. These cases are summarised at Some truly appalling reporting of science by the BBC. The worst was the case of Call You and Yours. There was also a totally imbalanced and ill-informed report on statutory regulation, and a very irresponsible video of a woman who claimed homeopathy cured her cancer. .
The question of balance is important. Ofcom imposes an obligation that reporting should reflect the balance of viewpoints. Section 5 of Ofcom’s broadcasting code says (emphasis is mine).
“Section 5: Due Impartiality and Due Accuracy and Undue Prominence of Views and Opinions”
“To ensure that news, in whatever form, is reported with due accuracy and presented with due impartiality.”
“Meaning of “due impartiality”: “Due” is an important qualification to the concept of impartiality. Impartiality itself means not favouring one side over another. “Due” means adequate or appropriate to the subject and nature of the programme. So “due impartiality” does not mean an equal division of time has to be given to every view, or that every argument and every facet of every argument has to be represented.”
The BBC Trust has a very similar definition of “due impartiality”.
Thus the rules stare quite explicitly that "impartial" does not mean giving equal time to any view, however batty,
In practice, though, producers often seem to play it too safe, and choose to give the same time time to the view that the earth is flat as is does to the view that the earth is spherical (OK, an oblate ellipsoid). This often gives a quite misleading impression of the state of play of informed opinion. Inappropriate use of “equal time” is the most common cause for misleading reports
on science.
Minority views should be heard of course, but they should not be given equal prominence to views that are held by the vast majority of informed people. Inevitably the worst cases arise in the areas of quack medicine, climate change and evolution.
Somebody said recently, it is as though after an air crash one gave equal time to the air accident investigator and a representative gravity-deniers association. That is scarcely an exaggeration of what happens on the BBC too often.
Worse still, far more time was given (especially on ‘Call You and Yours’) to the viewpoint that any scientist, indeed any informed person, would regard as quackery.
One thing that could be done about this false balance is to have better informed producers, or, more likely, to have better informed science reporters who can give advice on the state of opinion (and to make sure that their advice is sought).
Unless the BBC starts to be more critical in some of its reports, it could lose its preeminence. In the last few years it has become increasingly the case that the best critical evaluations of science are to be found not in the BBC or other mainstream media, but on blogs written by working scientists. Perhaps the BBC should ask them more often than it does at present.
Now give your opinions, below or email them to trust.science@bbc.co.uk
Follow-up
The Open University is a wonderful institution
I should have made it clear the Open University has played a big role in producing some of the best programmes. I was, quite rightly, corrected by a letter from an OU scientist. I’ll quote from it.
David Attenborough’s series on Life In Cold Blood, Charles Darwin and the Tree Of Life, Life – these are all Open University commissions. Life In Cold Blood also won a BAFTA and we have a string of other awards. I believe this is some of the very best science broadcasting the Open University puts out and I’m not surprised to see it at the top of your list.
More Or Less is an Open University commission – see e.g. http://www.open2.net/moreorless/
Material World is another one of our occasional commissions -see e.g. http://www.open2.net/materialworld/index.html
Geoff Watts contributed to the BBC Darwin season last year, which was heavily supported by Open University programmes.
See for example http://www.bbc.co.uk/darwin/We support the series with a considerable amount of on-line material both for credit and not for credit. This material is accessible to the public at these open2.net sites, and at http://www.open.ac.uk/openlearn/.
It is clear that the criticisms levelled at the Alternative Medicine series are very much the exception to the usual excellent work of the OU. I’m told that that programme had nothing to do with the science faculty. Clearly it was an unusual aberation. I presume it was connected with the OU’s course K221, which I wrote about in 2006, under the title Open University Quacks. That sort of thing is quite atypical of the Open University, and something of an embarrassment to the many top rate people who work there. As usual, the blame lies not with scientists but with senior managers. After hearing about course K221, I had a long correspondence with Professor David Vincent, a pro-vice chancellor. He made sympathetic noises, but did absolutely nothing. That’s par for the course with senior administrators.
The Open University has been a magnificent success from the outset. Its first vice-chancellor was Walter Laing Macdonald Perry . Before he took that job, he was professor of Pharmacology in Edinburgh (and one of my Ph.D. supervisors). He did a great job.
Failure to report negative results
A classic example of a sin of omission by the BBC (and the rest of the mainstream media too) occurred recently in the reporting of the alleged effect of B vitamins on the development of Alzheimer’s disease. A positive trial was widely reported, but two weeks later a trial appeared that measured the eight thing -cognitive deficiency – and that showed no effect at all. As far as I can tell it was barely reported at all, The details are at https://www.dcscience.net/?p=3516
I’m a bit late on this one, but better late than never.
The opinionated and ill-informed actress turned talk show host, Jeni Barnett, spent an hour or so endangering your children (and hers) with what most surely be one of the worst ever accounts of measles vaccination.
|
Chart from BBC report |
She was abominably rude to a well-informed nurse who phoned in to try to inject some sense into the conversation.
The LBC tried to stop Ben Goldacre from publicising this horrific show by legal action.
Blogs are the new journalism. The response has been wonderful. People of all ages sat up late into the night transcribing the entire broadcast. Unlike the doubtless highly-paid actress, they did it as a public service. They were not paid by anyone. It is all rather beautiful.. Within a day of the legal notice being sent to Goldacre, the offensive broadcast has spread like wildfire over the web.
The result of all this hard work is that if you type ‘Jeni Barnett MMR’ into Google, every item but one on the first page links to the sites that are highly critical of Barnett’s irresponsible and ill-mannered rant (at 7 am on 7 Feb).
You can listen to the entire broadcast here. Or read the entire transcript here.
The many people who have put work into this effort are listed, for example, on Ben Goldacre’s own site.
Holfordwatch lists many links, and also lists previous attempts of lawyers to suppress science.
When will people learn that lawyers are not the proper way to settle matters of truth and falsehood.
Dice, n . Small polka-dotted cubes of ivory, constructed like a lawyer to lie on any side, but commonly the wrong one. [ Bierce, Ambrose , The Enlarged Devil’s Dictionary , 1967]
Follow-up
The list of commentators, on Holfordwatch, grows by the minute. The story rapidly spread to the USA: for example the excellent Orac has spoken eloquently.
The condemnation extends far beyond the usual bad medicine writers. Anyone who wants to speak the truth as they see it sees legal actions like these as a threat to freedom of speech. A side effect is that I learned about several new blogs.
One, with a name as good as its content is A Somewhat Old, But Capacious Handbag, written by (you guessed it) Miss Prism, has Today’s irresponsible tripe courtesy of Jeni Barnett.”
Another one that was new to me is the Black Triangle blog, written by Dr Anthony Cox (a pharmacovigilance pharmacist). He writes in Conspiracy?
Anti-vaccinators have exploited the internet for years. Websites, blogs, and forums are widely used by activists to promote their wrong-headed cause. However, when the pro-science pro-vaccine lobby use similar methods a common accusation is leveled at them. Here it is posted at JABS, the UK’s leading anti-vaccine website.
“There is no way all of this could have happened so quickly without Pharma backing.” “
That is really priceless. These anti-vaccination fanatics just don’t seem to be able to grasp that there is a big army of people who care so much about the public interest that they do all this for no money and a considerable cost to themselves in time and lost sleep.
Besides which, anyone who thinks that a big corporation could whip up so much support and activity in 24 hours obviously has a rather better opinion of the efficiency of big companies than I do. They’d need 25 meetings and an awayday in Majorca before anything happened . Even a university can do better than that (perhaps only 20 meetings and an awayday in Uxbridge). One does wonder why, then, universities are always being told to be more like businesses. But that is another story.
Anthony Cox also deals with another of my favourite topics in The Today Programme’s irresponsible MMR interview. I listen to the Today Programme, I listen every morning. But I do wish they could bring their medical reporting up to the same standard as their political reporting. Their policy of ‘equal time for the flat earth society” is not my idea of impartiality.
The Sunday Times for 8th February, by coincidence, has a major article by the excellent investigative reporter, Brian Deer.
An excellent summary has appeared already Dynutrix on Holfordwatch.
Part 1.MMR doctor Andrew Wakefield fixed data on autism .
“However, our investigation, confirmed by evidence presented to the General Medical Council (GMC), reveals that: In most of the 12 cases, the children’s ailments as described in The Lancet were different from their hospital and GP records. Although the research paper claimed that problems came on within days of the jab, in only one case did medical records suggest this was true, and in many of the cases medical concerns had been raised before the children were vaccinated. Hospital pathologists, looking for inflammatory bowel disease, reported in the majority of cases that the gut was normal. This was then reviewed and the Lancet paper showed them as abnormal. “
Part 2. MMR: Key Dates in the Crisis .
Part 3. Most shockingly Hidden records show MMR truth
“A Sunday Times investigation has found that altered data was behind the decade-long scare over vaccination ”
Part 4. How the MMR scare led to the return of measles.
Let’s hope that some of the original documents appear on-line soon.
The Times on 10 February carried a beautifully hard-hitting column by David Aaronovitch: The preposterous prejudice of the anti-MMR lobby
“Last week there was a bust-up in blogland.”
“Last week, justifying herself on her blog, Barnett invoked the spirit of the insurgent ignoramus. Yes, she said, she should have been ready with facts and figures on MMR.”
“That’s why I’m passionately for Goldacre, and why I find myself wondering whether we can file a class action against LBC for permitting a presenter to inflict her preposterous prejudices on her listeners, to the detriment of someone else’s kids.”
Jeni Barnett: have you lost something?. Well well, first Jeni Barnett removed the critical comments from her blog. Then she removed the blog altogether. Seems she isn’t interested in debate at all.
Neither does she understand the internet. You can read the missing blog here, and the invaluable Quackometer has reproduced the whole blog post and all the missing comments. Great work Andy,
Stephen Fry left a comment (#223) on Goldacre’s site.
“The fatuity of the Jeni Barnett woman’s manner – her blend of self-righteousness and stupidity, her simply quite staggering inability to grasp, pursue or appreciate a sequence of logical steps – all these are signature characteristics of Britain these days. The lamentable truth is that most of the population wouldn’t really understand why we get so angry at this assault on reason, logic and sense. But we have to keep hammering away at these people and their superstitious inanities. We have to. Well done you and well done all you supporting. I’ve tweeted this site to my followers. I hope they all do their best to support you. Publish and be damned. We’ll fight them and fight them and fight them in the name of empricism, reason, double blind random testing and all that matters.”
London Evening Standard on 11 February. Nick Cohen on How my friends fell for the MMR panic.
Press Gazette covered the start of the srory on 6th February, here.
MSNBC TV broadcast by Keith Olberman votes Andrew Wakefield as “today’s worst person in the world” on February 10th. Click in video “Vaccine lie puts kids at risk”.
Write to your MP to ask him/her to sign Early Day Motion 754, MMR Vaccine and the Media
David Aaronovitch writes again in the Times, February 14th, “We need an inquiry into how Andrew Wakefield got away with it“.
Acupuncture in the BMJ
A new review appeared in the BMJ today. It is by Madsen et al., from the Nordic Cochrane Centre, Copenhagen. Here are the conclusions.
- The analgesic effect of acupuncture is small and cannot be distinguished from bias resulting from incomplete blinding.
- The analgesic effect of placebo acupuncture is moderate but very variable as some large trials report substantial effects.
- The effect of acupuncture seems to be unrelated to the type of placebo acupuncture used as control.
The results confirm, yet again, that there is essentially no difference between “real” acupuncture and sham acupuncture. All that talk about meridians and Qi really is so much mumbo jumbo.
The average effect (the diamond at the bottom) is essentially zero.
It has often been supposed that acupuncture is a theatrical placebo, but because of the placebo effect it produces more pain relief than (non-blind) controls. This study confirms that there is likely to be such an effect, but it also finds that the size of the placebo effect is too small to be useful to patients. Here is the comparison between sham acupuncture and no acupuncture at all.
The results of different trials are very variable, but the average effect (diamond at the bottom) favours sham acupuncture over no acupuncture at all.
But how big is the effect? The numbers along the bottom of the graph are ‘standardised mean differences’. The average value of -0.42 (95% confidence limits -0.6 to -0.23) between ‘no acupuncture’ and ‘sham acupuncture’ corresponds to a difference of about 10 points on a 100 point scale. This difference is big enough to be real, in the sense that it isn’t just chance. But is it big enough to be useful to the patient? Probably not. Madsen et al conclude
“a consensus report characterised a 10mm reduction on a 100 mm visual analogue scale as representing a “minimal” change or “little change”. Thus, the apparent analgesic effect of acupuncture seems to be below a clinically relevant pain improvement.”
This makes nonsense of the Pittilo report. Notice that these results, yet again, make nonsense of the proposals in the gamma-minus Pittilo report, to pseudo-regulate acupuncture, and to have degrees in the subject.
Cochrane reviews of Acupuncture : and a bad report from BBC
Two new Cochrane reviews appeared last week, Acupuncture for tension-type headache, and Acupuncture for migraine prophylaxis.
You can’t blame subeditors for the appalling title of the BBC’s report on Acupuncture ‘works for headaches’. The content is pretty misleading too. (The link is to a version saved at 19.35 on 21 January 2009.). Furthermore, like far too many of the BBC reports, it is anonymous. One has no idea to blame This is important, if only because the BBC news site is so influential. Twelve hours after posting the misleading title has been copied all round the world.
In the Guardian, Ian Sample had a much better report, Even’fake’ acupuncture reduces the severity of headaches and migraines. Sadly the print edition had the title “Acupuncture aids migraines, researchers
find”, but that can be blamed on subeditors who have a problem with reading (in fact Sample had seen neither title).
The BBC report seemed to call for a complaint. This is what I sent.
I wish to complain about the report on acupuncture at http://news.bbc.co.uk/1/hi/health/7838231.stm
The title itself was highly misleading “Acupuncture works for headaches” is precisely the opposite of what was shown, namely that it is no better than sham acupuncture controls. The article goes on to say
It is not said that there isn’t the slightest reason to think this is true (or even means anything) , and that the work which is being reported is strong evidence that it’s not true
That is wrong. University of Salford , for example has just closed its acupuncture course after a number of us have pointed out that it is teaching things that aren’t true
It is exactly the opposite of an endorsement. It is one more nail in the coffin of acupuncture as generally understood. This article, I contend, is partial and misleading. Unfortunately it is anonymous. |
Follow-up
Today’s BBC report on the BMJ paper is almost as partisan as the one that I just complained about. “Confusion on acupuncture benefit ” exaggerates greatly the amount of confusion. Worse still, it quotes only two well-known advocates of acupuncture, people who make their living from it. No independent scientific voice is to be heard.
One of those cited is Adrian White. He is described as “a researcher into acupuncture at the Peninsula Medical School”. There is no mention of the fact that he is also Editor in Chief of the journal Acupuncture in Medicine. The other person who is quoted is Mike O’Farrell, the chief executive of the British Acupuncture Council. What do you expect him to say?
I’ve just sent another complaint, about today’s BBC report. The more the merrier.
The BMJ itself published an editorial on the Madsen paper. It seems very odd that they should have chosen the editors of the BMJ-group journal, Acupuncture in Medicine, to comment on a paper that sounds the death knell for the subject from which they make their living. Needless to say, the editorial attempted to wriggle out of the obvious conclusions. Worse still, the Health editor of the BBC web site referred to the editorial in his defence, in response to my complaint.
Channel 4 News does a lot better than the BBC, or the BMJ editorial, with its report Acupuncture ‘fails to relieve pain’
“Claims that acupuncture can relieve pain have been undermined by the results of a new study.”
“senior researcher Asbjorn Hrobjartsson said: “Our findings question both the traditional foundation of acupuncture…and the prevailing theory that acupuncture has an important effect on pain in general”. “
The Daily Telegraph also, like almost everyone else, did better than the BBC, with “Acupuncture ‘has almost no effect in relieving pain’”
“The pain relieving effects of acupuncture are so small that they may be clinically irrelevant, according to a review of research into the treatment.”
Even Metro, the free London Newspaper, is more accurate then the BBC. They carry a short report “Acupuncture ‘has no medical point’.”
“The pain-relieving effects of acupuncture compared with a placebo are so small they may be clinically irrelevant.”
New review puts in doubt traditional foundation of acupuncture. Believe it or not, that is the title of the report on Madsen et al from the Prince’s Foundation for Integretad Health! Don’t get too excited though. They haven’t even bothered to look at the original paper, but merely cite the dreadful BBC report. Much prominence is given to the acupuncturists, White and O’Farrell, and the important finding isn’t mentioned at all. Another gamma minus.
I’m perfectly happy to think of alternative medicine as being a voluntary, self-imposed tax on the gullible (to paraphrase Goldacre again). But only as long as its practitioners do no harm and only as long as they obey the law of the land. Only too often, though, they do neither.
When I talk about law, I don’t mean lawsuits for defamation. Defamation suits are what homeopaths and chiropractors like to use to silence critics. heaven knows, I’ve becomes accustomed to being defamed by people who are, in my view. fraudsters, but lawsuits are not the way to deal with it.
I’m talking about the Trading Standards laws Everyone has to obey them, and in May 2008 the law changed in a way that puts the whole health fraud industry in jeopardy.
The gist of the matter is that it is now illegal to claim that a product will benefit your health if you can’t produce evidence to justify the claim.
I’m not a lawyer, but with the help of two lawyers and a trading standards officer I’ve attempted a summary. The machinery for enforcing the law does not yet work well, but when it does, there should be some very interesting cases.
The obvious targets are homeopaths who claim to cure malaria and AIDS, and traditional Chinese Medicine people who claim to cure cancer.
But there are some less obvious targets for prosecution too. Here is a selection of possibilities to savour..
- Universities such as Westminster, Central Lancashire and the rest, which promote the spreading of false health claims
- Hospitals, like the Royal London Homeopathic Hospital, that treat patients with mistletoe and marigold paste. Can they produce any real evidence that they work?
- Edexcel, which sets examinations in alternative medicine (and charges for them)
- Ofsted and the QCA which validate these exams
- Skills for Health and a whole maze of other unelected and unaccountable quangos which offer “national occupational standards” in everything from distant healing to hot stone therapy, thereby giving official sanction to all manner of treatments for which no plausible evidence can be offered.
- The Prince of Wales Foundation for Integrated Health, which notoriously offers health advice for which it cannot produce good evidence
- Perhaps even the Department of Health itself, which notoriously referred to “psychic surgery” as a profession, and which has consistently refused to refer dubious therapies to NICE for assessment.
The law, insofar as I’ve understood it, is probably such that only the first three or four of these have sufficient commercial elements for there to be any chance of a successful prosecution. That is something that will eventually have to be argued in court.
But lecanardnoir points out in his comment below that The Prince of Wales is intending to sell herbal concoctions, so perhaps he could end up in court too.
The laws
We are talking about The Consumer Protection from Unfair Trading Regulations 2008. The regulations came into force on 26 May 2008. The full regulations can be seen here, or download pdf file. They can be seen also on the UK Statute Law Database.
The Office of Fair Trading, and Department for Business, Enterprise & Regulatory Reform (BERR) published Guidance on the Consumer Protection from Unfair Trading Regulations 2008 (pdf file),
Statement of consumer protection enforcement principles (pdf file), and
The Consumer Protection from Unfair Trading Regulations: a basic guide for business (pdf file).
Has The UK Quietly Outlawed “Alternative” Medicine?
On 26 September 2008, Mondaq Business Briefing published this article by a Glasgow lawyer, Douglas McLachlan. (Oddly enough, this article was reproduced on the National Center for Homeopathy web site.)
“Proponents of the myriad of forms of alternative medicine argue that it is in some way “outside science” or that “science doesn’t understand why it works”. Critical thinking scientists disagree. The best available scientific data shows that alternative medicine simply doesn’t work, they say: studies repeatedly show that the effect of some of these alternative medical therapies is indistinguishable from the well documented, but very strange “placebo effect” ”
“Enter The Consumer Protection from Unfair Trading Regulations 2008(the “Regulations”). The Regulations came into force on 26 May 2008 to surprisingly little fanfare, despite the fact they represent the most extensive modernisation and simplification of the consumer protection framework for 20 years.”
The Regulations prohibit unfair commercial practices between traders and consumers through five prohibitions:-
- General Prohibition on Unfair Commercial
Practices (Regulation 3)- Prohibition on Misleading Actions (Regulations 5)
- Prohibition on Misleading Omissions (Regulation 6)
- Prohibition on Aggressive Commercial Practices (Regulation 7)
- Prohibition on 31 Specific Commercial Practices that are in all Circumstances Unfair (Schedule 1). One of the 31 commercial practices which are in all circumstances considered unfair is “falsely claiming that a product is able to cure illnesses, dysfunction or malformations”. The definition of “product” in the Regulations includes services, so it does appear that all forms medical products and treatments will be covered.
Just look at that!
One of the 31 commercial practices which are in all circumstances considered unfair is “falsely claiming that a product is able to cure illnesses, dysfunction or malformations” |
Section 5 is equally powerful, and also does not contain the contentious word “cure” (see note below)
Misleading actions
5.—(1) A commercial practice is a misleading action if it satisfies the conditions in either paragraph (2) or paragraph (3).
(2) A commercial practice satisfies the conditions of this paragraph—
(a) if it contains false information and is therefore untruthful in relation to any of the matters in paragraph (4) or if it or its overall presentation in any way deceives or is likely to deceive the average consumer in relation to any of the matters in that paragraph, even if the information is factually correct; and
(b) it causes or is likely to cause the average consumer to take a transactional decision he would not have taken otherwise.
These laws are very powerful in principle, But there are two complications in practice.
One complication concerns the extent to which the onus has been moved on to the seller to prove the claims are true, rather than the accuser having to prove they are false. That is a lot more favourable to the accuser than before, but it’s complicated.
The other complication concerns enforcement of the new laws, and at the moment that is bad.
Who has to prove what?
That is still not entirely clear. McLachlan says
“If we accept that mainstream evidence based medicine is in some way accepted by mainstream science, and alternative medicine bears the “alternative” qualifier simply because it is not supported by mainstream science, then where does that leave a trader who seeks to refute any allegation that his claim is false?
Of course it is always open to the trader to show that his the alternative therapy actually works, but the weight of scientific evidence is likely to be against him.”
On the other hand, I’m advised by a Trading Standards Officer that “He doesn’t have to refute anything! The prosecution have to prove the claims are false”. This has been confirmed by another Trading Standards Officer who said
“It is not clear (though it seems to be) what difference is implied between “cure” and “treat”, or what evidence is required to demonstrate that such a cure is false “beyond reasonable doubt” in court. The regulations do not provide that the maker of claims must show that the claims are true, or set a standard indicating how such a proof may be shown.”
The main defence against prosecution seems to be the “Due diligence defence”, in paragraph 17.
Due diligence defence
17. —(1) In any proceedings against a person for an offence under regulation 9, 10, 11 or 12 it is a defence for that person to prove—
(a) that the commission of the offence was due to—
(i) a mistake;
(ii) reliance on information supplied to him by another person;
(iii) the act or default of another person;
(iv) an accident; or
(v) another cause beyond his control; and
(b) that he took all reasonable precautions and exercised all due diligence to avoid the commission of such an offence by himself or any person under his control.
If “taking all reasonable precautions” includes being aware of the lack of any good evidence that what you are selling is effective, then this defence should not be much use for most quacks.
Douglas McLachlan has clarified, below, this difficult question
False claims for health benefits of foods
A separate bit of legislation, European regulation on nutrition and health claims made on food, ref 1924/2006, in Article 6, seems clearer in specifying that the seller has to prove any claims they make.
Article 6
Scientific substantiation for claims
1. Nutrition and health claims shall be based on and substantiated by generally accepted scientific evidence.
2. A food business operator making a nutrition or health claim shall justify the use of the claim.
3. The competent authorities of the Member States may request a food business operator or a person placing a product on the market to produce all relevant elements and data establishing compliance with this Regulation.
That clearly places the onus on the seller to provide evidence for claims that are made, rather than the complainant having to ‘prove’ that the claims are false.
On the problem of “health foods” the two bits of legislation seem to overlap. Both have been discussed in “Trading regulations and health foods“, an editorial in the BMJ by M. E. J. Lean (Professor of Human Nutrition in Glasgow).
“It is already illegal under food labelling regulations (1996) to claim that food products can treat or prevent disease. However, huge numbers of such claims are still made, particularly for obesity ”
“The new regulations provide good legislation to protect vulnerable consumers from misleading “health food” claims. They now need to be enforced proactively to help direct doctors and consumers towards safe, cost effective, and evidence based management of diseases.”
In fact the European Food Standards Agency (EFSA) seems to be doing a rather good job at imposing the rules. This, predictably, provoked howls of anguish from the food industry There is a synopsis here.
“Of eight assessed claims, EFSA’s Panel on Dietetic Products, Nutrition and Allergies (NDA) rejected seven for failing to demonstrate causality between consumption of specific nutrients or foods and intended health benefits. EFSA has subsequently issued opinions on about 30 claims with seven drawing positive opinions.”
“. . . EFSA in disgust threw out 120 dossiers supposedly in support of nutrients seeking addition to the FSD’s positive list.
If EFSA was bewildered by the lack of data in the dossiers, it needn’t hav been as industry freely admitted it had in many cases submitted such hollow documents to temporarily keep nutrients on-market.”
Or, on another industry site, “EFSA’s harsh health claim regime”
“By setting an unworkably high standard for claims substantiation, EFSA is threatening R&D not to mention health claims that have long been officially approved in many jurisdictions.”
Here, of course,”unworkably high standard” just means real genuine evidence. How dare they ask for that!
Enforcement of the law
Article 19 of the Unfair Trading regulations says
19. —(1) It shall be the duty of every enforcement authority to enforce these Regulations.
(2) Where the enforcement authority is a local weights and measures authority the duty referred to in paragraph (1) shall apply to the enforcement of these Regulations within the authority’s area.
Nevertheless, enforcement is undoubtedly a weak point at the moment. The UK is obliged to enforce these laws, but at the moment it is not doing so effectively.
A letter in the BMJ from Rose & Garrow describes two complaints under the legislation in which it appears that a Trading Standards office failed to enforce the law. They comment
” . . . member states are obliged not only to enact it as national legislation but to enforce it. The evidence that the government has provided adequate resources for enforcement, in the form of staff and their proper training, is not convincing. The media, and especially the internet, are replete with false claims about health care, and sick people need protection. All EU citizens have the right to complain to the EU Commission if their government fails to provide that protection.”
This is not a good start. A lawyer has pointed out to me
“that it can sometimes be very difficult to get Trading Standards or the OFT to take an interest in something that they don’t fully understand. I think that if it doesn’t immediately leap out at them as being false (e.g “these pills cure all forms of cancer”) then it’s going to be extremely difficult. To be fair, neither Trading Standards nor the OFT were ever intended to be medical regulators and they have limited resources available to them. The new Regulations are a useful new weapon in the fight against quackery, but they are no substitute for proper regulation.”
Trading Standards originated in Weights and Measures. It was their job to check that your pint of beer was really a pint. Now they are being expected to judge medical controversies. Either they will need more people and more training, or responsibility for enforcement of the law should be transferred to some more appropriate agency (though one hesitates to suggest the MHRA after their recent pathetic performance in this area).
Who can be prosecuted?
Any “trader”, a person or a company. There is no need to have actually bought anything, and no need to have suffered actual harm. In fact there is no need for there to be a complainant at all. Trading standards officers can act on their own. But there must be a commercial element. It’s unlikely that simply preaching nonsense would be sufficient to get you prosecuted, so the Prince of Wales is, sadly, probably safe.
Universities who teach that “Amethysts emit high Yin energy” make an interesting case. They charge fees and in return they are “falsely claiming that a product is able to cure illnesses”.
In my view they are behaving illegally, but we shan’t know until a university is taken to court. Watch this space.
The fact remains that the UK is obliged to enforce the law and presumably it will do so eventually. When it does, alternative medicine will have to change very radically. If it were prevented from making false claims, there would be very little of it left apart from tea and sympathy
Follow-up
New Zealand must have similar laws.
Just as I was about to post this I found that in New Zealand a
“couple who sold homeopathic remedies claiming to cure bird flu, herpes and Sars (severe acute respiratory syndrome) have been convicted of breaching the Fair Trading Act.”
They were ordered to pay fines and court costs totalling $23,400.
A clarification form Douglas McLachlan
On the difficult question of who must prove what, Douglas McLachlan, who wrote Has The UK Quietly Outlawed “Alternative” Medicine?, has kindly sent the following clarification.
“I would agree that it is still for the prosecution to prove that the trader committed the offence beyond a reasonable doubt, and that burden of proof is always on the prosecution at the outset, but I think if a trader makes a claim regarding his product and best scientific evidence available indicates that that claim is false, then it will be on the trader to substantiate the claim in order to defend himself. How will the trader do so? Perhaps the trader might call witness after witness in court to provide anecdotal evidence of their experiences, or “experts” that support their claim – in which case it will be for the prosecution to explain the scientific method to the Judge and to convince the Judge that its Study evidence is to be preferred.
Unfortunately, once human personalities get involved things could get clouded – I could imagine a small time seller of snake oil having serious difficulty, but a well funded homeopathy company engaging smart lawyers to quote flawed studies and lead anecdotal evidence to muddy the waters just enough for a Judge to give the trader the benefit of the doubt. That seems to be what happens in the wider public debate, so it’s easy to envisage it happening a courtroom.”
The “average consumer”.
(3) A commercial practice is unfair if—
(a) it contravenes the requirements of professional diligence; and
(b) it materially distorts or is likely to materially distort the economic behaviour of the average consumer with regard to the product.
It seems,therefore, that what matters is whether the “average consumer” would infer from what is said that a claim was being made to cure a disease. The legal view cited by Mojo (comment #2, below) is that expressions such as “can be used to treat” or “can help with” would be considered by the average consumer as implying successful treatment or cure.
The drugstore detox delusion. A nice analysis “detox” at .Science-based Pharmacy
Today brings a small setback for those of us interested in spreading sensible ideas about science. According to a press release
“The BMJ Group is to begin publishing a medical journal on acupuncture from next year, it was announced today (Tuesday 11 November 2008).
This will be the first complementary medicine title that the BMJ Group has published.”
And they are proud of that? What one earth is going on? The BMJ group is a publishing company which says, of itself,
“Our brand stands for medical credibility. We are one of the world’s best known and most respected medical publishers.”
Well perhaps it used to be.
They have certainly picked a very bad moment for this venture. In the last year there have been at least five good books that assess the evidence carefully and honestly. Of these, the ones that are perhaps the best on the subject of acupuncture are Singh & Ernst’s Trick or Treatment and Barker Bausell’s Snake Oil Science. Both Ernst and Bausell have first hand experience of acupuncture research. And crucially, none of these authors has any financial interest in whether the judgement goes for acupuncture or against it.
Here are quotations from Singh & Ernst’s conclusions
“Reliable conclusions from systematic reviews make it clear that acupuncture does not work for a whole range of conditions, except as a placebo.”
“There are some high quality trials that support the use of acupuncture for some types of pain and nausea, but there are also high quality trials that contradict this conclusion. In short, the evidence is neither consistent nor convincing – it is borderline.”
The House of Lords’ report in 2000 tended to give acupuncture the benefit of the (very considerable) doubt that existed at the time the report was written. Since that time there have been a lot of very well-designed trials of acupuncture.
Now it is quite clear that, for most (and quite possibly all) conditions, acupuncture is no more than a particularly theatrical placebo. Perhaps that is not surprising insofar as the modern western practice of acupuncture owes more to Chinese nationalistic propaganda that started in the time of Mao-Tse Tung than it owes to ancient wisdom (which often turns out to be bunk anyway).
The BMJ Group has decided to endorse acupuncture at a time when it is emerging that the evidence for any specific effect is very thin indeed. Well done.
The journal in question is this.
Acupuncture in Medicine is a quarterly title, which aims to build the evidence base for acupuncture. It is currently self-published by the British Medical Acupuncture Society (BMAS).
One good thing can be said about the Society and the Journal. That is that they don’t espouse all the mumbo-jumbo about ‘meridians’ and ‘Qi’. This, of course, puts them at odds with the vast majority of acupuncture teaching. This sort of internecine warfare between competing sects is characteristic of all sorts of alternative medicine. But that is just ideology. What matters is whether or not sticking needles in you is actually anything more than a placebo.
British Medical Acupuncture Society (BMAS)
The British Medical Acupuncture Society (BMAS). is “a registered charity established to encourage the use and scientific understanding of acupuncture within medicine for the public benefit.”. The phrase “encourage the use” suggests that they do not even envisage the possibility that it might not work. Their web site includes these claims.
Acupuncture can help in a variety of conditions:
- Pain relief for a wide range of painful conditions
- Nausea, especially postoperative nausea and vomiting
- Overactive bladder, also known as bladder detrusor instability
- Menstrual and menopausal problems, eg period pains and hot flushes
- Allergies such as hay fever and some types of allergic rashes
- Some other skin problems such as ulcers, itching and localised rashes
- Sinus problems and more
Presumably the word “help” is chosen carefully to fall just short of “cure”. The claims are vaguely worded, but let’s see what we can find about them from systematic reviews. It appears that the BMAS is being rather optimistic about the evidence.
BMJ Clinical Evidence is considered reliable and is particular interesting because it is owned by the BMJ Publishing Group.
Low Back Pain (chronic) Acupuncture is listed as being of “unknown effectiveness”.
Dysmenorrhoea Acupuncture is listed as being of “unknown effectiveness”.
Osteoarthritis of the knee. Acupuncture is listed as being of “unknown effectiveness”.
Psoriasis (chronic plaque) Acupuncture is listed as being of “unknown effectiveness”.
Neck pain “Acupuncture may be more effective than some types of sham treatment (not further defined) or inactive treatment (not further defined) at improving pain relief at the end of treatment or in the short term (less than 3 months), but not in the intermediate term (not defined) or in the long term (not defined)”
Headache (chronic tension-type) Acupuncture is listed as being of “unknown effectiveness”.
What about the greatest authority, the Cochrane Reviews?
Cochrane Reviews
Low back pain The data do not allow firm conclusions about the effectiveness of acupuncture for acute low-back pain. For chronic low-back pain, acupuncture is more effective for pain relief and functional improvement than no treatment or sham treatment immediately after treatment and in the short-term only. Acupuncture is not more effective than other conventional and “alternative” treatments. The data suggest that acupuncture and dry-needling may be useful adjuncts to other therapies for chronic low-back pain. Because most of the studies were of lower methodological quality, there certainly is a further need for higher quality trials in this area.
Chronic asthma. There is not enough evidence to make recommendations about the value of acupuncture in asthma treatment. Further research needs to consider the complexities and different types of acupuncture.
But most of the vaguely-worded claims made by BMAS have not been the subject of Cochrane reviews. The obvious interpretation of that is that there is not enough evidence to make it worth writing a review. In which case, why does BMAS claim that acupuncture can “help”?
Bandolier is another excellent source of high quality information, This was their view in September 2006
“Large, high-quality randomised trials of acupuncture have been published since the reviews. In fibromyalgia, chemotherapy-induced nausea and vomiting, breech presentation, tension headache, and migraine, all were negative compared with sham acupuncture. One in osteoarthritis of the knee, had statistical improvement over sham acupuncture at three months, but not later. Both large trials and this review of reviews come to the same general conclusion; that over a whole range of conditions and outcomes acupuncture cannot yet be shown to be effective.”
After thousands of years of acupuncture (or at least almost 40 years in the West) there seems to be very little to show for it.
The journal: Acupuncture in Medicine
What about the journal in question? Like all journals devoted to alternative medicine, it claims to be evidence-based. And like all journals devoted to alternative medicine it suffers from a fatal conflict of interest. If this journal were ever to conclude that acupuncture is a placebo, it would destroy the journal and the livelihoods of many of the people who write for it.
Scanning the first three issues of 2008 shows that it is very much like other alternative medicine journals. Most of the papers don’t address the critical question, is it a placebo. And most papers end up rather limply, with a statement along the lines “acupuncture may be useful for ***. More research is needed.”
The editor in chief of the journal is Dr Adrian White, and its editor is Michael Cummings. White is quoted by Ernst in the Guardian in 2004.
“We need to provide hard evidence to support what we all see in our clinics every day: that the modern approach to acupuncture works, and is highly relevant to the new, patient-centred NHS.” .
That means the answer is assumed in advance. That just isn’t science.. ‘We know the answer, all we have to do now is get some evidence’.
Why should the BMJ Group want to do a thing like this?
The press release says
Commenting on the move, BMJ and BMJ Journals Publishing Director, Peter Ashman, said “The journal is a good complement for our existing portfolio of journals and we’re certain that the Society’s members and other subscribers will appreciate the benefits of the decision the BMAS has made on their behalf.”
He continued: “The BMAS is ambitious for its journal to grow and flourish and we’re looking forward to working with the Society to develop an editorial and commercial strategy which will achieve the aims of BMAS and those of its members, while reaching out to the wider global community interested in this fascinating area of medicine.”
Yes, you got it. Money. The same motive that causes some vice-chancellors to bring their university into disrepute by awarding BSc degrees in subjects that are not only not science, but which are oftenly openly anti-science.
Conscience doesn’t seem to bother these people, so let’s put the problem in purely cash terms.
Both the BMJ Group and the vice-chancellors will have to decide whether the cash they gain is sufficient to counterbalance the corrosive effects of their actions on their own reputations.
Follow-up
Only a couple of days later, two new trials show acupuncture is no different from sham controls for helping IVF pregnancy rates. James Randerson in the Guardian writes thus.
“Acupuncture aimed at improving IVF success rates is widely offered by fertility clinics in the UK. In the first of the studies, researchers in Hong Kong split 370 women receiving IVF into two groups. One group received real acupuncture before and after having an IVF embryo implanted into their uterus. The other had the same procedure, except the treatment used retractable needles that did not penetrate the skin.”
“Of the 185 who received the sham treatment, 91 achieved a clinical pregnancy (foetal heartbeat identified using ultrasound) and 71 had a successful delivery. This compared with 72 clinical pregnancies in the true acupuncture group and 55 live births. The differences between the groups were not statistically significant.”
“In a second study, researchers in Chicago used a similar design in which 124 women received true or sham acupuncture. The control group had their skin punctured by real acupuncture needles, but not at genuine “Qi-lines” on the body. In the true acupuncture group, 43.9% achieved a clinical pregnancy, compared with 55.2% of the women given the sham treatment. “
The original paper for the first study can be seen here.
I have always thought that our undergraduates had difficulty in expressing themselves clearly, in simple words. But they are models of clear thought compared with Christine Barry’s recent paper (Social Science and Medicine, 62, 2464-2657, 2006).
Barry’s work rivals Alan Sokal’s famous spoof paper, “Transgressing the boundaries: the Hermeneutics of quantum gravity”. Sokal’s paper opens as follows.
“There are many natural scientists, and especially physicists, who continue to reject the notion that the disciplines concerned with social and cultural criticism can have anything to contribute, except perhaps peripherally, to their research. Still less are they receptive to the idea that the very foundations of their worldview must be revised or rebuilt in the light of such criticism. Rather, they cling to the dogma imposed by the long post-Enlightenment hegemony over the Western intellectual outlook, which can be summarized briefly as follows:
that there exists an external world, whose properties are independent of any individual human being and indeed of humanity as a whole; that these properties are encoded in “eternal” physical laws; and that human beings can obtain reliable, albeit imperfect and tentative, knowledge of these laws by hewing to the “objective” procedures and epistemological strictures prescribed by the (so-called) scientific method.”
Compare this with the opening of Barry’s paper.
“Calls for ‘gold standard’ randomised controlled trial evidence, by both biomedical and political establishments, to legitimise the integration of alternative medicine into healthcare systems, can be interpreted as deeply political. In this paper, the supposed objectivity of scientific, biomedical forms of evidence is questioned through an illumination of the multiple rhetorics embedded in the evidence-based medicine phenomenon, both within biomedicine itself and in calls for its use to evaluate alternative therapeutic systems.”
and, later,
“I wish to problematise the call from within biomedicine for more evidence of alternative medicine’s effectiveness via the medium of the randomised clinical trial (RCT).”
“Ethnographic research in alternative medicine is coming to be used politically as a challenge to the hegemony of a scientific biomedical construction of evidence.”
“The science of biomedicine was perceived as old fashioned and rejected in favour of the quantum and chaos theories of modern physics.”
“In this paper, I have deconstructed the powerful notion of evidence within biomedicine, . . .”
Just one difference, though, Sokal’s paper was a spoof, which brilliantly exploded the pretentious nonsense of post-modernism. Barry’s paper, is, I very much fear, intended to be serious.
To make matters worse, this work was funded by the Department of Health.
Sokal’s story is told in his devastating book, “Intellectual Impostures”. Strongly recommended, if you want to retain your sanity. [Link to Amazon]. Here is a quotation from the book, concerning Jacques-Marie-Emile Lacan (1901 – 1981), a French psychoanalyst and psychiatrist.
[Jouissance = enjoyment; the word appears in French in the translations.]
In it the square root of -1 is related to erectile function in a piece of gobbledygook which shows an understanding of mathematics about as profound as Barry’s understanding of “the quantum and chaos theories of modern physics”.
It seems that validation committees often don’t look beyond the official documents. As a result, the validations may not be worth the paper they are written on. Try this one.
One of the best bits of news recently was the downfall of Matthias Rath. He’s the man who peddled vitamin pills for AIDS in Africa, and encouraged the AIDS denialists in the South African government. Thabo Mbeki and his Health Minister, Mrs Beetroot, have gone now, thank heavens.
Rath was one of the best illustrations of the murderous effect of selling ineffective treatments. The fact that nobody in the “nutritional therapy” industry has uttered a word of condemnation for this man illustrates better than anything one can imagine the corrupt state of “nutritional therapy”. The people who kept silent include the British Association of Nutritional Therapists (BANT).
It might be surprising, then, to find the Northern College of Acupuncture proudly adding a course in alternative nutrition to its courses in acupuncture (now known to be a theatrical placebo) and Chinese herbal medicine (largely untested and sometimes toxic). It might be even more surprising to find the boast that the course is validated by the University of Wales. It seemed a good idea to find out a bit more about how this came about. Thanks to the Freedom of Information Act, some interesting things can be discovered.
Polly Toynbee’s superb article, Quackery and superstition – available soon on the NHS, written in January 2008, mentioned diplomas and degrees in complementary therapies offered by, among others, the University of Wales. This elicited a letter of protest to Toynbee from the Vice-Chancellor of the University of Wales, Professor Marc Clement BSc, PhD, MInstP, CEng,CPhys,FIET. He invited her to visit the university to see their “validation and monitoring procedures (including the University’s very specific guidelines on health studies disciplines”.
So let’s take a look at these validation procedures and guidelines.
The validation process
The Northern College of Acupuncture submitted a 148 page proposal for the course in October 2007. The document has all the usual edu-bollocks jargon, but of course doesn’t say much about clinical trials, though it does boast about an unblinded trial of acupuncture published in 2006 which, because of lack of appropriate controls, served only to muddy the waters. : This submission was considered by the University’s validation committee last December.
|
The whole validation document is only four pages long [download it]. The most interesting thing about it is that the words ‘evidence’ or ‘critical’ do not occur in it a single time. It has all the usual bureaucratic jargon of such documents but misses entirely the central point.
Does that mean that the University of Wales doesn’t care about evidence or critical thinking? Well, not on paper. Two years previously a short document called Health Studies Guidelines had been written by Dr Brian Spriggs (Health Studies Validation Consultant, since retired) for the Health Studies Committee, and it was approved on 21 April 2005. It starts well.
“Degrees in the Health Studies field are expected to promote an understanding of the importance of the scientific method and an evidence-base to underpin therapeutic interventions and of research to expand that base.”
It even goes on to say that a BSc degree in homeopathy is “unacceptable”. Don’t get too excited though, because it also says that acupuncture and Chinese herbal stuff is quite OK. How anyone can imagine they live up to the opening sentence beats me. And it gets worse. It says that all sorts of rather advanced forms of battiness are OK if they form only part of another degree. They include Homeopathy, Crystal therapy. Dowsing, Iridology; Kinesiology, Radionics, Reflexology, Shiatsu, Healing, and Maharishi Ayurvedic Medicine.
Dowsing? Crystal therapy? Just let me remind you. We are living in 2008. It is easy to forget that when ploughing through all this new age junk.
The Validation Handbook of Quality Assurance: Health Studies (2007) runs to an astonishing 256 pages [download the whole thing]. On page 12 we find the extent of the problem.
“The University of Wales validates a number of schemes in the Health Studies field. At the current time we have undergraduate and/or postgraduate degree schemes in Acupuncture, Animal Manipulation, Chiropractic, Herbal Medicine, Integrative Psychotherapy, Osteopathy, Osteopathic Studies, Traditional Chinese Medicine and Regulatory Affairs, both in the UK and overseas.”
That sounds pretty shocking. Further down on page 12, though, we find this.
“Degrees in the Health Studies field are expected to promote an understanding of the importance of the scientific method and an evidence-base to underpin therapeutic interventions and of research to expand that base. The mission is to promote and require the critical evaluation of the practices, doctrines, beliefs, theories and hypotheses that underlie the taught therapeutic measures of the discipline.”
They are indeed fine words. The problem is that I can detect no sign in the submission, nor in its consideration by the validation committee, that any attempt whatsoever was made to ensure that the course complied with these requirements.
The only sign of concern I could detect of any concern about the quality of what was being taught came in a minute to a meeting of the Health Studies Committee meeting on 24th April 2008.
“Members received a copy of an article entitled Quackery and superstition available soon on the NHS which appeared in The Guardian newspaper in January 2008, and a copy of the Vice- Chancellors response. Members agreed that this article was now historical but felt that if/when the issue were to arise again; the key matter of scientific rigour should be stressed. The Committee agreed that this was the most critical element of all degree schemes in the University of Wales portfolio of health studies schemes. It was felt it would be timely to re-examine the schemes within the portfolio as well as the guidelines for consideration of Health Studies schemes at the next meeting. The Committee might also decide that Institutions would be required to include literature reviews (as part of their validation submission) to provide evidence for their particular profession/philosophy. It was agreed that the guidelines would be a vital document in the consideration of new schemes and during preliminary visits to prospective Institutions. “
The Press Office had passed Polly Toynbee’s article to them. Curiously the Health Studies Committee dismissed it as “historical”, simply because it was written three months earlier. That is presumably “historical” in the sense that the public will have forgotten about it, rather than in the sense that the facts of the matter have changed since January. So, at least for the nutrition degree, Toynbee’s comments were simply brushed under the carpet.
After a few cosmetic changes of wording the validation was completed on 16th January 2008. For example the word “diagnosis” was removed in 43 places and “rewritten in terms of evaluation and assessment”. There was, needless to say, no indication that the change in wording would change anything in what was taught to students.
You may think that I am being a bit too harsh. Perhaps the course is just fine after all? The problem is that the submission and the reaction of the validation committee tell you next to nothing about what actually matters, and that is what is taught. There is only a vague outline of that in the submission (and part of it was redacted on the grounds that if it were made public somebody might copy ;it. Heaven forbid).
That is why I have to say, yet again, that this sort of validation exercise is not worth the paper it’s written on.
How can we find out a bit more? Very easily as it happens. Just Google. What matters is not so much formal course outlines but who teaches them.
The nutrition course
The title of the course is just “Nutrition”, not ‘Nutritional Therapy’ or ‘Alternative Nutrition’. That sounds quite respectable but a glance at the prospectus shows immediately that it is full-blown alternative medicine.
Already in July 2007, the glowing press releases for the course had attracted attention from the wonderfully investigative web site HolfordWatch. I see no sign that the validation committee was aware of this. But if not, why not? I would describe is as dereliction of academic duty.
“This pioneering course is unique in that it is firmly rooted in both Western nutritional science and naturopathic medicine and also covers concepts of nutrition within traditional Chinese, Japanese, Tibetan and Ayurvedic medicine.
This means that graduates will gain comprehensive understanding of both modern scientific knowledge and ancient wisdom concerning nutrition and dietetics.”
Ancient wisdom, of course, means something that your are supposed to believe though there is no good reason to think it’s true. In the end, though, almost the only thing that really matters about any course is who is running it. The brochure shows that all of the people are heavily into every form of alternative nuttiness.
Course Director and Tutor: Jacqueline Young nutritionist, naturopath, clinical psychologist and Oriental medical practitioner
Nutrition Tutors:
Elaine Aldred (qualified as a chiropractor with the Anglo European Chiropractic College, as an acupuncturist with the British College of Acupuncture and as a Western Medical Herbalist with the College of Phytotherapy. She recently also qualified in Chinese herbal medicine with the Northern College of Acupuncture.)
Sue Russell (3 year diploma in nutritional therapy at the Institute of Optimum Nutrition. She currently practises as a nutritional therapist and also works part-time as a manager at the Northern College of Homeopathic Medicine.)
Anuradha Sharma (graduated as a dietician from Leeds Metropolitan University in 2002 and subsequently completed a Naturopathy certificate and a post-graduate diploma in acupuncture).
Guest Lecturers include : Dr John Briffa, Professor Jane Plant, M.B.E. (a geochemist turned quack), and, most revealingly, none other than the UK’s most notorious media celebrity and pill peddler, Patrick Holford.
So much has been written about Holford’s appalling abuse of science, one would have thought that not even a validation committee could have missed it.
“The course has been created by Jacqueline Young“, so let’s look a bit further at her track record.
Jacqueline Young has written a book, ‘Complementary Medicine for Dummies’ [Ed: ahem shouldn’t that be Dummies for Complementary Medicine?]. You can see parts of it on Google Books. Did the validation committee bother to look at it? As far as I can tell, the words ‘randomised’ or ‘clinical trial’ occur nowhere in the book.
The chapter on Tibetan medicine is not very helpful when it comes to evidence but for research we are referred to the Tibetan Medical and Astrology Institute. Guess what? That site gives no evidence either. So far not a single university has endorsed Astrology (there is a profitable niche there for some vice-chancellor).
Here are few samples from the book. The advice seems to vary from the undocumented optimism of this
Well researched? No. Safe? Nobody knows. Or this
Mandarin peel prevents colds and flu? Old wive’s tale. Then there are things that verge on the weird, like this one
or the deeply bizarre like this
The problem of Jacqueline Young’s fantasy approach to facts was pointed out at least as far back as 2004, by Ray Girvan., who wrote about it again in May 2005. The problems were brought to wider attention when Ben Goldacre wrote two articles in his Badscience column, Imploding Researchers (September 2005), and the following week, Tangled Webs.
“we were pondering the ethics and wisdom of Jacqueline Young dishing out preposterous, made-up, pseudoscientific nonsense as if it was authoritative BBC fact, with phrases such as: “Implosion researchers have found that if water is put through a spiral its electrical field changes and it then appears to have a potent, restorative effect on cells.” “
and later
“Take this from her article on cranial osteopathy, riddled with half truths: “Sutherland found that the cranial bones (the skull bones encasing the brain) weren’t fused in adulthood, as was widely believed, but actually had a cycle of slight involuntary movement.” In fact the cranial bones do fuse in adulthood.
She goes on: “This movement was influenced by the rhythmic flow of cerebrospinal fluid (the nourishing and protective fluid that circulates through the spinal canal and brain) and could become blocked.” There have now been five studies on whether “cranial osteopaths” can indeed feel these movements, as they claim, and it’s an easy experiment to do: ask a couple of cranial osteopaths to write down the frequency of the rhythmic pulses on the same person’s skull, and see if they give the same answer. They don’t. A rather crucial well-replicated finding to leave out of your story.
That was in 2005 and since then all of Young’s “preposterous, made-up, pseudoscientific nonsense” (along with most of the other stuff about junk medicine) has vanished from the BBC’s web site, after some people with a bit of common sense pointed out what nonsense it was. But now we see them resurfacing in a course validated by a serious university. The BBC had some excuse (after all, it is run largely by arts graduates). I can see no excuses for the University of Wales.
Incidentally, thanks to web archive you can still read Young’s nonsense, long after the BBC removed it. Here is a quotation.
“Implosion researchers have found that if water is put through a spiral its ,field changes and it then appears to have a potent, restorative effect on cells. In one study, seedlings watered with spiralised water grew significantly faster, higher and stronger than those given ordinary water.”
The vice-chancellor of the University of Wales, Marc Clement, is a physicist (Department of Electrical and Electronic Engineering), so can he perhaps explain the meaning of this?
Selection committees for jobs (especially senior jobs) and validation committees for courses, might make fewer mistakes if they didn’t rely so much on formal documents and did a little more investigation themselves. That sort of thing is why the managerial culture not only takes a lot more time, but also gives a worse result.
It would have taken 10 minutes with Google to find out about Young’s track record, but they didn’t bother. As a result they have spent a long time producing a validation that isn’t worth the paper it’s written on. That makes the University of Wales a bit of a laughing stock. Worse still, it brings science itself into disrepute.
Follow-up
What does the University of Wales say? So far, nothing. Last week I sent brief and polite emails to Professor Palastanga and to Professor Clement to try to discover whether it is true that the validation process had indeed missed the fact that the course organiser’s writings had been described as “preposterous, made-up, pseudoscientific nonsense” in the Guardian.
So far I have had no reply from the vice-chancellor, but on .26 October I did get an answer from Prof Palastaga.
As regards the two people you asked questions about – J.Young – I personally am not familiar with her book and nobody on the validation panel raised any concerns about it. As for P.Holford similarly there were no concerns expressed about him or his work. In both cases we would have considered their CV’s as presented in the documentation as part of the teaching team. In my experience of conducting degree validations at over 16 UK Universities this is the normal practice of a validation panel. |
I have to say this reply confirms my worst fears. Validation committees such as this one simply don’t do their duty. They don’t show the curiosity that is needed to discover the facts about the things that they are meant to be judging. How could they not have looked at the book by the very person that they are validating? After all that has been written about Patrick Holford, it is simply mind-boggling that the committee seems to have been quite unaware of any of it.
It is yet another example of the harm done to science by an unthinking, box-ticking approach.
Pharmacology. A Handbook for Complementary Healthcare Professionals
Elsevier were kind enough to send me an inspection copy of this book, which is written by one of the nutrition course tutors, Elaine Aldred. She admits that pharmacology is “considered by most students to be nothing more that a ‘hoop-jumping’ exercise in the process of becoming qualified”. She also says. disarmingly. that “I was certainly not the most adept scientist at school and found my university course a trial”.
The book has all the feel of a cut and paste job. It is mostly very simple (if not simplistic). though for no obvious reason it starts with a long (and very amateur) discussion of chemical bonding Then molecules are admitted to be indivisible (but, guess what, the subject of homeopathy is avoided). There is a very short section on ion channels, though, bizarrely, it appears under the heading “How do drugs get into cells?”. Since the author is clearly not able to make the distinction between volts and coulombs, the discussion is more likely to confuse the reader than to help.
Then a long section on plants. It starts of by asserting that “approximately a quarter of prescription drugs contain at least one chemical that was originally isolated and extracted from a plant”.. This cannot be even remotely correct. There are vast tables showing complicated chemical structures, but the usual inadequate
list of their alleged actions This is followed by a quick gallop through some classes of conventional drugs, illustrated again mainly by chemical structures not data. Hormone replacement therapy is mentioned, but the chance to point out that it is one of the best illustrations of the need for RCTs is missed.
The one thing that one would really like to see in such a book is a good account of how you tell whether or not a drug works in man. This is relegated to five pages at the end of the book, and it is, frankly, pathetic. It
is utterly uncritical in the one area that matters more than any other for people who purport to treat patients. All you get is a list of unexplained bullet points.
If this book is the source of the “scientific content” of the nutrition course, things are as bad as we feared.
Alfred Joseph Clark FRS held the established chair of Pharmacology at UCL from 1919 to 1926, when he left for Edinburgh. In the 1920s and 30s, Clark was a great pioneer in the application of quantitative physical ideas to pharmacology. As well as his classic scientific works, like The Mode of Action of Drugs on Cells (1933) he wrote, and felt strongly, about the fraud perpetrated on the public by patent medicine salesmen. In 1938 (while in Edinburgh) he published a slim volume called Patent Medicines. The parallels with today are astonishing.
Alfred Joseph Clark FRS (1885 – 1941) |
I was lucky to be given a copy of this book by David Clark, A.J. Clark’s eldest son, who is now 88. I visited him in Cambridge on 17 September 2008, because he thought that, as holder of the A.J. Clark chair at UCL from 1985 to 2004, I’d be a good person to look after this and several other books from his father’s library. They would have gone to the Department of Pharmacology if we still had one, but that has been swept away by mindless administrators with little understanding of how to get good science.
Quotations from the book are in italic, and are interspersed with comments from me.
The book starts with a quotation from the House of Commons Select Committee report on Patent Medicines. The report was submitted to the House on 4 August 1914, so there is no need to explain why it had little effect. The report differs from recent ones in that it is not stifled by the sort of political correctness that makes politicians refer to fraudsters as “professions”.
The problem
“2.2 The situation, therefore, as regards the sale and advertisement of proprietary medicines and articles may be summarised as follows:
For all practical purposes British law is powerless to prevent any person from procuring any drug, or making any mixture, whether patent or without any therapeutical activity whatever (as long as it does not contain a scheduled poison), advertising it in any decent terms as a cure for any disease or ailment, recommending it by bogus testimonials and the invented opinions and facsimile signatures of fictitious physicians, and selling it under any name he chooses, on payment of a small stamp duty. For any price he can persuade a credulous public to pay.”
Select Committee on Patent Medicines. 1914
“The writer has endeavoured in the present article to analyse the reasons for the amazing immunity of patent medicines form all attempts to curb their activity, to estimate the results and to suggest the obvious measures of reform that are needed.”
Clark, writing in 1938, was surprised that so little had changed since 1914. What would he have thought if he had known that now, almost 100 years after the 1914 report, the fraudsters are still getting away with it? Chapter 2 starts thus. |
THE LAW
The Select Committee appointed by the House of Commons in 1914 ‘to consider and inquire into the question of the sale of Patent and Proprietary Medicines’ stated its opinion in 28 pages of terse and uncompromising invective. Its general conclusions were as follows:
That the trade in secret remedies constituted a grave and widespread public evil.
That the existing law was chaotic and had proved inoperative and that consequently the traffic in secret remedies was practically uncontrolled.
In particular it concluded ‘”that this is an intolerable state of things and that new legislation to deal with it, rather than merely the amendment of existing laws, is urgently needed in the public interest.”
The “widespread public evil”continues almost unabated, and rather than introduce sensible legislation to cope with it, the government has instead given a stamp of approval for quackery by introducing utterly ineffective voluntary “self-regulation”.
Another Bill to deal with patent medicines was introduced in 1931, without success, and finally in 1936, a Medical and Surgical Appliances (Advertisement) Bill was introduced. This Bill had a very limited scope. Its purpose was to alleviate some of the worst abuses of the quack medicine trade by prohibiting the advertisement of cures for certain diseases such as blindness, Bright’s disease [nephritis] , cancer, consumption [tuberculosis], epilepsy, fits, locomotor ataxy, fits, lupus or paralysis.
The agreement of many interests was secured for this measure. The president of the Advertising Association stated that the proposed Bill would not affect adversely any legitimate trade interest. Opposition to the Bill was, however, whipped up amongst psychic healers, anti-vivisectionists and other opponents of medicine and at the second reading in March 1936, the Bill was opposed and the House was counted out during the ensuing debate. The immediate reason for this fate was that the Bill came up for second reading on the day of the Grand National! This is only one example of the remarkable luck that has attended the patent medicine vendors.
(Page 14).
The “remarkable luck” of patent medicine vendors continues to this day, Although, in principle, advertisement of cures for venereal diseases was banned in 1917, and for cancer in 1939, it takes only a few minutes with Google to find that these laws are regularly flouted by quacks, In practice quacks get away with selling vitamin pills for AIDS, sugar pills for malaria and homeopathic pills for rabies, polio anthrax and just about anything else you can think of. Most of these advertisements are contrary to the published codes of ethics of the organisations to which the quack in question belongs but nothing ever happens.
Self-regulation simply does not work, and there is still no effective enforcement even of existing laws..
“It has already been stated that British law allows the advertiser of a secret remedy to tell any lie or make any claim that he fancies will sell his goods and the completeness of this licence is best illustrated by the consideration of a few specific points.
Advertisements for secret remedies very frequently contain a list of testimonials from medical men, which usually are in an anonymous form, stating that ………….. M.D., F.R.C.S., has found the remedy infallible. Occasionally, however, the name and address of a doctor is given and anyone unaware of the vagaries of English law would imagine that such use of a doctor’s name and professional reputation could not be made with impunity without his consent. In 1899, however, the Sallyco Mineral Water Company advertised that ‘Dr. Morgan Dochrill, physician to St. John’s Hospital, London and many of the leading physicians are presenting ‘Sallyco’ as an habitual drink. Dr. Dochrill says nothing has done his gout so much good.
Dr. Dochrill, whose name and title were correctly stated above, sued the company but failed in his case. ”
“The statement that the law does not prevent the recommending of a secret remedy by the use of bogus testimonials and facsimile signatures of fictitious physicians is obviously an understatement since it is doubtful how far it interferes with the use of bogus testimonials from real physicians.”
Dodgy testimonials are still a mainstay of dodgy salesman. One is reminded of the unauthorised citation of testimonials from Dr John Marks and Professor Jonathan Waxman by Patrick Holford to aid his sales of unnecessary vitamin supplements. There is more on this at Holfordwatch.
The man in the street knows that the merits of any article are usually exaggerated in advertisements and is in the habit of discounting a large proportion of such claims, but, outside the realm of secret remedies, the law is fairly strict as regards definite misstatements concerning goods offered for sale and hence the everyday experience of the man in the street does not prepare him for dealing with advertisements which are not merely exaggerations but plain straightforward lies from beginning to end.
Scientific training is undoubtedly a handicap in estimating popular gullibility as regards nostrums. One imagines that no one today would be willing to spend money on pills guaranteed to prevent earthquakes but yet the claims of many of the remedies offered appear equally absurd to anyone with an elementary
knowledge of physiology or even of chemistry. A study of the successes and failures suggests that success depends chiefly on not over-rating the public intelligence. (Page 34)
This may have changed a bit since A.J. Clark was writing in 1938. Now the main clients of quacks seem to be the well-off “worried-well”. But it remains as true as ever that “Scientific training is undoubtedly a handicap in estimating popular gullibility as regards nostrums.” In 2008, it is perhaps more a problem of Ben Goldacre’s dictum ““My basic hypothesis is this: the people who run the media are humanities graduates with little understanding of science, who wear their ignorance as a badge of honour.”
Clark refers (page 36) to a successful conviction for fraud in the USA in 1917. The subject was a widely advertised ‘get fat quick’ pill that contained lecithin, proteins and sugar. The BMA analysis (in 1912)
suggested that the cost of the ingredients in a box of 30 tablets sold for 4/6 was 1 1/4 d. [4/6 meant 4 shillings and six pence, or 22.5 pence since 1971, and 1 1/4 old pence, a penny farthing, is 0.52 new pence]. He comments thus.
The trial revealed many interesting facts. The formula was devised after a short consultation with the expert of one of the largest drug manufacturers in the U.S.A. This firm manufactured the tablets and sold them to the proprietary medicine company at about 3/- per 1000, whilst they were retailed to the public at the rate of £7 10s. per 1000. The firm is estimated to have made a profit of about $3,000,000.
These trials in the U.S.A. revealed the fact that in a considerable proportion of cases the ‘private formula’ department of the large and well known drug firm already mentioned had first provided the formula for the nostrum and subsequently had prepared it wholesale.
Nothing much has changed here either. The alternative medicine industry (and it is a very big industry) is fond of denouncing the evils of the pharmaceutical industry, and sadly, occasionally they are right. One of the less honest practices of the pharmaceutical industry (though one never mentioned by quacks) is buying heavily into alternative medicine. Goldacre points out
“there is little difference between the vitamin and pharmaceutical industries. Key players in both include multinationals such as Roche and Aventis; BioCare, the vitamin pill producer that media nutritionist Patrick Holford works for, is part-owned by Elder Pharmaceuticals.”
And then. of course, there is the deeply dishonest promotion by Boots the Chemists of homeopathic miseducation, of vitamins and of CoQ10 supplements.
The manner in which secret remedies can survive repeated exposure is shown by the following summary of the life history of a vendor of a consumption [tuberculosis] cure.
1904, 1906: Convicted of violating the law in South Africa.
1908: Exposed in British Medical Association report and also attacked by Truth.
1910: Sued by a widow. The judge stated: ‘I think this is an intentional and well-considered fraud. It is a scandalous thing that poor people should be imposed upon and led to part with their money, and to hope that those dear to them would be cured by those processes which were nothing but quack remedies and had not the slightest value of any kind.’
1914: A libel action against the British Medical Association was lost.
1915 The cure was introduced into the United States.
1919 The cure was sold in Canada.
1924 Articles by men with medical qualifications appeared in the Swiss medical journal boosting
the cure.
Secret remedies have a vitality that resembles that of the more noxious weeds and the examples mentioned suggest that nothing can do them any serious harm.
Most of the time, quacks get away with claims every bit as outrageous today. But Clark does give one example of a successful prosecution. It resulted from an exposé in the newspapers -wait for it -in the Daily Mail.
There is, however, one example which proves that a proprietary remedy can be squashed by exposure if this is accompanied by adequate publicity.
The preparation Yadil was introduced as an antiseptic and was at first advertised to the medical profession. The proprietor claimed that the remedy was not secret and that the active principle was ‘tri-methenal allylic carbide’. The drug acquired popularity in the influenza epidemic of 1918 and the proprietor became more and more ambitious in his therapeutic claims. The special virtue claimed for Yadil was that it would kill any harmful organism that had invaded the body. A more specific claim was that consumption in the first stage was cured with two or three pints whilst advanced cases might require a little more. Other advertisements suggested that it was a cure for most known diseases from cancer downwards.
These claims were supported by an extraordinarily intense advertising campaign. Most papers, and even magazines circulating amongst the wealthier classes, carried full page and even double page advertisements. The Daily Mail refused these advertisements and in 1924 published a three column article by Sir William Pope, professor of Chemistry in the University of Cambridge. He stated that
the name ‘tri-methenal allylic carbide’ was meaningless gibberish and was not the chemical definition of any known substance. He concluded that Yadil consisted of :
‘About one per cent of the chemical compound formaldehyde.
About four per cent of glycerine.
About ninety-five per cent of water and, lastly, a smell.
He calculated that the materials contained in a gallon cost about 1/6, whilst the mixture was sold at £4 10s. per gallon.
This exposure was completely successful and the matter is of historic interest in that it is the only example of the career of a proprietary medicine being arrested by the action of the Press.
Clark goes on to talk of the law of libel.
“On the other hand the quack medicine vendor can pursue his advertising campaigns in the happy assurance that, whatever lies he tells, he need fear nothing from the interference of British law. The law does much to protect the quack medicine vendor because the laws of slander and libel are so severe.”
The law of libel to this day remains a serious risk to freedom of speech of both individuals and the media. Its use by rogues to suppress fair comment is routine. My first encounter was when a couple of herbalists
threatened to sue UCL because I said that the term ‘blood cleanser’ is gobbledygook. The fact that the statement was obviously true didn’t deter them for a moment. The herbalists were bluffing no doubt, but they caused enough nuisance that I was asked to take my pages off UCL’s server. A week later I was invited back but by then I’d set up a much better blog and the publicity resulted in an enormous increase in readership, so the outcome was good for me (but bad for herbalists).
It was also good in the end for Andy Lewis when his immortal page “The gentle art of homoeopathic killing” (about the great malaria scandal) was suppressed. The Society of Homeopaths’ lawyers didn’t go for him personally but for his ISP who gave in shamefully and removed the page. As a result the missing page reappeared in dozens of web sites round the world and shot to the top in a Google search.
Chiropractors are perhaps the group most likely to try to suppress contrary opinions by law not argument. The only lawyers’ letter that has been sent to me personally, alleged defamation in an editorial that I wrote for the New Zealand Medical Journal. That was a little scary, but the journal stuck up for its right to speak and the threat went away after chiropractors were allowed right of reply (but we got the last word).
Simon Singh, one of the best science communicators we have, has not been so lucky. He is going to have to defend in court an action brought by the British Chiropractic Association because of innocent opinions expressed in the Guardian.
Chapter 6 is about “The harm done by patent medicines”. It starts thus.
“The trade in secret remedies obviously represents a ridiculous waste of money but some may argue that, since we are a free country and it pleases people to waste their money in this particular way, there is no call for any legislative interference. The trade in quack medicines cannot, however, be regarded as a harmless one. The Poisons Acts fortunately prevent the sale of a large number of dangerous drugs, but there are numerous other ways in which injury can be produced by these remedies.”
The most serious harm, he thought, resulted from self-medication, and he doesn’t mince his words.
“The most serious objection to quack medicines is however that their advertisements encourage self-medication as a substitute for adequate treatment and they probably do more harm in this than in any other manner.
The nature of the problem can best be illustrated by considering a simple example such as diabetes. In this case no actual cure is known to medicine but, on the other hand, if a patient is treated adequately by insulin combined with appropriate diet, he can be maintained in practically normal health, in spite of his disability, for an indefinite period. The expectation of life of the majority of intelligent diabetics, who make no mistakes in their regime, is not much less than that of normal persons. The regime is both irksome and unpleasant, but anyone who persuades diabetics to abandon it, is committing manslaughter as certainly as if he fired a machine gun into a crowded street.
As regards serious chronic disease the influence of secret remedies may be said to range from murderous to merely harmful.
‘Cures’ for consumption, cancer and diabetes may fairly be classed as murderous, since they are likely to cause the death of anyone who is unfortunate enough to believe in their efficacy and thus delay adequate treatment until too late.
The phrase “‘Cures for consumption, cancer and diabetes may fairly be classed as murderous” made Clark himself the victim of suppression of freedom of speech by lawyers. His son, David Clark, wrote of his father in “Alfred Joseph Clark, A Memoir” (C. & J. Clark Ltd 1985 ISBN 0-9510401-0-3)
“Although tolerant of many human foibles, A. J. had always disapproved fiercely of quacks, particularly the charlatans who sold fraudulent medicines. During his visits to London he met Raymond Postgate, then a crusading left wing journalist, who persuaded A.J. to write a pamphlet which was published in an ephemeral series called ‘Fact‘ in March 1938. It was a lively polemical piece. . To A.J.’s surprise and dismay he was sued for libel by a notorious
rogue who peddled a quack cure for for tuberculosis. This man said that A.J.’s remarks (such as “‘Cures’ for consumption, cancer and diabetes may fairly be classed as murderous”) were libellous and would damage his business. A.J. was determined to fight, and he and Trixie decided to put their savings at stake if necessary. The B.M.A. and the Medical Defence Union agreed to support him and they all went to lawyers. He was shocked when they advised him that he would be bound to lose for he had damaged the man’s livelihood! Finally, after much heart searching, he made an apology, saying that he had not meant that particular man’s nostrum”
Talk about déjà vu!
On page 68 there is another very familiar story. It could have been written today.
“The fact that the public is acquiring more knowledge of health matters and is becoming more suspicious of the cruder forms of lies is also helping to weed out the worst types of patent medicine advertisements. For example, in 1751 a bottle of oil was advertised as a cure for scurvy, leprosy and consumption but today such claims would not be effective in promoting the sale of a remedy. The modern advertiser would probably claim that the oil was rich in all the vitamins and the elements essential for life and would confine his claims to a statement that it would alleviate all minor forms of physical or mental ill-health.
The average patent medicine advertised today makes plausible rather than absurd claims and in general the advertisements have changed to conform with a change in the level of the public’s knowledge.
It is somewhat misleading, however, to speak of this as an improvement, since the law has not altered and hence the change only means that the public is being swindled in a somewhat more skilful manner.
The ideal method of obtaining an adequate vitamin supply is to select a diet containing an abundant supply of fresh foods, but unfortunately the populace is accustomed to live very largely on preserved or partially purified food stuffs and such processes usually remove most of the vitamins.”
The first part of the passage above is reminiscent of something that A.J Clark wrote in the BMJ in 1927. Nowadays it is almost unquotable and I was told by a journal editor that it was unacceptable even with asterisks. That seems to me a bit silly. Words had different connotations in 1927.
“The less intelligent revert to the oldest form of belief and seek someone who will make strong magic for them and defeat the evil spirits by some potent charm. This is the feeling to which the quack appeals; he claims to be above the laws of science and to possess some charm for defeating disease of any variety.
The nature of the charm changes with the growth of education. A naked n****r howling to the beat of a tom-tom does not impress a European, and most modern Europeans would be either amused or disgusted by the Black mass that was popular in the seventeenth century. Today some travesty of physical science appears to be the most popular form of incantation.”
A.J. Clark (1927) The historical aspect of quackery, BMJ October 1st 1927
Apart from some of the vocabulary, what better description could one have of the tendency of homeopaths to harp on meaninglessly about quantum theory or the “scienciness” and “referenciness” of
modern books on nutritional therapy?
So has anything changed?
Thus far, the outcome might be thought gloomy. Judging by Clark’s account, remarkably little has changed since 1938, or even since 1914. The libel law in the UK is as bad now as it was then. Recently the United Nations Human Rights Committee said UK laws block matters of public interest and encourage libel tourism (report here, see also here). It is unfit for a free society and it should be changed.
But there are positive sides too. Firstly the advent of scientific bloggers has begun to have some real influence. People are no longer reliant on journalists to interpret (or, often, misinterpret) results for them. They can now get real experts and links to original sources. Just one of these, Ben Goldacre’s badscience.net, and his weekly column in the Guardian has worked wonders in educating the public and improving journalism. Young people can, and do, contribute to the debate because they can blog anonymously if they are frightened that their employer might object.
Perhaps still more important, the law changed this year. Now, at last, it may be possible to prosecute successfully those who make fraudulent health claims. Sad to say, this was not an initiative of the UK government, which remains as devoted as ever to supporting quacks. Remember that, quite shamefully, the only reason given by the Medicines and Health Regulatory Authority (MHRA) gave for allowing false labelling of homeopathic pills was to support the “homeopathic industry”. They suggested (falsely) that the EU required them to take this irresponsible step, which was condemned by just about every scientific organisation. But the new unfair trading regulations did come from the EU. After almost 100 years since the 1914 report, we have at last some decent legislation. Let’s hope it’s enforced.
Postcript
The back cover of the series of ‘Fact‘ books in which A.J. Clark’s article appeared is reproduced below, simply because of the historical portrait of the 1930s that it gives.
Follow-up
This post got a lot of hits from Ben Goldacre’s miniblog which read
- Prof David Colquhoun gets into a time machine and meets himself
A truly classic DC post.
Thanks, Ben.
During the last year, there has been a very welcome flurry of good and informative books about alternative medicine. They are all written in a style that requires little scientific background, even the one that is intended for medical students.
CAM, Cumming | Trick or Treatment | Snake Oil Science |
Testing treatments | Suckers | Healing, Hype or Harm
I’ll start with the bad one, which has not been mentioned on this blog before.
Complementary and Alternative medicine. An illustrated text.
by Allan D. Cumming, Karen R. Simpson and David Brown (and 12 others). 94 pages, Churchill Livingstone; 1 edition (8 Dec 2006).
The authors of this book sound impressive
Allan Cumming, BSc(Hons), MBChB, MD, FRCP(E), Professor of Medical Education and Director of Undergraduate Learning and Teaching, and Honorary Consultant Physician, College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, UK; Karen Simpson, BA(Hons), RN, RNT, Fellow in Medical Education, College of Medicine and Veterinary Medicine David Brown, MBChB, DRCOG, General Practitioner, The Murrayfield Medical Centre, and Honorary Clinical Tutor, University of Edinburgh |
Sadly, this is a book so utterly stifled by political correctness that it ends up saying nothing useful at all. The slim volume is, I have to say, quite remarkably devoid of useful information. Partly that is a result of out-of-date and selective references (specially in the chapters written by alternative practitioners),
But the lack of information goes beyond the usual distortions and wishful thinking. I get the strong impression is that it results not so much for a strong commitment to alternative medicine (at least by Cumming) as from the fact that the first two authors are involved with medical education. It seems that they belong to that singularly barmy fringe of educationalists who hold that the teacher must not give information to s student for fear of imparting bias. Rather the student must be told how to find out the information themselves. There is just one little problem with this view. It would take about 200 years to graduate in medicine.
There is something that worries me about medical education specialists. Just look at the welcome given by Yale’s Dean of Medical Education, Richard Belitsky, to Yale’s own division of “fluid concepts of evidence”, as described at Integrative baloney @ Yale, and as featured on YouTube. There are a lot of cryptic allusions to alternative forms of evidence in Cumming’s book too, but nothing in enough detail to be useful to the reader.
What should a book about Alternative medicine tell you? My list would look something like this.
- Why people are so keen to deceive themselves about the efficacy of a treatment
- Why it is that are so often deceived into thinking that something works when it doesn’t
- How to tell whether a medicine works better than placebo or not,
- Summaries of the evidence concerning the efficacy and safety of the main types of alternative treatments.
The Cumming book contains chapters with titles like these. It asks most of the right questions, but fails to answer any of them. There is, time and time again, the usual pious talk about the importance of evidence, but then very little attempt to tell you what the evidence says. When an attempt is made to mention evidence, it is usually partial and out of date. Nowhere are you told clearly about the hazards that will be encountered when trying to find out whether a treatment works.
The usual silly reflexology diagram is reproduced in Cumming’s introductory chapter, but with no comment at all, The fact that it is obviously total baloney is carefully hidden from the reader.. What is the poor medical student meant to think when they perceive that it is totally incompatible with all the physiology they have learned? No guidance is offered. |
You will look in vain for a decent account of how to do a good randomised controlled trial, though you do get a rather puerile cartoon, The chapter about evidence is written by a librarian. Since the question of evidence is crucial, this is a fatal omission. |
Despite the lack of presentation of evidence that any of it works, there seems to be an assumption throughout the book that is is desirable to integrate alternative medicine into clinical practice. In Cumming’s chapter (page 6) we see
Since it would not be in the interests of patients to integrate treatments that don’t work with treatments that do work, I see only two ways to explain this attitude. Either the authors have assumed than most alternative methods work (in which case they haven’t read the evidence), or they think integration is a good idea even if the treatment doesn’t work. Neither case strikes me as good medical education.
The early chapters are merely vague and uninformative. Some of the later chapters are simply a disgrace.
Most obviously the chapter on homeopathy is highly selective and inaccurate, That is hardly surprising because it is was written by Thomas Whitmarsh, a consultant physician at Glasgow Homeopathic Hospital (one that has still survived). It has all the usual religious zeal of the homeopath. I honestly don’t know whether people like Whitmarsh are incapable of understanding what constitutes evidence, or are simply too blinded by faith to even try. Since the only other possibility is that they are dishonest, I suppose it must be one of the former.
The chapter on “Nutritional therapy” is also written by a convert and is equally misleading piece of special pleading.
The same is true of the chapter on Prayer and Faith Healing. This chapter reproduces the header of the Cochrane Review on “Intercessory prayer for the alleviation of Ill Health”, but then proceeds to ignore entirely its conclusion “Most of the studies show no real differences”).
If you want to know about alternative medicine, don’t buy this book. Although this book was written for medical students, you will learn a great deal more from any of the following books, all of which were written for the general public.
Trick or Treatment
by Simon Singh and Edzard Ernst, Bantam Press, 2008
Simon Singh is the author of many well-known science books, like Fermat’s Last Theorem. Edzard Ernst is the UK’s first professor of complementary and alternative medicine.
Ernst, unlike Cumming et. al is a real expert in alternative medicine. He practised it at an early stage in his career and has now devoted all his efforts to careful, fair and honest assessment of the evidence. That is what this book is about. It is a very good account of the subject and it should be read by everyone, and certainly by every medical student. |
Singh and Ernst follow the sensible pattern laid out above, The first chapter goes in detail into how you distinguish truth from fiction (a little detail often forgotten in this area).
The authors argue, very convincingly, that the development of medicine during the 19th and 20th century depended very clearly on the acceptance of evidence not anecdote. There is a fascinating history of clinical trials, from James Lind (lemons and scurvy), John Snow and the Broad Street pump, Florence Nightingale’s contribution not just to hygiene, but also to the statistical analysis that was needed to demonstrate the strength of her conclusions (she became the first female member of the Royal Statistical Society, and had studied under Cayley and Sylvester, pioneers of matrix algebra).
There are detailed assessments of the evidence for acupuncture, homeopathy, chiropractic and herbalism, and shorter synopses for dozens of others. The assessments are fair, even generous in marginal cases.
Acupunture. Like the other good books (but not Cumming’s), it is pointed out that acupuncture in the West is not so much the product of ancient wisdom (which is usually wrong anyway), but rather a product of Chinese nationalist propaganda engineered by Mao Tse-tung after 1949. It spread to the West after Nixon’s visit Their fabricated demonstrations of open heart surgery under acupuncture have been known since the 70s but quite recently they managed again to deceive the BBC It was Singh who revealed the deception. The conclusion is ” . . . this chapter demonstrates that acupuncture is very likely to be acting as nothing more than a placebo . . . ”
Homeopathy. “hundreds of trials have failed to deliver significant or convincing evidence to support the use of homeopathy for the treatment of any particular ailment. On the contrary, it would be to say that there is a mountain of evidence to suggest that homeopathic remedies simply do not work”.
Chiropractic. Like the other good books (but not Cumming’s) there is a good account of the origins of chiropractic (see, especially, Suckers). D.D. Palmer, grocer, spiritual healer, magnetic therapist and fairground quack, finally found a way to get rich by removing entirely imaginary ‘subluxations’. They point out the dangers of chiropractic (the subject of court action), and they point out that physiotherapy is just as effective and safer.
Herbalism. There is a useful table that summarises the evidence. They conclude that a few work and most don’t Unlike homeopathy, there is nothing absurd about herbalism, but the evidence that most of them do any good is very thin indeed.
“We argue that it is now the time for the tricks to stop, and for the real treatments to take priority. In the name of honesty, progress and good healthcare, we call for scientific standards, evaluation and regulation to be applied to all types of medicine, so that patients can be confident that they are receiving treatments that demonstrably generate more harm than good.” |
Snake Oil Science, The Truth about Complementary and Alternative Medicine.
R. Barker Bausell, Oxford University Press, 2007
Another wonderful book from someone who has been involved himself in acupuncture research, Bausell is a statistician and experimental designer who was Research Director of a Complementary and Alternative Medicine Specialised Research Center at the University of Maryland.
This book gives a superb account of how you find out the truth about medicines, and of how easy it is to be deceived about their efficacy.
I can’t do better than quote the review by Robert Park of the American Physical Society (his own book, Voodoo Science, is also excellent)
“Hang up your lantern, Diogenes, an honest man has been found. Barker Bausell, a biostatistician, has stepped out of the shadows to give us an insider’s look at how clinical evidence is manipulated to package and market the placebo effect. Labeled as ‘Complementary and Alternative Medicine’, the placebo effect is being sold, not just to a gullible public, but to an increasing number of health professionals as well. Bausell knows every trick and explains each one in clear language” |
Bausell’s conclusion is stronger than that of Singh and Ernst.
“There is no compelling, credible scientific evidence to suggest that any CAM therapy benefits any medical condition or reduces any medical symptom (pain or otherwise) better than a placebo”.
Here are two quotations from Bausell that I love.
[Page 22] ” seriously doubt, however, that there is a traditional Chinese medicine practitioner anywhere who ever stopped performing acupuncture on an afflicted body in the presence of similarly definitive negative evidence. CAM therapists simply do not value (and most cases, in my experience, do not understand) the scientific process”
And even better,
[Page39] “But why should nonscientists care one iota about something as esoteric as causal inference? I believe that the answer to this question is because the making of causal inferences is part of our job description as Homo Sapiens.”
Testing Treatments: Better Research for Better Healthcare
by Imogen Evans, Hazel Thornton, Iain Chalmers, British Library, 15 May 2006
You don’t even need to pay for this excellent book (but buy it anyway, eg from Amazon). If you can’t afford, £15 then download it from the James Lind Library.
This book is a unlike all the others, because it is barely mentions alternative medicine. What it does, and does very well, is to describe he harm that can be done to patients when they are treated on the basis of guesswork or ideology, rather than on the basis of proper tests. This, of course, is true whether or not the treatment is labelled ‘alternative’.
It is worth noting that one of the authors of this book is someone who has devoted much of his life to the honest assessement of evidence, Sir Iain Chalmers, one of the founders of the Cochrane Collaboration , and Editor of the James Lind Library . |
A central theme is that randomised double blind trial are essentially the only way to be sure you have the right answer. One of the examples that the authors use to illustrate this is Hormone Replacement Therapy (HRT). For over 20 years, women were told that HRT would reduce their risk of heart attacks and strokes. But when, eventually, proper randomised trials were done, it was found that precisely the opposite was true. The lives of many women were cut short because the RCT had not been done,
The reason why the observational studies gave the wrong answer is pretty obvious. HRT was used predominantly by the wealthier and better-educated women. Income is just about the best predictor of longevity. The samples were biassed, and when a proper RCT was done it was revealed that the people who used HRT voluntarily lived longer despite the HRT, not because of it. It is worth remembering that there are very few RCTs that test the effects of diet. And diet differs a lot between rich and poor people. That, no doubt, is why there are so many conflicting recommendations about diet. And that is why “nutritional therapy” is little more than quackery. Sadly, the media just love crap epidemiology. One of the best discussions of this topics was in Radio 4 Programme. “The Rise of the Lifestyle Nutritionists“, by Ben Goldacre.
One of the big problems in all assessment is the influence of money, in other words corruption, The alternative industry is entirely corrupt of course, but the pharmaceutical industry has been increasingly bad. Testing Treatments reproduces this trenchant comment.
Suckers. How Alternative Medicine Makes Fools of Us All
Rose Shapiro, Random House, London 2008
I love this book. It is well-researched, feisty and a thoroughly good read.
It was put well in the review by George Monbiot.
The chapters on osteopathy and chiropractic are particularly fascinating. |
This passage describes the founder of the chiropractic religion.
“By the 1890s Palmer had established a magnetic healing practice in Davenport, Iowa, and was styling himself ‘doctor’. Not everyone was convinced as a piece about him in an 1894 edition of the local paper, the Davenport Leader, shows.”
A crank on magnetism has a crazy notion hat he can cure the sick and crippled with his magnetic hands. His victims are the eak-minded, ignorant and superstitious, those foolish people who have been sick for years and have become tired of the regular physician and want health by the short-cut method . . . he has certainly profited by the ignorance of his victims . . . His increase in business shows what can be done in Davenport, even by a quack”
Over 100 years later, it seems that the “weak-minded, ignorant and superstitious” include the UK’s Department of Health, who have given these quacks a similar status to the General Medical Council.
The intellectual standards of a 19th Century mid-western provincial newspaper leader writer are rather better than the intellectual standards of the Department of Health, and of several university vice-chancellors in 2007.
Healing Hype or Harm
Edited by Imprint Academic (1 Jun 2008)
Download the contents page
My own chapter in this compilation of essays, “Alternative medicine in UK Universities” is an extended version of what was published in Nature last year (I don’t use the term CAM because I don’t believe anything can be labelled ‘complementary’ until it has been shown to work). Download a copy if the corrected proof of this chapter (pdf).
Perhaps the best two chapters, though, are “CAM and Politics” by Rose and Ernst, and “CAM in Court” by John Garrow.
CAM and politics gives us some horrifiying examples of the total ignorance of almost all politicians and civil servants about the scientific method (and their refusal to listen to anyone who does understand it). CAM in Court has some fascinating examples of prosecutions for defrauding the public. Recent changes in the law mean we may be seeing a lot more of these soon. Rational argument doesn’t work well very well with irrational people. But a few homeopaths in jail for killing people with malaria would probably be rather effective. |
Follow-up
Healing, Hype or Harm has had some nice reviews, That isn’t so surprising from the excellent Harriet Hall at Science-Based Medicine. The introduction to my chapter was a fable about the replacemment of the Department of Physics and Astronomy by the new Department of Alternative Physics and Astrology. It was an unashamedly based on Laurie Taylor’e University of Poppleton column. Hall refers to it as “Crislip-style”, a new term to me. I guess the incomparable Laurie Taylor is not well-known in the USA, Luckily Hall gives a link to Mark Crislip’s lovely article, Alternative Flight,
“Americans want choice. Americans are increasingly using alternative aviation. A recent government study suggests that 75% of Americans have attempted some form of alternative flight, which includes everything from ultralights to falling, tripping and use of bungee cords.”
“Current airplane design is based upon a white male Western European model of what powered flight should look like. Long metal tubes with wings are a phallic design that insults the sensibilities of women, who have an alternative, more natural, emotional, way of understanding airplane design. In the one size fits all design of allopathic airlines, alternative designs are ignored and airplane design utilizing the ideas and esthetics of indigenous peoples and ancient flying traditions are derided as primitive and unscientific, despite centuries of successful use.”
Metapsychology Online Reviews doesn’t sound like a promising title for a good review of Healing, Hype or Harm, but in fact their review by Kevin Purday is very sympathetic. I like the ending.
“One may not agree with everything that is written in this book but it is wonderful that academic honesty is still alive and well.”
In a wonderful demonstration of common sense, the BBC has removed all the alternative medicine pages from BBC Health web site. I expect that it was helped in making that decision by the many complaints it had received about statements on these pages that were simply not true, The existence of these pages was just not compatible with the BBC’s commitment to accuracy.
Needless to say, this decision was greeted with howls of rage from the alternative world. Some wrote to the BBC to complain (and I wrote to congratulate them). Until today I haven’t been able to find any BBC statement on the matter. This one appeared on Healthypages, one of the zaniest sites on the net. Nothing is too barmy for them. This is the picture they used to show how wrong the BBC is | Picture from Healthypages |
“I do a range of spiritual healing practices and can offer an energetic healing session including techniques from the Order of Melchizedek and about 10 forms of Reiki in exchange for a Theta Healing session.”
And just in case you don’t know about the Melchidezek method, here is a picture of a poster in the window of my local “health food” shop.
The explanation says
“Using ancient holographic technology, the basis of the techniques presented is the activation of the Merkaba, a rotating lightfield awakening your spherical consciousness. This raises one’s quotient of light vibration within the human atomic cell structure. Once activated. the merkaba assists us in accessing our naturally ascended consciousness state: the healing capabilities are enhanced a hundred-fold. The Hologram of Love has the ability to heal and rejuvenate any form of creation as it is the living conscious holographic pattern of God Source vibration.”
There, and I’ll bet you thought holography was a recent invention. As an example of sciencey-sounding words used in random order, this one takes a bit of beating. It really is an insult to human intellect.
Anyway, back to the BBC. This is their diplomatic response (what they should have said really was, ‘those pages were nonsense so we removed them’).
BBC issued the following statement, dated 22 Feb 2008:
Complaint We received complaints about the Complementary section of the BBC Health website being disbanded The BBC’s response The decision to remove the complementary medicine area of the health website was taken as part of a wider review of all the health content in order to enable the BBC to focus its efforts on creating new and exciting content. In order to release resources for this redevelopment work, we’re reviewing existing content from an editorial and value-for-money perspective.The complementary health section was incomplete and, therefore, not of a satisfactory editorial standard. It also represented a small proportion of traffic to the site but was disproportionately time-consuming.Therefore, the decision to take it down was based on a combination of factors: how much work it needed to maintain to a high editorial standard, how much this cost and how popular it was with site users.We have already removed other sections of the health site and plan to reduce or remove others. We appreciate people are disappointed this area of the site has been removed and apologise if the decision has appeared abrupt to site users or inconvenienced other sites linking to BBC Health. The BBC will continue to cover complementary health in other areas of its output, such as TV, radio and news programmes, and may reassess its complementary health content in future. |
” . . . However the site has in recent months been targetted by the self-appointed ‘quackbusters’ ( . . . such as David Colquhoun), who sent a deluge of letters and emails claiming that complementary therapies such as homeopathy and cranial osteopathy were ‘unscientific’ and should be removed.”
Well thanks for the credit, but sorry, there was no deluge. I wrote no more than a couple of times myself, and I suspect that a handful of friends did the same. I didn’t even ask them to remove the whole lot, merely to correct particular statements that were not true,
‘Mardi’ goes on
“Rather than taking a reasoned view and considering the evidence from good research studies on complementary medicine, these individuals seem simply hell bent on trying to stamp out complementary medicine”
That really is a bit rich. Suddenly the alternative industry are invoking evidence from good research studies. It is precisely the other way round. It is because that evidence is almost all negative that the BBC have decided to remove their coverage.
Of course it may have helped that the BBC had to spend a lot of time defending itself against criticism of the BBC 2 TV series on Alternative Medicine from February 2006. After initially rejecting complaints, an appeal to the highest level, the Board of Trustees, two of the most serious complaints were upheld against these programmes.
Boots the Chemists have proved themselves dishonest before, over their promotion of homeopathy and of B Vitamins “for vitality”
In a press release dated 12 March 2008, they have hit a new low in ethical standards
Boots help boost the nation’s energy levels in just one week “Health and beauty expert Boots has launched an exclusive energising vitamin supplement that helps boost depleted energy levels and maintain vitality. It is the first time that this exclusive form of CoQ10 has been made available on the high street.” ” . . .supplementation can help to supply higher levels of CoQ10 than are available in the diet. Boots Energy Super Strength CoQ10 containing natural Kaneka CoQ10 is a way of boosting energy levels that can help people who lack energy to see results in a week” |
You decide.
Last year there was an equally misleading press release about CoQ10 from Solgar/Boots Herbal. That one was headed “Need More Energy – Solgar’s Nutri Nano™ Uses Nanotechnology to Deliver Unprecedented Bioavailability of CoQ10”. Not only is the word ‘energy’ misused but notice that the trendy term ‘nanotechnology’ is worked in for extra sciencey effect. It turns out that all this means is that the preparation contains micelles. So nothing new there either. Micelles have been known for almost 100 years.
In contrast, the Boots online store is noticeably more restrained. Could that be because the Advertising Standards People can’t touch press releases, just as they can’t control what Boots Expert Team tell you face to face in the shop?
Boots PR contact is given as: Carrie Eames, PR Manager, Boots The Chemists, D90W WG14, Thane Road, Nottingham NG90 1BS. I’m not sure how Ms Eames sleeps at night. Perhaps you should write to her and let her know what you think.
You might point out to her Boots (anti) Social Corporate (ir)Responsibility Page. It says
“So it’s part of our heritage to treat our customers fairly and act with integrity in everything we do, rather than seizing on the quickest and easiest way to turn a profit.” |
CoQ10 and “energy”
Coenzyme Q10 (also known as ubiquinone) is a relatively small molecule. It cooperates with cytochrome enzymes (big proteins) to synthesize a molecule called ATP. This is a chemical form of energy that can be used to do work, such as making a muscle fibre contract.
The word “energy ” here is used in the sense that a physicist would use it. It is measured in joules or in calories. The meaning of the word ‘energy‘ is described nicely in the Wikipedia entry. For example, when an electric current passes through a resistor (like a kettle) the electrical energy is converted to heat energy, and the energy used is potential difference (volts) X current (amps) X time. In other words energy is power (in watts) times time. So another unit for energy is kilowatt-hours (one kilowatt-hour is about 3.6 megajoules).
Energy in this sense has nothing whatsoever to do with the everyday use of ‘energy’ to indicate your vitality, or how lively you feel.
Furthermore there is not the slightest empirical reason to think that CoQ10 makes you feel more lively. None. The press release cites a sciencey-sounding reference (Ernster L, Dallner G. Biochemical, physiological and medical aspects of ubiquinone function. Biochim Biophys Acta. 1995 May 24;1271(1):195-204.). But this paper is just a review of the biochemistry, nothing whatsoever to do with feeling good.
CoQ10 and the supplement business
There is nothing new in this big push by Boots. CoQ10 has been a staple of supplement business for a long time now. All sorts of medical claims have been made for it. Everything from migraine, to Parkinson’s disease to cancer has been raised as possible benefits of the magic drug, oops, I mean ‘supplement’. This is quite improper of course, since it is being sold as a food not as a medicine, but it is standard practice among supplement hucksters, and so far they have been allowed to get away with it.
What’s interesting though is that until Boots PR machine swung into action, one thing that hadn’t been claimed much is that it made you feel more lively. That’s one they just invented.
CoQ10 and the press
It’s standard technique to get free advertising by hoping that journalists will dash off an article on the basis of a press release, with the hope that they will be in too much hurry to check the spin. Too often it works.
The Daily Mail has big coverage of the press release, under the title “Can a 60p pill from the chemist really add years to your life?“. This was written by Anna Hodgekiss and it’s not bad. It starts with a nice note of scepticism
“Forget vitamins C, E or even B12. The real wonder supplement is Coenzyme Q10 or CoQ10. That’s what Boots would have you believe, anyway. ”
“So should we all be taking this supplement?Not according to David Colquhoun, professor of pharmacology at University College London, who says Boots’ claims are “deliberately misleading customers”.
“Yes, CoQ10 helps the body convert glucose into energy, but it’s not the psychological get-up-and-go energy you feel day to day.
“The type of energy it does produce powers our muscles and cells – physical energy. They have confused the two here to promote a product that I’m not convinced would make any difference to how you actually feel at all.”
The article goes on
Among the other sceptics is Scott Marsden, a senior dietician at The London Clinic.
“There haven’t been enough trials to warrant us all taking CoQ10,” he says.
“It sounds boring, but if you are healthy and eating a balanced diet, you will get all the nutrients you need and shouldn’t have to take supplements.
“Not only could you be spending money unnecessarily, you could also be putting your health at risk. Buy some wholesome food instead.” “
Dr Clare Gerada, vice chairman of the Royal College of General Practitioners, is more forthright.
“While there is some evidence to suggest CoQ10 supplements may help patients with heart failure or severe respiratory disorders, more work is needed,” she says.
“This is just another example of normal health being medicalised, and it’s an issue that worries me.
“The human body is an amazing machine, and we have never been in better health. The fact that more people are living well into their 80s and 90s is proof.
“People need to stop looking for a wonder pill in their quest to live for ever.”
But guess who comes out fighting for Boots? None other than my old friend Dr Ann Walker. Little wonder then that my Nutriprofile result recommended a co Q10 supplement, because she is involved in that too.
Ann Walker’s colleague on the Nutriprofile project, Dr Sarah Brewer comments on CoQ10 on the Healthspan site, thus.
“As CoQ10 is vital for energy production in muscle cells, lack of CoQ10 is linked with lack of energy, physical fatigues, muscle aches and pains . . .”
It seems that she also can’t distinguish between energy in joules and energy as vitality,
Female First and Marie Claire also carry a story “Boots Sell ‘Life Extending’ Pill”
“A new pill that claims to add years to our lives is due to hit shelves in Boots stores this week but scientists say the drug is misleading.”
“Despite these claims Professor David Colquhoun told Marie Clare that he believes the drug is ‘deliberately misleading customers’: “Yes, CoQ10 helps the body convert glucose into energy, but it’s not the psychological get-up-and-go energy you feel day to day,” he said.”
(Funny, I never consciously spoke to Marie Claire but the quotation is OK.)
The Times, in contrast, carries an appalling column by their Dr Thomas Stuttaford, “A natural solution to tiredness“. There isn’t even a question mark in the title, and the content is totally uncritical. Private Eye has nicknamed the author ‘Dr Thomas Utterfraud’. How very cruel.
See also, excellent articles on CoQ10 by Ben Goldacre in the Guardian, and at badscience.net, and at Holfordwatch and Dr Aust’s Spleen
Aha Boots have repeated their mendacious claims in newspaper advertisements
This appeared in the Guardian on 18 March, and I’m told it was in the Mail too.
The small print says
“The new Boots Energy supplement contains Kaneka Q10 to help boost your energy levels throughout the day”
Here is what I just sent to the Advertising Standard Authority, or email new.complaints@asa.org.uk . Why not have a go yourself?
“The words “boost your energy levels” and “still lacking energy” constitute a (presumably deliberate) confusion beteen ‘energy’ measured in joules and the everyday use of the word ‘energy’ to mean vitality. The former usage would be justified in viewof the role of Coenzyme Q10 in ATP production. There is neither theoretical justification nor any empirical evidence that CoQ10 helps your vitality or ‘energy’ in the latter sense.” |
A full size graphic to attach to your complaint can be downloaded here.
Thursday 24 Jan.
One of the original reasons for going to North America was an invitation from the Toronto Secular Alliance and Center for Inquiry. The talk for them was given a lot of publicity, for example here and here and from the totally admirable Orac.
Toronto seems to be no worse than anywhere else when it comes to delusional thinking about medicine. It is, of course, the home of Ryerson University, the place that produced one of the most outrageous pieces of postmodernist nonsense on record. But when this sort of thing gets into really good universities, it is more worrying.
As a result of the publicity there was some media coverage (and a record 7109 hits on this site on Sunday).
Friday 25th January, Reception and talk: Center for Inquiry. Science in an Age of Endarkenment: Some Examples from Scientific Fraud, Quackery, Religion and University Politics |
The interview was broadcast on Sunday morning (28 Jan) and elicited a lot of correspondence. CBC made it available as a podcast which can be downloaded from CBC here. The endarkenment interview was the last 22 minutes (out of 64 minutes) [play the interview here (mp3, 20 Mb)]. |
Sunday Edition: the follow-upThe week following this CBC show, the backlash started. The Sunday Edition wrote
“A stirred-up hornet’s nest is a mild disturbance compared to the firestorm we unleashed last week over my conversation with Dr. David Colquhoun. Dr. Colquhuon [sic] is a gangly, pipe-puffing British pharmacologist who thinks all alternative medicine, all of it, is a fraud perpetrated by quacks. But he went further, somehow suggesting that those who believe in it probably supported Margaret Thatcher, Ronald Reagan and the Ayatollah Khomeini. He pooh-poohed acupuncture, chiropractic, homeopathy, even vitamins.
Well, his remarks opened the floodgates of listener mail, screaming for Dr. Colquhoun’s head on a pike. In a few moments, alternative or complimentary [sic] medicine strikes back. With the help of two experts, we will try to give the other side of contentious Colquhounism.”
The programme for 3 Feb 2008 started with a few emails from listeners, mainly of the “homeopathy cured my granny” type. Nothing of much significance there. But then Enright interviewed Dugald Seely of the Canadian College of Naturopathic Medicine and Dr. Kien Trinh of the DeGroote School of medicine at McMaster University in Hamilton. You can download the podcast here.
The flat earth problem.
Michael Enright was a good interviewer, but Sunday Edition suffers, like the BBC, from a problem. It is admirable that CBC, like the BBC, should strive to be ‘fair and balanced’, but it is not always easy to see what that means in practice. Is it fair and balanced to give equal time to people who think that the earth is flat and those who think it is spherical (OK, an oblate ellipsoid)? Perhaps, but it also
quite misleading because it can easily convey a very distorted idea of the balance of informed opinion. In this case the flat-earthers are the homeopaths and other alternative medicine advocates. That would not matter so much if the interviewers had enough knowledge of the subject to pin down the falt-earth advocates with the sort of penetrating questions that people like John Humphrys (of the BBC’s Today programme) are so very good at. When it comes to science, though, the flat-earthers tend to get away with murder, and the public can easily be left with a very distorted view. Which “expert” should they believe? If I had been given the option, I would have loved to debate the problems of alternative medicine directly with Trinh and Seely I could have asked then a few questions that Enright missed.
Let’s take a look at what happened at the follow-up.
Quackery at McMaster University
McMaster is one of many universities in North America that has chosen to betray the intellectual tradition of the enlightenment by buying into superstition (see the roll of shame here). The ‘contemporary medical acupuncture program appears to run under the aegis of the anaesthesia
department, though the fact that is doen’t appear on the department’s front page suggests there may be some embarrassment about it. The medical acupuncture program itself, has separate web pages which don’t seem to be on the McMaster server at all (they are on a private server, ThePlanet.com Internet Services, Inc.
As so often, these pages pay lip service to an ‘evidence based’ or ‘scientific’ approach, while doing nothing of the sort. In his CBC interview Kien Trinh agreed (twice) with my contention that trials had shown that it doesn’t matter where you put the needles. But then he failed totally to draw the obvious conclusion that ‘meridians’ are mumbo jumbo. He went right on taking the conventional mystical view of meridians and “energy” flow. Like most proponents of alternative medicine, Trinh seems to live in some sort of parallel universe in which the normal rules of logic don’t apply.
On wouldn’t expect regular anaesthetists to accept this sort of mystical nonsense, but it seems one would be wrong. When I wrote to the Chair of the Department of Anesthesia, at McMaster to ask about their relationship with acupuncturists there was no hint of embarrassement. Dr Norman Buckley, BA (Psych), MD, FRCPC, wrote
“It operates under the principles of evidence based medicine, and relates the concepts raised by the Acupuncture/traditional Chinese medicine to physiology anatomy et as it is more usually taught in Western schools.”
That would be all very well if it were true, but it simply isn’t true. The evidence just isn’t there, and the departments involved make no serious attempts to get evidence. In a later letter, Dr Buckley seems to acknowledge that it may be all placebo, but seems reluctant to offend anyone by saying so. That, I suspect, is how quackery has gained such a foothold.
It is good to keep an open mind, but if it is too open your brains fall out. Or, in another variant, if it is too open, someone will fill it with trash.
One looks in vain on Trinh’s web site for any good evidence. They quote approvingly the conclusion of a 1997 NIH Consensus statement that says “There is sufficient evidence of acupuncture’s value to expand its use into conventional medicine and to encourage further studies of its physiology and clinical value.”, but forget to mention that this document is headed “This statement is more than five years old and is provided solely for historical purposes.”. The department doesn’t seem to do much original research, just to write endless reviews of other peoples’ work. The reviews aren’t too bad, and mostly they come to the right conclusion, that there is not enough evidence to come to firm conclusions. The difference from science is that this doesn’t dent their confidence for a moment. A typical sort of conclusion seems to be
Elbow pain. A review by Green et al. concluded “needle acupuncture [is] of short-term benefit with respect to pain, but this finding [is] based on the results of two small trials, the results of which [are] not able to be combined in metaanalysis.”
The results of thousands of years experience with acupuncture seem to be pretty pathetic so far..
Quackery at the Canadian College of Naturopathic Medicine (CCNM)
Unlike McMaster, CCNM isn’t a proper university, though nonetheless is hands out ‘doctorates’. Dugald Seely’s contribution was interesting insofar as he admitted that there was a lot of fraud and unjustified claims in the alternative medicine industry (never forget there are megabucks involved). What he didn’t explain was how he himself could be distinguished from the frauds. The problem, as always is the second-rate research that goes on in this area.
Take one of Seely’s papers, Adaptogenic Potential of a Polyherbal Natural Health Product: Report
on a Longitudinal Clinical Trial. Is only too typical: a small non-randomised, open-label (not blind) “trial” of a complex herbal mixture on 17 patients. The conclusion was, as it almost always is,
“Further research using a randomized controlled design is necessary to confirm the findings from this pilot study.”
In other words, no conclusion at all. Why is it that the proper trial never seems to appear? Could it be that naturopaths, and the wealthy industry behind them, are afraid to do proper trials? That is certainly the impression they give.
One way in which the alternative medicine industry operates is to invent new words with ill-defined meanings (and Big Pharma does it too). In case you were wondering about the word “adaptogen” it is defined as “Essentially the adaptogen supports the body’s ability to ‘adapt’ ideally to its environment. Essentially the adaptogen supports the body’s ability to ‘adapt’ ideally to its environment. ”
Whatever that means.
The Canadian College of Naturopathic Medicine offers the following “therapies”.
- acupuncture/Asian medicine
- botanical medicine
- physical medicine (massage, hydrotherapy, etc.)
- clinical nutrition
- homeopathic medicine
- lifestyle counseling
Well, nothing wrong with nutrition and lifestyle counseling as long as the claims aren’t exaggerated. But, as always, the claims that are made are vastly exaggerated. For example they claim
Homeopathic remedies are particularly effective for:
- depression
- anxiety
- allergies
- infections
- gynecological concerns
- skin conditions
- digestive problems
- chronic and acute conditions including colds and flu
These claims are simply not true, in my view. If you don’t believe me, check NELCAM (the NHS Complementary and Alternative Medicine Specialist Library). This is written by advocates of alternative medicine, yet it finds no convincing evidence for effectiveness of homeopathy in any of the conditions listed above.
Or, even more remarkably, from a report in Newsweek.
“Dr. Jack Killen, acting deputy director of the National Center for Complementary and Alternative Medicine, says homeopathy “goes beyond current understanding of chemistry and physics.” He adds: “There is, to my knowledge, no condition for which homeopathy has been proven to be an effective treatment.”
The National Center for Complementary and Alternative Medicine (NCCAM) has, incidentally, spent almost one billion US$ billion of US taxpayers’ money and has come up with next-to-nothing useful.
So the claims made by the Canadian College of Naturopathic Medicine are not backed up even by people who are directly involved in alternative medicine You don’t have to be a rocket scientist to understand that the medicine contains no medicine.