Download Lectures on Biostatistics (1971). Corrected and searchable version of Google books edition
Download review of Lectures on Biostatistics (THES, 1973).
BMJ
This piece is almost identical with today’s Spectator Health article.
This week there has been enormously wide coverage in the press for one of the worst papers on acupuncture that I’ve come across. As so often, the paper showed the opposite of what its title and press release, claimed. For another stunning example of this sleight of hand, try Acupuncturists show that acupuncture doesn’t work, but conclude the opposite: journal fails, published in the British Journal of General Practice).
Presumably the wide coverage was a result of the hyped-up press release issued by the journal, BMJ Acupuncture in Medicine. That is not the British Medical Journal of course, but it is, bafflingly, published by the BMJ Press group, and if you subscribe to press releases from the real BMJ. you also get them from Acupuncture in Medicine. The BMJ group should not be mixing up press releases about real medicine with press releases about quackery. There seems to be something about quackery that’s clickbait for the mainstream media.
As so often, the press release was shockingly misleading: It said
Acupuncture may alleviate babies’ excessive crying Needling twice weekly for 2 weeks reduced crying time significantly
This is totally untrue. Here’s why.
|
Luckily the Science Media Centre was on the case quickly: read their assessment. The paper made the most elementary of all statistical mistakes. It failed to make allowance for the jelly bean problem. The paper lists 24 different tests of statistical significance and focusses attention on three that happen to give a P value (just) less than 0.05, and so were declared to be "statistically significant". If you do enough tests, some are bound to come out “statistically significant” by chance. They are false postives, and the conclusions are as meaningless as “green jelly beans cause acne” in the cartoon. This is called P-hacking and it’s a well known cause of problems. It was evidently beyond the wit of the referees to notice this naive mistake. It’s very doubtful whether there is anything happening but random variability. And that’s before you even get to the problem of the weakness of the evidence provided by P values close to 0.05. There’s at least a 30% chance of such values being false positives, even if it were not for the jelly bean problem, and a lot more than 30% if the hypothesis being tested is implausible. I leave it to the reader to assess the plausibility of the hypothesis that a good way to stop a baby crying is to stick needles into the poor baby. If you want to know more about P values try Youtube or here, or here. |
|
One of the people asked for an opinion on the paper was George Lewith, the well-known apologist for all things quackish. He described the work as being a "good sized fastidious well conducted study ….. The outcome is clear". Thus showing an ignorance of statistics that would shame an undergraduate.
On the Today Programme, I was interviewed by the formidable John Humphrys, along with the mandatory member of the flat-earth society whom the BBC seems to feel obliged to invite along for "balance". In this case it was professional acupuncturist, Mike Cummings, who is an associate editor of the journal in which the paper appeared. Perhaps he’d read the Science media centre’s assessment before he came on, because he said, quite rightly, that
"in technical terms the study is negative" "the primary outcome did not turn out to be statistically significant"
to which Humphrys retorted, reasonably enough, “So it doesn’t work”. Cummings’ response to this was a lot of bluster about how unfair it was for NICE to expect a treatment to perform better than placebo. It was fascinating to hear Cummings admit that the press release by his own journal was simply wrong.
Listen to the interview here
Another obvious flaw of the study is that the nature of the control group. It is not stated very clearly but it seems that the baby was left alone with the acupuncturist for 10 minutes. A far better control would have been to have the baby cuddled by its mother, or by a nurse. That’s what was used by Olafsdottir et al (2001) in a study that showed cuddling worked just as well as another form of quackery, chiropractic, to stop babies crying.
Manufactured doubt is a potent weapon of the alternative medicine industry. It’s the same tactic as was used by the tobacco industry. You scrape together a few lousy papers like this one and use them to pretend that there’s a controversy. For years the tobacco industry used this tactic to try to persuade people that cigarettes didn’t give you cancer, and that nicotine wasn’t addictive. The main stream media obligingly invite the representatives of the industry who convey to the reader/listener that there is a controversy, when there isn’t.
Acupuncture is no longer controversial. It just doesn’t work -see Acupuncture is a theatrical placebo: the end of a myth. Try to imagine a pill that had been subjected to well over 3000 trials without anyone producing convincing evidence for a clinically useful effect. It would have been abandoned years ago. But by manufacturing doubt, the acupuncture industry has managed to keep its product in the news. Every paper on the subject ends with the words "more research is needed". No it isn’t.
Acupuncture is pre-scientific idea that was moribund everywhere, even in China, until it was revived by Mao Zedong as part of the appalling Great Proletarian Revolution. Now it is big business in China, and 100 percent of the clinical trials that come from China are positive.
if you believe them, you’ll truly believe anything.
Follow-up
29 January 2017
Soon after the Today programme in which we both appeared, the acupuncturist, Mike Cummings, posted his reaction to the programme. I thought it worth posting the original version in full. Its petulance and abusiveness are quite remarkable.
I thank Cummings for giving publicity to the video of our appearance, and for referring to my Wikipedia page. I leave it to the reader to judge my competence, and his, in the statistics of clinical trials. And it’s odd to be described as a "professional blogger" when the 400+ posts on dcscience.net don’t make a penny -in fact they cost me money. In contrast, he is the salaried medical director of the British Medical Acupuncture Society.
It’s very clear that he has no understanding of the error of the transposed conditional, nor even the mulltiple comparison problem (and neither, it seems, does he know the meaning of the word ‘protagonist’).
I ignored his piece, but several friends complained to the BMJ for allowing such abusive material on their blog site. As a result a few changes were made. The “baying mob” is still there, but the Wikipedia link has gone. I thought that readers might be interested to read the original unexpurgated version. It shows, better than I ever could, the weakness of the arguments of the alternative medicine community. To quote Upton Sinclair:
“It is difficult to get a man to understand something, when his salary depends upon his not understanding it.”
It also shows that the BBC still hasn’t learned the lessons in Steve Jones’ excellent “Review of impartiality and accuracy of the BBC’s coverage of science“. Every time I appear in such a programme, they feel obliged to invite a member of the flat earth society to propagate their make-believe.
Acupuncture for infantile colic – misdirection in the media or over-reaction from a sceptic blogger?26 Jan, 17 | by Dr Mike Cummings So there has been a big response to this paper press released by BMJ on behalf of the journal Acupuncture in Medicine. The response has been influenced by the usual characters – retired professors who are professional bloggers and vocal critics of anything in the realm of complementary medicine. They thrive on oiling up and flexing their EBM muscles for a baying mob of fellow sceptics (see my ‘stereotypical mental image’ here). Their target in this instant is a relatively small trial on acupuncture for infantile colic.[1] Deserving of being press released by virtue of being the largest to date in the field, but by no means because it gave a definitive answer to the question of the efficacy of acupuncture in the condition. We need to wait for an SR where the data from the 4 trials to date can be combined. So what about the research itself? I have already said that the trial was not definitive, but it was not a bad trial. It suffered from under-recruiting, which meant that it was underpowered in terms of the statistical analysis. But it was prospectively registered, had ethical approval and the protocol was published. Primary and secondary outcomes were clearly defined, and the only change from the published protocol was to combine the two acupuncture groups in an attempt to improve the statistical power because of under recruitment. The fact that this decision was made after the trial had begun means that the results would have to be considered speculative. For this reason the editors of Acupuncture in Medicine insisted on alteration of the language in which the conclusions were framed to reflect this level of uncertainty. DC has focussed on multiple statistical testing and p values. These are important considerations, and we could have insisted on more clarity in the paper. P values are a guide and the 0.05 level commonly adopted must be interpreted appropriately in the circumstances. In this paper there are no definitive conclusions, so the p values recorded are there to guide future hypothesis generation and trial design. There were over 50 p values reported in this paper, so by chance alone you must expect some to be below 0.05. If one is to claim statistical significance of an outcome at the 0.05 level, ie a 1:20 likelihood of the event happening by chance alone, you can only perform the test once. If you perform the test twice you must reduce the p value to 0.025 if you want to claim statistical significance of one or other of the tests. So now we must come to the predefined outcomes. They were clearly stated, and the results of these are the only ones relevant to the conclusions of the paper. The primary outcome was the relative reduction in total crying time (TC) at 2 weeks. There were two significance tests at this point for relative TC. For a statistically significant result, the p values would need to be less than or equal to 0.025 – neither was this low, hence my comment on the Radio 4 Today programme that this was technically a negative trial (more correctly ‘not a positive trial’ – it failed to disprove the null hypothesis ie that the samples were drawn from the same population and the acupuncture intervention did not change the population treated). Finally to the secondary outcome – this was the number of infants in each group who continued to fulfil the criteria for colic at the end of each intervention week. There were four tests of significance so we need to divide 0.05 by 4 to maintain the 1:20 chance of a random event ie only draw conclusions regarding statistical significance if any of the tests resulted in a p value at or below 0.0125. Two of the 4 tests were below this figure, so we say that the result is unlikely to have been chance alone in this case. With hindsight it might have been good to include this explanation in the paper itself, but as editors we must constantly balance how much we push authors to adjust their papers, and in this case the editor focussed on reducing the conclusions to being speculative rather than definitive. A significant result in a secondary outcome leads to a speculative conclusion that acupuncture ‘may’ be an effective treatment option… but further research will be needed etc… Now a final word on the 3000 plus acupuncture trials that DC loves to mention. His point is that there is no consistent evidence for acupuncture after over 3000 RCTs, so it clearly doesn’t work. He first quoted this figure in an editorial after discussing the largest, most statistically reliable meta-analysis to date – the Vickers et al IPDM.[2] DC admits that there is a small effect of acupuncture over sham, but follows the standard EBM mantra that it is too small to be clinically meaningful without ever considering the possibility that sham (gentle acupuncture plus context of acupuncture) can have clinically relevant effects when compared with conventional treatments. Perhaps now the best example of this is a network meta-analysis (NMA) using individual patient data (IPD), which clearly demonstrates benefits of sham acupuncture over usual care (a variety of best standard or usual care) in terms of health-related quality of life (HRQoL).[3] |
30 January 2017
I got an email from the BMJ asking me to take part in a BMJ Head-to-Head debate about acupuncture. I did one of these before, in 2007, but it generated more heat than light (the only good thing to come out of it was the joke about leprechauns). So here is my polite refusal.
|
Hello Thanks for the invitation, Perhaps you should read the piece that I wrote after the Today programme Why don’t you do these Head to Heads about genuine controversies? To do them about homeopathy or acupuncture is to fall for the “manufactured doubt” stratagem that was used so effectively by the tobacco industry to promote smoking. It’s the favourite tool of snake oil salesman too, and th BMJ should see that and not fall for their tricks. Such pieces night be good clickbait, but they are bad medicine and bad ethics. All the best David |
|
This is a very important book. Buy it now (that link is to Waterstone’s Amazon don’t pay tax in the UK, so don’t use them). When you’ve read it, do something about it. The book has lots of suggestions about what to do. |
Stolen from badscience.net |
Peter Medawar, the eminent biologist, in his classic book Advice to a Young Scientist, said this.
“Exaggerated claims for the efficacy of a medicament are very seldom the consequence of any intention to deceive; they are usually the outcome of a kindly conspiracy in which everybody has the very best intentions. The patient wants to get well, his physician wants to have made him better, and the pharmaceutical company would have liked to have put it into the physician’s power to have made him so. The controlled clinical trial is an attempt to avoid being taken in by this conspiracy of good will.”
There was a lot of truth in that 1979, towards the end of the heyday of small molecule pharmacology. Since then, one can argue, things have gone downhill.
First, though, think of life without general anaesthetics, local anaesthetics, antibiotics, anticoagulants and many others. They work well and have done incalculable good. And they were developed by the drug industry.
But remember also that remarkably little is known about medicine. There are huge areas in which neither causes nor cures are known. Treatments for chronic pain, back problems, many sorts of cancer and almost all mental problems are a mess. It just isn’t known what to do. Nobody is to blame for this. Serious medical research has been going on for little more than 60 years, and it turns out to be very complicated. We are doing our best, but are still ignorant about whole huge areas. That leads to a temptation to make things up. Clutching at straws is very evident when it comes to depression, pain and Alzheimer’s disease, among others.
In order to improve matters, one essential is to do fair tests on treatments that we have. Ben Goldacre’s book is a superb account of how this could be done, and how the process of testing has been subverted for commercial gain and to satisfy the vanities of academics.
Of course there is nothing new in criticisms of Big Pharma. The huge fines levied on them for false advertising are well known. The difference is that Goldacre’s book explains clearly what’s gone wrong in great detail, documents it thoroughly, and makes concrete suggestions for improving matters.
Big Pharma has undoubtedly sometimes behaved appallingly in recent years. Someone should be in jail for crimes against patients. They have behaved in much the same way that bankers have. In any huge globalised industry it is always possible to blame someone in another department for the dishonesty. But they aren’t the only people to blame. None of the problems could have arisen with the complicity of academics, universities, and a plethora of regulatory agencies and professional bodies.
The biggest scandal of all is missing data (chapter 1). Companies, and sometmes academics, have suppressed of trials that don’t favour the drugs that they are trying to sell. The antidepressant drug, reboxetine, appeared at first to be good. It had been approved by the Medicines and Healthcare products Regulatory Agency (MHRA) and there was at least one good randomized placebo-controlled trial (RCT) showing it worked. But it didn’t. The manufacturer didn’t provide a complete list of unpublished trials when asked for them. After much work it was found in 2010 that, as well as the published, favourable trial, there were six more trials which had not been published and all six showed reboxetine to be no better than placebo . In comparisons with other antidepressant drugs three small studies (507 patients) showed reboxetine to be as good as its competitors. These were published. But it came to light that data on 1657 patients had never been published and these showed reboxetine to be worse than its rivals.
When all the data for the SSRI antidepressants were unearthed (Kirsch et al., 2008) it turned out that they were no better than placebo for mild or moderate depression. This selective suppression of negative data has happened time and time again. It harms patients and deceives doctors, but, incredibly, it’s not illegal.
Disgracefully, Kirsch et al. had to use a Freedom of Information Act request to get the data from the FDA.
“The output of a regulator is often simply a crude, brief summary: almost a ‘yes’ or ‘no’ about side effects. This is the opposite of science, which is only reliable because everyone shows their working, explains how they know that something is effective or safe, shares their methods and their results, and allows others to decide if they agree with the way they processed and analysed the data.”
|
“the NICE document discussing whether it’s a good idea to have Lucentis, an extremely expensive drug, costing well over £ 1,000 per treatment, that is injected into the eye for a condition called acute macular degeneration. As you can see, the NICE document on whether this treatment is a good idea is censored. Not only is the data on the effectiveness of the treatment blanked out by thick black rectangles, in case any doctor or patient should see it, but absurdly, even the names of some trials are missing, preventing the reader from even knowing of their existence, or cross referencing information about them.Most disturbing of all, as you can see in the last bullet point, the data on adverse events is also censored.”
|
 |
The book lists all the tricks that are used by both industry and academics. Here are some of them.
- Regulatory agencies like the MHRA, the European Medicines Agency (EMA) and the US Food and Drugs Administration (FDA) set a low bar for approval of drugs.
- Companies make universities sign gagging agreements which allow unfavourable results to be suppressed, and their existence hidden.
- Accelerated approval schemes are abused to get quick approval of ineffective drugs and the promised proper tests often don’t materialise
- Disgracefully, even when all the results have been given to the regulatory agencies (which isn’t always). The MHRA, EMA and FDA don’t make them public. We are expected to take their word.
- Although all clinical trials are meant to be registered before they start, the EMA register, unbelievably, is not public. Furthermore there is no check that the results if trials ever get published. Despite mandates that results must be published within a year of finishing the trial, many aren’t. Journals promise to check this sort of thing, but they don’t.
- When the results are published, it is not uncommon for the primary outcome, specified before it started, to have been changed to one that looks like a more favourable result. Journals are meant to check, but mostly don’t.
- Companies use scientific conferences, phony journals, make-believe “seed trials” and “continuing medical education” for surreptitious advertising.
- Companies invent new diseases, plant papers to make you think you’re abnormal, and try to sell you a “cure”. For example, female sexual dysfunction , restless legs syndrome and social anxiety disorder (i.e. shyness). This is called disease-mongering, medicalisation or over-diagnosis. It’s bad.
- Spin is rife. Companies, and authors, want to talk up their results. University PR departments want to exaggerate benefits. Journal editors want sensational papers. Read the results, not the summary. This is universal (but particularly bad in alternative medicine).
- Companies fund patient groups to lobby for pills even when the pills are known to be ineffective. The lobby that demanded that Herceptin should be available to all on the breast cancer patients on the NHS was organised by a PR company working for the manufacturer, Roche. But Herceptin doesn’t work at all in 80% of patients and gives you at best a few extra months of life in advanced cases.
- Ghostwriting of papers is serious corruption. A company writes the paper and senior academics appear as the authors, though they may never have seen the original data. Even in cases where academics have admitted to lying about whether they have seen the data, they go unpunished by their universities. See for example, the case of Professor Eastell.
- By encouraging the funding of “continuing medical education” by companies, the great and the good of academic medicine have let us down badly.
This last point is where the book ends, and it’s worth amplification.
“So what have the great and good of British medicine done to help patients, in the face of this endemic corruption, and these systematic flaws? In 2012, a collaborative document was produced by senior figures in medicine from across the board, called ‘Guidance on Collaboration Between Healthcare Professionals and the Pharmaceutical Industry’. This document was jointly approved by the ABPI, the Department of Health, the Royal Colleges of Physicians, Nursing, Psychiatrists, GPs, the Lancet, the British Medical Association, the NHS Confederation, and so on. ”
“It contains no recognition of the serious problems we have seen in this book. In fact, quite the opposite: it makes a series of assertions about them that are factually incorrect.”
“It states that drug reps ‘can be a useful resource for healthcare professionals’. Again, I’m not sure why the Royal Colleges, the BMA, the Department of Health and the NHS Confederation felt the need to reassert this to the doctors of the UK, on behalf of industry, when the evidence shows that drug reps actively distort prescribing practices. But that is the battle you face, trying to get these issues taken seriously by the pinnacle of the medical establishment.”
This is perhaps the most shameful betrayal of all. The organisations that should protect patients have sold them out.
You may have been sold out by your “elders and betters”, but you can do something. The “What to do” sections of the book should be produced as a set of flash cards, as a reminder that matters can be improved.
It is shameful that this book was not written by a clinical pharmacologist, or a senior doctor, or a Royal College, or a senior academic. Why has the British Pharmacological Society said nothing?
It is shameful too that this book was not written by one of the quacks who are keen to defend the $60 billion alternative medicine industry (which has cured virtually nothing) and who are strident in their criticism of the 600 billion dollar Pharma industry. They haven’t done the work that Goldacre has to analyse the real problems. All they have done is to advocate unfair tests, because that is the only sort their treatments can pass.
It’s weird that medicine, the most caring profession, is more corrupt than any other branch of science. The reason, needless to say, is money. Well, money and vanity. The publish or perish mentality of senior academics encourages dishonesty. It is a threat to honest science.
Goldacre’s book shows the consequences: harm to patients and huge wastage of public money.
Read it.
Do something.
Follow-up
7 October, 2012, The Observer
Goldacre wrote
"I think it’s really disappointing that nobody, not the Royal Colleges, the Academy of Medical Sciences, the British Pharmacological Society, the British Medical Association, none of these organisations have stood up and said: selective non-publication of unflattering trial data is research misconduct, and if you do it you will be booted out. And I think they really urgently should."
Exactly.
It’s good to see the BMJ joining the campaign for free speech (only a month or two behind the blogs). The suing of Simon Singh for defamation by the British Chiropractic Association has stirred up a hornet’s nest that could (one hopes) change the law of the land, and destroy chiropractic altogether. The BMJ’s editor, Fiona Godlee, has a fine editorial, Keep the libel laws out of science. She starts “I hope all readers of the BMJ are signed up to organised scepticism” and says
“Weak science sheltered from criticism by officious laws means bad medicine. Singh is determined to fight the lawsuit rather than apologise for an article he believes to be sound. He and his supporters have in their sights not only the defence of this case but the reform of England’s libel laws.”
Godlee refers to equally fine articles by the BMJ’s Deputy Editor, Tony Delamothe, Thinking about Charles II“, and by Editor in Chief, Harvey Marcovitch, “Libel law in the UK“.
The comments on Godlee’s editorial, show strong support, (apart from one from the infamous quantum fantasist, Lionel Milgrom). But there was one that slighlty surprised me, from Tricia Greenhalgh, author of the superb book, “How to read a paper”. She comments
“the use of the term ‘bogus’ seems both unprofessional and unscholarly. The argument would be stronger if expressed in more reserved terms”
That set me thinking, not for the first time, about the difference between journalism and scholarship. I can’t imagine ever using a word like ‘bogus’ in a paper about single ion channels. But Singh was writing in a newspaper, not in a scientific paper. Even more to the point, his comments were aimed at people who are not scholars and who, quite explicitly reject the normal standards of science and evidence. The scholarly approach has been tried for centuries, and it just doesn’t work with such people. I’d defend Singh’s language. It is the only way to have any effect. That is why I sent the following comment.
The ultimate irony is that the comment was held up by the BMJ’s lawyers, and has still to appear.
|
Thanks for an excellent editorial. I doubt that it’s worth replying to Lionel Milgrom whose fantasy physics has been totally demolished by real physicists. Trisha Greenhalgh is, though, someone whose views I’d take very seriously. She raises an interesting question when she says “bogus” is an unprofessional word to use. Two things seem relevant. First, there is little point in writing rational scholarly articles for a group of people who do not accept the ordinary rules of evidence or scholarship. We are dealing with fantasists. Worse still, we are dealing with fantasists whose income depends on defending their fantasies. You can point out to your heart’s content that “subluxations” are figment of the chiropractors’ imagination, but they don’t give a damn. They aren’t interested in what’s true and what isn’t. Throughout my lifetime, pharmacologists and others have been writing scholarly articles about how homeopathy and other sorts of alternative medicine are bogus. All this effort had little effect. What made the difference was blogs and investigative journalism. When it became possible to reveal leaked teaching materials that taught students that “amethysts emit high yin energy“, and name and shame the vice-chancellors who allow that sort of thing to happen (in this case Prof Geoffrey Petts of Westminster University), things started to happen. In the last few years all five “BSc” degrees in homeopathy have closed and that is undoubtedly a consequence of the activities of bloggers and can assess evidence but who work more like investigative journalists. When the BCA released, 15 months after the event, its “plethora of evidence” a semi-organised effort by a group of bloggers produced, in less than 24 hours, thoroughly scholarly analyses of all of them (there is a summary here). As the editorial says, they didn’t amount to a hill of beans, They also pointed out the evidence that was omitted by the BCA. The conventional press just followed the bloggers. I find it really rather beautiful that a group of people who have other jobs to do, spent a lot of time doing these analyses, unpaid, in their own time, simply to support Singh, because they believed it is the right thing to do. Simon Singh has analysed the data coolly in his book. But In the case that gave rise to the lawsuit he was writing in a newspaper. It was perfectly clear from the context what ‘bogus’ meant. but Mr Justice Eady (aided by a disastrous law) chose to ignore entirely the context and the question of truth. The description ‘bogus’. as used by Singh, seems to be entirely appropriate for a newspaper article. To criticise him for using “unprofessional” language is inappropriate because we are not dealing with professionals. At the heart of the problem is the sort of stifling political correctness that has resulted in quacks being referred to as “professions” rather than fantasists and fraudsters [of course I use the word fraudster with no implication that it necessarily implies conscious lying]. At least there are some laughs to be had from the whole sorry affair. Prompted by that prince among lawyers known as Jack of Kent there was a new addition to my ‘Patients’ Guide to Magic Medicine‘, as featured in the Financial Times.
|
It is, perhaps, misplaced political correctness that lies at the heart of the problem. Who can forget the letter from Lord Hunt, while he was at the Department of Health, in which he described “psychic surgery” (one of the best known fraudulent conjuring tricks) as a “profession”.
Follow-up
Two days later, the comment has appeared in the BMJ at last. But it has been altered a bit.
|
Unprofessional language is appropriate when dealing with unprofessional people Thanks for an excellent editorial. I doubt that it’s worth replying to Lionel Milgrom whose fantasy physics has been totally demolished by real physicists. Trisha Greenhalgh is, though, someone whose views I’d take very seriously. She raises an interesting question when she says “bogus” is an unprofessional word to use. Two things seem relevant. First, there is little point in writing rational scholarly articles for a group of people who do not seem to accept the ordinary rules of evidence or scholarship. You can point out to your heart’s content that “subluxations” are figment of the chiropractors’ imagination, but they don’t give a damn. Throughout my lifetime, pharmacologists and others have been writing scholarly articles about how homeopathy and other sorts of alternative medicine are bogus. All this effort had little effect. What made the difference was blogs and investigative journalism. When it became possible to reveal leaked teaching materials that taught students that “amethysts emit high yin energy”, and name and shame the vice-chancellors who allow that sort of thing to happen (in this case Prof Geoffrey Petts of Westminster University), things started to happen. In the last few years all five “BSc” degrees in homeopathy have closed and that is undoubtedly a consequence of the activities of bloggers and can assess evidence but who work more like investigative journalists. When the BCA released, 15 months after the event, its “plethora of evidence” a semi-organised effort by a group of bloggers produced, in less than 24 hours, thoroughly scholarly analyses of all of them (there is a summary here). As the editorial says, they didn’t amount to a hill of beans, They also pointed out the evidence that was omitted by the BCA. The conventional press just followed the bloggers. I find it really rather beautiful that a group of people who have other jobs to do, spent a lot of time doing these analyses, unpaid, in their own time, simply to support Singh, because they believed it is the right thing to do. Simon Singh has analysed the data coolly in his book. But In the case that gave rise to the lawsuit he was writing in a newspaper. It was perfectly clear from the context what ‘bogus’ meant. but Mr Justice Eady (aided by a disastrous law) chose to ignore entirely the context. The description ‘bogus’. as used by Singh, seems to be entirely appropriate for a newspaper article. To criticise him for using “unprofessional” language is inappropriate because we are not dealing with professionals. At least there are some laughs to be had from the whole sorry affair. Prompted by that prince among lawyers known as Jack of Kent there was a new addition to my ‘Patients’ Guide to Magic Medicine’, as featured in the Financial Times.
|
Here are the changes that were made. Hmm.very interesting.
Today brings a small setback for those of us interested in spreading sensible ideas about science. According to a press release
“The BMJ Group is to begin publishing a medical journal on acupuncture from next year, it was announced today (Tuesday 11 November 2008).
This will be the first complementary medicine title that the BMJ Group has published.”
And they are proud of that? What one earth is going on? The BMJ group is a publishing company which says, of itself,
“Our brand stands for medical credibility. We are one of the world’s best known and most respected medical publishers.”
Well perhaps it used to be.
They have certainly picked a very bad moment for this venture. In the last year there have been at least five good books that assess the evidence carefully and honestly. Of these, the ones that are perhaps the best on the subject of acupuncture are Singh & Ernst’s Trick or Treatment and Barker Bausell’s Snake Oil Science. Both Ernst and Bausell have first hand experience of acupuncture research. And crucially, none of these authors has any financial interest in whether the judgement goes for acupuncture or against it.
Here are quotations from Singh & Ernst’s conclusions
“Reliable conclusions from systematic reviews make it clear that acupuncture does not work for a whole range of conditions, except as a placebo.”
“There are some high quality trials that support the use of acupuncture for some types of pain and nausea, but there are also high quality trials that contradict this conclusion. In short, the evidence is neither consistent nor convincing – it is borderline.”
The House of Lords’ report in 2000 tended to give acupuncture the benefit of the (very considerable) doubt that existed at the time the report was written. Since that time there have been a lot of very well-designed trials of acupuncture.
Now it is quite clear that, for most (and quite possibly all) conditions, acupuncture is no more than a particularly theatrical placebo. Perhaps that is not surprising insofar as the modern western practice of acupuncture owes more to Chinese nationalistic propaganda that started in the time of Mao-Tse Tung than it owes to ancient wisdom (which often turns out to be bunk anyway).
The BMJ Group has decided to endorse acupuncture at a time when it is emerging that the evidence for any specific effect is very thin indeed. Well done.
The journal in question is this.
Acupuncture in Medicine is a quarterly title, which aims to build the evidence base for acupuncture. It is currently self-published by the British Medical Acupuncture Society (BMAS).
One good thing can be said about the Society and the Journal. That is that they don’t espouse all the mumbo-jumbo about ‘meridians’ and ‘Qi’. This, of course, puts them at odds with the vast majority of acupuncture teaching. This sort of internecine warfare between competing sects is characteristic of all sorts of alternative medicine. But that is just ideology. What matters is whether or not sticking needles in you is actually anything more than a placebo.
British Medical Acupuncture Society (BMAS)
The British Medical Acupuncture Society (BMAS). is “a registered charity established to encourage the use and scientific understanding of acupuncture within medicine for the public benefit.”. The phrase “encourage the use” suggests that they do not even envisage the possibility that it might not work. Their web site includes these claims.
Acupuncture can help in a variety of conditions:
- Pain relief for a wide range of painful conditions
- Nausea, especially postoperative nausea and vomiting
- Overactive bladder, also known as bladder detrusor instability
- Menstrual and menopausal problems, eg period pains and hot flushes
- Allergies such as hay fever and some types of allergic rashes
- Some other skin problems such as ulcers, itching and localised rashes
- Sinus problems and more
Presumably the word “help” is chosen carefully to fall just short of “cure”. The claims are vaguely worded, but let’s see what we can find about them from systematic reviews. It appears that the BMAS is being rather optimistic about the evidence.
BMJ Clinical Evidence is considered reliable and is particular interesting because it is owned by the BMJ Publishing Group.
Low Back Pain (chronic) Acupuncture is listed as being of “unknown effectiveness”.
Dysmenorrhoea Acupuncture is listed as being of “unknown effectiveness”.
Osteoarthritis of the knee. Acupuncture is listed as being of “unknown effectiveness”.
Psoriasis (chronic plaque) Acupuncture is listed as being of “unknown effectiveness”.
Neck pain “Acupuncture may be more effective than some types of sham treatment (not further defined) or inactive treatment (not further defined) at improving pain relief at the end of treatment or in the short term (less than 3 months), but not in the intermediate term (not defined) or in the long term (not defined)”
Headache (chronic tension-type) Acupuncture is listed as being of “unknown effectiveness”.
What about the greatest authority, the Cochrane Reviews?
Cochrane Reviews
Low back pain The data do not allow firm conclusions about the effectiveness of acupuncture for acute low-back pain. For chronic low-back pain, acupuncture is more effective for pain relief and functional improvement than no treatment or sham treatment immediately after treatment and in the short-term only. Acupuncture is not more effective than other conventional and “alternative” treatments. The data suggest that acupuncture and dry-needling may be useful adjuncts to other therapies for chronic low-back pain. Because most of the studies were of lower methodological quality, there certainly is a further need for higher quality trials in this area.
Chronic asthma. There is not enough evidence to make recommendations about the value of acupuncture in asthma treatment. Further research needs to consider the complexities and different types of acupuncture.
But most of the vaguely-worded claims made by BMAS have not been the subject of Cochrane reviews. The obvious interpretation of that is that there is not enough evidence to make it worth writing a review. In which case, why does BMAS claim that acupuncture can “help”?
Bandolier is another excellent source of high quality information, This was their view in September 2006
“Large, high-quality randomised trials of acupuncture have been published since the reviews. In fibromyalgia, chemotherapy-induced nausea and vomiting, breech presentation, tension headache, and migraine, all were negative compared with sham acupuncture. One in osteoarthritis of the knee, had statistical improvement over sham acupuncture at three months, but not later. Both large trials and this review of reviews come to the same general conclusion; that over a whole range of conditions and outcomes acupuncture cannot yet be shown to be effective.”
After thousands of years of acupuncture (or at least almost 40 years in the West) there seems to be very little to show for it.
The journal: Acupuncture in Medicine
What about the journal in question? Like all journals devoted to alternative medicine, it claims to be evidence-based. And like all journals devoted to alternative medicine it suffers from a fatal conflict of interest. If this journal were ever to conclude that acupuncture is a placebo, it would destroy the journal and the livelihoods of many of the people who write for it.
Scanning the first three issues of 2008 shows that it is very much like other alternative medicine journals. Most of the papers don’t address the critical question, is it a placebo. And most papers end up rather limply, with a statement along the lines “acupuncture may be useful for ***. More research is needed.”
The editor in chief of the journal is Dr Adrian White, and its editor is Michael Cummings. White is quoted by Ernst in the Guardian in 2004.
“We need to provide hard evidence to support what we all see in our clinics every day: that the modern approach to acupuncture works, and is highly relevant to the new, patient-centred NHS.” .
That means the answer is assumed in advance. That just isn’t science.. ‘We know the answer, all we have to do now is get some evidence’.
Why should the BMJ Group want to do a thing like this?
The press release says
Commenting on the move, BMJ and BMJ Journals Publishing Director, Peter Ashman, said “The journal is a good complement for our existing portfolio of journals and we’re certain that the Society’s members and other subscribers will appreciate the benefits of the decision the BMAS has made on their behalf.”
He continued: “The BMAS is ambitious for its journal to grow and flourish and we’re looking forward to working with the Society to develop an editorial and commercial strategy which will achieve the aims of BMAS and those of its members, while reaching out to the wider global community interested in this fascinating area of medicine.”
Yes, you got it. Money. The same motive that causes some vice-chancellors to bring their university into disrepute by awarding BSc degrees in subjects that are not only not science, but which are oftenly openly anti-science.
Conscience doesn’t seem to bother these people, so let’s put the problem in purely cash terms.
Both the BMJ Group and the vice-chancellors will have to decide whether the cash they gain is sufficient to counterbalance the corrosive effects of their actions on their own reputations.
Follow-up
Only a couple of days later, two new trials show acupuncture is no different from sham controls for helping IVF pregnancy rates. James Randerson in the Guardian writes thus.
“Acupuncture aimed at improving IVF success rates is widely offered by fertility clinics in the UK. In the first of the studies, researchers in Hong Kong split 370 women receiving IVF into two groups. One group received real acupuncture before and after having an IVF embryo implanted into their uterus. The other had the same procedure, except the treatment used retractable needles that did not penetrate the skin.”
“Of the 185 who received the sham treatment, 91 achieved a clinical pregnancy (foetal heartbeat identified using ultrasound) and 71 had a successful delivery. This compared with 72 clinical pregnancies in the true acupuncture group and 55 live births. The differences between the groups were not statistically significant.”
“In a second study, researchers in Chicago used a similar design in which 124 women received true or sham acupuncture. The control group had their skin punctured by real acupuncture needles, but not at genuine “Qi-lines” on the body. In the true acupuncture group, 43.9% achieved a clinical pregnancy, compared with 55.2% of the women given the sham treatment. “
The original paper for the first study can be seen here.