Download Lectures on Biostatistics (1971). Corrected and searchable version of Google books edition
Download review of Lectures on Biostatistics (THES, 1973).
badscience
This piece is almost identical with today’s Spectator Health article.
This week there has been enormously wide coverage in the press for one of the worst papers on acupuncture that I’ve come across. As so often, the paper showed the opposite of what its title and press release, claimed. For another stunning example of this sleight of hand, try Acupuncturists show that acupuncture doesn’t work, but conclude the opposite: journal fails, published in the British Journal of General Practice).
Presumably the wide coverage was a result of the hyped-up press release issued by the journal, BMJ Acupuncture in Medicine. That is not the British Medical Journal of course, but it is, bafflingly, published by the BMJ Press group, and if you subscribe to press releases from the real BMJ. you also get them from Acupuncture in Medicine. The BMJ group should not be mixing up press releases about real medicine with press releases about quackery. There seems to be something about quackery that’s clickbait for the mainstream media.
As so often, the press release was shockingly misleading: It said
Acupuncture may alleviate babies’ excessive crying Needling twice weekly for 2 weeks reduced crying time significantly
This is totally untrue. Here’s why.
|
Luckily the Science Media Centre was on the case quickly: read their assessment. The paper made the most elementary of all statistical mistakes. It failed to make allowance for the jelly bean problem. The paper lists 24 different tests of statistical significance and focusses attention on three that happen to give a P value (just) less than 0.05, and so were declared to be "statistically significant". If you do enough tests, some are bound to come out “statistically significant” by chance. They are false postives, and the conclusions are as meaningless as “green jelly beans cause acne” in the cartoon. This is called P-hacking and it’s a well known cause of problems. It was evidently beyond the wit of the referees to notice this naive mistake. It’s very doubtful whether there is anything happening but random variability. And that’s before you even get to the problem of the weakness of the evidence provided by P values close to 0.05. There’s at least a 30% chance of such values being false positives, even if it were not for the jelly bean problem, and a lot more than 30% if the hypothesis being tested is implausible. I leave it to the reader to assess the plausibility of the hypothesis that a good way to stop a baby crying is to stick needles into the poor baby. If you want to know more about P values try Youtube or here, or here. |
|
One of the people asked for an opinion on the paper was George Lewith, the well-known apologist for all things quackish. He described the work as being a "good sized fastidious well conducted study ….. The outcome is clear". Thus showing an ignorance of statistics that would shame an undergraduate.
On the Today Programme, I was interviewed by the formidable John Humphrys, along with the mandatory member of the flat-earth society whom the BBC seems to feel obliged to invite along for "balance". In this case it was professional acupuncturist, Mike Cummings, who is an associate editor of the journal in which the paper appeared. Perhaps he’d read the Science media centre’s assessment before he came on, because he said, quite rightly, that
"in technical terms the study is negative" "the primary outcome did not turn out to be statistically significant"
to which Humphrys retorted, reasonably enough, “So it doesn’t work”. Cummings’ response to this was a lot of bluster about how unfair it was for NICE to expect a treatment to perform better than placebo. It was fascinating to hear Cummings admit that the press release by his own journal was simply wrong.
Listen to the interview here
Another obvious flaw of the study is that the nature of the control group. It is not stated very clearly but it seems that the baby was left alone with the acupuncturist for 10 minutes. A far better control would have been to have the baby cuddled by its mother, or by a nurse. That’s what was used by Olafsdottir et al (2001) in a study that showed cuddling worked just as well as another form of quackery, chiropractic, to stop babies crying.
Manufactured doubt is a potent weapon of the alternative medicine industry. It’s the same tactic as was used by the tobacco industry. You scrape together a few lousy papers like this one and use them to pretend that there’s a controversy. For years the tobacco industry used this tactic to try to persuade people that cigarettes didn’t give you cancer, and that nicotine wasn’t addictive. The main stream media obligingly invite the representatives of the industry who convey to the reader/listener that there is a controversy, when there isn’t.
Acupuncture is no longer controversial. It just doesn’t work -see Acupuncture is a theatrical placebo: the end of a myth. Try to imagine a pill that had been subjected to well over 3000 trials without anyone producing convincing evidence for a clinically useful effect. It would have been abandoned years ago. But by manufacturing doubt, the acupuncture industry has managed to keep its product in the news. Every paper on the subject ends with the words "more research is needed". No it isn’t.
Acupuncture is pre-scientific idea that was moribund everywhere, even in China, until it was revived by Mao Zedong as part of the appalling Great Proletarian Revolution. Now it is big business in China, and 100 percent of the clinical trials that come from China are positive.
if you believe them, you’ll truly believe anything.
Follow-up
29 January 2017
Soon after the Today programme in which we both appeared, the acupuncturist, Mike Cummings, posted his reaction to the programme. I thought it worth posting the original version in full. Its petulance and abusiveness are quite remarkable.
I thank Cummings for giving publicity to the video of our appearance, and for referring to my Wikipedia page. I leave it to the reader to judge my competence, and his, in the statistics of clinical trials. And it’s odd to be described as a "professional blogger" when the 400+ posts on dcscience.net don’t make a penny -in fact they cost me money. In contrast, he is the salaried medical director of the British Medical Acupuncture Society.
It’s very clear that he has no understanding of the error of the transposed conditional, nor even the mulltiple comparison problem (and neither, it seems, does he know the meaning of the word ‘protagonist’).
I ignored his piece, but several friends complained to the BMJ for allowing such abusive material on their blog site. As a result a few changes were made. The “baying mob” is still there, but the Wikipedia link has gone. I thought that readers might be interested to read the original unexpurgated version. It shows, better than I ever could, the weakness of the arguments of the alternative medicine community. To quote Upton Sinclair:
“It is difficult to get a man to understand something, when his salary depends upon his not understanding it.”
It also shows that the BBC still hasn’t learned the lessons in Steve Jones’ excellent “Review of impartiality and accuracy of the BBC’s coverage of science“. Every time I appear in such a programme, they feel obliged to invite a member of the flat earth society to propagate their make-believe.
Acupuncture for infantile colic – misdirection in the media or over-reaction from a sceptic blogger?26 Jan, 17 | by Dr Mike Cummings So there has been a big response to this paper press released by BMJ on behalf of the journal Acupuncture in Medicine. The response has been influenced by the usual characters – retired professors who are professional bloggers and vocal critics of anything in the realm of complementary medicine. They thrive on oiling up and flexing their EBM muscles for a baying mob of fellow sceptics (see my ‘stereotypical mental image’ here). Their target in this instant is a relatively small trial on acupuncture for infantile colic.[1] Deserving of being press released by virtue of being the largest to date in the field, but by no means because it gave a definitive answer to the question of the efficacy of acupuncture in the condition. We need to wait for an SR where the data from the 4 trials to date can be combined. So what about the research itself? I have already said that the trial was not definitive, but it was not a bad trial. It suffered from under-recruiting, which meant that it was underpowered in terms of the statistical analysis. But it was prospectively registered, had ethical approval and the protocol was published. Primary and secondary outcomes were clearly defined, and the only change from the published protocol was to combine the two acupuncture groups in an attempt to improve the statistical power because of under recruitment. The fact that this decision was made after the trial had begun means that the results would have to be considered speculative. For this reason the editors of Acupuncture in Medicine insisted on alteration of the language in which the conclusions were framed to reflect this level of uncertainty. DC has focussed on multiple statistical testing and p values. These are important considerations, and we could have insisted on more clarity in the paper. P values are a guide and the 0.05 level commonly adopted must be interpreted appropriately in the circumstances. In this paper there are no definitive conclusions, so the p values recorded are there to guide future hypothesis generation and trial design. There were over 50 p values reported in this paper, so by chance alone you must expect some to be below 0.05. If one is to claim statistical significance of an outcome at the 0.05 level, ie a 1:20 likelihood of the event happening by chance alone, you can only perform the test once. If you perform the test twice you must reduce the p value to 0.025 if you want to claim statistical significance of one or other of the tests. So now we must come to the predefined outcomes. They were clearly stated, and the results of these are the only ones relevant to the conclusions of the paper. The primary outcome was the relative reduction in total crying time (TC) at 2 weeks. There were two significance tests at this point for relative TC. For a statistically significant result, the p values would need to be less than or equal to 0.025 – neither was this low, hence my comment on the Radio 4 Today programme that this was technically a negative trial (more correctly ‘not a positive trial’ – it failed to disprove the null hypothesis ie that the samples were drawn from the same population and the acupuncture intervention did not change the population treated). Finally to the secondary outcome – this was the number of infants in each group who continued to fulfil the criteria for colic at the end of each intervention week. There were four tests of significance so we need to divide 0.05 by 4 to maintain the 1:20 chance of a random event ie only draw conclusions regarding statistical significance if any of the tests resulted in a p value at or below 0.0125. Two of the 4 tests were below this figure, so we say that the result is unlikely to have been chance alone in this case. With hindsight it might have been good to include this explanation in the paper itself, but as editors we must constantly balance how much we push authors to adjust their papers, and in this case the editor focussed on reducing the conclusions to being speculative rather than definitive. A significant result in a secondary outcome leads to a speculative conclusion that acupuncture ‘may’ be an effective treatment option… but further research will be needed etc… Now a final word on the 3000 plus acupuncture trials that DC loves to mention. His point is that there is no consistent evidence for acupuncture after over 3000 RCTs, so it clearly doesn’t work. He first quoted this figure in an editorial after discussing the largest, most statistically reliable meta-analysis to date – the Vickers et al IPDM.[2] DC admits that there is a small effect of acupuncture over sham, but follows the standard EBM mantra that it is too small to be clinically meaningful without ever considering the possibility that sham (gentle acupuncture plus context of acupuncture) can have clinically relevant effects when compared with conventional treatments. Perhaps now the best example of this is a network meta-analysis (NMA) using individual patient data (IPD), which clearly demonstrates benefits of sham acupuncture over usual care (a variety of best standard or usual care) in terms of health-related quality of life (HRQoL).[3] |
30 January 2017
I got an email from the BMJ asking me to take part in a BMJ Head-to-Head debate about acupuncture. I did one of these before, in 2007, but it generated more heat than light (the only good thing to come out of it was the joke about leprechauns). So here is my polite refusal.
|
Hello Thanks for the invitation, Perhaps you should read the piece that I wrote after the Today programme Why don’t you do these Head to Heads about genuine controversies? To do them about homeopathy or acupuncture is to fall for the “manufactured doubt” stratagem that was used so effectively by the tobacco industry to promote smoking. It’s the favourite tool of snake oil salesman too, and th BMJ should see that and not fall for their tricks. Such pieces night be good clickbait, but they are bad medicine and bad ethics. All the best David |
The last email of Stephan Grimm has had more views than any other on this blog. “Publish and perish at Imperial College London: the death of Stefan Grimm“. Since then it’s been viewed more than 210,000 times. The day after it was posted, the server failed under the load.
Since than, I posted two follow-up pieces. On December 23, 2014 “Some experiences of life at Imperial College London. An external inquiry is needed after the death of Stefan Grimm“. Of course there was no external inquiry.
And on April 9, 2015, after the coroner’s report, and after Imperial’s internal inquiry, “The death of Stefan Grimm was “needless”. And Imperial has done nothing to prevent it happening again“.
On September 24th 2015, I posted a memorial on the first anniversary of his death. It included some of Grimm’s drawings that his mother and sister sent to me.
That tragedy led to two actions by Imperial, the metrics report (2015) and the bullying report (2016).
Let’s look at the outcomes.
The 2015 metrics report
In February 2015 and investigation was set up into the use of metrics to evaluate people, In December 2015 a report was produced: Application and Consistency of Approach in the Use of Performance Metrics. This was an internal enquiry so one didn’t expect very much from it. Out of 1338 academic staff surveyed at the College, 309 (23% of the total) responded
another 217 started the survey but did not submit anything). One can only speculate about the low return. It could be that 87% of staff were happy, or it could be that 87% of staff were frightened to give their opinions. It’s true that some departments use few if any metrics to assess people so one wouldn’t expect strong responses from them.
My position is clear: metrics don’t measure the quality of science, in fact they corrupt science.
This is not Imperial’s view though. The report says:
5.1 In seeking to form a view on performance metrics, we started from the premise that, whatever their benefits or deficiencies, performance metrics pervade UK universities. From REF to NSS via the THE and their attendant league tables, universities are measured and ranked in many dimensions and any view of performance metrics has to be formed in this context.
In other words, they simply acquiesce in the use of measures that demonstrably don’t do what’s claimed for them.
Furthermore the statement that “performance metrics pervade UK universities” is not entirely true. At UCL we were told in 2015.
“We will evaluate the quality of staff contributions appropriately, focusing on the quality of individual research outputs and their impact rather than quantity or journal-level metrics.” .
And one of the comments quoted in Imperial’s report says
“All my colleagues at MIT and Harvard etc tell me they reject metrics because they lead to mediocre candidates. If Imperial really wants to be a leader, it has to be bold enough to judge based on quality.”
It is rather shameful that only five UK universities (out of 114 or so) have signed the San Francisco Declaration on Research Assessment (DORA). I’m very happy that UCL is one of them, along with Sussex and Manchester, Birmingham and Liverpool. Imperial has not signed.
Imperial’s report concludes
“each department should develop profiles of its academic staff based on a series of published (ie open and transparent [perhaps on the College intranet]:”
There seems to be a word missing here. Presumably this means “open and transparent metrics“.
The gist of the report seems to be that departments can carry on doing what they want, as long as they say what it is. That’s not good enough, in my opinion.
A review of Imperial College’s institutional culture and its impact on gender equality
Unlike the metrics report, this one was external: that’s good. But, unlike the metrics report, it is secret: that’s bad.
The report was written by Alison Phipps (Director of Gender Studies and Reader in Sociology University of Sussex). But all that’s been released is an 11 page summary, written by Imperial, not by the authors of the report. When I asked Phipps for a copy of the whole report I was told
“Unfortunately we cannot share the full report – this is an internal document to Imperial, and we have to protect our research participants who told us their stories on this basis.”
It’s not surprising that the people who told their stories are afraid of repercussions. But it’s odd that their stories are concealed from everyone but the people who are in a position to punish them.
The report seems to have been commissioned because of this incident.
“The university apologised to the women’s rugby team after they were left playing to an empty stadium when the coaches ferrying spectators back to campus were allowed to leave early.”
“a member of staff was overheard saying that they did not care “how those fat girls” got home,”
But the report wasn’t restricted to sexism. It covered the whole culture at Imperial. One problem was that only 127 staff
and 85 students participated. There is no way to tell whether those who didn’t respond were happy or whether they were scared.
Here are some quotations from Imperial’s own summary of the secret report.
“For most, the meaning was restricted to excellence in research despite the fact that the College’s publicised mission statement gives equal prominence to research and education in the excellence context”
“Participants saw research excellence in metricised terms, positioning the College as a top-level player within the UK and in the world.”
Words used by those critical of Imperial’s culture included ” ‘cutthroat’, ‘intimidating’, ‘blaming’ and ‘arrogant’ “.
“Many participants in the survey and other methods felt that the external focus on excellence had emphasised internal competition rather than collaboration. This competition was noted as often being individualistic and adversarial. ”
“It was felt that there was an all-consuming focus on academic performance, and negative attitudes towards those who did not do well or who were not as driven as others. There was a reported lack of community spirit in the College’s culture including departments being ‘played off against each other’”
“The research findings noted comments that the lack of communal space on the campus had contributed to a lack of a community spirit. It was suggested that the College had ‘an impersonal culture’ and groups could therefore self-segregate in the absence of mechanisms for them to connect. ”
“There were many examples given to the researchers of bullying and discriminatory behaviour towards staff and students. These examples predominantly reflected hierarchies in work or study arrangements. ”
“The researchers reported that many of the participants linked it with the ‘elite’ white masculinity of the majority population, although a few examples of unacceptable behaviour by female staff and students were also cited. Examples of misogynistic and homophobic conduct were given and one interviewee expressed concern that the ‘ingrained misogyny’ at Imperial was so deep that it had become normal.”
“Although the College describes itself as a supportive environment, and many positive examples of that support were cited, a number of participants felt that senior management would turn a blind eye to poor behaviour if the individual involved was of value to the College.”
“Despite Imperial’s ‘no tolerance’ stance on harassment and bullying and initiatives such as ‘Have Your Say’, the researchers heard that people did not ‘speak up’ about many issues, ranging from discrimination and abuse to more subtle practices that leave people feeling vulnerable, unheard or undermined.”
“Relations between PIs and contract researchers were especially difficult, and often gendered as the PI was very often a man and the researcher a woman.”
“It was reported that there was also a clear sense of staff and students feeling afraid to speak up about issues and not receiving clear information or answers due to unclear institutional processes and one-way communication channels.”
“This representation of Imperial College as machine rather than organism resonated with observations on a culture of fear and silence, and the lack of empathy and community spirit at the College.”
“Some of the participants identified a surface commitment to diversity and representation but a lack of substantive system processes to support this. The obstacles to participation in the way of doing things at Imperial, and the associated issues of fear and insecurity, were reported as leading to feelings of hopelessness, demotivation, and low morale among some staff and students.”
“Some participants felt that Athena SWAN had merely scratched the surface of issues or had just provided a veneer which concealed continuing inequalities and that events such as the annual Athena SWAN lecture were little more than a ‘box ticking exercise.’”
The conclusions are pretty weak: e.g.
“They [the report’s authors] urged the College to implement changes that would ensure that its excellence in research is matched by excellence in other areas.”
Of course, Imperial College says that it will fix the problems. “Imperial’s provost, James Stirling, said that the institution must do better and was committed to gender equality”.
But that is exactly what they said in 2003
“The rector [then Richard Sykes] acknowledged the findings that came out of the staff audit – Imperial College – A Good Place to Work? – undertaken in August 2002.”
“He reinforced the message that harassment or bullying would not be tolerated in the College, and promised commitment from Council members and the Executive Committee for their continuing support to equal opportunities.”
This was eleven years before the pressure applied to Stefan Grimm caused him to take his own life. As always, it sounds good. But it seems that, thirteen years later, Imperial is going through exactly the same exercise.
It would be interesting to know whether Imperial’s Department of Medicine is still adopting the same cruel assessment methods as it was in 2007. Other departments at Imperial have never used such methods. It’s a continual source of bafflement to me that medicine, the caring profession, seems to care less for its employees that most other departments.
Other universities
Imperial is certainly not unique in having these problems. They are endemic. For example, Queen Mary, Kings College London and Warwick University have had similar problems, among many others.
Managers must learn that organisations function better when employees have good morale and are happy to work. Once again, I quote Scott Burkun (The myths of Innovation, 2007).
“Creation is sloppy; discovery is messy; exploration is dangerous. What’s a manager to do? The answer in general is to encourage curiosity and accept failure. Lots of failure.”
All big organisations are much the same -dissent is squashed and punished. Committees are set up. Fine-sounding statements are issued. But nothing much changes.
It should not be so.
Follow-up
‘We know little about the effect of diet on health. That’s why so much is written about it’. That is the title of a post in which I advocate the view put by John Ioannidis that remarkably little is known about the health effects if individual nutrients. That ignorance has given rise to a vast industry selling advice that has little evidence to support it.
The 2016 Conference of the so-called "College of Medicine" had the title "Food, the Forgotten Medicine". This post gives some background information about some of the speakers at this event. I’m sorry it appears to be too ad hominem, but the only way to judge the meeting is via the track record of the speakers.
Quite a lot has been written here about the "College of Medicine". It is the direct successor of the Prince of Wales’ late, unlamented, Foundation for Integrated Health. But unlike the latter, its name is disguises its promotion of quackery. Originally it was going to be called the “College of Integrated Health”, but that wasn’t sufficently deceptive so the name was dropped.
For the history of the organisation, see
Don’t be deceived. The new “College of Medicine” is a fraud and delusion
The College of Medicine is in the pocket of Crapita Capita. Is Graeme Catto selling out?
The conference programme (download pdf) is a masterpiece of bait and switch. It is a mixture of very respectable people, and outright quacks. The former are invited to give legitimacy to the latter. The names may not be familiar to those who don’t follow the antics of the magic medicine community, so here is a bit of information about some of them.
The introduction to the meeting was by Michael Dixon and Catherine Zollman, both veterans of the Prince of Wales Foundation, and both devoted enthusiasts for magic medicne. Zollman even believes in the battiest of all forms of magic medicine, homeopathy (download pdf), for which she totally misrepresents the evidence. Zollman works now at the Penny Brohn centre in Bristol. She’s also linked to the "Portland Centre for integrative medicine" which is run by Elizabeth Thompson, another advocate of homeopathy. It came into being after NHS Bristol shut down the Bristol Homeopathic Hospital, on the very good grounds that it doesn’t work.
Now, like most magic medicine it is privatised. The Penny Brohn shop will sell you a wide range of expensive and useless "supplements". For example, Biocare Antioxidant capsules at £37 for 90. Biocare make several unjustified claims for their benefits. Among other unnecessary ingredients, they contain a very small amount of green tea. That’s a favourite of "health food addicts", and it was the subject of a recent paper that contains one of the daftest statistical solecisms I’ve ever encountered
"To protect against type II errors, no corrections were applied for multiple comparisons".
If you don’t understand that, try this paper.
The results are almost certainly false positives, despite the fact that it appeared in Lancet Neurology. It’s yet another example of broken peer review.
It’s been know for decades now that “antioxidant” is no more than a marketing term, There is no evidence of benefit and large doses can be harmful. This obviously doesn’t worry the College of Medicine.
Margaret Rayman was the next speaker. She’s a real nutritionist. Mixing the real with the crackpots is a standard bait and switch tactic.
Eleni Tsiompanou, came next. She runs yet another private "wellness" clinic, which makes all the usual exaggerated claims. She seems to have an obsession with Hippocrates (hint: medicine has moved on since then). Dr Eleni’s Joy Biscuits may or may not taste good, but their health-giving properties are make-believe.
Andrew Weil, from the University of Arizona
gave the keynote address. He’s described as "one of the world’s leading authorities on Nutrition and Health". That description alone is sufficient to show the fantasy land in which the College of Medicine exists. He’s a typical supplement salesman, presumably very rich. There is no excuse for not knowing about him. It was 1988 when Arnold Relman (who was editor of the New England Journal of Medicine) wrote A Trip to Stonesville: Some Notes on Andrew Weil, M.D..
“Like so many of the other gurus of alternative medicine, Weil is not bothered by logical contradictions in his argument, or encumbered by a need to search for objective evidence.”
This blog has mentioned his more recent activities, many times.
Alex Richardson, of Oxford Food and Behaviour Research (a charity, not part of the university) is an enthusiast for omega-3, a favourite of the supplement industry, She has published several papers that show little evidence of effectiveness. That looks entirely honest. On the other hand, their News section contains many links to the notorious supplement industry lobby site, Nutraingredients, one of the least reliable sources of information on the web (I get their newsletter, a constant source of hilarity and raised eyebrows). I find this worrying for someone who claims to be evidence-based. I’m told that her charity is funded largely by the supplement industry (though I can’t find any mention of that on the web site).
Stephen Devries was a new name to me. You can infer what he’s like from the fact that he has been endorsed byt Andrew Weil, and that his address is "Institute for Integrative Cardiology" ("Integrative" is the latest euphemism for quackery). Never trust any talk with a title that contains "The truth about". His was called "The scientific truth about fats and sugars," In a video, he claims that diet has been shown to reduce heart disease by 70%. which gives you a good idea of his ability to assess evidence. But the claim doubtless helps to sell his books.
Prof Tim Spector, of Kings College London, was next. As far as I know he’s a perfectly respectable scientist, albeit one with books to sell, But his talk is now online, and it was a bit like a born-again microbiome enthusiast. He seemed to be too impressed by the PREDIMED study, despite it’s statistical unsoundness, which was pointed out by Ioannidis. Little evidence was presented, though at least he was more sensible than the audience about the uselessness of multivitamin tablets.
Simon Mills talked on “Herbs and spices. Using Mother Nature’s pharmacy to maintain health and cure illness”. He’s a herbalist who has featured here many times. I can recommend especially his video about Hot and Cold herbs as a superb example of fantasy science.
Annie Anderson, is Professor of Public Health Nutrition and
Founder of the Scottish Cancer Prevention Network. She’s a respectable nutritionist and public health person, albeit with their customary disregard of problems of causality.
Patrick Holden is chair of the Sustainable Food Trust. He promotes "organic farming". Much though I dislike the cruelty of factory farms, the "organic" industry is largely a way of making food more expensive with no health benefits.
The Michael Pittilo 2016 Student Essay Prize was awarded after lunch. Pittilo has featured frequently on this blog as a result of his execrable promotion of quackery -see, in particular, A very bad report: gamma minus for the vice-chancellor.
Nutritional advice for patients with cancer. This discussion involved three people.
Professor Robert Thomas, Consultant Oncologist, Addenbrookes and Bedford Hospitals, Dr Clare Shaw, Consultant Dietitian, Royal Marsden Hospital and Dr Catherine Zollman, GP and Clinical Lead, Penny Brohn UK.
Robert Thomas came to my attention when I noticed that he, as a regular cancer consultant had spoken at a meeting of the quack charity, “YestoLife”. When I saw he was scheduled tp speak at another quack conference. After I’d written to him to point out the track records of some of the people at the meeting, he withdrew from one of them. See The exploitation of cancer patients is wicked. Carrot juice for lunch, then die destitute. The influence seems to have been temporary though. He continues to lend respectability to many dodgy meetings. He edits the Cancernet web site. This site lends credence to bizarre treatments like homeopathy and crystal healing. It used to sell hair mineral analysis, a well-known phony diagnostic method the main purpose of which is to sell you expensive “supplements”. They still sell the “Cancer Risk Nutritional Profile”. for £295.00, despite the fact that it provides no proven benefits.
Robert Thomas designed a food "supplement", Pomi-T: capsules that contain Pomegranate, Green tea, Broccoli and Curcumin. Oddly, he seems still to subscribe to the antioxidant myth. Even the supplement industry admits that that’s a lost cause, but that doesn’t stop its use in marketing. The one randomised trial of these pills for prostate cancer was inconclusive. Prostate Cancer UK says "We would not encourage any man with prostate cancer to start taking Pomi-T food supplements on the basis of this research". Nevertheless it’s promoted on Cancernet.co.uk and widely sold. The Pomi-T site boasts about the (inconclusive) trial, but says "Pomi-T® is not a medicinal product".
There was a cookery demonstration by Dale Pinnock "The medicinal chef" The programme does not tell us whether he made is signature dish "the Famous Flu Fighting Soup". Needless to say, there isn’t the slightest reason to believe that his soup has the slightest effect on flu.
In summary, the whole meeting was devoted to exaggerating vastly the effect of particular foods. It also acted as advertising for people with something to sell. Much of it was outright quackery, with a leavening of more respectable people, a standard part of the bait-and-switch methods used by all quacks in their attempts to make themselves sound respectable. I find it impossible to tell how much the participants actually believe what they say, and how much it’s a simple commercial drive.
The thing that really worries me is why someone like Phil Hammond supports this sort of thing by chairing their meetings (as he did for the "College of Medicine’s" direct predecessor, the Prince’s Foundation for Integrated Health. His defence of the NHS has made him something of a hero to me. He assured me that he’d asked people to stick to evidence. In that he clearly failed. I guess they must pay well.
Follow-up
This post was written for the Spectator Health section, at short notice after the release of the spider letters. The following version is almost the same as appeared there, with a few updates. Some of the later sections are self-plagiarised from earlier posts.
Picture: Getty
The age of enlightenment was a beautiful thing. People cast aside dogma and authority. They started to think for themselves. Natural science flourished. Understanding of the natural world increased. The hegemony of religion slowly declined. Eventually real universities were created and real democracy developed. The modern world was born.
People like Francis Bacon, Voltaire and Isaac Newton changed the world for the better. Well, that’s what most people think. But not Charles, Prince of Wales and Duke of Cornwall.
In 2010 he said
"I was accused once of being the enemy of the Enlightenment,” he told a conference at St James’s Palace. “I felt proud of that.” “I thought, ‘Hang on a moment’. The Enlightenment started over 200 years ago. It might be time to think again and review it and question whether it is really effective in today’s conditions."
It seems that the Prince preferred things as they were before 1650. That’s a remarkable point of view for someone who, if he succeeds, will become the patron of that product of the age of enlightenment, the Royal Society, a venture that got its Royal Charter from King Charles II in1622.
I suppose that the Prince cannot be blamed for his poor education. He may have been at Trinity College Cambridge, his 2.2 degree is the current euphemism for a fail (it seems that he even failed to learn the dates of the enlightenment).
His behaviour has brought to the fore the question of the role of the monarchy.
A constitutional monarch is purely ceremonial and plays no part in politics. Well actually in the UK it isn’t quite as simple as that. The first problem is that we have no constitution. Things haven’t changed much since the 19th century when Walter Bagehot said “the Sovereign has, under a constitutional monarchy… three rights—the right to be consulted, the right to encourage, the right to warn.”.
These are real powers in a country which is meant to be run by elected representatives. But nobody knows how these powers are used: it is all done in secret. Well, almost all. The Prince of Wales has been unusually public in expressing his views. His views bear directly on government policy in many areas: medicine, architecture, agriculture and the environment. These are mostly areas that involve at least an elementary knowledge of science. But that is something that he lacks. Worse still, he seems to have no consciousness of his ignorance.
The Royal family should clearly have no influence whatsoever on government policies in a democracy. And they should be seen to have no influence. The Queen is often praised for her neutrality, but the fact is that nobody has the slightest idea what happens at the weekly meetings between the Prime Minister and the Queen. I doubt that she advises the prime minister to create a National Health Service, or to tax the rich. We shall never know that. We should do.
Almost the only light that has been thrown on the secret activities of Charles was the release, on 13 May, of 27 letters that the Prince wrote to government ministers in the Blair government between 2004 and 2005. It has take 10 years of effort by the Guardian to get hold of the letters. It was ike getting blood from a stone. When the Information Commissioner ruled that the letters should be made public, the decision was vetoed by the Conservative attorney general, Dominic Grieve. He said. of the "particularly frank" letters,
" Disclosure of the correspondence could damage The Prince of Wales’ ability to perform his duties when he becomes King."
That, of course, is precisely why the documents should be revealed.
If Charles’ ability to perform his duty as King is damaged, should his subjects be kept unaware of that fact? Of course not.
In this case, the law prevailed over the attorney general. After passing through the hands of 16 different judges, the Supreme Court eventually ruled, in March, that the government’s attempts to block release were unlawful. The government spent over £400,000 in trying, and failing, to conceal what we should know. The Freedom of Information Act (2000) is the best thing that Tony Blair did, though he, and Jack Straw, thought it was the worst. I expect they are afraid of what it might reveal about their own records. Transparency is not favoured by governments of any hue.
What do the letters say?
You can read all the letters on the Guardian web site. They give the impression of being written by a rather cranky old man with bees in his bonnet and too much time on his hands. The problem is that not all cranky old men can write directly to the prime minister, and get an answer.
Not all the letters are wrong headed. But all attempt to change government policy. They represent a direct interference in the political process by the heir to the throne. That is unacceptable in a democracy. It disqualifies him from becoming king.
Some letters verged on the bizarre.
|
21 October 2004 I particularly hope that the illegal fishing of the Patagonian Toothfish will be high on your list of priorities because until the trade is stopped, there is little hope for the poor old albatross. |
No doubt illegal fishing is a problem, but not many people would write directly to a minister about the Patagonian Toothfish.
Others I agree with. But they are still attempts to influence the policies of the elected government. This one was about the fact that supermarkets pay so little to dairy farmers for milk that sometimes it’s cheaper than bottled water.
|
To Tony Blair 8 September 2004 ". . . unless United Kingdom co-operatives can grow sufficiently the processors and retailers will continue to have the farmers in an arm lock and we will continue to shoot ourselves in the foot! You did kindly say that you would look at this . . . ". Yours ever, Charles |
He wrote to the minister of education to try to influence education policy.
|
22 February 2005 "I understand from your predecessor, Charles Clarke, that he has spoken to you about my most recent letter of 24th November, and specifically about the impact of my Education Summer School for teachers of English and History. This Programme, which involves up to ninety state school teachers each year, has been held over the past three years in Dartington, Devon, at Dunston, in Norfolk and at Buxton, in Derbyshire. I believe that they have added fresh inspiration to the national debate about the importance of English Literature and History in schools." Despite having made substantial progress, as you may be aware I remain convinced that the correct approaches to teaching and learning need to be challenged |
It’s interesting that the meeting was in Dartington. That’s near Totnes ("twinned with Narnia") and it’s a centre for the bizarre educational cult promoted by the mystic and racist, Rudolf Steiner.
Then we get a reference to one of Charles’ most bizarre beliefs, alternative medicine.
|
24 February 2005 Dear Prime Minister, We briefly mentioned the European Union Directive on Herbal Medicines, which is having such a deleterious effect on complementary medicine sector in this country by effectively outlawing the use of certain herbal extracts. I think we both agreed this was using a sledgehammer to crack a nut. You rightly asked me what could be done about it and I am asking the Chief Executive of my Foundation for Integrated Health to provide a more detailed briefing which I hope to be able to send shortly so that your advisers can look at it. Meanwhile, I have given Martin Hurst a note suggesting someone he could talk to who runs the Herbal Practitioner’s Association. Yours ever, Charles |
In this he opposes the EU Directive on Herbal Medicines. All this directive did was to insist that there was some anecdotal evidence for the safety of things that are sold to you. It asked for no evidence at all that they work, and it allowed very misleading labels. It provided the weakest form of protection from the deluded and charlatans. It was put into effect in the UK by the Medicines and Healthcare Products Regulatory Authority (MHRA). They even allowed products that were registered under this scheme to display an impressive-looking “kite-mark”. Most people would interpret this as a government endorsement of herbal medicines.
This got a sympathetic response from Tony Blair, someone who, along with his wife, was notoriously sympathetic to magic medicine.
|
30 March 2005 Dear Prince Charles Thanks too for your contacts on herbal medicines who have been sensible and constructive. They feel that the directive itself is sound and the UK regulators excellent, but are absolutely correct in saying that the implementation as it is currently planned is crazy. We can do quite a lot here: we will delay implementation for all existing products to 2011; we will take more of the implementation upon ourselves; and I think we can sort out the problems in the technical committee – where my European experts have some very good ideas. We will be consulting with your contacts and others on the best way to do this we simply cannot have burdensome regulation here. Yours ever, Tony |
Note "absolutely correct in saying that the implementation as it is currently planned is crazy. We can do quite a lot here: we will delay implementation for all existing products to 2011".
Government support for acupuncture and herbal medicine was made explicit in a letter from Health Secretary, John Reid (February 2005). He assures the prince that government is taking action to "enhance the status of the herbal medicine and acupuncture professions".
Nothing could reveal more clearly the clueless attitude of the then government to quackery. In fact, after 15 years of wrangling, the promised recognition of herbalism by statutory regulation never happened. One is reminded of the time that an equally-clueless minister, Lord (Phillip) Hunt, referred to ‘psychic surgery’ as a “profession”.
We got a preview of the Prince’s letters a month before the release when Max Hastings wrote in the Spectator
I have beside me a copy of a letter allegedly written by him some years ago to a cultural institution, asserting the conviction that ‘there is a DIVINE Source which is ultimate TRUTH… that this Truth can be expressed by means of numbers… and that, if followed correctly, these principles can be expressed with infinite variety to produce Beauty’.
You can’t get much barmier than that.
Are the letters harmless?
That has been the reaction on the BBC. I can’t agree. In one sense they so trivial that it’s amazing that the government thought it was a good use of £400,000 to conceal them. But they are all the evidence that we’ll get of the Prince’s very direct attempts to influence the political process.
The Prince of Wales is more than just a crank. He has done real harm. Here are some examples.
When the generally admirable NHS Choices re-wrote their advice on homeopathy (the medicines that contain no medicine) the new advice took two years to appear. It was held up in the Department of Health while consultations were made with the Prince’s Foundation for Integrated Health. That’s Charles’ lobby organisation for crackpot medicine. (The word "integrated" is the euphemism for alternative medicine that’s in favour with its advocates.) If it were not for the fact that I used the Freedom of Information Act to find out what was going on, the public would have been given bad advice as a direct result of the Prince’s political interference.
The Prince’s Foundation for Integrated Health (FIH) folded in 2010 as a result of a financial scandal, but it was quickly reincarnated as the "College of Medicine". It was originally going to be named the College of Integrated Medicine, but it was soon decided that this sounded too much like quackery, so it was given the deceptive name, College of Medicine. It appears to be financed by well-known outsourcing company Capita. It’s closely connected with Dr Michael Dixon, who was medical advisor to the FIH, and who tried to derail the advice given by NHS Choices.
Perhaps the worst example of interference by the Prince of Wales, was his attempt to get an academic fired. Prof Edzard Ernst is the UK’s foremost expert on alternative medicine. He has examined with meticulous care the evidence for many sorts of alternative medicine.Unfortunately for its advocates, it turned out that there is very little evidence that any of it works. This attention to evidence annoyed the Prince, and a letter was sent from Clarence House to Ernst’s boss, the vice-chancellor of the University of Exeter, Steve Smith. Shamefully, Smith didn’t tell the prince to mind his ow business, but instead subjected Ernst to disciplinary proceedings, After subjecting him to a year of misery, he was let off with a condescending warning letter, but Ernst was forced to retire early. In 2011and the vice-chancellor was rewarded with a knighthood. His university has lost an honest scientist but continues to employ quacks.
Not just interfering but costing taxpayers’ money
The Prince’s influence seems to be big in the Department of Health (DH). He was given £37,000 of taxpayers’ money to produce his Patients’ Guide (I produced a better version for nothing). And he was paid an astonishing £900,000 by DH to prepare the ground for the setting up of the hapless self-regulator, the Complementary and Natural Healthcare Council (CNHC, also known as Ofquack).
The Prince of Wales’ business, Duchy Originals, has been condemned by the Daily Mail, (of all places) for selling unhealthy foods. And when his business branched into selling quack “detox” and herbal nonsense he found himself censured by both the MHRA and the Advertising Standards Authority (ASA) for making unjustifiable medical claims for these products.
It runs in the family
The Prince of Wales is not the only member of the royal family to be obsessed with bizarre forms of medicine. The first homeopath to the British royal family, Frederick Quin, was a son of the Duchess of Devonshire (1765-1824). Queen Mary (1865-1953), wife of King George V, headed the fundraising efforts to move and expand the London Homeopathic Hospital. King George VI was so enthusiastic that in 1948 he conferred the royal title on the London Homeopathic Hospital.
The Queen Mother loved homeopathy too (there is no way to tell whether this contributed to her need for a colostomy in the 1960s).
The present Queen’s homeopathic physician is Peter Fisher, who is medical director of what, until recently was called the Royal London Homeopathic Hospital (RLHH). In 2010 that hospital was rebranded as the Royal London Hospital for Integrated medicine (RLHIM) in another unsubtle bait and switch move.
The RLHIM is a great embarrassment to the otherwise excellent UCLH Trust. It has been repeatedly condemned by the Advertising Standards Authority for making false claims. As a consequence, it has been forced to withdraw all of its patient information.
The patron of the RLHIM is the Queen, not the Prince of Wales. It is hard to imagine that this anachronistic institution would still exist if it were not for the influence, spoken or unspoken, of the Queen. Needless to say we will never be told.
The royal warrant for a firm that sells "meningitis vaccine" that contains nothing
Ainsworth’s homeopathic pharmacy is endorsed by both Prince Charles and the Queen: it has two Royal Warrants, one from each of them. They sell “homeopathic vaccines” for meningitis, measles, rubella and whooping cough. These “vaccines” contain nothing whatsoever so they are obviously a real danger to public health.
Despite the fact that Ainsworth’s had already been censured by the ASA in 2011 for selling similar products, Ainsworth’s continued to recommend them with a “casual disregard for the law”.
The regulator (the MHRA) failed to step in to stop them until it was eventually stirred into action by a young BBC reporter, Sam Smith who made a programme for BBC South West. Then, at last, the somnolent regulator was stirred into action. The MHRA “told Ainsworths to stop advertising a number of products” (but apparently not to stop making them or selling them).
They still sell Polonium metal 30C and Swine Meningitis 36C, and a booklet that recommends homeopathic “vaccination”.
Ainsworth’s sales are no doubt helped by the Royal Warrants. The consequence is that people may die of meningitis. In 2011, the MHRA Chief Executive Professor Kent Woods, was knighted. It was commented, justly, that
"Children will be harmed by this inaction. Children will die. And the fault must lie with Professor Sir Kent Woods, chairman of the regulator "
But the regulator has to fight the political influence of the Queen and Prince Charles. They lost.
The attorney general, while trying to justify the secrecy of Charles’ letters, said
“It is a matter of the highest importance within our constitutional framework that the Monarch is a politically neutral figure”.
Questions about health policy are undoubtedly political, and the highly partisan interventions of the Prince in the political process make his behaviour unconstitutional.
The Prince’s petulant outbursts not only endanger patients. They endanger the monarchy itself. Whether that matters depends on how much you value the tourist business generated by the Gilbert & Sullivan flummery at which royals excel.
The least that one can ask of the royal family is that they should not endanger the health of the nation. It would help if they refrained from using their influence on matters that are beyond their intellectual grasp..
If I wanted to know the winner of the 2.30 at Ascot, I’d ask a royal. For any other question I’d ask someone with more education.
Follow-up
The letters have made headlines in just about every newspaper. The Guardian had extensive coverage, of course.
The Times had a front page story "Revealed: how Charles got Blair to alter health policy" [pdf]
The British Medical Journal wrote "Prince Charles delayed regulation of herbal medicines" [pdf]
For me, the most shocking item was an interview given by Jack Straw, on Radio 4’s Today Programme. He was Home Secretary from 1997 to 2001 and Foreign Secretary from 2001 to 2006 under Tony Blair. From 2007 to 2010 he was Lord Chancellor. His response to the letters sounded like that of a right-wing conservative.
Like Blair. he deplored the Freedom of Information Act that his own government passed. He defended the secrecy, and supported the Conservative attorney-general’s attempt to veto the release of the letters. Perhaps his defence of secrecy is not surprising, He has a lot to hide, His involvement in the mendacity that led to the Iraq war, the dodgy dossier, his role in covering up torture (the "rendition" scandal). And He was suspended by the Labour party in February 2015 due to allegation of cash bribes.
He is certainly a man with plenty of things to hide.
The tragedy of the apparent suicide of Stefan Grimm is now known worldwide. His last email has been read by more than 160,000 people from over 200 countries. This post gathers together some of the reactions to his death. It’s a Christmas card for the people who are responsible.
 Alice Gast (president) |
 James Stirling (provost) |
 Dermot Kelleher (VP (health) |
“This isn’t about science – it’s about bragging rights, or institutional willy-waving.” from Grimm’s Tale
The back story
On Monday 1st December I published Stefan Grimm’s last email. It has been read by more than 160,000 people from over 200 different countries.
On Tuesday 2nd December, Stefan Grimm’s immediate boss, Martin Wilkins, wrote to me. He claimed “We met from time to time to discuss science and general matters. These meetings were always cordial. ”
On Wednesday 3rd December, the Dean of Medicine, Dermot Kelleher, mailed all Faculty of Medicine staff (not the rest of the College). Read the letter. It said very little. But it did include the words
“I regret I did not know Stefan personally, and I looked to colleagues to describe to me his life and the impact of his work at Imperial “
It seems a bit odd that the Dean of Medicine did not know a senior professor, but that seems to be life at Imperial.
On Thursday 4th December, Times Higher Education printed the same last email, and also the text of a threatening letter sent to Grimm in March.by his boss, Martin Wilkins. The letter was very far from being cordial, contrary to what Wilkins claimed. It included these words.
“I am of the opinion that you are struggling to fulfil the metrics of a Professorial post at Imperial College which include maintaining established funding in a programme of research with an attributable share of research spend of £200k p.a and must now start to give serious consideration as to whether you are performing at the expected level of a Professor at Imperial College.”
For a successful 51 year old with a good publication record to get a letter like that must have been devastating.
On Friday 5th December, Imperial made its first public announcement of his death. more than three months after it happened. By this time a damning account of his death had appeared even in the Daily Mail. The announcement read as though the world was unaware of his last words. It was a PR disaster: weasel words and crocodile tears. It made Imperial College appear to be totally heartless. The official announcement was accompanied by the phone numbers for the Samaritans. the chaplaincy and mental health first-aiders. Giving a person a phone number to call when you’ve destroyed their life is not an adequate substitute for treating staff properly.
Imperial are still trying to pretend that Grimm’s death is nothing to do with them, despite the fact that the whole world now knows quite enough of the facts to see otherwise.
The Coroner’s Inquest
The inquest into Grimm’s death was adjourned on October 8th, pending investigations into its cause. If you know anything relevant you should email the Coroner’s officer who is responsible for the investigation. That’s Molly Stewart (Molly.Stewart@lbhf.gov.uk). It is rather important that all the information doesn’t come from the College authorities, which cannot be relied on to tell the truth.
Some reports about the regime at Imperial College
Since my post went up on December 1st, I’ve had a stream of emails which testify to the reign of terror operated by the senior management at Imperial. The problem is by no means restricted to the Faculty of Medicine, though the problems seem to be worst there.
Many of these correspondents don’t want to speak in public. That’s certainly true of people who still work at Imperial, who have been warned to deflect all enquiries to HR. Here are some of the stories that I can reveal.
The Research Excellence Framework (REF) results were announced on 18th December. All university PR people hunted through the results, and all found something to boast tediously about. The letter from Imperial’s provost, James Stirling (read it), is pretty standard stuff. as is the letter from the Dean of Medicine, Dermot Kelleher (read it). Needless to say, neither letter mentioned the price in human misery, and even death, that Imperial had paid for its high ranking. I felt compelled to tweet
Imperial is second in #REF2014. And cost only one life #Grimmdeath. Well worth it uh? #HRbollocks http://t.co/25QNe8agjs
— David Colquhoun (@david_colquhoun) December 18, 2014
Kelleher promoted. Astonishingly, the very next day, the Dean of Medicine, on whose watch Grimm died, was promoted. You can read the letter from Imperial’s president, Alice Gast, in which this is announced. He is to be Vice President (Health), as a reward no doubt, for the cruel regime he ran as Dean. The letter has all the usual vacuous managerial buzzwords, e.g. “to support and grow the multidisciplinary paradigm in health”. Remember DC’s rule number one: never trust anyone who uses the word ‘paradigm’. Needless to say, still no mention of treating staff better.
Dr William J Astle.
Dr Astle is one of many people who wrote to me about his experiences at Imperial College. Although he still appears on Imperial’s web site, he now works as a statistician in a bioinformatics team at the University of Cambridge (see their web site).
He wrote again on 23 October 2014, to pass on an email (read the mail) that was sent to Department staff after Grimm’s last email had been circulated.(on 21 October). It is from a Faculty Operating Officer and ends with a warning to refer media enquries to a PR person (the Press and Internal Communications Manager, John-Paul Jones).
When he saw the internal email from Provost James Stirling with the usual self-congratulatory stuff about the REF, Astle wrote again to Stirling, His letter ends thus.
“Putting university staff in fear of losing their jobs leads to an atmosphere of obsequiousness and obedience to authority that prevents academics from fulfilling their institutional role. In a free society it is essential academics have the autonomy to determine their line of work, to question institutional and state authority and to do risky research. Once again I emphasise – in my experience the atmosphere in the faculty of medicine at IC is not conducive to this.”
Stirling did not reply to this letter. Neither Gast nor Stirling have replied to mine either. Discourtesy seems to be part of the job description of senior managers.
Christine Yates
Christine Yates says
“I was employed at Imperial College London from s” August 2002 to October 2012. For these 10 years I was the College’s Equality and Diversity Consultant in the Human Resources Department, reporting to the HR deputy director, Kim Everitt. In turn, Kim Everitt reports to the HR director, Louise Lindsay. Throughout this time I was the College’s sole equalities consultant, and over time built up the Equalities Unit and managed a team of five.”
“I was dismissed on 8th October 2012 following a Disciplinary Hearing in response to an allegation of gross misconduct “for continued wilful refusal to follow your Head of Department’s (HOD) instructions not to be involved in individual cases”.
As part of her job, she was responsible for establishing and maintaining the Harassment Support Contact Scheme, which was designed to help staff who felt they were being harassed, bullied, and victimised. She was also responsible for the College’s first Athena SWAN (scientific women’s academic network) .successful application, along with the establishment of disabilities, race equalities, and sexual orientation networks, all of which attained quality professional kite marks over time. The Athena Swan award is particularly ironic, given that Imperial’s present brutal assessment system must be even more unfair to women than it is to men. In 2003 (when Richard Sykes was still in charge), a third of female employees at Imperial reported that they were bullied. The improvement since then seems to have been small.
One of many cases she dealt with involved the harassment and bullying of a senior female academic by her male boss. Yates maintains, with good evidence, that complaints about this behaviour were never investigated properly by HR. This displeased HR. Incidents like this undoubtedly contributed to her dismissal.
“In Dr ***’s [female] case, it is clear to me that no independent investigations have been held and that College procedures are being flouted or rnanipulated with the alleged harasser (Professor **** [male]) being protected and permitted to continue his misconduct.”
“In my position as the College’s Equalities Consultant, I was aware of many cases and outcomes. Or ***’s is one of the most distressing and badly handled cases I was witness to, and the manner in which HR protect senior academics who have gravely offended, and who under any reasonable circumstances would be found to be guilty of gross misconduct, is a sad indictment of Imperial College”
You can read the statement that Christine Yates has already sent to the Coroner’s officer. Unfortunately the attachments have had to be removed here because they deal with specific cases.
“The Coroner’s Office needs to be aware of the pattern of behaviour that ensues whenever bad practice is brought to the College’s attention. In response to whistle blows and other complaints the College tries to discredit the complainant. When this fails they will invariably state that they will hold a ‘review’ usually undertaken by those responsible for the bad behaviour and thus with a vested interest in covering up any misconduct and impropriety. It is noted this pattern remains unchanged, “
A problem with a paper
An anonymous correspondent has sent me a lot of emails that concern a paper that was in revision at the time of Grimm’s death. The title of the paper is “Role of non-coding RNAs in apoptosis revealed in a functional genome-wide screen”.
On October 6th, one author wrote to his co-authors “I worked closely with Stefan on the screen data this year. We re-interpreted the mathematical analysis performed in the original manuscript, providing a more rigorous statistical foundation of the gene rankings. As a result, the gene list Stefan and I have generated is now different.”.
Clearly Grimm was aware of the need for revision before he died. Given that everyone was under such intense pressure to publish, it’s likely that the prospect of a prolonged delay in publication might well have contributed to his depression and his death.
The author who wrote on October 6th outlined some options. One was to leave the paper as it was, but to include all the raw data and submit to a journal such as Plos One or the preprint server BioRxiv. This option “requires minimal work, and would result in no change in the author list. However we would aim for a lower-impact journal.”. His preferred option, though, was to rewrite the paper altogether (and for himself to become first co-author) “as it is in all our shared interest to get the work published in as good a journal as possible. “.
Two days later, on October 8th, the same author thanked his co-authors for their responses. As a result of the responses he got, he asked to have his name removed from the paper because he did not agree with what was contained in the manuscript. “However, given that I believe the gene list is wrong, I request my name to be removed from the author list. If any other authors do not wish for the raw data to be disclosed then I hope you think it’s reasonable for me to close off my involvement with the paper.”.
The paper has 11 authors, including Stefan Grimm. . I have written to all but one of the authors to try to ascertain the facts. Of the four co-authors who have replied, all but one said that they hadn’t seen the final paper. One said that they were unaware that they were on the author list, and said they probably shouldn’t be.
I have tried to protect the authors (some of whom are still at Imperial) by not mentioning their names. But one co-author is sufficiently senior to be mentioned by name. Alan Boobis answered by my mail cordially enough when I first wrote to him, but declined to give much useful information, apart from confirming that Grimm was the senior author on the paper. On October 9th he wrote to all co-authors, thus.
|
From: Boobis, Alan R [a.boobis@imperial.ac.uk] Dear all The situation regarding this manuscript needs to be dealt with rationally. There is a real danger that the reputations of individuals and of the College will be harmed. I suggest that we all need to agree the most appropriate way forward. I am out of the country this week but will have my secretary liaise with you next week to arrange a suitable time (face to face or by phone) to discuss this. Best wishes, Alan |
I have no idea what the outcome of this meeting was. Personally. I always worry a bit when people want meetings “face to face or by phone”. Written records are much more informative.
I should like to make it clear that I’m not suggesting any misconduct whatsoever. The author who wished to withdraw acted with principle and courage, and mistakes happen. They are perhaps especially likely in multi-author papers where some authors don’t understand the input from others. But it is sad to see the emphasis on the long-discredited journal impact factor that was forced on them by Imperial’s policies. And it’s sad to see that several co-authors had not actually seen the final paper. This smacks of “citation-mongering”, yet another bad effect of the metrics culture that has pervaded all of academia, and which is enforced in an especially simple-minded way at Imperial.
This sad episode is yet another illustration of the way that Imperial’s policies are damaging people, and, in the end, damaging science.
Some discussions of the Imperial problem
Since Grimm’s last email was revealed, it’s been discussed in many blogs and articles. Here are a few of them.
Grimm’s tale (2 December). This perceptive blog reproduces part of the nasty threatening letter sent by Martin Wilkins to Grimm.
“Your current level of funding does not constitute the appropriate level for a professor at Imperial College. Unless you submit and are awarded a Platform grant as PI in the next 12 months we will seek to initiate disciplinary action against you.”
The author comments (my emphasis).
“This isn’t about science – it’s about bragging rights, or institutional willy-waving. Grimm was informed – in public – that he was to be fired, and left waiting for the axe to fall while the axe-wielder marauded around the campus boasting about it like an even more pathetic Alan Sugar.”
That sums it up for me. It’s very sad.
Another blog comments
“Martin Wilkins to Professor Stefan Grimm, a few months before the latter committed suicide. Imperial College had been pressuring Grimm to get 200, 000 pounds in grants in order for him to remain employed. They threatened to sack him as he only had 135,000 pounds.
Sounds a lot like loan sharks.”
Clearly universities like Imperial are no longer places for scholarship. They are more like anxiety machines.
The Nuffield Council on Bioethics produced an important report in the midst of the scandal about Grimm: The culture of scientific research 2014. Paragraph 1.7 produced a chilling statistic
1.7 Compromising on research integrity and standards
- Almost six in ten (58%) respondents are aware of scientists feeling under pressure to
compromise on research integrity and standards, with poor methodology and data fraud
frequently mentioned in the free text responses.- Just over a quarter (26%) of those taking part in the survey have felt tempted to compromise
on research integrity.
Stefan Grimm and the British University system. This blog, written by a geneticist. Federico Calboli, based in Helsinki, gives an indication of the harm that Imperial is inflicting not only on itself, but on the whole of UK academia, and hence on the UK economy
“As always in the real world the best laid plans often conflict with how the world actually works, and this conflict gives rise to a number of unintended consequences. The first unintended consequence is that the pursuit of what managements defines as ‘novel’ and ‘glamorous’ will diminish the intellectual value of British academia as a whole.”
“Unfortunately, since academia, funding bodies and the editorial boards of papers have been taken over by top down management culture, solid rigorous science is penalised in favour of anything that can be branded as ’novel’, ‘cutting edge’, ‘state of the art’ and similar platitudes.”
“This policy will leave British academia directionless and intellectually empty, and will transform any research in technology and data driven drivel that can at most pick up low hanging fruits and will deliver less and less as time goes on.”
Still more shaming, Calboli continues thus.
“The second problem with how British academia is managed is the culture of intellectual dishonesty that is forced upon people. People are not allowed to just express their goals in simple honest terms. They are required to spin and embellish everything in order to have half a chance of getting some funding or publishing in a high impact journal – both crucial to contribute to the ‘excellence metric”.
“Only the shameless cynics thrive in such environment”.
The blog finishes with a rallying cry.
“the email that Prof Grimm sent in October did not magically make its way to the press by itself. While many people are feeling disenchanted with academia and leave, more and more insiders are taking a combative stance against the mindless hogwash that threatens the foundations of British academia and the people that push it. We should all stand up and be counted, or we will not be able to complain in the future. It would be great if management could live up to its role and abandon the idea that scientific research is simple, predictable and quickly profitable, and actually help build the future of British academia.”
All this reflects similar sentiments to those that I expressed in 2007 [the RAE was the predecessor of the REF]
“The policies described here will result in a generation of ‘spiv’ scientists, churning out 20 or even more papers a year, with very little originality. They will also, inevitably, lead to an increase in the sort of scientific malpractice that was recently pilloried viciously, but accurately, in the New York Times, and a further fall in the public’s trust in science. That trust is already disastrously low, and one reason for that is, I suggest, pressures like those described here which lead scientists to publish when they have nothing to say.”
““All of us who do research (rather than talk about it) know the disastrous effects that the Research Assessment Exercise has had on research in the United Kingdom: short-termism, intellectual shallowness, guest authorships and even dishonesty”. Now we can add to that list bullying, harassment and an incompetent box-ticking style of assessment that tends to be loved by HR departments.
This process might indeed increase your RAE score in the short term (though there is no evidence that it it does even that). But, over a couple of decades, it will rid universities of potential Nobel prize winners.”
Conclusions
The policies adopted by Imperial College have harmed Imperial’s reputation throughout the world. Worse still, they have tainted the reputation of all UK universities. They have contributed to the corruption of science. and they have, in all probability, killed a successful man,
I hope that Alice Gast (president), James Stirling (provost). Dermot Kelleher (Dean, now vice president), and Martin Wilkins (who was left to weild the knife) have a good Christmas. If I were in their shoes, I’d feel so guilty that I wouldn’t be able to sleep at night.
They should spend Christmas reading Peter Lawrence’s wonderful essay on The Mismeasurement of Science. Please download a copy
Their proposal that HR policies should be investigated by, inter alia, the head of HR has provoked worldwide derision.
Their refusal to set up an independent external inquiry is reprehensible.
Not for the first time, a fine institution is being brought into disrepute by its leadership. Council please note.
|
Alice Gast |
James Stirling |
Dermot Kelleher |
Perhaps the best description of what’s going on is from Grimm’s Tale “This isn’t about science – it’s about bragging rights, or institutional willy-waving.”. Gast, Stirling and Kelleher should stop the willy-waving. They should either set about rectifying the damage they’ve done. Or they should resign. Now.
The chair of universities HR association, Kim Frost, said
“Bullying is a very emotive term, and what one person experiences as bullying will often be simple performance management from their manager’s point of view.”.
That’s scary because it shows that she hasn’t the slightest idea about “performance management”. I have news for HR people. They are called experiments because we don’t know whether they will work. If they don’t work that’s not a reason to fire anyone. No manager can make an experiment come out as they wish. The fact of the matter is that it’s impossible to manage research. If you want real innovation you have to tolerate lots and lots of failure. “Performance management” is an oxymoron. Get used to it.
This sorry episode has far more general lessons for the way the REF is conducted and for the metrics sales industry. Both share some of the guilt.
That will have to wait for another post.
Follow-up
25 December 2014. Universities "eliminate tenure because Starbucks does not have tenure"
I was struck by this excerpt from a Christmas newsletter from a colleague. Buried among the family news was buried this lament. He’s writing about Rush University, Chicago, but much the same could be said about many universities, not only in the USA.
|
Rush Medical Center built an $800 million hospital building that is clinically state-of-the-art and architecturally unique. Now it is poised to become a world class center of basic and clinical research. Sadly, rather than listen to researchers who have devoted their careers to Rush, senior administration hears advice from fly-by-night financial consultants who apply the same “Business Model” to medical care, education, and research as to a shoe factory. Perhaps because fiscal consulting requires little skill or training*, they do not distinguish between a researcher and a Starbucks employee [literally true!]. They eliminate tenure because Starbucks does not have tenure. {To be fair, they have only eliminated “tenure of salary” – one may continue working with a title, but without pay!} They cannot imagine that world-class research is an art that requires years of training, cultivating an international network of colleagues, and most importantly, continuity of funding. Because their work is so trivial, they cannot fathom that researchers could be utterly unique and irreplacable. And they do not care – they will destroy research at Rush, collect their multi-million dollar fee, and move on to the next shoe factory. *Lesson 1. Fire people who do real work, cut wages, steal from pension funds, eliminate unions and job security. Congratulations you are now a qualified fiscal consultant! |
26 December 2014
|
Grimm is not the only one. In the same month, September 2014, Tony Veitch was found dead. He was a senior scientist in the lab at Kew Botanical Gardens. He was 49, much the same age as Stefan Grimm. It’s presumed that he committed suicide after being told to reapply for his own job. |
 |
!7 January 2015
I hear that Imperial College’s UCU passed this motion.
|
Motion 3: Branch condemns bullying and harassment of staff at Imperial This branch strongly condemns the bullying and harassment of staff at Imperial, particularly by some managers. We call upon the senior management of the College to ensure that all managers are properly trained to deal with staff in a fair and considerate manner and on how to refrain from bullying and harassment. In light of a recent tragic case at Imperial, the College management must ensure that they fulfil their duty of care to all staff at all times. |
Of course every employer claims that they do this.
I wonder how the officials can mouth these platitudes when the facts, now well known, show them to be untrue,
The first post and this one have been viewed over 173.000 times, from at least 170 countries (UK, USA,and then almost 10,000 views from China). I realise that this must have harmed Imperial, but they have brought it on themselves. Neither the president nor the rector have had the courtesy to answer perfectly polite letters.
I wrote also on 29 December to the chair of Council. Eliza Manningham-Buller. She has still not acknowledged receipt, never mind replied.
I am amazed by the discourtesy of people who regard themselves as too important to reply to letters.
|
To chair of Council, Imperial College London 29 December 2014 Dear Lady Manningham-Buller A problem with management at Imperial It cannot have escaped your notice that a senior member of Imperial’s staff was found dead, after being told that he’d lose his job if he didn’t raise £200,000 in grants within a year. When I posted Stefan Grimm’s last email on my blog on December 1st it went viral (Publish and perish at Imperial College London), It has been read by over 160,000 people from over 200 countries. That being the case, Imperial’s first official mention of the matter on December 4th looked pretty silly. It was written as though his email was not already common knowledge –totally hamfisted public relations. After posting Grimm’s last mail, I was deluged with mails about people who had been badly treated at Imperial. I posted a few of them on December 23rd (Some experiences of life at Imperial College London. An external inquiry is needed after the death of Stefan Grimm). The policy of telling staff that their research must be expensive is not likely to be appreciated by the taxpayer. Neither will it improve the quality of science. On the contrary, the actions of the College are very likely to deter good scientists from working there (I already heard of two examples of people who turned down jobs at Imperial). I think it is now clear that the senior management team is pursuing policies that are damaging the reputation of Imperial. I hope that Council will take appropriate action. Best regards David Colquhoun |
20 January 2015
Today I got a reply to the letter (above) that I sent to Eliza Manningham-Buller on 29 December. You can download it.
I guess it’s not surprising that the reply says nothing helpful.
It endorses the idea that HR should investigate their own practices, an idea that the outside world greets with ridicule.
It reprimands me for making "unprofessional" comments about individuals. That’s what happens when people behave badly. It would be unprofessional to fail to point out what’s going on. It’s the job of journalists to name people. All else is PR.
It suggests that I may have not followed the Samaritans’ guidelines for reporting of suicide. I’ve read their document and I don’t believe that either I, or Times Higher Education, have breached the guidelines.
The letter says. essentially, please shut up, you are embarrassing Imperial. It’s fascinating to see the rich and powerful close ranks when criticised. But it is very disappointing. It seems to me to be very much in the public interest to have published the last email of Stefan Grimm.
But I guess the last person you’d expect to champion transparency is an ex-head of MI5.
Felix, Imperial’s student newspaper, carried an interesting article Death of Professor Grimm: the world reacts. The events at Imperial have been noted all over the world (at least 170 countries according to my own Google analytics) but the response has been especially big in China. Alienating a country like China seems to me to rank as bringing the College into disrepute.
9 February 2015
Death in Academia and the mis-measurement of science. Good article in Euroscientist by Arren Frood
25 February 2015
I see that Dermot Kelleher is leaving Imperial for the University of British Columbia. Perhaps he hopes that he’ll be able to escape his share of the blame for the death of Stefan Grimm? Let’s hope, for the sake of UBC, that he’s learned a lesson from the episode.
10 March 2015
The Vancouver Sun has been asking questions. An article by Pamela Fayerman includes the following.
"Recently, Imperial College was engulfed in a controversy involving a tragedy. . . . a medical school professor, Stefan Grimm, took his own life last fall. He left an email that accused unnamed superiors of bullying through demands that he garner more research grants. The “publish or peril” adage that scientists so often cite seems like it may apply in this case.
The college said it would set up an internal inquiry into the circumstances around the toxicology professor’s death, but the results have not been released. UBC provost Dave Farrar said in an interview that the death of the professor at Imperial College was never even discussed during the recruitment process.
Kelleher said in a long distance phone interview that the tragedy had nothing to do with his reasons for leaving Imperial. And he can’t speak about the case since it is currently under review by a coroner."
Well, I guess he would say that, wouldn’t he? Kelleher has been at Imperial for less than three years, and the generous intepretation of his departure is that he didn’t like the bullying regime. It had been going on long before Kelleher arrived, as documented on this blog in in 2007.
It’s interesting to speculate about why he wasn’t asked about Grimm’s death (if that’s true).
Did the University of British Columbia think it was irrelevant?
Or did they want him to establish a similar regime of “performance management” at UBC?
Or were the senior people at UBC not even aware of the incident?
Perhaps the third option is the most likely: it’s only too characteristic of senior managers to be unaware of what’s happening on the shop floor. Just as in banks.
11 March 2015
It’s beginning to look like an exodus. The chair of Imperial’s council, Eliza Manningham-Buller, is also leaving. Despite her condescending response to my inquiries, perhaps she too is scared of what will be revealed about bullying. I just hope that she doesn’t bring Imperial’s ideas about "performance management" to the Wellcome Trust.
In the course of thinking about metrics, I keep coming across cases of over-promoted research. An early case was “Why honey isn’t a wonder cough cure: more academic spin“. More recently, I noticed these examples.
“Effect of Vitamin E and Memantine on Functional Decline in Alzheimer Disease".(Spoiler -very little), published in the Journal of the American Medical Association. ”
and ” Primary Prevention of Cardiovascular Disease with a Mediterranean Diet” , in the New England Journal of Medicine (which had second highest altmetric score in 2013)
and "Sleep Drives Metabolite Clearance from the Adult Brain", published in Science
In all these cases, misleading press releases were issued by the journals themselves and by the universities. These were copied out by hard-pressed journalists and made headlines that were certainly not merited by the work. In the last three cases, hyped up tweets came from the journals. The responsibility for this hype must eventually rest with the authors. The last two papers came second and fourth in the list of highest altmetric scores for 2013
Here are to two more very recent examples. It seems that every time I check a highly tweeted paper, it turns out that it is very second rate. Both papers involve fMRI imaging, and since the infamous dead salmon paper, I’ve been a bit sceptical about them. But that is irrelevant to what follows.
Boost your memory with electricity
That was a popular headline at the end of August. It referred to a paper in Science magazine:
“Targeted enhancement of cortical-hippocampal brain networks and associative memory” (Wang, JX et al, Science, 29 August, 2014)
This study was promoted by the Northwestern University "Electric current to brain boosts memory". And Science tweeted along the same lines.
|
Science‘s link did not lead to the paper, but rather to a puff piece, "Rebooting memory with magnets". Again all the emphasis was on memory, with the usual entirely speculative stuff about helping Alzheimer’s disease. But the paper itself was behind Science‘s paywall. You couldn’t read it unless your employer subscribed to Science.
|
 |
|
All the publicity led to much retweeting and a big altmetrics score. Given that the paper was not open access, it’s likely that most of the retweeters had not actually read the paper. |
|
When you read the paper, you found that is mostly not about memory at all. It was mostly about fMRI. In fact the only reference to memory was in a subsection of Figure 4. This is the evidence.
That looks desperately unconvincing to me. The test of significance gives P = 0.043. In an underpowered study like this, the chance of this being a false discovery is probably at least 50%. A result like this means, at most, "worth another look". It does not begin to justify all the hype that surrounded the paper. The journal, the university’s PR department, and ultimately the authors, must bear the responsibility for the unjustified claims.
Science does not allow online comments following the paper, but there are now plenty of sites that do. NHS Choices did a fairly good job of putting the paper into perspective, though they failed to notice the statistical weakness. A commenter on PubPeer noted that Science had recently announced that it would tighten statistical standards. In this case, they failed. The age of post-publication peer review is already reaching maturity
Boost your memory with cocoa
Another glamour journal, Nature Neuroscience, hit the headlines on October 26, 2014, in a paper that was publicised in a Nature podcast and a rather uninformative press release.
"Enhancing dentate gyrus function with dietary flavanols improves cognition in older adults. Brickman et al., Nat Neurosci. 2014. doi: 10.1038/nn.3850.".
The journal helpfully lists no fewer that 89 news items related to this study. Mostly they were something like “Drinking cocoa could improve your memory” (Kat Lay, in The Times). Only a handful of the 89 reports spotted the many problems.
A puff piece from Columbia University’s PR department quoted the senior author, Dr Small, making the dramatic claim that
“If a participant had the memory of a typical 60-year-old at the beginning of the study, after three months that person on average had the memory of a typical 30- or 40-year-old.”
|
Like anything to do with diet, the paper immediately got circulated on Twitter. No doubt most of the people who retweeted the message had not read the (paywalled) paper. The links almost all led to inaccurate press accounts, not to the paper itself. |
|
But some people actually read the paywalled paper and post-publication review soon kicked in. Pubmed Commons is a good site for that, because Pubmed is where a lot of people go for references. Hilda Bastian kicked off the comments there (her comment was picked out by Retraction Watch). Her conclusion was this.
"It’s good to see claims about dietary supplements tested. However, the results here rely on a chain of yet-to-be-validated assumptions that are still weakly supported at each point. In my opinion, the immodest title of this paper is not supported by its contents."
(Hilda Bastian runs the Statistically Funny blog -“The comedic possibilities of clinical epidemiology are known to be limitless”, and also a Scientific American blog about risk, Absolutely Maybe.)
NHS Choices spotted most of the problems too, in "A mug of cocoa is not a cure for memory problems". And so did Ian Musgrave of the University of Adelaide who wrote "Most Disappointing Headline Ever (No, Chocolate Will Not Improve Your Memory)",
Here are some of the many problems.
- The paper was not about cocoa. Drinks containing 900 mg cocoa flavanols (as much as in about 25 chocolate bars) and 138 mg of (−)-epicatechin were compared with much lower amounts of these compounds
- The abstract, all that most people could read, said that subjects were given "high or low cocoa–containing diet for 3 months". Bit it wasn’t a test of cocoa: it was a test of a dietary "supplement".
- The sample was small (37ppeople altogether, split between four groups), and therefore under-powered for detection of the small effect that was expected (and observed)
- The authors declared the result to be "significant" but you had to hunt through the paper to discover that this meant P = 0.04 (hint -it’s 6 lines above Table 1). That means that there is around a 50% chance that it’s a false discovery.
- The test was short -only three months
- The test didn’t measure memory anyway. It measured reaction speed, They did test memory retention too, and there was no detectable improvement. This was not mentioned in the abstract, Neither was the fact that exercise had no detectable effect.
- The study was funded by the Mars bar company. They, like many others, are clearly looking for a niche in the huge "supplement" market,
The claims by the senior author, in a Columbia promotional video that the drink produced "an improvement in memory" and "an improvement in memory performance by two or three decades" seem to have a very thin basis indeed. As has the statement that "we don’t need a pharmaceutical agent" to ameliorate a natural process (aging). High doses of supplements are pharmaceutical agents.
To be fair, the senior author did say, in the Columbia press release, that "the findings need to be replicated in a larger study—which he and his team plan to do". But there is no hint of this in the paper itself, or in the title of the press release "Dietary Flavanols Reverse Age-Related Memory Decline". The time for all the publicity is surely after a well-powered study, not before it.
The high altmetrics score for this paper is yet another blow to the reputation of altmetrics.
One may well ask why Nature Neuroscience and the Columbia press office allowed such extravagant claims to be made on such a flimsy basis.
What’s going wrong?
These two papers have much in common. Elaborate imaging studies are accompanied by poor functional tests. All the hype focusses on the latter. These led me to the speculation ( In Pubmed Commons) that what actually happens is as follows.
- Authors do big imaging (fMRI) study.
- Glamour journal says coloured blobs are no longer enough and refuses to publish without functional information.
- Authors tag on a small human study.
- Paper gets published.
- Hyped up press releases issued that refer mostly to the add on.
- Journal and authors are happy.
- But science is not advanced.
It’s no wonder that Dorothy Bishop wrote "High-impact journals: where newsworthiness trumps methodology".
It’s time we forgot glamour journals. Publish open access on the web with open comments. Post-publication peer review is working
But boycott commercial publishers who charge large amounts for open access. It shouldn’t cost more than about £200, and more and more are essentially free (my latest will appear shortly in Royal Society Open Science).
Follow-up
Hilda Bastian has an excellent post about the dangers of reading only the abstract "Science in the Abstract: Don’t Judge a Study by its Cover"
4 November 2014
I was upbraided on Twitter by Euan Adie, founder of Almetric.com, because I didn’t click through the altmetric symbol to look at the citations "shouldn’t have to tell you to look at the underlying data David" and "you could have saved a lot of Google time". But when I did do that, all I found was a list of media reports and blogs -pretty much the same as Nature Neuroscience provides itself.
More interesting, I found that my blog wasn’t listed and neither was PubMed Commons. When I asked why, I was told "needs to regularly cite primary research. PubMed, PMC or repository links”. But this paper is behind a paywall. So I provide (possibly illegally) a copy of it, so anyone can verify my comments. The result is that altmetric’s dumb algorithms ignore it. In order to get counted you have to provide links that lead nowhere.
So here’s a link to the abstract (only) in Pubmed for the Science paper http://www.ncbi.nlm.nih.gov/pubmed/25170153 and here’s the link for the Nature Neuroscience paper http://www.ncbi.nlm.nih.gov/pubmed/25344629
It seems that altmetrics doesn’t even do the job that it claims to do very efficiently.
It worked. By later in the day, this blog was listed in both Nature‘s metrics section and by altmetrics. com. But comments on Pubmed Commons were still missing, That’s bad because it’s an excellent place for post-publications peer review.
|
Maurice Nathan Saatchi, Baron Saatchi is an advertising man who, with his brother, Charles Saatchi ("‘why tell the truth when a good lie will do?), became very rich by advertising cigarettes and the Conservative party. After his second wife died of cancer he introduced a private members bill in the House of Lords in 2012. The Medical Innovation Bill came back to the Lords for its second reading on 24 October 2014. |
 |
The debate was deeply depressing: very pompous and mostly totally uninformed. You would never have guessed that the vast majority of those who understand the problem are against the bill. The Bill has failed to win the support of The British Medical Association, The Motor Neurone Disease Association, the Royal College of Physicians, the Royal College of Surgeons, the Medical Research Society, the Royal College of Radiologists, The Medical Defence Union, the Academy of Medical Sciences, the Royal College of Pathologists, the Royal College of General Practitioners, the Academy for Healthcare Science, the Wellcome Trust, Action Against Medical Accidents, and patient advocacy charities Health Watch and the Nightingale Collaboration, and others. Conservative MP Sarah Wollaston, a former GP, has “very serious concerns” about it.
Grief is not a good basis for legislation. In the Daily Telegraph, Lord Saatchi attributes the lack of a "cure for cancer" to fear of litigation.
“Any deviation by a doctor is likely to result in a verdict of guilt for medical negligence. The law defines medical negligence as deviation from that standard procedure. But as innovation is deviation, non-deviation is non-innovation. This is why there is no cure for cancer.”
This statement is utterly bizarre. Evidently Lord Saatchi knows much more about how to sell cigarettes than he does about how to discover new drugs. The reason there is no cure is that it’s a very difficult problem. It has nothing to do with litigation
Almost every medical organisation, and many lawyers, have pointed out the flaws in his ideas,. But slick, and often mendacious, advertising of his cure has deceived many of our scientifically-illiterate parliamentarians, and the bill is making progress
Lord Saatchi’s Bill would allow uncontrolled testing of treatments on any patient. It is not limited to cancer, nor to terminally ill patients (though some amendments, yet to be accepted, might change that). This sort of uncontrolled experimentation is likely to impede advances in treatment rather than to help them. And the vagueness of the wording of the bill could lead to an increase in litigation, rather than the intended decrease.
It is no coincidence that the legion of cancer quacks is in favour of the bill. It opens the door to their nonsense. Big pharma is likely to benefit too, because they will be able to sell improperly tested drugs with little or no effectiveness and immunity from prosecution if they do harm.
For more information about the Bill, see the excellent site, Stop the Saatchi Bill. There are sections on Dispelling Myths, on Professional Concerns, and on the many good articles and blogs that have pointed out the many problems with the bill.
Two recent articles are well worth reading
Saatchi’s ‘Medical Innovation Bill’ will benefit lawyers and charlatans, not patients, by neurologist David Nicholl. And it’s particularly interesting that the Telegraph’s arch-conservative, Peter Oborne has come out strongly against the bill, in The ‘Saatchi Bill’ is dangerous and will bring nothing but harm. I can’t agree with his opening words ("Lord Saatchi is rightly regarded as a national treasure. In his early days he was the advertising genius who played a fundamental role in selling the Thatcherite message"), but I agree entirely with
“The PR campaign distorts the facts, exploiting ignorance and desperation to plant false hope in the rich soil of multiple fallacies.”
“Quacks will be given free rein. No “treatment” is so loopy (or potentially dangerous) that the Saatchi Bill won’t protect doctors who prescribe or administer it from prosecution.”
“The propagation of falsehoods and fallacies surrounding and arising from this Bill will have a corrosive effect on medical ethics and society as a whole.”
Lord Saatchi, who proposed the bill said “To prevent more needless cancer deaths, doctors must be free to innovate”. This statement shows he has no idea of the efforts that go into cancer research. We are doing our best, but it is a very hard problem. In some areas (like breast cancer) there have been big advances. In others areas (e.g. pancreatic or ovarian cancer) there has not yet been any perceptible progress. It will take many more years of hard scientific work. Individual doctors tinkering with speculative untested treatments will not solve the problem.
- The only people who will benefit from the bill will be lawyers, quacks and big pharma.
- “Innovation” is undefined –the bill is based on the myth that there exists a miraculous cure waiting to be found. Sadly, the probability of this happening is vanishingly small.
- To some extent, medicine is victim of its own hype. The public feels it has a right to demand the latest miracle cure. Too often, they don’t exist.
- There is no need for the bill because doctors can already prescribe whatever they want. The Medical Defence Union says it has no reason to think that doctors are deterred from innovating by the fear of litigation.
- The bill won’t help to discover new cures –in fact it is more likely to hinder it, especially if it deters people from joining properly organised clinical trials. The bill will generate many separate anecdotes which may or may not be published. That’s the worst possible way to do research.
- The bill removes protections from patients and is more likely to increase litigation than reduce it.
- Anyone unlucky enough to get cancer is immediately a target for a legion of quacks trying to sell you crazy “treatments”. There is nothing to stop even the lunatic fringe of alternative medicine, homeopaths, promoting their sugar pills. There are, disgracefully, several hundred homeopaths with medical degrees –all a quack has to do is to ask another quack doctor to endorse their daft ideas.
- For untested treatments there is, by definition, no reason to think they’ll do more good than harm. The cancer surgeon, Michael Baum said, rightly, that in cases of terminal cancer "there will be many patients we can’t help. but none we can’t harm".
As a result of tweeting about the bill, I got a call on 21 October from RT TV (UK) asking me speak on the Saatchi Bill. That was a pleasure. Here it is (done from home, on Skype).
And on the day of the Lords debate (24 October 2014) I was asked to talk again about the Saatchi Bill, on the flagship Radio 4 morning news programme, Today. It was only a 3 minute interview with Mishal Husain, but it was unopposed so I managed to get in the main points.
Click to play interview
This is the Today Programme studio.
The BBC doesn’t pay you, but it’s good at driving you to/from the studio, though the cars are unnecessarily posh. On the way home, I had my first ever ride in an all-electric Tesla. It does 0 – 60 in 4.2 seconds but costs £50k -more than twice the price of a Prius.
|
|
Follow-up
25 October 2014 A summary of the misunderstandings in yesterday’s House of Lords debate: What the Lords Missed.
This discussion seemed to be of sufficient general interest that we submitted is as a feature to eLife, because this journal is one of the best steps into the future of scientific publishing. Sadly the features editor thought that " too much of the article is taken up with detailed criticisms of research papers from NEJM and Science that appeared in the altmetrics top 100 for 2013; while many of these criticisms seems valid, the Features section of eLife is not the venue where they should be published". That’s pretty typical of what most journals would say. It is that sort of attitude that stifles criticism, and that is part of the problem. We should be encouraging post-publication peer review, not suppressing it. Luckily, thanks to the web, we are now much less constrained by journal editors than we used to be.
Here it is.
Scientists don’t count: why you should ignore altmetrics and other bibliometric nightmares
David Colquhoun1 and Andrew Plested2
1 University College London, Gower Street, London WC1E 6BT
2 Leibniz-Institut für Molekulare Pharmakologie (FMP) & Cluster of Excellence NeuroCure, Charité Universitätsmedizin,Timoféeff-Ressowsky-Haus, Robert-Rössle-Str. 10, 13125 Berlin Germany.
Jeffrey Beall is librarian at Auraria Library, University of Colorado Denver. Although not a scientist himself, he, more than anyone, has done science a great service by listing the predatory journals that have sprung up in the wake of pressure for open access. In August 2012 he published “Article-Level Metrics: An Ill-Conceived and Meretricious Idea. At first reading that criticism seemed a bit strong. On mature consideration, it understates the potential that bibliometrics, altmetrics especially, have to undermine both science and scientists.
Altmetrics is the latest buzzword in the vocabulary of bibliometricians. It attempts to measure the “impact” of a piece of research by counting the number of times that it’s mentioned in tweets, Facebook pages, blogs, YouTube and news media. That sounds childish, and it is. Twitter is an excellent tool for journalism. It’s good for debunking bad science, and for spreading links, but too brief for serious discussions. It’s rarely useful for real science.
Surveys suggest that the great majority of scientists do not use twitter (7 — 13%). Scientific works get tweeted about mostly because they have titles that contain buzzwords, not because they represent great science.
What and who is Altmetrics for?
The aims of altmetrics are ambiguous to the point of dishonesty; they depend on whether the salesperson is talking to a scientist or to a potential buyer of their wares.
At a meeting in London , an employee of altmetric.com said “we measure online attention surrounding journal articles” “we are not measuring quality …” “this whole altmetrics data service was born as a service for publishers”, “it doesn’t matter if you got 1000 tweets . . .all you need is one blog post that indicates that someone got some value from that paper”.
These ideas sound fairly harmless, but in stark contrast, Jason Priem (an author of the altmetrics manifesto) said one advantage of altmetrics is that it’s fast “Speed: months or weeks, not years: faster evaluations for tenure/hiring”. Although conceivably useful for disseminating preliminary results, such speed isn’t important for serious science (the kind that ought to be considered for tenure) which operates on the timescale of years. Priem also says “researchers must ask if altmetrics really reflect impact” . Even he doesn’t know, yet altmetrics services are being sold to universities, before any evaluation of their usefulness has been done, and universities are buying them. The idea that altmetrics scores could be used for hiring is nothing short of terrifying.
The problem with bibliometrics
The mistake made by all bibliometricians is that they fail to consider the content of papers, because they have no desire to understand research. Bibliometrics are for people who aren’t prepared to take the time (or lack the mental capacity) to evaluate research by reading about it, or in the case of software or databases, by using them. The use of surrogate outcomes in clinical trials is rightly condemned. Bibliometrics are all about surrogate outcomes.
If instead we consider the work described in particular papers that most people agree to be important (or that everyone agrees to be bad), it’s immediately obvious that no publication metrics can measure quality. There are some examples in How to get good science (Colquhoun, 2007). It is shown there that at least one Nobel prize winner failed dismally to fulfil arbitrary biblometric productivity criteria of the sort imposed in some universities (another example is in Is Queen Mary University of London trying to commit scientific suicide?).
Schekman (2013) has said that science
“is disfigured by inappropriate incentives. The prevailing structures of personal reputation and career advancement mean the biggest rewards often follow the flashiest work, not the best.”
Bibliometrics reinforce those inappropriate incentives. A few examples will show that altmetrics are one of the silliest metrics so far proposed.
The altmetrics top 100 for 2103
The superficiality of altmetrics is demonstrated beautifully by the list of the 100 papers with the highest altmetric scores in 2013 For a start, 58 of the 100 were behind paywalls, and so unlikely to have been read except (perhaps) by academics.
The second most popular paper (with the enormous altmetric score of 2230) was published in the New England Journal of Medicine. The title was Primary Prevention of Cardiovascular Disease with a Mediterranean Diet. It was promoted (inaccurately) by the journal with the following tweet:
Many of the 2092 tweets related to this article simply gave the title, but inevitably the theme appealed to diet faddists, with plenty of tweets like the following:
The interpretations of the paper promoted by these tweets were mostly desperately inaccurate. Diet studies are anyway notoriously unreliable. As John Ioannidis has said
"Almost every single nutrient imaginable has peer reviewed publications associating it with almost any outcome."
This sad situation comes about partly because most of the data comes from non-randomised cohort studies that tell you nothing about causality, and also because the effects of diet on health seem to be quite small.
The study in question was a randomized controlled trial, so it should be free of the problems of cohort studies. But very few tweeters showed any sign of having read the paper. When you read it you find that the story isn’t so simple. Many of the problems are pointed out in the online comments that follow the paper. Post-publication peer review really can work, but you have to read the paper. The conclusions are pretty conclusively demolished in the comments, such as:
“I’m surrounded by olive groves here in Australia and love the hand-pressed EVOO [extra virgin olive oil], which I can buy at a local produce market BUT this study shows that I won’t live a minute longer, and it won’t prevent a heart attack.”
We found no tweets that mentioned the finding from the paper that the diets had no detectable effect on myocardial infarction, death from cardiovascular causes, or death from any cause. The only difference was in the number of people who had strokes, and that showed a very unimpressive P = 0.04.
Neither did we see any tweets that mentioned the truly impressive list of conflicts of interest of the authors, which ran to an astonishing 419 words.
“Dr. Estruch reports serving on the board of and receiving lecture fees from the Research Foundation on Wine and Nutrition (FIVIN); serving on the boards of the Beer and Health Foundation and the European Foundation for Alcohol Research (ERAB); receiving lecture fees from Cerveceros de España and Sanofi-Aventis; and receiving grant support through his institution from Novartis. Dr. Ros reports serving on the board of and receiving travel support, as well as grant support through his institution, from the California Walnut Commission; serving on the board of the Flora Foundation (Unilever). . . “
And so on, for another 328 words.
The interesting question is how such a paper came to be published in the hugely prestigious New England Journal of Medicine. That it happened is yet another reason to distrust impact factors. It seems to be another sign that glamour journals are more concerned with trendiness than quality.
One sign of that is the fact that the journal’s own tweet misrepresented the work. The irresponsible spin in this initial tweet from the journal started the ball rolling, and after this point, the content of the paper itself became irrelevant. The altmetrics score is utterly disconnected from the science reported in the paper: it more closely reflects wishful thinking and confirmation bias.
The fourth paper in the altmetrics top 100 is an equally instructive example.
|
This work was also published in a glamour journal, Science. The paper claimed that a function of sleep was to “clear metabolic waste from the brain”. It was initially promoted (inaccurately) on Twitter by the publisher of Science. After that, the paper was retweeted many times, presumably because everybody sleeps, and perhaps because the title hinted at the trendy, but fraudulent, idea of “detox”. Many tweets were variants of “The garbage truck that clears metabolic waste from the brain works best when you’re asleep”. |
|
But this paper was hidden behind Science’s paywall. It’s bordering on irresponsible for journals to promote on social media papers that can’t be read freely. It’s unlikely that anyone outside academia had read it, and therefore few of the tweeters had any idea of the actual content, or the way the research was done. Nevertheless it got “1,479 tweets from 1,355 accounts with an upper bound of 1,110,974 combined followers”. It had the huge Altmetrics score of 1848, the highest altmetric score in October 2013.
Within a couple of days, the story fell out of the news cycle. It was not a bad paper, but neither was it a huge breakthrough. It didn’t show that naturally-produced metabolites were cleared more quickly, just that injected substances were cleared faster when the mice were asleep or anaesthetised. This finding might or might not have physiological consequences for mice.
Worse, the paper also claimed that “Administration of adrenergic antagonists induced an increase in CSF tracer influx, resulting in rates of CSF tracer influx that were more comparable with influx observed during sleep or anesthesia than in the awake state”. Simply put, giving the sleeping mice a drug could reduce the clearance to wakeful levels. But nobody seemed to notice the absurd concentrations of antagonists that were used in these experiments: “adrenergic receptor antagonists (prazosin, atipamezole, and propranolol, each 2 mM) were then slowly infused via the cisterna magna cannula for 15 min”. Use of such high concentrations is asking for non-specific effects. The binding constant (concentration to occupy half the receptors) for prazosin is less than 1 nM, so infusing 2 mM is working at a million times greater than the concentration that should be effective. That’s asking for non-specific effects. Most drugs at this sort of concentration have local anaesthetic effects, so perhaps it isn’t surprising that the effects resembled those of ketamine.
The altmetrics editor hadn’t noticed the problems and none of them featured in the online buzz. That’s partly because to find it out you had to read the paper (the antagonist concentrations were hidden in the legend of Figure 4), and partly because you needed to know the binding constant for prazosin to see this warning sign.
The lesson, as usual, is that if you want to know about the quality of a paper, you have to read it. Commenting on a paper without knowing anything of its content is liable to make you look like an jackass.
A tale of two papers
Another approach that looks at individual papers is to compare some of one’s own papers. Sadly, UCL shows altmetric scores on each of your own papers. Mostly they are question marks, because nothing published before 2011 is scored. But two recent papers make an interesting contrast. One is from DC’s side interest in quackery, one was real science. The former has an altmetric score of 169, the latter has an altmetric score of 2.
|
The first paper was “Acupuncture is a theatrical placebo”, which was published as an invited editorial in Anesthesia and Analgesia [download pdf]. The paper was scientifically trivial. It took perhaps a week to write. Nevertheless, it got promoted it on twitter, because anything to do with alternative medicine is interesting to the public. It got quite a lot of retweets. And the resulting altmetric score of 169 put it in the top 1% of all articles altmetric have tracked, and the second highest ever for Anesthesia and Analgesia. As well as the journal’s own website, the article was also posted on the DCScience.net blog (May 30, 2013) where it soon became the most viewed page ever (24,468 views as of 23 November 2013), something that altmetrics does not seem to take into account. |
|
Compare this with the fate of some real, but rather technical, science.
|
My [DC] best scientific papers are too old (i.e. before 2011) to have an altmetrics score, but my best score for any scientific paper is 2. This score was for Colquhoun & Lape (2012) “Allosteric coupling in ligand-gated ion channels”. It was a commentary with some original material. The altmetric score was based on two tweets and 15 readers on Mendeley. The two tweets consisted of one from me (“Real science; The meaning of allosteric conformation changes http://t.co/zZeNtLdU ”). The only other tweet as abusive one from a cyberstalker who was upset at having been refused a job years ago. Incredibly, this modest achievement got it rated “Good compared to other articles of the same age (71st percentile)”. |
|
Conclusions about bibliometrics
Bibliometricians spend much time correlating one surrogate outcome with another, from which they learn little. What they don’t do is take the time to examine individual papers. Doing that makes it obvious that most metrics, and especially altmetrics, are indeed an ill-conceived and meretricious idea. Universities should know better than to subscribe to them.
Although altmetrics may be the silliest bibliometric idea yet, much this criticism applies equally to all such metrics. Even the most plausible metric, counting citations, is easily shown to be nonsense by simply considering individual papers. All you have to do is choose some papers that are universally agreed to be good, and some that are bad, and see how metrics fail to distinguish between them. This is something that bibliometricians fail to do (perhaps because they don’t know enough science to tell which is which). Some examples are given by Colquhoun (2007) (more complete version at dcscience.net).
Eugene Garfield, who started the metrics mania with the journal impact factor (JIF), was clear that it was not suitable as a measure of the worth of individuals. He has been ignored and the JIF has come to dominate the lives of researchers, despite decades of evidence of the harm it does (e.g.Seglen (1997) and Colquhoun (2003) ) In the wake of JIF, young, bright people have been encouraged to develop yet more spurious metrics (of which ‘altmetrics’ is the latest). It doesn’t matter much whether these metrics are based on nonsense (like counting hashtags) or rely on counting links or comments on a journal website. They won’t (and can’t) indicate what is important about a piece of research- its quality.
People say – I can’t be a polymath. Well, then don’t try to be. You don’t have to have an opinion on things that you don’t understand. The number of people who really do have to have an overview, of the kind that altmetrics might purport to give, those who have to make funding decisions about work that they are not intimately familiar with, is quite small. Chances are, you are not one of them. We review plenty of papers and grants. But it’s not credible to accept assignments outside of your field, and then rely on metrics to assess the quality of the scientific work or the proposal.
It’s perfectly reasonable to give credit for all forms of research outputs, not only papers. That doesn’t need metrics. It’s nonsense to suggest that altmetrics are needed because research outputs are not already valued in grant and job applications. If you write a grant for almost any agency, you can put your CV. If you have a non-publication based output, you can always include it. Metrics are not needed. If you write software, get the numbers of downloads. Software normally garners citations anyway if it’s of any use to the greater community.
When AP recently wrote a criticism of Heather Piwowar’s altmetrics note in Nature, one correspondent wrote: "I haven’t read the piece [by HP] but I’m sure you are mischaracterising it". This attitude summarizes the too-long-didn’t-read (TLDR) culture that is increasingly becoming accepted amongst scientists, and which the comparisons above show is a central component of altmetrics.
Altmetrics are numbers generated by people who don’t understand research, for people who don’t understand research. People who read papers and understand research just don’t need them and should shun them.
But all bibliometrics give cause for concern, beyond their lack of utility. They do active harm to science. They encourage “gaming” (a euphemism for cheating). They encourage short-term eye-catching research of questionable quality and reproducibility. They encourage guest authorships: that is, they encourage people to claim credit for work which isn’t theirs. At worst, they encourage fraud.
No doubt metrics have played some part in the crisis of irreproducibility that has engulfed some fields, particularly experimental psychology, genomics and cancer research. Underpowered studies with a high false-positive rate may get you promoted, but tend to mislead both other scientists and the public (who in general pay for the work). The waste of public money that must result from following up badly done work that can’t be reproduced but that was published for the sake of “getting something out” has not been quantified, but must be considered to the detriment of bibliometrics, and sadly overcomes any advantages from rapid dissemination. Yet universities continue to pay publishers to provide these measures, which do nothing but harm. And the general public has noticed.
It’s now eight years since the New York Times brought to the attention of the public that some scientists engage in puffery, cheating and even fraud.
Overblown press releases written by journals, with connivance of university PR wonks and with the connivance of the authors, sometimes go viral on social media (and so score well on altmetrics). Yet another example, from Journal of the American Medical Association involved an overblown press release from the Journal about a trial that allegedly showed a benefit of high doses of Vitamin E for Alzheimer’s disease.
This sort of puffery harms patients and harms science itself.
We can’t go on like this.
What should be done?
Post publication peer review is now happening, in comments on published papers and through sites like PubPeer, where it is already clear that anonymous peer review can work really well. New journals like eLife have open comments after each paper, though authors do not seem to have yet got into the habit of using them constructively. They will.
It’s very obvious that too many papers are being published, and that anything, however bad, can be published in a journal that claims to be peer reviewed . To a large extent this is just another example of the harm done to science by metrics –the publish or perish culture.
Attempts to regulate science by setting “productivity targets” is doomed to do as much harm to science as it has in the National Health Service in the UK. This has been known to economists for a long time, under the name of Goodhart’s law.
Here are some ideas about how we could restore the confidence of both scientists and of the public in the integrity of published work.
- Nature, Science, and other vanity journals should become news magazines only. Their glamour value distorts science and encourages dishonesty.
- Print journals are overpriced and outdated. They are no longer needed. Publishing on the web is cheap, and it allows open access and post-publication peer review. Every paper should be followed by an open comments section, with anonymity allowed. The old publishers should go the same way as the handloom weavers. Their time has passed.
- Web publication allows proper explanation of methods, without the page, word and figure limits that distort papers in vanity journals. This would also make it very easy to publish negative work, thus reducing publication bias, a major problem (not least for clinical trials)
- Publish or perish has proved counterproductive. It seems just as likely that better science will result without any performance management at all. All that’s needed is peer review of grant applications.
- Providing more small grants rather than fewer big ones should help to reduce the pressure to publish which distorts the literature. The ‘celebrity scientist’, running a huge group funded by giant grants has not worked well. It’s led to poor mentoring, and, at worst, fraud. Of course huge groups sometimes produce good work, but too often at the price of exploitation of junior scientists
- There is a good case for limiting the number of original papers that an individual can publish per year, and/or total funding. Fewer but more complete and considered papers would benefit everyone, and counteract the flood of literature that has led to superficiality.
- Everyone should read, learn and inwardly digest Peter Lawrence’s The Mismeasurement of Science.
A focus on speed and brevity (cited as major advantages of altmetrics) will help no-one in the end. And a focus on creating and curating new metrics will simply skew science in yet another unsatisfactory way, and rob scientists of the time they need to do their real job: generate new knowledge.
It has been said
“Creation is sloppy; discovery is messy; exploration is dangerous. What’s a manager to do?
The answer in general is to encourage curiosity and accept failure. Lots of failure.”
And, one might add, forget metrics. All of them.
Follow-up
17 Jan 2014
This piece was noticed by the Economist. Their ‘Writing worth reading‘ section said
"Why you should ignore altmetrics (David Colquhoun) Altmetrics attempt to rank scientific papers by their popularity on social media. David Colquohoun [sic] argues that they are “for people who aren’t prepared to take the time (or lack the mental capacity) to evaluate research by reading about it.”"
20 January 2014.
Jason Priem, of ImpactStory, has responded to this article on his own blog. In Altmetrics: A Bibliographic Nightmare? he seems to back off a lot from his earlier claim (cited above) that altmetrics are useful for making decisions about hiring or tenure. Our response is on his blog.
20 January 2014.
Jason Priem, of ImpactStory, has responded to this article on his own blog, In Altmetrics: A bibliographic Nightmare? he seems to back off a lot from his earlier claim (cited above) that altmetrics are useful for making decisions about hiring or tenure. Our response is on his blog.
23 January 2014
The Scholarly Kitchen blog carried another paean to metrics, A vigorous discussion followed. The general line that I’ve followed in this discussion, and those mentioned below, is that bibliometricians won’t qualify as scientists until they test their methods, i.e. show that they predict something useful. In order to do that, they’ll have to consider individual papers (as we do above). At present, articles by bibliometricians consist largely of hubris, with little emphasis on the potential to cause corruption. They remind me of articles by homeopaths: their aim is to sell a product (sometimes for cash, but mainly to promote the authors’ usefulness).
It’s noticeable that all of the pro-metrics articles cited here have been written by bibliometricians. None have been written by scientists.
28 January 2014.
Dalmeet Singh Chawla,a bibliometrician from Imperial College London, wrote a blog on the topic. (Imperial, at least in its Medicine department, is notorious for abuse of metrics.)
29 January 2014 Arran Frood wrote a sensible article about the metrics row in Euroscientist.
2 February 2014 Paul Groth (a co-author of the Altmetrics Manifesto) posted more hubristic stuff about altmetrics on Slideshare. A vigorous discussion followed.
5 May 2014. Another vigorous discussion on ImpactStory blog, this time with Stacy Konkiel. She’s another non-scientist trying to tell scientists what to do. The evidence that she produced for the usefulness of altmetrics seemed pathetic to me.
7 May 2014 A much-shortened version of this post appeared in the British Medical Journal (BMJ blogs)
The consistent failure of ‘regulators’ to do their job has been a constant theme on this blog. There is a synopsis of dozens of them at Regulation of alternative medicine: why it doesn’t work, and never can. And it isn’t only quackery where this happens. The ineptitude (and extravagance) of the Quality Assurance Agency (QAA) was revealed starkly when the University of Wales’ accreditation of external degrees was revealed (by me and by BBC TV Wales, not by the QAA) to be so bad that the University had to shut down.
Here is another example that you couldn’t make up.
 |
 |
Yes, the Professional Standards Authority (PSA) has agreed to accredit that bad-joke pseudo-regulator, the Complementary & Natural Healthcare Council (CNHC, more commonly known as Ofquack)
Ofquack was created at the instigation of HRH the Prince of Wales, at public expense, as a means of protecting the delusional beliefs of quacks from criticism. I worked for them for a while, and know from the inside that their regulation is a bad joke.
When complaints were made about untrue claims made by ‘reflexologists’, the complaints were upheld but they didn’t even reach the Conduct and Competence committee, on the grounds that the reflexologists really believed the falsehoods that they’d been taught. Therefore, by the Humpty Dumpty logic of the CNHC, their fitness to practise was not affected by their untrue claims. You can read the account of this bizarre incident by the person who submitted the complaints, Simon Perry.
In fact in the whole history of the CNHC, it has received a large number of complaints, but only one has ever been considered by their Conduct and Competence Committee. The rest have been dismissed before they were considered properly. That alone makes their claim to be a regulator seem ridiculous.
The CNHC did tell its registrants to stop making unjustified claims, but it has been utterly ineffective in enforcing that ruling. In May 2013, another 100 complaints were submitted and no doubt they will be brushed aside too: see Endemic problems with CNHC registrants..
As I said at the time
It will be fascinating to see how the CNHC tries to escape from the grave that it has dug for itself.
If the CNHC implements properly its own code of conduct, few people will sign up and the CNHC will die. If it fails to implement its own code of conduct it would be shown to be a dishonest sham.
In February of this year (2013), I visited the PSA with colleagues from the Nightingale Collaboration. We were received cordially enough, but they seemed to be bureaucrats with no real understanding of science. We tried to explain to them the fundamental dilemma of the regulation of quacks, namely that no amount of training will help when the training teaches things that aren’t true. They were made aware of all of the problems described above. But despite that, they ended up endorsing the CNHC.
How on earth did the PSA manage to approve an obviously ineffective ‘regulator’?
The job of the PSA is said to be “. . . protecting users of health and social care services and the public”. They (or at least their predecessor, the CHRE), certainly didn’t do that during the saga of the General Chiropractic Council.
The betrayal of reason is catalogued in a PSA document [get local copy]. Here is some nerdy detail.
It is too tedious to go through the whole document, so I’ll deal with only two of its many obvious flaws, the sections that deal with the evidence base, and with training.
The criteria for accreditation state
Standard 6: the organisation demonstrates that there is a defined knowledge base underpinning the health and social care occupations covered by its register or, alternatively, how it is actively developing one. The organisation makes the defined knowledge base or its development explicit to the public.
The Professional Standards Authority recognises that not all disciplines are underpinned by evidence of proven therapeutic value. Some disciplines are subject to controlled randomized trials, others are based on qualitative evidence. Some rely on anecdotes. Nevertheless, these disciplines are legal and the public choose to use them. The Authority requires organisations to make the knowledge base/its development clear to the public so that they may make informed decisions.
Since all 15 occupations that are “regulated” by the CNHC fall into the last category. they “rely on anecdotes”, you would imagine the fact that “The Authority requires organisations to make the knowledge base/its development clear to the public” would mean that the CNHC was required to make a clear statement that reiki, reflexology etc are based solely on anecdote. Of course the CNHC does no such thing. For example, the CNHC’s official definition of reflexology says
Reflexology is a complementary therapy based on the belief that there are reflex areas in the feet and hands which are believed to correspond to all organs and parts of the body
There is, of course, not the slightest reason to think such connections exist, but the CNHC gives no hint whatsoever of that inconvenient fact. The word “anecdote” is used by the PSA but occurs nowhere on the CNHC’s web site.
It is very clear that the CNHC fails standard 6.
But the PSA managed to summon up the following weasel words to get around this glaring failure:
“The professional associations (that verify eligibility for CNHC registration) were actively involved in defining the knowledge base for each of the 15 professions. The Panel further noted that Skills for Health has lead responsibility for writing and reviewing the National Occupational Standards (NOS) for the occupations CNHC registers and that all NOS have to meet the quality criteria set by the UK Commission for Employment and Skills (UKCES), who are responsible for the approval of all NOS across all industry sectors. The Panel considered evidence provided and noted that the applicant demonstrated that there is a defined knowledge base underpinning the occupations covered by its registers. The knowledge base was explicit to the public”.
The PSA, rather than engaging their own brains, simply defer to two other joke organisations, Skills for Health and National Occupational Standards. But it is quite obvious that for things like reiki, reflexology and craniosacral therapy, the “knowledge base” consists entirely of made-up nonsense. Any fool can see that (but not, it seems, the PSA).
Skills for Health lists made-up, HR style, “competencies” for everything under the sun. When I got them to admit that their efforts on distance-healing etc had been drafted by the Prince of Wales’ Foundation, the conversation with Skills for Health became surreal (recorded in January 2008)
DC. Well yes the Prince of Wales would like that. His views on medicine are well known, and they are nothing if not bizarre. Haha are you going to have competencies in talking to trees perhaps?
“You’d have to talk to LANTRA, the land-based organisation for that.”
DC. I’m sorry, I have to talk to whom?
“LANTRA which is the sector council for the land-based industries uh, sector, not with us sorry . . . areas such as horticulture etc.”
DC. We are talking about medicine aren’t we? Not horticulture.
“You just gave me an example of talking to trees, that’s outside our remit ”
You couldn’t make it up, but it’s true. And the Professional Standards Authority rely on what these jokers say.
The current Skills for Health entry for reflexology says
“Reflexology is the study and practice of treating reflex points and areas in the feet and hands that relate to corresponding parts of the body. Using precise hand and finger techniques a reflexologist can improve circulation, induce relaxation and enable homeostasis. These three outcomes can activate the body’s own healing systems to heal and prevent ill health.”
This is crass, made-up nonsense. Of course there are no connections between “areas in the feet and hands that relate to corresponding parts of the body” and no reason to think that reflexology is anything more than foot massage. That a very expensive body, paid for by you and me, can propagate such preposterous nonsense is worrying. That the PSA should rely on them is even more worrying.
National Occupational Standards is yet another organisation that is utterly dimwitted about medical matters, but if you look up reflexology you are simply referred to Skills for Health, as above.
UK Commission for Employment and Skills
(UKCES) is a new one on me. The PSA says that “the UK Commission for Employment and Skills (UKCES), who are responsible for the approval of all NOS across all industry sectors” It is only too obvious that the UKCES leadership team have failed utterly to do their job when it comes to made-up medicine. None of them know much about medicine. It’s true that their chairman did once work for SmithKline Beecham, but as a marketer of Lucozade, a job which anyone with much knowledge of science would not find comfortable..
You don’t need to know much medicine to spot junk. I see no excuse for their failure.
The training problem.
The PSA’s criteria for accreditation say
Standard 9: education and training
The organisation:
9a) Sets appropriate educational standards that enable its registrants to practise competently the occupation(s) covered by its register. In setting its standards the organisation takes account of the following factors:
- The nature and extent of risk to service users and the public
- The nature and extent of knowledge, skill and experience required to provide
service users and the public with good quality careand later
9b) Ensures that registrants who assess the health needs of service users and provide any form of care and treatment are equipped to:
- Recognise and interpret clinical signs of impairment
- Recognise where a presenting problem may mask underlying pathologies
- Have sufficient knowledge of human disease and social determinants of health to identify where service users may require referral to another health or social care professional.
Anyone who imagines for a moment that a reflexologist or a craniosacral therapist is competent to diagnose a subarachnoid haemorrhage or malaria must need their head examining. In any case, the CNHC has already admitted that their registrants are taught things that aren’t true, so more training presumably means more inculcation of myths.
So how does the PSA wriggle out of this? Their response started
“The Panel noted that practitioners must meet, as a minimum, the National Occupational Standards for safe and competent practice. This is verified by the professional associations, who have in turn provided written undertakings to CNHC affirming that there are processes in place to verify the training and skills outcomes of their members to the NOS”
Just two problems there. The NOS standards themselves are utterly delusional. And checking them is left to the quacks themselves. To be fair, the PSA weren’t quite happy with this, but after an exchange of letters, minor changes enabled the boxes to be ticked and the PSA said “The Panel was now satisfied from the evidence provided, that this Standard had been
met”.
What’s wrong with regulators?
This saga is typical of many other cases of regulators doing more harm than good. Regulators are sometimes quacks themselves, in which case one isn’t surprised at their failure to regulate.
But organisations like the Professional Standards Authority and Skills for Health are not (mostly) quacks themselves. So how do they end up giving credence to nonsense? I find that very hard to comprehend, but here are a few ideas.
(1) They have little scientific education and are not really capable of critical thought
(2) Perhaps even more important, they lack all curiosity. It isn’t very hard to dig under the carapace of quack organisations, but rather than finding out for themselves, the bureaucrats of the PSA are satisfied by reassuring letters that allow them to tick their boxes and get home.
(3) A third intriguing possibility is that people like the PSA yield to political pressure. The Department of Health is deeply unscientific and clearly has no idea what to do about alternative medicine. They have still done nothing at all about herbal medicine, traditional Chinese medicine or homeopathy, after many years of wavering. My guess is that they see the CNHC as an organisation that gives the appearance that they’ve done something about reiki etc. I wonder whether they applied pressure to the PSA to accredit CNHC, despite it clearly breaking their own rules. I have sent a request under the Freedom of Information Act in an attempt to discover if the Department of Health has misbehaved in the way it did when it attempted to override NHS Choices.
The responsibility for this cock-up has to rest squarely on the shoulders of the PSA’s director, Harry Cayton. He was director of the CHRE from which PSA evolved and is the person who so signally failed to do anything about the General Chiropractic Council fiasco,
What can be done?
This is just the latest of many examples of regulators who not only fail to help but actually do harm by giving their stamp of approval to mickey mouse organisations like the CNHC. Most of the worst quangos survived the “bonfire of the quangos”.. The bonfire should have started with the PSA, CNHC and Skills for Health. They cost a lot and do harm.
There is a much simpler answer. There is a good legal case that much of alternative medicine is illegal. All one has to do is to enforce the existing law. Nobody would object to quacks if they stopped making false claims (though whether they could stay in business if they stopped exaggerating is debatable). There is only one organisation that has done a good job when it comes to truthfulness. That is the Advertising Standards Authority. But the ASA can do nothing apart from telling people to change the wording of their advertisements, and even that is often ignored.
The responsibility for enforcing the Consumer Protection Law is Trading Standards. They have consistently failed to do their job (see Medico-Legal Journal, Rose et al., 2012. “Spurious Claims for Health-care Products“.
If they did their job of prosecuting people who defraud the public with false claims, the problem would be solved.
But they don’t, and it isn’t.
Follow-up
The indefatigable Quackometer has wriiten an excellent account of the PSA fiasco
Despite the First Amendment in the US and a new Defamation Act in the UK, fear of legal threats continue to suppress the expression of honest scientific opinion.
I was asked by Nature Medicine (which is published in the USA) to write a review of Paul Offit’s new book. He’s something of a hero, so of course I agreed. The editor asked me to make some changes to the first draft, which I did. Then the editor concerned sent me this letter.
|
Thank you for the revised version of the book review.
The chief editor of the journal took a look at your piece, and he thought that it would be a good idea to run it past our legal counsel owing to the strong opinions expressed in the piece in relation to specific individuals. I regret to say that the lawyers have advised us against publishing the review. |
After that I tried the UK Conversation. They had done a pretty good job with my post on the baleful influence of royals on medicine. They were more helpful then Nature Medicine, but for some reason that I can’t begin to understand, they insisted that I should not name Nature Medicine, but to refer only to "a leading journal". And they wanted me not to name Harvard in the last paragraph. I’m still baffled about why. But it seemed to me that editorial interference had gone too far, so rather than have an editor re-write my review, I withdrew it.
It is precisely this sort of timidity that allows purveyors of quackery such success with their bait and switch tactics. The fact that people seem so terrified to be frank must be part of the reason why Harvard, Yale and the rest have shrugged their shoulders and allowed nonsense medicine to penetrate so deeply into their medical schools. It’s also why blogs now are often better sources of information than established journals.
Here is the review. I see nothing defamatory in it.
Do You Believe in Magic? The Sense and Nonsense of Alternative Medicine
|
Paul A. Offit Reviewed by David Colquhoun Research Professor of Pharmacology, UCL. |
|
Here’s an odd thing. There is a group of people who advocate the silly idea that you can cure all ills by stuffing yourself with expensive pills, made by large and unscrupulous Pharma companies. No, I’m not talking about pharmacologists or doctors or dietitians. They mostly say that stuffing yourself with pills is often useless and sometimes harmful, because that’s what the evidence says .
Rather, the pill pushers are the true believers in the alternative realities of the “supplement” industry. They seem blithely unaware that the manufacturers are mostly the same big pharma companies that they blame for trying to suppress “natural remedies”. Far from trying to suppress them, pharma companies love the supplement industry because little research is needed and there are few restrictions on the claims that can be made.
Paul Offit’s excellent book concentrates on alternative medicine in the USA, with little mention of the rest of the world. He describes how American pork barrel politics have given supplement hucksters an almost unrestricted right to make stuff up.
Following the thalidomide tragedy, which led to birth defects in babies in the 1950s and 60s, many countries passed laws that required evidence that a drug was both effective and safe before it could be sold. This was mandate by the Kefauver-Harris amendment (1961) in the USA and the Medicines Act (1968) in the UK. Laws like that upset the quacks, and in the UK the quacks got a free pass, a ‘licence of right‘, largely still in existence.
|
In order to sell a herbal concoction in the UK you need to present no evidence at all that it works, just evidence of safety, in return for which you get a reassuring certification mark and freedom to use misleading brand names and labels. |
 Tradional herbal mark |
In the USA the restrictions didn’t last long. Offit describes how a lobby group for vitamin sellers, the National Health Federation, had a board made up of quacks, some of whom, according to Offit (page 73) had convictions. They found an ally in Senator William Proxmire who introduced in 1975 an amendment that banned the Food and Drugs Administration (FDA) from regulating the safety of megavitamins. Tragically, this bill was even supported by the previously-respected scientist Linus Pauling. Offit tells us that “to Proxmire” became a verb meaning to obstruct science for political gain.
The author then relates how the situation got worse with the passage of the Dietary Supplement Health and Education Act (DSHEA) in 1994. It was passed with the help of ex-vitamin salesman Senator Orin Hatch and lots of money from the supplement industry.
This act iniquitously defined a “supplement” as “a product intended to supplement the diet that bears or contains one or more of the following ingredients: a vitamin, a mineral, an herb or other botanical, or an amino acid”. At a stroke, herbs were redefined as foods. There was no need to submit any evidence of either efficacy or even of safety, before marketing anything. All a manufacturer had to do to sell almost any herbal drug or megadose vitamin was to describe it as a “dietary supplement”. The lobbying to get this law through was based on appealing to the Tea Party tendency –get the government’s hands off our vitamins. And it was helped by ‘celebrities’ such as Sissy Spacek and Mel Gibson (it’s impossible to tell whether they really believed in the magic of vitamins, or whether they were paid, or had Tea Party sympathies).
Offit’s discussion of vaccination is a heartbreaking story of venom and misinformation. As co-inventor of the first rotavirus vaccine he’s responsible for saving many lives around the world. But he, perhaps more than anyone, suffered from the autism myth started by the falsified work of Andrew Wakefield.
The scientific community took the question seriously and soon many studies showed absolutely no link between vaccination and autism. But evidence did not seem to interest the alternative world. Rather than Offit being lauded as a saver of children’s lives, he describes how he was subjected to death threats and resorted to having armed guards at meetings.
Again, Offit tells us how celebrities were able to sway public opinion For example (chapter 6), the actress Jenny McCarthy and talk-show hostess Oprah Winfrey promoted, only too successfully, the vaccine-autism link despite abundant evidence that it didn’t exist, and promoted a number of theories that were not supported by any evidence, such as the idea that autism can be “cured” by mega-doses of vitamins and supplements.
Of course vaccines like the one for rotavirus can’t be developed without pharmaceutical companies because, as Offit says, only they "have the resources and expertise to make a vaccine. We can’t make it in our garage". When the Children’s Hospital of Philadelphia sold its royalty stake in the rotavirus vaccine for $182 million, Offit received an undisclosed share of the intellectual property, “in the millions ”.
That’s exactly what universities love. We are encouraged constantly to collaborate with industry, and, in the process, make money for the university. It’s also what Wakefield, and the Royal Free Hospital where he worked, hoped to do. But sadly, these events led to Offit being called names such as “Dr Proffit” and “Biostitute” (to rhyme with “prostitute”) by people like Jenny McCarthy and Robert F. Kennedy Jr. The conspiritorialist public lapped up this abuse, but appeared not to notice that many quacks have become far richer by peddling cures that do not work.
One lesson from this sad story is that we need to think more about the potential for money to lead to good science being disbelieved, and sometimes to corrupt science.
Everyone should buy this book, and weep for the gullibility and corruption that it describes.
I recommend it especially to the deans of US Medical schools, from Harvard downwards, who have embraced “integrative medicine” departments. In doing so they have betrayed both science and their patients.
Abraham Flexner, whose 1910 report first put US medicine on a sound scientific footing, must be turning in his grave.
|
|
Follow-up
30 August 2013
Quack lobby groups got a clause inserted into Obamacare that will make any attempt to evaluate whether a treatment actually works will leave insurance companies open to legal action for "discrimination".
"Discrimination? Yes! We must not allow the government to exclude health care providers just because those providers don’t cure anything."
The latest piece of well-organised corporate corruption by well-funded lobbyists is revealed by Steven Salzberg, in Forbes Magazine. The chaos in the US health system makes one even more grateful for the NHS and for the evaluation of effectiveness of treatments by NICE.
A constant theme of this blog is that the NHS should not pay for useless treatments. By and large, NICE does a good job of preventing that. But NICE has not been allowed by the Department of Health to look at quackery.
I have the impression that privatisation of many NHS services will lead to an increase in the provision of myth-based therapies. That is part of the "bait and switch" tactic that quacks use in the hope of gaining respectability. A prime example is the "College of Medicine", financed by Capita and replete with quacks, as one would expect since it is the reincarnation of the Prince’s Foundation for Integrated Health.
One such treatment is acupuncture. Having very recently reviewed the evidence, we concluded that "Acupuncture is a theatrical placebo: the end of a myth". Any effects it may have are too small to be useful to patients. That’s the background for an interesting case study.
A colleague got a very painful frozen shoulder. His GP referred him to the Camden & Islington NHS Trust physiotherapy service. That service is now provided by a private company, Connect Physical Health.
That proved to be a big mistake. The first two appointments were not too bad, though they resulted in little improvement. But at the third appointment he was offered acupuncture. He hesitated, but agreed, in desperation to try it. It did no good. At the next appointment the condition was worse. After some very painful manipulation, the physiotherapist offered acupuncture again. This time he refused on the grounds that "I hadn’t noticed any effect the first time, because there is no evidence that it works and that I was concerned by her standards of hygiene". The physiotherapist then became "quite rude" and said that she would put down that the patient had refused treatment.
The lack of response was hardly surprising. NHS Evidence says
"There is no clinical evidence to show that other treatments, such as transcutaneous electrical nerve stimulation (TENS), Shiatsu massage or acupuncture are effective in treating frozen shoulder."
In fact it now seems beyond reasonable doubt that acupuncture is no more than a theatrical placebo.
According to Connect’s own web site “Our services are evidence-based”. That is evidently untrue in this case, so I asked them for the evidence that acupuncture was effective.
I’d noticed that in other places, Connect Physical Health also offers the obviously fraudulent craniosacral therapy (for example, here) and discredited chiropractic quackery. So I asked them about the evidence for their effectiveness too.
This is what they said.
|
Many thanks for your comments via our web site. In response, we thought you might like to access the sources for some of the evidence which underpins our MSK services: Integrating Evidence-Based Acupuncture into Physiotherapy for the Benefit of the Patient – you can obtain the information you require from www.aacp.org.uk The General Chiropractic Council www.gcc-uk.org/page.cfm We have also attached a copy of the NICE Guidelines. |
So, no Cochrane reviews, no NHS Evidence. Instead I was referred to the very quack organisations that promote the treatments in question, the Acupuncture Association of Chartered Physiotherapists, and the totally discredited General Chiropractic Council.
The NICE guidelines that they sent were nothing to do with frozen shoulder. They were the guidelines for low back pain which caused such a furore when they were first issued (and which, in any case, don’t recommend chiropractic explicitly).
When I pointed out these deficiencies I got this.
|
Your email below has been forwarded to me. I am sorry if you feel that that that information we pointed you towards to enable you to make your own investigations into the evidence base for the services provided by Connect Physical Health and your hospital did not meet with your expectations. ‘ ‘ ‘ Please understand that our NHS services in Camden were commissioned by the Primary Care Trust. The fully integrated MSK service model included provision for acupuncture and other manual therapy provided by our experienced Chartered Physiotherapists. If you have any problems with the evidence base for the use of acupuncture or manual therapy within the service, which has been commissioned on behalf of the GPs in Camden Borough, then I would politely recommend that you direct your observations to the clinical commissioning authorities and other professional bodies who do spend time evidencing best practice and representing the academic arguments. I am sure they will be pleased to pick up discussions with you about the relative merits of the interventions being procured by the NHS. Yours sincerely, Mark Mark Philpott BSc BSc MSc MMACP MCSP
Head of Operations, Community MSK Services Connect Physical Health 35 Apex Business Village Cramlington Northumberland NE23 7BF |
So, "don’t blame us, blame the PCT". A second letter asked why they were shirking the little matter of evidence.
|
In response to your last email, I would like to say that Connect does not wish to be drawn into a debate over two therapeutic options (acupuncture and craniosacral therapy) that are widely practiced [sic] within and outside the NHS by very respectable practitioners. You will be as aware, as Connect is, that there are lots of treatments that don’t have a huge evidence base that are practiced in mainstream medicine. Connect has seen many carefully selected patients helped by acupuncture and manual therapy (craniosacral therapy / chiropractic) over many years. Lack of evidence doesn’t mean they don’t work, just that benefit is not proven. Furthermore, nowhere on our website do we state that ALL treatments / services / modalities that Connect offer are ‘Evidence Based’. We do however offer many services that are evidence based, where the evidence exists. We aim to offer ‘choice’ to patients, from a range of services that are safe and delivered by suitably trained professionals and practitioners in line with Codes of Practice and Guidelines from the relevant governing bodies. Connect’s service provision in Camden is meticulously assessed and of a high standard and we are proud of the services provided. |
This response is so wrong, on so many levels, that I gave up on Mr Philpott at this point. At least he admitted implicitly that all of their treatments are not evidence-based. In that case their web site needs to change that claim.
If, by "governing bodies" he means jokes like the GCC or the CNHC then I suppose the behaviour of their employees is not surprising. Mr Philpott is evidently not aware that "craniosacral therapy" has been censured by the Advertising Standards Authority. Well he is now, but evidently doesn’t seem to give a damn.
Next I wrote to the PCT and it took several mails to find out who was responsible for the service. Three mails produced no response so I sent a Freedom of Information Act request. In the end I got some
"Connect PHC provide the Community musculoskeletal service for Camden. The specification for the service specifically asks for the provision of evidence based management and treatments see paragraph on Governance page 14 of attached.. Patients are treated with acupuncture as per the NICE Guidelines (May 2009) for the management of low back pain … . .. Chiropractors are not employed in the service and craniosacral therapy is not provided as part of the service either."
Another letter, pointing out that they were using acupuncture for things other than low back pain got no more information. They did send a copy of the contract with Connect. It makes no mention whatsoever of alternative treatments. It should have done, so part of the responsibility for the failure must lie with the PCT.
The contract does, however, say (page 18)
The service to be led by a lead clinician/manager who can effectively demonstrate ongoing and evidence-based development of clinical guidelines, policies and protocols for effective working practices within the service
In my opinion, Connect Physical Health are in breach of contract
Another example of Connect ignoring evidence
The Connect Physical Health web site has an article about osteoarthritis of the knee
Physiotherapy can be extremely beneficial to help to reduce the symptoms of OA. Treatments such as mobilizations, rehab exercises, acupuncture and taping can help to reduce pain, increase range of movement, increase muscle strength and aid return to functional activities and sports.
There is little enough evidence that physiotherapy does any of these things, but at least it is free of mystical mumbo-jumbo. Although at one time the claim for acupuncture was thought to have some truth, the 2010 Cochrane review concludes otherwise
Sham-controlled trials show statistically significant benefits; however, these benefits are small, do not meet our pre-defined thresholds for clinical relevance, and are probably due at least partially to placebo effects from incomplete blinding.
This conclusion is much the same as has been reached for acupuncture treatments of almost everything. Two major meta-analyses come to similar conclusions. Madsen Gøtzsche & Hróbjartsson (2009) and Vickers et al (2012) both conclude that if there is an effect at all (dubious) then it is too small to be noticeable to the patient. (Be warned that in the case of Vickers et al. you need to read the paper itself because of the spin placed on the results in the abstract.). These papers are discussed in detail in our recent paper.
Why is Connect Physical Health not aware of this?
Their head of operations told me (see above) that
"Connect does not wish to be drawn into a debate [about acupuncture and craniosacral therapy]".
That outlook was confirmed when I left a comment on their osteoarthritis post. This is what it looked like almost a month later.
Guess what? The comment has never appeared..
The attitude of Connect Physical Health to evidence is simply to ignore it if it gets in the way of making money, and to censor any criticism.
What have Camden NHS done about it?
The patient and I both complained to Camden NHS in August 2012. At first, they simply forwarded the complaints to Connect Physical Health with the unsatisfactory results shown above. It took until May 2013 to get any sort of reasonable response. That seems a very long time. In fact by the time the response arrived the PCT had been renamed a Clinical Commissioning Group (CCG) because of the vast top-down reorganisation inposed by Lansley’s Health and Social Care Act.
On 8 May 2013, this response was sent to the patient, Here is part of it.
|
I have received your email of complaint from the NHSNCL complaints department regarding your care. You raise some very clear concerns and I will attempt to address these in order. 1) The fact that you felt pressurised into having acupuncture is a concern as everybody should be given a choice. As part of the informed consent relating to acupuncture you should have been told about the treatment, it’s [sic] benefits and risks and then you sign to confirm you are happy to proceed. I understand that this was the case in your situation but I have reinforced that the consent is important and must be adhered to by the provider Connect Physical Health. There are clear standards of clinical practice that all Chartered Physiotherapists must follow which I will discuss further with the Connect Camden team Manager Nick Downing. I do disagree with you around acupuncture; there is no conclusive evidence for acupuncture in frozen shoulder but I have referenced a systematic review which concludes the studies were too small to draw any conclusions although shoulder function was significantly improved at 4 weeks (Green S et al. Acupuncture for shoulder pain. Cochrane Database Syst Rev 2005; 18: CD005319). There is a growing body of evidence supporting the use of acupuncture and until such time as there is specific evidence against it I don’t think we would be absolutely against the practice of this modality alongside other treatments. .Best wishes Strategy and Planning Directorate |
This response raises more questions than it answers.
For example, what is "informed consent" worth if the therapist is his/herself misinformed about the treatment? It is the eternal dilemma of alternative medicine that it is no use referring to well-trained practitioners, when their training has inculcated myths and untruths.
There is not a "growing body of evidence supporting the use of acupuncture". Precisely the opposite is true.
And the statement "until such time as there is specific evidence against it I don’t think we would be absolutely against the practice of this modality alongside" betrays a basic misunderstanding of the scientific process.
So I sent the writer of this letter a reprint of our paper, "Acupuncture is a theatrical placebo: the end of a myth" (the blog version alone has had over 12,000 page views). A few days later we had an amiable lunch together and we had a constructive discussion about the problems of deciding what should be commissioned and what shouldn’t.
It seems to me to be clear that CCGs should take better advice before boasting that they commission evidence-based treatments.
Postscript
Stories like this are worrying to the majority of physiotherapists who don’t go in for mystical mumbo-jumbo of acupuncture. One of the best is Neil O’Connell who blogs at BodyInMind. He tweeted
Physio fail, sigh RT@david_colquhoun: Yet more #acupuncture. Sold to the NHS by private contractor @ConnectPHC http://t.co/HylkwMCVTh
— Neil O'Connell (@NeilOConnell) June 10, 2013
It isn’t clear how many physiotherapists embrace nonsense, but the Acupuncture Association of Chartered Physiotherapists has around 6000 members, compared with 47,000 chartered physiotherapists (AACP), so it’s a smallish minority. The AACP claims that it is “Integrating Evidence-Based Acupuncture into Physiotherapy”. Like most politicians, the term “evidence-based” is thrown around with gay abandon. Clearly they don’t understand evidence.
Follow-up
12 June 2013
The Advertising Standards Authority has, one again, upheld complaints against the UCLH Trust, for making false claims in its advertising. This time, appropriately, it’s about acupuncture. Just about everything in their advertising leaflets was held to be unjustifiable. They’ve been in trouble before about false claims for homeopathy, hypnosis and craniosacral "therapy".
Of course all of these embarrassments come from one very small corner of the UCLH Trust, the Royal London Hospital for Integrated Medicine (previously known as the Royal London Homeopathic Hospital).
Why is it tolerated in an otherwise excellent NHS Trust? Well, the patron is the Queen herself (not Charles, aka the Quacktitioner Royal), She seems to exert more power behind the scenes than is desirable in In a constitutional monarchy
29 June 2013
I wrote to Dr Gill Gaskin about the latest ASA judgement against RLHIM. She is the person at the UCLH Trust who has responsibility for the quack hospital. She previously refused to do anything about the craniosacral nonsense that is promoted there. This time the ASA seems to have stung them into action at long last. I was told
|
In response to your question about proposed action: All written information for patients relating to the services offered by the Royal London Hospital for Integrated Medicine are being withdrawn for review in the light of the ASA’s rulings (and the patient leaflets have already been withdrawn). It will be reviewed and modified where necessary item by item, and only reintroduced after sign-off through the Queen Square divisional clinical governance processes and the Trust’s patient information leaflet team. With best wishes Gill Gaskin Dr Gill Gaskin |
It remains to be seen whether the re-written information is accurate or not.
The rules for advertising
The Advertising Standards Authority gives advice for advertisers about what’s permitted and what isn’t.
Acupuncture The CAP advice
Craniosacral therapy The CAP advice
Homeopathy The CAP advice and 2013 update
Chiropractic The CAP advice.
Here is a record of a couple of recent newspaper pieces. Who says the mainstream media don’t matter any longer? Blogs may be in the lead now when it comes to critical analysis. The best blogs have more expertise and more time to read the sources than journalists. But the mainstream media get the message to a different, and much larger, audience.
The Observer ran a whole page interview with me as part of their “Rational Heroes” series. I rather liked their subtitle [pdf of article]
“Professor of pharmacology David Colquhoun is the take-no-prisoners debunker of pseudoscience on his unmissable blog”
It was pretty accurate apart from the fact that the picture was labelled as “DC in his office”. Actually it was taken (at the insistence of the photographer) in Lucia Sivilotti’s lab.
Photo by Karen Robinson.
The astonishing result of this was that on Sunday the blog got a record 24,305 hits. Normally it gets 1,000-1,400 hits a day . between posts, fewer on Sunday, and the previous record was around 7000/day
A week later it was still twice normal. It remains to be seen whether the eventual plateau stays up.
I also gained around 1000 extra followers on twitter, though some dropped away quite soon, and 100 or so people signed for email updates. The dead tree media aren’t yet dead. I’m happy to say.
3 June 2013
Perhaps as a result of the foregoing piece, I got asked to write a column for The Observer, at barely 48 hours notice. This is the best I could manage in the time. The web version has links.
This attracted the usual "it worked for me" anecdotes in the comments, but I spent an afternoon answering them. It seems important to have a dialogue, not just to lecture the public. In fact when I read a regular scientific paper, I now find myself looking for the comment section. That may say something about the future of scientific publishing.
It is for others to judge how succesfully I engage with the public, but I was quite surprised to discover that UCL’s public engagement unit, @UCL_in_public, has blocked me on twitter. Hey ho. They have 1574 follower and I have 7597. I wish them the best of luck.
Follow-up
Anesthesia & Analgesia is the official journal of the International Anesthesia Research Society. In 2012 its editor, Steven Shafer, proposed a head-to-head contest between those who believe that acupuncture works and those who don’t. I was asked to write the latter. It has now appeared in June 2013 edition of the journal [download pdf]. The pro-acupuncture article written by Wang, Harris, Lin and Gan appeared in the same issue [download pdf].
Acupuncture is an interesting case, because it seems to have achieved greater credibility than other forms of alternative medicine, despite its basis being just as bizarre as all the others. As a consequence, a lot more research has been done on acupuncture than on any other form of alternative medicine, and some of it has been of quite high quality. The outcome of all this research is that acupuncture has no effects that are big enough to be of noticeable benefit to patients, and it is, in all probablity, just a theatrical placebo.
After more than 3000 trials, there is no need for yet more. Acupuncture is dead.
Acupuncture is a theatrical placebo
David Colquhoun (UCL) and Steven Novella (Yale)
Anesthesia & Analgesia, June 2013 116:1360-1363.
Pain is a big problem. If you read about pain management centres you might think it had been solved. It hasn’t. And when no effective treatment exists for a medical problem, it leads to a tendency to clutch at straws. Research has shown that acupuncture is little more than such a straw.
Although it is commonly claimed that acupuncture has been around for thousands of years, it hasn’t always been popular even in China. For almost 1000 years it was in decline and in 1822 Emperor Dao Guang issued an imperial edict stating that acupuncture and moxibustion should be banned forever from the Imperial Medical Academy.
Acupuncture continued as a minor fringe activity in the 1950s. After the Chinese Civil War, the Chinese Communist Party ridiculed traditional Chinese medicine, including acupuncture, as superstitious. Chairman Mao Zedong later revived traditional Chinese Medicine as part of the Great Proletarian Cultural Revolution of 1966 (Atwood, 2009). The revival was a convenient response to the dearth of medically-trained people in post-war China, and a useful way to increase Chinese nationalism. It is said that Chairman Mao himself preferred Western medicine. His personal physician quotes him as saying “Even though I believe we should promote Chinese medicine, I personally do not believe in it. I don’t take Chinese medicine” Li {Zhisui Li. Private Life Of Chairman Mao: Random House, 1996}.
The political, or perhaps commercial, bias seems to still exist. It has been reported by Vickers et al. (1998) (authors who are sympathetic to alternative medicine) that
"all trials [of acupuncture] originating in China, Japan, Hong Kong, and Taiwan were positive"(4).
Acupuncture was essentially defunct in the West until President Nixon visited China in 1972. Its revival in the West was largely a result of a single anecdote promulgated by journalist James Reston in the New York Times, after he’d had acupuncture in Beijing for post-operative pain in 1971. Despite his eminence as political journalist, Reston had no scientific background and evidently didn’t appreciate the post hoc ergo propter hoc fallacy, or the idea of regression to the mean.
After Reston’s article, acupuncture quickly became popular in the West. Stories circulated that patients in China had open heart surgery using only acupuncture (Atwood, 2009). The Medical Research Council (UK) sent a delegation, which included Alan Hodgkin, to China in 1972 to investigate these claims , about which they were skeptical. In 2006 the claims were repeated in 2006 in a BBC TV program, but Simon Singh (author of Fermat’s Last Theorem) discovered that the patient had been given a combination of three very powerful sedatives (midazolam, droperidol, fentanyl) and large volumes of local anaesthetic injected into the chest. The acupuncture needles were purely cosmetic.
Curiously, given that its alleged principles are as bizarre as those on any other sort of pre-scientific medicine, acupuncture seemed to gain somewhat more plausibility than other forms of alternative medicine. The good thing about that is that more research has been done on acupuncture than on just about any other fringe practice.
The outcome of this research, we propose, is that the benefits of acupuncture, if any, are too small and too transient to be of any clinical significance. It seems that acupuncture is little or no more than a theatrical placebo. The evidence for this conclusion will now be discussed.
Three things that are not relevant to the argument
There is no point in discussing surrogate outcomes such as fMRI studies or endorphine release studies until such time as it has been shown that patients get a useful degree of relief. It is now clear that they don’t.
There is also little point in invoking individual studies. Inconsistency is a prominent characteristic of acupuncture research: the heterogeneity of results poses a problem for meta-analysis. Consequently it is very easy to pick trials that show any outcome whatsoever. Therefore we shall consider only meta-analyses.
The argument that acupuncture is somehow more holistic, or more patient-centred, than medicine seems us to be a red herring. All good doctors are empathetic and patient-centred. The idea that empathy is restricted to those who practice unscientific medicine seems both condescending to doctors, and it verges on an admission that empathy is all that alternative treatments have to offer.
There is now unanimity that the benefits, if any, of acupuncture for analgesia, are too small to be helpful to patients.
Large multicenter clinical trails conducted in Germany {Linde et al., 2005; Melchart et, 2005; Haake et al, 2007, Witt et al, 2005), and in the United States {Cherkin et al, 2009) consistently revealed that verum (or true) acupuncture and sham acupuncture treatments are no different in decreasing pain levels across multiple chronic pain disorders: migraine, tension headache, low back pain, and osteoarthritis of the knee.
If, indeed, sham acupuncture is no different from real acupuncture the apparent improvement that may be seen after acupuncture is merely a placebo effect. Furthermore it shows meridians don’t exist, so the "theory" memorized by qualified acupuncturists is just myth. All that remains to be discussed is whether or not the placebo effect is big enough to be useful, and whether it is ethical to prescribe placebos.
Some recent meta-analyses have found that there may be a small difference between sham and real acupuncture. Madsen Gøtzsche & Hróbjartsson {2009) looked at thirteen trials with 3025 patients, in which acupuncture was used to treat a variety of painful conditions. There was a small difference between ‘real’ and sham acupuncture (it didn’t matter which sort of sham was used), and a somewhat bigger difference between the acupuncture group and the no-acupuncture group. The crucial result was that even this bigger difference corresponded to only a 10 point improvement on a 100 point pain scale. A consensus report (Dworkin, 2009) that a change of this sort should be described as a “minimal” change or “little change”. It isn’t big enough for the patient to notice much effect.
The acupuncture and no-acupuncture groups were, of course, not blind to the patients and neither were they blind to the practitioner giving the treatment. It isn’t possible to say whether the observed difference is a real physiological action or whether it’s a placebo effect of a rather dramatic intervention. Interesting though it would be to know this, it matters not a jot, because the effect just isn’t big enough to produce any tangible benefit.
Publication bias is likely to be an even greater problem for alternative medicine than it is for real medicine, so it is particularly interesting that the result just described has been confirmed by authors who practise, or sympathise with, acupuncture. Vickers et al. (2012) did a meta-analysis for 29 RCTs, with 17,922 patients. The patients were being treated for a variety of chronic pain conditions. The results were very similar to those of Madsen et al.{2009). Real acupuncture was better than sham, but by a tiny amount that lacked any clinical significance. Again there was a somewhat larger difference in the non-blind comparison of acupuncture and no-acupuncture, but again it was so small that patients would barely notice it.
Comparison of these two meta-analyses shows how important it is to read the results, not just the summaries. Although the outcomes were similar for both, the spin on the results in the abstracts (and consequently the tone of media reports) was very different.
An even more extreme example of spin occurred in the CACTUS trial of acupuncture for " ‘frequent attenders’ with medically unexplained symptoms” (Paterson et al., 2011). In this case, the results showed very little difference even between acupuncture and no-acupuncture groups, despite the lack of blinding and lack of proper controls. But by ignoring the problems of multiple comparisons the authors were able to pick out a few results that were statistically significant, though trivial in size. But despite this unusually negative outcome, the result was trumpeted as a success for acupuncture. Not only the authors, but also their university’s PR department and even the Journal editor issued highly misleading statements. This gave rise to a flood of letters to the British Journal of General Practice and much criticism on the internet.
From the intellectual point of view it would be interesting to know if the small difference between real and sham acupuncture found in some, but not all, recent studies is a genuine effect of acupuncture or whether it is a result of the fact that the practitioners are never blinded, or of publication bias. But that knowledge is irrelevant for patients. All that matters for them is whether or not they get a useful degree of relief.
There is now unanimity between acupuncturists and non-acupuncturists that any benefits that may exist are too small to provide any noticeable benefit to patients. That being the case it’s hard to see why acupuncture is still used. Certainly such an accumulation of negative results would result in the withdrawal of any conventional treatment.
Specific conditions
Acupuncture should, ideally, be tested separately for effectiveness for each individual condition for which it has been proposed (like so many other forms of alternative medicine, that’s a very large number). Good quality trials haven’t been done for all of them. It’s unlikely that acupuncture works for rheumatoid arthritis, stopping smoking, irritable bowel syndrome or for losing weight. And there is no good reason to think it works for addictions, asthma, chronic pain, depression, insomnia, neck pain, shoulder pain or frozen shoulder, osteoarthritis of the knee, sciatica, stroke or tinnitus and many other conditions (Ernst et al., 2011).
In 2009, the UK’s National Institute for Clinical Excellence (NICE) did recommend both acupuncture and chiropractic for back pain. This exercise in clutching at straws caused something of a furore. In the light of NICE’s judgement the Oxford Centre for Evidence-based medicine updated its analysis of acupuncture for back pain. Their verdict was
“Clinical bottom line. Acupuncture is no better than a toothpick for treating back pain.”
The paper by Artus et al. (2010) is of particular interest for the problem of back pain. Their Fig 2 shows that there is a modest improvement in pain scores after treatment, but much the same effect, with the same time course is found regardless of what treatment is given, and even with no treatment at all. They say
“we found evidence that these responses seem to follow a common trend of early rapid improvement in symptoms that slows down and reaches a plateau 6 months after the start of treatment, although the size of response varied widely. We found a similar pattern of improvement in symptoms following any treatment, regardless of whether it was index, active comparator, usual care or placebo treatment”.
It seems that most of what’s being seen is regression to the mean. And that is very likely to be the main reason why acupuncture sometimes appears to work when it doesn’t.
Although the article by Wang et al (2012) was written to defend the continued use of acupuncture, the only condition for which they claim that there is any reasonably strong evidence is for post-operative nausea and vomiting (PONV). It would certainly be odd if a treatment that had been advocated for such a wide variety of conditions turned out to work only for PONV. Nevertheless, let’s look at the evidence.
The main papers that are cited to support the efficacy of acupuncture in alleviation of PONV are all from the same author: Lee & Done (1999), and two Cochrane reviews, Lee & Done (2004), updated in Lee & Fan (2009). We need only deal with this latest updated meta-analysis.
Although the authors conclude “P6 acupoint stimulation prevented PONV”, closer examination shows that this conclusion is very far from certain. Even taken at face value, a relative risk of 0.7 can’t be described as “prevention”. The trials that were included were not all tests of acupuncture but included several other more or less bizarre treatments (“acupuncture, electro-acupuncture, transcutaneous nerve stimulation, laser stimulation, capsicum plaster, an acu-stimulation device, and acupressure”). The number needed to treat varied from a disastrous 34 to a poor 5 for patients with control rates of PONV of 10% and 70% respectively.
The meta-analysis showed, on average, similar effectiveness for acupumcture and anti-emetic drugs. The problem is that the effectiveness of drugs is in doubt because an update to the Cochrane review has been delayed (Carlisle, 2012) by the discovery of major fraud by a Japanese anesthetist, Yoshitaka Fujii (Sumikawa, 2012). It has been suggested that metclopramide barely works at all (Bandolier, 2012; Henzi, 1999).
Of the 40 trials (4858 participants) that were included; only four trials reported adequate allocation concealment. Ninety percent of trials were open to bias from this source. Twelve trials did not report all outcomes. The opportunities for bias are obvious. The authors themselves describe all estimates as being of “Moderate quality” which is defined this: “Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate”. That being the case, perhaps the conclusion should have been “more research needed”. In fact almost all trials of alternative medicines seem to end up with the conclusion that more research is needed.
Conclusions
It is clear from meta-analyses that results of acupuncture trials are variable and inconsistent, even for single conditions. After thousands of trials of acupuncture, and hundreds of systematic reviews (Ernst et al., 2011), arguments continue unabated. In 2011, Pain carried an editorial which summed up the present situation well.
“Is there really any need for more studies? Ernst et al. (2011) point out that the positive studies conclude that acupuncture relieves pain in some conditions but not in other very similar conditions. What would you think if a new pain pill was shown to relieve musculoskeletal pain in the arms but not in the legs? The most parsimonious explanation is that the positive studies are false positives. In his seminal article on why most published research findings are false, Ioannidis (2005) points out that when a popular but ineffective treatment is studied, false positive results are common for multiple reasons, including bias and low prior probability.”
Since it has proved impossible to find consistent evidence after more than 3000 trials, it is time to give up. It seems very unlikely that the money that it would cost to do another 3000 trials would be well-spent.
A small excess of positive results after thousands of trials is most consistent with an inactive intervention. The small excess is predicted by poor study design and publication bias. Further, Simmons et al (2011) demonstrated that exploitation of "undisclosed flexibility in data collection and analysis" can produce statistically positive results even from a completely nonexistent effect. With acupuncture in particular there is documented profound bias among proponents (Vickers et al., 1998). Existing studies are also contaminated by variables other than acupuncture – such as the frequent inclusion of "electroacupuncture" which is essentially transdermal electrical nerve stimulation masquerading as acupuncture.
The best controlled studies show a clear pattern – with acupuncture the outcome does not depend on needle location or even needle insertion. Since these variables are what define "acupuncture" the only sensible conclusion is that acupuncture does not work. Everything else is the expected noise of clinical trials, and this noise seems particularly high with acupuncture research. The most parsimonious conclusion is that with acupuncture there is no signal, only noise.
The interests of medicine would be best-served if we emulated the Chinese Emperor Dao Guang and issued an edict stating that acupuncture and moxibustion should no longer be used in clinical practice.
No doubt acupuncture will continue to exist on the High Streets where they can be tolerated as a voluntary self-imposed tax on the gullible (as long as they don’t make unjustified claims).
|
REFERENCES
|
|
|
1. Acupuncture Centre. . About Acupuncture. Available at: http://www.acupuncturecentre.org/aboutacupuncture.html. Accessed March 30, 2013 |
|
|
2. Atwood K. “Acupuncture Anesthesia”: a Proclamation from Chairman Mao (Part IV). Available at: http://www.sciencebasedmedicine.org/index.php/acupuncture-anesthesia-a-proclamation-from-chairman-mao-part-iv/. Accessed September 2, 2012 |
|
|
3. Li Z Private Life of Chairman Mao: The Memoirs of Mao’s Personal Physician. 1996 New York: Random House |
|
|
4. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19:159–66 Available at: http://bit.ly/WqVGWN. Accessed September 2, 2012 |
|
|
5. Reston J. Now, About My Operation in Peking; Now, Let Me Tell You About My Appendectomy in Peking … The New York Times. 1971 Available at: http://select.nytimes.com/gst/abstract.html?res=FB0D11FA395C1A7493C4AB178CD85F458785F9. Accessed March 30, 2013 |
|
|
6. Atwood K. “Acupuncture anesthesia”: a proclamation from chairman Mao (part I). Available at: http://www.sciencebasedmedicine.org/index.php/acupuncture-anesthesia-a-proclamation-of-chairman-mao-part-i/. Accessed September 2, 2012 |
|
|
7. Linde K, Streng A, Jürgens S, Hoppe A, Brinkhaus B, Witt C, Wagenpfeil S, Pfaffenrath V, Hammes MG, Weidenhammer W, Willich SN, Melchart D. Acupuncture for patients with migraine: a randomized controlled trial. JAMA. 2005;293:2118–25 |
|
|
8. Melchart D, Streng A, Hoppe A, Brinkhaus B, Witt C, Wagenpfeil S, Pfaffenrath V, Hammes M, Hummelsberger J, Irnich D, Weidenhammer W, Willich SN, Linde K. Acupuncture in patients with tension-type headache: randomised controlled trial. BMJ. 2005;331:376–82 |
|
|
9. Haake M, Müller HH, Schade-Brittinger C, Basler HD, Schäfer H, Maier C, Endres HG, Trampisch HJ, Molsberger A. German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups. Arch Intern Med. 2007;167:1892–8 |
|
|
10. Witt C, Brinkhaus B, Jena S, Linde K, Streng A, Wagenpfeil S, Hummelsberger J, Walther HU, Melchart D, Willich SN. Acupuncture in patients with osteoarthritis of the knee: a randomised trial. Lancet. 2005;366:136–43 |
|
|
11. Cherkin DC, Sherman KJ, Avins AL, Erro JH, Ichikawa L, Barlow WE, Delaney K, Hawkes R, Hamilton L, Pressman A, Khalsa PS, Deyo RA. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009;169:858–66 |
|
|
12. Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ. 2009;338:a3115 |
|
|
13. Dworkin RH, Turk DC, McDermott MP, Peirce-Sandner S, Burke LB, Cowan P, Farrar JT, Hertz S, Raja SN, Rappaport BA, Rauschkolb C, Sampaio C. Interpreting the clinical importance of group differences in chronic pain clinical trials: IMMPACT recommendations. Pain. 2009;146:238–44 |
|
|
14. Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, Sherman KJ, Witt CM, Linde K. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172:1444–53 |
|
|
15. Paterson C, Taylor RS, Griffiths P, Britten N, Rugg S, Bridges J, McCallum B, Kite G. Acupuncture for ‘frequent attenders’ with medically unexplained symptoms: a randomised controlled trial (CACTUS study). Br J Gen Pract. 2011;61:e295–e305 |
|
|
16. . Letters in response to Acupuncture for ‘frequent attenders’ with medically unexplained symptoms. Br J Gen Pract. 2011;61 Available at: http://www.ingentaconnect.com/content/rcgp/bjgp/2011/00000061/00000589. Accessed March 30, 2013 |
|
|
17. Colquhoun D. Acupuncturists show that acupuncture doesn’t work, but conclude the opposite: journal fails. 2011 Available at: https://www.dcscience.net/?p=4439. Accessed September 2, 2012 |
|
|
18. Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. Pain. 2011;152:755–64 |
|
|
19. Colquhoun D. NICE falls for Bait and Switch by acupuncturists and chiropractors: it has let down the public and itself. 2009 Available at: https://www.dcscience.net/?p=1516. Accessed September 2, 2012 |
|
|
20. Colquhoun D. The NICE fiasco, part 3. Too many vested interests, not enough honesty. 2009 Available at: https://www.dcscience.net/?p=1593. Accessed September 2, 2012 |
|
|
21. Bandolier. . Acupuncture for back pain—2009 update. Available at: http://www.medicine.ox.ac.uk/bandolier/booth/painpag/Chronrev/Other/acuback.html. Accessed March 30, 2013 |
|
|
22. Artus M, van der Windt DA, Jordan KP, Hay EM. Low back pain symptoms show a similar pattern of improvement following a wide range of primary care treatments: a systematic review of randomized clinical trials. Rheumatology (Oxford). 2010;49:2346–56 |
|
|
23. Wang S-M, Harris RE., Lin Y-C, Gan TJ. Acupuncture in 21st century anesthesia: is there a needle in the haystack? Anesth Analg. 2013;116:1356–9 |
|
|
24. Lee A, Done ML. The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesth Analg. 1999;88:1362–9 |
|
|
25. Lee A, Done ML. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2004:CD003281 |
|
|
26. Lee A, Fan LT. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2009:CD003281 |
|
|
27. Carlisle JB. A meta-analysis of prevention of postoperative nausea and vomiting: randomised controlled trials by Fujii etal. compared with other authors. Anaesthesia. 2012;67:1076–90 |
|
|
28. Sumikawa K. The results of investigation into Dr.Yoshitaka Fujii’s papers. Report of the Japanese Society of Anesthesiologists Special Investigation Committee. http://www.anesth.or.jp/english/pdf/news20120629.pdf |
|
|
29. Bandolier. . Metoclopramide is ineffective in preventing postoperative nausea and vomiting. Available at: http://www.medicine.ox.ac.uk/bandolier/band71/b71-8.html. Accessed March 30, 2013 |
|
|
30. Henzi I, Walder B, Tramèr MR. Metoclopramide in the prevention of postoperative nausea and vomiting: a quantitative systematic review of randomized, placebo-controlled studies. Br J Anaesth. 1999;83:761–71 |
|
|
31. Hall H. Acupuncture’s claims punctured: not proven effective for pain, not harmless. Pain. 2011;152:711–2 |
|
|
32. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2:e124 |
|
|
33. Simmons JP, Leif DN, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22:1359–66 |
Follow-up
30 May 2013 Anesthesia & Analgesia has put the whole paper on line. No paywall now!
9 June 2013. Since this page was posted on May 30, it has had over 20,000 page views. Not bad.
26 July 2013. The Observer had a large double-page spread about acupuncture. It was written by David Derbyshire, largely on the basis of this article.
26 December 2013
Over christmas the flow of stuff that misrepresents the "thousands of years" of Chinese medicine has continued unabated. Of course one expects people who are selling Chinese herbs and acupuncture to lie. All businesses do. One does not expect such misrepresentation from British Columbia, Cardiff University School of medicine, or from Yale University. I left a comment on the Yale piece. Whether it passes moderation remains to be seen. Just in case, here it is.
One statement is undoubtedly baseless ““If it’s still in use after a thousand years there must be something right,” It’s pretty obvious to the most casual observer that many beliefs that have been round for a thousand years have proved to be utterly wrong.
In any case, it’s simply not true that most “Traditional” Chinese medicine has been around for thousands of years. Acupuncture was actually banned by the Emperor Dao Guang in 1822. The sort of Chinese medicine that is sold (very profitably) to the west was essentially dead in China until it was revived by Mao as part of the great proletarian cultural revolution (largely to stir up Chinese nationalism at that time). Of course he didn’t use it himself.
This history has been documented in detail now, and it surprises me to see it misrepresented, yet again, from a Yale academic.
Of course there might turn out to be therapeutically useful chemicals in Chinese herbs (it has happened with artemesinin). But it is totally irresponsible to pretend that great things are coming in the absence of good RCTs in human patients.
Yale should be ashamed of PR like this. And so should Cardiff University. It not only makes the universities look silly. It corrupts the whole of the rest of these institutions. Who knows how much more of their PR is mere puffery.
18 January 2014. I checked the Yale posting and found that the comment, above, had indeed been deleted. There is little point in having comments if you are going to delete anything that’s mildly critical. It is simply dishonest.
The bulletin of the British Pharmacological Society, Pharmacology Matters, declined to publish the following article. Sadly the Society seems to be more interested in "reputation management" than in truth. Luckily, it is not easy to suppress criticism these days. A version of the article has appeared in Research Fortnight where it will be seen by far more people than it would have been in Pharmacology Matters. This is the original version that I submitted to them. They would not allow me to quote Lewis’s comment (apropos of the sale of homeopathic meningitis vaccine)
“Children will be harmed by this inaction. Children will die. And the fault must lie with Professor Sir Kent Woods, chairman of the regulator [MHRA].”
If a child were to die of whooping cough or meningitis as a result of buying the fraudulent "vaccines", that would be true. It’s a sad reflection on the state of defamation law that journals are not willing to say so. Blogs are fast becoming the best source of reliable information.
|
|
Stop press. The BPS has now signed up to Alltrials (too late for the printed version)
Pharmacology society does little to defend its subject
David Colquhoun
Over the past few years a courageous group of writers, researchers and activists has worked to expose the truth about the medicines we are sold, be they conventional or alternative.
Thanks, above all, to Ben Goldacre (1), more people than ever know that the big pharma companies have been concealing evidence of the harm that their products do, or the good they fail to do. Thanks to a small army of bloggers the preposterous claims made by peddlers of homeopathic remedies and other quackery are less likely to go unchallenged.
And yet, the organization charged with safeguarding the good name of pharmacology in this country, the British Pharmacological Society (BPS), has remained silent throughout.
The pharmaceutical industry
The BPS, rather than helping, became actively complicit when, along with much of the medical establishment, it signed an agreement with the ABPI (2). This document, developed under the aegis of the Ethical Standards in Health & Life Sciences Group (ESHLSG), contained two objectionable clauses:
“Industry plays a valid and important role in the provision of medical education”
And
“Medical representatives can be a useful resource for healthcare professionals”
Given that clinical “education” has long been part of Pharma’s marketing strategy, this seems disgraceful. And most of the doctors I know and respect refuse to see reps altogether. It is hopelessly optimistic to think that can an industry person can teach clinical pharmacology without bias in favour of his own company’s products. The BPS has many members who teach pharmacology. Can they really cope so badly that we need to have industry to educate clinicians?
It’s fine, and sometimes desirable, for academics and industry to work together on drug development. But only as long as the industry partner has no say in how, or whether, the results are published. Without that proviso we can expect more corruption of the sort that’s been seen at the University of Sheffield (3).
This is very sad, because I have great reason to like the drug industry. I’ve benefitted from several of their products myself. But the industry is in trouble. Many of its products provide only marginal benefits. Furthermore, some of the things that seemed to be useful, like SSRI antidepressants, have turned out to be next to useless once hidden trials were revealed (4). The MHRA’s learning module on SSRIs doesn’t seem to have caught up with this yet.
Sadly, the reaction of industry has been to resort to dishonesty, to hide unfavourable data and to increase yet more what it spends on marketing. Between 2009 and 2012, fines of at least 10 billion dollars (5) have been imposed on some of the most eminent companies. They include Lilly, Pfizer, AstraZeneca, Merck, Abbott and GlaxoSmithKline (GSK). The biggest fine of all ($3 bn, in July 2012) went to a British company, GSK. This succession of large fines seems to be regarded by the companies as mere marketing expenses.
All these fines were levied in the USA. Where, one might ask, are the regulators in the UK? Why have there been no fines here? Why, indeed, are some of the senior managers of these companies not in jail? Why has the BPS remained silent about the prostitution of its subject? And why have the MHRA done so little to stop it?
I suggest that you support the petition for release of the results of all trials (6). It’s been supported by many individuals and a lot of organisations, including the BMJ and the Royal Statistical Society. But, disgracefully, not by the BPS.
Quackery
At least in the case of the pharmaceutical industry some of its products work. But pharmacologists should also be concerned about the quackery industry, worth about 60 billion dollars per year (as opposed to $600 bn for the pharmaceutical industry). Virtually none of their products work (7). Why has the BPS said so little about it? It has, along with most of the medical and university establishment, shrugged its shoulders about the fact that students at Westminster University have been shown dowsing with a pendulum as a method for selection of herbal “remedies”, as part of a Bachelor or “Science” degree. It is an area in which every regulatory agency has failed to ensure even minimal levels of honesty (8). And the BPS has just shrugged.
The MHRA has been worse than useless in this area: it has been actively unhelpful (9). The senior staff of the MHRA are members of the BPS which has, as usual, said next to nothing. The MHRA’s herbal medicine committee has allowed misleading labels that give indications to be put on herbal potions, and these labels fail to make it clear that no evidence whatsoever of efficacy is required to get the MHRA kitemark. The wording was suggested (not required) by European law, but that law does not prevent the MHRA from saying, as it should, “there is no reason to think that this product is effective for any of the indications on the label” (10). Arguably, the MHRA is in breach of Consumer Protection law (11, 12).
At the time, the BPS did make some objection to the labelling (13), but only under great pressure from me (indeed I wrote it) . That has not been followed up, and I can no longer find it on the BPS web site. Indeed Philip Routledge, one of the people who is responsible for the misinformation in his capacity as chair of the MHRA Herbal Medicines Advisory Committee, is, at present the president of the BPS.
The MHRA has also been responsible for misleading labelling of the products of the most obviously fraudulent products of the lot: homeopathic pills, the medicines that contain no medicine. Most of the pills (anything beyond 12x dilution) contain not a single molecule of the substance on the label. Yet they have been given a get-out clause that enables them to evade prosecution by Trading Standards (an organisation that consistently fails to apply consumer protection laws. Rose et al (2011) (12) concluded
"EU directive 200s5/29/EC is largely ineffective in preventing misleading health claims for consumer products in the UK".
It is simply bizarre that the people at the MHRA, many of who are BPS members, have sat round a table and approved the following label. This example is for Arnica 30C pills, which, of course, contain no trace of arnica (14, 8). The outcome of their deliberations is simply surreal (see the actual labels here).
"Used within the homeopathic tradition for the symptomatic relief of sprains, muscular ache; and bruising or swelling after contusions."
This will deceive the naïve into thinking that it will have some effect on sprains etc. It won’t. And the MHRA have declined to test how the label is perceived by the public, though it took some effort to get them to admit it.
"If you are allergic to any of the ingredients in this medicine, consult your doctor before taking this medicine."
The ingredients aren’t stated apart from “contains lactose and sucrose”. That’s all they contain. No arnica.
"If pregnant or breastfeeding consult your doctor before use."
Why should a few mg of lactose and sucrose have the slightest effect on a pregnant or breast-feeding mother. This is pure make-believe
"If you forget to take this product, continue to take your usual dose at the usual time, it does not matter if you have missed a dose. Do not take a double dose to make up for a missed dose."
This statement is even more bizarre. There is nothing in the pills.
"If you take too much of this product (overdose) speak to a doctor or pharmacist and take this label with you."
And this is the ultimate in nonsense. The 1023 campaign regularly swallows whole bottles, and of course nothing happens. You can’t overdose on nothing. The fact that the MHRA can insist on this label, with a straight face, is the ultimate betrayal of science and reason.
"When asked to comment, as part of the consultation on these rules, this was the response from the BPS."
This is extracted from page 16 of the “selected response” provided by the MHRA under a Freedom of Information Act request [download all]
Things have changed little since A.J. Clark wrote his book on Patent Medicines in 1938 (15). And the BPS has done next to nothing to help. Neither has the MHRA. In fact both have colluded in the failures of both honesty and reason.
A BBC South West program recently revealed that a pharmacist was selling “homeopathic vaccines” for whooping cough and meningitis (16). The MHRA have know about his homicidal practice for years, but have done nothing. The General Pharmaceutical Council let him off with a rap on the knuckles. It has been left to bloggers and TV reporters to focus attention on these scoundrels. The well-respected blogger, Andy Lewis, wrote (17)
“Children will be harmed by this inaction. Children will die. And the fault must lie with Professor Sir Kent Woods, chairman of the regulator [MHRA].”
And the full clinical data for Tamiflu are still being concealed by Roche (18).
I think that is rather shameful.
I have been a member of the BPS for all my working life. I was happy when they made me an honorary fellow. But I now find myself asking if I can remain a member of an organisation that has done so little to defend honest scientific behaviour.
References
(1) Ben Goldacre’s Bad Pharma. Buy it now. Then do something. https://www.dcscience.net/?p=5538
(2) ABPI 2012 Guidance on collaboration between healthcare professionals and the pharmaceutical industry. http://www.abpi.org.uk/our-work/library/guidelines/Pages/collaboration-guidance.aspx
(3) Colquhoun, D. (2007) The Corporate Corruption of Higher Education: part 2 https://www.dcscience.net/?p=193
(4) Kirsch,I., B.J.Deacon, T.B.Huedo-Medina, A.Scoboria, T.J.Moore, and B.T.Johnson. 2008. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS. Med. 5:e45. http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0050045
(5) Groeger, L. (2012) Big Pharma’s Big Fines http://www.propublica.org/special/big-pharmas-big-fines
(6) All trials registered. All results reported, http://www.alltrials.net/supporters/
(7) Singh,S. and E.Ernst. 2009. Trick or Treatment. Corgi. http://en.wikipedia.org/wiki/Trick_or_Treatment
(8) Colquhoun, D. (2012) Regulation of alternative medicine: why it doesn’t work, and never can. https://www.dcscience.net/?p=5562
(9) Colquhoun, D, (2006) The MHRA breaks its founding principle: it is an intellectual disgrace. https://www.dcscience.net/?p=32
(10) Colquhoun, D. (2011). Why does the MHRA refuse to label herbal products honestly? Kent Woods and Richard Woodfield tell me. https://www.dcscience.net/?p=4269
(11) Colquhoun, D. (2009) Most alternative medicine is illegal. https://www.dcscience.net/?p=30
(12) Rose,L.B., P.Posadzki, and E.Ernst. 2012. Spurious claims for health-care products: an experimental approach to evaluating current UK legislation and its implementation. Med. Leg. J. 80:13-18. https://www.dcscience.net/Rose-medico-legal-2012.pdf
(13) Colquhoun, D. (2006) Learned Societies speak out against CAM, and the MHRA. https://www.dcscience.net/?p=30
(14) MHRA Arnicare Arnica 30c pillules NR 01175/0181 http://www.mhra.gov.uk/home/groups/par/documents/websiteresources/con049307.pdf
(15) Colquhoun, D. (2008) Patent medicines in 1938 and now: A.J.Clark’s book. https://www.dcscience.net/?p=257
(16) BBC South West on the evils of homeopathic "vaccines" http://www.youtube.com/watch?v=TZf9mUzI4RI
(17) Why Does the MHRA Not Close Down these Homeopaths? The regulators have known of serious problems for years, Nothing is done. http://www.quackometer.net/blog/2013/01/why-does-the-mhra-not-close-down-these-homeopaths.html
(18) Payne, D. (2012). Tamiflu: the battle for secret drug data http://www.bmj.com/content/345/bmj.e7303
Follow-up
25 February 2013 This post has some follow-up even before it appeared on Research Research. I noticed on the BPS web site a press release “BPS announces intention to sign All Trials Petition“. It was dated 20th February, but I didn’t notice it until after the printed edition went to press. It was expressed as a future intention to sign, though in fact they signed almost straight away (though over 100 organisations had already done so). That’s good. I suspect that when my old friend, Humphrey Rang, who is president elect of the BPS, takes charge, the Society may start to take its responsibilities to the public more seriously than it has in the past.
|
The MHRA, on the other hand, is still evading its self-declared job of ensuring the public that drugs work. |
|
13 March 2013. A reply to this piece appeared in red on Research Research, The British Pharmacological Society champions its science. It was written by Humphrey Rang who, as president elect of the BPS naturally felt obliged to defend its record. He defends the BPS membership of ESHLSG, but fails to mention that first the Lancet and then the BMA withdrew their support. Nor does he mention that medical students and doctors launched a campaign, BadGuidelines.org, against the agreement. The Medical Schools Council, which also signed the agreement, said "the scrutiny of the guidance has ‘identified deficiencies in the current statement". Didn’t they read it before signing? Rang says that the BPS is also working to improve the joint statement with the ABPI. That’s good, but one must wonder why the BPS signed up to the original form.
On the points about quackery, Rang sites the BPS statement on homeopathy (now vanished) but fails to mention that that statement was written by me in an attempt to wake the BPS from it’s slumbers on the matter of medicines that don’t work. But he doesn’t mention at all the matter of mislabelling of both homeopathic and herbal preparations.
Of course, the BPS does many good things. But like most organisations, it is too reluctant to speak out when it sees wrongdoing, and too reluctant to say "sorry we made a cock-up that time".
|
“In causing NHS Choices to publish content that is less than completely frank about the evidence on homeopathy, the DH have compromised the editorial standards of a website that they themselves established”. . . “. . . they have failed the general public, by putting special interests, politics, and the path of least resistance (as they saw it) before the truth about health and healthcare.” David Mattin, lately of NHS Choices |
NHS Choices is usually a good source of information for the public. But there is one exception: the information they provide about alternative medicine is poor. A Freedom of Information Act request has revealed that the attempt of NHS Choices to rewrite their pages more accurately was censored by the Department of Health in conjunction with the late Prince’s Foundation for Integrated Health. The Department of Health (DH) has misled the public.
The earliest version of the homeopathy information page recorded by the Wayback Machine was November 12 2007. It was still there on December 5 2010. The comments were mostly critical. One said, quite correctly,
|
I find it most regrettable that the way NHS has covered this subject is to give uncritical voice to the claims of homoeopathy without giving readers the information they need to evaluate those claims. To refer readers to the websites of the British Homeopathy Association is like settling the question of the shape of planet by a reference to the website of the Flat Earth Society |
There were a lot of complaints, and to the credit of NHS Choices, the page vanished. Throughout 2011, and up to October 2012 the information page on homeopathy read
|
Introduction Content on homeopathy has been removed from the website pending a review by the Department of Health policy team responsible for complementary and alternative medicines. Homeopathy is not part of mainstream medicine. Instead it is defined as a complementary or alternative medicine. If you are considering using homeopathy, talk to your GP first. For more information about homeopathy see the House of Commons Science and Technology Committee report on homeopathy published on 8 February 2010 and the Department of Health response to that report published in July 2010 (PDF, 69KB). |
Then, at the end of 2012, the page reappeared. It was a bit better than the original, but not much. Many of the comments criticise the misleading nature of the information (as well as the usual “it worked for me” comments). The “useful links” still has six links to flat-earth organisations like the Society of Homeopaths, and only one to a sensible source, the excellent pamphlet from Sense about Science. They do link at the end to the 2010 Science and Technology Committee Report: Evidence Check 2: Homeopathy (PDF, 1.61Mb), and to the Government Response to the Science and Technology Committee Report, Evidence Check 2: Homeopathy (PDF, 69kb) but no comment is made on the findings.
Policy-based evidence
I wondered why the NHS Choices page, after an absence of almost two years, had returned in such an unsatisfactory form. So I asked them. After a reminder, I was told that my queries were being dealt with not by NHS Choices, but had been referred to Dr Sunjai Gupta “the DH official with responsibility for this area”. Dr Sunjai Gupta OBE is Deputy Director of Public Health Strategy and Social Marketing, Department of Health.
Dr Gupta is not obviously sympathetic to woo. It’s hard to tell since he doesn’t seem to have published much. But one is not reassured by an article that he wrote for the Journal of Holistic Healthcare. It appears straight after an article by fantasy herbalist, Simon Mills.
Despite assurances that I’d hear from Dr Gupta shortly, nothing happened. So I sent a request for the correspondence under the Freedom of Information Act (2000). Although the request was addressed to NHS Choices, a public body, strenuous efforts were made to divert it to the Department of Health. These were resisted. Nevertheless when, after a long delay, the material arrived, it came not from NHS Choices, but from DH, who had evidently vetted it,. The emails were rather shocking [download all].
A mail dated 1 December 2009 said
This is the most direct statement I’ve seen that, in the Department of Health, policy dictates evidence. NHS Choices is meant to provide evidence, but what they say has to be checked by DH to make sure they “don’t clash with any policy messages”.
The re-written page
The original version of the re-written page was sent to me by David Mattin, who worked for NHS Choices until September 2012. You can download the whole draft here. It is an enormous improvement on the original page. For example, it says
|
“Does it work?
Many independent experts would respond to this question ‘no, homeopathy does not work’ There is no good quality clinical evidence to show that homeopathy is more successful than placebo in the treatment of any type of condition. Furthermore, if the principles of homeopath were true it would violate all the existing theories of science that we make use of today; not just our theory of medicine, but also chemistry, biology and physics. |
This original draft was sent to Mattin on 29 January 2010. After editing it for length Mattin sent it to DH for approval. Over the next two years, DH removed much of the accurate content. Mattin’s own comments on this evisceration are reproduced below.
The DH emails
All the names have been redacted. Needless to say, nobody is willing to take responsibility. But the number of people who support magic medicine is really quite small so the main players were easy to identify.
During the nearly 2 year absence of the homeopathy page, dozens of changes were made by DH. It seems that the policy message with which the NHS Choices draft failed to comply were those of the Prince’s Foundation for Integrated Health, and its successor (after April 2010), the College of Integrated Health, now known as the College of Medicine.
NHS Choices sought advice about their redrafted pages from the right person, Sir Iain Chalmers, one of the founders of the Cochrane Collaboration. On 3 Nov 2009, Chalmers advised
The most reliable source in the country - and one of the most reliable in the world - is Professor Edzard Ernst, professor of Complementary Medicine at the Peninsular Medical School, |
Ernst returned his suggestions in July 2010, but it seems that few of them survived the subsequent 18 months of revisions by DH.
On 2 December 2009, a mail from the NHS headquarters (Quarry House, Leeds) was sent to NHS Choices
This makes it perfectly clear that DH regards the Prince’s Foundation, and the equally flaky Complementary and Natural Healthcare Council (CNHC: known on the web as Ofquack) as appropriate guides for public health policy. The fact of the matter is that regulation of magic medicine by the government has been a total disaster, because, it seems, DH regards the Prince of Wales as a reliable source.
On 29 December 2009, the Prince’s Foundation went on the attack.
On 10th January 2010, two more letters were sent to DH by the Prince’s Foundation. At 13.48 they wrote
And at 22.14 on the same day, it was followed up with
The references to Devon and to Thought Field Therapy, make it very obvious that these letters were written by Dr Michael Dixon OBE, who was medical director of the Prince’s Foundation, and who is now a director of the “College of Medicine”. And the object of Dixon’s bile is obviously Edzard Ernst (the quotation is from his book, Trick or Treatment).
I find it fascinating to see just how venomous quacks become when the evidence contradicts their views. The cuddly “holistic” veneer quickly vanishes.
It gets worse. On 21 January 2010, a mail from NHS Choices to DH said
The only person in the country who fits this description is the (in)famous George Lewith. It is simply mind-boggling that DH regards him as an appropriate person to advise on anything.
After that, NHS Choices kept asking DH to sign off the documents, and changes continued to be made. Almost two years later, DH were still stalling.
The admission that “We are a bit short of doctors within DH these days” is interesting.
A bit short of anyone capable of critical thinking would be more accurate.
The most interesting document that I got from DH was an intermediate draft of the rewritten page on homeopathy (undated). Download the document. Here are a couple of extracts.
It’s a story of two years of meddling and procrastination. The end result misinforms the public.
Right at the start, the NHS Choices draft says, reasonably enough
A House of Commons Science and Technology Committee report said that homeopathic remedies perform no better than placebos and that the principles on which homeopathv is based are “scientifically implausible”.
But a comment, added apparently by DH, says
Can we remove this statement? This report is really quite contentious and we may well be subject to quite a lot of challenge from the Homeopathic community if published.
What on earth? The DH seems to think that that its job is not to present the evidence, but to avoid challenges from the homeopathic community! And true enough, this piece is missing from the final version.
A bit later, the NHS Choices draft was censored again
“A 2010 Science and Technology Committee report said that scientific tests had shown that homeopathic treatments don’t work”
But again this doesn’t appear in the final version. The comment, apparently from DH, says
“The DH response to this report (point 24) doesn’t support this statement though”
That’s a gross distortion of point 24, which actually concludes
“The Government Chief Scientific Adviser cannot envisage scientifically credible proposals for funding for research into homeopathy in the future”
NHS Choices was not happy with the result
Shortly before the revised page was published, Paul Nuki, Editor in Chief of NHS Choicea, sent an email to DH.
|
Date: 7th September 2011 Time: 3:33:42 pm Hi I’ve been through the CAM articles and asked that we publish them asap as requested. For the record, we will be publishing these pieces outside of the normal editorial process. Although originally signed off by a suitably qualified clinician, the time lapse and policy changes have been so substantial as to render that null and void. We also don’t have a formal |
It doesn’t need much reading between the lines to see that he was unhappy with the result. It will be interesting to see whether the Information Standard people at the Royal Society for Public Health do anything about it.
The Department of Health has not just ignored evidence but actively opposed it.
That’s the only possible conclusion from the documents that I was sent. And it’s pretty shocking that the DH has preferred advice from the Prince’s Foundation and its handful of acolytes (in particular Michael Dixon and George Lewith) to the findings of the Science and Technology Select Committee and the views of the Chief Scientific Advisor.
In January this year, the Chief medical Officer, Dame Sally Davies, said, in a rare outburst of candour
|
‘I’m very concerned when homeopathic practitioners try to peddle this way of life to prevent malaria or other infectious disease,” she said. “I am perpetually surprised that homeopathy is available on the NHS.” Dame Sally, who is England’s most senior doctor, concluded by remarking that homeopathy “is rubbish”. |
 |
So one part of DH is working to contradict another part. the Chief Medical Officer. Perhaps Sally Davies should have a word with Dr Gupta.
This all predates the advent of Jeremy Hunt (and known defender of homeopathy) as health minister. But the sympathies of some DH people are made obvious by the presence on the DH web site of an article “Personal health budgets: A new way of accessing complementary therapies?”. This astonishing piece confirms the worst fears that quacks will see personal health budgets as a commercial opportunity to peddle their wares. The article is by Jim Rogers of Lincoln University. What his paper does not mention is Rogers’ conflict of interest. He’s a homeopath, and he has a paper in the International Journal of High Dilution Research (yes, there is a journal for every form of make-believe). You can download a reprint of this paper. It advocates more research into homeopathic provings, something that even George Lewith seems to have given up on.
It’s about time that the DH started to listen to the Chief Medical Officer. As it is, some people at DH seem to prefer the advice of the Prince’s Foundation, and to actively suppress employees who prefer evidence to anecdote.
One thing is clear. The DH is an unholy mess. Parts of it are intent on producing policy-based evidence.
Comment by David Mattin, who edited the first draft for NHS Choices
David Mattin left NHS Choices in September, 2012. He edited the new version and lived through the two years of wrangling with DH during which much of the best content was eviscerated. He sent me this statement about the affair.
|
As an editor at NHS Choices, I viewed it as my job to present evidence-based information to the public. The article we prepared on homeopathy stayed true to that central purpose: it made clear to readers that there is no good quality evidence that homeopathy is an effective treatment for any health condition, and also presented the broad scientific consensus that the supposed method of action of homeopathy is implausible.
What followed was a two year story of delay, and eventual suppression, of that article. My strong impression was of DH civil servants who lacked the courage, and, frankly, the energy to stand up to the criticism from special interest groups that they anticipated would arise because of the article; and that did indeed arise when a draft of the article and other draft content on complementary and alternative medicines fell into the hands of the Prince’s Foundation and other CAM groups.
The attitude of DH civil servants, broadly, was simply to tell us ‘we can’t say this about homeopathy, people will complain’. They seemed to have no interest in making an appraisal of the evidence on homeopathy themselves to see if what we were saying was actually true or not. We repeatedly pushed back with the message: ‘some people may very well complain, but if what we are saying about the evidence base is true – and it is – then we must simply weather those complaints, and stand by our content. Our duty is to supply our readers with the best information, not to please the homeopathy community.’ But these arguments were disregarded. The DH civil servants were almost entirely concerned with the politics of the situation – that is, the politics as they saw them – and the possibility that this article may create new work for them, and very little concerned with the evidence itself, or the presentation of this evidence to the public.
The whole episode is an insight into the way special interest groups can influence the workings of government and the public sector simply by making a lot of noise, and having a few powerful friends. In causing NHS Choices to publish content that is less than completely frank about the evidence on homeopathy, the DH have compromised the editorial standards of a website that they themselves established, and that they fund. They have sold out the NHS Choices editorial team, who work tirelessly to fulfil their remit. And, most seriously, they have failed the general public, by putting special interests, politics, and the path of least resistance (as they saw it) before the truth about health and healthcare.
|
Follow-up
13 February 2013 The Guardian version of this story, written by Sarah Boseley, is Prince’s charity lobbied government to water down homeopathy criticism. It’s fine as far as it goes but it doesn’t name any names. There are some good comments though.
14 February 2013. The printed Guardian gave the story full 5 column-width coverage. [download print version]
And news has reached the USA: there’s an account of the affair on the Neurologica blog: Politics trumping science at the NHS.
On 14 February, the Guardian version was Editor’s Choice by lunchtime, and the Guardian web version already had 414 comments, mostly sensible (though this blog got far more referrals from twitter than from the Guardian)
And news has reached the USA: there’s an account of the affair on the Neurologica blog: Politics trumping science at the NHS.
15 February 2013. The Daily Mail had very fair coverage of the story.
The Guardian closed the comments on the story when it had got 642 comments, most of them very sensible. And this page got almost 6000 hits in 24 hours. The majority of the referrals came from Twitter rather than from the Guardian, despite the direct link to the page from the Guardian.
18 February 2013. The affair featured in BMJ News [download the reprint]. The item featured prominently on the BMJ news page.
19 February 2013 Only six days of this post, the NHS Choices page has been re-written again, in a much improved form. That looks like bloggers 1, DH 0. It is baffling that it’s left to bloggers, working for nothing, to extract a bit of sense from the highly-paid civil servants at the Department of Health. But at least they listened this time, which is a lot more than happens often. Paul Nuki, who runs NHS Choices, deserves congratulations. Of course the revised page still doesn’t call a spade a spade, but it gets close at times. I like the way it starts "Homeopathy is a ‘treatment’ based". Notice the quotation marks.
Reputation management?
Incidentally, NHS Choices is outsourced to the (in)famous company, Capita. And the moderation of the comments on their site is outsourced again to Tempero, which describes itself as a "reputation management" company. Each of them creams off money meant of patient care. This discovery might explain why I and others have had comments rejected by NHS Choices several times. "Reputation management" is the antithesis of evidence. It is public relations, i.e. paid lying. That is quite wrong for a site that is meant to provide dispassionate information.
21 February 2013. Sadly a step backwards. Part of the improved page was removed. This bit.
The Chief Medical Officer, Professor Dame Sally Davies, has said there is no scientifically plausible way that homeopathy can prevent or cure diseases. She has made it clear she is particularly concerned about the use of homeopathy in developing countries as a so-called cure for malaria.
We can only speculate why this was removed, because it was true. In fact she accurately described homeopathy as "rubbish". Why she should not be quoted beats me.