Why you should ignore altmetrics and other bibliometric nightmares

Published January 16, 2014

This discussion seemed to be of sufficient general interest that we submitted is as a feature to eLife, because this journal is one of the best steps into the future of scientific publishing. Sadly the features editor thought that " too much of the article is taken up with detailed criticisms of research papers from NEJM and Science that appeared in the altmetrics top 100 for 2013; while many of these criticisms seems valid, the Features section of eLife is not the venue where they should be published". That’s pretty typical of what most journals would say. It is that sort of attitude that stifles criticism, and that is part of the problem. We should be encouraging post-publication peer review, not suppressing it. Luckily, thanks to the web, we are now much less constrained by journal editors than we used to be.

Here it is.

Scientists don’t count: why you should ignore altmetrics and other bibliometric nightmares

David Colquhoun¹ and Andrew Plested²

¹ University College London, Gower Street, London WC1E 6BT

² Leibniz-Institut für Molekulare Pharmakologie (FMP) & Cluster of Excellence NeuroCure, Charité Universitätsmedizin,Timoféeff-Ressowsky-Haus, Robert-Rössle-Str. 10, 13125 Berlin Germany.

Jeffrey Beall is librarian at Auraria Library, University of Colorado Denver. Although not a scientist himself, he, more than anyone, has done science a great service by listing the predatory journals that have sprung up in the wake of pressure for open access. In August 2012 he published “Article-Level Metrics: An Ill-Conceived and Meretricious Idea. At first reading that criticism seemed a bit strong. On mature consideration, it understates the potential that bibliometrics, altmetrics especially, have to undermine both science and scientists.

Altmetrics is the latest buzzword in the vocabulary of bibliometricians. It attempts to measure the “impact” of a piece of research by counting the number of times that it’s mentioned in tweets, Facebook pages, blogs, YouTube and news media. That sounds childish, and it is. Twitter is an excellent tool for journalism. It’s good for debunking bad science, and for spreading links, but too brief for serious discussions. It’s rarely useful for real science.

Surveys suggest that the great majority of scientists do not use twitter (7 — 13%). Scientific works get tweeted about mostly because they have titles that contain buzzwords, not because they represent great science.

What and who is Altmetrics for?

The aims of altmetrics are ambiguous to the point of dishonesty; they depend on whether the salesperson is talking to a scientist or to a potential buyer of their wares.

At a meeting in London , an employee of altmetric.com said “we measure online attention surrounding journal articles” “we are not measuring quality …” “this whole altmetrics data service was born as a service for publishers”, “it doesn’t matter if you got 1000 tweets . . .all you need is one blog post that indicates that someone got some value from that paper”.

These ideas sound fairly harmless, but in stark contrast, Jason Priem (an author of the altmetrics manifesto) said one advantage of altmetrics is that it’s fast “Speed: months or weeks, not years: faster evaluations for tenure/hiring”. Although conceivably useful for disseminating preliminary results, such speed isn’t important for serious science (the kind that ought to be considered for tenure) which operates on the timescale of years. Priem also says “researchers must ask if altmetrics really reflect impact” . Even he doesn’t know, yet altmetrics services are being sold to universities, before any evaluation of their usefulness has been done, and universities are buying them. The idea that altmetrics scores could be used for hiring is nothing short of terrifying.

The problem with bibliometrics

The mistake made by all bibliometricians is that they fail to consider the content of papers, because they have no desire to understand research. Bibliometrics are for people who aren’t prepared to take the time (or lack the mental capacity) to evaluate research by reading about it, or in the case of software or databases, by using them. The use of surrogate outcomes in clinical trials is rightly condemned. Bibliometrics are all about surrogate outcomes.

If instead we consider the work described in particular papers that most people agree to be important (or that everyone agrees to be bad), it’s immediately obvious that no publication metrics can measure quality. There are some examples in How to get good science (Colquhoun, 2007). It is shown there that at least one Nobel prize winner failed dismally to fulfil arbitrary biblometric productivity criteria of the sort imposed in some universities (another example is in Is Queen Mary University of London trying to commit scientific suicide?).

Schekman (2013) has said that science

“is disfigured by inappropriate incentives. The prevailing structures of personal reputation and career advancement mean the biggest rewards often follow the flashiest work, not the best.”

Bibliometrics reinforce those inappropriate incentives. A few examples will show that altmetrics are one of the silliest metrics so far proposed.

The altmetrics top 100 for 2103

The superficiality of altmetrics is demonstrated beautifully by the list of the 100 papers with the highest altmetric scores in 2013 For a start, 58 of the 100 were behind paywalls, and so unlikely to have been read except (perhaps) by academics.

The second most popular paper (with the enormous altmetric score of 2230) was published in the New England Journal of Medicine. The title was Primary Prevention of Cardiovascular Disease with a Mediterranean Diet. It was promoted (inaccurately) by the journal with the following tweet:

Many of the 2092 tweets related to this article simply gave the title, but inevitably the theme appealed to diet faddists, with plenty of tweets like the following:

The interpretations of the paper promoted by these tweets were mostly desperately inaccurate. Diet studies are anyway notoriously unreliable. As John Ioannidis has said

"Almost every single nutrient imaginable has peer reviewed publications associating it with almost any outcome."

This sad situation comes about partly because most of the data comes from non-randomised cohort studies that tell you nothing about causality, and also because the effects of diet on health seem to be quite small.

The study in question was a randomized controlled trial, so it should be free of the problems of cohort studies. But very few tweeters showed any sign of having read the paper. When you read it you find that the story isn’t so simple. Many of the problems are pointed out in the online comments that follow the paper. Post-publication peer review really can work, but you have to read the paper. The conclusions are pretty conclusively demolished in the comments, such as:

“I’m surrounded by olive groves here in Australia and love the hand-pressed EVOO [extra virgin olive oil], which I can buy at a local produce market BUT this study shows that I won’t live a minute longer, and it won’t prevent a heart attack.”

We found no tweets that mentioned the finding from the paper that the diets had no detectable effect on myocardial infarction, death from cardiovascular causes, or death from any cause. The only difference was in the number of people who had strokes, and that showed a very unimpressive P = 0.04.

Neither did we see any tweets that mentioned the truly impressive list of conflicts of interest of the authors, which ran to an astonishing 419 words.

“Dr. Estruch reports serving on the board of and receiving lecture fees from the Research Foundation on Wine and Nutrition (FIVIN); serving on the boards of the Beer and Health Foundation and the European Foundation for Alcohol Research (ERAB); receiving lecture fees from Cerveceros de España and Sanofi-Aventis; and receiving grant support through his institution from Novartis. Dr. Ros reports serving on the board of and receiving travel support, as well as grant support through his institution, from the California Walnut Commission; serving on the board of the Flora Foundation (Unilever). . . “

And so on, for another 328 words.

The interesting question is how such a paper came to be published in the hugely prestigious New England Journal of Medicine. That it happened is yet another reason to distrust impact factors. It seems to be another sign that glamour journals are more concerned with trendiness than quality.

One sign of that is the fact that the journal’s own tweet misrepresented the work. The irresponsible spin in this initial tweet from the journal started the ball rolling, and after this point, the content of the paper itself became irrelevant. The altmetrics score is utterly disconnected from the science reported in the paper: it more closely reflects wishful thinking and confirmation bias.

The fourth paper in the altmetrics top 100 is an equally instructive example.

This work was also published in a glamour journal, Science. The paper claimed that a function of sleep was to “clear metabolic waste from the brain”. It was initially promoted (inaccurately) on Twitter by the publisher of Science.

After that, the paper was retweeted many times, presumably because everybody sleeps, and perhaps because the title hinted at the trendy, but fraudulent, idea of “detox”. Many tweets were variants of “The garbage truck that clears metabolic waste from the brain works best when you’re asleep”.

But this paper was hidden behind Science’s paywall. It’s bordering on irresponsible for journals to promote on social media papers that can’t be read freely. It’s unlikely that anyone outside academia had read it, and therefore few of the tweeters had any idea of the actual content, or the way the research was done. Nevertheless it got “1,479 tweets from 1,355 accounts with an upper bound of 1,110,974 combined followers”. It had the huge Altmetrics score of 1848, the highest altmetric score in October 2013.

Within a couple of days, the story fell out of the news cycle. It was not a bad paper, but neither was it a huge breakthrough. It didn’t show that naturally-produced metabolites were cleared more quickly, just that injected substances were cleared faster when the mice were asleep or anaesthetised. This finding might or might not have physiological consequences for mice.

Worse, the paper also claimed that “Administration of adrenergic antagonists induced an increase in CSF tracer influx, resulting in rates of CSF tracer influx that were more comparable with influx observed during sleep or anesthesia than in the awake state”. Simply put, giving the sleeping mice a drug could reduce the clearance to wakeful levels. But nobody seemed to notice the absurd concentrations of antagonists that were used in these experiments: “adrenergic receptor antagonists (prazosin, atipamezole, and propranolol, each 2 mM) were then slowly infused via the cisterna magna cannula for 15 min”. Use of such high concentrations is asking for non-specific effects. The binding constant (concentration to occupy half the receptors) for prazosin is less than 1 nM, so infusing 2 mM is working at a million times greater than the concentration that should be effective. That’s asking for non-specific effects. Most drugs at this sort of concentration have local anaesthetic effects, so perhaps it isn’t surprising that the effects resembled those of ketamine.

The altmetrics editor hadn’t noticed the problems and none of them featured in the online buzz. That’s partly because to find it out you had to read the paper (the antagonist concentrations were hidden in the legend of Figure 4), and partly because you needed to know the binding constant for prazosin to see this warning sign.

The lesson, as usual, is that if you want to know about the quality of a paper, you have to read it. Commenting on a paper without knowing anything of its content is liable to make you look like an jackass.

A tale of two papers

Another approach that looks at individual papers is to compare some of one’s own papers. Sadly, UCL shows altmetric scores on each of your own papers. Mostly they are question marks, because nothing published before 2011 is scored. But two recent papers make an interesting contrast. One is from DC’s side interest in quackery, one was real science. The former has an altmetric score of 169, the latter has an altmetric score of 2.

The first paper was “Acupuncture is a theatrical placebo”, which was published as an invited editorial in Anesthesia and Analgesia [download pdf]. The paper was scientifically trivial. It took perhaps a week to write.

Nevertheless, it got promoted it on twitter, because anything to do with alternative medicine is interesting to the public. It got quite a lot of retweets. And the resulting altmetric score of 169 put it in the top 1% of all articles altmetric have tracked, and the second highest ever for Anesthesia and Analgesia.

As well as the journal’s own website, the article was also posted on the DCScience.net blog (May 30, 2013) where it soon became the most viewed page ever (24,468 views as of 23 November 2013), something that altmetrics does not seem to take into account.

Compare this with the fate of some real, but rather technical, science.

My [DC] best scientific papers are too old (i.e. before 2011) to have an altmetrics score, but my best score for any scientific paper is 2. This score was for Colquhoun & Lape (2012) “Allosteric coupling in ligand-gated ion channels”. It was a commentary with some original material.

The altmetric score was based on two tweets and 15 readers on Mendeley. The two tweets consisted of one from me (“Real science; The meaning of allosteric conformation changes http://t.co/zZeNtLdU ”).

The only other tweet as abusive one from a cyberstalker who was upset at having been refused a job years ago. Incredibly, this modest achievement got it rated “Good compared to other articles of the same age (71st percentile)”.

Conclusions about bibliometrics

Bibliometricians spend much time correlating one surrogate outcome with another, from which they learn little. What they don’t do is take the time to examine individual papers. Doing that makes it obvious that most metrics, and especially altmetrics, are indeed an ill-conceived and meretricious idea. Universities should know better than to subscribe to them.

Although altmetrics may be the silliest bibliometric idea yet, much this criticism applies equally to all such metrics. Even the most plausible metric, counting citations, is easily shown to be nonsense by simply considering individual papers. All you have to do is choose some papers that are universally agreed to be good, and some that are bad, and see how metrics fail to distinguish between them. This is something that bibliometricians fail to do (perhaps because they don’t know enough science to tell which is which). Some examples are given by Colquhoun (2007) (more complete version at dcscience.net).

Eugene Garfield, who started the metrics mania with the journal impact factor (JIF), was clear that it was not suitable as a measure of the worth of individuals. He has been ignored and the JIF has come to dominate the lives of researchers, despite decades of evidence of the harm it does (e.g.Seglen (1997) and Colquhoun (2003) ) In the wake of JIF, young, bright people have been encouraged to develop yet more spurious metrics (of which ‘altmetrics’ is the latest). It doesn’t matter much whether these metrics are based on nonsense (like counting hashtags) or rely on counting links or comments on a journal website. They won’t (and can’t) indicate what is important about a piece of research- its quality.

People say – I can’t be a polymath. Well, then don’t try to be. You don’t have to have an opinion on things that you don’t understand. The number of people who really do have to have an overview, of the kind that altmetrics might purport to give, those who have to make funding decisions about work that they are not intimately familiar with, is quite small. Chances are, you are not one of them. We review plenty of papers and grants. But it’s not credible to accept assignments outside of your field, and then rely on metrics to assess the quality of the scientific work or the proposal.

It’s perfectly reasonable to give credit for all forms of research outputs, not only papers. That doesn’t need metrics. It’s nonsense to suggest that altmetrics are needed because research outputs are not already valued in grant and job applications. If you write a grant for almost any agency, you can put your CV. If you have a non-publication based output, you can always include it. Metrics are not needed. If you write software, get the numbers of downloads. Software normally garners citations anyway if it’s of any use to the greater community.

When AP recently wrote a criticism of Heather Piwowar’s altmetrics note in Nature, one correspondent wrote: "I haven’t read the piece [by HP] but I’m sure you are mischaracterising it". This attitude summarizes the too-long-didn’t-read (TLDR) culture that is increasingly becoming accepted amongst scientists, and which the comparisons above show is a central component of altmetrics.

Altmetrics are numbers generated by people who don’t understand research, for people who don’t understand research. People who read papers and understand research just don’t need them and should shun them.

But all bibliometrics give cause for concern, beyond their lack of utility. They do active harm to science. They encourage “gaming” (a euphemism for cheating). They encourage short-term eye-catching research of questionable quality and reproducibility. They encourage guest authorships: that is, they encourage people to claim credit for work which isn’t theirs. At worst, they encourage fraud.

No doubt metrics have played some part in the crisis of irreproducibility that has engulfed some fields, particularly experimental psychology, genomics and cancer research. Underpowered studies with a high false-positive rate may get you promoted, but tend to mislead both other scientists and the public (who in general pay for the work). The waste of public money that must result from following up badly done work that can’t be reproduced but that was published for the sake of “getting something out” has not been quantified, but must be considered to the detriment of bibliometrics, and sadly overcomes any advantages from rapid dissemination. Yet universities continue to pay publishers to provide these measures, which do nothing but harm. And the general public has noticed.

It’s now eight years since the New York Times brought to the attention of the public that some scientists engage in puffery, cheating and even fraud.

Overblown press releases written by journals, with connivance of university PR wonks and with the connivance of the authors, sometimes go viral on social media (and so score well on altmetrics). Yet another example, from Journal of the American Medical Association involved an overblown press release from the Journal about a trial that allegedly showed a benefit of high doses of Vitamin E for Alzheimer’s disease.

This sort of puffery harms patients and harms science itself.

We can’t go on like this.

What should be done?

Post publication peer review is now happening, in comments on published papers and through sites like PubPeer, where it is already clear that anonymous peer review can work really well. New journals like eLife have open comments after each paper, though authors do not seem to have yet got into the habit of using them constructively. They will.

It’s very obvious that too many papers are being published, and that anything, however bad, can be published in a journal that claims to be peer reviewed . To a large extent this is just another example of the harm done to science by metrics –the publish or perish culture.

Attempts to regulate science by setting “productivity targets” is doomed to do as much harm to science as it has in the National Health Service in the UK. This has been known to economists for a long time, under the name of Goodhart’s law.

Here are some ideas about how we could restore the confidence of both scientists and of the public in the integrity of published work.

Nature, Science, and other vanity journals should become news magazines only. Their glamour value distorts science and encourages dishonesty.
Print journals are overpriced and outdated. They are no longer needed. Publishing on the web is cheap, and it allows open access and post-publication peer review. Every paper should be followed by an open comments section, with anonymity allowed. The old publishers should go the same way as the handloom weavers. Their time has passed.
Web publication allows proper explanation of methods, without the page, word and figure limits that distort papers in vanity journals. This would also make it very easy to publish negative work, thus reducing publication bias, a major problem (not least for clinical trials)
Publish or perish has proved counterproductive. It seems just as likely that better science will result without any performance management at all. All that’s needed is peer review of grant applications.
Providing more small grants rather than fewer big ones should help to reduce the pressure to publish which distorts the literature. The ‘celebrity scientist’, running a huge group funded by giant grants has not worked well. It’s led to poor mentoring, and, at worst, fraud. Of course huge groups sometimes produce good work, but too often at the price of exploitation of junior scientists
There is a good case for limiting the number of original papers that an individual can publish per year, and/or total funding. Fewer but more complete and considered papers would benefit everyone, and counteract the flood of literature that has led to superficiality.
Everyone should read, learn and inwardly digest Peter Lawrence’s The Mismeasurement of Science.

A focus on speed and brevity (cited as major advantages of altmetrics) will help no-one in the end. And a focus on creating and curating new metrics will simply skew science in yet another unsatisfactory way, and rob scientists of the time they need to do their real job: generate new knowledge.

It has been said

“Creation is sloppy; discovery is messy; exploration is dangerous. What’s a manager to do?
The answer in general is to encourage curiosity and accept failure. Lots of failure.”

And, one might add, forget metrics. All of them.

Follow-up

17 Jan 2014

This piece was noticed by the Economist. Their ‘Writing worth reading‘ section said

"Why you should ignore altmetrics (David Colquhoun) Altmetrics attempt to rank scientific papers by their popularity on social media. David Colquohoun [sic] argues that they are “for people who aren’t prepared to take the time (or lack the mental capacity) to evaluate research by reading about it.”"

20 January 2014.

Jason Priem, of ImpactStory, has responded to this article on his own blog. In Altmetrics: A Bibliographic Nightmare? he seems to back off a lot from his earlier claim (cited above) that altmetrics are useful for making decisions about hiring or tenure. Our response is on his blog.

20 January 2014.

Jason Priem, of ImpactStory, has responded to this article on his own blog, In Altmetrics: A bibliographic Nightmare? he seems to back off a lot from his earlier claim (cited above) that altmetrics are useful for making decisions about hiring or tenure. Our response is on his blog.

23 January 2014

The Scholarly Kitchen blog carried another paean to metrics, A vigorous discussion followed. The general line that I’ve followed in this discussion, and those mentioned below, is that bibliometricians won’t qualify as scientists until they test their methods, i.e. show that they predict something useful. In order to do that, they’ll have to consider individual papers (as we do above). At present, articles by bibliometricians consist largely of hubris, with little emphasis on the potential to cause corruption. They remind me of articles by homeopaths: their aim is to sell a product (sometimes for cash, but mainly to promote the authors’ usefulness).

It’s noticeable that all of the pro-metrics articles cited here have been written by bibliometricians. None have been written by scientists.

28 January 2014.

Dalmeet Singh Chawla,a bibliometrician from Imperial College London, wrote a blog on the topic. (Imperial, at least in its Medicine department, is notorious for abuse of metrics.)

29 January 2014 Arran Frood wrote a sensible article about the metrics row in Euroscientist.

2 February 2014 Paul Groth (a co-author of the Altmetrics Manifesto) posted more hubristic stuff about altmetrics on Slideshare. A vigorous discussion followed.

5 May 2014. Another vigorous discussion on ImpactStory blog, this time with Stacy Konkiel. She’s another non-scientist trying to tell scientists what to do. The evidence that she produced for the usefulness of altmetrics seemed pathetic to me.

7 May 2014 A much-shortened version of this post appeared in the British Medical Journal (BMJ blogs)

Tagged Academia, acupuncture, altmetrics, badscience, bibliobollocks, bibliometrics, open access, peer review, publication, regulation | 20 Comments

Blogs lead in critical thinking, but newspapers still matter

Published June 10, 2013

Jump to follow-up

Here is a record of a couple of recent newspaper pieces. Who says the mainstream media don’t matter any longer? Blogs may be in the lead now when it comes to critical analysis. The best blogs have more expertise and more time to read the sources than journalists. But the mainstream media get the message to a different, and much larger, audience.

The Observer ran a whole page interview with me as part of their “Rational Heroes” series. I rather liked their subtitle [pdf of article]

“Professor of pharmacology David Colquhoun is the take-no-prisoners debunker of pseudoscience on his unmissable blog”

It was pretty accurate apart from the fact that the picture was labelled as “DC in his office”. Actually it was taken (at the insistence of the photographer) in Lucia Sivilotti’s lab.

DC-Observer
Photo by Karen Robinson.

The astonishing result of this was that on Sunday the blog got a record 24,305 hits. Normally it gets 1,000-1,400 hits a day . between posts, fewer on Sunday, and the previous record was around 7000/day

hits

A week later it was still twice normal. It remains to be seen whether the eventual plateau stays up.

I also gained around 1000 extra followers on twitter, though some dropped away quite soon, and 100 or so people signed for email updates. The dead tree media aren’t yet dead. I’m happy to say.

3 June 2013

Perhaps as a result of the foregoing piece, I got asked to write a column for The Observer, at barely 48 hours notice. This is the best I could manage in the time. The web version has links.

This attracted the usual "it worked for me" anecdotes in the comments, but I spent an afternoon answering them. It seems important to have a dialogue, not just to lecture the public. In fact when I read a regular scientific paper, I now find myself looking for the comment section. That may say something about the future of scientific publishing.

It is for others to judge how succesfully I engage with the public, but I was quite surprised to discover that UCL’s public engagement unit, @UCL_in_public, has blocked me on twitter. Hey ho. They have 1574 follower and I have 7597. I wish them the best of luck.

Follow-up

Tagged Academia, Anti-science, antiscience, badscience, CAM, cancer, communication, public engagement, quackademia, science communication, Universities | 4 Comments

Acupuncture is a theatrical placebo: the end of a myth

Published May 30, 2013

Jump to follow-up

Anesthesia & Analgesia is the official journal of the International Anesthesia Research Society. In 2012 its editor, Steven Shafer, proposed a head-to-head contest between those who believe that acupuncture works and those who don’t. I was asked to write the latter. It has now appeared in June 2013 edition of the journal [download pdf]. The pro-acupuncture article written by Wang, Harris, Lin and Gan appeared in the same issue [download pdf].

Acupuncture is an interesting case, because it seems to have achieved greater credibility than other forms of alternative medicine, despite its basis being just as bizarre as all the others. As a consequence, a lot more research has been done on acupuncture than on any other form of alternative medicine, and some of it has been of quite high quality. The outcome of all this research is that acupuncture has no effects that are big enough to be of noticeable benefit to patients, and it is, in all probablity, just a theatrical placebo.

After more than 3000 trials, there is no need for yet more. Acupuncture is dead.

aa1

aa2

Acupuncture is a theatrical placebo

David Colquhoun (UCL) and Steven Novella (Yale)

Anesthesia & Analgesia, June 2013 116:1360-1363.

Pain is a big problem. If you read about pain management centres you might think it had been solved. It hasn’t. And when no effective treatment exists for a medical problem, it leads to a tendency to clutch at straws. Research has shown that acupuncture is little more than such a straw.

Although it is commonly claimed that acupuncture has been around for thousands of years, it hasn’t always been popular even in China. For almost 1000 years it was in decline and in 1822 Emperor Dao Guang issued an imperial edict stating that acupuncture and moxibustion should be banned forever from the Imperial Medical Academy.

Acupuncture continued as a minor fringe activity in the 1950s. After the Chinese Civil War, the Chinese Communist Party ridiculed traditional Chinese medicine, including acupuncture, as superstitious. Chairman Mao Zedong later revived traditional Chinese Medicine as part of the Great Proletarian Cultural Revolution of 1966 (Atwood, 2009). The revival was a convenient response to the dearth of medically-trained people in post-war China, and a useful way to increase Chinese nationalism. It is said that Chairman Mao himself preferred Western medicine. His personal physician quotes him as saying “Even though I believe we should promote Chinese medicine, I personally do not believe in it. I don’t take Chinese medicine” Li {Zhisui Li. Private Life Of Chairman Mao: Random House, 1996}.

The political, or perhaps commercial, bias seems to still exist. It has been reported by Vickers et al. (1998) (authors who are sympathetic to alternative medicine) that

"all trials [of acupuncture] originating in China, Japan, Hong Kong, and Taiwan were positive"(4).

Acupuncture was essentially defunct in the West until President Nixon visited China in 1972. Its revival in the West was largely a result of a single anecdote promulgated by journalist James Reston in the New York Times, after he’d had acupuncture in Beijing for post-operative pain in 1971. Despite his eminence as political journalist, Reston had no scientific background and evidently didn’t appreciate the post hoc ergo propter hoc fallacy, or the idea of regression to the mean.

After Reston’s article, acupuncture quickly became popular in the West. Stories circulated that patients in China had open heart surgery using only acupuncture (Atwood, 2009). The Medical Research Council (UK) sent a delegation, which included Alan Hodgkin, to China in 1972 to investigate these claims , about which they were skeptical. In 2006 the claims were repeated in 2006 in a BBC TV program, but Simon Singh (author of Fermat’s Last Theorem) discovered that the patient had been given a combination of three very powerful sedatives (midazolam, droperidol, fentanyl) and large volumes of local anaesthetic injected into the chest. The acupuncture needles were purely cosmetic.

Curiously, given that its alleged principles are as bizarre as those on any other sort of pre-scientific medicine, acupuncture seemed to gain somewhat more plausibility than other forms of alternative medicine. The good thing about that is that more research has been done on acupuncture than on just about any other fringe practice.

The outcome of this research, we propose, is that the benefits of acupuncture, if any, are too small and too transient to be of any clinical significance. It seems that acupuncture is little or no more than a theatrical placebo. The evidence for this conclusion will now be discussed.

Three things that are not relevant to the argument

There is no point in discussing surrogate outcomes such as fMRI studies or endorphine release studies until such time as it has been shown that patients get a useful degree of relief. It is now clear that they don’t.

There is also little point in invoking individual studies. Inconsistency is a prominent characteristic of acupuncture research: the heterogeneity of results poses a problem for meta-analysis. Consequently it is very easy to pick trials that show any outcome whatsoever. Therefore we shall consider only meta-analyses.

The argument that acupuncture is somehow more holistic, or more patient-centred, than medicine seems us to be a red herring. All good doctors are empathetic and patient-centred. The idea that empathy is restricted to those who practice unscientific medicine seems both condescending to doctors, and it verges on an admission that empathy is all that alternative treatments have to offer.

There is now unanimity that the benefits, if any, of acupuncture for analgesia, are too small to be helpful to patients.

Large multicenter clinical trails conducted in Germany {Linde et al., 2005; Melchart et , 2005; Haake et al, 2007, Witt et al, 2005), and in the United States {Cherkin et al, 2009) consistently revealed that verum (or true) acupuncture and sham acupuncture treatments are no different in decreasing pain levels across multiple chronic pain disorders: migraine, tension headache, low back pain, and osteoarthritis of the knee.

If, indeed, sham acupuncture is no different from real acupuncture the apparent improvement that may be seen after acupuncture is merely a placebo effect. Furthermore it shows meridians don’t exist, so the "theory" memorized by qualified acupuncturists is just myth. All that remains to be discussed is whether or not the placebo effect is big enough to be useful, and whether it is ethical to prescribe placebos.

Some recent meta-analyses have found that there may be a small difference between sham and real acupuncture. Madsen Gøtzsche & Hróbjartsson {2009) looked at thirteen trials with 3025 patients, in which acupuncture was used to treat a variety of painful conditions. There was a small difference between ‘real’ and sham acupuncture (it didn’t matter which sort of sham was used), and a somewhat bigger difference between the acupuncture group and the no-acupuncture group. The crucial result was that even this bigger difference corresponded to only a 10 point improvement on a 100 point pain scale. A consensus report (Dworkin, 2009) that a change of this sort should be described as a “minimal” change or “little change”. It isn’t big enough for the patient to notice much effect.

The acupuncture and no-acupuncture groups were, of course, not blind to the patients and neither were they blind to the practitioner giving the treatment. It isn’t possible to say whether the observed difference is a real physiological action or whether it’s a placebo effect of a rather dramatic intervention. Interesting though it would be to know this, it matters not a jot, because the effect just isn’t big enough to produce any tangible benefit.

Publication bias is likely to be an even greater problem for alternative medicine than it is for real medicine, so it is particularly interesting that the result just described has been confirmed by authors who practise, or sympathise with, acupuncture. Vickers et al. (2012) did a meta-analysis for 29 RCTs, with 17,922 patients. The patients were being treated for a variety of chronic pain conditions. The results were very similar to those of Madsen et al.{2009). Real acupuncture was better than sham, but by a tiny amount that lacked any clinical significance. Again there was a somewhat larger difference in the non-blind comparison of acupuncture and no-acupuncture, but again it was so small that patients would barely notice it.

Comparison of these two meta-analyses shows how important it is to read the results, not just the summaries. Although the outcomes were similar for both, the spin on the results in the abstracts (and consequently the tone of media reports) was very different.

An even more extreme example of spin occurred in the CACTUS trial of acupuncture for " ‘frequent attenders’ with medically unexplained symptoms” (Paterson et al., 2011). In this case, the results showed very little difference even between acupuncture and no-acupuncture groups, despite the lack of blinding and lack of proper controls. But by ignoring the problems of multiple comparisons the authors were able to pick out a few results that were statistically significant, though trivial in size. But despite this unusually negative outcome, the result was trumpeted as a success for acupuncture. Not only the authors, but also their university’s PR department and even the Journal editor issued highly misleading statements. This gave rise to a flood of letters to the British Journal of General Practice and much criticism on the internet.

From the intellectual point of view it would be interesting to know if the small difference between real and sham acupuncture found in some, but not all, recent studies is a genuine effect of acupuncture or whether it is a result of the fact that the practitioners are never blinded, or of publication bias. But that knowledge is irrelevant for patients. All that matters for them is whether or not they get a useful degree of relief.

There is now unanimity between acupuncturists and non-acupuncturists that any benefits that may exist are too small to provide any noticeable benefit to patients. That being the case it’s hard to see why acupuncture is still used. Certainly such an accumulation of negative results would result in the withdrawal of any conventional treatment.

Specific conditions

Acupuncture should, ideally, be tested separately for effectiveness for each individual condition for which it has been proposed (like so many other forms of alternative medicine, that’s a very large number). Good quality trials haven’t been done for all of them. It’s unlikely that acupuncture works for rheumatoid arthritis, stopping smoking, irritable bowel syndrome or for losing weight. And there is no good reason to think it works for addictions, asthma, chronic pain, depression, insomnia, neck pain, shoulder pain or frozen shoulder, osteoarthritis of the knee, sciatica, stroke or tinnitus and many other conditions (Ernst et al., 2011).

In 2009, the UK’s National Institute for Clinical Excellence (NICE) did recommend both acupuncture and chiropractic for back pain. This exercise in clutching at straws caused something of a furore. In the light of NICE’s judgement the Oxford Centre for Evidence-based medicine updated its analysis of acupuncture for back pain. Their verdict was

“Clinical bottom line. Acupuncture is no better than a toothpick for treating back pain.”

The paper by Artus et al. (2010) is of particular interest for the problem of back pain. Their Fig 2 shows that there is a modest improvement in pain scores after treatment, but much the same effect, with the same time course is found regardless of what treatment is given, and even with no treatment at all. They say

“we found evidence that these responses seem to follow a common trend of early rapid improvement in symptoms that slows down and reaches a plateau 6 months after the start of treatment, although the size of response varied widely. We found a similar pattern of improvement in symptoms following any treatment, regardless of whether it was index, active comparator, usual care or placebo treatment”.

It seems that most of what’s being seen is regression to the mean. And that is very likely to be the main reason why acupuncture sometimes appears to work when it doesn’t.

Although the article by Wang et al (2012) was written to defend the continued use of acupuncture, the only condition for which they claim that there is any reasonably strong evidence is for post-operative nausea and vomiting (PONV). It would certainly be odd if a treatment that had been advocated for such a wide variety of conditions turned out to work only for PONV. Nevertheless, let’s look at the evidence.

The main papers that are cited to support the efficacy of acupuncture in alleviation of PONV are all from the same author: Lee & Done (1999), and two Cochrane reviews, Lee & Done (2004), updated in Lee & Fan (2009). We need only deal with this latest updated meta-analysis.

Although the authors conclude “P6 acupoint stimulation prevented PONV”, closer examination shows that this conclusion is very far from certain. Even taken at face value, a relative risk of 0.7 can’t be described as “prevention”. The trials that were included were not all tests of acupuncture but included several other more or less bizarre treatments (“acupuncture, electro-acupuncture, transcutaneous nerve stimulation, laser stimulation, capsicum plaster, an acu-stimulation device, and acupressure”). The number needed to treat varied from a disastrous 34 to a poor 5 for patients with control rates of PONV of 10% and 70% respectively.

The meta-analysis showed, on average, similar effectiveness for acupumcture and anti-emetic drugs. The problem is that the effectiveness of drugs is in doubt because an update to the Cochrane review has been delayed (Carlisle, 2012) by the discovery of major fraud by a Japanese anesthetist, Yoshitaka Fujii (Sumikawa, 2012). It has been suggested that metclopramide barely works at all (Bandolier, 2012; Henzi, 1999).

Of the 40 trials (4858 participants) that were included; only four trials reported adequate allocation concealment. Ninety percent of trials were open to bias from this source. Twelve trials did not report all outcomes. The opportunities for bias are obvious. The authors themselves describe all estimates as being of “Moderate quality” which is defined this: “Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate”. That being the case, perhaps the conclusion should have been “more research needed”. In fact almost all trials of alternative medicines seem to end up with the conclusion that more research is needed.

Conclusions

It is clear from meta-analyses that results of acupuncture trials are variable and inconsistent, even for single conditions. After thousands of trials of acupuncture, and hundreds of systematic reviews (Ernst et al., 2011), arguments continue unabated. In 2011, Pain carried an editorial which summed up the present situation well.

“Is there really any need for more studies? Ernst et al. (2011) point out that the positive studies conclude that acupuncture relieves pain in some conditions but not in other very similar conditions. What would you think if a new pain pill was shown to relieve musculoskeletal pain in the arms but not in the legs? The most parsimonious explanation is that the positive studies are false positives. In his seminal article on why most published research findings are false, Ioannidis (2005) points out that when a popular but ineffective treatment is studied, false positive results are common for multiple reasons, including bias and low prior probability.”

Since it has proved impossible to find consistent evidence after more than 3000 trials, it is time to give up. It seems very unlikely that the money that it would cost to do another 3000 trials would be well-spent.

A small excess of positive results after thousands of trials is most consistent with an inactive intervention. The small excess is predicted by poor study design and publication bias. Further, Simmons et al (2011) demonstrated that exploitation of "undisclosed flexibility in data collection and analysis" can produce statistically positive results even from a completely nonexistent effect. With acupuncture in particular there is documented profound bias among proponents (Vickers et al., 1998). Existing studies are also contaminated by variables other than acupuncture – such as the frequent inclusion of "electroacupuncture" which is essentially transdermal electrical nerve stimulation masquerading as acupuncture.

The best controlled studies show a clear pattern – with acupuncture the outcome does not depend on needle location or even needle insertion. Since these variables are what define "acupuncture" the only sensible conclusion is that acupuncture does not work. Everything else is the expected noise of clinical trials, and this noise seems particularly high with acupuncture research. The most parsimonious conclusion is that with acupuncture there is no signal, only noise.

The interests of medicine would be best-served if we emulated the Chinese Emperor Dao Guang and issued an edict stating that acupuncture and moxibustion should no longer be used in clinical practice.

No doubt acupuncture will continue to exist on the High Streets where they can be tolerated as a voluntary self-imposed tax on the gullible (as long as they don’t make unjustified claims).

REFERENCES
1. Acupuncture Centre. . About Acupuncture. Available at: http://www.acupuncturecentre.org/aboutacupuncture.html. Accessed March 30, 2013
2. Atwood K. “Acupuncture Anesthesia”: a Proclamation from Chairman Mao (Part IV). Available at: http://www.sciencebasedmedicine.org/index.php/acupuncture-anesthesia-a-proclamation-from-chairman-mao-part-iv/. Accessed September 2, 2012
3. Li Z Private Life of Chairman Mao: The Memoirs of Mao’s Personal Physician. 1996 New York: Random House
4. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19:159–66 Available at: http://bit.ly/WqVGWN. Accessed September 2, 2012
5. Reston J. Now, About My Operation in Peking; Now, Let Me Tell You About My Appendectomy in Peking … The New York Times. 1971 Available at: http://select.nytimes.com/gst/abstract.html?res=FB0D11FA395C1A7493C4AB178CD85F458785F9. Accessed March 30, 2013
6. Atwood K. “Acupuncture anesthesia”: a proclamation from chairman Mao (part I). Available at: http://www.sciencebasedmedicine.org/index.php/acupuncture-anesthesia-a-proclamation-of-chairman-mao-part-i/. Accessed September 2, 2012
7. Linde K, Streng A, Jürgens S, Hoppe A, Brinkhaus B, Witt C, Wagenpfeil S, Pfaffenrath V, Hammes MG, Weidenhammer W, Willich SN, Melchart D. Acupuncture for patients with migraine: a randomized controlled trial. JAMA. 2005;293:2118–25
8. Melchart D, Streng A, Hoppe A, Brinkhaus B, Witt C, Wagenpfeil S, Pfaffenrath V, Hammes M, Hummelsberger J, Irnich D, Weidenhammer W, Willich SN, Linde K. Acupuncture in patients with tension-type headache: randomised controlled trial. BMJ. 2005;331:376–82
9. Haake M, Müller HH, Schade-Brittinger C, Basler HD, Schäfer H, Maier C, Endres HG, Trampisch HJ, Molsberger A. German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups. Arch Intern Med. 2007;167:1892–8
10. Witt C, Brinkhaus B, Jena S, Linde K, Streng A, Wagenpfeil S, Hummelsberger J, Walther HU, Melchart D, Willich SN. Acupuncture in patients with osteoarthritis of the knee: a randomised trial. Lancet. 2005;366:136–43
11. Cherkin DC, Sherman KJ, Avins AL, Erro JH, Ichikawa L, Barlow WE, Delaney K, Hawkes R, Hamilton L, Pressman A, Khalsa PS, Deyo RA. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009;169:858–66
12. Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ. 2009;338:a3115
13. Dworkin RH, Turk DC, McDermott MP, Peirce-Sandner S, Burke LB, Cowan P, Farrar JT, Hertz S, Raja SN, Rappaport BA, Rauschkolb C, Sampaio C. Interpreting the clinical importance of group differences in chronic pain clinical trials: IMMPACT recommendations. Pain. 2009;146:238–44
14. Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, Sherman KJ, Witt CM, Linde K. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172:1444–53
15. Paterson C, Taylor RS, Griffiths P, Britten N, Rugg S, Bridges J, McCallum B, Kite G. Acupuncture for ‘frequent attenders’ with medically unexplained symptoms: a randomised controlled trial (CACTUS study). Br J Gen Pract. 2011;61:e295–e305
16. . Letters in response to Acupuncture for ‘frequent attenders’ with medically unexplained symptoms. Br J Gen Pract. 2011;61 Available at: http://www.ingentaconnect.com/content/rcgp/bjgp/2011/00000061/00000589. Accessed March 30, 2013
17. Colquhoun D. Acupuncturists show that acupuncture doesn’t work, but conclude the opposite: journal fails. 2011 Available at: https://www.dcscience.net/?p=4439. Accessed September 2, 2012
18. Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. Pain. 2011;152:755–64
19. Colquhoun D. NICE falls for Bait and Switch by acupuncturists and chiropractors: it has let down the public and itself. 2009 Available at: https://www.dcscience.net/?p=1516. Accessed September 2, 2012
20. Colquhoun D. The NICE fiasco, part 3. Too many vested interests, not enough honesty. 2009 Available at: https://www.dcscience.net/?p=1593. Accessed September 2, 2012
21. Bandolier. . Acupuncture for back pain—2009 update. Available at: http://www.medicine.ox.ac.uk/bandolier/booth/painpag/Chronrev/Other/acuback.html. Accessed March 30, 2013
22. Artus M, van der Windt DA, Jordan KP, Hay EM. Low back pain symptoms show a similar pattern of improvement following a wide range of primary care treatments: a systematic review of randomized clinical trials. Rheumatology (Oxford). 2010;49:2346–56
23. Wang S-M, Harris RE., Lin Y-C, Gan TJ. Acupuncture in 21st century anesthesia: is there a needle in the haystack? Anesth Analg. 2013;116:1356–9
24. Lee A, Done ML. The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesth Analg. 1999;88:1362–9
25. Lee A, Done ML. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2004:CD003281
26. Lee A, Fan LT. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2009:CD003281
27. Carlisle JB. A meta-analysis of prevention of postoperative nausea and vomiting: randomised controlled trials by Fujii etal. compared with other authors. Anaesthesia. 2012;67:1076–90
28. Sumikawa K. The results of investigation into Dr.Yoshitaka Fujii’s papers. Report of the Japanese Society of Anesthesiologists Special Investigation Committee. http://www.anesth.or.jp/english/pdf/news20120629.pdf
29. Bandolier. . Metoclopramide is ineffective in preventing postoperative nausea and vomiting. Available at: http://www.medicine.ox.ac.uk/bandolier/band71/b71-8.html. Accessed March 30, 2013
30. Henzi I, Walder B, Tramèr MR. Metoclopramide in the prevention of postoperative nausea and vomiting: a quantitative systematic review of randomized, placebo-controlled studies. Br J Anaesth. 1999;83:761–71
31. Hall H. Acupuncture’s claims punctured: not proven effective for pain, not harmless. Pain. 2011;152:711–2
32. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2:e124
33. Simmons JP, Leif DN, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22:1359–66

Follow-up

30 May 2013 Anesthesia & Analgesia has put the whole paper on line. No paywall now!

9 June 2013. Since this page was posted on May 30, it has had over 20,000 page views. Not bad.

26 July 2013. The Observer had a large double-page spread about acupuncture. It was written by David Derbyshire, largely on the basis of this article.

26 December 2013

Over christmas the flow of stuff that misrepresents the "thousands of years" of Chinese medicine has continued unabated. Of course one expects people who are selling Chinese herbs and acupuncture to lie. All businesses do. One does not expect such misrepresentation from British Columbia, Cardiff University School of medicine, or from Yale University. I left a comment on the Yale piece. Whether it passes moderation remains to be seen. Just in case, here it is.

One statement is undoubtedly baseless ““If it’s still in use after a thousand years there must be something right,” It’s pretty obvious to the most casual observer that many beliefs that have been round for a thousand years have proved to be utterly wrong.

In any case, it’s simply not true that most “Traditional” Chinese medicine has been around for thousands of years. Acupuncture was actually banned by the Emperor Dao Guang in 1822. The sort of Chinese medicine that is sold (very profitably) to the west was essentially dead in China until it was revived by Mao as part of the great proletarian cultural revolution (largely to stir up Chinese nationalism at that time). Of course he didn’t use it himself.

This history has been documented in detail now, and it surprises me to see it misrepresented, yet again, from a Yale academic.

Of course there might turn out to be therapeutically useful chemicals in Chinese herbs (it has happened with artemesinin). But it is totally irresponsible to pretend that great things are coming in the absence of good RCTs in human patients.

Yale should be ashamed of PR like this. And so should Cardiff University. It not only makes the universities look silly. It corrupts the whole of the rest of these institutions. Who knows how much more of their PR is mere puffery.

18 January 2014. I checked the Yale posting and found that the comment, above, had indeed been deleted. There is little point in having comments if you are going to delete anything that’s mildly critical. It is simply dishonest.

Tagged Academia, acupuncture, badscience, CAM, quackademia, quackery | 63 Comments

The British Pharmacological Society and the MHRA don’t help the cause of honest science. They hinder it.

Published February 27, 2013

Jump to follow-up

The bulletin of the British Pharmacological Society, Pharmacology Matters, declined to publish the following article. Sadly the Society seems to be more interested in "reputation management" than in truth. Luckily, it is not easy to suppress criticism these days. A version of the article has appeared in Research Fortnight where it will be seen by far more people than it would have been in Pharmacology Matters. This is the original version that I submitted to them. They would not allow me to quote Lewis’s comment (apropos of the sale of homeopathic meningitis vaccine)

“Children will be harmed by this inaction. Children will die. And the fault must lie with Professor Sir Kent Woods, chairman of the regulator [MHRA].”

If a child were to die of whooping cough or meningitis as a result of buying the fraudulent "vaccines", that would be true. It’s a sad reflection on the state of defamation law that journals are not willing to say so. Blogs are fast becoming the best source of reliable information.

Stop press. The BPS has now signed up to Alltrials (too late for the printed version)

Pharmacology society does little to defend its subject

David Colquhoun

Over the past few years a courageous group of writers, researchers and activists has worked to expose the truth about the medicines we are sold, be they conventional or alternative.

Thanks, above all, to Ben Goldacre (1), more people than ever know that the big pharma companies have been concealing evidence of the harm that their products do, or the good they fail to do. Thanks to a small army of bloggers the preposterous claims made by peddlers of homeopathic remedies and other quackery are less likely to go unchallenged.

And yet, the organization charged with safeguarding the good name of pharmacology in this country, the British Pharmacological Society (BPS), has remained silent throughout.

The pharmaceutical industry

The BPS, rather than helping, became actively complicit when, along with much of the medical establishment, it signed an agreement with the ABPI (2). This document, developed under the aegis of the Ethical Standards in Health & Life Sciences Group (ESHLSG), contained two objectionable clauses:

“Industry plays a valid and important role in the provision of medical education”

And

“Medical representatives can be a useful resource for healthcare professionals”

Given that clinical “education” has long been part of Pharma’s marketing strategy, this seems disgraceful. And most of the doctors I know and respect refuse to see reps altogether. It is hopelessly optimistic to think that can an industry person can teach clinical pharmacology without bias in favour of his own company’s products. The BPS has many members who teach pharmacology. Can they really cope so badly that we need to have industry to educate clinicians?

It’s fine, and sometimes desirable, for academics and industry to work together on drug development. But only as long as the industry partner has no say in how, or whether, the results are published. Without that proviso we can expect more corruption of the sort that’s been seen at the University of Sheffield (3).

This is very sad, because I have great reason to like the drug industry. I’ve benefitted from several of their products myself. But the industry is in trouble. Many of its products provide only marginal benefits. Furthermore, some of the things that seemed to be useful, like SSRI antidepressants, have turned out to be next to useless once hidden trials were revealed (4). The MHRA’s learning module on SSRIs doesn’t seem to have caught up with this yet.

Sadly, the reaction of industry has been to resort to dishonesty, to hide unfavourable data and to increase yet more what it spends on marketing. Between 2009 and 2012, fines of at least 10 billion dollars (5) have been imposed on some of the most eminent companies. They include Lilly, Pfizer, AstraZeneca, Merck, Abbott and GlaxoSmithKline (GSK). The biggest fine of all ($3 bn, in July 2012) went to a British company, GSK. This succession of large fines seems to be regarded by the companies as mere marketing expenses.

All these fines were levied in the USA. Where, one might ask, are the regulators in the UK? Why have there been no fines here? Why, indeed, are some of the senior managers of these companies not in jail? Why has the BPS remained silent about the prostitution of its subject? And why have the MHRA done so little to stop it?

I suggest that you support the petition for release of the results of all trials (6). It’s been supported by many individuals and a lot of organisations, including the BMJ and the Royal Statistical Society. But, disgracefully, not by the BPS.

Quackery

At least in the case of the pharmaceutical industry some of its products work. But pharmacologists should also be concerned about the quackery industry, worth about 60 billion dollars per year (as opposed to $600 bn for the pharmaceutical industry). Virtually none of their products work (7). Why has the BPS said so little about it? It has, along with most of the medical and university establishment, shrugged its shoulders about the fact that students at Westminster University have been shown dowsing with a pendulum as a method for selection of herbal “remedies”, as part of a Bachelor or “Science” degree. It is an area in which every regulatory agency has failed to ensure even minimal levels of honesty (8). And the BPS has just shrugged.

The MHRA has been worse than useless in this area: it has been actively unhelpful (9). The senior staff of the MHRA are members of the BPS which has, as usual, said next to nothing. The MHRA’s herbal medicine committee has allowed misleading labels that give indications to be put on herbal potions, and these labels fail to make it clear that no evidence whatsoever of efficacy is required to get the MHRA kitemark. The wording was suggested (not required) by European law, but that law does not prevent the MHRA from saying, as it should, “there is no reason to think that this product is effective for any of the indications on the label” (10). Arguably, the MHRA is in breach of Consumer Protection law (11, 12).

At the time, the BPS did make some objection to the labelling (13), but only under great pressure from me (indeed I wrote it) . That has not been followed up, and I can no longer find it on the BPS web site. Indeed Philip Routledge, one of the people who is responsible for the misinformation in his capacity as chair of the MHRA Herbal Medicines Advisory Committee, is, at present the president of the BPS.

The MHRA has also been responsible for misleading labelling of the products of the most obviously fraudulent products of the lot: homeopathic pills, the medicines that contain no medicine. Most of the pills (anything beyond 12x dilution) contain not a single molecule of the substance on the label. Yet they have been given a get-out clause that enables them to evade prosecution by Trading Standards (an organisation that consistently fails to apply consumer protection laws. Rose et al (2011) (12) concluded

"EU directive 200s5/29/EC is largely ineffective in preventing misleading health claims for consumer products in the UK".

It is simply bizarre that the people at the MHRA, many of who are BPS members, have sat round a table and approved the following label. This example is for Arnica 30C pills, which, of course, contain no trace of arnica (14, 8). The outcome of their deliberations is simply surreal (see the actual labels here).

"Used within the homeopathic tradition for the symptomatic relief of sprains, muscular ache; and bruising or swelling after contusions."

This will deceive the naïve into thinking that it will have some effect on sprains etc. It won’t. And the MHRA have declined to test how the label is perceived by the public, though it took some effort to get them to admit it.

"If you are allergic to any of the ingredients in this medicine, consult your doctor before taking this medicine."

The ingredients aren’t stated apart from “contains lactose and sucrose”. That’s all they contain. No arnica.

"If pregnant or breastfeeding consult your doctor before use."

Why should a few mg of lactose and sucrose have the slightest effect on a pregnant or breast-feeding mother. This is pure make-believe

"If you forget to take this product, continue to take your usual dose at the usual time, it does not matter if you have missed a dose. Do not take a double dose to make up for a missed dose."

This statement is even more bizarre. There is nothing in the pills.

"If you take too much of this product (overdose) speak to a doctor or pharmacist and take this label with you."

And this is the ultimate in nonsense. The 1023 campaign regularly swallows whole bottles, and of course nothing happens. You can’t overdose on nothing. The fact that the MHRA can insist on this label, with a straight face, is the ultimate betrayal of science and reason.

"When asked to comment, as part of the consultation on these rules, this was the response from the BPS."

no comment

This is extracted from page 16 of the “selected response” provided by the MHRA under a Freedom of Information Act request [download all]

Things have changed little since A.J. Clark wrote his book on Patent Medicines in 1938 (15). And the BPS has done next to nothing to help. Neither has the MHRA. In fact both have colluded in the failures of both honesty and reason.

A BBC South West program recently revealed that a pharmacist was selling “homeopathic vaccines” for whooping cough and meningitis (16). The MHRA have know about his homicidal practice for years, but have done nothing. The General Pharmaceutical Council let him off with a rap on the knuckles. It has been left to bloggers and TV reporters to focus attention on these scoundrels. The well-respected blogger, Andy Lewis, wrote (17)

“Children will be harmed by this inaction. Children will die. And the fault must lie with Professor Sir Kent Woods, chairman of the regulator [MHRA].”

And the full clinical data for Tamiflu are still being concealed by Roche (18).

I think that is rather shameful.

I have been a member of the BPS for all my working life. I was happy when they made me an honorary fellow. But I now find myself asking if I can remain a member of an organisation that has done so little to defend honest scientific behaviour.

References

(1) Ben Goldacre’s Bad Pharma. Buy it now. Then do something. https://www.dcscience.net/?p=5538

(2) ABPI 2012 Guidance on collaboration between healthcare professionals and the pharmaceutical industry. http://www.abpi.org.uk/our-work/library/guidelines/Pages/collaboration-guidance.aspx

(3) Colquhoun, D. (2007) The Corporate Corruption of Higher Education: part 2 https://www.dcscience.net/?p=193

(4) Kirsch,I., B.J.Deacon, T.B.Huedo-Medina, A.Scoboria, T.J.Moore, and B.T.Johnson. 2008. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS. Med. 5:e45. http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0050045

(5) Groeger, L. (2012) Big Pharma’s Big Fines http://www.propublica.org/special/big-pharmas-big-fines

(6) All trials registered. All results reported, http://www.alltrials.net/supporters/

(7) Singh,S. and E.Ernst. 2009. Trick or Treatment. Corgi. http://en.wikipedia.org/wiki/Trick_or_Treatment

(8) Colquhoun, D. (2012) Regulation of alternative medicine: why it doesn’t work, and never can. https://www.dcscience.net/?p=5562

(9) Colquhoun, D, (2006) The MHRA breaks its founding principle: it is an intellectual disgrace. https://www.dcscience.net/?p=32

(10) Colquhoun, D. (2011). Why does the MHRA refuse to label herbal products honestly? Kent Woods and Richard Woodfield tell me. https://www.dcscience.net/?p=4269

(11) Colquhoun, D. (2009) Most alternative medicine is illegal. https://www.dcscience.net/?p=30

(12) Rose,L.B., P.Posadzki, and E.Ernst. 2012. Spurious claims for health-care products: an experimental approach to evaluating current UK legislation and its implementation. Med. Leg. J. 80:13-18. https://www.dcscience.net/Rose-medico-legal-2012.pdf

(13) Colquhoun, D. (2006) Learned Societies speak out against CAM, and the MHRA. https://www.dcscience.net/?p=30

(14) MHRA Arnicare Arnica 30c pillules NR 01175/0181 http://www.mhra.gov.uk/home/groups/par/documents/websiteresources/con049307.pdf

(15) Colquhoun, D. (2008) Patent medicines in 1938 and now: A.J.Clark’s book. https://www.dcscience.net/?p=257

(16) BBC South West on the evils of homeopathic "vaccines" http://www.youtube.com/watch?v=TZf9mUzI4RI

(17) Why Does the MHRA Not Close Down these Homeopaths? The regulators have known of serious problems for years, Nothing is done. http://www.quackometer.net/blog/2013/01/why-does-the-mhra-not-close-down-these-homeopaths.html

(18) Payne, D. (2012). Tamiflu: the battle for secret drug data http://www.bmj.com/content/345/bmj.e7303

Follow-up

25 February 2013 This post has some follow-up even before it appeared on Research Research. I noticed on the BPS web site a press release “BPS announces intention to sign All Trials Petition“. It was dated 20th February, but I didn’t notice it until after the printed edition went to press. It was expressed as a future intention to sign, though in fact they signed almost straight away (though over 100 organisations had already done so). That’s good. I suspect that when my old friend, Humphrey Rang, who is president elect of the BPS, takes charge, the Society may start to take its responsibilities to the public more seriously than it has in the past.

The MHRA, on the other hand, is still evading its self-declared job of ensuring the public that drugs work.

13 March 2013. A reply to this piece appeared in red on Research Research, The British Pharmacological Society champions its science. It was written by Humphrey Rang who, as president elect of the BPS naturally felt obliged to defend its record. He defends the BPS membership of ESHLSG, but fails to mention that first the Lancet and then the BMA withdrew their support. Nor does he mention that medical students and doctors launched a campaign, BadGuidelines.org, against the agreement. The Medical Schools Council, which also signed the agreement, said "the scrutiny of the guidance has ‘identified deficiencies in the current statement". Didn’t they read it before signing? Rang says that the BPS is also working to improve the joint statement with the ABPI. That’s good, but one must wonder why the BPS signed up to the original form.

On the points about quackery, Rang sites the BPS statement on homeopathy (now vanished) but fails to mention that that statement was written by me in an attempt to wake the BPS from it’s slumbers on the matter of medicines that don’t work. But he doesn’t mention at all the matter of mislabelling of both homeopathic and herbal preparations.

Of course, the BPS does many good things. But like most organisations, it is too reluctant to speak out when it sees wrongdoing, and too reluctant to say "sorry we made a cock-up that time".

Tagged arnica, badscience, Big Pharma, BPS, British Pharmacological Society, CAM, Kent Woods, Medicines and Health Regulatory Authority, MHRA, Phillip Routledge, regulation, Richard Eastell | 9 Comments

Academic role model? Moi?

Published October 26, 2012

Nobody could have been more surprised than I when I found myself nominated as an academic role model at UCL. I had to answer a few questions. It is not obvious to me what the object of the stunt is, but the person who asked me to do it seemed to find the answers amusing, so I’ll reproduce here what I said. I apologise for the temporary lapse into narcissism.

The final version has now been printed [download a copy]. Sadly the printed edition was “corrected” by someone who replaced “whom I asked to submit the first theoretical paper by Hawkes and me to the Royal Society” (as written below), with “paper by Hawkes and I”. Aaargh.

Your nomination – Why you were nominated as an Academic Role Model?

"David Colquhoun has made major contributions to our understanding of how ion channels (proteins which allow charged ions to pass across cell membranes) function to mediate electrical signalling in nerve and muscle cells. This work elegantly combines experimental and theoretical aspects, and resulted in David being made a Fellow of the Royal Society. David Colquhoun played a key role in resisting the notion that UCL should merge with Imperial College in 2002, by running a website opposed to the merger. He thus facilitated the continued existence of an independent UCL. He is also well-known for his principled opposition to therapies that are not based on scientific evidence, and for his blog which comments on this issue as well as on university bureaucracy and politics."

Role models’ questions

1. What is your response to being nominated?

Astounded.

We are interested in giving people a very brief ‘snapshot’ description of their career trajectory, to help a broad range of people see how you got to where you are:

2. What has your career path been?

My first job (in 1950s) was as an apprentice pharmacist in Timothy Whites & Taylors (Homeopathic Chemists) in Grange Road, Birkenhead. You can’t get a more humble start than that. But it got me interested in drugs, and thanks to my schoolmaster father, I got to the University of Leeds.

My father (1907 – 2001), in 1955

One of the courses involved some statistics, and that interested me. I think I made a semi-conscious decision that it would be sensible to be good at something that others were bad at, so I learned quite a lot of statistics and mathematics. I recall buying a Methuen’s Monograph on Determinants and Matrices in my final year, and, with the help of an Argentinian PhD student in physical chemistry (not my lecturers) I began to make sense of it.

I purposely went into my final viva with it sticking out of my pocket. The examiner was Walter Perry, then professor of Pharmacology in Edinburgh (he later did a great job setting up the Open University). That’s how I came to be a PhD student in Edinburgh.

Although Perry was one of my supervisors, the only time I saw him was when he came into my lab between committee meetings for a cigarette. But he did make me an honorary lecturer so I could join the Staff Club, where I made many friends, including a young physics lecturer called Peter Higgs. The staff club exists no longer, having been destroyed in one of those acts of short-sighted academic vandalism that vice-chancellors seem so fond of.

The great university expansion in the 1960s made it easy to get a job. The most famous pharmacology department in the world was at UCL so I asked someone to introduce me to its then head, Heinz Schild, and asked him if he had a job. While interned during WW2 he had written a paper on the statistics of biological assay and wanted someone to teach it to students, so I got a job (in 1964), and have been at UCL ever since apart from 9 years. Between 1964 and 1970 I published little, but learned a great deal by writing a textbook on statistics.

That sort of statistics is now thought too difficult for undergraduates, and the famous department that attracted me was itself destroyed in another act of academic vandalism, in 2007.

I have spent my life doing things that I enjoy. Such success as I’ve had, I attribute to a liking for spending time with people cleverer than I am, and wasting time drinking coffee. I found a very clever statistician, Alan Hawkes, in the Housman Room in the late 1960s, and we began to collaborate on the theory of single ion channel analysis in a series of papers that still isn’t quite finished. He did the hard mathematics, but I knew enough about it to write it up in a more or less comprehensible form and to write computer programs to evaluate the algebra. When I got stuck, I would often ask Hyman Kestelman (co-author of the famous mathematics textbook, Massie & Kestelman) to explain, usually in what was then the Joint Staff Common Room at lunch time (it is now the Haldane room, the common room having been confiscated by unenlightened management). Before leaving for the USA in 1970, I, in league with the then professor of French, Brian Woledge, eventually got through a motion that allowed women into the Housman room.

I’d also talk as much as I could to Bernard Katz, whom I asked to submit the first theoretical paper by Hawkes and me to the Royal Society. His comments on the first draft led to the published version making a prediction about single ion channel behaviour before channels could be observed.

The next step was sheer luck. As this was going on, two young Germans, Neher & Sakmann, succeeded in observing the tiny currents that flow through single ion channel molecules, so it became possible to test the theory. In series of visits to Göttingen, Sakmann and I did experiments late into the night. Neher & Sakmann got a well-deserved Nobel Prize in 1991, and I expect I benefitted from a bit of reflected glory

The work that I have done is nothing if not basic. It doesn’t fit in with the current vogue for translational research (most of which will fail), although I would regard it as laying the basis for rational drug design. My only regret is that rational drug design has proved to be so difficult that it won’t be achieved in my lifetime (please don’t believe the hype).

We’d also like you to take a slightly more personal view:

3. What have been the highs (and the lows?) of your career so far?

The highs have been the chance to work with brilliant people and write a handful of papers that have a chance of having a lasting influence. Because I have been able to take my time on those projects there haven’t been too many lows, apart from observing the continuous loss of academic integrity caused by the intense pressure to publish or perish, and the progressive decline in collegiality in universities caused by that pressure combined with the rise in power of managerialism. Luckily the advent of blogs has allowed me to do a little about that.

I’m saddened by the fact that the innumeracy of biologists that I noticed as an undergraduate has not really improved at all (though I don’t believe it is worse). Most biologists still have difficulty with even the simplest equations. Worse still, they don’t know enough maths to communicate their problem to a mathematician, so only too often one sees collaborations with mathematicians produce useless results.

The only real failure I’ve had was when, in a fit of vanity, I applied for the chair of Pharmacology in Oxford, in 1984, and failed to get it. But in retrospect that was really a success too. I would have hated the flummery of Oxford, and as head of department (an increasingly unattractive job) I would have spent my time on pushing paper, not ion channels. In retrospect, it was a lucky escape. UCL is my sort of place (most of the time).

We would like to hear what our role models have to say about the next generation:

4. What advice would you give to people finishing off their PhD?

My career course would be almost impossible now. In fact it is very likely that I would have been fired before I got going in the present climate. There were quite long periods when I didn’t publish much. I was learning the tools of my trade, both mathematical and experimental. Now there is no time to do that. You are under pressure to publish a paper a week (for the glory of your PI and your university) and probably rarely find time to leave the lab to talk to inspiring people. If you are given any courses they’ll probably be in some inane HR nonsense, not in algebra. That is one reason we started our summer workshop, though bizarrely that has now been dropped by the graduate school in favour of Advanced Powerpoint.

The plight of recent PhDs is dire. Too many are taken on (for the benefit of the university, not of the student) and there aren’t many academic jobs. If you want to stay in academia, all I can suggest is that you get good at doing something that other people can’t do, and to resist the pressure to publish dozens of trivial papers.

Try to maintain some academic integrity despite the many pressures to do the opposite that are imposed on you by your elders (but not always betters). That may or may not be enough to get you the job that you want, but at least you’ll be able to hold your head high.

Finally, we want to give a balanced impression of our role models because many were nominated for their ability to motivate others, and to balance life and work:

5. How do you keep motivated?

Work-life balance is much talked about by HR, though they are one of the reasons why it is now almost impossible, In the past it wasn’t a great problem. I’m fascinated by the problems that I’m trying to puzzle out. I’ve had periods of a year or two when things haven’t gone well and I’ve felt as though I was a failure, but luckily they haven’t lasted too long, and they occurred in a time before some idiotic performance manager would harass you for failing to publish for a year or two. The climate of “performance management” is doing a lot to kill innovation and creativity.

6. What do you do when are not working in SLMS?

I’ve had various phases. For a while I carried on boxing (which had been compulsory at school). When I was first at UCL in 1964 I bought a 21 foot sloop (and as a consequence could barely afford to eat), and in 1970 (at Yale) I learned to fly. I had a lot of fun sailing right up to the early 1980s, when I found I could not afford a son as well as a boat. That was when running came into fashion and that could be done for the price of a pair of shoes. I did marathons and half marathons for fun (the London in 1988 was great fun). And that was supplanted by walking country trails in the early 2000s.

There is never a clear division between work and play, especially with algebra. You can continue to struggle with a derivation on a boat, or even get a new angle on it while running. That, of course, is why the transparency review is such total nonsense.

The main cause of stress has never been work for me. Stress comes mainly from the imposition of dim-witted managerialism and incompetent HR policies. And that has become progressively worse. I doubt that if I were a young academic now I’d have the time to spend the weekend sailing.

I’m not sure whether the blogging that has taken up something like half my time since my nominal retirement in 2004 counts as work or not. It certainly depends on things that I have learned in my academic work. And it’s fun to have effects in the real world after a life spent on problems that many would regard as esoteric.

If you want a hobby that costs very little, and allows you to say what you want, start a blog.

Follow-up

Tagged Academia, alternative medicine, badscience, CAM, HR, Human resources, managerialism, quackademia, vice-chancellors | 8 Comments

Regulation of alternative medicine: why it doesn’t work, and never can

Published October 15, 2012

Jump to follow-up

The Scottish Universities Medical Journal asked me to write about the regulation of alternative medicine. It’s an interesting topic and not easy to follow because of the veritable maze of more than twenty overlapping regulators and quangos which fail utterly to protect the public against health fraud. In fact they mostly promote health fraud. The paper is now published, and here is a version with embedded links (and some small updates).

We are witnessing an increasing commercialisation of medicine. It’s really taken off since the passage of the Health and Social Security Bill into law. Not only does that mean having NHS hospitals run by private companies, but it means that “any qualified provider” can bid for just about any service. The problem lies, of course, in what you consider “qualified” to mean. Any qualified homeopath or herbalist will, no doubt, be eligible. University College London Hospital advertised for a spiritual healer. The "person specification" specified a "quallfication", but only HR people think that a paper qualification means that spiritual healing is anything but a delusion.

uclh-spirit

The vocabulary of bait and switch

First, a bit of vocabulary. Alternative medicine is a term that is used for medical treatments that don’t work (or at least haven’t been shown to work). If they worked, they’d be called “medicine”. The anti-malarial, artemesinin, came originally from a Chinese herb, but once it had been purified and properly tested, it was no longer alternative. But the word alternative is not favoured by quacks. They prefer their nostrums to be described as “complementary” –it sounds more respectable. So CAM (complementary and alternative medicine became the politically-correct euphemism. Now it has gone a stage further, and the euphemism in vogue with quacks at the moment is “integrated” or “integrative” medicine. That means, very often, integrating things that don’t work with things that do. But it sounds fashionable. In reality it is designed to confuse politicians who ask for, say, integrated services for old people.

Put another way, the salespeople of quackery have become rather good at bait and switch. The wikepedia definition is as good as any.

Bait-and-switch is a form of fraud, most commonly used in retail sales but also applicable to other contexts. First, customers are “baited” by advertising for a product or service at a low price; second, the customers discover that the advertised good is not available and are “switched” to a costlier product.

As applied to the alternative medicine industry, the bait is usually in the form of some nice touchy-feely stuff which barely mentions the mystical nonsense. But when you’ve bought into it you get the whole panoply of nonsense. Steven Novella has written eloquently about the use of bait and switch in the USA to sell chiropractic, acupuncture, homeopathy and herbal medicine: "The bait is that CAM offers legitimate alternatives, the switch is that it primarily promotes treatments that don’t work or are at best untested and highly implausible.".

The "College of Medicine" provides a near-perfect example of bait and switch. It is the direct successor of the Prince of Wales’ Foundation for Integrated Health. The Prince’s Foundation was a consistent purveyor of dangerous medical myths. When it collapsed in 2010 because of a financial scandal, a company was formed called "The College for Integrated Health". A slide show, not meant for public consumption, said "The College represents a new strategy to take forward the vision of HRH Prince Charles". But it seems that too many people have now tumbled to the idea that "integrated", in this context, means barmpottery. Within less than a month, the new institution was renamed "The College of Medicine". That might be a deceptive name, but it’s a much better bait. That’s why I described the College as a fraud and delusion.

Not only did the directors, all of them quacks, devise a respectable sounding name, but they also succeeded in recruiting some respectable-sounding people to act as figureheads for the new organisation. The president of the College is Professor Sir Graham Catto, emeritus professor of medicine at the University of Aberdeen. Names like his make the bait sound even more plausible. He claims not to believe that homeopathy works, but seems quite happy to have a homeopathic pharmacist, Christine Glover, on the governing council of his college. At least half of the governing Council can safely be classified as quacks.

So the bait is clear. What about the switch? The first thing to notice is that the whole outfit is skewed towards private medicine: see The College of Medicine is in the pocket of ~~Crapita~~ Capita. The founder, and presumably the main provider of funds (they won’t say how much) is the huge outsourcing company, Capita. This is company known in Private Eye as Crapita. Their inefficiency is legendary. They are the folks who messed up the NHS computer system and the courts computer system. After swallowing large amounts of taxpayers’ money, they failed to deliver anything that worked. Their latest failure is the court translation service.. The president (Catto), the vice president (Harry Brunjes) and the CEO (Mark Ratnarajah) are all employees of Capita.

The second thing to notice is that their conferences and courses are a bizarre mixture of real medicine and pure quackery. Their 2012 conference had some very good speakers, but then it had a "herbal workshop" with Simon Mills (see a video) and David Peters (the man who tolerates dowsing as a way to diagnose which herb to give you). The other speaker was Dick Middleton, who represents the huge herbal company, Schwabe (I debated with him on BBC Breakfast), In fact the College’s Faculty of Self-care appears to resemble a marketing device for Schwabe.

Why regulation isn’t working, and can’t work

There are various levels of regulation. The "highest" level is the statutory regulation of osteopathy and chiropractic. The General Chiropractic Council (GCC) has exactly the same legal status as the General Medical Council (GMC). This ludicrous state of affairs arose because nobody in John Major’s government had enough scientific knowledge to realise that chiropractic, and some parts of osteopathy, are pure quackery,

The problem is that organisations like the GCC function more to promote chiropractic than to regulate them. This became very obvious when the British Chiropractic Association (BCA) decided to sue Simon Singh for defamation, after he described some of their treatments as “bogus”, “without a jot of evidence”.

In order to support Singh, several bloggers assessed the "plethora of evidence" which the BCA said could be used to justify their claims. When, 15 months later, the BCA produced its "plethora" it was shown within 24 hours that the evidence was pathetic. The demolition was summarised by lawyer, David Allen Green, in The BCA’s Worst Day.

In the wake of this, over 600 complaints were made to the GCC about unjustified claims made by chiropractors, thanks in large part to heroic work by two people, Simon Perry and Allan Henness. Simon Perry’s Fishbarrel (browser plugin) allows complaints to be made quickly and easily -try it). The majority of these complaints were rejected by the GCC, apparently on the grounds that chiropractors could not be blamed because the false claims had been endorsed by the GCC itself.

My own complaint was based on phone calls to two chiropractors, I was told such nonsense as "colic is down to, er um, faulty movement patterns in the spine". But my complaint never reached the Conduct and Competence committee because it had been judged by a preliminary investigating committee that there was no case to answer. The impression one got from this (very costly) exercise was that the GCC was there to protect chiropractors, not to protect the public.

The outcome was a disaster for chiropractors, wno emerged totally discredited. It was also a disaster for the GCC which was forced to admit that it hadn’t properly advised chiropractors about what they could and couldn’t claim. The recantation culminated in the GCC declaring, in August 2010, that the mythical "subluxation" is a "historical concept " "It is not supported by any clinical research evidence that would allow claims to be made that it is the cause of disease.". Subluxation was a product of the fevered imagination of the founder of the chiropractic cult, D.D. Palmer. It referred to an imaginary spinal lesion that he claimed to be the cause of most diseases. .Since ‘subluxation’ is the only thing that’s distinguished chiropractic from any other sort of manipulation, the admission by the GCC that it does not exist, after a century of pretending that it does, is quite an admission.

The President of the BCA himself admitted in November 2011

“The BCA sued Simon Singh personally for libel. In doing so, the BCA began one of the darkest periods in its history; one that was ultimately to cost it financially,”

As a result of all this, the deficiencies of chiropractic, and the deficiencies of its regulator were revealed, and advertisements for chiropractic are somewhat less misleading. But this change for the better was brought about entirely by the unpaid efforts of bloggers and a few journalists, and not at all by the official regulator, the GCC. which was part of the problem. not the solution. And it was certainly not helped by the organisation that is meant to regulate the GCC, the Council for Health Regulatory Excellence (CHRE) which did nothing whatsoever to stop the farce.

At the other end of the regulatory spectrum, voluntary self-regulation, is an even worse farce than the GCC. They all have grand sounding "Codes of Practice" which, in practice, the ignore totally.

The Society of Homeopaths is just a joke. When homeopaths were caught out recommending sugar pills for prevention of malaria, they did nothing (arguably such homicidal advice deserves a jail sentence).

The Complementary and Natural Healthcare Council (CNHC) is widely know in the blogosphere as Ofquack. I know about them from the inside, having been a member of their Conduct and Competence Committee, It was set up with the help of a £900,000 grant from the Department of Health to the Prince of Wales, to oversee voluntary self-regulation. It fails utterly to do anything useful.. The CNHC code of practice, paragraph 15 , states

“Any advertising you undertake in relation to your professional activities must be accurate. Advertisements must not be misleading, false, unfair or exaggerated”.

When Simon Perry made a complaint to the CNHC about claims being made by a CNHC-registered reflexologist, the Investigating Committee upheld all 15 complaints. But it then went on to say that there was no case to answer because the unjustified claims were what the person had been taught, and were made in good faith.
This is precisely the ludicrous situation which will occur again and again if reflexologists (and many other alternative therapies) are “accredited”. The CNHC said, correctly, that the reflexologist had been taught things that were not true, but then did nothing whatsoever about it apart from toning down the advertisements a bit. They still register reflexologists who make outrageously false claims.

Once again we see that no sensible regulation is possible for subjects that are pure make-believe.

The first two examples deal (or rather, fail to deal) with regulation of outright quackery. But there are dozens of other quangos that sound a lot more respectable.

European Food Standards Agency (EFSA). One of the common scams is to have have your favourite quack treatment classified as a food not as a medicine. The laws about what you can claim have been a lot laxer for foods. But the EFSA has done a pretty good job in stopping unjustified claims for health benefits from foods. Dozens of claims made by makers of probiotics have been banned. The food industry, needless to say, objects very strongly to be being forced to tell the truth. In my view, the ESFA has not gone far enough. They recently issued a directive about claims that could legally be made. Some of these betray the previously high standards of the EFSA. For example you are allowed to say that "Vitamin C contributes to the reduction of tiredness and fatigue" (as long as the product contains above a specified amount of Vitamin C. I’m not aware of any trials that show vitamin C has the slightest effect on tiredness or fatigue, Although these laws do not come into effect until December 2012, they have already been invoked by the ASA has a reason not to uphold a complaint about a multivitamin pill which claimed that it “Includes 8 nutrients that can contribute to the reduction in tiredness and fatigue”

The Advertising Standards Authority (ASA). This is almost the only organisation that has done a good job on false health claims. Their Guidance on Health Therapies & Evidence says

"Whether you use the words ‘treatment’, ‘treat’ or ‘cure’, all are likely to be seen by members of the public as claims to alleviate effectively a condition or symptom. We would advise that they are not used"

"Before and after’ studies with little or no control, studies without human subjects, self-assessment studies and anecdotal evidence are unlikely to be considered acceptable"

"Before and after’ studies with little or no control, studies without human subjects, self-assessment studies and anecdotal evidence are unlikely to be considered acceptable"

They are spot on.

The ASA’s Guidance for Advertisers of Homeopathic Services is wonderful.

"In the simplest terms, you should avoid using efficacy claims, whether implied or direct,"

"To date, the ASA has have not seen persuasive evidence to support claims that homeopathy can treat, cure or relieve specific conditions or symptoms."

That seems to condemn the (mis)labelling allowed by the MHRA as breaking the rules.. Sadly, though, the ASA has no powers to enforce its decisions and only too often they are ignored. The Nightingale collaboration has produced an excellent letter that you can hand to any pharmacist who breaks the rules

The ASA has also judged against claims made by "Craniosacral therapists" (that’s the lunatic fringe of osteopathy). They will presumably uphold complaints about similar claims made (I’m ashamed to say) by UCLH Hospitals.

The private examination company Edexcel sets exams in antiscientific subjects, so miseducating children. The teaching of quackery to 16 year-olds has been approved by a maze of quangos, none of which will take responsibility, or justify their actions. So far I’ve located no fewer than eight of them. The Office of the Qualifications and Examinations Regulator (OfQual), Edexcel, the Qualifications and Curriculum Authority (QCA), Skills for Health, Skills for Care, National Occupational Standards (NOS), private exam company VTCT and the schools inspectorate, Ofsted.. Asking any of these people why they approve of examinations in imaginary subjects meets with blank incomprehension. They fail totally to protect tha public from utter nonsense.

The Department of Education has failed to do anything about the miseducation of children in quackery. In fact it has encouraged it by, for the first time, giving taxpayers’ money to a Steiner (Waldorf) school (at Frome, in Somerset). Steiner schools are run by a secretive and cult-like body of people (read about it). They teach about reincarnation, karma, gnomes, and all manner of nonsense, sometimes with unpleasant racial overtones. The teachers are trained in Steiner’s Anthroposophy, so if your child gets ill at school they’ll probably get homeopathic sugar pills. They might well get measles or mumps too, since Steiner people don’t believe in vaccination.

Incredibly, the University of Aberdeen came perilously close to appointing a chair in anthroposophical medicine. This disaster was aborted by bloggers, and a last minute intervention from journalists. Neither the university’s regulatory mechanisms. nor any others, seemed to realise that a chair in mystical barmpottery was a bad idea.

Trading Standards offices and the Office of Fair Trading.

It is the statutory duty of Trading Standards to enforce the Consumer Protection Regulations (2008) This European legislation is pretty good. it caused a lawyer to write " Has The UK Quietly Outlawed “Alternative” Medicine?". Unfortunately Trading Standards people have consistently refused to enforce these laws. The whole organisation is a mess. Its local office arrangement fails totally to deal with the age of the internet. The situation is so bad that a group of us decided to put them to the test. The results were published in the Medico-Legal Journal, Rose et al., 2012. "Spurious Claims for Health-care Products: An Experimental Approach to Evaluating Current UK Legislation and its Implementation". They concluded "EU directive 2005/29/EC is
largely ineffective in preventing misleading health claims for consumer products in
the UK"

Skills for Health is an enormous quango which produces HR style "competences" for everything under the son. They are mostly quite useless. But those concerned with alternative medicine are not just useless. They are positively harmful. Totally barmy. There are competences and National Occupational Standards for every lunatic made-up therapy under the sun. When I phoned them to discover who’d written them, I learned that the had been drafted by the Prince of Wales’ Foundation for Magic Medicine. And when I joked by asking if they had a competence for talking to trees, I was told, perfectly seriously, “You’d have to talk to LANTRA, the land-based organisation for that.”

That was in January 2008. A lot of correspondence with the head of Skills for Health got nowhere at all. She understood nothing and it hasn’t improved a jot.

This organisation costs a lot of taxpayers’ money and it should have been consigned to the "bonfire of the quangos" (but of course there was no such bonfire in reality). It is a disgrace.

The Quality Assurance Agency (QAA) is supposed to ensure the quality of university courses. In fact it endorses courses in nonsense alternative medicine and so does more harm than good. The worst recent failure of the QAA was in the case of the University of Wales: see Scandal of the University of Wales and the Quality Assurance Agency. The university was making money by validating thousands of external degrees in everything from fundamentalist theology to Chinese Medicine. These validations were revealed as utterly incompetent by bloggers, and later by BBC Wales journalist Ciaran Jenkins (now working for Channel 4).

The mainstream media eventually caught up with bloggers. In 2010, BBC1 TV (Wales) produced an excellent TV programme that exposed the enormous degree validation scam run by the University of Wales. The programme can be seen on YouTube (Part 1, and Part 2). The programme also exposed, incidentally, the uselessness of the Quality Assurance Agency (QAA) which did nothing until the scam was exposed by TV and blogs. Eventually the QAA sent nine people to Malaysia to investigate a dodgy college that had been revealed by the BBC. The trip cost £91,000. It could have been done for nothing if anyone at the QAA knew how to use Google.

The outcome was that the University of Wales stopped endorsing external courses, and it was soon shut down altogether (though bafflingly, its vice-chancellor, Marc Clement was promoted). The credit for this lies entirely with bloggers and the BBC. The QAA did nothing to help until the very last moment.

Throughout this saga Universities UK (UUK), has maintained its usual total passivity. They have done nothing whatsoever about their members who give BSc degrees in anti-scientific subjects. (UUK used to known as the Committee of Vice-Chancellors and Principals).

Council for Health Regulatory Excellence (CHRE), soon to become the PSAHSC,

Back now to the CHRE, the people who failed so signally to sort out the GCC. They are being reorganised. Their consultation document says

"The Health and Social Care Act 20122 confers a new function on the Professional Standards Authority for Health and Social Care (the renamed Council for Healthcare Regulatory Excellence). From November 2012 we will set standards for organisations that hold voluntary registers for people working in health and social care occupations and we will accredit the register if they meet those standards. It will then be known as an ‘Accredited Register’. "

They are trying to decide what the criteria should be for "accreditation" of a regulatory body. The list of those interested has some perfectly respectable organisations, like the British Psychological Society. It also contains a large number of crackpot organisations, like Crystal and Healing International, as well as joke regulators like the CNHC.

They already oversee the Health Professions Council (HPC) which is due to take over Herbal medicine and Traditional Chinese Medicine, with predictably disastrous consequences.

Two of the proposed criteria for "accreditation" appear to be directly contradictory.

Para 2.5 makes the whole accreditation pointless from the point of view of patients

2.5 It will not be an endorsement of the therapeutic validity or effectiveness of any particular discipline or treatment.

Since the only thing that matters to the patient is whether the therapy works (and is safe), accrediting of organisations that ignore this will merely give the appearance of official approval of crystal healing etc etc. This appears to contradict directly

A.7 The organisation can demonstrate that there either is a sound knowledge base underpinning the profession or it is developing one and makes that explicit to the public.

A "sound knowledge base", if it is to mean anything useful at all, means knowledge that the treatment is effective. If it doesn’t mean that, what does it mean?

It seems that the official mind has still not grasped the obvious fact that there can be no sensible regulation of subjects that are untrue nonsense. If it is nonsense, the only form of regulation that makes any sense is the law.

Please fill in the consultation. My completed return can be downloaded as an example, if you wish.

Medicines and Healthcare products Regulatory Agency (MHRA) should be a top level defender of truth. Its strapline is

"We enhance and safeguard the health of the public by ensuring that medicines and medical devices work and are acceptably safe."

The MHRA did something (they won’t tell me exactly what) about one of the most cruel scams that I’ve ever encountered, Esperanza Homeopathic Neuropeptide, peddled for multiple sclerosis, at an outrageous price ( £6,759 for 12 month’s supply). Needless to say there was not a jot of evidence that it worked (and it wasn’t actually homeopathic).

Astoundingly, Trading Standards officers refused to do anything about it.

The MHRA admit (when pushed really hard) that there is precious little evidence that any of the herbs work, and that homeopathy is nothing more than sugar pills. Their answer to that is to forget that bit about "ensuring that medicines … work"

Here’s the MHRA’s Traditional Herbal Registration Certificate for devils claw tablets.

vitabiotics

The wording "based on traditional use only" has to be included because of European legislation. Shockingly, the MHRA have allowed them to relegate that to small print, with all the emphasis on the alleged indications. The pro-CAM agency NCCAM rates devil’s claw as "possibly effective" or "insufficient evidence" for all these indications, but that doesn’t matter because the MHRA requires no evidence whatsoever that the tablets do anything. They should, of course, added a statement to this effect to the label. They have failed in their duty to protect and inform the public by allowing this labelling.

But it gets worse. Here is the MHRA’s homeopathic marketing authorisation for the homeopathic medicinal product Arnicare Arnica 30c pillules

It is nothing short of surreal.

Since the pills contain nothing at all, they don’t have the slightest effect on sprains, muscular aches or bruising. The wording on the label is exceedingly misleading.

If you "pregnant or breastfeeding" there is no need to waste you doctor’s time before swallowing a few sugar pills.

"Do not take a double dose to make up for a missed one". Since the pills contain nothing, it doesn’t matter a damn.

"If you overdose . . " it won’t have the slightest effect because there is nothing in them

And it gets worse. The MHRA-approved label specifies ACTIVE INGREDIENT. Each pillule contains 30c Arnica Montana

No, they contain no arnica whatsoever.

It truly boggles the mind that men with dark suits and lots of letters after their names have sat for hours only to produce dishonest and misleading labels like these.

When this mislabeling was first allowed, it was condemned by just about every scientific society, but the MHRA did nothing.

The Nightingale Collaboration.

This is an excellent organisation, set up by two very smart skeptics, Alan Henness and Maria MacLachlan. Visit their site regularly, sign up for their newsletter Help with their campaigns. Make a difference.

Conclusions

The regulation of alternative medicine in the UK is a farce. It is utterly ineffective in preventing deception of patients.

Such improvements as have occurred have resulted from the activity of bloggers, and sometime the mainstream media. All the official regulators have, to varying extents, made things worse.

The CHRE proposals promise to make matters still worse by offering "accreditation" to organisations that promote nonsensical quackery. None of the official regulators seem to be able to grasp the obvious fact that is impossible to have any sensible regulation of people who promote nonsensical untruths. One gets the impression that politicians are more concerned to protect the homeopathic (etc, etc) industry than they are to protect patients.

Deception by advocates of alternative medicine harms patients. There are adequate laws that make such deception illegal, but they are not being enforced. The CHRE and its successor should restrict themselves to real medicine. The money that they spend on pseudo-regulation of quacks should be transferred to the MHRA or a reformed Trading Standards organisation so they can afford to investigate and prosecute breaches of the law. That is the only form of regulation that makes sense.

Follow-up

The shocking case of the continuing sale of “homeopathic vaccines” for meningitis, rubella, pertussis etc was highlighted in an excellent TV programme by BBC South West. The failure of the MHRA and the GPC do take any effective action is a yet another illustration of the failure of regulators to do their job. I have to agree with Andy Lewis when he concludes

“Children will die. And the fault must lie with Professor Sir Kent Woods, chairman of the regulator.”

Tagged Academia, acupuncture, alternative medicine, badscience, British Chiropractic Association, BTEC, chiropractic, CNHC, College of medicine, Complementary and Natural Healthcare Council, Department of Health, Edexcel, evidence, General Chiropractic Council, Health Professions Council, herbalism, homeopathy, National Occupational Stardards, OfQual, Ofsted, Prince of Wales, Prince's Foundation, QCA, quackademia, quackery, Skills for Care, Skills for Health, Steiner, TCM, Traditional Chinese medicine, University of Westminster, VTCT | 14 Comments

Ben Goldacre’s Bad Pharma. Buy it now. Then do something.

Published September 25, 2012

Jump to follow-up

This is a very important book.

Buy it now (that link is to Waterstone’s Amazon don’t pay tax in the UK, so don’t use them).

When you’ve read it, do something about it. The book has lots of suggestions about what to do.

Stolen from badscience.net

Peter Medawar, the eminent biologist, in his classic book Advice to a Young Scientist, said this.

“Exaggerated claims for the efficacy of a medicament are very seldom the consequence of any intention to deceive; they are usually the outcome of a kindly conspiracy in which everybody has the very best intentions. The patient wants to get well, his physician wants to have made him better, and the pharmaceutical company would have liked to have put it into the physician’s power to have made him so. The controlled clinical trial is an attempt to avoid being taken in by this conspiracy of good will.”

There was a lot of truth in that 1979, towards the end of the heyday of small molecule pharmacology. Since then, one can argue, things have gone downhill.

First, though, think of life without general anaesthetics, local anaesthetics, antibiotics, anticoagulants and many others. They work well and have done incalculable good. And they were developed by the drug industry.

But remember also that remarkably little is known about medicine. There are huge areas in which neither causes nor cures are known. Treatments for chronic pain, back problems, many sorts of cancer and almost all mental problems are a mess. It just isn’t known what to do. Nobody is to blame for this. Serious medical research has been going on for little more than 60 years, and it turns out to be very complicated. We are doing our best, but are still ignorant about whole huge areas. That leads to a temptation to make things up. Clutching at straws is very evident when it comes to depression, pain and Alzheimer’s disease, among others.

In order to improve matters, one essential is to do fair tests on treatments that we have. Ben Goldacre’s book is a superb account of how this could be done, and how the process of testing has been subverted for commercial gain and to satisfy the vanities of academics.

Of course there is nothing new in criticisms of Big Pharma. The huge fines levied on them for false advertising are well known. The difference is that Goldacre’s book explains clearly what’s gone wrong in great detail, documents it thoroughly, and makes concrete suggestions for improving matters.

Big Pharma has undoubtedly sometimes behaved appallingly in recent years. Someone should be in jail for crimes against patients. They have behaved in much the same way that bankers have. In any huge globalised industry it is always possible to blame someone in another department for the dishonesty. But they aren’t the only people to blame. None of the problems could have arisen with the complicity of academics, universities, and a plethora of regulatory agencies and professional bodies.

The biggest scandal of all is missing data (chapter 1). Companies, and sometmes academics, have suppressed of trials that don’t favour the drugs that they are trying to sell. The antidepressant drug, reboxetine, appeared at first to be good. It had been approved by the Medicines and Healthcare products Regulatory Agency (MHRA) and there was at least one good randomized placebo-controlled trial (RCT) showing it worked. But it didn’t. The manufacturer didn’t provide a complete list of unpublished trials when asked for them. After much work it was found in 2010 that, as well as the published, favourable trial, there were six more trials which had not been published and all six showed reboxetine to be no better than placebo . In comparisons with other antidepressant drugs three small studies (507 patients) showed reboxetine to be as good as its competitors. These were published. But it came to light that data on 1657 patients had never been published and these showed reboxetine to be worse than its rivals.

When all the data for the SSRI antidepressants were unearthed (Kirsch et al., 2008) it turned out that they were no better than placebo for mild or moderate depression. This selective suppression of negative data has happened time and time again. It harms patients and deceives doctors, but, incredibly, it’s not illegal.

Disgracefully, Kirsch et al. had to use a Freedom of Information Act request to get the data from the FDA.

“The output of a regulator is often simply a crude, brief summary: almost a ‘yes’ or ‘no’ about side effects. This is the opposite of science, which is only reliable because everyone shows their working, explains how they know that something is effective or safe, shares their methods and their results, and allows others to decide if they agree with the way they processed and analysed the data.”

“the NICE document discussing whether it’s a good idea to have Lucentis, an extremely expensive drug, costing well over £ 1,000 per treatment, that is injected into the eye for a condition called acute macular degeneration. As you can see, the NICE document on whether this treatment is a good idea is censored. Not only is the data on the effectiveness of the treatment blanked out by thick black rectangles, in case any doctor or patient should see it, but absurdly, even the names of some trials are missing, preventing the reader from even knowing of their existence, or cross referencing information about them.Most disturbing of all, as you can see in the last bullet point, the data on adverse events is also censored.”

The book lists all the tricks that are used by both industry and academics. Here are some of them.

Regulatory agencies like the MHRA, the European Medicines Agency (EMA) and the US Food and Drugs Administration (FDA) set a low bar for approval of drugs.
Companies make universities sign gagging agreements which allow unfavourable results to be suppressed, and their existence hidden.
Accelerated approval schemes are abused to get quick approval of ineffective drugs and the promised proper tests often don’t materialise
Disgracefully, even when all the results have been given to the regulatory agencies (which isn’t always). The MHRA, EMA and FDA don’t make them public. We are expected to take their word.
Although all clinical trials are meant to be registered before they start, the EMA register, unbelievably, is not public. Furthermore there is no check that the results if trials ever get published. Despite mandates that results must be published within a year of finishing the trial, many aren’t. Journals promise to check this sort of thing, but they don’t.
When the results are published, it is not uncommon for the primary outcome, specified before it started, to have been changed to one that looks like a more favourable result. Journals are meant to check, but mostly don’t.
Companies use scientific conferences, phony journals, make-believe “seed trials” and “continuing medical education” for surreptitious advertising.
Companies invent new diseases, plant papers to make you think you’re abnormal, and try to sell you a “cure”. For example, female sexual dysfunction , restless legs syndrome and social anxiety disorder (i.e. shyness). This is called disease-mongering, medicalisation or over-diagnosis. It’s bad.
Spin is rife. Companies, and authors, want to talk up their results. University PR departments want to exaggerate benefits. Journal editors want sensational papers. Read the results, not the summary. This is universal (but particularly bad in alternative medicine).
Companies fund patient groups to lobby for pills even when the pills are known to be ineffective. The lobby that demanded that Herceptin should be available to all on the breast cancer patients on the NHS was organised by a PR company working for the manufacturer, Roche. But Herceptin doesn’t work at all in 80% of patients and gives you at best a few extra months of life in advanced cases.
Ghostwriting of papers is serious corruption. A company writes the paper and senior academics appear as the authors, though they may never have seen the original data. Even in cases where academics have admitted to lying about whether they have seen the data, they go unpunished by their universities. See for example, the case of Professor Eastell.
By encouraging the funding of “continuing medical education” by companies, the great and the good of academic medicine have let us down badly.

This last point is where the book ends, and it’s worth amplification.

“So what have the great and good of British medicine done to help patients, in the face of this endemic corruption, and these systematic flaws? In 2012, a collaborative document was produced by senior figures in medicine from across the board, called ‘Guidance on Collaboration Between Healthcare Professionals and the Pharmaceutical Industry’. This document was jointly approved by the ABPI, the Department of Health, the Royal Colleges of Physicians, Nursing, Psychiatrists, GPs, the Lancet, the British Medical Association, the NHS Confederation, and so on. ”

“It contains no recognition of the serious problems we have seen in this book. In fact, quite the opposite: it makes a series of assertions about them that are factually incorrect.”

“It states that drug reps ‘can be a useful resource for healthcare professionals’. Again, I’m not sure why the Royal Colleges, the BMA, the Department of Health and the NHS Confederation felt the need to reassert this to the doctors of the UK, on behalf of industry, when the evidence shows that drug reps actively distort prescribing practices. But that is the battle you face, trying to get these issues taken seriously by the pinnacle of the medical establishment.”

This is perhaps the most shameful betrayal of all. The organisations that should protect patients have sold them out.

You may have been sold out by your “elders and betters”, but you can do something. The “What to do” sections of the book should be produced as a set of flash cards, as a reminder that matters can be improved.

It is shameful that this book was not written by a clinical pharmacologist, or a senior doctor, or a Royal College, or a senior academic. Why has the British Pharmacological Society said nothing?

It is shameful too that this book was not written by one of the quacks who are keen to defend the $60 billion alternative medicine industry (which has cured virtually nothing) and who are strident in their criticism of the 600 billion dollar Pharma industry. They haven’t done the work that Goldacre has to analyse the real problems. All they have done is to advocate unfair tests, because that is the only sort their treatments can pass.

It’s weird that medicine, the most caring profession, is more corrupt than any other branch of science. The reason, needless to say, is money. Well, money and vanity. The publish or perish mentality of senior academics encourages dishonesty. It is a threat to honest science.

Goldacre’s book shows the consequences: harm to patients and huge wastage of public money.

Read it.

Do something.

Follow-up

7 October, 2012, The Observer

Goldacre wrote

"I think it’s really disappointing that nobody, not the Royal Colleges, the Academy of Medical Sciences, the British Pharmacological Society, the British Medical Association, none of these organisations have stood up and said: selective non-publication of unflattering trial data is research misconduct, and if you do it you will be booted out. And I think they really urgently should."

Exactly.

Tagged Academia, badscience, Big Pharma, CAM, Clinical trials, corruption, EMA, European Medicines Agency, FDA, NICE, RCT, Richard Eastell, Universities | 16 Comments

In which Simon Gaskell, of Queen Mary, University of London, makes a cock-up

Published August 16, 2012

Jump to follow-up

The row about ~~redundancies~~ firings at Queen Mary rumbles on.

I’ve already written about it twice in Is Queen Mary University of London trying to commit scientific suicide?, and in Queen Mary, University of London in The Times. Does Simon Gaskell care?. But wait, there is more to come.

The harm done to teaching at Queen Mary was outlined in a report written at the request of Simon Gaskell. He appears to have ignored it entirely. So let’s concentrate on research.

Simon Gaskell

Some explanation of the bizarre behaviour of the Queen Mary management can be gleaned from Queen Mary’s Frequently Asked Questions:
Restructures and Reviews in Academic Departments 2011-12. This says

“research-related metrics” used in the School of Biological and Chemical Sciences are the result of “extensive consultation with staff and include both the Australian [Research Council] journal classification system as well as impact factor”.

Let’s skip over the fact that the "extensive consultation" was largely sham. That’s standard procedure (and not only in universities).

Australian journal classification

The reference to the Australian journal classification is revealing. Australia has been noted in the past for being one of the worst places for abuse of publication metrics. Its 2010 Journal classification was utterly bizarre [download Excel]. It ranks 20,712 journals as being A*, A, B, or C (did the rankers really look at all of them?).

I’ll take one example from my own area: the Journal of General Physiology is probably one of the most-respected journals for electrophysiology with emphasis on mechanisms. Many of its papers are quite mathematical. In its niche, it is hugely respected for the quality of papers and for its high integrity [declaration of interest, I’m an editor, and was hugely flattered by that honour]. But the Australian 2010 list ranks the Journal of General Physiology as B, the same as Brain Research, British Journal of Religious Education and Chinese Medicine (a journal of quack medicine).

In contrast, the Nature journals all get A*, as do many review journals which don’t publish original research at all. But if you can’t get a paper into Nature, there are other A* journals that might help you keep your job, for example Television and New Media, or Tourism Management as well as some joke journals like the Journal of Alternative and Complementary Medicine and Complementary Therapies in Medicine.

No doubt you’ll find similar ludicrous anomalies in your own field.

Simon Gaskell has not answered emails from me, and he hasn’t answered those from his own staff either, but he did emerge briefly from his shell last week, in The Times and at greater length in today’s Times Higher Education. The latter, though longer, consists almost entirely of the vapid self-congratulation which all vice-chancellors feel compelled to spout. Only one short paragraph is devoted to answering his critics.

“Where academic performance has been assessed, it has been important to do so on the basis of objective criteria including metrics – any subjective assessment would be quite unacceptable. These objective criteria were based on generally recognised academic expectations . . . “

I find it almost impossible to believe that a vice-chancellor should be so out of touch as to believe that counting papers, and using impact factors to judge people are “generally recognised”. The extent of the misunderstanding of metrics is illustrated by two statements for Matthew Evans, head of the School of Biological and Chemical Sciences. is quoted as saying:

“Impact factor reflects the number of times an average paper is cited, [so] is a good indication of how many citations a particular paper is likely to achieve,”

Anyone who understands the difference between mode, median and mean of the highly skewed distribution of citations would not make an elementary mistake like that. It is simply statistical illiteracy.

Evans also

"…described metrics as a “vital tool” in assessing academics’ contributions to research and “the only empirical way of measuring success in science”."

Is Matthew Evans not aware that there isn’t the slightest evidence that any metric predicts the future success of a scientist? It’s an evidence-free zone Perhaps he should read about how to test social interventions.

It’s certainly evident that Gaskell’s management team can’t use Google. After getting the hint about the use of Australian rankings, it took five minutes to discover this statement.

"On 30 May 2011, the decision not to use ranked outlets for ERA 2012 was announced".

It’s more than a year since Kim Carr, at that time the Australian minister for innovation, industry, science and research, announced the result of a review of the way the next ce of Research in Australia (ERA) exercise would be conducted by the Australian Research Council (ARC).

“There is clear and consistent evidence that the rankings were being deployed inappropriately within some quarters of the sector, in ways that could produce harmful outcomes, and based on a poor understanding of the actual role of the rankings. One common example was the setting of targets for publication in A and A* journals by institutional research managers.

“their existence was focussing ill-informed, undesirable behaviour in the management of research – I have made the decision to remove the rankings, based on the ARC’s expert advice”.

It seems that Professor Gaskell is unaware of this, since he is enforcing the very “ill-informed, undesirable behaviour in the management of research” that Australia has dropped.

That is what I’d call a major cock-up.

HEFCE also reviewed the role of metrics. Their pilot tests were, almost needless to say, not properly randomised, but the conclusion was much the same as in Australia. Is Gaskell not aware that REF instructions say

“No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs”

The very people Gaskell seeks to please have condemned strongly his methods.

If that is not “bringing your university into disrepute”, I don’t know what is.

The responses to Simon Gaskell

This week’s Times Higher Education published three letters in response to Gaskell’s article.

The lead letter was signed by thirteen Queen Mary academics. This gives the flavour.

“Here we point to the unintended consequences of restructuring already in evidence. These include undermining morale in the schools and departments concerned; the flight of talented colleagues to other institutions; the consignment of teaching to lecturers in casual employment or those deemed unfit for research; scandalous gender disparity; and the lopsided, counterproductive allocation of resources. When staff are dismissed, replacements can come only from other institutions that have been willing to invest in people, research and scholarship. As a part of normal academic life, mobility is acceptable, even desirable, but when enforced on the scale envisaged at Queen Mary, it is random slaughter offset by poaching.”

My letter described the cock-up over the ARC Journal classification.

The third letter was from Fanis Missirlis, who was co-authot of the Lancet letter, Queen Mary: nobody expects the Spanish Inquisition. As a result he was fired at one day’s notice. He is one of the academics whose “careers are destroyed by decimal points in spurious calculations”

The Queen Mary process was summed up rather well when the University of Sydney went through a similar convulsion. An Australian academic referred to

"retrenchment exercises driven by “crass, bureaucratic, quantifiable simulacra of genuine research”.

Follow-up

August 23 2012.

I see it is now official. One of the great stars of Queen Mary, Lisa Jardine, is leaving Queen Mary (and, as it happens, coming to UCL). Learn more about her in an interview with Laurie Taylor.

She discovered lost papers by the great renaissance scientist, Robert Hooke –watch the video.

Lisa Jardine

20 December 2012

It seems that the predicted bad effects are coming true. Times Higher Education carries a letter from a biology undergraduate, Matthew James Erickson, which is highly critical of the effect of Gaskell’s policies on teaching at Queen Mary.

“From my perspective as a first-year undergraduate, the aggressive restructuring has had a profoundly negative effect on my opinion of my university.”

It is rather wonderful when a first year undergraduate dares to speak truth to power. Well done Matthew James Erickson.

Tagged impact factor, Queen Mary, Queen Mary University of London, Simon Gaskell | 11 Comments

Is Queen Mary University of London trying to commit scientific suicide?

Published June 29, 2012

Jump to follow-up

Academic staff are going to be fired at Queen Mary University of London (QMUL). It’s possible that universities may have to contract a bit in hard times, so what’s wrong?

What’s wrong is that the victims are being selected in a way that I can describe only as insane. The criteria they use are guaranteed to produce a generation of second-rate spiv scientists, with a consequent progressive decline in QMUL’s reputation.

The firings, it seems, are nothing to do with hard financial times, but are a result of QMUL’s aim to raise its ranking in university league tables.

In the UK university league table, a university’s position is directly related to its government research funding. So they need to do well in the 2014 ‘Research Excellence Framework’ (REF). To achieve that they plan to recruit new staff with high research profiles, take on more PhD students and post-docs, obtain more research funding from grants, and get rid of staff who are not doing ‘good’ enough research.

So far, that’s exactly what every other university is trying to do. This sort of distortion is one of the harmful side-effects of the REF. But what’s particularly stupid about QMUL’s behaviour is the way they are going about it. You can assess your own chances of survival at QMUL’s School of Biological and Chemical Sciences from the following table, which is taken from an article by Jeremy Garwood (Lab Times Online. July 4, 2012). The numbers refer to the four year period from 2008 to 2011.

Category of staff	Research Output Quantity (No. of Papers)	Research Output Quality (No. of high quality papers)	Research Income (£) (Total)	Research Income (£) As Principal Investigator
Professor	11	2	400,000	at least 200,000
Reader	9	2	320,000	at least 150,000
Senior Lecturer	7	1	260,000	at least 120,000
Lecturer	5	1	200,000	at least 100,000

In addition to the three criteria, ‘Research Output ‐ quality’, ‘Research Output – quantity’, and ‘Research Income’, there is a minimum threshold of 1 PhD completion for staff at each academic level. All this data is “evidenced by objective metrics; publications cited in Web of Science, plus official QMUL metrics on grant income and PhD completion.”

To survive, staff must meet the minimum threshold in three out of the four categories, except as follows:

Demonstration of activity at an exceptional level in either ‘research outputs’ or ‘research income’, termed an ‘enhanced threshold’, is “sufficient” to justify selection regardless of levels of activity in the other two categories. And what are these enhanced thresholds?
For research quantity: a mere 26 published items with at least 11 as significant author (no distinction between academic level); research quality: a modest 6 items published in numerically-favoured journals (e.g. impact factor > 7). Alternatively you can buy your survival with a total ‘Research Income’ of £1,000,000 as PI.

The university notes that the above criteria “are useful as entry standards into the new school, but they fall short of the levels of activity that will be expected from staff in the future. These metrics should not, therefore, be regarded as targets for future performance.”

This means that those who survived the redundancy criteria will simply have to do better. But what is to reassure them that it won’t be their turn next time should they fail to match the numbers?

To help them, Queen Mary is proposing to introduce ‘D³’ performance management (www.unions.qmul.ac.uk/ucu/docs/d3-part-one.doc). Based on more ‘administrative physics’, D³ is shorthand for ‘Direction × Delivery × Development.’ Apparently “all three are essential to a successful team or organisation. The multiplication indicates that where one is absent/zero, then the sum is zero!”

D³ is based on principles of accountability: “A sign of a mature organisation is where its members acknowledge that they face choices, they make commitments and are ready to be held to account for discharging these commitments, accepting the consequences rather than seeking to pass responsibility.” Inspired?

I presume the D³ document must have been written by an HR person. It has all the incoherent use of buzzwords so typical of HR. And it says "sum" when it means "product" (oh dear, innumeracy is rife).

The criteria are utterly brainless. The use of impact factors for assessing people has been discredited at least since Seglen (1997) showed that the number of citations that a paper gets is not perceptibly correlated with the impact factor of the journal in which it’s published. The reason for this is the distribution of the number of citations for papers in a particular journal is enormously skewed. This means that high-impact journals get most of their citations from a few articles.

The distribution for Nature is shown in Fig. 1. Far from being gaussian, it is even more skewed than a geometric distribution; the mean number of citations is 114, but 69% of papers have fewer than the mean, and 24% have fewer than 30 citations. One paper has 2,364 citations but 35 have 10 or fewer. ISI data for citations in 2001 of the 858 papers published in Nature in 1999 show that the 80 most-cited papers (16% of all papers) account for half of all the citations (from Colquhoun, 2003)

The Institute of Scientific Information, ISI, is guilty of the unsound statistical practice of characterizing a distribution by its mean only, with no indication of its shape or even its spread. School of Biological and Chemical Sciences-QMUL is expecting everyone has to be above average in the new regime. Anomalously, the thresholds for psychologists are lower because it is said that it’s more difficult for them to get grants. This undermines even the twisted logic applied at the outset.

All this stuff about skewed distributions is, no doubt, a bit too technical for HR people to understand. Which, of course, is precisely why they should have nothing to do with assessing people.

At a time when so may PhDs fail to get academic jobs we should be limiting the numbers. But QMUL requires everyone to have a PhD student, not for the benefit of the student, but to increase its standing in league tables. That is deeply unethical.

The demand to have two papers in journals with impact factor greater than seven is nonsense. In physiology, for example, there are only four journals with an impact factor greater that seven and three of them are review journals that don’t publish original research. The two best journals for electrophysiology are Journal of Physiology (impact factor 4.98, in 2010) and Journal of General Physiology (IF 4.71). These are the journals that publish papers that get you into the Royal Society or even Nobel prizes. But for QMUL, they don’t count.

I have been lucky to know well three Nobel prize winners. Andrew Huxley. Bernard Katz, and Bert Sakmann. I doubt that any of them would pass the criteria laid down for a professor by QMUL. They would have been fired.

The case of Sakmann is analysed in How to Get Good Science, [pdf version]. In the 10 years from 1976 to 1985, when Sakmann rose to fame, he published an average of 2.6 papers per year (range 0 to 6). In two of these 10 years he had no publications at all. In the 4 year period (1976 – 1979 ) that started with the paper that brought him to fame (Neher & Sakmann, 1976) he published 9 papers, just enough for the Reader grade, but in the four years from 1979 – 1982 he had 6 papers, in 2 of which he was neither first nor last author. His job would have been in danger if he’d worked at QMUL. In 1991 Sakmann, with Erwin Neher, got the Nobel Prize for Physiology or Medicine.

The most offensive thing of the lot is the way you can buy yourself out if you publish 26 papers in the 4 year period. Sakmann came nowhere near this. And my own total, for the entire time from my first paper (1963) until I was elected to the Royal Society (May 1985) was 27 papers (and 7 book chapters). I would have been fired.

Peter Higgs had no papers at all from the time he moved to Edinburgh in 1960, until 1964 when his two paper’s on what’s now called the Higgs’ Boson were published in Physics Letters. That journal now has an impact factor less than 7 so Queen Mary would not have counted them as “high quality” papers, and he would not have been returnable for the REF. He too would have been fired.

The encouragement to publish large numbers of papers is daft. I have seen people rejected from the Royal Society for publishing too much. If you are publishing a paper every six weeks, you certainly aren’t writing them, and possibly not even reading them. Most likely you are appending your name to somebody else’s work with little or no checking of the data. Such numbers can be reached only by unethical behaviour, as described by Peter Lawrence in The Mismeasurement of Science. Like so much managerialism, the rules provide an active encouragement to dishonesty.

In the face of such a boneheaded approach to assessment of your worth, it’s the duty of any responsible academic to point out the harm that’s being done to the College. Richard Horton, in the Lancet, did so in Bullying at Barts. There followed quickly letters from Stuart McDonald and Nick Wright, who used the Nuremburg defence, pointing out that the Dean (Tom Macdonald) was just obeying orders from above. That has never been as acceptable defence. If Macdonald agreed with the procedure, he should be fired for incompetence. If he did not agree with it he should have resigned.

It’s a pity, because Tom Macdonald was one of the people with whom I corresponded in support of Barts’ students who, very reasonably, objected to having course work marked by homeopaths (see St Bartholomew’s teaches antiscience, but students revolt, and, later, Bad medicine. Barts sinks further into the endarkenment). In that case he was not unreasonable, and, a mere two years later I heard that he’d taken action.

To cap it all, two academics did their job by applying a critical eye to what’s going on at Queen Mary. They wrote to the Lancet under the title Queen Mary: nobody expects the Spanish Inquisition

"For example, one of the “metrics” for research output at professorial level is to have published at least two papers in journals with impact factors of 7 or more. This is ludicrous, of course—a triumph of vanity as sensible as selecting athletes on the basis of their brand of track suit. But let us follow this “metric” for a moment. How does the Head of School fair? Zero, actually. He fails. Just consult Web of Science. Take care though, the result is classified information. HR’s “data” are marked Private and Confidential. Some things must be believed. To question them is heresy."

Astoundingly, the people who wrote this piece are now under investigation for “gross misconduct”. This is behaviour worthy of the University of Poppleton, as pointed out by the inimitable Laurie Taylor, in Times Higher Education (June 7)

The rustle of censorship

It appears that last week’s edition of our sister paper, The Poppleton Evening News, carried a letter from Dr Gene Ohm of our Biology Department criticising this university’s metrics-based redundancy programme.

We now learn that, following the precedent set by Queen Mary, University of London, Dr Ohm could be found guilty of “gross misconduct” and face “disciplinary proceedings leading to dismissal” for having the effrontery to raise such issues in a public place.

Louise Bimpson, the corporate director of our ever-expanding human resources team, admitted that this response might appear “severe” but pointed out that Poppleton was eager to follow the disciplinary practices set by such soon-to-be members of the prestigious Russell Group as Queen Mary. Thus it was only to be expected that we would seek to emulate its espousal of draconian censorship. She hoped this clarified the situation.

David Bignell, emeritus professor of zoology at Queen Mary hit the nail on the head.

"These managers worry me. Too many are modest achievers, retired from their own studies, intoxicated with jargon, delusional about corporate status and forever banging the metrics gong. Crucially, they don’t lead by example."

What the managers at Queen Mary have failed to notice is that the best academics can choose where to go.

People are being told to pack their bags and move out with one day’s notice. Access to journals stopped, email address removed, and you may need to be accompanied to your (ex)-office. Good scientists are being treated like criminals.

What scientist in their right mind would want to work at QMUL, now that their dimwitted assessment methods, and their bullying tactics, are public knowledge?

The responsibility must lie with the principal, Simon Gaskell. And we know what the punishment is for bringing your university into disrepute.

Follow-up

Send an email. You may want to join the many people who have already written to QMUL’s principal, Simon Gaskell (principal@qmul.ac.uk), and/or to Sir Nicholas Montagu, Chairman of Council, n.montagu@qmul.ac.uk.

Sunday 1 July 2012. Since this blog was posted after lunch on Friday 29th June, it has had around 9000 visits from 72 countries. Here is one of 17 maps showing the origins of 200 of the hits in the last two days

map

The tweets about QMUL are collected in a Storify timeline.

I’m reminded of a 2008 comment, on a post about the problems imposed by HR, In-human resources, science and pizza.

Thanks for that – I LOVED IT. It’s fantastic that the truth of HR (I truly hate that phrase) has been so ruthlessly exposed. Should be part of the School Handbook. Any VC who stripped out all the BS would immediately retain and attract good people and see their productivity soar.

That’s advice that Queen Mary should heed.

Part of the reason for that popularity was Ben Goldacre’s tweet, to his 201,000 followers

“destructive, unethical and crude metric incentives in academia (spotlight QMUL) bit.ly/MFHk2H by @david_colquhoun”

3 July 2012. I have come by a copy of this email, which was sent to Queen Mary by a senior professor from the USA (word travels fast on the web). It shows just how easy it is to destroy the reputation of an institution.

Sir Nicholas Montagu, Chairman of Council, and Principal Gaskell,

I was appalled to read the criteria devised by your University to evaluate its faculty. There are so flawed it is hard to know where to begin.

Your criteria are antithetical to good scientific research. The journals are littered with weak publications, which are generated mainly by scientists who feel the pressure to publish, no matter whether the results are interesting, valid, or meaningful. The literature is flooded by sheer volume of these publications.

Your attempt to require “quality” research is provided by the requirement for publications in “high Impact Factor” journals. IF has been discredited among scientists for many reasons: it is inaccurate in not actually reflecting the merit of the specific paper, it is biased toward fields with lots of scientists, etc. The demand for publications in absurdly high IF journals encourages, and practically enforces scientific fraud. I have personally experienced those reviews from Nature demanding one or two more “final” experiments that will clinch the publication. The authors KNOW how these experiments MUST turn out. If they want their Nature paper (and their very academic survival if they are at a brutal, anti-scientific university like QMUL), they must get the “right” answer. The temptation to fudge the data to get this answer is extreme. Some scientists may even be able to convince themselves that each contrary piece of data that they discard to ensure the “correct” answer is being discarded for a valid reason. But the result is that scientific misconduct occurs. I did not see in your criteria for “success” at QMUL whether you discount retracted papers from the tally of high IF publications, or perhaps the retraction itself counts as yet another high IF publication!

Your requirement for each faculty to have one or more postdocs or students promotes the abusive exploitation of these individuals for their cheap labor, and ignores the fact that they are being “trained” for jobs that do not exist.

The “standards” you set are fantastically unrealistic. For example, funding is not graded, but a sharp step function – we have 1 or 2 or 0 grants and even if the average is above your limits, no one could sustain this continuously. Once you have fired every one of your faculty, which will almost certainly happen within 1-2 rounds of pogroms, where will you find legitimate scientists who are willing to join such a ludicrous University?

4 July 2012.

Professor John F. Allen is Professor of Biochemistry at Queen Mary, University of London, and distinguished in the fields of Photosynthesis, Chloroplasts, Mitochondria, Genome function and evolution and Redox signalling. He, with a younger colleague, wrote a letter to the Lancet, Queen Mary: nobody expects the Spanish Inquisition. It is an admirable letter, the sort of thing any self-respecting academic should write. But not according to HR. On 14 May, Allen got a letter from HR, which starts thus.

14th May 2012

Dear Professor Allen

I am writing to inform you that the College had decided to commence a factfinding investigation into the below allegation: That in writing and/or signing your name to a letter entitled "Queen Mary: nobody expects the Spanish Inquisition," (enclosed) which was published in the Lancet online on 4th May 2012, you sought to bring the Head of School of Biological and Chemical Sciences and the Dean for Research in the School of Medicine and Dentistry into disrepute.

. . . .

Sam Holborn
HR Consultant- Science & Engineering

Download the entire letter. It is utterly disgraceful bullying. If anyone is bringing Queen Mary into disrepute, it is Sam Holborn and the principal, Simon Gaskell.

Here’s another letter, from the many that have been sent. This is from a researcher in the Netherlands.

Dear Sir Nicholas,

I am addressing this to you in the hope that you were not directly involved in creating this extremely stupid set of measures that have been thought up, not to improve the conduct of science at QMUL, but to cheat QMUL’s way up the league tables over the heads of the existing academic staff.

Others have written more succinctly about the crass stupidity of your Human Resources department than I could, and their apparent ignorance of how science actually works. As your principal must bear full responsibility for the introduction of these measures, I am not sending him a copy of this mail. I am pretty sure that his “principal” mail address will no longer be operative.

We have had a recent scandal in the Netherlands where a social psychology professor, who even won a national “Man of the Year” award, as well as as a very large amount of research money, was recently exposed as having faked all the data that went into a total number of articles running into three figures. This is not the sort of thing one wants to happen to one’s own university. He would have done well according to your REF .. before he was found out.

Human Resources departments have gained too much power, and are completely incompetent when it comes to judging academic standards. Let them get on with the old dull, and gobbledigook-free, tasks that personnel departments should be carrying out.

Yours,

5 July 2012.

Here’s another letter. It’s from a member of academic staff at QMUL, someone who is not himself threatened with being fired. It certainly shows that I’m not making a fuss about nothing. Rather, I’m the only person old enough to say what needs to be said without fear of losing my job and my house.

Dear Prof. Colquhoun,

I am an academic staff member in SBCS, QMUL. I am writing from my personal email account because the risks of using my work account to send this email are too great.

I would like to thank you for highlighting our problems and how we have been treated by our employer (Queen Mary University of London), in your blog. I would please urge you to continue to tweet and blog about our plight, and staff in other universities experiencing similarly horrific working conditions.

I am not threatened with redundancy by QMUL, and in fact my research is quite successful. Nevertheless, the last nine months have been the most stressful of all my years of academic life. The best of my colleagues in SBCS, QMUL are leaving already and I hope to leave, if I can find another job in London.

Staff do indeed feel very unfairly treated, intimidated and bullied. I never thought a job at a university could come to this.

Thank you again for your support. It really does matter to the many of us who cannot really speak out openly at present.

Best regards,

In a later letter, the same person pointed out

"There are many of us who would like to speak more openly, but we simply cannot."

"I have mortgage . . . . Losing my job would probably mean losing my home too at this point."

"The plight of our female staff has not even been mentioned. We already had very few female staff. And with restructuring, female staff are more likely to be forced into teaching-only contracts or indeed fired"."

"total madness in the current climate – who would want to join us unless desperate for a job!"

“fuss about nothing” – absolutely not. It is potentially a perfect storm leading to teaching and research disaster for a university! Already the reputation of our university has been greatly damaged. And senior staff keep blaming and targeting the “messengers"."

6 July 2012.

Througn the miracle of WiFi, this is coming from Newton, MA. The Lancet today has another editorial on the Queen Mary scandal.

"As hopeful scientists prepare their applications to QMUL, they should be aware that, behind the glossy advertising, a sometimes harsh, at times repressive, and disturbingly unforgiving culture awaits them."

That sums it up nicely.

24 July 2012. I’m reminded by Nature writer, Richard van Noorden (@Richvn) that Nature itself has wriiten at least twice about the iniquity of judging people by impact factors. In 2005 Not-so-deep impact said

"Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations."

"None of this would really matter very much, were it not for the unhealthy reliance on impact factors by administrators and researchers’ employers worldwide to assess the scientific quality of nations and institutions, and often even to judge individuals."

And, more recently, in Assessing assessment” (2010).

29 July 2012. Jonathan L Rees. of the University of Edinburgh, ends his blog:

"I wonder what career advice I should offer to a young doctor circa 2012. Apart from not taking a job at Queen Mary of course. "

How to select candidates

I have, at various times, been asked how I would select candidates for a job, if not by counting papers and impact factors. This is a slightly modified version of a comment that I left on a blog, which describes roughly what I’d advocate

After a pilot study the entire Research Excellence Framework (which attempts to assess the quality of research in every UK university) made the following statement.

“No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs”

It seems that the REF is paying attention to the science not to bibliometricians.

It has been the practice at UCL to ask people to nominate their best papers (2 -4 papers depending on age). We then read the papers and asked candidates hard questions about them (not least about the methods section). It’s a method that I learned a long time ago from Stephen Heinemann, a senior scientist at the Salk Institute. It’s often been surprising to learn how little some candidates know about the contents of papers which they themselves select as their best. One aim of this is to find out how much the candidate understands the principles of what they are doing, as opposed to following a recipe.

Of course we also seek the opinions of people who know the work, and preferably know the person. Written references have suffered so much from ‘grade inflation’ that they are often worthless, but a talk on the telephone to someone that knows both the work, and the candidate, can be useful, That, however, is now banned by HR who seem to feel that any knowledge of the candidate’s ability would lead to bias.

It is not true that use of metrics is universal and thank heavens for that. There are alternatives and we use them.

Incidentally, the reason that I have described the Queen Mary procedures as insane, brainless and dimwitted is because their aim to increase their ratings is likely to be frustrated. No person in their right mind would want to work for a place that treats its employees like that, if they had any other option. And it is very odd that their attempt to improve their REF rating uses criteria that have been explicitly ruled out by the REF. You can’t get more brainless than that.

This discussion has been interesting to me, if only because it shows how little bibliometricians understand how to get good science.

Tagged Academia, assessment, citations, David Bignell, impact factor, metrics, Quality assessment, Queen Mary, Queen Mary University of London, Simon Gaskell, Tom Macdonald, Universities | 76 Comments

Bait and switch by herbalists, Chinese and Western. Simon Mills speaks.

Published June 10, 2012

Jump to follow-up

Although many university courses in quackery have now closed, two subjects that hang on in a few places are western herbalism, and traditional Chinese medicine (including acupuncture). The University of Westminster still runs Chinese medicine, and Western herbal medicine (with dowsing). So do the University of Middlesex and University of East London.

Since the passing of the Health and Social Security Act, these people have been busy with their customary bait and switch tactics, trying to get taxpayers’ money. It’s worth looking again at the nonsense these people talk.

Take for example, the well known herbalist, Simon Mills. At one time he was associated with the University of Exeter, but no longer. Perhaps his views are too weird even for their Third Gap section (the folks who so misrepresented their results in a trial of acupuncture). Unsurprisingly, he was involved in the late Prince’s Foundation for Magic Medicine, and, unsurprisingly, he is involved with its successor, the "College of Medicine", where he spoke along similar lines. You can get a good idea about his views from the video of a talk that he gave at Schumacher College in 2005. It’s rather long, and exceedingly uncritical, so here’s a shorter version to which some helpful captions have been added.

That talk is weird by any standards. He says, apparently with a straight face, that "all modern medicines are cold in the third degree"..And with ginger and cinnamon "You can stop a cold, generally speaking, in its tracks" (at 21′ 30" in the video). This is simply not true, but he says it, despite the fact that the Plant Medicine with site (of which he’s a director) which he is associated gives them low ratings

Simon Mills is also a director of SustainCare. Their web site says

SustainCare Community Interest Company is a social enterprise set up to return health care to its owners: “learning to look after ourselves and our families in ways that make sense and do not cost the earth“. It is founded on the principle that one’s health is a personal story, and that illness is best managed when we make our health care our own. The enterprise brings clinical expertise, long experience of academia, education and business, and the connections and resources to deliver new approaches.

"As its own social enterprise contribution to this project Sustaincare set up and supported Café Sustain as a demonstration Intelligent Waiting Room at Culm Valley Integrated Centre for Health in Devon". (yes, that’s Michael Dixon, again]

In the talk (see video) Mills appears to want to take medicine back to how it was 1900 years ago, in the time of Galen. The oblique speaking style is fascinating. He never quite admits that he thinks all that nonsense is true, but presumably it is how he treats patients. Yet a person with these bizarre pre-scientific ideas is thought appropriate to advise the MHRA

It’s characteristic of herbalists that they have a very long list of conditions for which each herb is said to be good. The sort of things said by Mills differ little from the 1900-year old ideas of Galen, io the 17th century ideas of Culpepper.

You can see some of the latter in my oldest book, Blagrave’s supplement to Culpepper’s famous herbal, published in 1674.

See what he has to say about daffodils

daff

It is "under the dominion of Mars, and the roots hereof are hot and dry almost in the third degree".

"The root, boyled in posset drink, and drunk, causeth vomiting, and is used with good successe in the beginning of Agues, especiallyTertians, which frequently rage in the spring-time: a plaister made of the roots with parched Barley meal, and applied to swellings and imposthumes do dissolve them; the juice mingled with hony, frankincense, wine and myrrhe, and dropped into the Eares, is good against the corrupt filth and running matter of the Eares; the roots made hollow and boyled in oyl doth help Kib’d heels [or here ]: the juice of the root is good for Morphew, and discolourings of the skin."

It seems that daffodils would do a lot in 1674. Even herbalists don’t seem to use it much now. A recent herbal site describes daffodil as "poisonous".

But the descriptions are very like those used by present day herbalists, as you can hear in Simon Mills’ talk.

Chinese medicine is even less tested than western herbs. Not a single Chinese herb has been shown to be useful for treating anything (though in a very few case, they have been found to contain drugs that are useful when purified, notably the anti-malarial compound, artemesinin). They are often contaminated, some are dangerously toxic. And they contribute to the extinction of tigers and rhinoceros because of the silly myths that these make useful medicines. The cruelty of bear bile farming is legendary.

In a recent report in China Daily (my emphasis).

In a congratulation letter, Vice-Premier Li Keqiang called for integration of TCM and Western medicine.

TCM, as a time-honored treasure of Chinese civilization, has contributed to the prosperity of China and brought impacts to world civilization, Li said.

He also urged medical workers to combine the merits of TCM with contemporary medicine to better facilitate the ongoing healthcare reform in China.

The trade in Chinese medicines survives only for two reasons. One is that thay are a useful tool for promoting Chinese nationalism. The other is that they are big business. Both are evident in the vice-premier’s statement.

I presume that it’s the business bit that is the reason why London South Bank University (ranked 114 ou ot 114) that led to one of their main lecture theatres being decorated with pictures like this.“Mr Li Changchun awarding 2010 Confucius Institute of the year to LSBU Vice Chancellor” . I’ll bet Mr Li Changchun uses real medicine himself, as most Chinese who can afford it do.

SBU

Presumably, what’s taught in their Confucius Institute is the same sort of dangerous make-believe nonsense.that’s taught on other such courses.

The "College of Medicine" run a classical bait and switch operation. Their "First Thursday lectures" have several good respectable speakers, but then they have Andrew Flower, He is "a former president of the Register of Chinese Herbal Medicine, a medical herbalist and acupuncturist. He has recently completed a PhD exploring the role of Chinese herbal medicine in the treatment of endometriosis". He’s associated with the Avicenna Centre for Chinese Medicine, and with the University of Southampton’s quack division The only bit of research I could find by Andrew Flower was a Cochrane review, Chinese herbal medicine for endometriosis. The main results tell us

"Two Chinese RCTs involving 158 women were included in this review. Both these trials described adequate methodology. Neither trial compared CHM with placebo treatment."

But the plain language summary says

"This review suggests that Chinese herbal medicine (CHM) may be useful in relieving endometriosis-related pain with fewer side effects than experienced with conventional treatment."

It sounds to me as though people as partisan as the authors of this should not be allowed to write Cochrane reviews.

Flower’s talk is followed by one from the notorious representative of the herbal industry, Michael McIntyre, talking on Herbal medicine: A major resource for the 21st century. That’s likely to be about as objective as if they’d invited a GSK drug rep to talk about SSRIs.

The people at Kings College London Institute of Pharmaceutical Sciences are most certainly not quacks. They have made a database of chemicals found in traditional Chinese medicine. It’s sold by a US company, Chem-TCM and it’s very expensive (Commercial license: $3,740.00. Academic/government license: $1,850.00). Not much open access there. It’s a good idea to look at chemicals of plant origin, but only as long as you don’t get sucked into the myths. It’s only too easy to fall for the bait and switch of quacks (like TCM salespeople). The sample page shows good chemical and botanical information, and predicted (not observed) pharmacological activity. More bizarrely, it shows also analysis of the actions claimed by TCM people.

It does seem odd to me to apply sophisticated classification methods to things that are mostly myth.

The multiple uses claimed for Chinese medicines are very like the make-believe claims made for western herbs by Galen, Culpepper and (with much less excuse) by Mills.

They are almost all untrue, but their proponents are good salesmen. Don’t let them get a foot in your door.

Follow-up

10 June 2012. No sooner did this post go public when I can across what must be one of the worst herbal scams ever: “Arthroplex“

31 July 2012. Coffee is the subject of another entry in the 1674 edition of Blagrave.

blagrave2

Blagrave evidently had a lower regard for coffee than I have.

“But being pounded and baked, as do it to make the Coffee-liquor with, it then stinks most loathsomly, which is an argument of some Saturnine quality in it.”

“But there is no mention of an medicinal use thereof, by any Author either Antient of Modern”

Blagrave says also

“But this I may truly say of it [coffee]: Quod Anglorum Corpora quae huic liquori, tantopere indulgent, in Barbarorum naturam degenerasse videntur,”

This was translated expertly by Benet Salway, of UCL’s History department

“that the bodies of the English that indulge in this liquor to such an extent seem to degenerate into the nature of barbarians”

My boss, Lucia Sivolotti got something very like that herself. Be very impressed.

Salway suggested that clearer Latin would have been “quod corpora Anglorum, qui tantopere indulgent huic liquori, degenerasse in naturam barbarorum videntur”.

I’d have passed that on to Blagrave, but I can’t find his email address.

I much prefer Alfréd Rényi’s aphorism (often misattributed to Paul Erdös)

“A mathematician is a machine for turning coffee into theorems”

Tagged acupuncture, alternative medicine, Anti-science, CAM, Chinese herbal mediicine, herbal medicine, herbalism, Middlesex university, quackademia, Simon Mills, TCM, Traditional Chinese medicine, University of East London, University of Westminster, Westminster university | 16 Comments

The demise of quackademia. Progress in the last 5 years leaves Michael Driscoll and Geoffrey Petts isolated.

Published January 1, 2012

Jump to follow-up

Since writing about anti-scientific degrees in Nature (March 2007), much has been revealed about the nonsense that is taught on these degrees. New Year’s day seems like a good time to assess how far we’ve got, five years on.

At the beginning of 2007 UCAS (the universities central admission service) offered 45 different BSc degrees in quackery, at 16 universities.

Now there are only 24 such degrees.

If you exclude chiropractic and osteopathy, which all run at private colleges, with some sort of "validation" from a university, there are now only 18 BSc/MSc courses being offered in eight universities.

Degrees in homeopathy, naturopathy and "nutritional therapy", reflexology and aromatherapy have vanished altogether from UCAS.

In the race to provide BScs in anti-science, Middlesex University has now overhauled the long-standing leader, Westminster, by a short head.

Michael Driscoll, vice-chancellor of Middlesex

Petts
Geoffrey Petts, vice-chancellor of Westminster

Let’s see what’s gone.

The University of Central Lancashire (UCLAN) was the first to see sense. In August 2008 they announced closure of their “BSc” degree in homeopathy. On September 2008 they announced an internal review of their courses in homeopathy. herbalism and acupuncture. The report of this review closed down all of them in July 2009. I first asked for their teaching materials in July 2006. I finally got them in December 2010, after winning an appeal to the Information Commissioner, and then winning an appeal against that decision at an Information tribunal . By the time I got them, the course had been closed for over two years. That is just as well, because it turned out that UCLAN’s students were being taught dangerous nonsense. No wonder they tried so hard to conceal it.

Salford University was the next to go. They shut down their courses in complementary medicine, homeopathy and acupuncture. In January 2009 they announced " they are no longer considered “a sound academic fit” ". Shortly afterwards. a letter appeared in The Times from three heavyweights (plus me) congratulating the vice-chancellor on his decision.

University of Westminster

For many years, Westminster was the biggest supplier of BSc degrees in quackery. At the beginning of 2007 they offered 14 different BSc degrees in homeopathy, naturopathy, nutritional therapy, "complementary therapies", (western) herbal medicine and traditional Chinese medicine with acupuncture. Some of their courses were so bizarre that some of the students and even staff sent me slides which taught things like "amethysts emit high Yin energy". Like UCLAN, Westminster also held an internal review. Unlike UCLAN it came to the absurd conclusion that all would be well if they injected more science into the courses. The incompetence of the review meant that those who wrote it hadn’t noticed that if you try to put science into homeopathy or naturopathy, the whole subject vanishes in a puff of smoke. Nevertheless Westminster closed down entry to BSc homeopathy in March 2009 (though the subject remained as part of other courses).

Three years after the Nature article, all five BSc homeopathy degrees had shut their doors.

During 2011, Westminster shut down Naturopathy, Nutritional therapy, Therapeutic bodywork and Complementary Medicine. See, for example,
More dangerous nonsense from the University of Westminster: when will Professor Geoffrey Petts do something about it?

Professor Geoffrey Petts of the University of Westminster says they “are not teaching pseudo-science”. The facts show this is not true

University of Westminster shuts down naturopathy, nutritional therapy, but keeps Acupuncture and Herbal Medicine

Now Westminster has only four courses in two subjects. They still teach some dangerous and untrue things, but I suspect the writing is on the wall for these too.

I have seen a document, dated 11 April 2011, which states

“The following courses have been identified as ‘at risk’ (School definition) and will be discussed at the APRG and University Review Group2, due to poor recruitment and high cost of delivery:
 Integrated Health Scheme: BSc Complementary Medicine, Naturopathy; BSc Chinese Medicine; BSc Nutritional Therapy; BSc Herbal Medicine”

All but Chinese medicine and Herbal medicine have already gone. Almost there.

University of Wales

Since my first post in 2008 about the validation scam operated by the University of Wales, and some good investigations by BBC Wales TV, the outcome was the most spectacular so far. The entire institution collapsed. They no longer "validate" external degrees at dodgy business colleges, loony religious colleges or magic medicine colleges.

Another worthless validation: the University of Wales and nutritional therapy (October 2008) This is a ‘degree’ in nutrtional therapy. It is even more hilarious than usual, but it passed the validation anyway.

Scandal of the University of Wales and the Quality Assurance Agency (November 2010). This post followed the BBC Wales TV programme. At last the QAA began to notice, yet further confirmation of its utter ineptitude.

The University of Wales disgraced (but its vice chancellor is promoted) (October, 2011) The eventual collapse of the university was well-deserved. But it is very weird that the people who were responsible for it have still got their jobs. In fact the vice-chancellor, Marc Clement, was promoted despite his mendacious claim to be unaware of what was going on.

It remains to be seen how many of the many quack courses that were validated by the University of Wales will be taken on by other universities. The McTimoney College of Chiropractic is owned by BPP University (so much for their quality control, as explained in Private Eye). but still claims to be validated by Wales until 2017.

Some of the more minor players

Edinburgh Napier University. After an FOI request (rejected), Napier closed their herbal medicine degree in 2010.

Hot and cold herbal nonsense from Napier University Edinburgh: another course shuts. (June 2010)

As expected, the Scottish Information Commissioner agreed with that for England and Wales and ordered material to be sent. Edinburgh Napier University teaches reflexology, aromatherapy and therapeutic touch. Scottish Information Commissioner says you should know. Some of the horrors so discovered appeared in Yet more dangerous nonsense inflicted on students by Edinburgh Napier University. The embarrassment seems to have worked. Their remaining degrees in aromatherapy and reflexology have now vanished from UCAS too. All that remains is a couple of part time “Certificates of Credit” for aromatherapy and reflexology

Anglia Ruskin Univerity Not only have BSc degrees gone in aromatherapy and reflexology, but their midwifery degree now states "We are unable to accept qualifications in aromatherapy, massage and reflexology."

University of Derby Reflexology and aromatherapy have gone, though doubtless Spa management therapies have much nonsense left

University of Greenwich. BSc in Complementary Therapies (Nutritional Health) and BSc in Complementary Therapies (Nutritional Health) have been shut. The BSc Acupuncture is listed on their web site but it is under review, and is not listed in UCAS for 2012. (Acupuncture is run at International College of Oriental medicine, validated by Greenwich.). Only osteopathy (MOst) is still running, and that is a validation of an external course run at The European School of Osteopathy, in Maidstone

Thames Valley University was renamed the University of West London in 2010. The nonsense that was run there (e.g. Nutritional Fairy Tales from Thames Valley University) seems to have vanished. Their previous alt med guru, Nicola Robinson, appears now to be at London South Bank University (ranked 116 out of the 116 UK universities)

What’s left?

Chiropractic Surprisingly, given the total discreditation of chiropractic in the wake of the Simon Singh affair, and the internecine warfare that followed it, none of the chiropractic courses have shut yet. Some are clearly in trouble, so watch this space.

Osteopathy has also had no course closures since 2007. Like chiropractic it also suffers from internecine warfare. The General Osteopathic Council refuses to disown the utter nonsense of "craniosacral" osteopathy. But the more sensible practitioners do so and are roughly as effective as physiotherapists (though there are real doubts about how effective that is).

Excluding chiropractic and osteopathy, this is all that’s left. It now consists almost entirely of Chinese medicine and a bit of herbal.

Glyndwr university (Known as North East Wales Institute until 2008) Ranked 104 out of 116 UK universities

BSc Acupuncture (B341) BSc
BSc Complementary Therapies for Healthcare (B343)

Cardiff Metropolitan University (UWIC) (Known as University of Wales Institute Cardiff (UWIC) until Nov 2011.) The vice-chancellor of Cardiff Metropolitan, Antony Chapman, is in the QAA’s board of directors, so perhaps it isn’t surprising that the QAA has done nothing.

BSc Complementary Therapies (3 years) (B390)
BSc Complementary Therapies (4 yrs inc Foundation) (B300)

University of Lincoln

Acupuncture (B343) 3FT Hon BSc
Herbal Medicine (B342) 3FT Hon BSc

University of East London Ranked 113 out of 116 UK universities

Acupuncture (B343) 3FT Hon BSc

London South Bank University Ranked 116 out of 116 UK universities

Acupuncture (B343) 4FT Deg MCM

The Manchester Metropolitan University Ranked 93 out of 116 UK universities

Acupuncture (B343) 3FT Hon BSc

Middlesex University

Acupuncture (B348) 3FT Hon BSc
Ayurvedic Medicine (A900) 4FT Oth MCM
Herbal Medicine (B347) 3FT Hon BSc
Traditional Chinese Medicine (BT31) 4FT Hon BSc

University of Westminster

Chinese Medicine: Acupuncture (B343) 3FT Hon BSc
Chinese Medicine: Acupuncture with Foundation (B341) 4FT/5FT Hon BSc/MSci
Herbal Medicine (B342) 3FT Hon BSc
Herbal Medicine with Foundation Year (B340) 4FT/5FT Hon BSc/MSci

It seems that acupuncture hangs on in universities that are right at the bottom of the rankings.

Manchester Metropolitan gets the booby prize for actually starting a new course, just as all around are closing theirs. Dr Peter Banister, who was on the committee that approved the course (but now retired), has told me ” I am sceptical in the current economic climate whether it will prove to be successful”. Let’s hope he’s right.

But well done Westminster. Your position as the leader in antiscientific degrees has now been claimed by Middlesex University. Their "degrees" in Ayurveda mark out Middlesex University as the new King of Woo.

Over to you, Professor Driscoll. As vice-chancellor of Middlesex University, the buck stops with you.

Both still teach Chinese and herbal medicine, which are potentially dangerous. There is not a single product from either that has marketing authorisation from the MHRA, though the MHRA has betrayed its trust by allowing misleading labelling of herbal medicines without requiring any evidence whatsoever that they work, see, for example

Why degrees in Chinese medicine are a danger to patients
More quackedemia. Dangerous Chinese medicine taught at Middlesex University
Why does the MHRA refuse to label herbal products honestly? Kent Woods and Richard Woodfield tell me

Sub-degree courses

In contrast to the large reduction in the number of BSc and MSc degrees, there has actually been an increase in two year foundation degrees and HND courses in complementary medicine, at places right near the bottom of the academic heap. The subject is sinking to the bottom. With luck it will vanish entirely from universities before too long.

Research-intensive Universities

Although all of the degrees in magic medicine are from post-1992 universities, the subject has crept into more prestigious universities. Of these, the University of Southampton is perhaps the worst, because of the presence of George Lewith, and his defender, Stephen Holgate. Others have staunch defenders of quackery, including the University of Warwick, University of Edinburgh and St Batholomew’s.

Why have all these courses closed?

One reason is certainly the embarrassment caused by exposure of what’s taught on the courses. Professors Petts (Westminster) and Driscoll (Middlesex) must be aware that googling their names produces references to this and other skeptical blogs on the front page. Thanks to some plain brown emails, and, after a three year battle, the Freedom of Information Act, it has been possible to show here the nonsense that has been foisted on students by some universities. Not only is this a burden on the taxpayer, but, more importantly, some of it is a danger to patients.

When a course closes, it is often said that it is because of falling student numbers (though UCLAN and Salford did not use that excuse). Insofar as that is true, the credit must go to the whole of the skeptical movement that has grown so remarkably in the last few years. Ben Goldacre’s "ragged band of bloggers" have produced a real change in universities and in society as a whole.

The people who should have done the job have either been passive or an active hindrance. The list is long. Vice-chancellors and Universities UK (UUK), the Quality Assurance Agency (QAA), the Hiigher Education Funding Council England (HEFCE), Skills for Health, the Medicines and Health Regulatory Authority ( MHRA) , the Health Professions Council (HPC), the Department of Health, the Prince of Wales and his reincarnated propaganda organisation, the "College of Medicine", the King’s Fund, the Universities and Colleges Union (UCU), OfQual, Edexcel, National Occupational Standards and Qualifications and the Curriculum Authority (QCA).

Whatever happened to that "bonfire of the quangos"?

Follow-up

2 January 2012 The McTimoney College of Chiropractic (owned by BPP University) claims that its “validation” by the University of Wales will continue until 2017. This contradicts the statement from UoW. Watch this space.

3 January 2012. Thanks to Neil O’Connell for drawing my attention to a paper in Pain. The paper is particularly interesting because it comes from the Southampton group which has previously been sympathetic to acupuncture. Its authors include George Lewith. It shows, yet again that there is no detectable difference between real and sham acupuncture treatment. It also shows that the empathy of the practitioner has little effect: in fact the stern authoritarian practitioner may have been more effective.

Patients receiving acupuncture demonstrated clinically important improvements from baseline (i.e., a 29.5% reduction in pain), but despite this, acupuncture has no specific efficacy over placebo for this group of patients. The clinical effect of acupuncture treatment and associated controls is not related to the use of an acupuncture needle, nor mediated by empathy, but is practitioner related and may be linked to the perceived authority of the practitioner.”

Sadly. the trial didn’t include a no-treatment group, so it is impossible to say how much of the improvement is regression to the mean and how much is a placebo effect. The authors admit that it could be mostly the former.

Surely now the misplaced confidence in acupuncture shown by some medical and university people must be in tatters.

In yet another sign that even acupuncture advovates are beginning to notice that it doesn’t work, a recent article Paradoxes in Acupuncture Research: Strategies for Moving Forward, shows some fascinating squirming.

3 January 2012. The Daily Telegraph has carried a piece about closure of university courses, written by Michael Hanlon. On 31 January they carried a much longer piece.

3 January 2012. It is a great pity that some physiotherapists seem to have fallen hook, line and sinker for the myths of acupuncture. Physiotherapists are, by and large, the respectable face of manipulative therapy. Their evidence base is certainly not all one would wish, but at least they are free of the outrageous mumbo humbo of chiropractors. Well, most of them are, but not the Acupuncture Association of Chartered Physiotherapists (AACP), or, still worse, The Association of Chartered Physiotherapists in Energy Medicine, a group that is truly away with the fairies. These organisations are bringing a very respectable job into disrepute. And the Health Professions Council, which is meant to be their regulator, has, like most regulators, done nothing whatsoever to stop it.

5 January 2012. Times Higher Education gives a history of the demise of the University of Wales, Boom or Bust. It’s a useful timeline, but like so many journalists, it’s unwilling to admit that bloggers were on to the problem long before the BBC, never mind the QAA.

There was also a leader on the same topic, Perils of the export business. It again fails to take the QAA to task for its failures.

Interviews for Deutsche Welle and Middle East Broadcasting Center TV.

17 January 2012 Another question answered. I just learned that the ludicrous course in Nutritional Therapy, previously validated by the University of Wales (and a contributor to its downfall), is now being validated by, yes, you guessed, Middlesex University. Professor Driscoll seems determined to lead his univerity to the bottom of the academic heap. His new partnership with the Northern college of Acupuncture is just one of a long list of validations that almost rivals that of the late University of Wales. The course has, of course, an enthusiastic testimonial, from a student. It starts

I work full time as a team leader for a pension company but I am also a kinesiologist and work in my spare time doing kinesiology, reiki and Indian head massage.

Evidently she’s a believer in the barmiest and totally disproved forms of magic medicine. And Middlesex University will give her a Master of Science degree. I have to say I find it worrying that she’s a team leader for a pension company. Does she also believe in the value of worthless derivatives. I wonder?

18 January 2012. the story has gone international, with an interview that I did for Deutsche Welle, UK universities drop alternative medicine degree programs. I’m quoted as saying “They’re dishonest, they teach things that aren’t true, and things that are dangerous to patients in some cases”. That seems fair enough.

There is also an interesting item from July 2010 about pressure to drop payment for homeopathy by German health insurance

31 January 2012

The Daily Telegraph carried a prominent 1200 word account (the title wasn’t mine). The published version was edited slightly.

Tagged Academia, alternative medicine, Anti-science, antiscience, badscience, CAM, Central Lancashire, Geoffrey Petts, herbal medicine, homeopathy, Michael Driscoll, Middlesex university, nutritional therapy, TCM, Traditional Chinese medicine, University of Westminster, Westminster university | 39 Comments

Open access, peer review, grants and other academic conundrums

Published December 16, 2011

Jump to follow-up

Open access is in the news again.

Index on Censorship held a debate on open data on December 6th.

The video of of the meeting is now on YouTube. A couple of dramatic moments in the video: At 48 min O’Neill & Monbiot face off about "competent persons" (and at 58 min Walport makes fun of my contention that it’s better to have more small grants rather than few big ones, on the grounds that it’s impossible to select the stars).

The meeting has been written up on the Bishop Hill Blog, with some very fine cartoon minutes.

Bishop Hill blog (I love the Josh cartoons -pity he seems to be a climate denier, spoken of approvingly by the unspeakable James Delingpole.)

It was gratifying that my remarks seemed to be better received by the scientists in the audience than they were by some other panel members. The Bishop Hill blog comments "As David Colquhoun, the only real scientist there and brilliant throughout, said “Give them everything!” " Here’s a subsection of the brilliant cartoon minutes

notes-dc

The bit about "I just lied -but he kept his job" referred to the notorious case of Richard Eastell and the University of Sheffield.

We all agreed that papers should be open for anyone to read, free. Monbiot and I both thought that raw data should be available on request, though O’Neill and Walport had a few reservations about that.

A great deal of time and money would be saved if data were provided on request. It shouldn’t need a Freedom of Information Act (FOIA) request, and the time and energy spent on refusing FOIA requests is silly. It simply gives the impression that there is something to hide (Climate scientists must be ruthlessly honest about data). The University of Central Lancashire spent £80,000 of taxpayers’ money trying (unsuccessfully) to appeal against the judgment of the Information Commissioner that they must release course material to me. It’s hard to think of a worse way to spend money.

A few days ago, the Department for Business, Innovation and Skills (BIS) published a report which says (para 6.6)

“The Government . . . is committed to ensuring that publicly-funded research should be accessible
free of charge.”

That’s good, but how it can be achieved is less obvious. Scientific publishing is, at the moment, an unholy mess. It’s a playground for profiteers. It runs on the unpaid labour of academics, who work to generate large profits for publishers. That’s often been said before, recently by both George Monbiot (Academic publishers make Murdoch look like a socialist) and by me (Publish-or-perish: Peer review and the corruption of science). Here are a few details.

Extortionate cost of publishing

Mark Walport has told me that

The Wellcome Trust is currently spending around £3m pa on OA publishing costs and, looking at the Wellcome papers that find their way to UKPMC, we see that around 50% of this content is routed via the “hybrid option”; 40% via the “pure” OA journals (e.g. PLoS, BMC etc), and the remaining 10% through researchers self-archiving their author manuscripts.

I’ve found some interesting numbers, with help from librarians, and through access to The Journal Usage Statistics Portal (JUSP).

Elsevier

UCL pays Elsevier the astonishing sum of €1.25 million, for access to its journals. And that’s just one university. That price doesn’t include any print editions at all, just web access and there is no open access. You have to have a UCL password to see the results. Elsevier has, of course, been criticised before, and not just for its prices.

Elsevier publish around 2700 scientific journals. UCL has bought a package of around 2100 journals. There is no possibility to pick the journals that you want. Some of the journals are used heavily ("use" means access of full text on the web). In 2010, the most heavily used journal was The Lancet, followed by four Cell Press journals

elsevier top

But notice the last bin. Most of the journals are hardly used at all. Among all Elsevier journals, 251 were not accessed even once in 2010. Among the 2068 journals bought by UCL, 56 were never accessed in 2010 and the most frequent number of accesses per year is between 1 and 10 (the second bin in the histogram, below). 60 percent of journals have 300 or fewer usages in 2010, Above 300, the histogram tails on up to 51878 accesses for The Lancet. The remaining 40 percent of journals are represented by the last bin (in red). The distribution is exceedingly skewed. The median is 187, i.e. half of the journals had fewer than 187 usages in 2010), but the mean number of usages (which is misleading for such a skewed distribution, was 662 usages).

histo

Nature Publishing Group

UCL bought 65 journals from NPG in 2010. They get more use than Elsevier, though surprisingly three of them were never accessed in 2010, and 17 had fewer than 1000 accesses in that year. The median usage was 2412, better than most. The leader, needless to say, was Nature itself, with 153,321.

Oxford University Press

The situation is even more extreme for 248 OUP journals, perhaps because many of the journals are arts or law rather than science.

OUP-jisto

The most frequent (modal) usage of was zero (54 journals), followed by 1 to 10 accesses (42 journals) 64 percent of journals had fewer than 200 usages, and the 36 percent with over 200 are pooled in the last (red) bin. The histogram extends right up to 16060 accesses for Brain. The median number of usages in 2010 was 66.

So far I haven’t been able to discover the costs of the contracts with OUP or Nature Publishing group. It seems that the university has agreed to confidentiality clauses. This itself is a shocking lack of transparency. If I can find the numbers I shall -watch this space.

Almost all of these journals are not open access. The academics do the experiments, most often paid for by the taxpayer. They write the paper (and now it has to be in a form that is almost ready for publication without further work), they send it to the journal, where it is sent for peer review, which is also unpaid. The journal sells the product back to the universities for a high price, where the results of the work are hidden from the people who paid for it.

It’s even worse than that, because often the people who did the work and wrote the paper, have to pay "page charges". These vary, but can be quite high. If you send a paper to the Journal of Neuroscience, it will probably cost you about $1000. Other journals, like the excellent Journal of Physiology, don’t charge you to submit a paper (unless you want a colour figure in the print edition, £200), but the paper is hidden from the public for 12 months unless you pay $3000.

The major medical charity, the Wellcome Trust, requires that the work it funds should be available to the public within 6 months of publication. That’s nothing like good enough to allow the public to judge the claims of a paper which hits the newspapers the day that it’s published. Nevertheless it can cost the authors a lot. Elsevier journals charge $3000 except for their most-used journals. The Lancet charges £400 per page and Cell Press journals charge $5000 for this unsatisfactory form of open access.

Open access journals

The outcry about hidden results has resulted in a new generation of truly open access journals that are open to everyone from day one. But if you want to publish in them you have to pay quite a lot.

Furthermore, although all these journals are free to read, most of them do not allow free use of the material they publish. Most are operating under all-rights-reserved copyrights. In 2009 under 10 percent of open access journals had true Creative Commons licence.

Nature Publishing Group has a true open access journal, Nature Communications, but it costs the author $5000 to publish there. The Public Library of Science journals are truly open access but the author is charged $2900 for PLoS Medicine though PLoS One costs the author only $1350

A 2011 report considered the transition to open access publishing but it doesn’t even consider radical solutions, and makes unreasonably low estimates of the costs of open access publishing.

Scam journals have flourished under the open access flag

Open access publishing has, so far, almost always involved paying a hefty fee. That has brought the rats out of the woodwork and one gets bombarded daily with offers to publish in yet another open access journal. Many of these are simply scams. You pay, we put it on the web and we won’t fuss about quality. Luckily there is now a guide to these crooks: Jeffrey Beall’s List of Predatory, Open-Access Publishers.

One that I hear from regularly is Bentham Open Journals

(a name that is particularly inappropriate for anyone at UCL). Jeffery Beall comments

"Among the first, large-scale gold OA publishers, Bentham Open continues to expand its fleet of journals, now numbering over 230. Bentham essentially operates as a scholarly vanity press."

They undercut real journals. A research article in The Open Neuroscience Journal will cost you a mere $800. Although these journals claim to be peer-reviewed, their standards are suspect. In 2009, a nonsensical computer-generated spoof paper was accepted by a Bentham Journal (for $800),

What can be done about publication, and what can be done about grants?

Both grants and publications are peer-reviewed, but the problems need to be discussed separately.

Peer review of papers by journals

One option is clearly to follow the example of the best open access journals, such as PLoS. The cost of $3000 to 5000 per paper would have to be paid by the research funder, often the taxpayer. It would be money subtracted from the research budget, but it would retain the present peer review system and should cost no more if the money that were saved on extortionate journal subscriptions were transferred to research budgets to pay the bills, though there is little chance of this happening.

The cost of publication would, in any case, be minimised if fewer papers were published, which is highly desirable anyway.

But there are real problems with the present peer review system. It works quite well for journals that are high in the hierarchy. I have few grumbles myself about the quality of reviews, and sometimes I’ve benefitted a lot from good suggestions made by reviewers. But for the user, the process is much less satisfactory because peer review has next to no effect on what gets published in journals. All it influences is which journal the paper appears in. The only effect of the vast amount of unpaid time and effort put into reviewing is to maintain a hierarchy of journals, It has next to no effect on what appears in Pubmed.

For authors, peer review can work quite well, but

from the point of view of the consumer, peer review is useless.

It is a myth that peer review ensures the quality of what appears in the literature.

A more radical approach

I made some more radical suggestions in Publish-or-perish: Peer review and the corruption of science.

It seems to me that there would be many advantages if people simply published their own work on the web, and then opened the comments. For a start, it would cost next to nothing. The huge amount of money that goes to publishers could be put to better uses.

Another advantage would be that negative results could be published. And proper full descriptions of methods could be provided because there would be no restrictions on length.

Under that system, I would certainly send a draft paper to a few people I respected for comments before publishing it. Informal consortia might form for that purpose.

The publication bias that results from non-publication of negative results is a serious problem, mainly, but not exclusively, for clinical trials. It is mandatory to register a clinical trial before it starts, but many of the results never appear. (see, for example, Deborah Cohen’s report for Index on Censorship). Although trials now have to be registered before they start, there is no check on whether or not the results are published. A large number of registered trials do not result in any publication, and this publication bias can costs thousands of lives. It is really important to ensure that all results get published,

The ArXiv model

There are many problems that would have to be solved before we could move to self-publication on the web. Some have already been solved by physicists and mathematicians. Their archive, ArXiv.org provides an example of where we should be heading. Papers are published on the web at no cost to either user or reader, and comments can be left. It is an excellent example of post-publication peer review. Flame wars are minimised by requiring users to register, and to show they are bona fide scientists before they can upload papers or comments. You may need endorsement if you haven’t submitted before.

Peer review of grants

The problems for grants are quite different from those for papers. There is no possibility of doing away with peer review for the award of grants, however imperfect the process may be. In fact candidates for the new Wellcome Trust investigator awards were alarmed to find that the short listing of candidates for their new Investigator Awards was done without peer review.

The Wellcome Trust has been enormously important for the support of medical and biological support, and never more than now, when the MRC has become rather chaotic (let’s hope the new CEO can sort it out). There was, therefore, real consternation when Wellcome announced a while ago its intention to stop giving project and programme grants altogether. Instead it would give a few Wellcome Trust Investigator Awards to prominent people. That sounds like the Howard Hughes approach, and runs a big risk of “to them that hath shall be given”.

The awards have just been announced, and there is a good account by Colin Macilwain in Science [pdf]. UCL did reasonable well with four awards, but four is not many for a place the size of UCL. Colin Macilwain hits the nail on the head.

"While this is great news for the 27 new Wellcome Investigators who will share £57 million, hundreds of university-based researchers stand to lose Wellcome funds as the trust phases out some existing programs to pay for the new category of investigators".

There were 750 applications, but on the basis of CV alone, they were pared down to a long-list if 173. The panels then cut this down to a short-list of 55. Up to this point no external referees were used, quite unlike the normal process for award of grants. This seems to me to have been an enormous mistake. No panel, however distinguished, can have the knowledge to distinguish the good from the bad in areas outside their own work, It is only human nature to favour the sort of work you do yourself. The 55 shortlisted people were interviewed, but again by a panel with an even narrower range of expertise, Macilwain again:

"Applications for MRC grants have gone up “markedly” since the Wellcome ones closed, he says: “We still see that as unresolved.” Leszek Borysiewicz, vice-chancellor of the University of Cambridge, which won four awards, believes the impact will be positive: “Universities will adapt to this way of funding research."

It certainly isn’t obvious to most people how Cambridge or UCL will "adapt" to funding of only four people.

The Cancer Research Campaign UK has recently made the same mistake.

One problem is that any scheme of this sort will inevitably favour big groups, most of whom are well-funded already. Since there is some reason to believe that small groups are more productive (see also University Alliance report), it isn’t obvious that this is a good way to go. I was lucky enough to get 45 minutes with the director of the Wellcome Trust, Mark Walport, to put these views. He didn’t agree with all I said, but he did listen.

One of the things that I put to him was a small statistical calculation to illustrate the great danger of a plan that funds very few people. The funding rate was 3.6% of the original applications, and 15.6% of the long-listed applications. Let’s suppose, as a rough approximation, that the 173 long-listed applications were all of roughly equal merit. No doubt that won’t be exactly true, but I suspect it might be more nearly true than the expert panels will admit. A quick calculation in Mathcad gives this, if we assume a 1 in 8 chance of success for each application.

Distribution of the number of successful applications

Suppose $ n $ grant applications are submitted. For example, the same grant submitted $ n $ times to selection boards of equal quality, OR $ n $ different grants of equal merit are submitted to the same board.

Define $ p $ = probability of success at each application

Under these assumptions, it is a simple binomial distribution problem.

According to the binomial distribution, the probability of getting $ r $ successful applications in $ n $ attempts is

\[ P(r)=\frac{n!}{r!\left(n-r\right)! }\; {p}^{r} \left(1-p \right)^{n-r} \]

For a success rate of 1 in 8, $ p = 0.125 $, so if you make $ n = 8 $ applications, the probability that $ r $ of them will succeed is shown in the graph.

Despite equal merit, almost as many people end up with no grant at all as almost as many people end up with no grant at all as get one grant. And 26% of people will get two or more grants.

mcd1-g

Of course it would take an entire year to write 8 applications. If we take a more realistic case of making four applications we have $ n = 4 $ (and $ p = 0.125 $, as before). In this case the graph comes out as below. You have a nearly 60% chance of getting nothing at all, and only a 1 in 3 chance of getting one grant.

mcd3-g

These results arise regardless of merit, purely as consequence of random chance. They are disastrous, and especially disastrous for the smaller, better-value, groups for which a gap in funding can mean loss of vital expertise. It also has the consequence that scientists have to spend most of their time not doing science, but writing grant applications. The mean number of applications before a success is 8, and a third of people will have to write 9 or more applications before they get funding. This makes very little sense.

Grant-awarding panels are faced with the near-impossible task of ranking many similar grants. The peer review system is breaking down, just as it has already broken down for journal publications.

I think these considerations demolish the argument for funding a small number of ‘stars’. The public might expect that the person making the application would take an active part in the research. Too often, now, they spend most of their time writing grant applications. What we need is more responsive-mode smallish programme grants and a maximum on the size of groups.

Conclusions

We should be thinking about the following changes,

Limit the number of papers that an individual can publish. This would increase quality, it would reduce the impossible load on peer reviewers and it would reduce costs.
Limit the size of labs so that more small groups are encouraged. This would increase both quality and value for money.
More (and so smaller) grants are essential for innovation and productivity.
Move towards self-publishing on the web so the cost of publishing becomes very low rather than the present extortionate costs. It would also mean that negative results could be published easily and that methods could be described in proper detail.

The entire debate is now on YouTube.

Follow-up

24 January 2012. The eminent mathematician, Tim Gowers, has a rather hard-hitting blog on open access and scientific publishing, Elsevier – my part in its downfall. I’m right with him. Although his post lacks the detailed numbers of mine, it shows that mathematicians has exactly the same problems of the rest of us.

11 April 2012. Thanks to Twitter, I came across a remarkably prescient article, in the Guardian, in 2001.
Science world in revolt at power of the journal owners, by James Meek. Elsevier have been getting away with murder for quite a while.

19 April 2012.

I got invited to give after-dinner talk on open access at Cumberland Lodge. It was for the retreat of out GEE Department (that is the catchy brand name we’ve had since 2007: I’m in the equally memorable NPP). I think it stands for Genetics, Evolution and Environment. The talk seemed to stiir up a lot of interest: the discussions ran on to the next day.

It was clear that younger people are still as infatuated with Nature and Science as ever. And that, of course is the fault of their elders.

The only way that I can see, is to abandon impact factor as a way of judging people. It should have gone years ago,and good people have never used it. They read the papers. Access to research will never be free until we think oi a way to break the hegemony of Nature, Science and a handful of others. Stephen Curry has made some suggestions

Probably it will take action from above. The Wellcome Trust has made a good start. And so has Harvard. We should follow their lead (see also, Stephen Curry’s take on Harvard)

And don’t forget to sign up for the Elsevier boycott. Over 10,000 academics have already signed. Tim Gowers’ initiative took off remarkably.

24 July 2012. I’m reminded by Nature writer, Richard van Noorden (@Richvn) that Nature itself has written at least twice about the iniquity of judging people by impact factors. In 2005 Not-so-deep impact said

"Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations."

"None of this would really matter very much, were it not for the unhealthy reliance on impact factors by administrators and researchers’ employers worldwide to assess the scientific quality of nations and institutions, and often even to judge individuals."

And, more recently, in Assessing assessment” (2010).

27 April 2014

The brilliant mathematician,Tim Gowers, started a real revolt against old-fashioned publishers who are desperately trying to maintain extortionate profits in a world that has changed entirely. In his 2012 post, Elsevier: my part in its downfall, he declared that he would no longer publish in, or act as referee for, any journal published by Elsevier. Please follow his lead and sign an undertaking to that effect: 14,614 people have already signed.

Gowers has now gone further. He’s made substantial progress in penetrating the wall of secrecy with which predatory publishers (of which Elsevier is not the only example) seek to prevent anyone knowing about the profitable racket they are operating. Even the confidentiality agreements, which they force universities to sign, are themselves confidential.

In a new post, Tim Gowers has provided more shocking facts about the prices paid by universities. Please look at Elsevier journals — some facts. The jaw-dropping 2011 sum of €1.25 million paid by UCL alone, is now already well out-of-date. It’s now £1,381,380. He gives figures for many other Russell Group universities too. He also publishes some of the obstructive letters that he got in the process of trying to get hold of the numbers. It’s a wonderful aspect of the web that it’s easy to shame those who deserve to be shamed.

I very much hope the matter is taken to the Information Commissioner, and that a precedent is set that it’s totally unacceptable to keep secret what a university pays for services.

Tagged Academia, acupuncture, George Monbiot, Mark Walport, Onora O'Neill, open access, peer review, Public interest, publishing | 26 Comments

Why philosophy is largely ignored by science

Published October 28, 2011

Jump to follow-up

I have in the past, taken an occasional interest in the philosophy of science. But in a lifetime doing science, I have hardly ever heard a scientist mention the subject. It is, on the whole, a subject that is of interest only to philosophers.

It’s true that some philosophers have had interesting things to say about the nature of inductive inference, but during the 20th century the real advances in that area came from statisticians, not from philosophers. So I long since decided that it would be more profitable to spend my time trying to understand R.A Fisher, rather than read even Karl Popper. It is harder work to do that, but it seemed the way to go.

This post is based on the last part of chapter titled “In Praise of Randomisation. The importance of causality in medicine and its subversion by philosophers of science“. A talk was given at the meeting at the British Academy in December 2007, and the book will be launched on November 28th 2011 (good job it wasn’t essential for my CV with delays like that). The book is published by OUP for the British Academy, under the title Evidence, Inference and Enquiry (edited by Philip Dawid, William Twining, and Mimi Vasilaki, 504 pages, £85.00). The bulk of my contribution has already appeared here, in May 2009, under the heading Diet and health. What can you believe: or does bacon kill you?. It is one of the posts that has given me the most satisfaction, if only because Ben Goldacre seemed to like it, and he has done more than anyone to explain the critical importance of randomisation for assessing treatments and for assessing social interventions.

Having long since decided that it was Fisher, rather than philosophers, who had the answers to my questions, why bother to write about philosophers at all? It was precipitated by joining the London Evidence Group. Through that group I became aware that there is a group of philosophers of science who could, if anyone took any notice of them, do real harm to research. It seems surprising that the value of randomisation should still be disputed at this stage, and of course it is not disputed by anybody in the business. It was thoroughly established after the start of small sample statistics at the beginning of the 20th century. Fisher’s work on randomisation and the likelihood principle put inference on a firm footing by the mid-1930s. His popular book, The Design of Experiments made the importance of randomisation clear to a wide audience, partly via his famous example of the lady tasting tea. The development of randomisation tests made it transparently clear (perhaps I should do a blog post on their beauty). By the 1950s. the message got through to medicine, in large part through Austin Bradford Hill.

Despite this, there is a body of philosophers who dispute it. And of course it is disputed by almost all practitioners of alternative medicine (because their treatments usually fail the tests). Here are some examples.

“Why there’s no cause to randomise” is the rather surprising title of a report by Worrall (2004; see also Worral, 2010), from the London School of Economics. The conclusion of this paper is

“don’t believe the bad press that ‘observational studies’ or ‘historically controlled trials’ get – so long as they are properly done (that is, serious thought has gone in to the possibility of alternative explanations of the outcome), then there is no reason to think of them as any less compelling than an RCT.”

In my view this conclusion is seriously, and dangerously, wrong –it ignores the enormous difficulty of getting evidence for causality in real life, and it ignores the fact that historically controlled trials have very often given misleading results in the past, as illustrated by the diet problem.. Worrall’s fellow philosopher, Nancy Cartwright (Are RCTs the Gold Standard?, 2007), has made arguments that in some ways resemble those of Worrall.

Many words are spent on defining causality but, at least in the clinical setting the meaning is perfectly simple. If the association between eating bacon and colorectal cancer is causal then if you stop eating bacon you’ll reduce the risk of cancer. If the relationship is not causal then if you stop eating bacon it won’t help at all. No amount of Worrall’s “serious thought” will substitute for the real evidence for causality that can come only from an RCT: Worrall seems to claim that sufficient brain power can fill in missing bits of information. It can’t. I’m reminded inexorably of the definition of “Clinical experience. Making the same mistakes with increasing confidence over an impressive number of years.” In Michael O’Donnell’s A Sceptic’s Medical Dictionary.

At the other philosophical extreme, there are still a few remnants of post-modernist rhetoric to be found in obscure corners of the literature. Two extreme examples are the papers by Holmes et al. and by Christine Barry. Apart from the fact that they weren’t spoofs, both of these papers bear a close resemblance to Alan Sokal’s famous spoof paper, Transgressing the boundaries: towards a transformative hermeneutics of quantum gravity (Sokal, 1996). The acceptance of this spoof by a journal, Social Text, and the subsequent book, Intellectual Impostures, by Sokal & Bricmont (Sokal & Bricmont, 1998), exposed the astonishing intellectual fraud if postmodernism (for those for whom it was not already obvious). A couple of quotations will serve to give a taste of the amazing material that can appear in peer-reviewed journals. Barry (2006) wrote

“I wish to problematise the call from within biomedicine for more evidence of alternative medicine’s effectiveness via the medium of the randomised clinical trial (RCT).”

“Ethnographic research in alternative medicine is coming to be used politically as a challenge to the hegemony of a scientific biomedical construction of evidence.”

“The science of biomedicine was perceived as old fashioned and rejected in favour of the quantum and chaos theories of modern physics.”
“In this paper, I have deconstructed the powerful notion of evidence within biomedicine, . . .”

The aim of this paper, in my view, is not obtain some subtle insight into the process of inference but to try to give some credibility to snake-oil salesmen who peddle quack cures. The latter at least make their unjustified claims in plain English.

The similar paper by Holmes, Murray, Perron & Rail (Holmes et al., 2006) is even more bizarre.

“Objective The philosophical work of Deleuze and Guattari proves to be useful in showing how health sciences are colonised (territorialised) by an all-encompassing scientific research paradigm “that of post-positivism ” but also and foremost in showing the process by which a dominant ideology comes to exclude alternative forms of knowledge, therefore acting as a fascist structure. “,

It uses the word fascism, or some derivative thereof, 26 times. And Holmes, Perron & Rail (Murray et al., 2007)) end a similar tirade with

“We shall continue to transgress the diktats of State Science.”

It may be asked why it is even worth spending time on these remnants of the utterly discredited postmodernist movement. One reason is that rather less extreme examples of similar thinking still exist in some philosophical circles.

Take, for example, the views expressed papers such as Miles, Polychronis and Grey (2006), Miles & Loughlin (2006), Miles, Loughlin & Polychronis (Miles et al., 2007) and Loughlin (2007).. These papers form part of the authors’ campaign against evidence-based medicine, which they seem to regard as some sort of ideological crusade, or government conspiracy. Bizarrely they seem to think that evidence-based medicine has something in common with the managerial culture that has been the bane of not only medicine but of almost every occupation (and which is noted particularly for its disregard for evidence). Although couched in the sort of pretentious language favoured by postmodernists, in fact it ends up defending the most simple-minded forms of quackery. Unlike Barry (2006), they don’t mention alternative medicine explicitly, but the agenda is clear from their attacks on Ben Goldacre. For example, Miles, Loughlin & Polychronis (Miles et al., 2007) say this.

“Loughlin identifies Goldacre [2006] as a particularly luminous example of a commentator who is able not only to combine audacity with outrage, but who in a very real way succeeds in manufacturing a sense of having been personally offended by the article in question. Such moralistic posturing acts as a defence mechanism to protect cherished assumptions from rational scrutiny and indeed to enable adherents to appropriate the ‘moral high ground’, as well as the language of ‘reason’ and ‘science’ as the exclusive property of their own favoured approaches. Loughlin brings out the Orwellian nature of this manoeuvre and identifies a significant implication.”

If Goldacre and others really are engaged in posturing then their primary offence, at least according to the Sartrean perspective adopted by Murray et al. is not primarily intellectual, but rather it is moral. Far from there being a moral requirement to ‘bend a knee’ at the EBM altar, to do so is to violate one’s primary duty as an autonomous being.”

This ferocious attack seems to have been triggered because Goldacre has explained in simple words what constitutes evidence and what doesn’t. He has explained in a simple way how to do a proper randomised controlled trial of homeopathy. And he he dismantled a fraudulent Qlink pendant, purported to shield you from electromagnetic radiation but which turned out to have no functional components (Goldacre, 2007). This is described as being “Orwellian”, a description that seems to me to be downright bizarre.

In fact, when faced with real-life examples of what happens when you ignore evidence, those who write theoretical papers that are critical about evidence-based medicine may behave perfectly sensibly. Although Andrew Miles edits a journal, (Journal of Evaluation in Clinical Practice), that has been critical of EBM for years. Yet when faced with a course in alternative medicine run by people who can only be described as quacks, he rapidly shut down the course (A full account has appeared on this blog).

It is hard to decide whether the language used in these papers is Marxist or neoconservative libertarian. Whatever it is, it clearly isn’t science. It may seem odd that postmodernists (who believe nothing) end up as allies of quacks (who’ll believe anything). The relationship has been explained with customary clarity by Alan Sokal, in his essay Pseudoscience and Postmodernism: Antagonists or Fellow-Travelers? (Sokal, 2006).

Conclusions

Of course RCTs are not the only way to get knowledge. Often they have not been done, and sometimes it is hard to imagine how they could be done (though not nearly as often as some people would like to say).

It is true that RCTs tell you only about an average effect in a large population. But the same is true of observational epidemiology. That limitation is nothing to do with randomisation, it is a result of the crude and inadequate way in which diseases are classified (as discussed above). It is also true that randomisation doesn’t guarantee lack of bias in an individual case, but only in the long run. But it is the best that can be done. The fact remains that randomization is the only way to be sure of causality, and making mistakes about causality can harm patients, as it did in the case of HRT.

Raymond Tallis (1999), in his review of Sokal & Bricmont, summed it up nicely

“Academics intending to continue as postmodern theorists in the interdisciplinary humanities after S & B should first read Intellectual Impostures and ask themselves whether adding to the quantity of confusion and untruth in the world is a good use of the gift of life or an ethical way to earn a living. After S & B, they may feel less comfortable with the glamorous life that can be forged in the wake of the founding charlatans of postmodern Theory. Alternatively, they might follow my friend Roger into estate agency — though they should check out in advance that they are up to the moral rigours of such a profession.”

The conclusions that I have drawn were obvious to people in the business a half a century ago. (Doll & Peto, 1980) said

“If we are to recognize those important yet moderate real advances in therapy which can save thousands of lives, then we need more large randomised trials than at present, not fewer. Until we have them treatment of future patients will continue to be determined by unreliable evidence.”

The towering figures are R.A. Fisher, and his followers who developed the ideas of randomisation and maximum likelihood estimation. In the medical area, Bradford Hill, Archie Cochrane, Iain Chalmers had the important ideas worked out a long time ago.

In contrast, philosophers like Worral, Cartwright, Holmes, Barry, Loughlin and Polychronis seem to me to make no contribution to the accumulation of useful knowledge, and in some cases to hinder it. It’s true that the harm they do is limited, but that is because they talk largely to each other. Very few working scientists are even aware of their existence. Perhaps that is just as well.

References

Cartwright N (2007). Are RCTs the Gold Standard? Biosocieties (2007), 2: 11-20

Colquhoun, D (2010) University of Buckingham does the right thing. The Faculty of Integrated Medicine has been fired. https://www.dcscience.net/?p=2881

Miles A & Loughlin M (2006). Continuing the evidence-based health care debate in 2006. The progress and price of EBM. J Eval Clin Pract 12, 385-398.

Miles A, Loughlin M, & Polychronis A (2007). Medicine and evidence: knowledge and action in clinical practice. J Eval Clin Pract 13, 481-503.

Miles A, Polychronis A, & Grey JE (2006). The evidence-based health care debate – 2006. Where are we now? J Eval Clin Pract 12, 239-247.

Murray SJ, Holmes D, Perron A, & Rail G (2007).
Deconstructing the evidence-based discourse in health sciences: truth, power and fascis. Int J Evid Based Healthc 2006; : 4, 180–186.

Sokal AD (1996). Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity. Social Text 46/47, Science Wars, 217-252.

Sokal AD (2006). Pseudoscience and Postmodernism: Antagonists or Fellow-Travelers? In Archaeological Fantasies, ed. Fagan GG, Routledge,an imprint of Taylor & Francis Books Ltd.

Sokal AD & Bricmont J (1998). Intellectual Impostures, New edition, 2003, Economist Books ed. Profile Books.

Tallis R. (1999) Sokal and Bricmont: Is this the beginning of the end of the dark ages in the humanities?

Worrall J. (2004) Why There’s No Cause to Randomize. Causality: Metaphysics and Methods.Technical Report 24/04 . 2004.

Worrall J (2010). Evidence: philosophy of science meets medicine. J Eval Clin Pract 16, 356-362.

Follow-up

Iain Chalmers has drawn my attention to a some really interesting papers in the James Lind Library

An account of early trials is given by Chalmers I, Dukan E, Podolsky S, Davey Smith G (2011). The adoption of unbiased treatment allocation schedules in clinical trials during the 19th and early 20th centuries. Fisher was not the first person to propose randomised trials, but he is the person who put it on a sound mathematical basis.

Another fascinating paper is Chalmers I (2010). Why the 1948 MRC trial of streptomycin used treatment allocation based on random numbers.

The distinguished statistician, David Cox contributed, Cox DR (2009). Randomization for concealment.

Incidentally, if anyone still thinks there are ethical objections to random allocation, they should read the account of retrolental fibroplasia outbreak in the 1950s, Silverman WA (2003). Personal reflections on lessons learned from randomized trials involving newborn infants, 1951 to 1967.

Chalmers also pointed out that Antony Eagle of Exeter College Oxford had written about Goldacre’s epistemology. He describes himself as a “formal epistemologist”. I fear that his criticisms seem to me to be carping and trivial. Once again, a philosopher has failed to make a contribution to the progress of knowledge.

Tagged acupuncture, CAM, infererence, philosophy, randomisation, randomization, RCT, Ronald Fisher, statistics | 32 Comments

University of Westminster shuts down naturopathy, nutritional therapy, but keeps Acupuncture and Herbal Medicine.

Published August 17, 2011

There’s been no official announcement, but four more of Westminster’s courses in junk medicine have quietly closed.

For entry in 2011 they offer

University of Westminster (W50)	qualification

Chinese Medicine: Acupuncture (B343)	3FT Hon BSc
Chinese Medicine: Acupuncture with Foundation (B341)	4FT/5FT Hon BSc/MSci
Complementary Medicine (B255)	3FT Hon BSc
Complementary Medicine (B301)	4FT Hon MHSci
Complementary Medicine: Naturopathy (B391)	3FT Hon BSc
Herbal Medicine (B342)	3FT Hon BSc
Herbal Medicine with Foundation Year (B340)	4FT/5FT Hon BSc/MSci
Nutritional Therapy (B400)	3FT Hon BSc

But for entry in 2012

University of Westminster (W50)	qualification

Chinese Medicine: Acupuncture (B343)	3FT Hon BSc
Chinese Medicine: Acupuncture with Foundation (B341)	4FT/5FT Hon BSc/MSci
Herbal Medicine (B342)	3FT Hon BSc
Herbal Medicine with Foundation Year (B340)	4FT/5FT Hon BSc/MSc

At the end of 2006, Westminster was offering 14 different BSc degrees in seven flavours of junk medicine. In October 2008, it was eleven. This year it’s eight, and next year only four degrees in two subjects. Since "Integrated Health" was ‘merged’ with Biological Sciences in May 2010, two of the original courses have been dropped each year. This September there will be a final intake for Nutrition Therapy and Naturopathy. That leaves only two, Chinese Medicine (acupuncture and (Western) Herbal Medicine.

The official reason given for the closures is always that the number of applications has fallen. I’m told that the number of applications has halved over the last five or six years. If that’s right, it counts as a big success for the attempts of skeptics to show the public the nonsense that’s taught on these degrees. Perhaps it is a sign that we are emerging from the endarkenment.

Rumour has it that the remaining degrees will eventually close too. Let’s hope so. Meanwhile, here is another helping hand.

There is already quite a bit here about the dangers of Chinese medicine, e.g. here and, especially, here. A submission to the Department of Health gives more detail. There has been a lot on acupuncture here too. There is now little doubt that it’s no more than a theatrical, and not very effective, placebo. So this time I’ll concentrate on Western herbal medicine.

Western Herbal Medicine

Herbal medicine is just a branch of pharmacology and it could be taught as such. But it isn’t. It comes overlaid with much superstitious nonsense. Some of it can be seen in slides from Edinburgh Napier University (the difference being that Napier closed that course, and Westminster hasn’t)

Even if it were taught properly, it wouldn’t be appropriate for a BSc for several reasons.

First, there isn’t a single herbal that has full marketing authorisation from the MHRA. In other words, there isn’t a single herb for which there is good evidence that it works to a useful extent.

Second, the fact that the active principals in plants are virtually always given in an unknown dose makes them potentially dangerous. This isn’t 1950s pharmacology. It’s 1920s pharmacology, dating from a time before methods were worked out for standardising the potency of natural products (see Plants as Medicines).

Third, if you are going to treat illness with chemicals, why restrict yourself to chemicals that occur in plants?

It was the herbal medicine course that gave rise to the most virulent internal complaints at the University of Westminster. These complaints revealed the use of pendulum dowsing by some teachers on the course and the near-illegal, and certainly dangerous, teaching about herbs in cancer.

Here are a few slides from Principles of Herbal Medicine(3CT0 502). The vocabulary seems to be stuck in a time warp. When I first started in the late 1950s, words like tonic, carminative, demulcent and expectorant were common Over the last 40 years all these words have died out in pharmacology, for the simple reason that it became apparent that there were no such actions. But these imaginary categories are still alive and well in the herbal world.

There was a lecture on a categories of drugs so old-fashioned that I’ve never even heard the words: "nervines". and "adaptogens".

The "tonics" listed here seem quite bizarre. In the 1950s, “tonics” containing nux vomica (a small dose of strychnine) and gentian (tastes nasty) were common, but they vanished years ago, because they don’t work. None of those named here even get a mention in NCCAM’s Herbs-at-a-glance. Oats? Come on!

The only ‘relaxant’ here for which there is the slightest evidence is Valerian. I recall tincture of Valerian in a late 1950s pharmacy. It smells terrible,

According to NCCAM

Research suggests that valerian may be helpful for insomnia, but there is not enough evidence from well-designed studies to confirm this.
There is not enough scientific evidence to determine whether valerian works for other conditions, such as anxiety or depression.

Not much, for something that’s been around for centuries.

And for chamomile

Chamomile has not been well studied in people so there is little evidence to support its use for any condition.

None of this near-total lack of evidence is mentioned on the slides.

What about the ‘stimulants‘? Rosemary? No evidence at all. Tea and coffee aren’t medicine (and not very good stimulants for me either).

Ginseng, on the other hand, is big business. That doesn’t mean it works of course. NCCAM says of Asian ginseng (Panax Ginseng).

Some studies have shown that Asian ginseng may lower blood glucose. Other studies indicate possible beneficial effects on immune function.
Although Asian ginseng has been widely studied for a variety of uses, research results to date do not conclusively support health claims associated with the herb. Only a few large, high-quality clinical trials have been conducted. Most evidence is preliminary—i.e., based on laboratory research or small clinical trials.

Thymoleptics – antidepressants are defined as "herbs that engender a feeling of wellbeing. They uplift the spirit, improve the mood and counteract depression".

Oats, Lemon balm, Damiana, Vervain. Lavender and Rosemary are just old bits of folklore

NCCAM says

Some “sleep formula” products combine valerian with other herbs such as hops, lavender, lemon balm, and skullcap. Although many of these other herbs have sedative properties, there is no reliable evidence that they improve insomnia.

The only serious contender here is St John’s Wort. At one time this was the prize exhibit for herbalists. It has been shown to be as good as the conventional SSRIs for treatment of mild to moderate depression. Sadly it has turned out that the SSRIs are themselves barely better than placebos. NCCAM says

There is scientific evidence that St. John’s wort may be useful for short-term treatment of mild to moderate depression. Although some studies have reported benefits for more severe depression, others have not; for example, a large study sponsored by NCCAM found that the herb was no more effective than placebo in treating major depression of moderate severity.

"Adaptogens" are another figment of the herbalists’ imaginations. They are defined in the lecture thus.

Herbs that have a normalising or balancing effect.
Mind and body are restored to optimum normal peak,
Increase threshold to physical and mental trauma and damage
Mental and physical activity and performance improved.

Well, it would be quite nice if such drugs existed. Sadly they don’t.

NCCAM says

The evidence for using astragalus for any health condition is limited. High-quality clinical trials (studies in people) are generally lacking.

Another lecture dealt with "stimulating herbs". No shortage of them, it seems.

Well at least one of these has quite well-understood effects in pharmacology, ephedrine, a sympathomimetic amine. It isn’t used much because it can be quite dangerous, even with the controlled dose that’s used in real medicine. In the uncontrolled dose in herbal medicines it is downright dangerous.

This is what NCCAM says about Ephedra

An NCCAM-funded study that analyzed phone calls to poison control centers found a higher rate of side effects from ephedra, compared with other herbal products.
Other studies and systematic reviews have found an increased risk of heart, psychiatric, and gastrointestinal problems, as well as high blood pressure and stroke, with ephedra use.
According to the U.S. Food and Drug Administration (FDA), there is little evidence of ephedra’s effectiveness, except for short-term weight loss. However, the increased risk of heart problems and stroke outweighs any benefits.

It seems that what is taught in the BSc Herbal Medicine degree consists largely of folk-lore and old wives’ tales. Some of it could be quite dangerous for patients.

A problem for pharmacognosists

While talking about herbal medicine, it’s appropriate to mention a related problem, though it has nothing to do with the University of Westminster.

My guess is that not many people have even heard of pharmacognosy. If it were not for my humble origins as an apprentice pharmacist in Grange Road, Birkenhead (you can’t get much more humble than that) I might not know either.

Pharmacognosy is a branch of botany, the study of plant drugs. I recall inspecting powered digitalis leaves under a microscope. In Edinburgh, in the time of the great pharmacologist John Henry Gaddum, medical students might be presented in the oral exam with a jar of calabar beans and required to talk about their anticholinesterase effects of the physostigmine that they contain.

The need for pharmacognosy has now all but vanished, but it hangs on in the curriculum for pharmacy students. This has engendered a certain unease about the role of pharmacognists. They often try to justify their existence by rebranding themselves as "phytotherapists". There are even journals of phytotherapy. It sounds a lot more respectable that herbalism. At its best, it is more respectable, but the fact remains that there no herbs whatsoever that have well-documented medical uses.

The London School of Pharmacy is a case in point. Simon Gibbons (Professor of Phytochemistry, Department of Pharmaceutical and Biological Chemistry). The School of Pharmacy) has chosen, for reasons that baffle me, to throw in his lot with the reincarnated Prince of Wales Foundation known as the “College of Medicine“. That organisation exists largely (not entirely) to promote various forms of quackery under the euphemism “integrated medicine”. On their web site he says "Western science is now recognising the extremely high value of herbal medicinal products . . .", despite the fact that there isn’t a single herbal preparation with efficacy sufficient for it to get marketing authorisation in the UK. This is grasping at straws, not science.

The true nature of the "College of Medicine" is illustrated, yet again, by their "innovations network". Their idea of "innovation" includes the Bristol Homeopathic Hospital and the Royal London Hospital for Integrated medicine, both devoted to promoting the utterly discredited late-18th century practice of giving people pills that contain no medicine. Some "innovation".

It baffles me that Simon Gibbons is willing to appear on the same programme as Simon Mills and David Peters, and George Lewith. Mills’ ideas can be judged by watching a video of a talk he gave in which he ‘explains’ “hot and cold herbs”. It strikes me as pure gobbledygook. Make up your own mind. He too has rebranded himself as "phytotherapist" though in fact he’s an old-fashioned herbalist with no concern for good evidence. David Peters is the chap who, as Clinical Director of the University of Westminster’s ever-shrinking School of Quackery, tolerates dowsing as a way to select ‘remedies’.

The present chair of Pharmacognosy at the School of Pharmacy is Michael Heinrich. He, with Simon Gibbons, has written a book Fundamentals of pharmacognosy and phytotherapy. As well as much good chemistry, it contains this extraordinary statement

“TCM [traditional Chinese medicine] still contains very many remedies which were selected by their symbolic significance rather than their proven effects; however this does not mean that they are all ‘quack’remedies! There may even be some value in medicines such as tiger bone, bear gall, turtle shell, dried centipedes, bat dung and so on. The herbs, however, are well researched and are becoming increasingly popular as people become disillusioned with Western Medicine.”

It is irresponsible to give any solace at all to the wicked industries that kill tigers and torture bears to extract their bile. And it is simple untrue that “herbs are well-researched”. Try the test,

A simple test for herbalists. Next time you encounter a herbalist, ask them to name the herb for which there is the best evidence of benefit when given for any condition. Mostly they refuse to answer, as was the case with Michael McIntyre (but he is really an industry spokesman with few scientific pretensions). I asked Michael Heinrich, Professor of Pharmacognosy at the School of Pharmacy. Again I couldn’t get a straight answer. Usually, when pressed, the two things that come up are St John’s Wort and Echinacea. Let’s see what The National Center for Complementary and Alternative Medicine (NCCAM) has to say about them. NCCAM is the branch of the US National Institutes of Health which has spent around a billion dollars of US taxpayers’ money on research into alternative medicine, For all that effort they have failed to come up with a single useful treatment. Clearly they should be shut down. Nevertheless, as an organisation that is enthusiastic about alternative medicine, their view can only be overoptimistic.

For St John’s Wort . NCCAM says

There is scientific evidence that St. John’s wort may be useful for short-term treatment of mild to moderate depression. Although some studies have reported benefits for more severe depression, others have not; for example, a large study sponsored by NCCAM found that the herb was no more effective than placebo in treating major depression of moderate severity.

For Echinacea NCCAM says

Study results are mixed on whether echinacea can prevent or effectively treat upper respiratory tract infections such as the common cold. For example, two NCCAM-funded studies did not find a benefit from echinacea, either as Echinacea purpurea fresh-pressed juice for treating colds in children, or as an unrefined mixture of Echinacea angustifolia root and Echinacea purpurea root and herb in adults. However, other studies have shown that echinacea may be beneficial in treating upper respiratory infections.

If these are the best ones, heaven help the rest.

Follow-up

Tagged Academia, acupuncture, alternative medicine, CAM, Central Lancashire, College of medicine, David Peters, Geoffrey Petts, George Lewith, herbal medicine, herbalism, Jane Lewis, Michael Heinrich, quackademia, Simon Gibbons, Simon Mills, University of Westminster, Westminster university | 14 Comments

Professor Geoffrey Petts of the University of Westminster says they “are not teaching pseudo-science”. The facts show this is not true

Published August 11, 2011

Jump to follow-up

On 23rd May 2008 a letter was sent to the vice-chancellor of the University of Westminster, Professor Geoffrey Petts

Dear Professor Petts

You may be aware an article by Zoe Corbyn, published in Times Higher Education 24 April 2008, with the title Experts criticise ‘pseudo-scientific’ complementary medicine degrees. The subtitle of the article was Vice-chancellors should re-examine courses, say campaigners. In the light of that, we wondered whether you had anything to add to the comments made by David Peters in todays THE. We are preparing a response to that, and it seems fair to ask your view before we proceed.
(In order to save you time, copies of the two articles are attached.)

As an expert on oceans and geomorphology we would have imagined that you would view with some scepticism statements like amethysts emit high yin energy, as taught in your first year course, common to most complementary medicine students, as part if an honours BSc. Unfortunately Peters does not deal with this in his response, so your opinion would be welcome.

Best regards

Edzard Ernst MD, PhD, FRCP, FRCP (Edin.)
Professor of Complementary Medicine, Univerisity of Exeter

Simon Singh Ph.D,
Science writer, Author of Fermat’s Last Theorem, The Code Book, etc

David Colquhoun FRS Professor of Pharmacology UCL

Despite reminders, we were never afforded the courtesy of a reply to this, or any other letter.

Professor Petts has, however, replied to a letters sent to him recently by the Nightingale Collaboration. He said

“Whilst I understand your concerns, colleagues of the School of Life Sciences where these courses are offered do not share them. They are not teaching pseudo-science, as you claim,…”.

Neither of thse claims is true.I know at least two members of the Life Sciences Faculty who are very worried. One has now left and one has retired. The rest are presumably too scared to speak out.

It is most certainly not true to say they "are not teaching pseudo-science". Both the vice-chancellor and the Dean of Life Sciences, Jane Lewis, have been made aware of what;s happening repeatedly over several years, I can think of no other way to put it but to say Professor Petts is lying. Tha is not a good thing for vice-chancellors to do.

Much of what is taught at Westminster has now been revealed. This seems like a good moment to summarise what we know. Searching this blog for "University of Westminster" yields 39 hits. Of these, 11 show what’s taught at Westminster, so I’ll summarise them here for easy reference.

Westminster’s response

March 26th, 2007

The day after “Science degrees without the Science“ appeared in Nature, the University of Westminster issued a statement (now vanished, but see debate in THE). In my view, their statement provides the strongest grounds so far to believe that the BSc is inappropriate.

Let’s take a look at it.

“The BSc (Hons) Health Sciences: Homeopathy is a fully validated degree that satisfies internal and external quality assurance standards.”

Since that time, the university has closed down the BSc in Homeopathy. I have now seen many such validation documents and they are mostly box ticking exercises, not worth the paper they are written on. The worst offender is the University or Wales.

Westminster University BSc: “amethysts emit high yin energy”

April 23rd, 2008

http://dcscience.net/vib-therapies-43s.jpg

This shows the first set of slides that I got from Westminster (leaked by an angry insider). It has slides on crystal healing and dowsing, things that are at the lunatic fringe even by the standards of alternative medicine. it also relates that an academic who invited me to give a talk at the University of Westminster on the evidence for alternative medal was leaned on heavily by “VC, Provosts and Deans” to prevent that talk taking place.

Crystal balls. Professor Petts in Private Eye

February 19th, 2009

Professor Petts makes an appearance in Private Eye. It could held that this counts as bringing your university into disrepute.

The opposite of Science.

February 24th, 2009

BSc courses in homeopathy are closing. Is it a victory for campaigners, or just the end of the Blair/Bush era?

Professor Petts of Westminster seems to think that the problem can be solved by putting more science into the courses The rest of the world realises that as soon as you apply science to homeopathy or naturopathy, the whole subject vanishes in a puff of smoke, I fear that Professor Petts will have to do better,

The Guardian carries a nice article by Anthea Lipsett, The Opposite of Science (or download pdf of print version).

The last BSc (Hons) Homeopathy closes! But look at what they still teach at Westminster University.

March 30th, 2009

In March 2007 I wrote a piece in Nature on Science degrees without the science. At that time there were five “BSc” degrees in homeopathy. A couple of weeks ago I checked the UCAS site for start in 2009, and found there was only one full “BSc (hons)” left and that was at Westminster University.

Today I checked again and NOW THERE ARE NONE.

A phone call to the University of Westminster tonight confirmed that they have suspended entry to their BSc (Hons) homeopathy degree.

Then I revealed another set of slides, showing a misunderstanding (by the teacher) of statistics, but most chillingly,some very dangerous ideas conveyed to students by Westminster’s naturopaths, and in the teaching of Traditional Chinese Medicine, and the great “detox” scam.

“if you get tuberculosis, it isn’t caused by Mycobacterium tuberculosis? And the symptoms are “constructive”? So you don’t need to do anything. It’s all for the best really.

This isn’t just nonsense. It’s dangerous nonsense.”

“Remember when shopping to favour fruits and vegetables which are in season and locally grown (and ideally organic) as they are more vibrationally compatible with the body.”

Locally grown vegetables are “more vibrationally compatible with the body”? Pure mystical gobbledygook. Words fail me.

More make-believe from the University of Westminster. This time it’s Naturopathy

June 25th, 2009

Some truly mind-boggling stuff that’s taught to students at Westminster, It includes "Emotrance". A primer on Emotrance says

"And then I thought of the lady in the supermarket whose husband had died, and I spend the following time sending her my best wishes, and my best space time quantum healing efforts for her void."

Then there are slides on pendulum diagnosis and “kinesiology”, a well-known fraudulent method of diagnosis. It is all perfectly mad.

Why degrees in Chinese medicine are a danger to patients

August 10th, 2009

More lunatic fantasies from Westminster, this time about Chinese medicine.

“Teaching students that the brain is made of marrow is not just absurd, but desperately dangerous for anyone unlucky (or stupid) enough to go to such a person when they are ill.”

There is a lot of stuff about cancer that is potentially homicidal.

Legally, you cannot claim to cure cancer

This is not a problem because:
‘we treat people not diseases‘

This is outrageous and very possibly illegal under the Cancer Act (1939). It certainly poses a huge danger to patients. It is a direct incentive to make illegal, and untrue claims by using weasel words in an attempt to stay just on the right side of the law. But that, of course, is standard practice in alternative medicine,

Emergent Chinese Omics at the University of Westminster/p>

August 27th, 2010

Systems biology is all the rage, No surprise then, to see the University of Westminster advertising a job for a systems biologist in the The Department of Molecular and Applied Biosciences. Well, no surprise there -until you read the small print.

Much has been written here about the University of Westminster, which remains the biggest provider of junk science degrees in the UK, despite having closed two of them.

If there is one thing more offensive than the use of meaningless mystical language, it is the attempt to hijack the vocabulary of real science to promote nonsense. As soon as a quack uses the words "quantum", "energy", "vibration", or now, "systems biology", you can be sure that it’s pretentious nonsense.

Hot of the press. Within a few hours of posting this, I was told that Volker Scheid, the man behind the pretentious Chinese medidine omics nonsense has been promoted to a full chair, And the Dean, Jane Lewis has congratulated him for speaking at a Chines Medicine symposium. Even quite sensible people like Lewis are being corrupted. The buck stops with Petts.

More dangerous nonsense from the University of Westminster: when will Professor Geoffrey Petts do something about it?

May 3rd, 2011

Yet more ghastly slides that are inflicted on Westminster students. How’s this for sheer barminess, taught as part of a Bachelor of Science degree?

enviro-4

And

" Just in case you happen to have run out of Alaskan Calling All Angels Essence, you can buy it from Baldwin’s for £19.95. It’s “designed to invoke the nurturing, uplifting and joyful qualities of the angelic kingdom.”, and what’s more “can also use them any time to cleanse, energize, and protect your auric field.” Well that’s what it says.in the ad.

Freedom of information reveals some unusual testimonials for the University of Westminster: when will Professor Geoffrey Petts do something about it?

June 20th, 2011

This post gives details of two complaints, one from a student and one from a lecturer. The vice-chancellor certainly knows about them. So why, I wonder, did he say "“Whilst I understand your concerns, colleagues of the School of Life Sciences where these courses are offered do not share them.". He must know that this simply isn’t true. It is over a year now ( 10 July 2009 ) that a lecturer wrote to the vice-chancellor and Dean

“I expect that were the Department of Health to be aware of the unscientific teaching and promotion of practices like dowsing, (and crystals, iridology, astrology, and tasting to determine pharmacological qualities of plant extracts) on the Wmin HM [Westminster Herbal Medicine] Course, progress towards the Statutory Regulation of Herbal Medicine could be threatened.”

There’s only one thing wrong with this. The lecturer underestimated the stupidiity of the Department of Health which went ahead with statutory regulation despite being made aware of what was going on.

The latest example to come to light is cited by Andy Lewis on his Quackometer blog

“There are some even odder characters too, such as Roy Riggs B.Sc who describes himself as a “Holistic Geobiologist” and is “an “professional Earth Energy dowser”. He guest lectures at the London Westminster University’s School of Integrative Medicine and The Baltic Dowser’s Association of Lithuania.”

I do wonder who Professor Petts thinks he’s fooling. His denial of the obvious fact that his university is teaching pseudo-science serves only to discredit further the University of Westminster and his own integrity.

Follow-up

13 August 2011 I’m intrigued to notice that two days after posting this summary, googling “Geoffrey Petts” brings up this post as #3 on the first page. Actions have consequences.

Tagged Academia, alternative medicine, Anti-science, antiscience, CAM, Department of Health, Geoffrey Petts, quackademia, Universities, University of Westminster, vice-chancellors, Westminster university | 5 Comments

Academia

Scientists don’t count: why you should ignore altmetrics and other bibliometric nightmares

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

Acupuncture is a theatrical placebo

Follow-up

Share this:

Like this:

Pharmacology society does little to defend its subject

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

The vocabulary of bait and switch

Why regulation isn’t working, and can’t work

The Nightingale Collaboration.

Conclusions

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

Australian journal classification

The responses to Simon Gaskell

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

What’s left?

Why have all these courses closed?

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this:

Western Herbal Medicine

A problem for pharmacognosists

Follow-up

Share this:

Like this:

Follow-up

Share this:

Like this: