Download Lectures on Biostatistics (1971). Corrected and searchable version of Google books edition
Download review of Lectures on Biostatistics (THES, 1973).
Human resources
There is a widespread belief that science is going through a crisis of reproducibility. A meeting was held to discuss the problem. It was organised by Academy of Medical Sciences, the Wellcome Trust, MRC and BBSRC, and It was chaired by Dorothy Bishop (of whose blog I’m a huge fan). It’s good to see that scientific establishment is beginning to take notice. Up to now it’s been bloggers who’ve been making the running. I hadn’t intended to write a whole post about it, but some sufficiently interesting points arose that I’ll have a go.
The first point to make is that, as far as I know, the “crisis” is limited to, or at least concentrated in, quite restricted areas of science. In particular, it doesn’t apply to the harder end of sciences. Nobody in physics, maths or chemistry talks about a crisis of reproducibility. I’ve heard very little about irreproducibility in electrophysiology (unless you include EEG work). I’ve spent most of my life working on single-molecule biophysics and I’ve never encountered serious problems with irreproducibility. It’s a small and specialist field so I think if I would have noticed if it were there. I’ve always posted on the web our analysis programs, and if anyone wants to spend a year re-analysing it they are very welcome to do so (though I have been asked only once).
The areas that seem to have suffered most from irreproducibility are experimental psychology, some areas of cell biology, imaging studies (fMRI) and genome studies. Clinical medicine and epidemiology have been bad too. Imaging and genome studies seem to be in a slightly different category from the others. They are largely statistical problems that arise from the huge number of comparisons that need to be done. Epidemiology problems stem largely from a casual approach to causality. The rest have no such excuses.
The meeting was biased towards psychology, perhaps because that’s an area that has had many problems. The solutions that were suggested were also biased towards that area. It’s hard to see some of them could be applied to electrophysiology for example.
There was, it has to be said, a lot more good intentions than hard suggestions. Pre-registration of experiments might help a bit in a few areas. I’m all for open access and open data, but doubt they will solve the problem either, though I hope they’ll become the norm (they always have been for me).
All the tweets from the meeting hve been collected as a Storify. The most retweeted comment was from Liz Wager
@SideviewLiz: Researchers are incentivised to publish, get grants, get promoted but NOT incentivised to be right! #reprosymp
This, I think, cuts to the heart if the problem. Perverse incentives, if sufficiently harsh, will inevitably lead to bad behaviour. Occasionally it will lead to fraud. It’s even led to (at least) two suicides. If you threaten people in their forties and fifties with being fired, and losing their house, because they don’t meet some silly metric, then of course people will cut corners. Curing that is very much more important than pre-registration, data-sharing and concordats, though the latter occupied far more of the time at the meeting.
The primary source of the problem is that there is not enough money for the number of people who want to do research (a matter that was barely mentioned). That leads to the unpalatable conclusion that the only way to cure the problem is to have fewer people competing for the money. That’s part of the reason that I suggested recently a two-stage university system. That’s unlikely to happen soon. So what else can be done in the meantime?
The responsibility for perverse incentives has to rest squarely on the shoulders of the senior academics and administrators who impose them. It is at this level that the solutions must be found. That was said, but not firmly enough. The problems are mostly created by the older generation It’s our fault.
IncidentalIy, I was not impressed by the fact that the Academy of Medical Sciences listed attendees with initials after peoples’ names. There were eight FRSs but I find it a bit embarrassing to be identified as one, as though it made any difference to the value of what I said.
It was suggested that courses in research ethics for young scientists would help. I disagree. In my experience, young scientists are honest and idealistic. The problems arise when their idealism is shattered by the bad example set by their elders. I’ve had a stream of young people in my office who want advice and support because they feel they are being pressured by their elders into behaviour which worries them. More than one of them have burst into tears because they feel that they have been bullied by PIs.
One talk that I found impressive was Ottloline Leyser who chaired the recent report on The Culture of Scientific Research in the UK, from the Nuffield Council on Bioethics. But I found that report to be bland and its recommendations, though well-meaning, unlikely to result in much change. The report was based on a relatively small, self-selected sample of 970 responses to a web survey, and on 15 discussion events. Relatively few people seem to have spent time filling in the text boxes, For example
“Of the survey respondents who provided a negative comment on the effects of competition in science, 24 out of 179 respondents (13 per cent) believe that high levels of competition between individuals discourage research collaboration and the sharing of data and methodologies.&rdquo:
Such numbers are too small to reach many conclusions, especially since the respondents were self-selected rather than selected at random (poor experimental design!). Nevertheless, the main concerns were all voiced. I was struck by
“Almost twice as many female survey respondents as male respondents raise issues related to career progression and the short term culture within UK research when asked which features of the research environment are having the most negative effect on scientists”
But no conclusions or remedies were put forward to remedy this problem. It was all put rather better, and much more frankly, some time ago by Peter Lawrence. I do have the impression that bloggers (including Dorothy Bishop) get to the heart of the problems much more directly than any official reports.
The Nuffield report seemed to me to put excessive trust in paper exercises, such as the “Concordat to Support the Career Development of Researchers”. The word “bullying” does not occur anywhere in the Nuffield document, despite the fact that it’s problem that’s been very widely discussed and a problem that’s critical for the problems of reproducibility. The Concordat (unlike the Nuffield report) does mention bullying.
"All managers of research should ensure that measures exist at every institution through which discrimination, bullying or harassment can be reported and addressed without adversely affecting the careers of innocent parties. "
That sounds good, but it’s very obvious that there are many places simply ignore it. All universities subscribe to the Concordat. But signing is as far as it goes in too many places. It was signed by Imperial College London, the institution with perhaps the worst record for pressurising its employees, but official reports would not dream of naming names or looking at publicly available documentation concerning bullying tactics. For that, you need bloggers.
On the first day, the (soon-to-depart) Dean of Medicine at Imperial, Dermot Kelleher, was there. He seemed a genial man, but he would say nothing about the death of Stefan Grimm. I find that attitude incomprehensible. He didn’t reappear on the second day of the meeting.
The San Francisco Declaration on Research Assessment (DORA) is a stronger statement than the Concordat, but its aims are more limited. DORA states that the impact factor is not to be used as a substitute “measure of the quality of individual research articles, or in hiring, promotion, or funding decisions”. That’s something that I wrote about in 2003, in Nature. In 2007 it was still rampant, including at Imperial College. It still is in many places. The Nuffield Council report says that DORA has been signed by “over 12,000 individuals and 500 organisations”, but fails to mention the fact that only three UK universities have signed up to DORA (oneof them, I’m happy to say, is UCL). That’s a pretty miserable record. And, of course, it remains to be seen whether the signatories really abide by the agreement. Most such worthy agreements are ignored on the shop floor.
The recommendations of the Nuffield Council report are all worthy, but they are bland and we’ll be lucky if they have much effect. For example
“Ensure that the track record of researchers is assessed broadly, without undue reliance on journal impact factors”
What on earth is “undue reliance”? That’s a far weaker statement than DORA. Why?
And
“Ensure researchers, particularly early career researchers, have a thorough grounding in research ethics”
In my opinion, what we should say to early career researchers is “avoid the bad example that’s set by your elders (but not always betters)”. It’s the older generation which has produced the problems and it’s unbecoming to put the blame on the young. It’s the late career researchers who are far more in need of a thorough grounding in research ethics than early-career researchers.
Although every talk was more or less interesting, the one I enjoyed most was the first one, by Marcus Munafo. It assessed the scale of the problem (though with a strong emphasis on psychology, plus some genetics and epidemiology), and he had good data on under-powered studies. It also made a fleeting mention of the problem of the false discovery rate. Since the meeting was essentially about the publication of results that aren’t true, I would have expected the statistical problem of the false discovery rate to have been given much more prominence than it was. Although Ioannidis’ now-famous paper “Why most published research is wrong” got the occasional mention, very little attention (apart from Munafo and Button) was given to the problems which he pointed out.
I’ve recently convinced myself that, if you declare that you’ve made a discovery when you observe P = 0.047 (as is almost universal in the biomedical literature) you’ll be wrong 30 – 70% of the time (see full paper, "An investigation of the false discovery rate and the misinterpretation of p-values".and simplified versions on Youtube and on this blog). If that’s right, then surely an important way to reduce the publication of false results is for journal editors to give better advice about statistics. This is a topic that was almost absent from the meeting. It’s also absent from the Nuffield Council report (the word “statistics” does not occur anywhere).
In summary, the meeting was very timely, and it was fun. But I ended up thinking it had a bit too much of preaching good intentions to the converted. It failed to grasp some of the nettles firmly enough. There was no mention of what’s happening at Imperial, or Warwick, or Queen Mary, or at Kings College London. Let’s hope that when it’s written up, the conclusion will be a bit less bland than those of most official reports.
It’s overdue that we set our house in order, because the public has noticed what’s going on. The New York Times was scathing in 2006. This week’s Economist said
"Modern scientists are doing too much trusting and not enough verifying -to the detriment of the whole of science, and of humanity.
Too many of the findings that fill the academic ether are the result of shoddy experiments or poor analysis""Careerism also encourages exaggeration and the cherrypicking of results."
This is what the public think of us. It’s time that vice-chancellors did something about it, rather than willy-waving about rankings.
Conclusions
After criticism of the conclusions of official reports, I guess that I have to make an attempt at recommendations myself. Here’s a first attempt.
- The heart of the problem is money. Since the total amount of money is not likely to increase in the short term, the only solution is to decrease the number of applicants. This is a real political hot-potato, but unless it’s tackled the problem will persist. The most gentle way that I can think of doing this is to restrict research to a subset of universities. My proposal for a two stage university system might go some way to achieving this. It would result in better postgraduate education, and it would be more egalitarian for students. But of course universities that became “teaching only” would see (wrongly) as demotion, and it seems that UUK is unlikely to support any change to the status quo (except, of course, for increasing fees).
- Smaller grants, smaller groups and fewer papers would benefit science.
- Ban completely the use of impact factors and discourage use of all metrics. None has been shown to measure future quality. All increase the temptation to “game the system” (that’s the usual academic euphemism for what’s called cheating if an undergraduate does it).
- “Performance management” is the method of choice for bullying academics. Don’t allow people to be fired because they don’t achieve arbitrary targets for publications or grant income. The criteria used at Queen Mary London, and Imperial, and Warwick and at Kings, are public knowledge. They are a recipe for employing spivs and firing Nobel Prize winners: the 1991 Nobel Laureate in Physiology or Medicine would have failed Imperial’s criteria in 6 years out of 10 years when he was doing the work which led to the prize.
- Universities must learn that if you want innovation and creativity you have also to tolerate a lot of failure.
- The ranking of universities by ranking businesses or by the REF encourages bad behaviour by encouraging vice-chancellors to improve their ranking, by whatever means they can. This is one reason for bullying behaviour. The rankings are totally arbitrary and a huge waste of money. I’m not saying that universities should be unaccountable to taxpayers. But all you have to do is to produce a list of publications to show that very few academics are not trying. It’s absurd to try to summarise a whole university in a single number. It’s simply statistical illiteracy
- Don’t waste money on training courses in research ethics. Everyone already knows what’s honest and what’s dodgy (though a bit more statistics training might help with that). Most people want to do the honest thing, but few have the nerve to stick to their principles if the alternative is to lose your job and your home. Senior university people must stop behaving in that way.
- University procedures for protecting the young are totally inadequate. A young student who reports bad behaviour of his seniors is still more likely to end up being fired than being congratulated (see, for example, a particularly bad case at the University of Sheffield). All big organisations close ranks to defend themselves when criticised. Even extreme cases, as when an employee commits suicide after being bullied, universities issue internal reports which blame nobody.
- Universities must stop papering over the cracks when misbehaviour is discovered. It seems to be beyond the wit of PR people to realise that often it’s best (and always the cheapest) to put your hands up and say “sorry, we got that wrong”
- There an urgent need to get rid of the sort of statistical illiteracy that allows P = 0.06 to be treated as failure and P = 0.04 as success. This is almost universal in biomedical papers, and given the hazards posed by the false discovery rate, could well be a major contribution to false claims. Journal editors need to offer much better statistical advice than is the case at the moment.
Follow-up
|
This week’s Times Higher Education carried a report of the death, at age 51, of Professor Stefan Grimm: Imperial College London to ‘review procedures’ after death of academic. He was professor of toxicology in the Faculty of Medicine at Imperial. |
 |
Now Stefan Grimm is dead. Despite having a good publication record, he failed to do sufficiently expensive research, so he was fired (or at least threatened with being fired).
“Speaking to Times Higher Education on condition of anonymity, two academics who knew Professor Grimm, who was 51, said that he had complained of being placed under undue pressure by the university in the months leading up to his death, and that he had been placed on performance review.”
Having had cause to report before on bullying at Imperial’s Department of Medicine, I was curious to know more.
Martin Wilkins wrote to Grimm on 10 March 2014. The full text is on THE.
"I am of the opinion that you are struggling to fulfil the metrics of a Professorial post at Imperial College which include maintaining established funding in a programme of research with an attributable share of research spend of £200k p.a and must now start to give serious consideration as to whether you are performing at the expected level of a Professor at Imperial College."
"Please be aware that this constitutes the start of informal action in relation to your performance, however should you fail to meet the objective outlined, I will need to consider your performance in accordance with the formal College procedure for managing issues of poor performance (Ordinance D8) which can be found at the following link.
http://www3.imperial.ac.uk/secretariat/collegegovernance/provisions/ordinances/d8"
[The link to ordinances in this letter doesn’t work now. But you can still read them here (click on the + sign).]
It didn’t take long to get hold of an email from Grimm that has been widely circulated within Imperial. The mail is dated a month after his death. It isn’t known whether it was pre-set by Grimm himself or whether it was sent by someone else. It’s even possible that it wasn’t written by Grimm himself, though if it is an accurate description of what happened, that’s not crucial.
No doubt any Imperial staff member would be in great danger if they were to publish the mail. So, as a public service, I shall do so.
The email from Stefan Grimm, below, was prefaced by an explanation written by the person who forwarded it (I don’t know who that was).
|
Dear Colleagues,
You may have already heard about the tragic death of Professor Stefan Grimm a former member of the Faculty of Medicine at Imperial College. He died suddenly and unexpectedly in early October. As yet there is no report about the cause of his death. Some two weeks later a delayed email from him was received by many of the senior staff of the medical school, and other researchers worldwide. It has been forwarded to me by one of my research collaborators. From my reading of it I believe that Stefan wanted it circulated as widely as possible and for that reason I am sending it to you. It is appended below. This email represents just one side of an acrimonious dispute, but it may be indicative of more deep seated problems. best wishes Begin forwarded message: From: Stefan Grimm <professorstefangrimm@gmail.com> Date: 21 October 2014 23:41:03 BST To: <big-email-list> Subject: How Professors are treated at Imperial College Dear all, If anyone is interested how Professors are treated at Imperial College: Here is my story. On May 30th ’13 my boss, Prof Martin Wilkins, came into my office together with his PA and ask me what grants I had. After I enumerated them I was told that this was not enough and that I had to leave the College within one year – “max” as he said. He made it clear that he was acting on behalf of Prof Gavin Screaton, the then head of the Department of Medicine, and told me that I would have a meeting with him soon to be sacked. Without any further comment he left my office. It was only then that I realized that he did not even have the courtesy to close the door of my office when he delivered this message. When I turned around the corner I saw a student who seems to have overheard the conversation looking at me in utter horror. Prof Wilkins had nothing better to do than immediately inform my colleagues in the Section that he had just sacked me. Why does a Professor have to be treated like that? All my grant writing stopped afterwards, as I was waiting for the meeting to get sacked by Prof Screaton. This meeting, however, never took place. In March ’14 I then received the ultimatum email below. 200,000 pounds research income every year is required. Very interesting. I was never informed about this before and cannot remember that this is part of my contract with the College. Especially interesting is the fact that the required 200,000.- pounds could potentially also be covered by smaller grants but in my case a programme grant was expected. Our 135,000.- pounds from the University of Dammam? Doesn’t count. I have to say that it was a lovely situation to submit grant applications for your own survival with such a deadline. We all know what a lottery grant applications are. There was talk that the Department had accepted to be in dept for some time and would compensate this through more teaching. So I thought that I would survive. But the email below indicates otherwise. I got this after the student for whom I “have plans” received the official admission to the College as a PhD student. He waited so long to work in our group and I will never be able to tell him that this should now not happen. What these guys don’t know is that they destroy lives. Well, they certainly destroyed mine. The reality is that these career scientists up in the hierarchy of this organization only look at figures to judge their colleagues, be it impact factors or grant income. After all, how can you convince your Department head that you are working on something exciting if he not even attends the regular Departmental seminars? The aim is only to keep up the finances of their Departments for their own career advancement. These formidable leaders are playing an interesting game: They hire scientists from other countries to submit the work that they did abroad under completely different conditions for the Research Assessment that is supposed to gauge the performance of British universities. Afterwards they leave them alone to either perform with grants or being kicked out. Even if your work is submitted to this Research Assessment and brings in money for the university, you are targeted if your grant income is deemed insufficient. Those submitted to the research assessment hence support those colleagues who are unproductive but have grants. Grant income is all that counts here, not scientific output. We had four papers with original data this year so far, in Cell Death and Differentiation, Oncogene, Journal of Cell Science and, as I informed Prof Wilkins this week, one accepted with the EMBO Journal. I was also the editor of a book and wrote two reviews. Doesn’t count. This leads to a interesting spin to the old saying “publish or perish”. Here it is “publish and perish”. Did I regret coming to this place? I enormously enjoyed interacting with my science colleagues here, but like many of them, I fell into the trap of confusing the reputation of science here with the present reality. This is not a university anymore but a business with very few up in the hierarchy, like our formidable duo, profiteering and the rest of us are milked for money, be it professors for their grant income or students who pay 100.- pounds just to extend their write-up status. If anyone believes that I feel what my excellent coworkers and I have accomplished here over the years is inferior to other work, is wrong. With our apoptosis genes and the concept of Anticancer Genes we have developed something that is probably much more exciting than most other projects, including those that are heavily supported by grants. Was I perhaps too lazy? My boss smugly told me that I was actually the one professor on the whole campus who had submitted the highest number of grant applications. Well, they were probably simply not good enough. I am by far not the only one who is targeted by those formidable guys. These colleagues only keep quiet out of shame about their situation. Which is wrong. As we all know hitting the sweet spot in bioscience is simply a matter of luck, both for grant applications and publications. Why does a Professor have to be treated like that? One of my colleagues here at the College whom I told my story looked at me, there was a silence, and then said: “Yes, they treat us like sh*t”. Best regards, Stefan Grimm |
There is now a way for staff to register their opinions of their employers.The entries for Imperial College on Glassdoor.com suggest that bullying there is widespread (on contrast, the grumbles about UCL are mostly about lack of space).
Googling ‘imperial college employment tribunal’ shows a history of bullying that is not publicised. In fact victims are often forced to sign gagging clauses. In fairness, AcademicFOI.com shows that the problems are not unique to Imperial. Over 3 years (it isn’t clear which years) , 810 university staff went to employment tribunals. And 5528 staff were gagged. Not a proud record
Imperial’s Department of Medicine web site says that one of its aims is to “build a strong and supportive academic community”. Imperial’s spokesman said “Stefan Grimm was a valued member of the Faculty of Medicine”.
The ability of large organisations to tell barefaced lies never ceases to amaze me.
I asked Martin Wilkins to comment on the email from Grimm. His response is the standard stuff that HR issues on such occasions. Not a word of apology, no admission of fault. It says “Imperial College London seeks to give every member of its community the opportunity to excel and to create a supportive environment in which their careers may flourish.”. Unless, that is, your research is insufficiently expensive, in which case we’ll throw you out on the street at 51. For completeness, you can download Wilkins’ mail.
After reading this post, Martin Wilkins wrote again to me (12.21 on 2nd December), He said
“You will appreciate that I am unable to engage in any further discussion – not because of any institutional policy but because there is an ongoing inquest into the circumstances of his death. What I can say is that there was no ongoing correspondence. We met from time to time to discuss science and general matters. These meetings were always cordial. My last meeting with him was to congratulate him on his recent paper, accepted by EMBOL "
The emails now revealed show that the relationship could hardly have been less “cordial”. Martin Wilkins appears to be less than frank about what happened.
If anyone has more correspondence which ought to be known, please send it to me. I don’t reveal sources (if you prefer, use my non-College email david.colquhoun72 (at) gmail.com).
The problem is by no means limited to Imperial. Neither is it universal at Imperial: some departments are quite happy about how they are run. Kings College London, Warwick University and Queen Mary College London have been just as brutal as Imperial. But in these places nobody has died. Not yet.
Follow-up
Here are a few of the tweets that appeared soon after this post appeared.
OMG. If you are an academic at a UK uni read this. Saddest thing I’ve read. Both for the individual and us all. http://t.co/g8xlQbLvDD
— GaryFoster (@Prof_GD_Foster) December 1, 2014
Utterly tragic consequences of running a University Department as a business, via @david_colquhoun. #UofABudgetModel http://t.co/Ql2AJcQqWR
— Dr. Andy Holt (@DrAndyHolt) December 1, 2014
Accusation of a culture of bullying at Imperial from Stefan Grimm, written just before he died. http://t.co/NeXysWFjZM
— Suzi Gage (@soozaphone) December 1, 2014
Bastards “@david_colquhoun: Publish *and* perish at Imperial College: the last email of Stefan Grimm http://t.co/DcQQlfwf78 #bullying”
— John Canning (@johngcanning) December 1, 2014
@david_colquhoun thank you for publishing this. Tragic.
— Melanie Byng (@ThetisMercurio) December 1, 2014
Good grief. This is what happens when science is run like a business http://t.co/3eArX3pRPz (via @david_colquhoun)
— Stephan Neuhaus (@stephanneuhaus1) December 1, 2014
Having worked at Imperial as a PostDoc, I recognise the vile atmosphere portrayed here (via @david_colquhoun). http://t.co/7ogAekAcLR
— Michel Valstar (@MichelValstar) December 1, 2014
Haunting stuff. @david_colquhoun writes of a tragic death and potential case of bullying at #Imperial College London http://t.co/uicA2wPE7S
— Michael Head (@michaelghead) December 1, 2014
@Cesar_F1000 @david_colquhoun Not the kind of "life of the mind" that anyone would want. Can Academia ever be reformed?
— Eric Kansa (@ekansa) December 1, 2014
@david_colquhoun @AMCELL Thank you. This is like something out of Terminator: the rise of the (administrator) machines.
— Dawn Bazely (@dawnbazely) December 1, 2014
@david_colquhoun @timeshighered @ChrisParrTHE "review" by Director of HR? Wonder if they'll find any fault with HR processes?
— calloutloud (@blowthatwhistl1) December 2, 2014
@david_colquhoun @timeshighered @ChrisParrTHE Stefan Grimm case order up a lorry load of whitewash for delivery to @imperialcollege DofHR
— calloutloud (@blowthatwhistl1) December 2, 2014
Shame on my old Uni for this! @david_colquhoun Publish and perish at @imperialcollege: the death of Stefan Grimm http://t.co/l0LZf8u8BX
— Scott Edmunds (@SCEdmunds) December 2, 2014
I saw the best minds of my generation destroyed by madness. @imperialcollege chews up profs. http://t.co/8ibqkk9fjJ @david_colquhoun
— Robert Davidson (@bobbledavidson) December 2, 2014
Saddest blog post I've read for ages: about the pressure on high performing academics by @david_colquhoun http://t.co/gH95I1bMCa
— Mark Brandon (@icey_mark) December 2, 2014
This breaks my heart @david_colquhoun Publish *and* perish at Imperial College London: last email of Stefan Grimm http://t.co/mUsEiM6OJU
— Sylvia McLain (@girlinterruptin) December 2, 2014
"Not a university but a business": Publish AND perish | http://t.co/6aOU6j9CRS | Via @david_colquhoun | Tragic, shameful, urgent
— Tom Farsides (@TomFarsides) December 2, 2014
3 December 2014
The day after this post went public, I wrote to the vice-chancellor of Imperial College, thus.
|
To: alice.gast@imperial.ac.uk cc: w.j.stirling@imperial.ac.uk, s.johal@imperial.ac.uk. d.humphris@imperial.ac.uk Dear Professor Gast You may be aware that last night, at 18.30, I published Stefan Grimm’s last email, see https://www.dcscience.net/?p=6834 In the 12 hours that it’s been public it’s had at least 10,000 views. At the moment, 230 people. from all round the world, are reading it. It seems to be going viral. I appreciate that you are new to the job of rector, so you may not realise that this sort of behaviour has been going on for years at Imperial (especially in Medicine) -I last wrote about the dimwitted methods being used to assess people in Medicine on 2007 -see https://www.dcscience.net/?p=182 Now it seems likely that the policy has actually killed someone (itwas quite predictable that this would happen, sooner or later). I hope that your your humanity will ensure a change of policy in your approach to “performance management”. Failing that, the bad publicity that you’re getting may be enough to persuade you to do so. Best regards David Colquhoun __________________________________ |
Today I updated the numbers: 44,000 hits after 36 hours.
I tried to put it politely, but I have not yet had a reply.
4 December 2014
More than one source at Imperial has sent me a copy of an email sent to staff by the dean of the Faculty of Medicine. It’s dated 03 December 2014 16:44. It was sent almost 24 hours after my post. It is, I suppose, just possible that Kelleher was unaware of my post. But he must surely have seen the internally-circulated version of Grimm’s letter. It isn’t mentioned: that makes the weasel words and crocodile tears in the email even more revolting than they otherwise would be. Both his account and Wilkins’ account contradict directly the account in Grimm’s mail.
Somebody is not telling the truth.
This post has broken all records (for this blog). It has been viewed over 50,000 times in 48 hours. It is still getting 35-40 visitors per minute, as it has for the last 2 days. How much longer will managers at Imperial be able to pretend that the cat hasn’t escaped from the bag?
5 December 2014
|
Late last night. Imperial made, at last. a public comment on the death of Stefan Grimm: Statement on Professor Stefan Grimm by Caroline Davis (Communications and Public Affairs). This bit of shameless public relations appears under a tasteful picture of lilies. |
|
It says “Members of Imperial’s community may be aware of media reports of the tragic loss of Stefan Grimm, professor of toxicology in the Faculty of Medicine”. They could hardly have missed the reports. As of 07.25 this morning, this post alone has been viewed 97,626 times, from all over the world. The statement is a masterpiece of weasel words, crocodile tears and straw man arguments. “Contrary to claims appearing on the internet, Professor Grimm’s work was not under formal review nor had he been given any notice of dismissal”. I saw no allegations that he had actually been fired. He was undoubtedly threatened with being fired. That’s entirely obvious from the email sent by Martin Wilkins to Stefan Grimm. on 10 March. The full text of that mail was published yesterday in Times Higher Education.
It’s worth reproducing the full text of that mail. To write like that to a successful professor, aged 51, is simply cruel. It is obviously incompatible with the PR guff that was issued yesterday. It seems to me to be very silly of Imperial College to try to deny the obvious.
I don’t know how people like Martin Wilkins and Caroline Davis manage to sleep at night.
|
Date: 10 March 2014 Dear Stefan I am writing following our recent meetings in which we discussed your current grant support and the prospects for the immediate future. The last was our discussion around your PRDP, which I have attached. As we discussed, any significant external funding you had has now ended. I know that you have been seeking further funding support with Charities such as CRUK and the EU commission but my concern is that despite submitting many grants, you have been unsuccessful in persuading peer-review panels that you have a competitive application. Your dedication to seek funding is not in doubt but as time goes by, this can risk becoming a difficult situation from which to extricate oneself. In other words, grant committees can become fatigued from seeing a series of unsuccessful applications from the same applicant. I am of the opinion that you are struggling to fulfil the metrics of a Professorial post at Imperial College which include maintaining established funding in a programme of research with an attributable share of research spend of £200k p.a and must now start to give serious consideration as to whether you are performing at the expected level of a Professor at Imperial College. Over the course of the next 12 months I expect you to apply and be awarded a programme grant as lead PI. This is the objective that you will need to achieve in order for your performance to be considered at an acceptable standard. I am committed to doing what I can to help you succeed and will meet with you monthly to discuss your progression and success in achieving the objective outlined. You have previously initiated discussions in our meetings regarding opportunities outside of Imperial College and I know you have been exploring opportunities elsewhere. Should this be the direction you wish to pursue, then I will do what I can to help you succeed. Please be aware that this constitutes the start of informal action in relation to your performance, however should you fail to meet the objective outlined, I will need to consider your performance in accordance with the formal College procedure for managing issues of poor performance (Ordinance D8) which can be found at the following link. Should you have any questions on the above, please do get in touch. Best wishes Martin |
These fixed performance targets are simply absurd. It’s called "research" because you don’t know how ir will come out. I’m told that if you apply for an Academic Clinical Fellowship at Imperial you are told
“Objectives and targets: The goal would be to impart sufficient training in the chosen subspecialty, as to enable the candidate to enter a MD/PhD programme at the end of the fellowship. During the entire academic training programme, the candidate is expected to publish at least five research articles in peer-reviewed journals of impact factor greater than 4.”
That’s a recipe for short term, unoriginal research. It’s an incentive to cut corners. Knowing that a paper has been written under that sort of pressure makes me less inclined to believe that the work has been done thoroughly. It is a prostitution of science.
Later on 5 December. This post has now had 100,000 views in a bit less that four days. At 13.30, I was at Kings College London, to talk to medical students about quackery etc. They were a smart lot, but all the questions were about Stefan Grimm.
The national press have begun to notice the tragedy. The Daily Mail, of all "newspapers" has a fair account of the death. It quotes Professor James Stirling, Provost of Imperial College London, as intoning the standard mantra:
“Imperial seeks to give every member of its community the opportunity to excel and to create a supportive environment in which their careers may flourish. Where we become aware that the College is falling short of this standard of support to its members, we will act”.
In my opinion the email above shows this is simply untrue. This sort of absurd and counterproductive pressure has been the rule in the Department of Medicine for years. I can’t believe that James Stirling didn’t now about it. If he did know, he should be fired for not anticipating the inevitable tragic consequences of his policies. If he didn’t know what was going on, he should be fired for not knowing. .
It is simply absurd for Imperial to allow (In)human resources to investigate itself. Nobody will believe the result.
An independent external inquiry is needed. Soon.
Stefan Grimm’s death is, ultimately, the fault of the use of silly metrics to mismeasure people. If there were no impact factors, no REF, no absurd university rankings, and no ill-educated senior academics and HR people who take them seriously, he’d probably still be alive.
8 December 2014
After one week, I wrote again to the senior management at Imperial (despite the fact that my earlier letters had been ignored). This time I had one simple suggestion. If Imperial want genuinely to set things right they should get an independent external inquiry. Their present proposal that the people who let things go so far should investigate themselves has been greeted with the scepticism that it so richly deserves. I still live in hope that someone will be sufficiently courteous to answer this time.
|
To: alice.gast@imperial.ac.uk cc: w.j.stirling@imperial.ac.uk, s.johal@imperial.ac.uk. d.humphris@imperial.ac.uk, d.kelleher@imperial.ac.uk Dear Professor Gast My post of Stefan Grimm’s email last Monday evening, has been viewed 130,000 times from at least 175 different countries. Your failure to respond to my letters is public knowledge. When you finally posted a statement about Grimm on Thursday it so obviously contradicted the emails which I, and Times Higher Education had already published, that it must have done your reputation more harm than good. May I suggest that the best chance to salvage your reputation would be to arrange for an independent external inquiry into the policies that contributed to Grimm’s death. You must surely realise that your announcement that HR will investigate its own policies has been greeted with universal scepticism. Rightly or wrongly, its conclusions will simply not be believed. I believe that an external inquiry would show Imperial is genuine in wishing to find out how to improve the way it treats the academics who are responsible for its reputation. Best regards David Colquhoun __________________________________ |
Here is a map of the location of 200 hits on 4 December (one of 20 such maps in a 4 hour period).
10 December 2014
Eventually I got a reply, of sorts, from Dermot Kelleher. It’s in the style of the true apparatchik "shut up and go away".
|
Dear Dr Colquhoun
Many thanks for your enquiry. Can I just say that College will liaise with the Coroner as required on this issue. In light of this, I do not believe that further correspondence will be helpful at present.
Best wishes
Dermot
|
Nobody could have been more surprised than I when I found myself nominated as an academic role model at UCL. I had to answer a few questions. It is not obvious to me what the object of the stunt is, but the person who asked me to do it seemed to find the answers amusing, so I’ll reproduce here what I said. I apologise for the temporary lapse into narcissism.
The final version has now been printed [download a copy]. Sadly the printed edition was “corrected” by someone who replaced “whom I asked to submit the first theoretical paper by Hawkes and me to the Royal Society” (as written below), with “paper by Hawkes and I”. Aaargh.
Your nomination – Why you were nominated as an Academic Role Model?
"David Colquhoun has made major contributions to our understanding of how ion channels (proteins which allow charged ions to pass across cell membranes) function to mediate electrical signalling in nerve and muscle cells. This work elegantly combines experimental and theoretical aspects, and resulted in David being made a Fellow of the Royal Society. David Colquhoun played a key role in resisting the notion that UCL should merge with Imperial College in 2002, by running a website opposed to the merger. He thus facilitated the continued existence of an independent UCL. He is also well-known for his principled opposition to therapies that are not based on scientific evidence, and for his blog which comments on this issue as well as on university bureaucracy and politics."
Role models’ questions
1. What is your response to being nominated?
Astounded.
We are interested in giving people a very brief ‘snapshot’ description of their career trajectory, to help a broad range of people see how you got to where you are:
2. What has your career path been?
|
My first job (in 1950s) was as an apprentice pharmacist in Timothy Whites & Taylors (Homeopathic Chemists) in Grange Road, Birkenhead. You can’t get a more humble start than that. But it got me interested in drugs, and thanks to my schoolmaster father, I got to the University of Leeds. |
 My father (1907 – 2001), in 1955 |
|
One of the courses involved some statistics, and that interested me. I think I made a semi-conscious decision that it would be sensible to be good at something that others were bad at, so I learned quite a lot of statistics and mathematics. I recall buying a Methuen’s Monograph on Determinants and Matrices in my final year, and, with the help of an Argentinian PhD student in physical chemistry (not my lecturers) I began to make sense of it. |
|
I purposely went into my final viva with it sticking out of my pocket. The examiner was Walter Perry, then professor of Pharmacology in Edinburgh (he later did a great job setting up the Open University). That’s how I came to be a PhD student in Edinburgh.
Although Perry was one of my supervisors, the only time I saw him was when he came into my lab between committee meetings for a cigarette. But he did make me an honorary lecturer so I could join the Staff Club, where I made many friends, including a young physics lecturer called Peter Higgs. The staff club exists no longer, having been destroyed in one of those acts of short-sighted academic vandalism that vice-chancellors seem so fond of.
The great university expansion in the 1960s made it easy to get a job. The most famous pharmacology department in the world was at UCL so I asked someone to introduce me to its then head, Heinz Schild, and asked him if he had a job. While interned during WW2 he had written a paper on the statistics of biological assay and wanted someone to teach it to students, so I got a job (in 1964), and have been at UCL ever since apart from 9 years. Between 1964 and 1970 I published little, but learned a great deal by writing a textbook on statistics.
That sort of statistics is now thought too difficult for undergraduates, and the famous department that attracted me was itself destroyed in another act of academic vandalism, in 2007.
I have spent my life doing things that I enjoy. Such success as I’ve had, I attribute to a liking for spending time with people cleverer than I am, and wasting time drinking coffee. I found a very clever statistician, Alan Hawkes, in the Housman Room in the late 1960s, and we began to collaborate on the theory of single ion channel analysis in a series of papers that still isn’t quite finished. He did the hard mathematics, but I knew enough about it to write it up in a more or less comprehensible form and to write computer programs to evaluate the algebra. When I got stuck, I would often ask Hyman Kestelman (co-author of the famous mathematics textbook, Massie & Kestelman) to explain, usually in what was then the Joint Staff Common Room at lunch time (it is now the Haldane room, the common room having been confiscated by unenlightened management). Before leaving for the USA in 1970, I, in league with the then professor of French, Brian Woledge, eventually got through a motion that allowed women into the Housman room.
I’d also talk as much as I could to Bernard Katz, whom I asked to submit the first theoretical paper by Hawkes and me to the Royal Society. His comments on the first draft led to the published version making a prediction about single ion channel behaviour before channels could be observed.
The next step was sheer luck. As this was going on, two young Germans, Neher & Sakmann, succeeded in observing the tiny currents that flow through single ion channel molecules, so it became possible to test the theory. In series of visits to Göttingen, Sakmann and I did experiments late into the night. Neher & Sakmann got a well-deserved Nobel Prize in 1991, and I expect I benefitted from a bit of reflected glory
The work that I have done is nothing if not basic. It doesn’t fit in with the current vogue for translational research (most of which will fail), although I would regard it as laying the basis for rational drug design. My only regret is that rational drug design has proved to be so difficult that it won’t be achieved in my lifetime (please don’t believe the hype).
We’d also like you to take a slightly more personal view:
3. What have been the highs (and the lows?) of your career so far?
The highs have been the chance to work with brilliant people and write a handful of papers that have a chance of having a lasting influence. Because I have been able to take my time on those projects there haven’t been too many lows, apart from observing the continuous loss of academic integrity caused by the intense pressure to publish or perish, and the progressive decline in collegiality in universities caused by that pressure combined with the rise in power of managerialism. Luckily the advent of blogs has allowed me to do a little about that.
I’m saddened by the fact that the innumeracy of biologists that I noticed as an undergraduate has not really improved at all (though I don’t believe it is worse). Most biologists still have difficulty with even the simplest equations. Worse still, they don’t know enough maths to communicate their problem to a mathematician, so only too often one sees collaborations with mathematicians produce useless results.
The only real failure I’ve had was when, in a fit of vanity, I applied for the chair of Pharmacology in Oxford, in 1984, and failed to get it. But in retrospect that was really a success too. I would have hated the flummery of Oxford, and as head of department (an increasingly unattractive job) I would have spent my time on pushing paper, not ion channels. In retrospect, it was a lucky escape. UCL is my sort of place (most of the time).
We would like to hear what our role models have to say about the next generation:
4. What advice would you give to people finishing off their PhD?
My career course would be almost impossible now. In fact it is very likely that I would have been fired before I got going in the present climate. There were quite long periods when I didn’t publish much. I was learning the tools of my trade, both mathematical and experimental. Now there is no time to do that. You are under pressure to publish a paper a week (for the glory of your PI and your university) and probably rarely find time to leave the lab to talk to inspiring people. If you are given any courses they’ll probably be in some inane HR nonsense, not in algebra. That is one reason we started our summer workshop, though bizarrely that has now been dropped by the graduate school in favour of Advanced Powerpoint.
The plight of recent PhDs is dire. Too many are taken on (for the benefit of the university, not of the student) and there aren’t many academic jobs. If you want to stay in academia, all I can suggest is that you get good at doing something that other people can’t do, and to resist the pressure to publish dozens of trivial papers.
Try to maintain some academic integrity despite the many pressures to do the opposite that are imposed on you by your elders (but not always betters). That may or may not be enough to get you the job that you want, but at least you’ll be able to hold your head high.
Finally, we want to give a balanced impression of our role models because many were nominated for their ability to motivate others, and to balance life and work:
5. How do you keep motivated?
Work-life balance is much talked about by HR, though they are one of the reasons why it is now almost impossible, In the past it wasn’t a great problem. I’m fascinated by the problems that I’m trying to puzzle out. I’ve had periods of a year or two when things haven’t gone well and I’ve felt as though I was a failure, but luckily they haven’t lasted too long, and they occurred in a time before some idiotic performance manager would harass you for failing to publish for a year or two. The climate of “performance management” is doing a lot to kill innovation and creativity.
6. What do you do when are not working in SLMS?
I’ve had various phases. For a while I carried on boxing (which had been compulsory at school). When I was first at UCL in 1964 I bought a 21 foot sloop (and as a consequence could barely afford to eat), and in 1970 (at Yale) I learned to fly. I had a lot of fun sailing right up to the early 1980s, when I found I could not afford a son as well as a boat. That was when running came into fashion and that could be done for the price of a pair of shoes. I did marathons and half marathons for fun (the London in 1988 was great fun). And that was supplanted by walking country trails in the early 2000s.
There is never a clear division between work and play, especially with algebra. You can continue to struggle with a derivation on a boat, or even get a new angle on it while running. That, of course, is why the transparency review is such total nonsense.
The main cause of stress has never been work for me. Stress comes mainly from the imposition of dim-witted managerialism and incompetent HR policies. And that has become progressively worse. I doubt that if I were a young academic now I’d have the time to spend the weekend sailing.
I’m not sure whether the blogging that has taken up something like half my time since my nominal retirement in 2004 counts as work or not. It certainly depends on things that I have learned in my academic work. And it’s fun to have effects in the real world after a life spent on problems that many would regard as esoteric.
If you want a hobby that costs very little, and allows you to say what you want, start a blog.
Follow-up
Academic staff are going to be fired at Queen Mary University of London (QMUL). It’s possible that universities may have to contract a bit in hard times, so what’s wrong?
What’s wrong is that the victims are being selected in a way that I can describe only as insane. The criteria they use are guaranteed to produce a generation of second-rate spiv scientists, with a consequent progressive decline in QMUL’s reputation.
The firings, it seems, are nothing to do with hard financial times, but are a result of QMUL’s aim to raise its ranking in university league tables.
In the UK university league table, a university’s position is directly related to its government research funding. So they need to do well in the 2014 ‘Research Excellence Framework’ (REF). To achieve that they plan to recruit new staff with high research profiles, take on more PhD students and post-docs, obtain more research funding from grants, and get rid of staff who are not doing ‘good’ enough research.
So far, that’s exactly what every other university is trying to do. This sort of distortion is one of the harmful side-effects of the REF. But what’s particularly stupid about QMUL’s behaviour is the way they are going about it. You can assess your own chances of survival at QMUL’s School of Biological and Chemical Sciences from the following table, which is taken from an article by Jeremy Garwood (Lab Times Online. July 4, 2012). The numbers refer to the four year period from 2008 to 2011.
|
Category of staff |
Research Output Quantity |
Research Output |
Research Income (£) |
Research Income (£) |
|
Professor |
11 |
2 |
400,000 |
at least 200,000 |
|
Reader |
9 |
2 |
320,000 |
at least 150,000 |
|
Senior Lecturer |
7 |
1 |
260,000 |
at least 120,000 |
|
Lecturer |
5 |
1 |
200,000 |
at least 100,000 |
|
In addition to the three criteria, ‘Research Output ‐ quality’, ‘Research Output – quantity’, and ‘Research Income’, there is a minimum threshold of 1 PhD completion for staff at each academic level. All this data is “evidenced by objective metrics; publications cited in Web of Science, plus official QMUL metrics on grant income and PhD completion.” To survive, staff must meet the minimum threshold in three out of the four categories, except as follows: Demonstration of activity at an exceptional level in either ‘research outputs’ or ‘research income’, termed an ‘enhanced threshold’, is “sufficient” to justify selection regardless of levels of activity in the other two categories. And what are these enhanced thresholds? |
|
The university notes that the above criteria “are useful as entry standards into the new school, but they fall short of the levels of activity that will be expected from staff in the future. These metrics should not, therefore, be regarded as targets for future performance.” This means that those who survived the redundancy criteria will simply have to do better. But what is to reassure them that it won’t be their turn next time should they fail to match the numbers? To help them, Queen Mary is proposing to introduce ‘D3’ performance management (www.unions.qmul.ac.uk/ucu/docs/d3-part-one.doc). Based on more ‘administrative physics’, D3 is shorthand for ‘Direction × Delivery × Development.’ Apparently “all three are essential to a successful team or organisation. The multiplication indicates that where one is absent/zero, then the sum is zero!” D3 is based on principles of accountability: “A sign of a mature organisation is where its members acknowledge that they face choices, they make commitments and are ready to be held to account for discharging these commitments, accepting the consequences rather than seeking to pass responsibility.” Inspired? |
I presume the D3 document must have been written by an HR person. It has all the incoherent use of buzzwords so typical of HR. And it says "sum" when it means "product" (oh dear, innumeracy is rife).
The criteria are utterly brainless. The use of impact factors for assessing people has been discredited at least since Seglen (1997) showed that the number of citations that a paper gets is not perceptibly correlated with the impact factor of the journal in which it’s published. The reason for this is the distribution of the number of citations for papers in a particular journal is enormously skewed. This means that high-impact journals get most of their citations from a few articles.
|
The distribution for Nature is shown in Fig. 1. Far from being gaussian, it is even more skewed than a geometric distribution; the mean number of citations is 114, but 69% of papers have fewer than the mean, and 24% have fewer than 30 citations. One paper has 2,364 citations but 35 have 10 or fewer. ISI data for citations in 2001 of the 858 papers published in Nature in 1999 show that the 80 most-cited papers (16% of all papers) account for half of all the citations (from Colquhoun, 2003)
|
|
The Institute of Scientific Information, ISI, is guilty of the unsound statistical practice of characterizing a distribution by its mean only, with no indication of its shape or even its spread. School of Biological and Chemical Sciences-QMUL is expecting everyone has to be above average in the new regime. Anomalously, the thresholds for psychologists are lower because it is said that it’s more difficult for them to get grants. This undermines even the twisted logic applied at the outset.
All this stuff about skewed distributions is, no doubt, a bit too technical for HR people to understand. Which, of course, is precisely why they should have nothing to do with assessing people.
At a time when so may PhDs fail to get academic jobs we should be limiting the numbers. But QMUL requires everyone to have a PhD student, not for the benefit of the student, but to increase its standing in league tables. That is deeply unethical.
The demand to have two papers in journals with impact factor greater than seven is nonsense. In physiology, for example, there are only four journals with an impact factor greater that seven and three of them are review journals that don’t publish original research. The two best journals for electrophysiology are Journal of Physiology (impact factor 4.98, in 2010) and Journal of General Physiology (IF 4.71). These are the journals that publish papers that get you into the Royal Society or even Nobel prizes. But for QMUL, they don’t count.
I have been lucky to know well three Nobel prize winners. Andrew Huxley. Bernard Katz, and Bert Sakmann. I doubt that any of them would pass the criteria laid down for a professor by QMUL. They would have been fired.
The case of Sakmann is analysed in How to Get Good Science, [pdf version]. In the 10 years from 1976 to 1985, when Sakmann rose to fame, he published an average of 2.6 papers per year (range 0 to 6). In two of these 10 years he had no publications at all. In the 4 year period (1976 – 1979 ) that started with the paper that brought him to fame (Neher & Sakmann, 1976) he published 9 papers, just enough for the Reader grade, but in the four years from 1979 – 1982 he had 6 papers, in 2 of which he was neither first nor last author. His job would have been in danger if he’d worked at QMUL. In 1991 Sakmann, with Erwin Neher, got the Nobel Prize for Physiology or Medicine.
The most offensive thing of the lot is the way you can buy yourself out if you publish 26 papers in the 4 year period. Sakmann came nowhere near this. And my own total, for the entire time from my first paper (1963) until I was elected to the Royal Society (May 1985) was 27 papers (and 7 book chapters). I would have been fired.
Peter Higgs had no papers at all from the time he moved to Edinburgh in 1960, until 1964 when his two paper’s on what’s now called the Higgs’ Boson were published in Physics Letters. That journal now has an impact factor less than 7 so Queen Mary would not have counted them as “high quality” papers, and he would not have been returnable for the REF. He too would have been fired.
The encouragement to publish large numbers of papers is daft. I have seen people rejected from the Royal Society for publishing too much. If you are publishing a paper every six weeks, you certainly aren’t writing them, and possibly not even reading them. Most likely you are appending your name to somebody else’s work with little or no checking of the data. Such numbers can be reached only by unethical behaviour, as described by Peter Lawrence in The Mismeasurement of Science. Like so much managerialism, the rules provide an active encouragement to dishonesty.
In the face of such a boneheaded approach to assessment of your worth, it’s the duty of any responsible academic to point out the harm that’s being done to the College. Richard Horton, in the Lancet, did so in Bullying at Barts. There followed quickly letters from Stuart McDonald and Nick Wright, who used the Nuremburg defence, pointing out that the Dean (Tom Macdonald) was just obeying orders from above. That has never been as acceptable defence. If Macdonald agreed with the procedure, he should be fired for incompetence. If he did not agree with it he should have resigned.
It’s a pity, because Tom Macdonald was one of the people with whom I corresponded in support of Barts’ students who, very reasonably, objected to having course work marked by homeopaths (see St Bartholomew’s teaches antiscience, but students revolt, and, later, Bad medicine. Barts sinks further into the endarkenment). In that case he was not unreasonable, and, a mere two years later I heard that he’d taken action.
To cap it all, two academics did their job by applying a critical eye to what’s going on at Queen Mary. They wrote to the Lancet under the title Queen Mary: nobody expects the Spanish Inquisition
"For example, one of the “metrics” for research output at professorial level is to have published at least two papers in journals with impact factors of 7 or more. This is ludicrous, of course—a triumph of vanity as sensible as selecting athletes on the basis of their brand of track suit. But let us follow this “metric” for a moment. How does the Head of School fair? Zero, actually. He fails. Just consult Web of Science. Take care though, the result is classified information. HR’s “data” are marked Private and Confidential. Some things must be believed. To question them is heresy."
Astoundingly, the people who wrote this piece are now under investigation for “gross misconduct”. This is behaviour worthy of the University of Poppleton, as pointed out by the inimitable Laurie Taylor, in Times Higher Education (June 7)
|
The rustle of censorship It appears that last week’s edition of our sister paper, The Poppleton Evening News, carried a letter from Dr Gene Ohm of our Biology Department criticising this university’s metrics-based redundancy programme. We now learn that, following the precedent set by Queen Mary, University of London, Dr Ohm could be found guilty of “gross misconduct” and face “disciplinary proceedings leading to dismissal” for having the effrontery to raise such issues in a public place. Louise Bimpson, the corporate director of our ever-expanding human resources team, admitted that this response might appear “severe” but pointed out that Poppleton was eager to follow the disciplinary practices set by such soon-to-be members of the prestigious Russell Group as Queen Mary. Thus it was only to be expected that we would seek to emulate its espousal of draconian censorship. She hoped this clarified the situation. |
David Bignell, emeritus professor of zoology at Queen Mary hit the nail on the head.
"These managers worry me. Too many are modest achievers, retired from their own studies, intoxicated with jargon, delusional about corporate status and forever banging the metrics gong. Crucially, they don’t lead by example."
What the managers at Queen Mary have failed to notice is that the best academics can choose where to go.
People are being told to pack their bags and move out with one day’s notice. Access to journals stopped, email address removed, and you may need to be accompanied to your (ex)-office. Good scientists are being treated like criminals.
What scientist in their right mind would want to work at QMUL, now that their dimwitted assessment methods, and their bullying tactics, are public knowledge?
The responsibility must lie with the principal, Simon Gaskell. And we know what the punishment is for bringing your university into disrepute.
Follow-up
Send an email. You may want to join the many people who have already written to QMUL’s principal, Simon Gaskell (principal@qmul.ac.uk), and/or to Sir Nicholas Montagu, Chairman of Council, n.montagu@qmul.ac.uk.
Sunday 1 July 2012. Since this blog was posted after lunch on Friday 29th June, it has had around 9000 visits from 72 countries. Here is one of 17 maps showing the origins of 200 of the hits in the last two days
The tweets about QMUL are collected in a Storify timeline.
I’m reminded of a 2008 comment, on a post about the problems imposed by HR, In-human resources, science and pizza.
Thanks for that – I LOVED IT. It’s fantastic that the truth of HR (I truly hate that phrase) has been so ruthlessly exposed. Should be part of the School Handbook. Any VC who stripped out all the BS would immediately retain and attract good people and see their productivity soar.
That’s advice that Queen Mary should heed.
Part of the reason for that popularity was Ben Goldacre’s tweet, to his 201,000 followers
“destructive, unethical and crude metric incentives in academia (spotlight QMUL) bit.ly/MFHk2H by @david_colquhoun”
3 July 2012. I have come by a copy of this email, which was sent to Queen Mary by a senior professor from the USA (word travels fast on the web). It shows just how easy it is to destroy the reputation of an institution.
|
Sir Nicholas Montagu, Chairman of Council, and Principal Gaskell, I was appalled to read the criteria devised by your University to evaluate its faculty. There are so flawed it is hard to know where to begin. Your criteria are antithetical to good scientific research. The journals are littered with weak publications, which are generated mainly by scientists who feel the pressure to publish, no matter whether the results are interesting, valid, or meaningful. The literature is flooded by sheer volume of these publications. Your attempt to require “quality” research is provided by the requirement for publications in “high Impact Factor” journals. IF has been discredited among scientists for many reasons: it is inaccurate in not actually reflecting the merit of the specific paper, it is biased toward fields with lots of scientists, etc. The demand for publications in absurdly high IF journals encourages, and practically enforces scientific fraud. I have personally experienced those reviews from Nature demanding one or two more “final” experiments that will clinch the publication. The authors KNOW how these experiments MUST turn out. If they want their Nature paper (and their very academic survival if they are at a brutal, anti-scientific university like QMUL), they must get the “right” answer. The temptation to fudge the data to get this answer is extreme. Some scientists may even be able to convince themselves that each contrary piece of data that they discard to ensure the “correct” answer is being discarded for a valid reason. But the result is that scientific misconduct occurs. I did not see in your criteria for “success” at QMUL whether you discount retracted papers from the tally of high IF publications, or perhaps the retraction itself counts as yet another high IF publication! Your requirement for each faculty to have one or more postdocs or students promotes the abusive exploitation of these individuals for their cheap labor, and ignores the fact that they are being “trained” for jobs that do not exist. The “standards” you set are fantastically unrealistic. For example, funding is not graded, but a sharp step function – we have 1 or 2 or 0 grants and even if the average is above your limits, no one could sustain this continuously. Once you have fired every one of your faculty, which will almost certainly happen within 1-2 rounds of pogroms, where will you find legitimate scientists who are willing to join such a ludicrous University? |
4 July 2012.
Professor John F. Allen is Professor of Biochemistry at Queen Mary, University of London, and distinguished in the fields of Photosynthesis, Chloroplasts, Mitochondria, Genome function and evolution and Redox signalling. He, with a younger colleague, wrote a letter to the Lancet, Queen Mary: nobody expects the Spanish Inquisition. It is an admirable letter, the sort of thing any self-respecting academic should write. But not according to HR. On 14 May, Allen got a letter from HR, which starts thus.
|
14th May 2012 Dear Professor Allen I am writing to inform you that the College had decided to commence a factfinding investigation into the below allegation: That in writing and/or signing your name to a letter entitled "Queen Mary: nobody expects the Spanish Inquisition," (enclosed) which was published in the Lancet online on 4th May 2012, you sought to bring the Head of School of Biological and Chemical Sciences and the Dean for Research in the School of Medicine and Dentistry into disrepute. . . . . Sam Holborn |
Download the entire letter. It is utterly disgraceful bullying. If anyone is bringing Queen Mary into disrepute, it is Sam Holborn and the principal, Simon Gaskell.
Here’s another letter, from the many that have been sent. This is from a researcher in the Netherlands.
|
Dear Sir Nicholas,
I am addressing this to you in the hope that you were not directly involved in creating this extremely stupid set of measures that have been thought up, not to improve the conduct of science at QMUL, but to cheat QMUL’s way up the league tables over the heads of the existing academic staff. Others have written more succinctly about the crass stupidity of your Human Resources department than I could, and their apparent ignorance of how science actually works. As your principal must bear full responsibility for the introduction of these measures, I am not sending him a copy of this mail. I am pretty sure that his “principal” mail address will no longer be operative. We have had a recent scandal in the Netherlands where a social psychology professor, who even won a national “Man of the Year” award, as well as as a very large amount of research money, was recently exposed as having faked all the data that went into a total number of articles running into three figures. This is not the sort of thing one wants to happen to one’s own university. He would have done well according to your REF .. before he was found out. Human Resources departments have gained too much power, and are completely incompetent when it comes to judging academic standards. Let them get on with the old dull, and gobbledigook-free, tasks that personnel departments should be carrying out. |
5 July 2012.
Here’s another letter. It’s from a member of academic staff at QMUL, someone who is not himself threatened with being fired. It certainly shows that I’m not making a fuss about nothing. Rather, I’m the only person old enough to say what needs to be said without fear of losing my job and my house.
|
Dear Prof. Colquhoun,
I am an academic staff member in SBCS, QMUL. I am writing from my personal email account because the risks of using my work account to send this email are too great. I would like to thank you for highlighting our problems and how we have been treated by our employer (Queen Mary University of London), in your blog. I would please urge you to continue to tweet and blog about our plight, and staff in other universities experiencing similarly horrific working conditions. I am not threatened with redundancy by QMUL, and in fact my research is quite successful. Nevertheless, the last nine months have been the most stressful of all my years of academic life. The best of my colleagues in SBCS, QMUL are leaving already and I hope to leave, if I can find another job in London. Staff do indeed feel very unfairly treated, intimidated and bullied. I never thought a job at a university could come to this.
Thank you again for your support. It really does matter to the many of us who cannot really speak out openly at present.
Best regards,
|
In a later letter, the same person pointed out
"There are many of us who would like to speak more openly, but we simply cannot."
"I have mortgage . . . . Losing my job would probably mean losing my home too at this point."
"The plight of our female staff has not even been mentioned. We already had very few female staff. And with restructuring, female staff are more likely to be forced into teaching-only contracts or indeed fired"."
"total madness in the current climate – who would want to join us unless desperate for a job!"
“fuss about nothing” – absolutely not. It is potentially a perfect storm leading to teaching and research disaster for a university! Already the reputation of our university has been greatly damaged. And senior staff keep blaming and targeting the “messengers"."
6 July 2012.
Througn the miracle of WiFi, this is coming from Newton, MA. The Lancet today has another editorial on the Queen Mary scandal.
"As hopeful scientists prepare their applications to QMUL, they should be aware that, behind the glossy advertising, a sometimes harsh, at times repressive, and disturbingly unforgiving culture awaits them."
That sums it up nicely.
24 July 2012. I’m reminded by Nature writer, Richard van Noorden (@Richvn) that Nature itself has wriiten at least twice about the iniquity of judging people by impact factors. In 2005 Not-so-deep impact said
"Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations."
"None of this would really matter very much, were it not for the unhealthy reliance on impact factors by administrators and researchers’ employers worldwide to assess the scientific quality of nations and institutions, and often even to judge individuals."
And, more recently, in Assessing assessment” (2010).
29 July 2012. Jonathan L Rees. of the University of Edinburgh, ends his blog:
"I wonder what career advice I should offer to a young doctor circa 2012. Apart from not taking a job at Queen Mary of course. "
How to select candidates
I have, at various times, been asked how I would select candidates for a job, if not by counting papers and impact factors. This is a slightly modified version of a comment that I left on a blog, which describes roughly what I’d advocate
After a pilot study the entire Research Excellence Framework (which attempts to assess the quality of research in every UK university) made the following statement.
“No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs”
It seems that the REF is paying attention to the science not to bibliometricians.
It has been the practice at UCL to ask people to nominate their best papers (2 -4 papers depending on age). We then read the papers and asked candidates hard questions about them (not least about the methods section). It’s a method that I learned a long time ago from Stephen Heinemann, a senior scientist at the Salk Institute. It’s often been surprising to learn how little some candidates know about the contents of papers which they themselves select as their best. One aim of this is to find out how much the candidate understands the principles of what they are doing, as opposed to following a recipe.
Of course we also seek the opinions of people who know the work, and preferably know the person. Written references have suffered so much from ‘grade inflation’ that they are often worthless, but a talk on the telephone to someone that knows both the work, and the candidate, can be useful, That, however, is now banned by HR who seem to feel that any knowledge of the candidate’s ability would lead to bias.
It is not true that use of metrics is universal and thank heavens for that. There are alternatives and we use them.
Incidentally, the reason that I have described the Queen Mary procedures as insane, brainless and dimwitted is because their aim to increase their ratings is likely to be frustrated. No person in their right mind would want to work for a place that treats its employees like that, if they had any other option. And it is very odd that their attempt to improve their REF rating uses criteria that have been explicitly ruled out by the REF. You can’t get more brainless than that.
This discussion has been interesting to me, if only because it shows how little bibliometricians understand how to get good science.
Being interested in science communication, I was pleased when the BMJ asked me to review Unscientific America , by Chris Monney and Sheril Kirshenbaum.
The BMJ provides a link that allows you access to the whole review. They have made very few changes from the submitted version, which is reproduced below (with live links in the text. [Download pdf of print version]
I very soon discovered that the book had already caused ructions in the USA, as a result of its advocacy of appeasement of religious groups. In particular there was all out war with P.Z.Myers, whose very popular blog, Pharyngula. documented the battle in detail).
It is an American book through and through, and in the USA the biggest threat to reason comes from the far-right religious fundamentalists who preach young-earth creationism. It is said that 46% of US citizens believe that the earth is less than 10,000 years old. The same far-right religious groups also preach that carrying guns is good, that Iraq was responsible for 9/11, that climate change is a socialist conspiracy and that health care for everyone is a communist plot. And they never hesitate to lie in the promotion of their ‘religious’ views. The US situation is totally different from that in Europe, where religion is all but dead, and young earth creationism is the preserve of a few cranky used-car dealers (and possibly Tony Blair?)
|
Review of the Week Trust me, I’m a scientist David Colquhoun, professor of pharmacology, University College London Unscientific America sounds like a fascinating topic, not least because the book is a follow-up from Mooney’s The Republican War on Science. It is written entirely from a US perspective (the USA sequenced the genome and invented the internet, apparently unaided). It’s reported that 46 percent of Americans believe that the earth is less than 10,000 years old. That’s certainly cause for alarm and Mooney & Kirshenbaum are certainly alarmed. They think that the public needs to be educated in science. They identify the obvious problems, evolution, climate change and quackery and ask what can be done. The problem is that they propose no good solutions, and some bad ones. Their aims are worthy but sometimes the book reads like an over-long and somewhat condescending whine about why science and scientists are not sufficiently appreciated. I simply don’t think that it’s true that the public are not interested in science, nor that they can’t understand it at a level that is sufficient to be useful. It’s true that they have been let down badly by some sections of the media. Think particularly of the “great MMR hoax”1. The disastrous fall in vaccination is more attributable to talk show presenters and air-headed celebrities than to lack of interest from the public. People are systematically deceived by anti-vaxers, climate change denialists, vitamin pill salesmen and a horde of crackpot alternative therapists. There is one problem that Mooney & Kirshenbaum don’t talk about at all, yet it seems to me to be one of the biggest problems in science communication. It isn’t lack of interest by the public, nor even lack of understanding, but lack of trust. The tendency of real science to indulge in hyperbolic self-promotion is one reason for the lack of trust. Sometimes this descends into outright dishonesty2,3. That is a tendency that is promoted by government and funding agencies by their insistence on imposing silly performance measures. The public is quite sensible enough to take with a pinch of salt the almost daily announcements of “cancer cures” that emanate from university press offices. On the face of it, one should be encouraged that ‘public engagement in science’ is the mode du jour. It isn’t quite that simple though. Only too often, universities regard public engagement as a branch of their own PR machine4. They even instruct you about what tone of voice to use when talking publicly. One reason why scientists need to talk to people outside the lab is precisely to counteract this tide of nonsense from PR people, who are paid to deceive. The problem for academics is usually time. We already do three jobs, teaching, research and coping with HR bollocks. How can we find time for a fourth job? That’s not easy, especially for the best researchers (those that do research themselves, not just lead a team). Mooney & Kirshenbaum suggest that the solution is to create a “cadre of communication and outreach experts”. I don’t think this would work. They would, by and large, be outsiders, writing uncritical paeans, dictated by big name scientists. A new cadre of PR hangers-on does not sound like a great idea. A better, and very much cheaper, solution would be to provide a course in free blogging software and we’ll do it ourselves. The two chapters that I looked forward to reading, on religion and on “The bloggers cannot save us”, proved deeply disappointing. The authors are firmly in the camp of what Richard Dawkins called the “Neville Chamberlain school of evolutionists&rdquo.; They maintain “if the goal is to create an America more friendly to science and reason, the combativeness of the New Atheists is strongly counterproductive”. They are particularly critical of P.Z. Myers5, the University of Minnesota developmental biologist who is splendidly clear in his views. Of the communion wafer, he famously said “It’s a frackin’ cracker”. But he, and Dawkins, are right. When it comes to young earth creationists we have a war on our hands, and nowhere more than in the USA. What’s more it’s a winnable war. Mooney & Kirshenbaum are all for appeasement, but appeasement won’t work. It might please the more moderate wings of the church, but they already believe in evolution and are regarded by fundamentalists as being just as big an enemy as Myers and Dawkins. And, one must ask, who has done best at getting a wide public readership? P.Z Myers’ blog, Pharyngula, has up to two million page views a month. Dawkins’ book The God Delusion has sold three million copies. In comparison the bland and often rather condescending corporate science web sites get tiny numbers of hits. In Europe in general, and the UK in particular, young earth creationists are not the major problem that they are in the USA, despite being supported by Tony Blair6. Perhaps the nearest analogy in Europe is the threat to reason from various sorts of crackpot medicine. The appeasers are widespread. The Royal Colleges and the Department of Health are at the forefront of the Neville Chamberlain approach. But appeasement hasn’t worked there either. What has worked is the revelation that university courses are teaching that “amethysts emit high yin energy”7. Or, in a lecture on herbal approaches for patients with cancer, “Legally, you cannot claim to cure cancer. This is not a problem because: ‘we treat people, not diseases’ “8. This is shocking stuff but it has not been unearthed by the corporate media, but by bloggers. I think Mooney and Kirshenbaum have it all wrong. They favour corporate communications, which are written by people outside science and which easily become mere PR machines for individuals and institutions. Such blogs are rarely popular and at their worst they threaten the honesty of science. More and more individual scientists have found that they can write their own blog. It costs next to nothing and you can say what you think. A few clicks and the world can read what you have to say. Forget corporate communications. Just do it yourself. It’s fun. And think of the money you’d save for doing science if the PR people were just fired. (1) Goldacre, B. The media’s MMR hoax. 2008 http://www.badscience.net/2008/08/the-medias-mmr-hoax/ (2) PLoS One. Ghostwriting documents now fully available on PLoS Medicine website. 21-8-2009 http://speakingofmedicine.plos.org/2009/08/21/ghostwriting-documents-now-fully-available-on-plos-medicine-website/ (3) Colquhoun, D. Universities Inc. in the UK. The Corporate Corruption of Higher Education: part 2. 6-12-2007 https://www.dcscience.net/?p=193 (4) Corbyn, Z. Nottingham raises eyebrows over definition of ‘public engagement’. 21-8-2008 http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=403234 (5) Myers, P. Z. Pharyngula. 2009 http://scienceblogs.com/pharyngula/ (6) Pyke, N. Revealed: Blair’s link to schools that take the Creation literally (Independent 13 June 2004). 13-7-2004 http://www.independent.co.uk/news/uk/politics/revealed-blairs-link-to-schools-that-take-the-creation-literally-732032.html (7) Colquhoun, D. Westminster University BSc: "amethysts emit high yin energy". 23-4-2008 https://www.dcscience.net/?p=227 (8) Colquhoun, D. Herbal approaches for patients with cancer. 10-8-2009 https://www.dcscience.net/?p=2043 |
Follow-up
P.Z. Myers has posted about thie book review, on Pharyngula, as Is this book dead yet? There are a lot more comments there than here, though few of them address the question of science communications..
Butterflies and Wheels is generating a lot of hits
The King’s Fund recently published Assessing complementary practice Building consensus on appropriate research methods [or download pdf].
It is described as being the “Report of an independent advisory group”. I guess everyone knows by now that an “expert report” can be produced to back any view whatsoever simply by choosing the right “experts”, so the first things one does is to see who wrote it. Here they are.
- Chair: Professor Dame Carol Black
- Harry Cayton, Chief Executive, Council for Healthcare Regulatory Excellence
- Professor Adrian Eddleston, then Vice-Chairman, The King’s Fund
- Professor George Lewith, Professor of Health Research, Complementary and Integrated Medicine Research Unit, University of Southampton
- Professor Stephen Holgate, MRC Clinical Professor of Immunopharmacology, University of Southampton
- Professor Richard Lilford, Head of School of Health and Population Sciences, University of Birmingham
We see at once two of the best known apologists for alternative medicine, George Lewith (who has appeared here more than once) and Stephen Holgate.
Harry Cayton is CEO of Council for Healthcare Regulatory Excellence (CHRE) which must be one of the most useless box-ticking quangos in existence. It was the CHRE that praised the General Chiropractic Council (GCC) for the quality of its work. That is the same GCC that is at present trying to cope with 600 or so complaints about the people it is supposed to regulate (not to mention a vast number of complaints to Trading Standards Offices). The GCC must be the prime example of the folly of giving government endorsement to things that don’t work. But the CHRE were not smart enough to spot that little problem. No doubt Mr Cayton did good work for the Alzheimer’s Society. His advocacy of patient’s choice may have helped me personally. But it isn’t obvious to me that he is the least qualified to express an opinion on research methods in anything whatsoever. According to the Guardian he is “BA in English and linguistics from the University of Ulster; diploma in anthropology from the University of Durham; B Phil in philosophy of education from the University of Newcastle.”
Adrian Eddlestone is a retired Professor of Medicine. He has been in academic administration since 1983. His sympathy for alternative medicine is demonstrated by the fact that he is also Chair of the General Osteopathic Council, yet another “regulator” that has done nothing to protect the public
from false health claims (and which may, soon, find itself in the same sort of trouble as the GCC).
Richard Lilford is the only member of the group who has no bias towards alternative medicine and also the only member with expertise in clinical research methods His credentials look impressive, and his publications show how he is the ideal person for this job. I rather liked also his article Stop meddling and let us get on.. He has written about the harm done by postmodernism and relativism, the fellow-travellers of alternative medicine.
Most damning of all, Lewith, Eddlestone and Holgate (along with Cyril Chantler, chair of the King’s Fund, and homeopaths, spiritual healers and Karol Sikora) are Foundation Fellows of the Prince of Wales Foundation for Magic Medicine, an organisation that is at the forefront of spreading medical misinformation.
I shall refer here to ‘alternative medicine’ rather than ‘complementary medicine’ which is used in the report. It is not right to refer to a treatment as ‘complementary’ until such time as it has been shown to work. The term ‘complementary’ is a euphemism that, like ‘integrative’, is standard among alternative medicine advocates whose greatest wish is to gain respectability.
The Report
The recommendations
On page 10 we find a summary of the conclusions.
The report identifies five areas of consensus, which together set a framework for moving forward. These are:
- the primary importance of controlled trials to assess clinical and cost effectiveness.
- the importance of understanding how an intervention works
- the value of placebo or non-specific effects
- the need for investment and collaboration in creating a sound evidence base
- the potential for whole-system evaluation to guide decision-making and subsequent research.
The first recommendation is just great. The rest sound to me like the usual excuses for incorporating ineffective treatments into medical practice. Notice the implicit assumption in the fourth point
that spending money on research will establish “a sound evidence base". There is a precedent, but it is ignored. A huge omission from the report is that it fails to mention anywhere that a lot of research has already been done.
Much research has already been done (and failed)
The report fails to mention at all the single most important fact in this area. The US National Institutes of Health has spent over a billion dollars on research on alternative medicines, over a period
of more than 10 years. It has failed to come up with any effective treatments whatsoever. See, for example Why the National Center for Complementary and Alternative Medicine (NCCAM) Should Be Defunded; Should there be more alternative research?; Integrative baloney @ Yale, and most recently, $2.5B Spent, No Alternative Med Cures found. .
Why did the committee think this irrelevant? I can’t imagine. You guess.
The report says
“This report outlines areas of potential consensus to guide research funders, researchers, commissioners and complementary practitioners in developing and applying a robust evidence base for complementary practice.”
As happens so often, there is implicit in this sentence the assumption that if you spend enough money evidence will emerge. That is precisely contrary to the experence in the USA where spending a billion dollars produced nothing beyond showing that a lot of things we already thought didn’t work were indeed ineffective.
And inevitably, and tragically, NICE’s biggest mistake is invoked.
“It is noteworthy that the evidence is now sufficiently robust for NICE to include acupuncture as a treatment for low back pain.” [p ]
Did the advisory group not read the evidence used (and misinterpeted) by NICE? It seems not. Did the advisory group not read the outcome of NIH-funded studies on acupuncture as summarised by Barker Bausell in his book, Snake Oil Science? Apparently not. It’s hard to know because the report has no references.
George Lewith is quoted [p. 15] as saying “to starve the system of more knowledge means we will continue to make bad decisions”. No doubt he’d like more money for research, but if a billion dollars
in the USA gets no useful result, is Lewith really likely to do better?
The usual weasel words of the alternative medicine industry are there in abundance
“First, complementary practice often encompasses an intervention (physical treatment or manipulation) as well as the context for that intervention. Context in this setting means both the physical setting for the delivery of care and the therapeutic relationship between practitioner and patient.” [p. 12]
Yes, but ALL medicine involves the context of the treatment. This is no different whether the medicine is alternative or real. The context (or placebo) effect comes as an extra bonus with any sort of treatment.
“We need to acknowledge that much of complementary practice seeks to integrate the positive aspects of placebo and that it needs to be viewed as an integral part of the treatment rather than an aspect that should be isolated and discounted.” [p. 13]
This is interesting. It comes very close (here and elsewhere) to admitting that all you get is a placebo effect, and that this doesn’t matter. This contradicts directly the first recommendation of the House of Lords report (2000).. Both the House of Lords report on Complementary and Alternative Medicine, and the Government’s response to it, state clearly
“. . . we recommend that three important questions should be addressed in the following order”. (1) does the treatment offer therapeutic benefits greater than placebo? (2) is the treatment safe? (3) how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?.
The crunch comes when the report gets to what we should pay for.
“Should we be prepared to pay for the so-called placebo effect?
The view of the advisory group is that it is appropriate to pay for true placebo (rather than regression to the mean or temporal effects).” [p 24]
Perhaps so, but there is very little discussion of the emormous ethical questions:that this opinion raises:
- How much is one allowed to lie to patients in order to elicit a placebo effect?
- Is is OK if the practitioner believes it is a placebo but gives it anyway?
- Is it OK if the pratitioner believes that it is not a placebo when actually it is?
- Is it OK for practitioners to go degrees taught by people who believe that it is not a placebo when actually it is?
The report fails to face frankly these dilemmas. The present rather absurd position in which it is considered unethical for a medical practitioner to give a patient a bottle of pink water, but
perfectly acceptable to refer them to a homeopath. There is no sign either of taking into account the cultural poison that is spread by telling people about yin, yang and meridians and such like preposterous made-up mumbo jumbo. That is part of the cost of endorsing placebos. And just when one thought that believing things because you wished they were true was going out of fashion
Once again we hear a lot about the alleged difficulties posed by research on alternative medicine. These alleged difficulties are, in my view, mostly no more than excuses. There isn’t the slightest
difficulty in testing things like herbal medicine or homeopathy, in a way that preserves all the ‘context’ and the ways of working of homeopaths and herbalists. Anyone who reads the Guardian knows
how to do that.
In the case of acupuncture, great ingenuity has gone into divising controls. The sham and the ‘real’ acupuncture always come out the same. In a non-blind comparison between acupuncture and no acupuncture the latter usually does a bit worse, but the effects are small and transient and entirely compatible with the view that it is a theatrical placebo.
Despite these shortcomings, some of the conclusions [p. 22] are reasonable.
“The public needs more robust evidence to make informed decisions about the use of complementary practice.
Commissioners of public health care need more robust evidence on which to base decisions about expenditure of public money on complementary practice.”
What the report fails to do is to follow this with the obvious conclusion that such evidence is largely missing and that until such time as it is forthcoming there should be no question of the NHS paying for alternative treatments.
Neither should there be any question of giving them official government recognition in the form of ‘statutory regulation’. The folly of doing that is illustrated graphically by the case of chiropractic which is now in deep crisis after inspection of its claims in the wake of the Simon Singh defamation case. Osteopathy will, I expect, suffer the same fate soon.
In the summary on p.12 we see a classical case of the tension
Controlled trials of effectiveness and cost-effectiveness are of primary importance
We recognise that it is the assessment of effectiveness that is of primary importance in reaching a judgement of different practices. Producing robust evidence that something works in practice – that it is effective – should not be held up by the inevitably partial findings and challenged interpretations arising from inquiries into how the intervention works.
The headline sounds impeccable, but directly below it we see a clear statement that we should use treatments before we know whether they work. “Effectiveness”, in the jargon of the alternative medicine business, simply means that uncontrolled trials are good enough. The bit about “how it works” is another very common red herring raised by alternative medicine people. Anyone who knows anything about pharmacology that knowledge about how any drug works is incomplete and often turns out to be wrong. That doesn’t matter a damn if it performs well in good double-blind randomised controlled trials.
One gets the impression that the whole thing would have been a lot worse without the dose of reality injected by Richard Lilford. He is quoted as a saying
“All the problems that you find in complementary medicine you will encounter in some other kind of treatment … when we stop and think about it… how different is it to any branch of health care – the answer to emerge from our debates is that it may only be a matter of degree.” [p. 17]
I take that to mean that alternative medicine poses problems that are no different from other sorts of treatment. They should be subjected to exactly the same criteria. If they fail (as is usually the case) they should be rejected. That is exactly right. The report was intended to produce consensus, but throughout the report, there is a scarcely hidden tension between believers on one side, and Richard Lilford’s impeccable logic on the other.
Who are the King’s Fund?
The King’s Fund is an organisation that states its aims thus.
“The King’s Fund creates and develops ideas that help shape policy, transform services and bring about behaviour change which improve health care.”
It bills this report on its home page as “New research methods needed to build evidence for the effectiveness of popular complementary therapies”. But in fact the report doesn’t really recommend ‘new research methods’ at all, just that the treatments pass the same tests as any other treatment. And note the term ‘build evidence’. It carries the suggestion that the evidence will be positive. Experience in the USA (and to a smaller extent in the UK) suggests that every time some good research is done, the effect is not to ‘build evidence’ but for the evidence to crumble further
If the advice is followed, and the results are largely negative, as has already happened in the USA, the Department of Health would look pretty silly if it had insisted on degrees and on statutory regulation.
The King’s Fund chairman is Sir Cyril Chantler and its Chief Executive is Niall Dickson. It produces reports, some of which are better than this one. I know it’s hard to take seriously an organisation that wants to “share its vision” withyou, but they are trying.
“The King’s Fund was formed in 1897 as an initiative of the then Prince of Wales to allow for the collection and distribution of funds in support of the hospitals of London. Its initial purpose was to raise money for London’s voluntary hospitals,”
It seems to me that the King’s Fund is far too much too influenced by the present Prince of Wales. He is, no doubt, well-meaning but he has become a major source of medical misinformation and his influence in the Department of Health is deeply unconstitutional. I was really surprised to see thet Cyril Chantler spoke at the 2009 conference of the Prince of Wales Foundation for Integrated Health, despite having a preview of the sort of make-believe being propagated by other speakers. His talk there struck me as evading all the essential points. Warm, woolly but in the end, a danger to patients. Not only did he uncritically fall for the spin on the word “integrated”, but he also fell for the idea that “statutory regulation” will safeguard patients.
Revelation of what is actually taught on degrees in these subjects shows very clearly that they endanger the public.
But the official mind doesn’t seem ever to look that far. It is happy ticking boxes and writing vacuous managerialese. It lacks curiosity.
Follow-up
The British Medical Journal published today an editorial which also recommends rebranding of ‘pragmatic’ trials. No surprise there, because the editorial is written by Hugh MacPherson, senior research fellow, David Peters, professor of integrated healthcare and Catherine Zollman, general practitioner. I find it a liitle odd that the BMJ says “Competing Interests: none. David Peters interest is obvious from his job description. It is less obvious that Hugh MacPherson is an acupuncture enthusiast who publishes mostly in alternative medicine journals. He has written a book with the extraordinary title “Acupuncture Research, Strategies for Establishing an Evidence Base”. The title seems to assume that the evidence base will materialise eventually despite a great deal of work that suggests it won’t. Catherine Zollman is a GP who is into homeopathy as well as acupuncture. All three authors were speakers at the Prince of Wales conference, described at Prince of Wales Foundation for magic medicine: spin on the meaning of ‘integrated’.
The comments that follow the editorial start with an excellent contribution from James Matthew May. His distinction between ‘caring’ and ‘curing’ clarifies beautifully the muddled thinking of the editorial.
Then a comment from DC, If your treatments can’t pass the test, the test must be wrong. It concludes
“At some point a stop has to be put to this continual special pleading. The financial crisis (caused by a quite different group of people who were equally prone to wishful thinking) seems quite a good time to start.”
Des Spence, a general practitioner in Glasgow, has revealed a memorandum that was allegedly leaked from the Department of Health. It was published in the Britsh Medical Journal (17 June 2009, doi:10.1136/bmj.b2466, BMJ 2009;338:b2466). It seemed to me to deserve wider publicity, so with the author’s permission, I reproduce it here. It may also provide a suitable introduction to a forthcoming analysis of a staff survey.
|
Re: The use of ‘note pads’ in the NHS and allied service based agencies. Hi, all care providers, managers of care, care managers, professions allied to care providers, carers’ carers, and stakeholders whose care is in our care. (And a big shout to all those service users who know me.) We report the findings from a quality based review, with a strong strategic overview, on the use of “note pads” across all service user interfaces. This involved extensive consultation with focus groups and key stakeholders at blue sky thinking events (previously erroneously known as brain storming). This quality assured activity has precipitated some heavy idea showers, allowing opinion leaders to generate a national framework of joined-up thinking. This will take this important quality agenda forward. A 1000 page report is available to cascade to all relevant stakeholders. The concentric themes underpinning this review are of confidentiality. Notes have been found on the visual interface devices on computers and writing workstations throughout the NHS work space. Although no actual breach of confidentiality has been reported, the independent external consultants reported that note pads “present a clear and present danger” to the NHS, and therefore there is an overarching responsibility to protect service users from scribbled messages in felt tip pen. Accordingly all types of note pads will be phased out in the near time continuum. A validated algorithm is also attached to aid this process going forward. This modernising framework must deliver a paradigm shift in the use of note pads. Care provider leaders must employ all their influencing and leverage talents to win the hearts and minds of the early adopter. A holistic cradle to grave approach is needed, with ownership being key, and with a 360 degree rethink of the old think. All remaining note pads must be handed over in the next four week ” note pad armistice” to be shredded by a facilitator (who is currently undergoing specialist training) and who will sign off and complete the audit trail. (Please note that the NHS’s email system blocks all attachments, so glossy, sustainable, wood based hard copies will be sent directly to everyone’s waste recycling receptacles.) Cite this as: BMJ 2009;338:b2466 |
Spence added a footnote, Note: The BMJ’s lawyers have insisted that I make it clear that this is a spoof, just in case you were wondering.
Here are a few more
| There is an initiative underway to determine what we do as an organisation in the realms of drug discovery. The intention is to identify internal and appropriate external capabilities to foster a pipeline of competencies that enable some of our basic research outputs to better impact healthcare. |
Follow-up
Two days ago I wrote NICE falls for Bait and Switch by acupuncturists and chiropractors: it has let down the public and itself
Now the official ‘guidance’ is out, and it is indeed quite as bad as the draft.
The relevant bits now read thus.
- Offer one of the following treatment options, taking into account patient preference: an exercise programme, a course of manual therapy or a course of acupuncture. Consider offering another of these options if the chosen treatment does not result in satisfactory improvement.
- Consider offering a structured exercise programme tailored to the person:
- This should comprise of up to a maximum of 8 sessions over a period of up to 12 weeks.
- Offer a group supervised exercise programme, in a group of up to 10 people.
- A one-to-one supervised exercise programme may be offered if a group programme is not suitable for a particular person.
- Consider offering a course of manual therapy, including spinal manipulation, comprising up to a maximum of nine sessions over a period of up to 12 weeks
- Consider offering a course of acupuncture needling comprising up to a maximum of 10 sessions over a period of up to 12 weeks.
The summary guidance still doesn’t mention chiropractic explicitly, just a coy reference to “spinal manipulation”. At a time when the British Chiropractic Association is busy trying to suppress free speech via the law courts, I guess it isn’t surprising that they don’t like to use the word.
The bias in the recommendations is perhaps not surprising because the guidance development group had a heavy representation from alternative medicine advocates, and of people with a record of what, is, in my view, excessive tolerance of mumbo-jumbo.
Royal College of General Practitioners (RCGP) played a large part in writing the guidance. That is an organisation thar has steadfastly refused to make any sort of sensible statement of policy about magic medicine. It isn’t long since I was told by a senior person at the RCGP that there was nothing odd about neuro-linguistic programming. That does not bode well.
Professor Martin Underwood, Professor of Primary Care Research Warwick Medical School, University of Warwick. Underwood chaired the guidance development group. Martin Underwood is also a GP in Coventry See also the British Osteopathic Association. Underwood was lead author of the BEAM trial (download reprint). It is well worth reading the comments on this trial too. It was a randomised trial (though not, of course, blind).
|
Our main aim was to estimate, for patients consulting their general practitioner with back pain, the effectiveness of adding the following to best care in general practice: a class based exercise programme (“back to fitness”), a package of treatment by a spinal manipulator (chiropractor, osteopath, or physiotherapist), or manipulation followed by exercise Conclusions |
In other words, none of them worked very well. The paper failed to distinguish between manipulation by physiotherapists, chiropractors and osteopaths and so missed a valuable chance to find out whether there is an advantage to employing people from alternative medicine (the very problem that this NICE guidance should have dealt with)
Steve Vogel, another member of the guidance development group, is an osteopath. Osteopathy has cast off its shady origins better than chiropractic, but it remains a largely evidence-free zone, and is still usually classified as alternative medicine.
Peter Dixon DC FCC FBCA is chairman of the General Chiropractic Council and founder of Peter Dixon Associates. They run six chiropractic clinics. The claims made by these clinics on their web site are mostly related to musculo-skeletal conditions (not, all spine-related however). But they also include
“Chiropractic is also suitable for relieving the symptoms of stress, and optimizing the way the body works is one of the best ways of enhancing health and wellbeing.”
I wonder how they could justify that claim? Then we get
A chiropractor is always willing to discuss individual conditions or answer questions in order to assess whether chiropractic may be of benefit.
Thanks for the invitation. I accepted it. After the first two phone calls, I had learned all I needed.
The first of Peter Dixon’s clinics that I called, was asked about my son who was suffeiing from perpetual colic. I asked for advice. I was put through to the chiropractor. Here are some extracts from the conversation.
“I think now it’s termed irritable baby syndrome. . . . . We’ve found chiropractic is very effective for colic . . . . £50 up to 3/4 hour which would involve taking a case history, examining the baby, with regard to seeing how the joints in the spine work because often colic is down to, er um, faulty movement patterns in the spine. We deal with an awful lot of things to do with the frame -how the spine moves -and it’s often problems with the way that the bones of the spine move in infants and babies that is the underlying cause in colicky symptoms. . . usually on the whole, I’ve shot myself in the foot by saying this in the past, if there are -um when I say back problems they’re usually temporary issues -when they are there the response for most babies is quite quick . . . it depends on how it’s related to what’s gone on in the birth process -there is usually some traumatic cause and it usually sets up a problem in the upper neck or the mid-back and that will drive those colic symptoms. . . . Yes it’s quite a straightforward thing we just check the baby’s spine and see if it’s problem we can deal with or not.”
Yes, I think you just shot yourself in the foot again. I have no idea in what fraction of cases a chiropractor would fail to claim that it was a case that they could deal with. At least that possibility was envisaged. But when I asked where he’d refer me to in such case I was told it might be a result of bottle feeding -“what they consume”. So I expect it would have been sent to some sort of “nutritional therapist” who would have used one of their usual battery of unreliable tests for food allergies.
I tried a second clinic in Peter Dixon’s empire and asked if chiropractic could help with asthma.”It can do, depending on what type of asthma you have". After some questions I was asked
“Do you have any general aches and pains otherwise, any tension .problems round the rib cage or thoracic spine . . . There are two things we’d look as chiropractors as to whether we could help or not and that would include the tension round the rib cage -in any type of asthma you’ll become tight round the rib cage and merely by keeping that area loose you’d help to stop the asthma from becoming quite as bad. And the second part is we’d look at the neck area with you to see if there is any restriction there -which houses the nerve supply to the lungs which can be indicated in some asthma cases also.”
I was invited to come for a free screen, so I asked if it usually worked quickly, “It depends what the problem is but you’d need a course I suspect”. £45 for the first appointment then £28 per session. “In most cases you’d need to look at a six to eight session course”.
Would anyone with experience of crying babies or of asthma like to comment on these proposed treatments? They are not my idea of evidence-based treatments and I find it quite surprising that someone who sponsors them is thought appropriate to write guidance for NICE.
This sort of ‘sting’ always makes me feel a bit uneasy, but it seems to be the only way to find out what actually goes on. And what goes on has all the appearance of classic bait and switch. You go in for your backache, and before you know where you are you are being sold a course of treatment to stop your baby crying.
Media follow-up after the NICE announcement
Sadly, I heard that the Today programme (my favourite news programme) was pretty wet. if only their science reporting was as good as their politics reporting.
The 27th May was certainly pretty busy for me, Apart from a couple of local paper interviews, this is what happened,
Sky News TV. Richard Suchet and two cameraman came to UCL and filmed a lot of stuff out of which a 20 second clip was used. And then they grumble that they don’t make money.
BBC TV 6 o’clock News. Similar, but at least only two people this time. Both TV stations spent ages showing pictures of people having needles pushed into them and very little time discussing the problems. A pathetically bad attempt at science reporting. Radio, on the whole, did much better
BBC Three Counties Radio (09.20) Host Ronnie Barbour, The daytime talk shows may be appalling to some of us, but the hosts did a far better job of airing the problems than TV [download the mp3].
Radio 5 Live Victoria Derbyshire [download the mp3]
BBC Radio Sheffield (12.05) was the best talk show by far. First speaker was Steve Vogel, the osteopath from the guidance development group. Although invited by the rather sensible host, Rony, to respond to the idea that a secondary consequence of the guidance would be to introduce hocus-pocus, he steadfastly refused to answer the question. At the end a rather sensible GP summed up the view from the coalface. [play the mp3]
Radio 4 PM programme This excellent early evening news programme is run by Eddie Mair. It was the best interview yet. The other side was put by Martin Underwood, chair of the guidance development group [play the mp3]
Underwood said “the evidence shows that it [acupuncture] works”. I disagree. The evidence shows that acupuncture, in a non-blind comparison with no acupuncture shows a small, variable additional effect that doesn’t last and is of marginal clinical significance, That is not what I call “works”. Underwood then indulges in the subgroup analysis fallacy by asserting that a few people get a large benefit from acupuncture. Or perhaps a few people just happen to get better that day. I was not convinced.
More 4 News TV 20.20 was an interview, with the other side being put by my old friend, George Lewith.
During the introduction, an acupuncturist. Lisa Sherman, ‘explained’ acupuncture in the usual sort of utterly meaningless words that illustrate perfectly the problem.
George Lewith said that “we don’t have a good pretend form of acupuncture”. “We don’t have a good placebo and we can demonstrate that acupuncture is literally twice as effective as conventional care”. The first is simply nonsense: huge efforts have gone into developing good controls in acupuncture studies (see, for example, Barker Bausell’s book, Snake Oil Science). And I hope that George will send the references for his “twice as effective” claim. It seems to me to be nothing short of preposterous.
My conclusions
So what went wrong?
One problem could be regarded as medical arrogance. The fact is that the problem of low back pain has not been solved, either by drugs or anything else. It is a failure of medicine (and of pharmacology). That should be the premise of all discussions, and it smacks of arrogance not to tell people straight out. Of course there are some cases when causes can be identified, and perhaps remedied, usually by surgery, but these are the exceptions not the rule.
One consequence of the inability of medicine to help much is that patients get desperate, and willing to try anything. And of course if they happen to have a remission, that is attributed to the treatment, however preposterous that treatment may have been. Conditions like back pain that come and go unpredictable are a gift for quacks.
Another consequence, for researchers rather than patients, is to clutch at straws. Even small and inconsistent effects are seized on as ‘successes’. This phenomenon seems to be part of the reason for the NICE guidance. Another reason is, almost certainly, the grinding of axes by some of the people who wrote it.
What should be done now?
Professor Sir Michael Rawlins is a sensible chap. He cares about evidence, But it does seem that in this case, he might have taken his eye off the ball, for once. In my opinion, he should restore the reputation of NICE by withdrawing this guidance and starting again.
Follow-up
An interesting document has come my way. It shows the responses of the guidance group to the consultation on the draft guidance. Many people made comments not unlike mone, but they were all brushed aside in a way that looks to me rather partisan. The document appears to have vanished from NICE’s web site, but you can download it here.
Friday 29th May. The Times prints a letter from two consultants in pain medicine, Joan Hestor and Stephen Ward. It says, inter alia
“As experienced pain specialists we feel that NICE has lost its way in publishing these guidelines.”
“We are saddened that NICE has chosen to ignore our important role and promote seemingly unworkable and for the most part clinically ineffective treatments”
Friday 29th May. NICE has had a good reputation in the USA for its important, and usually high-quality, attempts to assess what works and what doesn’t. That makes it all the sadder to see it condemned already from the USA for its latest effort. The excellent Yale neurologist, Steven Novella, has written about it on his Neurologica blog.
Friday 29th May. A correspondent points out that the costings of the guidance can be found on the NICE web site. Table 1 has a strong air of make-believe.
The Daily Mail (29 May 2009). Their article quotes serious criticisms of NICE.
Dr Ron Cooper, past chairman of the group and a consultant pain specialist in Northern Ireland, said: ‘I have never known so many pain medicine specialists to be so furious. More patients will end up having more expensive surgery, which is unnecessary, risky and has worse results.
‘NICE made it difficult for us to submit evidence to a committee on which there was not one experienced pain physician.
‘The guidelines will make us the laughing stock of Europe, Australia and the U.S. where pain specialists will continue to have full access to a wide range of treatments.’
Chirowatch suspended. 30 May 2009. The invaluable chiro-watch site, http://www.chirowatch.com, run by Dr Canadian physician, Dr Tom Polevoy, has been “suspended”. In fact the whole of his domain, healthwatcher.net, has been disabled. It seems that this was the result of a legal threat to his ISP, not by chiropractors this time, but by William O’Neill of the Canadian Cancer Research Group, which is far from being what the respectable-sounding title suggests, according to Dr Stephen Barrett. The site will be back soon, I’m told. Meanwhile you can read the suspended page here, and the only result of their hamfisted bullying will be to vastly increase the number of people who read it. You can read lots more at Quackwatch’s chirobase and at ebm-first.
A new blog, Not as NICE as you think appeared on May 30th. It is written by a pain physician, Stephen Ward. and is devoted to pointing out the serious problems raised by NICE’s guidance. It starts with the (very critical) World Institute of Pain Press Release.
British Medical Journal has more flak for the NICE guidelines
And the BMJ has published a letter signed by 50 consultants in pain medicine, NICE guidelines on low back pain are flawed.(this will probably appear as a letter in the print edition of the BMJ).
First the MHRA lets down the public by allowing deceptive labelling of sugar pills (see here, and this this blog). Now it is the turn of NICE to betray its own principles.
The National Institute for Health and Clinical Excellence (NICE) describes its job thus
“NICE is an independent organisation responsible for providing national guidance on promoting good health and preventing and treating ill health.”
Its Guidance document on Low Back Pain will be published on Wednesday 27 May 2009, but the newspapers have already started to comment, presumably on the assumption that it will have changed little from the Draft Guidance of September 2008. These comments may have to be changed as soon as the final version becomes available.
The draft guidance, though mostly sensible, has two recommendations that I believe to be wrong and dangerous. The recommendations include (page 7) these three.
- Consider offering a course of manual therapy including spinal manipulation of up to 9 sessions over up to 12 weeks.
- Consider offering a course of acupuncture needling comprising up to 10 sessions over a period of up to 12 weeks.
- Consider offering a structured exercise programme tailored to the individual.
All three of this options are accompanied by a footnote that reads thus.
“A choice of any of these therapies may be offered, taking into account patient preference.”
On the face if it, this might seem quite reasonable. All three choices seem to be about as effective (or ineffective) as each other, so why not let patients choose between them?
Actually there are very good reasons, but NICE does not seem to have thought about them. In the past I have had a high opinion of NICE but it seems that even they are now getting bogged down in the morass of political correctness and officialdom that is the curse of the Department of Health. It is yet another example of DC’s rule number one.
Never trust anyone who uses the word ‘stakeholder’.
They do use it, often.
So what is so wrong?
For a start, I take it that the reference to “spinal manipulation” in the first recommendation is a rather cowardly allusion to chiropractic. Why not say so, if that’s whar you mean? Chiropractic is mentioned in the rest of the report but the word doesn’t seem to occur in the recommendations. Is NICE perhaps nervous that it would reduce the credibility of the report if the word chiropractic were said out loud?
Well, they have a point, I suppose. It would.
That aside, here’s what’s wrong.
The Evidence
I take as my premise that the evidence says that no manipulative therapy has any great advantage over the others. They are all more or less equally effective. Perhaps I should say, more or less equally ineffective, because anyone who claims to have the answer to low back pain is clearly deluded (and I should know: nobody has fixed mine yet). So for effectiveness there are no good grounds to choose between exercise, physiotherapy, acupuncture or chiropractic. There is, though, an enormous cultural difference. Acupuncture and chiropractic are firmly in the realm of alternative medicine. They both invoke all sorts of new-age nonsense for which there isn’t the slightest good evidence. That may not poison your body, but it certainly poisons your mind.
Acupuncturists talk about about “Qi”, “meridians”, “energy flows”. The fact that “sham” and “real” acupuncture consistently come out indistinguishable is surely all the evidence one needs to dismiss such nonsense. Indeed there is a small group of medical acupuncturists who do dismiss it. Most don’t. As always in irrational subjects, acupuncture is riven by internecine strife between groups who differ in the extent of their mystical tendencies,
Chiropractors talk of “subluxations”, an entirely imaginary phenomenon (but a cause of much unnecessary exposure to X-rays). Many talk of quasi-religious things like “innate energy”. And Chiropractic is even more riven by competing factions than acupuncture. See, for example, Chiropractic wars Part 3: internecine conflict.
The bait and switch trick
This is the basic trick used by ‘alternative therapists’ to gain respectability.
There is a superb essay on it by the excellent Yale neurologist Steven Novella: The Bait and Switch of Unscientific Medicine. The trick is to offer some limited and reasonable treatment (like back manipulation for low back pain). This, it seems, is sufficient to satisfy NICE. But then, once you are in the showroom, you can be exposed to all sorts of other nonsense about “subluxations” or “Qi”. Still worse, you will also be exposed to the claims of many chiropractors and acupuncturists to be able to cure all manner of conditions other than back pain. But don’t even dare to suggest that manipulation of the spine is not a cure for colic or asthma or you may find yourself sued for defamation. The shameful legal action of the British Chiropractic Association against Simon Singh (follow it here) led to an addition to DC’s Patients’ Guide to Magic Medicine.
(In the face of such tragic behaviour, one has to be able to laugh).
Libel: A very expensive remedy, to be used only when you have no evidence. Appeals to alternative practitioners because truth is irrelevant.
NICE seems to have fallen for the bait and switch trick, hook line and sinker.
The neglected consequences
Once again, we see the consequences of paying insufficient attention to the Dilemmas of Alternative Medicine.
The lying dilemma
If acupuncture is recommended we will have acupuncturists telling patients about utterly imaginary things like “Qi” and “meridians”. And we will have chiropractors telling them about subluxations and innate energy. It is my opinion that these things are simply make-believe (and that is also the view of a minority of acupuncturist and chiropractors). That means that you have to decide whether the supposed benefits of the manipulation are sufficient to counterbalance the deception of patients.
Some people might think that it was worth it (though not me). What is unforgivable is not to consider even the question. The NICE guidance says not a word about this dilemma. Why not?
The training dilemma
The training dilemma is even more serious. Once some form of alternative medicine has successfully worked the Bait and Switch trick and gained a toehold in the NHS, there will be an army of box-ticking HR zombies employed to ensure that they have been properly trained in “subluxations” or “Qi”. There will be quangos set up to issue National Occupational Standards in “subluxations” or “Qi”. Skills for Health will issue “competences” in “subluxations” or “Qi” (actually they already do). There will be courses set up to teach about “subluxations” or “Qi”, some even in ‘universities’ (there already are).
The respectability problem
But worst of all, it will become possible for aupuncturists and chiropractors to claim that they now have official government endorsement from a prestigious evidence-based organisation like NICE for “subluxations” or “Qi”. Of course this isn’t true. In fact the words “subluxations” or “Qi” are not even mentioned in the draft report. That is the root of the problem. They should have been. But omitting stuff like that is how the Bait and Switch trick works.
Alternative medicine advocates crave, above all, respectability and acceptance. It is sad that NICE seems to have given them more credibility and acceptance without having considered properly the secondary consequences of doing so,
How did this failure of NICE happen?
It seems to have been a combination of political correctness, failure to consider secondary consequences, and excessive influence of the people who stand to make money from the acceptance of alternative medicine.
Take, for example, the opinion of the British Pain Society. This organisation encompasses not just doctors. It
includes “doctors, nurses, physiotherapists, scientists, psychologists, occupational therapists and other healthcare professionals actively engaged in the diagnosis and treatment of pain and in pain research for the benefit of patients”. Nevertheless, their response to the draft guidelines pointed out that the manipulative therapies as a whole were over-represented.
|
Manipulation The guidelines assess 9 large groups of interventions of which manual therapies are only one part. The full GDG members panel of 13 individuals included two proponents of spinal manipulation/mobilisation (P Dixon and S Vogel). In addition, the chair of the panel (M Underwood) is the lead author of the UKBEAM trial on which the positive recommendation for |
It seems that the Pain Society were quite right.
LBC 97.3 Breakfast Show (25 May 2009) had a quick discussion on acupuncture (play mp3 file). After I had my say, the other side was put by Rosey Grandage. She has (among other jobs) a private acupuncture practice so she is not quite as unbiassed as me). As usual, she misrepresents the evidence by failing to distinguish between blind and non-blind studies. She also misrepresented what I said by implying that I was advocating drugs. That was not my point and I did not mention drugs (they, like all treatments, have pretty limited effectiveness, and they have side effects too). She said “there is very good evidence to show they (‘Qi’ and ‘meridians’] exist”. That is simply untrue.
There can’t be a better demonstration of the consequences of falling for bait and switch than the defence mounted by Rosey Grandage. NICE may not mention “Qi” and “meridians”; but the people they want to allow into the NHS have no such compunctions.
I first came across Rosey Grandage when I discovered her contribution to the Open University/BBC course K221. That has been dealt with elsewhere. A lot more information about acupuncture has appeared since then. She doesn’t seem to have noticed it. Has she not seen the Nordic Cochrane Centre report? Nor read Barker Bausell, or Singh & Ernst? Has she any interest in evidence that might reduce her income? Probably not.
Where to find out more
An excellent review of chiropractic can be found at the Layscience site. It was written by the indefatigable ‘Blue Wode’ who has provided enormous amounts of information at the admirable ebm-first site (I am authorised to reveal that ‘Blue Wode’ is the author of that site). There you will also find much fascinating information about both acupuncture and about chiropractic.
I’m grateful to ‘Blue Wode’ for some of the references used here.
Follow-up
It is almost six months now since I posted Quackery creeps into good universities too -but through Human Resources. One example given there was the University of Leicester. This is an excellent university. It does first class research and it was the alma mater of the incomparable David Attenborough who has done more than anyone to show us the true beauty and wonder of the natural world.
Nevertheless, their well-meaning occupational health department had a section about “complementary therapies” that contained a lot of statements that were demonstrably untrue. They even recommended the utterly outrageous SCENAR device. So I pointed this out to them, and I had a quick and sympathetic response from their HR director.
But three months later, nothing had changed. Every now and then, I’d send a polite reminder, but it seemed the occupational health staff were very wedded to their quackery. The last reminder went on 6th February, but this time I copied it to Leicester’s vice-chancellor. This time it worked. There is still a link to Complementary Therapies on the Wellbeing site, but if you click on it, this is what you see.
Complementary therapiesSome employees may have an interest in complementary therapies such as acupuncture, yoga, Indian head massage, Reiki, sports & remedial massage, reflexology and hypnotherapy. If you have an interest in any of these, Staff Counselling can happily provide details of practitioners in the local area. Some of these practitioners offer discounts from their normal rates for University of Leicester staff. However, the University of Leicester cannot vouch for, or recommend any of these therapies to staff as being effective. We would urge members of staff who believe that such therapies might be effective to contact their GP prior to undertaking any of them. Further, the University of Leicester shall not be liable for any damage of any kind arising out of or related to the services of any complementary therapists or treatments listed here. If you would like further information, please contact Chris Wilson at: staffcounsel&welfare@le.ac.uk or telephone 1702. |
That’s not bad. Pity it doesn’t say alternative, rather than complementary though. Euphemisms aren’t really helpful.
In fact I have a bit of a problem with “wellbeing” too. It is a harmless word that has been highjacked so that its use now makes one think of mud baths provided by expensive hotels for their rich and gullible customers.
Leicester’s HR director wrote
“Unfortunately an instruction I had given previously had not been fully complied with. I spoke to the manager of the Staff Counselling team on Friday and gave clear instructions as to the content of this site. I had been assured that the offending information had been removed, but found that it had not.
I have now checked the site for myself and can say, with confidence, that all claims for the efficacy of complimentary [sic] therapies have been removed including SCENAR.”
The similarity between quack treatments and religion is intriguing. It seems that the devotion of the occupational health people to their baloney was so great that they wouldn’t take it down even when told to do so. The more irrational the belief, the greater the fervour with which it is defended,
What’s the lesson from this minor saga? It seems that most VCs and many HR people are too sensible to believe in alternative baloney, but that they are a bit too ready to tolerate it, perhaps on grounds of political correctness. Tolerance is a virtue, but lies about health are not in the least virtuous. If you point out that people are saying things for which there isn’t the slightest evidence, they will often respond. Just be prepared to send a few reminders.
It may also be useful to point out that some of the claims made are almost certainly illegal. Even people who care little about evidence of efficacy are impressed by the idea that they might be prosecuted by Trading Standards officers.
Who needs mystical medicine when you have real wonders like these.
(Click the google video logo for a bigger version)
Or try the pitcher plant video, or the Bird of paradise, or the Bower Bird, or the giant Amazon lilies.
Follow-up
We know all about the sixteen or so universities that run “BSc” degrees in hokum. They are all “post-1992” universities, which used to be polytechnics. That is one reason why it saddens me to see them destroying their own attempts to achieve parity with older universities by running courses that I would regard as plain dishonest.
Older universities do not run degree courses in such nonsense. Academics (insofar as they still have any influence) certainly would not put up with it if they tried. But nevertheless you can find quackery in some of the most respected universities, and it gets there not via academics but (guess what) via Human Resources. It creeps in through two routes. One is the “training courses” that research staff now have to do (the “Roberts agenda”). The other route is through occupational health services.
Quackery in training courses
It isn’t easy to find out what happens elsewhere, but I was certainly surprised to find out that UCL’s own HR department was offering a course that promised to teach you the “core principles” of Brain Gym and Neurolinguistic Programming, both totally discredited bits of psycho-babble, more appropriate to the lifestyle section of a downmarket.women’s magazine than a university. I gather that HR’s reaction after I brought this to light was not to ask what was wrong with it, but just to get angry.
In a spirit of collegiality I offered to run a transferable skills course myself. I even offered to do it for nothing (rather than the rumoured £700 per day charged by the life style consultants). I proposed a course in ‘How to read critically’ (subtitle ‘How to detect bullshit’). This seems to me to be the ultimate transferable skill. Bullshit occurs in every walk of life. My proposal was moderately worded and perfectly serious.
Guess what? Despite several reminders, I have never had any response to my suggestion. Well, I suppose that HR people now regard themselves as senior to mere professors and there is really no need to reply to their
letters.
Quackery in occupational health. Leicester sets a good example
If you work at a university, why not search the university’s web site for “complementary medicice” or complementary therapies”. If it is a real university, you won’t find any degrees in homeopathy, or in amethysts
that emit high yin energy. But some quite surprising places are found to be recommending magic medicine through their Occupational Health service, which usually seems to be part of HR. In fact at one time even UCL was doing it, but no soon had somebody sent me the link than it disappeared. As a matter of historical record, you can see it here (it had all the usual junk, as well as harmless stuff like yoga and pilates).
While looking for something else I stumbled recently some other cases. One was at the University of Leicester, a very good university (and alma mater to the great David Attenborough who must have done more to point out the beauty of science than just about anyone). But we find on their staff wellbeing site, alongside some perfectly sensible stuff, a link to complementary therapies.
The list of ‘therapies’ includes not only the usual placebos, acupuncture, reiki, reflexology, but, even more exotically, a fraudulent Russian device called SCENAR therapy. They have a nice leaflet that explains all these things in words that run the whole gamut from meaningless gobbledygook to plain wrong. Here are some examples from the leaflet.
Reflexology
“In the feet, there are reflex areas corresponding to all the parts of the body and these areas are arranged in such a way as to form a map of the body in the feet”
Reflexology has been shown to be effective for:
- Back Pain
- Migraine
- Infertility
- Arthritis
Well no, there are no such areas in your feet. That is sheer imagination. And reflexology has not “been shown to be effective” for any of those conditions. These claims for therapeutic efficacy are not only lies. They are also illegal.
“Reiki
Each hand position is held for a few minutes, and during this time healing energy will flow into you, balancing your energy system, releasing stress, soothing pain, and promoting your body’s natural ability to heal itself.”
This is sheer idiotic mumbo-jumbo. The “flow of healing energy” is totally imaginary. Such talk is offensive to anyone with half a brain. Insofar as they claim to heal anything, it is also illegal. The comes SCENAR.
“What is SCENAR?
SCENAR is an acronym for Self Controlled Energo- Neuro Adaptive Regulator. It is a reflex biofeedback device which when used by a qualified practitioner, can help to alleviate acute and chronic pain. It is licensed in the UK for pain relief but experience has shown that it is helpful in a wide variety of conditions.”
This is even more seriously nuts than the others. The term “licensed” means merely that it is electrically safe. It certainly does not mean that it works. Pubmed shows only three publications about the SCENAR device, all in Russian,
One sales site (apparently Russian) makes the following modest claim.
“A prime goal of the Russian Space Program was to provide space travelers with a portable medical device that would become their “universal medical assistant” in space. So from the beginning, the SCENAR was designed to replace an entire medical hospital, with all its staff, diagnostic and treatment facilities, even the pharmacy. A universal, non-invasive, portable regulator of body functions (among other things) was envisaged.”
The SCENAR device (right) looks like a TV remote control (perhaps it IS a TV remote control -we aren’t anywhere told in comprehensible terms what’s in the box. The Russian site sells also the rather baffling accessory on the right. The mind boggles.
SCENAR device |
Remote rectal-vaginal electrode for SCENAR |
How does this rubbish get onto the web site of a good university?
I presume that it is just another sign of what happens when universities come to be run by non-academics. No doubt the occupational health people are well meaning and kind, but just scientifically illiterate. What about the HR person in charge of them? They are not known for scientific literacy either (which would not matter if they stuck to their job). But perhaps they just didn’t notice. There is only one way to find out. Ask. So I sent this letter.on 10th September.
| Hello
I am a pharmacologist and I have a side interest in public understanding of science, alternative medicine.and medical fraud I was quite surprised when Google led me unexpectedly to your complementary therapies page at http://www.le.ac.uk/staffwellbeing/complementary_therapy.html There is, sad to say, a great deal of information on these pages that is simply not true. For example it has NOT been shown that reflexology has been shown to be effective in any of the conditions which you list, as far as I know To take only one more example from this page, the SCENAR device is an even more extreme example. It is well known to be fraudulent. and has been investigated by the Washington State Attorney General. This sort of thing is not what one would expect from a very respectable university, and it must be a great embarrassment to your excellent medical scientists. Apart from the many scientific inaccuracies (which greatly impede the efforts of those of us who try to improve public understanding of science), you are, I hope, aware that there is a legal aspect. Since May this year, new regulations have made it illegal to make claims for health benefits if evidence cannot be produced to show that the claims are justified. I would like to put it to you that many of the claims made on this page are not only immoral, but also illegal. I wondered whether you , or your HR department, would like to make any comments Best regards David Colquhoun |
I got an immediate and very sympathetic response from the Director of HR and a week later, on17th September, he wrote
“Hi David,
I have discussed the matter with my manager of Staff Counselling and Welfare and have agreed that it is probably safest that we remove the references to ‘complimentary’[sic] therapies from the site entirely.
Thank you for your helpful input and the recommendations for reading matter.”
So there is a lesson here. If you find this sort or stuff on your own institution’s web site, all that may be needed is a simple letter that points out what nonsense it is. Admittedly the HR man seemed rather more worried about whether the claims were illegal than whether they were true, but either way, it worked.
Only one little snag. As of 6 October the pages still have not been removed.
On the assumption that they eventually will be removed, I have kept copies of the Wellbeing page, of the Complementary Therapies page, and of the ‘explanatory leaflet’. They stand as part of a historical record that
shows, once again, what can happen when scientific matters get into the hands of HR. Fortunately Leicester University has an HR director who is willing to listen to advice.
Follow-up
Something seems to have gone seriously wrong. Despite the rapid response, virtually all the nonsense is still there on 13th October. It seems not to be so simple after all.
And despite several reminders, the advertisement for SCENAR ‘therapy’ is still on the University web site on December 14th. I know that no decision by HR can be made with fewer than 25 meetings and an awayday in Majorca, but this is getting ridiculous.
There have been some really excellent books about quackery this year. This isn’t one of them, because
 |
Nice dedication uh? |
it is about a lot more than quackery It is about the scientific method in general. and in particular about how often it is misunderstood by journalists. Abuse of evidence by the pharmaceutical industry is treated just as harshly as abuse of evidence by homeopaths and you get the low-down on both.
“More importantly, you will also see how a health myth can be created, fostered and maintained by the alternative medicine industry using all the same tricks on you, the public, which big pharma uses on doctors. This is about something much bigger than homeopathy.” (p.28)
Sir Iain Chalmers, a founder of the Cochrane Collaboration , co-author of the best lay text on evidence says: “Bad Science introduces the basic scientific principles to help everyone become a more effective bullshit detector”. And there is no more invaluable skill than being a bullshit detector.
Chalmers says also “Ben Goldacre has succeeded where the ‘public engagement in science’ organisations have so signally failed.” That is exactly right. ‘Public engagement’ has rapidly become bureaucratised, and at its worst, is no better than a branch of the university’s marketing department. This sort of public engagement corrupts as much as it enlightens. Goldacre enlightens, and he also makes you laugh.
In the introduction, Goldacre says
“You cannot reason people out of positions that they didn’t reason themselves into.” (p xii)
It’s a nice point, but the rest of the book makes a magnificent attempt to do just that.
There is quite a lot about medicine, of course, that’s his job, after all. But it isn’t all quackery by a long chalk Quackery is merely a good hook to hang the arguments on about how you distinguish what’s true from what isn’t. That’s partly because quacks make every mistake known to mankind (sometimes through ignorance, sometimes just to boost sales), and partly just because it is a topic that interests people, and with which they are bombarded every day I feel exactly the same. If I were to talk about the statistics of single ion channels, nobody would read it (big mistake -it’s fascinating), but if one can use the case of honey versus cough medicine to explain the analysis of variance, there is a chance that someone might find it interesting.
As much as anything, Goldacre’s book is about C.P. Snow’s two cultures. The chapters on the distortion and trivialisation of science in the media are just terrific.
“My basic hypothesis is this: the people who run the media are humanities graduates with little understanding of science, who wear their ignorance as a badge of honour. Secretly, deep down, perhaps they resent the fact that they have denied themselves access to the most significant developments in the history of Western thought from the past two hundred years.” Chapter 11, p. 207
“.. . . here is the information I would like from a newspaper to help me make decisions about my health, when reporting on a risk: I want to know who you’re talking about (e.g. men in their fifties): I want to know what the baseline risk is (e.g. four men out of a hundred will have a heart attack over ten years); and I want to know what the increase in risk is , as a natural frequency [not as relative risk] (two extra men out of that hundred will have a heart attack over ten years). I also want to know exactly what’s causing that increase in risk -an occasional headache pill or a daily tub full of pain-relieving medication for arthritis. Then I will consider reading your newspapers again, instead of blogs which are written by people who understand research , and which link reliably back to the original academic paper, so that I can double check their précis when I wish. ” (p. 242)
I detect some ambiguity in references to things that aren’t true. Sometimes there is magnanimity. At other times he is a grade one kick-ass ninja. For example
I can very happily view posh cosmetics -and other forms of quackery -as a special, self-administered, voluntary tax on people who don’t understand science properly (p. 26)
Of course nobody wants to ban cosmetics, or even homeopathy. But a lot of bad consequences flow from being over-tolerant of lies if you take it too far (he doesn’t). The lying dilemma and the training dilemma are among them. Some unthinking doctors will refer troublesome patients to a reflexologist. That gets the worried-well out of their surgery but neglects the inevitable consequence that Human Resources box-ticking zombies will then insist on having courses that teach the big toe is connected to the kidney (or whatever) so that reflexologists can have an official qualification in mystical mumbo-jumbo.
Is there anything missing from the book? Well inevitably. There are plenty of villains among the peddlers of nutri-bollocks, and in the media. But there isn’t much about the people who seem to me to be in some ways even worse. What about the black-suited men and women in the Ministry of Health and in some vice-chancellors’ chairs who betray their institutions and betray the public through some unfathomable
mixture of political correctness, scientific ignorance and greed? What about the ludicrous behaviour of quangos like Skills for Health? You have to wait right to the end of the book to hear about universities. But when it comes, it is well worth the wait.
“I’m not surprised that there are people with odd ideas about medicine, or that they sell those ideas. But I am spectacularly, supremely, incandescently unimpressed when a a university starts to offer BSc science courses in them.” (p. 317)
It’s almost worth buying Ben Goldacre’s book for that sentence alone.
This book is a romp through the folly, greed and above all the ignorance of much in our society. It’s deeply educational. And it makes you laugh. What more could you want?
| A report has appeared on Regulation of Practitioners of Acupuncture, Herbal Medicine, Traditional Chinese Medicine. The report is written by people all of whom have vested interests in spreading quackery. It shows an execrable ability to assess evidence, and it advocates degrees in antiscience It would fail any examination. Sorry, Prof Pittilo, but it’s gamma minus.[Download the report] |  |
Alice Miles put it well in The Times, today.
“This week came the publication of the “Report to Ministers from the Department of Health Steering Group on the Statutory Regulation of Practitioners of Acupuncture, Herbal Medicine, Traditional Chinese Medicine and Other Traditional Medicine Systems Practised in the UK”. Otherwise known as twaddle.” . . .
“Regulate the practitioners – for safety, note, not for efficacy, as that is impossible to prove – and you give them official recognition. From recognition it is but a short hop to demand and then prescription: packet of Prozac, bit of yoga and a bag of dodgy herbs for you, sir.” . . .
“The Government responded on Monday – with a three-month consultation. So join in. Write to the Health Minister Ben Bradshaw at Richmond House, 79 Whitehall, SW1A 2NS. Write, on behalf of the NHS: “What I want for my 60th birthday is… the chance to provide medical, dental, and nursing care to all. And absolutely nothing else.”
Judging by Ben Bradshaw’s speech to the Prince’s Foundation, there may be a problem in conveying to him the evidence, but one can and must try.
Why is it that a health joutnalist can do so much better than a university head? Yes, the chair of the steering group is Professor R. Michael Pittilo BSc PhD CBiol FIBiol FIBMS FRSH FLS FRSA, Principal and Vice-Chancellor of The Robert Gordon University, Aberdeen. Despite all those impressive-lookin initials after his name, I believe that this is a very bad report.
Here is something about Prof Pittilo from his university’s web site (the emphasis is mine).
Professor Michael Pittilo joined The Robert Gordon University, Aberdeen, as Principal and Vice-Chancellor on 5th September, 2005.
After postdoctoral research on arterial disease at the University of London, he was appointed to Kingston University where he became Head of Life Sciences. In 1995 he became Foundation Dean of the Faculty of Health and Social Care Sciences at Kingston University and St George’s Medical School (University of London). He was appointed Pro Vice Chancellor at the University of Hertfordshire in 2001.
Professor Pittilo has held a number of additional roles, including chairing Department of Health working groups, and as a trustee for the Prince of Wales’s Foundation for Integrated Health. “
Notice that Prof Pittilo is a Trustee of the Prince’s Foundation for Integrated Health, source of some of the least reliable information about alternative medicine to be found anywhere.
This steering group is, as so often, a nest of vested interests. It does not seem to have on it any regular medical or clinical scientist whatsoever. Why not? They just might produce some embarrassing facts perhaps? Like most government committees its members seem to have been chosen to produce the desired outcome.
For a start, the university run by Prof Pittilo, Robert Gordon’s University, is itself involved in a few antiscientific courses. Since his report recommends that degrees in quackery should become mandatory, I expect he’d welcome the chance to run more. Amazingly, Robert Gordon’s University runs an Introduction to Homeopathy, just about the daftest of all the common sorts of magic medicine.
Most of the the members of the steering group represent vested interests, though strangely this is not made clear in the list of members. An earlier report, in 2006, from the steering group was more open about this. Twelve of the members of the group represent Herbal Medicine, Chinese Medicine and Acupuncture (four from each). Most of the rest are lay members or bureaucrats. With membership like that it is, I suppose, not surprising that the assessment of evidence is, to put it kindly, grossly distorted and woefully inadequate.
The report starts badly by failing to mention that the House of Lords report (2000), and the government’s response to it, set the following priorities. Both state clearly
“… we recommend that three important questions should be addressed in the following order . .
- (1) does the treatment offer therapeutic benefits greater than placebo?
- (2) is the treatment safe?
- (3) how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?
The word ‘placebo’ does not occur a single time in the main report (and only twice in the text of the seven appendices). But they do say (page 11):
“We recommend that public funding from the NHS should be used to fund CAM therapies where there is evidence of efficacy, safety and quality assurance.”
The evidence
The problem is that the assessment of the evidence for efficacy in the report is pathetically poor. The report, sad to say, consists essentially of 161 pages of special pleading by the alternative medicine industry, served up with the usual large dose of HR gobbledygook.
There is really no excuse for this utterly incompetent assessment. There have been plenty of books this year alone that make excellent summaries of the evidence, mostly written for the lay public. They should, therefore, be understandable by any university vice-chancellor (president). The one benefit of the upsurge in public interest in magic medicine is that there are now quite a lot of good clinical trials, and when the trials are done properly, they mostly confirm what we thought before: in most cases the effects are no more than placebo.
Here is one example. Annexe1 concerns “Developing Research and Providing an Evidence Base for Acupuncture and Herbal/Traditional Medicine Treatment”. The wording of the title itself suggests, rightly, that this evidence base does not exist, in which case why on earth are we talking about them as “professions”? The discussion of the evidence in Annexe 1 is nothing if not partial. But what do you expect if you ask herbalists to assess herbal medicine? An honest assessment would put them out of business. The eternal mantra of the alternative industry appears as usual, “Absence of evidence is, of course, not evidence of absence”. True of course, but utterly irrelevant. Annexe 1 says
“Acupuncture is a complex intervention and lack of a suitable placebo control has hindered efforts to evaluate efficacy”
This is simply untrue, In recent years enormous efforts have been put into devising controls for assessment of acupuncture, but they are entirely ignored here. One thing that has been established quite clearly is that it makes no difference where you put the needles, so all the talk of Qi and meridians is obvious mumbo-jumbo.
Have the authors of Annexe 1, and Professor Pittilo, not read the relevant studies? Two books this year have dealt with the question of evidence with great care. They are both by people who have been involved personally with acupuncture research, Prof Edzard Ernst and Dr Barker Bausell. Edzard Ernst is the UK’s first Professor of Complementary Medicine. Barker Bausell was research director of an NIH-funded Complementary and Alternative Medicine Specialized Research Center at the University of Maryland.
 |
 |
Singh and Ernst discuss thoroughly the question of controls and assess all the evidence carefully. Their conclusions include the following.
- The traditional principles of acupuncture are deeply flawed, as there is no evidence at all to demonstrate the existence of Ch’i [Qi] or meridians.
- By focussing on the increasing number of high-quality research papers, reliable conclusions from systematic reviews make it clear that acupuncture does not work for a whole range of conditions, except as a placebo.
- In short, the evidence is neither consistent nor convincing. It is borderline.
Barker Bausell was himself involved in designing and analysing trialsof acupuncture. His conclusions are even less positive.
“There is no compelling, credible scientific evidence to suggest that any CAM therapy benefits any medical condition or reduces any medical symptom (pain or otherwise) better than a placebo”.
These are serious authors with direct experience in CAM research, which is more than can be said of anyone on the steering group. Why are their conclusions ignored entirely? That is sheer incompetence.
Degrees in anti-science
One conclusion of the report is that
“The threshold entry route to the register will normally be through a Bachelor degree with Honours”
This is utter nonsense. It is quite obvious surely that you can’t award honours degrees until after you have the evidence. You can read on page 55 of the report
“3a: Registrant acupuncturists must:
understand the following aspects and concepts for traditional East-Asian acupuncture:
– yin/yang, /5 elements/phases, eight principles, cyclical rhythms, qi ,blood and body fluids, different levels of qi, pathogenic factors, 12 zang fu and 6 extraordinary fu, jing luo/ meridians, the major acupuncture points, East-Asian medicine disease categorisation, the three burners, the 4 stages/levels and 6 divisions
– causes of disharmony/disease causation
– the four traditional diagnostic methods: questioning, palpation, listening and observing”
This is utter baloney. Anyone who advocates giving honours degrees in such nonsense deserves to be fired for bringing his university into disrepute (and, in the process, bringing all universities and science itself into disrepute).
That includes also degrees that teach that “amethysts emit high yin energy“.
So what should be done?
If making peole do degrees in mumbo-jumbo is not the answer, what is? Clearly it would be far too draconian to try to ban quackery (and it would only increase its popularity anyway).
The answer seems to me to be quite simple. All that needs to done is to enforce existing laws. It is already illegal to sell contaminated and poisonous goods to the public. It is already illegal to make fraudulent advertisemants and to sell goods that are not as described on the label.
The only problem is that the agencies that enforce these rules are toothless and that there are a lot of loopholes and exceptions that work in favour of quackery. I have tried myself to complain about mislabelling of homeopathic pills to the Office of Fair Trading on the grounds that are labelled Arnica 30C but contain no Arnica. They solemnly bought a bottle and sent it to an analyst and of course they found no arnica, But nothing happened, because an exception to the usual law applies to homeopathic pills.
The Advertising Standards Authority is good as far as it goes. They quickly told Boots Pharmacies to withdraw advertisements that claimed CoQ10 “increased vitality”. But they can exact no penalties and they can’t deal with lies that are told to you orally, or with anything at all on the web.
The Health Professions Council (HPC) says that one of the criteria for registering new professions is aspirant groups must “Practise based on evidence of efficacy”. If that were actually applied, none of this process would occur anyway. No doubt the HPC will fail to apply its own criteria. On past form, it can be expected to adopt a “fluid concept of evidence“.,
One more thing, New European legislation was described recently in the BMJ
“Consumers in the United Kingdom are to receive stronger legal safeguards against products that claim, without any identifiable scientific evidence, to provide physical and mental health benefits such as tackling obesity or depression.”
“The scope of the legislation is deliberately wide and is the biggest shake up in consumer law for decades. It targets any unfair selling to consumers by any business.”
Politicians seem to be immune to rational argument when it comes to quackery. But a few legal actions under these laws could bring the house of cards tumbling so fast that this gamma-minus report would become rapidly irrelevant. There will be no shortage of people to bring the actions. I can’t wait.
Follow up
Dominic Lawson, 24 June 2008. An excellent column appeared today in the Independent. Dominic Lawson writes about the Pittilo report: “So now we will have degrees in quackery. What, really, is the difference between acupuncture and psychic surgery?“. The reference to that well known conjuring trick, “psychic surgery” as a “profession”, revealed here, causes Lawson to say
“It makes it clear that the lunatics have taken over the asylum. For a start, how could Philip Hunt, previously director of the National Association of Health Authorities and Trusts, possibly have thought that “psychic healing” constituted a “profession” – let alone one which would “develop its own system of voluntary self-regulation?”
“One can see how this might fit in with the Government’s “never mind the quality, feel the width” approach to university education. One can also see how established practitioners of such therapies might see this as a future source of income – how pleasant it might be to become Visiting Professor of Vibrational Medicine at the University of Westminster.
Thus garlanded with the laurels of academic pseudo-science, the newly professionalised practitioners of “alternative medicine” can look down on such riff-raff as the “psychic surgeons”
Once again I have to ask, how is it that we have to rely on journalists to prevent vice-chancellors eroding academic standards; indeed eroding simple common sense? I guess it is just another sign of the delusional thinking engendered by the culture of managerialism that grips universities.
This is a fuller version, with links, of the comment piece published in Times Higher Education on 10 April 2008. Download newspaper version here.
If you still have any doubt about the problems of directed research, look at the trenchant editorial in Nature (3 April, 2008. Look also at the editorial in Science by Bruce Alberts. The UK’s establishment is busy pushing an agenda that is already fading in the USA.
Since this went to press, more sense about “Brain Gym” has appeared. First Jeremy Paxman had a good go on Newsnight. Skeptobot has posted links to the videos of the broadcast, which have now appeared on YouTube.
Then, in the Education Guardian, Charlie Brooker started his article about “Brain Gym” thus
Dr Aust’s cogent comments are at “Brain Gym” loses its trousers. |
 |
The Times Higher’s subeditor removed my snappy title and substituted this.
So here it is.
“HR is like many parts of modern businesses: a simple expense, and a burden on the backs of the productive workers”, “They don’t sell or produce: they consume. They are the amorphous support services” .
So wrote Luke Johnson recently in the Financial Times. He went on, “Training advisers are employed to distract everyone from doing their job with pointless courses”. Luke Johnson is no woolly-minded professor. He is in the Times’ Power 100 list, he organised the acquisition of PizzaExpress before he turned 30 and he now runs Channel 4 TV.
Why is it that Human Resources (you know, the folks we used to call Personnel) have acquired such a bad public image? It is not only in universities that this has happened. It seems to be universal, and worldwide. Well here are a few reasons.
Like most groups of people, HR is intent on expanding its power and status. That is precisely why they changed their name from Personnel to HR. As Personnel Managers they were seen as a service, and even, heaven forbid, on the side of the employees. As Human Resources they become part of the senior management team, and see themselves not as providing a service, but as managing people. My concern is the effect that change is having on science, but it seems that the effects on pizza sales are not greatly different.
The problem with having HR people (or lawyers, or any other non-scientists) managing science is simple. They have no idea how it works. They seem to think that every activity
can be run as though it was Wal-Mart That idea is old-fashioned even in management circles. Good employers have hit on the bright idea that people work best when they are not constantly harassed and when they feel that they are assessed fairly. If the best people don’t feel that, they just leave at the first opportunity. That is why the culture of managerialism and audit. though rampant, will do harm in the end to any university that embraces it.
As it happens, there was a good example this week of the damage that can be inflicted on intellectual standards by the HR mentality. As a research assistant, I was sent the Human Resources Division Staff Development and Training booklet. Some of the courses they run are quite reasonable. Others amount to little more than the promotion of quackery. Here are three examples. We are offered a courses in “Self-hypnosis”, in “Innovations for Researchers” and in “Communication and Learning: Recent Theories and Methodologies”. What’s wrong with them?
“Self-hypnosis” seems to be nothing more than a pretentious word for relaxation. The person who is teaching researchers to innovate left science straight after his PhD and then did courses in “neurolinguistic programming” and life-coaching (the Carole Caplin of academia perhaps?). How that qualifies him to teach scientists to be innovative in research may not be obvious.
The third course teaches, among other things, the “core principles” of neurolinguistic programming, the Sedona method (“Your key to lasting happiness, success, peace and well-being”), and, wait for it, Brain Gym. This booklet arrived within a day or two of Ben
Goldacre’s spectacular demolition of Brain Gym “Nonsense dressed up as neuroscience”
“Brain Gym is a set of perfectly good fun exercise break ideas for kids, which costs a packet and comes attached to a bizarre and entirely bogus pseudoscientific explanatory framework”
“This ridiculousness comes at very great cost, paid for by you, the taxpayer, in thousands of state schools. It is peddled directly to your children by their credulous and apparently moronic teachers”
And now, it seems, peddled to your researchers by your credulous and
moronic HR department.
Neurolinguistic programming is an equally discredited form of psycho-babble, the dubious status of which was highlighted in a Beyerstein’s 1995 review, from Simon Fraser University.
“ Pop-psychology. The human potential movement and the fringe areas of psychotherapy also harbor a number of other scientifically questionable panaceas. Among these are Scientology, Neurolinguistic Programming, Re-birthing and Primal Scream Therapy which have never provided a scientifically acceptable rationale or evidence to support their therapeutic claims.”
The intellectual standards for many of the training courses that are inflicted on young researchers seem to be roughly on a par with the self-help pages of a downmarket women’s magazine. It is the Norman Vincent Peale approach to education. Uhuh, sorry, not education, but training. Michael O’Donnell defined Education as “Elitist activity. Cost ineffective. Unpopular with Grey Suits . Now largely replaced by Training .”
In the UK most good universities have stayed fairly free of quackery (the exceptions being the sixteen post-1992 universities that give BSc degrees in things like homeopathy). But now it is creeping in though the back door of credulous HR departments. Admittedly UCL Hospitals Trust recently advertised for spiritual healers, but that is the NHS not a university. The job specification form for spiritual healers was, it’s true, a pretty good example of the HR box-ticking mentality. You are in as long as you could tick the box to say that you have a “Full National Federation of Spiritual Healer certificate. or a full Reiki Master qualification, and two years post certificate experience”. To the HR mentality, it doesn’t matter a damn if you have a certificate in balderdash, as long as you have the piece of paper. How would they know the difference?
A lot of the pressure for this sort of nonsense comes, sadly, from a government that is obsessed with measuring the unmeasurable. Again, real management people have already worked this out. The management editor of the Guardian, said
“What happens when bad measures drive out good is strikingly described in an article in the current Economic Journal. Investigating the effects of competition in the NHS, Carol Propper and her colleagues made an extraordinary discovery. Under competition, hospitals improved their patient waiting times. At the same time, the death-rate e emergency heart-attack admissions substantially increased.”
Two new government initiatives provide beautiful examples of the HR mentality in action, They are Skills for Health, and the recently-created Complementary and Natural Healthcare Council.(already dubbed OfQuack).
The purpose of the Natural Healthcare Council .seems to be to implement a box-ticking exercise that will have the effect of giving a government stamp of approval to treatments that don’t work. Polly Toynbee summed it up when she wrote about “ Quackery
and superstition – available soon on the NHS “ . The advertisement for its CEO has already appeared, It says that main function of the new body will be to enhance public protection and confidence in the use of complementary therapists. Shouldn’t it be decreasing confidence in quacks, not increasing it? But, disgracefully, they will pay no attention at all to whether the treatments work. And the advertisement refers you to
the Prince of Wales’ Foundation for Integrated Health for more information (hang on, aren’t we supposed to have a constitutional monarchy?).
Skills for Health, or rather that unofficial branch of government, the Prince of Wales’ Foundation, had been busy making ‘competences’ for distant healing, with a helpful bulletted list.
“This workforce competence is applicable to:
- healing in the presence of the client
- distant healing in contact with the client
- distant healing not in contact with the client”
And they have done the same for homeopathy and its kindred delusions. The one thing they never consider is whether they are writing ‘competences’ in talking gobbledygook. When I phoned them to try to find out who was writing this stuff (they wouldn’t say), I made a passing joke about writing competences in talking to trees. The answer came back, in all seriousness,
“You’d have to talk to LANTRA, the land-based organisation for that”,
“LANTRA which is the sector council for the land-based industries uh, sector, not with us sorry . . . areas such as horticulture etc.”.
Anyone for competences in sense of humour studies?
The “unrepentant capitalist” Luke Johnson, in the FT, said
“I have radically downsized HR in several companies I have run, and business has gone all the better for it.”
Now there’s a thought.
The follow-up
The provost’s newletter for 24th June 2008 could just be a delayed reaction to this piece? For no obvious reason, it starts thus.
“(1) what’s management about?
Human resources often gets a bad name in universities, because as academics we seem to sense instinctively that management isn’t for us. We are autonomous lone scholars who work hours well beyond those expected, inspired more by intellectual curiosity than by objectives and targets. Yet a world-class institution like UCL obviously requires high quality management, a theme that I reflect on whenever I chair the Human Resources Policy Committee, or speak at one of the regular meetings to welcome new staff to UCL. The competition is tough, and resources are scarce, so they need to be efficiently used. The drive for better management isn’t simply a preoccupation of some distant UCL bureaucracy, but an important responsibility for all of us. UCL is a single institution, not a series of fiefdoms; each of us contributes to the academic mission and good management permeates everything we do. I despair at times when quite unnecessary functional breakdowns are brought to my attention, sometimes even leading to proceedings in the Employment Tribunal, when it is clear that early and professional management could have stopped the rot from setting in years before. UCL has long been a leader in providing all newly appointed heads of department with special training in management, and the results have been impressive. There is, to say the least, a close correlation between high performing departments and the quality of their academic leadership. At its best, the ethos of UCL lies in working hard but also in working smart; in understanding that UCL is a world-class institution and not the place for a comfortable existence free from stretch and challenge; yet also a good place for highly-motivated people who are also smart about getting the work-life balance right.”
I don’t know quite what to make of this. Is it really a defence of the Brain Gym mentality?
Of course everyone wants good management. That’s obvious, and we really don’t need a condescending lecture about it. The interesting question is whether we are getting it.
There is nothing one can really object to in this lecture, apart from the stunning post hoc ergo propter hoc fallacy implicit in “UCL has long been a leader in providing all newly appointed heads of department with special training in management, and the results have been impressive.”. That’s worthy of a nutritional therapist.
Before I started writing this response at 08.25 I had already got an email from a talented and hard-working senior postdoc. “Let’s start our beautiful working day with this charging thought of the week:”.
He was obviously rather insulted at the suggestion that it was necessary to lecture academics with words like ” not the place for a comfortable existence free from stretch and challenge; yet also a good place for highly-motivated people who are also smart about getting the work-life balance right.”. I suppose nobody had thought of that until HR wrote it down in a “competence”?
To provoke this sort of reaction in our most talented young scientists could, arguably, be regarded as unfortunate.
I don’t blame the postdoc for feeling a bit insulted by this little homily.
So do I.
Now back to science.
This is an old joke which can be found in many places on the web, with minor variations. I came across it in an article by Gustav Born in 2002 (BIF Futura, 17, 78 – 86) and reproduce what he said. It has never been more relevant, so it’s well worth repeating. The title of the article was British medical education and research in the new century.
“The other deleterious development in UK research is increased bureaucratic control. Bureaucracy is notoriously bad for all creative activities. The story is told of a company chairman who was given a ticket to a concert in which Schubert’s Unfinished Symphony was to be played. Unable to go himself, he passed the ticket on to his colleague, the director in charge of administration and personnel. The next day the chairman asked, ‘Did you enjoy the concert?’ His colleague replied, ‘My report will be on your desk this afternoon’. This puzzled the chairman, who later received the following:
Report on attendance at a musical concert dated 14 November 1989. Item 3.
Schubert’s Unfinished Symphony.
- The attendance of the orchestra conductor is unnecessary for public performance. The orchestra has obviously practiced and has the prior authorization from the conductor to play the symphony at a predetermined level of quality. Considerable money could be saved by merely having the conductor critique the performance during a retrospective peer-review meeting.
- For considerable periods, the four oboe players had nothing to do. Their numbers should be reduced, and their work spread over the whole orchestra, thus eliminating the peaks and valleys of activity.
- All twelve violins were playing identical notes with identical motions. This was unnecessary duplication. If a larger volume is required, this could be obtained through electronic amplification which has reached very high levels of reproductive quality.
- Much effort was expended in playing sixteenth notes, or semi-quavers. This seems to me an excessive refinement, as listeners are unable to distinguish such rapid playing. It is recommended that all notes be rounded up to the nearest semi-quaver. If this were done, it would be possible to use trainees and lower-grade operatives more extensively.
- No useful purpose would appear to be served by repeating with horns the same passage that has already been handled by the strings. If all such redundant passages were eliminated, as determined by a utilization committee, the concert could have been reduced from two hours to twenty minutes with great savings in salaries and overhead.
- In fact, if Schubert had attended to these matters on a cost containment basis, he probably would have been able to finish his symphony.
In research, as in music, blind bureaucracy has the effect of destroying imaginative creativity. If that is truly valued, it must remain free from bureaucratic excesses. And indeed, the great strength of British science has always been the ability of curiosity-driven individuals to follow up original ideas, and the support that these individuals receive from organizations such as the MRC and the Wellcome Trust. This has given research workers the possibility to twist and turn in following up intuitions and ideas in other fields as well as their own. This freedom has been significantly eroded by job insecurity in universities and in commercial enterprises.”
Later in the article, Born talks about innovation in the pharmaceutical industry
“This ‘urge to merge’, which is affecting the pharmaceutical industry worldwide, is almost always claimed to be justified by the need for a larger research budget to sustain innovation. The actual evidence indicates that such mergers hide -for a while failure of innovation. An almost universal response to this problem has been to ‘streamline’ and ‘commercialize’ the process of research, with ultimate control vested in accountants rather than in pharmaceutically informed scientists. This has meant that the industry’s research and development programmes are being driven by technical novelties, notably computer- aided drug design, combinatorial chemistry, high-throughput screening and genomics. All these techniques are very cost-intensive and, what is worse, are superseding individual scientists with profound appreciation of disease mechanisms and knowledge of biochemical and pharmacological mechanisms. It is they whose ability to ask the crucial, often seemingly simple questions, that have led to blockbuster drugs. Outstanding British examples are James Black’s discovery of the gastric acid secretion inhibitors, and Hans Kosterlitz’s question whether the brain might perhaps contain some analgesic chemical like that in -of all things the poppy plant.”
“The fact is that drug discovery, like all discoveries, is more an individual than a team achievement, at least at the beginning. With a few notable exceptions, the trouble with the industry is summarized by a group research director at one of the leading pharmaceutical companies:
“Creative individuals are being driven out of the industry and being replaced by functionaries wbo parrot strategic maxims. Research is being driven by lawyers, financial experts, salesmen and market strategists who are completely unable to develop new ideas. It is doubtful whether there are any senior executives who understand the problem” (Drews, J., 1999 In quest of tomorrow’s medicine, Springer-Verlag, New York).
And further:
“Partly as a result of mismanagement and partly as a result of a search for solutions which takes no account of disease mechanisms and biomedical complexity, substantial parts of the pharmaceutical industry are failing to innovate at a rate which is needed for their health and for the health of the general public. Research management needs to be rethought with a much greater emphasis on creative individuals with a broad knowledge of biology and medicine, a lower emphasis on market research, and a greater openness to the information to be gained from clinical studies (Horrobin, D.F., 2000, Innovation in the pharmaceutical industry. J. R. Soc. Med. 93, 341 – 345. ).”
For more on keeping univeristies honest, see the excellent new blog, The storm breaking upon the university.