Jump to follow-up

It seems that validation committees often don’t look beyond the official documents. As a result, the validations may not be worth the paper they are written on. Try this one.

One of the best bits of news recently was the downfall of Matthias Rath.  He’s the man who peddled vitamin pills for AIDS in Africa, and encouraged the AIDS denialists in the South African government.  Thabo Mbeki and his Health Minister, Mrs Beetroot, have gone now, thank heavens.

Rath was one of the best illustrations of the murderous effect of selling ineffective treatments.  The fact that nobody in the “nutritional therapy” industry has uttered a word of condemnation for this man illustrates better than anything one can imagine the corrupt state of “nutritional therapy”.  The people who kept silent include the British Association of Nutritional Therapists (BANT).

It might be surprising, then, to find the Northern College of Acupuncture proudly adding a course in alternative nutrition to its courses in acupuncture (now known to be a theatrical placebo) and Chinese herbal medicine (largely untested and sometimes toxic).  It might be even more surprising to find the boast that the course is validated by the University of Wales.  It seemed a good idea to find out a bit more about how this came about.  Thanks to the Freedom of Information Act, some interesting things can be discovered.

Polly Toynbee’s superb article, Quackery and superstition – available soon on the NHS,  written in January 2008, mentioned diplomas and degrees in complementary therapies offered by, among others, the University of Wales.  This elicited a letter of protest to Toynbee from the Vice-Chancellor of the University of Wales, Professor Marc Clement BSc, PhD, MInstP, CEng,CPhys,FIET.   He invited her to visit the university to see their “validation and monitoring procedures (including the University’s very specific guidelines on health studies disciplines”.

So let’s take a look at these validation procedures and guidelines.

The validation process

The Northern College of Acupuncture submitted a 148 page proposal for the course in October 2007.   The document has all the usual edu-bollocks jargon, but of course doesn’t say much about clinical trials, though it does boast about an unblinded trial of acupuncture published in 2006 which, because of lack of appropriate controls, served only to muddy the waters.  : This submission was considered by the University’s validation committee last December.

Panel of Assessors: Professor Nigel Palastanga (Chair), Cardiff University Dr Celia Bell (School of Health and Social Sciences Middlesex University) Dr John Fish (Moderator designate) (Institute of Biological Sciences University of Wales, Aberystwyth) Ms Rhiannon Harris (Centre for Nutrition & Dietetics University of Wales Institute, Cardiff (UWIC)) Ms Felicity Moir (School of Integrated Health University of Westminster)

The whole validation document is only four pages long [download it]. The most interesting thing about it is that the words ‘evidence’ or ‘critical’ do not occur in it a single time.  It has all the usual bureaucratic jargon of such documents but misses entirely the central point.

Does that mean that the University of Wales doesn’t care about evidence or critical thinking?  Well, not on paper. Two years previously a short document called Health Studies Guidelines had been written by Dr Brian Spriggs (Health Studies Validation Consultant, since retired) for the Health Studies Committee, and it was approved on 21 April 2005.  It starts well.

“Degrees in the Health Studies field are expected to promote an understanding of the  importance of the scientific method and an evidence-base to underpin therapeutic interventions and of research to expand that base.”

It even goes on to say that a BSc degree in homeopathy is “unacceptable”.  Don’t get too excited though, because it also says that acupuncture and Chinese herbal  stuff is quite OK.  How anyone can imagine they live up to the  opening sentence beats me.   And it gets worse. It says that all sorts of rather advanced forms of battiness are OK if they form only part of another degree.  They include Homeopathy, Crystal therapy. Dowsing, Iridology; Kinesiology, Radionics, Reflexology, Shiatsu, Healing, and Maharishi Ayurvedic Medicine.

Dowsing? Crystal therapy?  Just let me remind you. We are living in 2008. It is easy to forget that when ploughing through all this new age junk.

The Validation Handbook of Quality Assurance: Health Studies (2007) runs to an astonishing 256 pages [download the whole thing]. On page 12 we find the extent of the problem.

“The University of Wales validates a number of schemes in the Health Studies field. At the current time we have undergraduate and/or postgraduate degree schemes in Acupuncture, Animal Manipulation, Chiropractic, Herbal Medicine, Integrative Psychotherapy, Osteopathy, Osteopathic Studies, Traditional Chinese Medicine and Regulatory Affairs, both in the UK and overseas.”

That sounds pretty shocking.   Further down on page 12, though, we find this.

“Degrees in the Health Studies field are expected to promote an understanding of the importance of the scientific method and an evidence-base to underpin therapeutic interventions and of research to expand that base. The mission is to promote and require the critical evaluation of the practices, doctrines, beliefs, theories and hypotheses that underlie the taught therapeutic measures of the discipline.”

They are indeed fine words. The problem is that I can detect no sign in the submission, nor in its consideration by the validation committee, that any attempt whatsoever was made to ensure that the course complied with these requirements.

The only sign of concern I could detect of any concern about the quality of what was being taught came in a minute to a meeting of the Health Studies Committee meeting on 24th April 2008.

“Members received a copy of an article entitled Quackery and superstition available soon on the NHS which appeared in The Guardian newspaper in January 2008, and a copy of the Vice- Chancellors response. Members agreed that this article was now historical but felt that if/when the issue were to arise again; the key matter of scientific rigour should be stressed. The Committee agreed that this was the most critical element of all degree schemes in the University of Wales portfolio of health studies schemes. It was felt it would be timely to re-examine the schemes within the portfolio as well as the guidelines for consideration of Health Studies schemes at the next meeting. The Committee might also decide that Institutions would be required to include literature reviews (as part of their validation submission) to provide evidence for their particular profession/philosophy. It was agreed that the guidelines would be a vital document in the consideration of new schemes and during preliminary visits to prospective Institutions. “

The Press Office had passed Polly Toynbee’s article to them.  Curiously the Health Studies Committee dismissed it as “historical”, simply because it was written three months earlier.  That is presumably “historical” in the sense that the public will have forgotten about it, rather than in the sense that the facts of the matter have changed since January.  So, at least for the nutrition degree, Toynbee’s comments  were simply brushed under the carpet.

After a few cosmetic changes of wording the validation was completed on 16th January 2008. For example the word “diagnosis” was removed in 43 places and “rewritten in terms of evaluation and assessment”.  There was, needless to say, no indication that the change in wording would change anything in what was taught to students.

You may think that I am being a bit too harsh. Perhaps the course is just fine after all? The problem is that the submission and the reaction of the validation committee tell you next to nothing about what actually matters, and that is what is taught. There is only a vague outline of that in the submission (and part of it was redacted on the grounds that if it were made public somebody might copy ;it.  Heaven forbid).

That is why I have to say, yet again, that this sort of validation exercise is not worth the paper it’s written on.

How can we find out a bit more? Very easily as it happens. Just Google. What matters is not so much formal course outlines but who teaches them.

The nutrition course

The title of the course is just “Nutrition”, not ‘Nutritional Therapy’ or ‘Alternative Nutrition’.  That sounds quite respectable but a glance at the prospectus shows immediately that it is full-blown alternative medicine.

Already in July 2007, the glowing press releases for the course had attracted attention from the wonderfully investigative web site HolfordWatch.   I see no sign that the validation committee was aware of this. But if not, why not? I would describe is as dereliction of academic duty.

“This pioneering course is unique in that it is firmly rooted in both Western nutritional science and naturopathic medicine and also covers concepts of nutrition within traditional Chinese, Japanese, Tibetan and Ayurvedic medicine.

This means that graduates will gain comprehensive understanding of both modern scientific knowledge and ancient wisdom concerning nutrition and dietetics.”

Ancient wisdom, of course, means something that your are supposed to believe though there is no good reason to think it’s true.  In the end, though, almost the only thing that really matters about any course is who is running it.  The brochure shows that all of the people are heavily into every form of alternative nuttiness.

Course Director and Tutor: Jacqueline Young nutritionist, naturopath, clinical psychologist and Oriental medical practitioner

Nutrition Tutors:

Elaine Aldred  (qualified as a chiropractor with the Anglo European Chiropractic College, as an acupuncturist with the British College of Acupuncture and as a Western Medical Herbalist with the College of Phytotherapy. She recently also qualified in Chinese herbal medicine with the Northern College of Acupuncture.)

Sue Russell (3 year diploma in nutritional therapy at the Institute of Optimum Nutrition. She currently practises as a nutritional therapist and also works part-time as a manager at the Northern College of Homeopathic Medicine.)

Anuradha Sharma (graduated as a dietician from Leeds Metropolitan University in 2002 and subsequently completed a Naturopathy certificate and a post-graduate diploma in acupuncture).

Guest Lecturers include : Dr John Briffa, Professor Jane Plant, M.B.E. (a geochemist turned quack), and, most revealingly, none other than the UK’s most notorious media celebrity and pill peddler, Patrick Holford.

So much has been written about Holford’s appalling abuse of science, one would have thought that not even a validation committee could have missed it.

“The course has been created by Jacqueline Young“, so let’s look a bit further at her track record.

Jacqueline Young has written a book, ‘Complementary Medicine for Dummies’ [Ed: ahem shouldn’t that be Dummies for Complementary Medicine?].  You can see parts of it on Google Books. Did the validation committee bother to look at it?  As far as I can tell, the words ‘randomised’ or ‘clinical trial’ occur nowhere in the book.

The chapter on Tibetan medicine is not very helpful when it comes to evidence but for research we are referred to the Tibetan Medical and Astrology Institute.  Guess what? That site gives no evidence either. So far not a single university has endorsed Astrology (there is a profitable niche there for some vice-chancellor).

Here are few samples from the book. The advice seems to vary from the undocumented optimism of this

Well researched?  No. Safe? Nobody knows.  Or this

Mandarin peel prevents colds and flu?  Old wive’s tale.  Then there are things that verge on the weird, like this one

or the deeply bizarre like this

The problem of Jacqueline Young’s fantasy approach to facts was pointed out at least as far back as 2004, by Ray Girvan., who wrote about it again in May 2005. The problems were brought to wider attention when Ben Goldacre wrote two articles in his Badscience column, Imploding Researchers (September 2005), and the following week, Tangled Webs.

“we were pondering the ethics and wisdom of Jacqueline Young dishing out preposterous, made-up, pseudoscientific nonsense as if it was authoritative BBC fact, with phrases such as: “Implosion researchers have found that if water is put through a spiral its electrical field changes and it then appears to have a potent, restorative effect on cells.” “

and later

“Take this from her article on cranial osteopathy, riddled with half truths: “Sutherland found that the cranial bones (the skull bones encasing the brain) weren’t fused in adulthood, as was widely believed, but actually had a cycle of slight involuntary movement.” In fact the cranial bones do fuse in adulthood.

She goes on: “This movement was influenced by the rhythmic flow of cerebrospinal fluid (the nourishing and protective fluid that circulates through the spinal canal and brain) and could become blocked.” There have now been five studies on whether “cranial osteopaths” can indeed feel these movements, as they claim, and it’s an easy experiment to do: ask a couple of cranial osteopaths to write down the frequency of the rhythmic pulses on the same person’s skull, and see if they give the same answer. They don’t. A rather crucial well-replicated finding to leave out of your story.

That was in 2005 and since then all of Young’s “preposterous, made-up, pseudoscientific nonsense” (along with most of the other stuff about junk medicine) has vanished from the BBC’s web site, after some people with a bit of common sense pointed out what nonsense it was.  But now we see them resurfacing in a course validated by a serious university.  The BBC had some excuse (after all, it is run largely by arts graduates). I can see no excuses for the University of Wales.

Incidentally,  thanks to web archive you can still read Young’s nonsense,  long after the BBC removed it.  Here is a quotation.

“Implosion researchers have found that if water is put through a spiral its ,field changes and it then appears to have a potent, restorative effect on cells. In one study, seedlings watered with spiralised water grew significantly faster, higher and stronger than those given ordinary water.”

The vice-chancellor of the University of Wales, Marc Clement, is a physicist (Department of Electrical and Electronic Engineering), so can he perhaps explain the meaning of this?

Selection committees for jobs (especially senior jobs) and validation committees for courses, might make fewer mistakes if they didn’t rely so much on formal documents and did a little more investigation themselves. That sort of thing is why the managerial culture not only takes a lot more time, but also gives a worse result.

It would have taken 10 minutes with Google to find out about Young’s track record, but they didn’t bother. As a result they have spent a long time producing a validation that isn’t worth the paper it’s written on. That makes the University of Wales a bit of a laughing stock. Worse still, it brings science itself into disrepute.

Follow-up

What does the University of Wales say? So far, nothing.   Last week I sent brief and polite emails to Professor Palastanga and to Professor Clement to try to discover whether it is true that the validation process had indeed missed the fact that the course organiser’s writings had been described as “preposterous, made-up, pseudoscientific nonsense” in the Guardian.

So far I have had no reply from the vice-chancellor, but on .26 October I did get an answer from Prof Palastaga.

I have to say this reply confirms my worst fears. Validation committees such as this one simply don’t do their duty. They don’t show the curiosity that is needed to discover the facts about the things that they are meant to be judging. How could they not have looked at the book by the very person that they are validating? After all that has been written about Patrick Holford, it is simply mind-boggling that the committee seems to have been quite unaware of any of it.

It is yet another example of the harm done to science by an unthinking, box-ticking approach.

Pharmacology. A Handbook for Complementary Healthcare Professionals

Elsevier were kind enough to send me an inspection copy of this book, which is written by one of the nutrition course tutors, Elaine Aldred. She admits that pharmacology is “considered by most students to be nothing more that a ‘hoop-jumping’ exercise in the process of becoming qualified”.  She also says. disarmingly. that “I was certainly not the most adept scientist at school and found my university course a trial”.

The book has all the feel of a cut and paste job. It is mostly very simple (if not simplistic). though for no obvious reason it starts with a long (and very amateur) discussion of chemical bonding  Then molecules are admitted to be indivisible (but, guess what, the subject of homeopathy is avoided).  There is a very short section on ion channels, though, bizarrely, it appears under the heading “How do drugs get into cells?”. Since the author is clearly not able to make the distinction between volts and coulombs, the discussion is more likely to confuse the reader than to help.

Then a long section on plants. It starts of by asserting that “approximately a quarter of prescription drugs contain at least one chemical that was originally isolated and extracted from a plant”.. This cannot be even remotely correct.  There are vast tables showing complicated chemical structures, but the usual inadequate
list of their alleged actions  This is followed by a quick gallop through some classes of conventional drugs, illustrated again mainly by chemical structures not data.  Hormone replacement therapy is mentioned, but the chance to point out that it is one of the best illustrations of the need for RCTs is missed.

The one thing that one would really like to see in such a book is a good account of how you tell whether or not a drug works in man. This is relegated to five pages at the end of the book, and it is, frankly, pathetic.  It
is utterly uncritical in the one area that matters more than any other for people who purport to treat patients. All you get is a list of unexplained bullet points.

If this book is the source of the “scientific content” of the nutrition course, things are as bad as we feared.