Download Lectures on Biostatistics (1971). Corrected and searchable version of Google books edition
Download review of Lectures on Biostatistics (THES, 1973).
David Colquhoun
[This an update of a 2006 post on my old blog]
The New York Times (17 January 2006) published a beautiful spoof that illustrates only too clearly some of the bad practices that have developed in real science (as well as in quackery). It shows that competition, when taken to excess, leads to dishonesty.
More to the point, it shows that the public is well aware of the dishonesty that has resulted from the publish or perish culture, which has been inflicted on science by numbskull senior administrators (many of them scientists, or at least ex-scientists). Part of the blame must attach to "bibliometricians" who have armed administrators with simple-minded tools the usefulness is entirely unverified. Bibliometricians are truly the quacks of academia. They care little about evidence as long as they can sell the product.
The spoof also illustrates the folly of allowing the hegemony of a handful of glamour journals to hold scientists in thrall. This self-inflicted wound adds to the pressure to produce trendy novelties rather than solid long term work.
It also shows the only-too-frequent failure of peer review to detect problems.
The future lies on publication on the web, with post-publication peer review. It has been shown by sites like PubPeer that anonymous post-publication review can work very well indeed. This would be far cheaper, and a good deal better than the present extortion practised on universities by publishers. All it needs is for a few more eminent people like mathematician Tim Gowers to speak out (see Elsevier – my part in its downfall).
Recent Nobel-prizewinner Randy Schekman has helped with his recent declaration that "his lab will no longer send papers to Nature, Cell and Science as they distort scientific process"
The spoof is based on the fraudulent papers by Korean cloner, Woo Suk Hwang, which were published in Science, in 2005. As well as the original fraud, this sad episode exposed the practice of ‘guest authorship’, putting your name on a paper when you have done little or no work, and cannot vouch for the results. The last (‘senior’) author on the 2005 paper, was Gerald Schatten, Director of the Pittsburgh Development Center. It turns out that Schatten had not seen any of the original data and had contributed very little to the paper, beyond lobbying Scienceto accept it. A University of Pittsburgh panel declared Schatten guilty of “research misbehavior”, though he was, amazingly, exonerated of “research misconduct”. He still has his job. Click here for an interesting commentary.
The New York Times carried a mock editorial to introduce the spoof..
One Last Question: Who Did the Work? By NICHOLAS WADE In the wake of the two fraudulent articles on embryonic stem cells published in Science by the South Korean researcher Hwang Woo Suk, Donald Kennedy, the journal’s editor, said last week that he would consider adding new requirements that authors “detail their specific contributions to the research submitted,” and sign statements that they agree with the conclusions of their article. A statement of authors’ contributions has long been championed by Drummond Rennie, deputy editor of The Journal of the American Medical Association, Explicit statements about the conclusions could bring to light many reservations that individual authors would not otherwise think worth mentioning. The article shown [below] from a future issue of the Journal of imaginary Genomics, annotated in the manner required by Science‘s proposed reforms, has been released ahead of its embargo date. |
The old-fashioned typography makes it obvious that the spoof is intended to mock a paper in Science.
The problem with this spoof is its only too accurate description of what can happen at the worst end of science.
Something must be done if we are to justify the money we get and and we are to retain the confidence of the public
My suggestions are as follows
- Nature Science and Cell should become news magazines only. Their glamour value distorts science and encourages dishonesty
- All print journals are outdated. We need cheap publishing on the web, with open access and post-publication peer review. The old publishers would go the same way as the handloom weavers. Their time has past.
- Publish or perish has proved counterproductive. You’d get better science if you didn’t have any performance management at all. All that’s needed is peer review of grant applications.
- It’s better to have many small grants than fewer big ones. The ‘celebrity scientist’, running a huge group funded by many grants has not worked well. It’s led to poor mentoring and exploitation of junior scientists.
- There is a good case for limiting the number of original papers that an individual can publish per year, and/or total grant funding. Fewer but more complete papers would benefit everyone.
- Everyone should read, learn and inwardly digest Peter Lawrence’s The Mismeasurement of Science.
Follow-up
3 January 2014.
Yet another good example of hype was in the news. “Effect of Vitamin E and Memantine on Functional Decline in Alzheimer Disease“. It was published in the Journal of the American Medical Association. The study hit the newspapers on January 1st with headlines like Vitamin E may slow Alzheimer’s Disease (see the excellent analyis by Gary Schwitzer). The supplement industry was ecstatic. But the paper was behind a paywall. It’s unlikely that many of the tweeters (or journalists) had actually read it.
The trial was a well-designed randomised controlled trial that compared four treatments: placebo, vitamin E, memantine and Vitamin E + memantine.
Reading the paper gives a rather different impression from the press release. Look at the pre-specified primary outcome of the trial.
The primary outcome measure was
" . . the Alzheimer’s Disease Cooperative Study/Activities of Daily Living (ADCSADL) Inventory.12 The ADCS-ADL Inventory is designed to assess functional abilities to perform activities of daily living in Alzheimer patients with a broad range of dementia severity. The total score ranges from 0 to 78 with lower scores indicating worse function."
It looks as though any difference that might exist between the four treaments is trivial in size. In fact the mean difference between Vitamin E and placebos was only 3.15 (on a 78 point scale) with 95% confidence limits from 0.9 to 5.4. This gave a modest P = 0.03 (when properly corrected for multiple comparisons), a result that will impress only those people who regard P = 0.05 as a sort of magic number. Since the mean effect is so trivial in size that it doesn’t really matter if the effect is real anyway.
It is not mentioned in the coverage that none of the four secondary outcomes achieved even a modest P = 0.05 There was no detectable effect of Vitamin E on
- Mean annual rate of cognitive decline (Alzheimer Disease Assessment Scale–Cognitive Subscale)
- Mean annual rate of cognitive decline (Mini-Mental State Examination)
- Mean annual rate of increased symptoms
- Mean annual rate of increased caregiver time,
The only graph that appeared to show much effect was The Dependence Scale. This scale
“assesses 6 levels of functional dependence. Time to event is the time to loss of 1 dependence level (increase in dependence). We used an interval-censored model assuming a Weibull distribution because the time of the event was known only at the end of a discrete interval of time (every 6 months).”
It’s presented as a survival (Kaplan-Meier) plot. And it is this somewhat obscure secondary outcome that was used by the Journal of the American Medical Assocciation for its publicity.
Note also that memantine + Vitamin E was indistinguishable from placebo. There are two ways to explain this: either Vitamin E has no effect, or memantine is an antagonist of Vitamin E. There are no data on the latter, but it’s certainly implausible.
The trial used a high dose of Vitamin E (2000 IU/day). No toxic effects of Vitamin E were reported, though a 2005 meta-analysis concluded that doses greater than 400 IU/d "may increase all-cause mortality and should be avoided".
In my opinion, the outcome of this trial should have been something like “Vitamin E has, at most, trivial effects on the progress of Alzheimer’s disease”.
Both the journal and the authors are guilty of disgraceful hype. This continual raising of false hopes does nothing to help patients. But it does damage the reputation of the journal and of the authors.
|
This paper constitutes yet another failure of altmetrics. (see more examples on this blog). Not surprisingly, given the title, It was retweeted widely, but utterly uncritically. Bad science was promoted. And JAMA must take much of the blame for publishing it and promoting it. |
|
This article has been reposted on The Winnower, and now has a digital object identifier DOI: 10.15200/winn.142934.47856
This post is not about quackery, nor university politics. It is about inference, How do we know what we should eat? The question interests everyone, but what do we actually know? Not as much as you might think from the number of column-inches devoted to the topic. The discussion below is a synopsis of parts of an article called “In praise of randomisation”, written as a contribution to a forthcoming book, Evidence, Inference and Enquiry.
About a year ago just about every newspaper carried a story much like this one in the Daily Telegraph,
|
Sausage a day can increase bowel cancer risk By Rebecca Smith, Medical Editor Last Updated: 1:55AM BST 31/03/2008
|
||||
What, I wondered, was the evidence behind these dire warnings. They did not come from a lifestyle guru, a diet faddist or a supplement salesman. This is nothing to do with quackery. The numbers come from the 2007 report of the World Cancer Research Fund and American Institute for Cancer Research, with the title ‘Food, Nutrition, Physical Activity, and the Prevention of Cancer: a Global Perspective‘. This is a 537 page report with over 4,400 references. Its panel was chaired by Professor Sir Michael Marmot, UCL’s professor of Epidemiology and Public Health. He is a distinguished epidemiologist, renowned for his work on the relation between poverty and health.
Nevertheless there has never been a randomised trial to test the carcinogenicity of bacon, so it seems reasonable to ask how strong is the evidence that you shouldn’t eat it? It turns out to be surprisingly flimsy.
In praise of randomisation
Everyone knows about the problem of causality in principle. Post hoc ergo propter hoc; confusion of sequence and consequence; confusion of correlation and cause. This is not a trivial problem. It is probably the main reason why ineffective treatments often appear to work. It is traded on by the vast and unscrupulous alternative medicine industry. It is, very probably, the reason why we are bombarded every day with conflicting advice on what to eat. This is a bad thing, for two reasons. First, we end up confused about what we should eat. But worse still, the conflicting nature of the advice gives science as a whole a bad reputation. Every time a white-coated scientist appears in the media to tell us that a glass of wine per day is good/bad for us (delete according to the phase of the moon) the general public just laugh.
In the case of sausages and bacon, suppose that there is a correlation between eating them and developing colorectal cancer. How do we know that it was eating the bacon that caused the cancer – that the relationship is causal? The answer is that there is no way to be sure if we have simply observed the association. It could always be that the sort of people who eat bacon are also the sort of people who get colorectal cancer. But the question of causality is absolutely crucial, because if it is not causal, then stopping eating bacon won’t reduce your risk of cancer. The recommendation to avoid all processed meat in the WCRF report (2007) is sensible only if the relationship is causal. Barker Bausell said:
[Page39] “But why should nonscientists care one iota about something as esoteric as causal inference? I believe that the answer to this question is because the making of causal inferences is part of our job description as Homo Sapiens.”
That should be the mantra of every health journalist, and every newspaper reader.
|
The essential basis for causal inference was established over 70 years ago by that giant of statistics Ronald Fisher, and that basis is randomisation. Its first popular exposition was in Fisher’s famous book, The Design of Experiments (1935). The Lady Tasting Tea has become the classical example of how to design an experiment. . |
|
Briefly, a lady claims to be able to tell whether the milk was put in the cup before or after the tea was poured. Fisher points out that to test this you need to present the lady with an equal number of cups that are ‘milk first’ or ‘tea first’ (but otherwise indistinguishable) in random order, and count how many she gets right. There is a beautiful analysis of it in Stephen Senn’s book, Dicing with Death: Chance, Risk and Health. As it happens, Google books has the whole of the relevant section Fisher’s tea test (geddit?), but buy the book anyway. Such is the fame of this example that it was used as the title of a book, The Lady Tasting Tea was published by David Salsburg (my review of it is here)
Most studies of diet and health fall into one of three types, case-control studies, cohort (or prospective) studies, or randomised controlled trials (RCTs). Case-control studies are the least satisfactory: they look at people who already have the disease and look back to see how they differ from similar people who don’t have the disease. They are retrospective. Cohort studies are better because they are prospective: a large group of people is followed for a long period and their health and diet is recorded and later their disease and death is recorded. But in both sorts of studies,each person decides for him/herself what to eat or what drugs to take. Such studies can never demonstrate causality, though if the effect is really big (like cigarette-smoking and lung cancer) they can give a very good indication. The difference in an RCT is that each person does not choose what to eat, but their diet is allocated randomly to them by someone else. This means that, on average, all other factors that might influence the response are balanced equally between the two groups. Only RCTs can demonstrate causality.
Randomisation is a rather beautiful idea. It allows one to remove, in a statistical sense, bias that might result from all the sources that you hadn’t realised were there. If you are aware of a source of bias, then measure it. The danger arises from the things you don’t know about, or can’t measure (Senn, 2004; Senn, 2003). Although it guarantees freedom from bias only in a long run statistical sense, that is the best that can be done. Everything else is worse.
Ben Goldacre has referred memorably to the newspapers’ ongoing “Sisyphean task of dividing all the inanimate objects in the world into the ones that either cause or cure cancer” (Goldacre, 2008). This has even given rise to a blog. “The Daily Mail Oncological Ontology Project“. The problem arises in assessing causality.
It wouldn’t be so bad if the problem were restricted to the media. It is much more worrying that the problem of establishing causality often seems to be underestimated by the authors of papers themselves. It is a matter of speculation why this happens. Part of the reason is, no doubt, a genuine wish to discover something that will benefit mankind. But it is hard not to think that hubris and self-promotion may also play a role. Anything whatsoever that purports to relate diet to health is guaranteed to get uncritical newspaper headlines.
At the heart of the problem lies the great difficulty in doing randomised studies of the effect of diet and health. There can be no better illustration of the vital importance of randomisation than in this field. And, notwithstanding the generally uncritical reporting of stories about diet and health, one of the best accounts of the need for randomisation was written by a journalist, Gary Taubes, and it appeared in the New York Times (Taubes, 2007).
The case of hormone replacement therapy
In the 1990s hormone replacement therapy (HRT) was recommended not only to relieve the unpleasant symptoms of the menopause, but also because cohort studies suggested that HRT would reduce heart disease and osteoporosis in older women. For these reasons, by 2001, 15 million US women (perhaps 5 million older women) were taking HRT (Taubes, 2007). These recommendations were based largely on the Harvard Nurses’ Study. This was a prospective cohort study in which 122,000 nurses were followed over time, starting in 1976 (these are the ones who responded out of the 170,000 requests sent out). In 1994, it was said (Manson, 1994) that nearly all of the more than 30 observational studies suggested a reduced risk of coronary heart disease (CHD) among women receiving oestrogen therapy. A meta-analysis gave an estimated 44% reduction of CHD. Although warnings were given about the lack of randomised studies, the results were nevertheless acted upon as though they were true. But they were wrong. When proper randomised studies were done, not only did it turn out that CHD was not reduced: it was actually increased.
The Women’s Health Initiative Study (Rossouw et al., 2002) was a randomized double blind trial on 16,608 postmenopausal women aged 50-79 years and its results contradicted the conclusions from all the earlier cohort studies. HRT increased risks of heart disease, stroke, blood clots, breast cancer (though possibly helped with osteoporosis and perhaps colorectal cancer). After an average 5.2 years of follow-up, the trial was stopped because of the apparent increase in breast cancer in the HRT group. The relative risk (HRT relative to placebo) of CHD was 1.29 (95% confidence interval 1.02 to 1.63) (286 cases altogether) and for breast cancer 1.26 (1.00 -1.59) (290 cases). Rather than there being a 44% reduction of risk, it seems that there was actually a 30% increase in risk. Notice that these are actually quite small risks, and on the margin of statistical significance. For the purposes of communicating the nature of the risk to an individual person it is usually better to specify the absolute risk rather than relative risk. The absolute number of CHD cases per 10,000 person-years is about 29 on placebo and 36 on HRT, so the increased risk of any individual is quite small. Multiplied over the whole population though, the number is no longer small.
Several plausible reasons for these contradictory results are discussed by Taubes,(2007): it seems that women who choose to take HRT are healthier than those who don’t. In fact the story has become a bit more complicated since then: the effect of HRT depends on when it is started and on how long it is taken (Vandenbroucke, 2009).
This is perhaps one of the most dramatic illustrations of the value of randomised controlled trials (RCTs). Reliance on observations of correlations suggested a 44% reduction in CHD, the randomised trial gave a 30% increase in CHD. Insistence on randomisation is not just pedantry. It is essential if you want to get the right answer.
Having dealt with the cautionary tale of HRT, we can now get back to the ‘Sisyphean task of dividing all the inanimate objects in the world into the ones that either cause or cure cancer’.
The case of processed meat
The WCRF report (2007) makes some pretty firm recommendations.
- Don’t get overweight
- Be moderately physically active, equivalent to brisk walking for at least 30 minutes every day
- Consume energy-dense foods sparingly. Avoid sugary drinks. Consume ‘fast foods’ sparingly, if at all
- Eat at least five portions/servings (at least 400 g or 14 oz) of a variety of non-starchy vegetables and of fruits every day. Eat relatively unprocessed cereals (grains) and/or pulses (legumes) with every meal. Limit refined starchy foods
- People who eat red meat to consume less than 500 g (18 oz) a week, very little if any to be processed.
- If alcoholic drinks are consumed, limit consumption to no more than two drinks a day for men and one drink a day for women.
- Avoid salt-preserved, salted, or salty foods; preserve foods without using salt. Limit consumption of processed foods with added salt to ensure an intake of less than 6 g (2.4 g sodium) a day.
- Dietary supplements are not recommended for cancer prevention.
These all sound pretty sensible but they are very prescriptive. And of course the recommendations make sense only insofar as the various dietary factors cause cancer. If the association is not causal, changing your diet won’t help. Note that dietary supplements are NOT recommended. I’ll concentrate on the evidence that lies behind “People who . . . very little if any to be processed.”
The problem of establishing causality is dicussed in the report in detail. In section 3.4 the report says
” . . . causal relationships between food and nutrition, and physical activity can be confidently inferred when epidemiological evidence, and experimental and other biological findings, are consistent, unbiased, strong, graded, coherent, repeated, and plausible.”
The case of processed meat is dealt with in chapter 4.3 (p. 148) of the report.
“Processed meats” include sausages and frankfurters, and ‘hot dogs’, to which nitrates/nitrites or other preservatives are added, are also processed meats. Minced meats sometimes, but not always, fall inside this definition if they are preserved chemically. The same point applies to ‘hamburgers’.
The evidence for harmfulness of processed meat was described as “convincing”, and this is the highest level of confidence in the report, though this conclusion has been challenged (Truswell, 2009) .
How well does the evidence obey the criteria for the relationship being causal?
Twelve prospective cohort studies showed increased risk for the highest intake group when compared to the lowest, though this was statistically significant in only three of them. One study reported non-significant decreased risk and one study reported that there was no effect on risk. These results are summarised in this forest plot (see also Lewis & Clark, 2001)
Each line represents a separate study. The size of the square represents the precision (weight) for each. The horizontal bars show the 95% confidence intervals. If it were possible to repeat the observations many times on the same population, the 95% CL would be different on each repeat experiment, but 19 out of 20 (95%) of the intervals would contain the true value (and 1 in 20 would not contain the true value). If the bar does not overlap the vertical line at relative risk = 1 (i.e. no effect) this is equivalent to saying that there is a statistically significant difference from 1 with P < 0.05. That means, very roughly, that there is a 1 in 20 chance of making a fool of yourself if you claim that the association is real, rather than being a result of chance (more detail here),
There is certainly a tendency for the relative risks to be above one, though not by much, Pooling the results sounds like a good idea. The method for doing this is called meta-analysis .
Meta-analysis was possible on five studies, shown below. The outcome is shown by the red diamond at the bottom, labelled “summary effect”, and the width of the diamond indicates the 95% confidence interval. In this case the final result for association between processed meat intake and colorectal cancer was a relative risk of 1.21 (95% CI 1.04–1.42) per 50 g/day. This is presumably where the headline value of a 20% increase in risk came from.
Support came from a meta-analysis of 14 cohort studies, which reported a relative risk for processed meat of 1.09 (95% CI 1.05 – 1.13) per 30 g/day (Larsson & Wolk, 2006). Since then another study has come up with similar numbers (Sinha etal. , 2009). This consistency suggests a real association, but it cannot be taken as evidence for causality. Observational studies on HRT were just as consistent, but they were wrong.
The accompanying editorial (Popkin, 2009) points out that there are rather more important reasons to limit meat consumption, like the environmental footprint of most meat production, water supply, deforestation and so on.
So the outcome from vast numbers of observations is an association that only just reaches the P = 0.05 level of statistical significance. But even if the association is real, not a result of chance sampling error, that doesn’t help in the least in establishing causality.
There are two more criteria that might help, a good relationship between dose and response, and a plausible mechanism.
Dose – response relationship
|
It is quite possible to observe a very convincing relationship between dose and response in epidemiological studies, The relationship between number of cigarettes smoked per day and the incidence of lung cancer is one example. Indeed it is almost the only example. |
 Doll & Peto, 1978 |
There have been six studies that relate consumption of processed meat to incidence of colorectal cancer. All six dose-response relationships are shown in the WCRG report. Here they are.
This Figure was later revised to
This is the point where my credulity begins to get strained. Dose – response curves are part of the stock in trade of pharmacologists. The technical description of these six curves is, roughly, ‘bloody horizontal’. The report says “A dose-response relationship was also apparent from cohort studies that measured consumption in times/day”. I simply cannot agree that any relationship whatsoever is “apparent”.
They are certainly the least convincing dose-response relationships I have ever seen. Nevertheless a meta-analysis came up with a slope for response curve that just reached the 5% level of statistical significance.
The conclusion of the report for processed meat and colorectal cancer was as follows.
“There is a substantial amount of evidence, with a dose-response relationship apparent from cohort studies. There is strong evidence for plausible mechanisms operating in humans. Processed meat is a convincing cause of colorectal cancer.”
But the dose-response curves look appalling, and it is reasonable to ask whether public policy should be based on a 1 in 20 chance of being quite wrong (1 in 20 at best –see Senn, 2008). I certainly wouldn’t want to risk my reputation on odds like that, never mind use it as a basis for public policy.
So we are left with plausibility as the remaining bit of evidence for causality. Anyone who has done much experimental work knows that it is possible to dream up a plausible explanation of any result whatsoever. Most are wrong and so plausibility is a pretty weak argument. Much play is made of the fact that cured meats contain nitrates and nitrites, but there is no real evidence that the amount they contain is harmful.
The main source of nitrates in the diet is not from meat but from vegetables (especially green leafy vegetables like lettuce and spinach) which contribute 70 – 90% of total intake. The maximum legal content in processed meat is 10 – 25 mg/100g, but lettuce contains around 100 – 400 mg/100g with a legal limit of 200 – 400 mg/100g. Dietary nitrate intake was not associated with risk for colorectal cancer in two cohort studies.(Food Standards Agency, 2004; International Agency for Research on Cancer, 2006).
To add further to the confusion, another cohort study on over 60,000 people compared vegetarians and meat-eaters. Mortality from circulatory diseases and mortality from all causes were not detectably different between vegetarians and meat eaters (Key et al., 2009a). Still more confusingly, although the incidence of all cancers combined was lower among vegetarians than among meat eaters, the exception was colorectal cancer which had a higher incidence in vegetarians than in meat eaters (Key et al., 2009b).
Mente et al. (2009) compared cohort studies and RCTs for effects of diet on risk of coronary heart disease. “Strong evidence” for protective effects was found for intake of vegetables, nuts, and “Mediterranean diet”, and harmful effects of intake of trans–fatty acids and foods with a high glycaemic index. There was also a bit less strong evidence for effects of mono-unsaturated fatty acids and for intake of fish, marine ω-3 fatty acids, folate, whole grains, dietary vitamins E and C, beta carotene, alcohol, fruit, and fibre. But RCTs showed evidence only for “Mediterranean diet”, and for none of the others.
As a final nail in the coffin of case control studies, consider pizza. According to La Vecchia & Bosetti (2006), data from a series of case control studies in northern Italy lead to: “An inverse association was found between regular pizza consumption (at least one portion of pizza per week) and the risk of cancers of the digestive tract, with relative risks of 0.66 for oral and pharyngeal cancers, 0.41 for oesophageal, 0.82 for laryngeal, 0.74 for colon and 0.93 for rectal cancers.”
What on earth is one meant to make of this? Pizza should be prescribable on the National Health Service to produce a 60% reduction in oesophageal cancer? As the authors say “pizza may simply represent a general and aspecific indicator of a favourable Mediterranean diet.” It is observations like this that seem to make a mockery of making causal inferences from non-randomised studies. They are simply uninterpretable.
Is the observed association even real?
The most noticeable thing about the effects of red meat and processed meat is not only that they are small but also that they only just reach the 5 percent level of statistical significance. It has been explained clearly why, in these circumstances, real associations are likely to be exaggerated in size (Ioannidis, 2008a; Ioannidis, 2008b; Senn, 2008). Worse still, there as some good reasons to think that many (perhaps even most) of the effects that are claimed in this sort of study are not real anyway (Ioannidis, 2005). The inflation of the strength of associations is expected to be bigger in small studies, so it is noteworthy that the large meta-analysis by Larsson & Wolk, 2006 comments “In the present meta-analysis, the magnitude of the relationship of processed meat consumption with colorectal cancer risk was weaker than in the earlier meta-analyses”.
This is all consistent with the well known tendency of randomized clinical trials to show initially a good effect of treatment but subsequent trials tend to show smaller effects. The reasons, and the cures, for this worrying problem are discussed by Chalmers (Chalmers, 2006; Chalmers & Matthews, 2006; Garattini & Chalmers, 2009)
What do randomized studies tell us?
The only form of reliable evidence for causality comes from randomised controlled trials. The difficulties in allocating people to diets over long periods of time are obvious and that is no doubt one reason why there are far fewer RCTs than there are observational studies. But when they have been done the results often contradict those from cohort studies. The RCTs of hormone replacement therapy mentioned above contradicted the cohort studies and reversed the advice given to women about HRT.
Three more illustrations of how plausible suggestions about diet can be refuted by RCTs concern nutritional supplements and weight-loss diets
Many RCTs have shown that various forms of nutritional supplement do no good and may even do harm (see Cochrane reviews). At least we now know that anti-oxidants per se do you no good. The idea that anti-oxidants might be good for you was never more than a plausible hypothesis, and like so many plausible hypotheses it has turned out to be a myth. The word anti-oxidant is now no more than a marketing term, though it remains very profitable for unscrupulous salesmen.
The randomised Women’s Health Initiative Dietary Modification Trial (Prentice et al., 2007; Prentice, 2007) showed minimal effects of dietary fat on cancer, though the conclusion has been challenged on the basis of the possible inaccuracy of reported diet (Yngve et al., 2006).
Contrary to much dogma about weight loss (Sacks et al., 2009) found no differences in weight loss over two years between four very different diets. They assigned randomly 811 overweight adults to one of four diets. The percentages of energy derived from fat, protein, and carbohydrates in the four diets were 20, 15, and 65%; 20, 25, and 55%; 40, 15, and 45%; and 40, 25, and 35%. No difference could be detected between the different diets: all that mattered for weight loss was the total number of calories. It should be added, though, that there were some reasons to think that the participants may not have stuck to their diets very well (Katan, 2009).
The impression one gets from RCTs is that the details of diet are not anything like as important as has been inferred from non-randomised observational studies.
So does processed meat give you cancer?
After all this, we can return to the original question. Do sausages or bacon give you colorectal cancer? The answer, sadly, is that nobody really knows. I do know that, on the basis of the evidence, it seems to me to be an exaggeration to assert that “The evidence is convincing that processed meat is a cause of bowel cancer”.
In the UK there were around 5 cases of colorectal cancer per 10,000 population in 2005, so a 20% increase, even if it were real, and genuinely causative. would result in 6 rather than 5 cases per 10,000 people, annually. That makes the risk sound trivial for any individual. On the other hand there were 36,766 cases of colorectal cancer in the UK in 2005. A 20% increase would mean, if the association were causal, about 7000 extra cases as a result of eating processed meat, but no extra cases if the association were not causal.
For the purposes of public health policy about diet, the question of causality is crucial. One has sympathy for the difficult decisions that they have to make, because they are forced to decide on the basis of inadequate evidence.
If it were not already obvious, the examples discussed above make it very clear that the only sound guide to causality is a properly randomised trial. The only exceptions to that are when effects are really big. The relative risk of lung cancer for a heavy cigarette smoker is 20 times that of a non-smoker and there is a very clear relationship between dose (cigarettes per day) and response (lung cancer incidence), as shown above. That is a 2000% increase in risk, very different from the 20% found for processed meat (and many other dietary effects). Nobody could doubt seriously the causality in that case.
The decision about whether to eat bacon and sausages has to be a personal one. It depends on your attitude to the precautionary principle. The observations do not, in my view, constitute strong evidence for causality, but they are certainly compatible with causality. It could be true so if you want to be on the safe side then avoid bacon. Of course life would not be much fun if your actions were based on things that just could be true.
My own inclination would be to ignore any relative risk based on observational data if it was less than about 2. The National Cancer Institute (Nelson, 2002) advises that relative risks less than 2 should be “viewed with caution”, but fails to explain what “viewing with caution” means in practice, so the advice isn’t very useful.
In fact hardly any of the relative risks reported in the WCRF report (2007) reach this level. Almost all relative risks are less than 1.3 (or greater than 0.7 for alleged protective effects). Perhaps it is best to stop worrying and get on with your life. At some point it becomes counterproductive to try to micromanage `people’s diet on the basis of dubious data. There is a price to pay for being too precautionary. It runs the risk of making people ignore information that has got a sound basis. It runs the risk of excessive medicalisation of everyday life. And it brings science itself into disrepute when people laugh at the contradictory findings of observational epidemiology.
The question of how diet and other ‘lifestyle interventions’ affect health is fascinating to everyone. There is compelling reason to think that it matters. For example one study demonstrated that breast cancer incidence increased almost threefold in first-generation Japanese women who migrated to Hawaii, and up to fivefold in the second generation (Kolonel, 1980). Since then enormous effort has been put into finding out why. The first great success was cigarette smoking but that is almost the only major success. Very few similar magic bullets have come to light after decades of searching (asbestos and mesothelioma, or UV radiation and skin cancer count as successes).
The WCRF report (2007) has 537 pages and over 4400 references and we still don’t know.
Sometimes I think we should say “I don’t know” rather more often.
More material
-
Listen to Ben Goldacre’s Radio 4 programmes. The Rise of the Lifetsyle Nutritionists. Part 1 and Part 2 (mp3 files), and at badscience.net.
-
Risk The Science and Politics of Fear, Dan Gardner. Virgin
Books, 2008 -
Some bookmarks about diet and supplements
Follow up
Dan Gardner, the author of Risk, seems to like the last line at least, according to his blog.
Report of the update, 2010
The 2010 report has been updated in WCRF/AICR Systematic Literature Review Continuous Update Project Report [big pdf file]. This includes studies up to May/June 2010.
The result of addition of the new data was to reduce slightly the apparent risk from eating processed meat from 1.21 (95% CI = 1.04-1.42) in the original study to 1.18 (95% CI = 1.10-1.28) in the update. The change is too small to mean much, though it is in direction expected for false correlations. More importantly, the new data confirm that the dose-response curves are pathetic. The evidence for causality is weakened somewhat by addition of the new data.
Dose-response graph of processed meat and colorectal cancer
|
It seems that bits of good news don’t come singly. First honours degrees in acupuncture vanish, Now a big chain of shops selling Chinese herbs and acupuncture has gone into administration. It seems that, at last, people are getting fed up with being conned out of their hard-earned money |
 Herbmedic Barking |
A local newspaper, The North Herts Comet reported thus.
Customers of Herbmedic, which trades under the name Herbs and Acupuncture, on Queensway in Stevenage have been left counting the cost after shelling out hundreds of pounds for treatment they never received.
The company, which has practices across the country, is now in the hands of receivers, Macintyre Hudson.
Sandra Emery, of The Paddocks in Stevenage, paid £350 for 10 treatment sessions, but only received one before the practice closed.
She said: “A standard course of treatment is 10 sessions, so most customers will have bought this package.
Claudia Gois, of Walden End in Stevenage, paid £240 for 12 treatment sessions but only received four before the practice closed.
She said: “I went there on Friday and it was closed. There was no warning or anything.
“I got in touch with head office and they said it’s very unlikely I will get money back.
This report was on 1st April, The company’s web site shows no sign of any problems, In fact they are still advertising jobs. So was this an April Fool joke?
No it wasn’t. A visit to Companies House soon settled the matter. The whole company is insolvent, as of 27 March 2009..
Download the whole administration notice and the company report.
Criticisms of Herbmedic
This chain of shops was investigated by the BBC’s
Inside Out programme. (September 25th 2006).
“We sent an undercover reporter to branches of the Herbmedic chain in southern England.
On each occasion, the reporter claimed to be suffering from tiredness and was prescribed herbal remedies after a consultation lasting less than five minutes.
The herbalists, who describe themselves as “doctors”, didn’t ask any questions about the patient’s medical history or take any notes.”
This is so bad that even Andrew Fowler, a past President of the Register of Chinese Herbal Medicine, described it as “malpractice”.
“Herbmedic has been investigated by the authorities in the past.
In 2002, trading standards officers prosecuted the branch in Southampton for selling herbal remedies with 26 times the permitted legal limit of lead.
And in October 2003, the Advertising Standards Authority banned Herbmedic from describing its practitioners as “doctors”.
Despite the ban, all three of the stores visited by Inside Out referred to the herbalist as the doctor.”
See also the BBC report Herbalists’ customers ‘at risk’, and a report in the Sunday Times, Herbmedic accused of high-pressure selling.
Read the Advertising Standards report. Seven different complaints against Herbmedic were upheld.
This is entirely consistent with my own experience. I went into one of their shops and asked about a cure for diabetes (hoping the be able to refer them to Trading Standards, but the young lady behind the counter had such a poor grasp of English that her reply was incomprehensible. She just kept trying to push me into having a consultation with “the doctor” who appeared to speak no English at all. I left.
The chequered history of Herbmedic
The company that his just gone into administration is Herbmedic Centre Ltd. It has been in existence for only two years. Its predecessor, known simply as Herbmedic, was dissolved on 13 March 2007, Companies House said
Company Filing History Type Date Description Order GAZ2(A) 13/03/2007 FINAL GAZETTE: DISSOLVED VIA VOLUNTARY STRIKE-OFF GAZ1(A) 28/11/2006 FIRST GAZETTE NOTICE FOR VOLUNTARY STRIKE-OFF 652a 16/10/2006 APPLICATION FOR STRIKING-OFF
Another Chinese medicine chain seems to be having a few problems too
Harmony Medical Distribution Ltd (“specialists in acupuncture and holistic medicine”) seems to be still in business(web site here), but several very similar companies have been dissolved, Harmomy Medics Ltd (dissolved 19 Sep 2006) ,, Harmony Medical Services (UK ) Ltd. (dissolved 6 May 2008) and Harmony Medical Services Ltd (dissolved 17 Oct 2008)
Given this history of companies that dissolve every couple of years and then mysteriously reincarnate with a slightly different name, one wonders if this really is the end of herbmedic, or it is just a device for shedding bad debts. Is this just another “pre-pack administration“?
Watch this space for more.
What’s the latest evidence on acupuncture anyway?
A correspondent drew my attention to the 2009 Annual Evidence Update on acupuncture complied by the NHS Complementary and Alternative Medicine Specialist Library. This includes no fewer than 56 systematic reviews and meta-analyses. Although the reviews are complied by alternative medicine sympathisers, they seem mostly to be pretty fair. Well apart form one thing.
Almost all of the reviews fail to come up with any positive evidence that acupuncture works well enough to be clinically useful. Only two come close, and they are the two singled out as “editor’s picks”. Perhaps that’s not entirely surprising given that the editor is Dr Mike Cummings.
Again and again, the results are inconclusive: #8 is pretty typical
Acupuncture for tension-type headache: a meta-analysis of randomized, controlled trials.
This meta-analysis suggests that acupuncture compared with sham for tension-type headache has limited efficacy for the reduction of headache frequency. There exists a lack of standardization of acupuncture point selection and treatment course among randomized, controlled trials. More research is needed to investigate the treatment of specific tension-type headache subtypes.
Vast effort and a lot of money is being put into trials, yet there are very few (if any) positive results. Very often there are no results at whatsoever. All we hear, again and again, is “more research is needed”.
At some point someone will have to decide it is all a charade and start to spend time and money on investigating things that are more promising.
Follow-up
A correspondent checked with Companies House to discover more about two of the directors of Herbmedic, Mr. Li Mao and Mr Xiao Xuan Chen. They have a chequered history indeed. [download the complete list]
Mr. Li Mao is, or has been, on the board of 31 different companies. Of these 6 are active, 5 are in administration, 14 were dissolved, 4 were liquidated and 2 are active with proposal to strike off. Not only is Her Medic centre Ltd in administration, but so is Dr China (UK) Ltd, and Great Chinese Herbal Medicine Ltd
With record like that, my correspondent wonders whether they should be disqualified.
The Nutrition Society is the interim professional body for nutrition. It seems that, unlike so many ‘regulatory bodies’, it may actually take its responsibilities seriously. The following announcement has appeared on their web site.
|
The UK Voluntary Register of Nutritionists acts to protect the public and the reputation the nutrition profession On March 4th 2009, a Fitness to Practice Panel was convened to consider an allegation against a registrant, Dr Ann Walker, that her fitness to practise was impaired. The panel considered whether the registrant, in advocating the use of a web based personal nutritional profiling service had complied with the Code of Ethics’ clause 3: This expects all registered nutritionists to “maintain the highest standards of professionalism and scientific integrity”. In particular, the panel considered whether the registrant showed “knowledge, skills and performance of high quality, up-to-date, and relevant to their field of practice”, in keeping with the Statement of Professional Conduct (para 9). The Panel accepted the allegation of impaired fitness to practice. Mindful of its duty to protect the public, it recommended that Dr Walker be removed from the register. Dr Walker has a right of appeal. |
Well. well, this must be none other than the Dr Ann Walker who caused UCL,and me, such trouble a few years ago. And just because I described her use of the word “blood cleanser” as gobbledygook. She has appeared a few times on this blog.
- Nutribollocks: antioxidants useless, some are dangerous
- Red clover, herbal spin and vested interests
- The fallout from DC’s de-excommunication
- So what is a “blood cleanser”? Quinion speaks.
- Herbal medicines fail test
- Nutriprofile: useful aid or sales scam?
- She even gets a brief mention at Boots reaches new level of dishonesty with CoQ10 promotion
Presumably the “web-based personal nutritional profiling service” that is referred to is Nutriprofile, on which, with the help of a dietition, we had a bit of fun a while ago. However ideal your diet it still recommended at the end of the questionnaire that you should buy some expensive supplements. Read about it at Nutriprofile: useful aid or sales scam?
I have no idea who lodged the complaint (but it wasn’t me).
It is interesting to compare the high standards of the Nutrition Society with the quite different standards of BANT (the British Association for Applied Nutrition and Nutritional Therapy). They bill themselves as the “Professional Body for Nutritional Therapists”. Nutritional therapists are those fantasists who believe you can cure any ill by buying some supplement pills. Their standards can be judged by, for example, BANT ethics code: BANT nutritional therapists are allowed to earn commission from selling pills and tests.
It seems Dr Ann Walker may have joined the wrong society.
Follow-up
On 24 July 2006, I sent a request to the University of Central Lancashire (UCLAN), under the Freedom of Information Act (2000) I asked to see the teaching materials that were used on their BSc Homeopathy course. The request was refused, citing the exemption under section 43(2) of the Act (Commercial Interests).
Two internal reviews were then held. These reviews upheld and the original refusal on the grounds of commercial interests, Section 43(3), and additionally claimed exemption under Section 21 “that is reasonably accessible to applicants by other means (upon the payment of a fee)….i.e. by enrolling on the course….”
In 21 October 2006 I appealed to the Office of the Information commisioner. (The”public authority” means UCLAN, and “the complainant” is me.)
|
“The complainant specifically asked the Commissioner to consider the application of section 43(2) to the course materials he had requested. The main thrust of his argument in this regard was that the public authority could not be considered a ‘commercial organisation’ for the purposes of the Act, and that the public authority had confused ‘commercial interests’ with ‘financial interests’. He however added that if the Commissioner decided section 43(2) was correctly engaged, then it was in the public interest to order disclosure.” |
In May 2008, my appeal got to the top of the pile, and on 30th March 2009 a judgement was delivered. In all respects but one trivial one, the appeal was upheld. In future universities will not be able to refuse requests for teaching materials.
The Decision Notice is on the web site of the Office of the Information Commissioner, [or download pdf file].
This whole thing has taken so long that the course at which it was aimed has already closed its doors last August (and blamed that, in part, on the problems caused by the Freedom of Information Act). UCLAN also announced a review of all its alternative medicine activities (and asked me to give evidence to it). That review is due to report its findings any time now.
Tha particular course that prompted the request is no longer the point. What matters is that all the usual exemptions claimed by universities have been ruled invalid. Here are a few details
What the decision notice says (the short version)
The full text of the Act is here.
The following three exemptions were judged NOT to apply the requests for university teaching materials. I’ll quote some bits from the Decision Notice.
Section 21 provides that –
“Information which is reasonably accessible to the applicant otherwise than under section 1 is exempt information.”
|
34. The public authority’s argument suggests that the requested information is reasonably accessible to the complainant if he enrols as a student on the course, and is therefore not accessible to him by any other means outside the Act unless he decides to make a total payment of £9,345 as a combined payment of three years tuition fees. 40. The Commissioner therefore finds that the public authority incorrectly applied the exemption contained at section 21 of the Act. |
Section 42(2) provides that –
“Information is exempt information if its disclosure under this Act would, or would be likely to, prejudice the commercial interests of any person (including the public authority holding it).”
|
71. The Commissioner therefore finds that the section 43(2) was incorrectly engaged by virtue of the fact that the public authority’s ability to recruit students is not a commercial interest within the contemplation of section 43(2). 76. In addition to his finding on commercial interests the Commissioner finds that section 43(2) would in any case not be engaged as the likelihood of prejudice to the public authority’s ability to recruit students as a result of disclosure under the Act is no more than the likelihood of prejudice resulting from the availability of the course materials to students already enrolled on the course. |
Section 36(2)C provides that –
“Information to which this section applies is exempt information if, in the reasonable opinion of a qualified person, disclosure of the information under this Act-
(c) would otherwise prejudice, or would be likely otherwise to prejudice, the effective conduct of public affairs
| 98. For the reasons set out above, the Commissioner finds that section 36(2)(c) is not engaged as he does not accept the opinion of the qualified person is an objectively reasonable one. He does not find that disclosure would be likely to prejudice the effective conduct of public affairs. |
Section 41(1) provides that –
“Information is exempt information if-
(a) it was obtained by the public authority from any other person (including another public authority), and
(b) the disclosure of the information to the public (otherwise than under this Act) by the public authority holding it would constitute a breach of confidence actionable by that or any other person.”
| 56. The Commissioner therefore finds the public authority correctly applied the exemption contained at section 41 to the case studies listed in Annex A. In the Commissioner’s view, even though the patients would not be identifiable if the case studies were disclosed, this disclosure would still be actionable by the patients. |
The Decision
|
99. The Commissioner finds that section 41 is engaged 100. He however finds that the exemptions at sections 21, 43(2), and 36(2)(c) are not engaged. 101. The Commissioner therefore finds the public authority in breach of; • Sections 1(1)(b) and 10(1), because it failed to disclose the remainder of the course materials (i.e. excluding the case studies) to the complainant within 20 working days. • Section 17(1), because it did not specify in its refusal notice that it was also relying on sections 41 and 36(2)(c). |
Steps required
|
103. The Commissioner requires the public authority to take the following steps to ensure compliance with the Act: • Disclose all the course materials for the BSc (Hons) in Homeopathy apart from the case studies listed in Annex A of this Notice. 104. The public authority must take the steps required by this notice within 35 calendar days of the date of this notice. |
Follow-up
In March 2007 I wrote a piece in Nature on Science degrees without the science. At that time there were five “BSc” degrees in homeopathy. A couple of weeks ago I checked the UCAS site for start in 2009, and found there was only one full “BSc (hons)” left and that was at Westminster University.
Today I checked again and NOW THERE ARE NONE.
A phone call to the University of Westminster tonight confirmed that they have suspended entry to their BSc (Hons) homeopathy degree.
They say that they have done so because of “poor recruitment”. It was a purely financial decision. Nothing to do with embarrasment. Gratifying though it is that recruits for the course are vanishing, that statement is actually pretty appalling It says that the University of Westminster doesn’t care whether it’s nonsense, but only about whether it makes money.
Nevertheless the first part of this post is not entirely outdated before it even appeared, because homeopathy will still be taught as part of Complementary Therapies. And Naturopathy and “Nutritional Therapy” are still there..
According to their ‘School of Integrated Health‘, “The University of Westminster has a vision of health care for the 21st Century”. Yes, but it is what most people would call a vision of health care in the 18th century.
The revelation that the University of Westminster teaches that Amethysts emit high Yin energy caused something of a scandal.
Since then I have acquired from several sources quite a lot more of their teaching material, despite the fact that the university has refused to comply with the Freedom of Information Act.
In view of the rather silly internal review conducted by Westminster’s Vice-Chancellor, Professor Geoffrey Petts, this seems like a good moment to make a bit more of it public,
I think that revelation of the material is justified because it is in the
public interest to know how the University if Westminster is spending taxpayers’ money. Another motive is to defend the reputation of the post-1992 universities. I have every sympathy with the ex-polytechnics in their efforts to convert themselves into universities. In many ways they have succeeded. That effort
is impeded by teaching mystical versions of medicine.
If the University of Westminster is being brought into disrepute, blame its vice-chancellor, not me.
Homeopathic spiders
Here are a few slides from a lecture on how good spider venom is for you. It is from Course 3CTH502 Homeopathic Materia Medica II. No need to worry though, because they are talking about homeopathic spider venom, so there is nothing but sugar in the pills. The involvement of spiders is pure imagination. No more than mystical gobbledygook.
You are in hurry, or play with your fingers? You need spider venom pills (that contain no spider venom).
You break furniture? Time goes too fast for you? Try the tarantula-free tarantula pills.
You are preoccupied with sex? You play with ropes? What you need is Mygale (which contains no Mygale)
Much more seriously, the same sugar pills are recommended for serious conditions, chorea, ‘dim sight’, gonorrhoea, syphilis and burning hot urine.
This isn’t just preposterous made-up stuff. It is dangerous.
There is a whole lot more fantasy stuff in the handouts for Homeopathy Materia Medica II (3CTH502). Here are a couple of examples.
Aurum metallicum (metallic gold) [Download the whole handout]
Affinities MIND, VASCULAR SYSTEM, Nerves, Heart, Bones, Glands, Liver, Kidneys, RIGHT SIDE, Left side.
Causations Emotions. Ailments from disappointed love and grief, offence or unusual responsibility, abuse of mercury or allopathic drugs.
Aurum belongs to the syphilitic miasm but has elements of sycosis (Aur-Mur).
Potassium salts are the subject of some fine fantasy, in “The Kali’s” [sic]. (there is much more serious stuff to worry about here than a few misplaced apostrophes.). [Download the whole handout]
“The radioactive element of potassium emits negative electrons from the atom nucleus and is thought to be significant in the sphere of cell processes especially in relation to functions relating to automatism and rhythmicity.”
“Kali people are very conscientious with strong principles. They have their rules and they stick to them, ‘a man of his word’.”
“Potassium acts in a parasympathetic way, tending towards depression”
“They [“Kali people=] are not melancholic like the Natrum’s but rather optimistic.”
Radioactive potassium is involved in automaticity? Total nonsense.
Where is the science?
Yes, it is true that the students get a bit of real science. There isn’t the slightest trace that I can find of any attempt to resolve the obvious fact that what they are taught in the science bits contradict directly what they are told in the other bits. Sounds like a recipe for stress to me.
They even get a bit of incredibly elementary statistics. But they can’t even get that right. This slide is from PPP – Res Quant data analysis.
“Involves parameters and/or distributions”. This has no useful meaning whatsoever, that I can detect.
“Tests hypotheses when population distributions are skewed”. Well yes, though nothing there about forms of non-Gaussian properties other than skew, nothing about normalising transformations, and nothing about the Central Limit theorem.
“Ranks data rather than the actual data itself”. This is plain wrong. Randomisation tests on the original data are generally the best (uniformly most powerful) sort of non-parametric test. It seems to have escaped the attention of the tutor that ranking is a short-cut approximation that allowed tables to be constructed, before we had computers.
The students are told about randomised controlled trials. But amazingly in the lecture PPP-RCTs, the little matter of blinding is barely mantioned. And the teacher’s ideas about randomisation are a bit odd too.
Sorry, but if you fiddle the randomisation, no amount of “careful scrutiny” will rescue you from bias.
An Introduction to Naturopathic Philosophy
Naturopathy is just about as barmy as homeopathy. You can see something about it at the University of Wales. How about this slide from Westminster’s An Introduction to Naturopathic Philosophy.
So if you get tuberculosis, it isn’t caused by Mycobacterium tuberculosis? And the symptoms are “constructive”? So you don;t need to do anything. It’s all for the best really.
This isn’t just nonsense. It’s dangerous nonsense.
Traditional Chinese Medicine
Ever wondered what the mysterious “Qi” is? Worry no more. All is explained on this slide.
It means breath, air, vapour, gas, energy, vitalism. Or perhaps prana? Is that quite clear now?
What can we make of this one? Anyone can see that the description is barely written in English and that vital information is missing (such as the age of the woman). And it’s nonsense to suggest that “invasion of cold” (during keyhole surgery!) would cause prolonged constriction of blood vessels (never mind that it would “consume yang qi”). Not being a clinician, I showed it to an oncologist friend. He said that it was impossible to tell from the description whether the problem was serious or not, but that any abdominal pain should be investigated properly. There isn’t anything here about referral for proper investigation. Just a lot of stuff about ginger and cinnamon. Anyone who was taught in this way could be a real danger to the public. It isn’t harmless nonsense It’s potentially harmful nonsense.
And finally, it’s DETOX
Surely everyone knows by now that ‘detox’ is no more than a marketing word? Well not at the University of Westminster. They have a long handout that tells you all the usual myths and a few new ones.
It is written by Jennifer Harper-Deacon, who describes herself modestly, thus.
Jennifer Harper-Deacon is a qualified and registered Naturopath and acupuncturist who holds a PhD in Natural Health and MSc in Complementary Therapies. She is a gifted healer and Reiki Master who runs her own clinic in Surrey where she believes in treating the ‘whole’ person by using a combination of Chinese medicine and naturopathic techniques that she has qualified in, including nutritional medicine, Chinese and Western herbalism, homoeopathy, applied kinesiology, reflexology, therapeutic massage, aromatherapy and flower remedies.
It seems that there is no limit on the number of (mutually incompatible) forms of nuttiness that she believes. Here are a few quotations from her handout for Westminster students.
“Detoxification is the single most powerful tool used by natural health professionals to prevent and reverse disease”
What? To “prevent and reverse” malaria? tuberculosis? Parkinson’s disease? AIDS? cancer?
“When you go on to a raw food only diet, especially fruit, the stored toxins are brought up from the deep organs such as the liver and kidneys, to the superficial systems of elimination.”;
Very odd. I always though that kidneys were a system of elimination.
“The over-use and mis-use of antibiotics has weakened the body’s ability to attack and destroy new strains of resistant bacteria, virulent viruses, which have led to our immune system becoming compromised.”
Certainly over-use and mis-use are problems. But I always thought it was the bacteria that became resistant.
“The beauty about detoxification therapy is that it addresses the very causative issues of health problems”
That is another dangerous and silly myth. Tuberculosis is not caused by mythical and un-named “toxins”. It is caused by Mycobacterium tuberculosis.
“Naturopathy follows the logic of cause and effect therefore believes that we simply need pure food and water, sunshine, air, adequate rest and sleep coupled with the right amount of exercise for health.”
Try telling that to someone with AIDS.
“Colon cleansing is one of the most important parts of any detoxification programme.”
The strange obsession with enemas in the alternative world is always baffling.
“Frankincense: holds the capacity to physically strengthen our defence system and can rebuild energy levels when our immune system is weak. Revered as a herb of protection, frankincense can also strengthen our spiritual defences when our Wei qi is low, making us more susceptible to negative energies. This calming oil has the ability to deepen the breath, helping us to let go of stale air and emotions, making it ideal oil to use inhale prior to meditating.”
This is so much hot air. There is a bit of evidence that frankincense might have some anti-inflammatory action and that’s it.
But this has to be my favourite.
“Remember when shopping to favour fruits and vegetables which are in season and locally grown (and ideally organic) as they are more vibrationally compatible with the body.”
Locally grown vegetables are “more vibrationally compatible with the body”? Pure mystical gobbledygook. Words fail me.
OK there’s a whole lot more, but that will do for now.
It’s good that Westminster is shutting down its Homeopathy BSc, but it seems they have a bit further to go.
The Health Professions Council (HPC) is yet another regulatory quango.
The HPC’s strapline is
|
 |
At present the HPC regulates; Arts therapists, biomedical scientists, chiropodists/podiatrists, clinical scientists, dietitians, occupational therapists, operating department practitioners, orthoptists, paramedics, physiotherapists, prosthetists/orthotists, radiographers and speech & language therapists.
These are thirteen very respectable jobs. With the possible exception of art therapists, nobody would doubt for a moment that they are scientific jobs, based on evidence. Dietitians, for example, are the real experts on nutrition (in contrast to “nutritional therapists” and the like, who are part of the alternative industry). That is just as well because the ten criteria for registration with the HPC say that aspirant groups must have
“Practise based on evidence of efficacy”
But then came the Pittilo report, about which I wrote a commentary in the Times, and here, A very bad report: gamma minus for the vice-chancellor, and here.
Both the Pittilo report, the HPC, and indeed the Department of Health itself (watch this space), seem quite unable to grasp the obvious fact that you cannot come up with any sensible form of regulation until after you have decided whether the ‘therapy’ works or whether it is so much nonsense.
In no sense can “the public be protected” by setting educational standards for nonsense. But this obvioua fact seems to be beyond the intellectual grasp of the quangoid box-ticking mentality.
That report recommended that the HPC should regulate also Medical Herbalists, Acupuncturists and Traditional Chinese Medicine Practitioners. Even more absurdly, it recommended degrees in these subjects, just at the moment that those universities who run them are beginning to realise that they are anti-scientific subjects and closing down degrees in them.
How could these three branches of the alternative medicine industry possibly be eligible to register with the HPC when one of the criteria for registration is that there must be “practise based on evidence of efficacy”?
Impossible, I hear you say. But if you said that, I fear you may have underestimated the capacity of the official mind for pure double-speak.
The HPC published a report on 11 September 2008, Regulation of Medical Herbalists, Acupuncturists and Traditional Chinese Medicine Practitioners.
The report says
1. Medical herbalists, acupuncturists and traditional Chinese medicine practitioners should be statutorily regulated in the public interest and for public safety reasons.
2. The Health Professions Council is appropriate as the regulator for these professions.
3. The accepted evidence of efficacy overall for these professions is limited, but regulation should proceed because it is in the public interest.
But the last conclusion contradicts directly the requirement for “practise based on evidence of efficacy”. I was curious about how this contradiction
could be resolved so I sent a list of questions. The full letter is here.
The letter was addressed to the president of the HPC, Anna van der Gaag, but with the customary discourtesy of such organisations, it was not answered by her but by Michael Guthrie, Head of Policy and Standards He said
“Our Council considered the report at its meeting in July 2008 and decided that the regulation of these groups was necessary on the grounds of public protection. The Council decided to make a recommendation to the Secretary of State for Health that these groups be regulated.
http://www.hpc-uk.org/assets/documents/100023FEcouncil_20080911_enclosure07.pdf “.
This, of course, doesn’t answer any of my questions. It does not explain how the public is protected by insisting on formal qualifications, if the qualifications
happen to teach mythical nonsense. Later the reply got into deeper water.
“I would additionally add that the new professions criteria are more focused on the process and structures of regulation, rather than the underlying rationale for regulation – the protection of members of the public. The Council considered the group’s report in light of a scoring against the criteria. The criteria on efficacy was one that was scored part met. As you have outlined in your email (and as discussed in the report itself) the evidence of efficacy (at least to western standards) is limited overall, particularly in the areas of herbal medicines and traditional Chinese medicine. However, the evidence base is growing and there was a recognition in the report that the individualised approach to practice in these areas did not lend themselves to traditional RCT research designs.”
Yes, based on process and structures (without engaging the brain it seems). Rather reminiscent of the great scandal in UK Social Services. It is right in one respect though.
The evidence base is indeed growing, But it is almost all negative evidence. Does the HPC not realise that? And what about “at least by Western standards”? Surely the HPC is not suggesting that UK health policy should be determined by the standards of evidence of Chinese herbalists? Actually it is doing exactly that since its assessment of evidence was based on the Pittilo report in which the evidence was assessed (very badly) by herbalists.
One despairs too about the statement that
“there was a recognition in the report that the individualised approach to practice in these areas did not lend themselves to traditional RCT research designs”
Yes of course the Pittilo report said that, because it was written by herbalists! Had the HPC bothered to read Ben Goldacre’s column in the Guardian they would have realised that there is no barrier at all to doing proper tests. It isn’t rocket science, though it seems that it is beyond the comprehension of the HPC.
So I followed the link to try again to find out why the HPC had reached the decision to breach its own rules. Page 10 of the HPC Council report says
3. The occupation must practise based on evidence of efficacy This criterion covers how a profession practises. The Council recognizes the centrality of evidence-based practice to modern health care and will assess applicant occupations for evidence that demonstrates that:
- Their practice is subject to research into its effectiveness. Suitable evidence would include publication in journals that are accepted as
learned by the health sciences and/or social care communities- There is an established scientific and measurable basis for measuring outcomes of their practice. This is a minimum—the Council welcomes
evidence of there being a scientific basis for other aspects of practice and the body of knowledge of an applicant occupation- It subscribes to the ethos of evidence-based practice, including being open to changing treatment strategies when the evidence is in favour
of doing so.
So that sounds fine. Except that research is rarely published in “journals that are accepted as learned by the health sciences”. And of course most of the good evidence is negative anyway. Nobody with the slightest knowledge of the literature could possibly think that these criteria are satisfied by Medical Herbalists, Acupuncturists and Traditional Chinese Medicine Practitioners.
So what does the HPC make of the evidence? Appendix 2 tells us. It goes through the criteria for HPS registration.
“Defined body of knowledge: There is a defined body of knowledge, although approaches to practice can vary within each area.”
There is no mention that the “body of knowledge” is, in many cases, nonsensical gobbledygook and, astonishingly this criterion was deemed to be “met”!.
This shows once again the sheer silliness of trying to apply a list of criteria without first judging whether the subject is based in reality,
Evidence of efficacy. There is limited widely accepted evidence of efficacy, although this could be partly explained by the nature of the professions in offering bespoke treatments to individual patients. This criterion is scored part met overall.
Sadly we are not told who deemed this criterion to be “part met”. But it does say that “This scoring has been undertaken based on the information outlined in the [Pittilo] report”. Since the assessment of evidence in that report was execrably bad (having been made by people who would lose their jobs if
they said anything negative). it is no wonder that the judgement is overoptimistic!
Did the HPC not notice the quality of the evidence presented in the Pittilo report? Apparently not. That is sheer incompetence.
Nevertheless the criterion was not “met”, so they can’t join HPC, right? Not at all. The Council simply decided to ignore its own rules.
On page 5 of the Council’s report we see this.
The Steering Group [Pittilo] argues that a lack of evidence of efficacy should not prevent regulation but that the professions should be encouraged and funded to strengthen the evidence base (p.11, p. 32, p.34).
This question can be a controversial area and the evidence base of these professions was the focus of some press attention following the report’s publication. An often raised argument against regulation in such circumstances is that it would give credibility in the public’s eyes to treatments that are not proven to be safe or efficacious.
This second point is dead right, but it is ignored. The Council then goes on to say
In terms of the HPC’s existing processes, a lack of ‘accepted’ evidence of efficacy is not a barrier to producing standards of proficiency or making decisions about fitness to practise cases.
This strikes me as ludicrous, incompetent, and at heart, dishonest.
There will be no sense in policy in this area until the question of efficacy is referred to NICE. Why didn’t the HPC recommend that? Why has it not been done?
One possible reason is that I discovered recently that, although there are two scientific advisers in the Department of Health,. both of them claim that it is “not their role” to give scientific advice in this area. So the questions get referred instead to the Prince of Wales Foundation. That is no way to run a ship.
The fact of the matter is that the HPC, like so many other regulatory agencies, fails utterly to protect the public from fraudulent and incompetent practitioners. In fact it actually protects them, in the same way that the financial ‘regulators’ protected fraudulent bankers. They all seem to think that ticking boxes and passing exams is an effective process. Even if the exams require you to memorise that amethysts “emit high Yin energy so transmuting lower energies and clearing and aligning energy disturbance as all levels of being”.
Recently I wrote a piece for the National Health Executive (“the Independent Journal for Senior Health Service Managers”), with the title Medicines that contain no medicine and other follies
In the interests of what journalists call balance (but might better be called equal time for the Flat Earth Society), an article appeared straight after mine, Integrating Homeopathy into Primary Care. It was by Rachel Roberts “Research consultant for the Society of Homeopaths”.
This defence was so appalling that I sent them a response (after first doing a bit of checking on its author). To my surprise, they published the response in full [download pdf of printed version]. Their title was
As always, the first step is to Google the author, to find out a bit more. It seems that Rachel Roberts runs a business Integrated Homeopathic Training. (a financial interest that was not mentioned in her article). She will sell you flash cards (‘Matmedcards’) for £70 (+£9 p&p) for 120 cards (yes, seriously). The card for Conium maculatum is remarkable. It says on the reverse side
Yes, it says (my emphasis)
“The poison used to execute Socrates. No 1 remedy for scirrhous breast cancer. Esp after blow to the breast”
No doubt she would claim that the word “remedy” was a special weasel word of homeopaths that did not imply any therapeutic efficacy. But its use in this context seems to me to be cruel deception, even murderous. It also appears to breach the Cancer Act 1939, as well as the Unfair Trading laws.
I asked the Bristol Trading Standards Office, and got a reply remarkably quickly. It ended thus.
“. . . . the use of the card for “hemlock” as an example amounts to advice in connection with the treatment (of cancer)”. I will initially write to IHT and require that they remove this, and any other, reference to cancer treatment from their website.
When I checked again a couple of weeks later, the hemlock sample card had been been replaced by one about chamomile (it is described as the opium of homeopathy. Luckily the pills contain no opium (and no chamomile either) or that would be breaking another law. Bafflingly, it is not (yet) against the law to sell pills that contain no trace of the ingredient on the label, if they are labelled ‘homeopathic’.
Presumably the packs still contain a claim to cure cancer. And what is said in the privacy of the consulting room will never be known.
Political correctness is a curious thing. I felt slightly guilty when I reported this breach of the Cancer Act. It felt almost sneaky. The feeling didn’t last long though. We are talking about sick people here.
It isn’t hard to imagine a desperate woman suffering from cancer reading that Ms Roberts knows the “No 1 remedy for scirrhous breast cancer”. She might actually believe it. She might buy some hemlock pills that contain no hemlock (or anything else). She might die as a result. It is not a joke. It is, literally, deadly serious.
It is also deadly serious that the Department of Health and some NHS managers are so stifled by political correctness that they refer to homeopaths as “professionals” and pay them money.
Ms Roberts, in her article, is at pains to point out that
|
 |
Well it is already well known that the the Code of Ethics of the Society of Homeopaths is something of a joke. This is just one more example.
The Code of Ethics, para 72 says homeopaths have a legal obligation
“To avoid making claims (whether explicit or implied; orally or in writing) implying cure of any named disease.”
Like, perhaps, claiming to have the “No 1 remedy for scirrhous breast cancer”?
Obviously voluntary self-regulation isn’t worth the paper it’s written on.
You don’t need to go to her web site to find “claims . . . implying cure of any named disease”. In her article she says
“Conditions which responded well to homeopathy included childhood eczema and asthma, migraine, menopausal problems, inflammatory bowel disease, irritable bowel syndrome, arthritis, depression and chronic fatigue syndrome.”
No doubt they will say that the claim that asthma and migraine “responded well” to their sugar pills carries no suggestion that they can cure a named disease. And if you believe that, you’ll believe anything.
I have to say that I find Ms Robert’s article exceedingly puzzling. It comes with 29 references, so it looks, to use Goldacre’s word, ‘sciencey’. If you read the references, and more importantly, know about all the work that isn’t referred to, you see it is the very opposite of science. I see only two options.
Either it is deliberate deception designed to make money, or it shows, to a mind-boggling extent, an inability to understand what constitutes evidence.
The latter, more charitable, view is supported by the fact that Ms Roberts trots out, yet again, the infamous 2005 Spence paper, as though it constituted evidence for anything at all. In this paper 6544 patients at the Bristol Homeopathic Hospital were asked if the felt better after attending the out-patient department. Half of them reported that the felt ‘better’ or ‘much better’. Another 20% said they were ‘slightly better’ (but that is what you say to be nice to the doctor). The patients were not compared with any other group at all. What could be less surprising than that half of the relatively minor complaints that get referred for homeopathy get ‘better or much better’ on their own?
This sort of study can’t even tell you if homeopathic treatment has a placebo effect, never mind that it has a real effect of its own. It is a sign of the desperation of homeopaths that they keep citing this work.
Whatever the reason, the conclusion is clear. Never seek advice from someone who has a financial interest in the outcome. Ms Roberts makes her living from homeopathy. If she were to come to the same conclusion as the rest of the world, that it is a placebo and a fraud, her income would vanish. It is asking too much of anyone to do that.
| This is the mistake made time and time again by the Department of Health and by the NHS. The Pittilo report does the same thing The execrably bad assessment of evidence in that report is, one suspects, not unrelated to the fact that it was done entirely by people who would lose their jobs if they were to come to any conclusion other than their treatments work. |  |
At present , the regulation of alternative medicine is chaotic because the government and the dozen or so different quangos involved are trying to regulate while avoiding the single most important question – do the treatments work?
They should now grasp that nettle and refer the question to NICE.
Follow-up
BSc courses in homeopathy are closing. Is it a victory for campaigners, or just the end of the Blair/Bush era?
The Guardian carries a nice article by Anthea Lipsett, The Opposite of Science (or download pdf of print version).
Dr Peter Davies, dean of Westminster’s school of integrated health, says
“he welcomes the debate but it isn’t as open as he would like.”
Well you can say that again. The University of Westminster has refused to send me anything much, and has used flimsy excuses to avoid complying with the Freedom of Information Act. Nevertheless a great deal has leaked out. Not just amethysts emit hig Yin energy, but a whole lot more (watch this space). Given what is already in the public, arena, how can they possibly say things like this?
“Those teaching the courses insist they are academically rigorous and scientific.”
There’s another remark from an unlikely source that I can agree with too. George Lewith, of Southampton University and Upper Harley Street, is quoted as saying
“The quality of degrees is an open joke . . . ”
Whatever next? [Note: Lewith told me later that he was quoted out of context by the Guardian, so it seems that after all he is happy with the courses. So sadly I have to withdraw the credit that I was giving him].
The article emphasises nicely the view that universities that run BSc degrees in things that are fundamentally the opposite of science are deceiving young people and corrupting science itself.
Professor Petts of Westminster seems to think that the problem can be solved by putting more science into the courses The rest of the world realises that as soon as you apply science to homeopathy or naturopathy, the whole subject vanishes in a puff of smoke, I fear that Professor Petts will have to do better,
“He [DC] believes the climate is starting to change after the Bush/Blair era where people believed in things because they wished they were true. “This has been going on for a generation and it’s about time for a swing in the other direction,” he suggests.”
Well, one can always hope.
Follow-up
It is almost six months now since I posted Quackery creeps into good universities too -but through Human Resources. One example given there was the University of Leicester. This is an excellent university. It does first class research and it was the alma mater of the incomparable David Attenborough who has done more than anyone to show us the true beauty and wonder of the natural world.
Nevertheless, their well-meaning occupational health department had a section about “complementary therapies” that contained a lot of statements that were demonstrably untrue. They even recommended the utterly outrageous SCENAR device. So I pointed this out to them, and I had a quick and sympathetic response from their HR director.
But three months later, nothing had changed. Every now and then, I’d send a polite reminder, but it seemed the occupational health staff were very wedded to their quackery. The last reminder went on 6th February, but this time I copied it to Leicester’s vice-chancellor. This time it worked. There is still a link to Complementary Therapies on the Wellbeing site, but if you click on it, this is what you see.
Complementary therapiesSome employees may have an interest in complementary therapies such as acupuncture, yoga, Indian head massage, Reiki, sports & remedial massage, reflexology and hypnotherapy. If you have an interest in any of these, Staff Counselling can happily provide details of practitioners in the local area. Some of these practitioners offer discounts from their normal rates for University of Leicester staff. However, the University of Leicester cannot vouch for, or recommend any of these therapies to staff as being effective. We would urge members of staff who believe that such therapies might be effective to contact their GP prior to undertaking any of them. Further, the University of Leicester shall not be liable for any damage of any kind arising out of or related to the services of any complementary therapists or treatments listed here. If you would like further information, please contact Chris Wilson at: staffcounsel&welfare@le.ac.uk or telephone 1702. |
That’s not bad. Pity it doesn’t say alternative, rather than complementary though. Euphemisms aren’t really helpful.
In fact I have a bit of a problem with “wellbeing” too. It is a harmless word that has been highjacked so that its use now makes one think of mud baths provided by expensive hotels for their rich and gullible customers.
Leicester’s HR director wrote
“Unfortunately an instruction I had given previously had not been fully complied with. I spoke to the manager of the Staff Counselling team on Friday and gave clear instructions as to the content of this site. I had been assured that the offending information had been removed, but found that it had not.
I have now checked the site for myself and can say, with confidence, that all claims for the efficacy of complimentary [sic] therapies have been removed including SCENAR.”
The similarity between quack treatments and religion is intriguing. It seems that the devotion of the occupational health people to their baloney was so great that they wouldn’t take it down even when told to do so. The more irrational the belief, the greater the fervour with which it is defended,
What’s the lesson from this minor saga? It seems that most VCs and many HR people are too sensible to believe in alternative baloney, but that they are a bit too ready to tolerate it, perhaps on grounds of political correctness. Tolerance is a virtue, but lies about health are not in the least virtuous. If you point out that people are saying things for which there isn’t the slightest evidence, they will often respond. Just be prepared to send a few reminders.
It may also be useful to point out that some of the claims made are almost certainly illegal. Even people who care little about evidence of efficacy are impressed by the idea that they might be prosecuted by Trading Standards officers.
Who needs mystical medicine when you have real wonders like these.
(Click the google video logo for a bigger version)
Or try the pitcher plant video, or the Bird of paradise, or the Bower Bird, or the giant Amazon lilies.
Follow-up
From time to time, Private Eye Magazine takes a look at university vice-chancellors (aka presidents/rectors/principals) in its High Principals column.
The current issue (No, 1239, 20, Feb – 5 Mar, 2009) features Professor Geoffrey Petts, vice-chancellor of the University of Westminster,
Well well. Who’d have thought such things were possible?
Follow-up
Notice
I heard, in January 2011, that Barts has a new Dean of Education, and no longer teaches about alternative medicine in the way that has caused so much criticism in the last two years. That’s good news.
What on earth has gone wrong at the Barts and The London School of Medicine and Dentistry (SMD)?
It is not so long ago that I discovered that the very sensible medical students at Barts were protesting vigourously about being forced to mix with various quacks. A bit of investigation soon showed that the students were dead right: see St Bartholomew’s teaches antiscience, but students revolt.
Now it seems that these excellent students have not yet succeeded in educating their own Dr Mark Carroll who, ironically, has the title Associate Dean (Education Quality) in the Centre for Medical Education (SMD), specialising in all aspects of quality assurance in the SMD,
Recently this letter was sent to all medical students. They are so indignant at the way they are being treated, it didn’t take long for a copy of the letter to reach me via a plain brown email.
| Does any medical student have a particular interest in Complementary Medicine? If so, a group at Westminster University would like to contact you (see email message below) with a view to some collaborative work. Further details from Dr Mark Carroll ( m.carroll@qmul.ac.uk ).
|
A student of naturopathy? Does Mark Carroll have the slightest idea what naturopathy is (or pretends to be)? If so, why is he promoting it? If not, he clearly hasn’t done his homework.
You can get a taste of naturopathy in Another worthless validation: the University of Wales and nutritional therapy, or in Nutritional Fairy Tales from Thames Valley University.
It is a branch of quackery that is so barmy that it’s actually banned in some US states. A pharmacist was fined $1 miilion for practising it. But Barts encourages it.
Or read here about the College of Natural Nutrition: bizarre teaching revealed. They claim to cure thyroid cancer with castor oil compresses, and a holder of their diploma was fined £800 000 for causing brain damage to a patient.
I removed the name of the hapless naturopathy student, I have no wish for her to get abusive mail. It isn’t her fault that she has been misled by people who should know better. If you feel angry about this sort of thing then that should be directed to the people who mislead them. The poor student has been misled in to taking courses that teach amethysts emit high yin energy by the University of Westminster’s Vice-chancellor, Professor Geoffrey Petts, But note that Professor Petts has recently set up a review of the teaching of what he must know to be nonsense (though it hasn’t got far yet). In contrast, Dr Carroll appears to be quite unrepentant. He is the person you to whom you should write if you feel indignant.
He claims Barts is "ahead of the game". Which game? Apparently the game of leading medicine back to the dark ages and the High Street quack shop. But, Dr Carroll, it isn’t a game. Sick people are involved.
Dr Carroll is the Associate Dean (Education Quality) in the Centre for Medical Education (SMD), specialising in all aspects of quality assurance in the SMD. This has to be the ultimate irony. It’s true that the Prince of Wales approach to medicine has penetrated slightly into other, otherwise good, medical schools (for example, Edinburgh) but I’m not aware of any other that has gone so far down the road of irrationality as at Barts.
Dr Carroll, I suggest you listen to your students a bit more closely.
You might also listen to President Obama. He has just allocated $1.1 billion “to compare drugs, medical devices, surgery and other ways of treating specific conditions“. This has infuriated the drug industry and far-right talk show host Rush Limbaugh. Doubtless it will infuriate quacks too, if any of it is spent on testing their treatments properly.
Follow-up
This classic was published in issue 1692 of New Scientist magazine, 25 November 1989. Frank Watt was head of the Scanning Proton Microprobe Unit in the Department of Physics at Oxford University. He wrote for the New Scientist an article on Microscopes with proton power.
In the light of yesterday’s fuss about research funding, Modest revolt to save research from red tape , this seemed like a good time to revive Watt’s article. It was written near to the end of the reign of Margaret Thatcher, about a year before she was deposed by her own party. It shows how little has changed.
It is now the fashion for grant awarding agencies to ask what percentage of your time will be spent on the project. I recently reviewed a grant in which the applicants had specified this to four significant figures, They’d
pretended they knew, a year before starting the work that they could say how much time they’d spend with an accuracy better than three hours. Stupid questions evoke stupid answers.
The article gave rise to some political follow-up in New Scientist, here and here.
Playing the game – The art of getting money for research
WELL, that’s it,’ thought the Captain. ‘We have the best ship in the world, the most experienced crew, and navigators par excellence. All we need now is a couple of tons of ship’s biscuits and it’s off to the ends of the world.’ NRC (Nautical Research Council) grant application no MOD2154, September 1761. Applicant: J. Cook. Aim: To explore the world. Requirements: Supplies for a three-year voyage. Total cost: Pounds sterling 21 7s 6d. January 1762: Application MOD2154,rejected by the Tall Ships subcommittee Reasons for rejection: ‘Damn,’ thought the Captain, ‘that’s nailed our vitals to the plank good and NRC grant application no MOD2279, April 1762. Applicant J. Cook. Aim: To explore the world beyond Africa in a clockwise direction and discover a large continent positioned between New Guinea and the South Pole. Requirements: Supplies for a three-year voyage. Total cost: Pounds sterling 21 7s 7d (adjusted for inflation). September 1762: Application MOD2279 rejected by the Castle and Moats subcommittee Reasons for rejection. (a) Why discover a large continent when there are hundreds of castles in Britain. (b) It is highly unlikely that Cook’s ship will fit into the average moat. ‘Damn,’ thought the Captain, ‘shiver me timbers, the application has gone to the wrong committee.’ ‘Come back in six months,’ he told his sturdy crew. Dear NRC, Why did our application MOD2279 go to the Castle and Moats subcommittee instead of the Tall Ships committee? Yours sincerely, James Cook. Dear Mr Cock, The Tall Ships subcommittee has been rationalised, and has been replaced by the Castle and Moats subcommittee and the Law and Order subcommittee. Yours sincerely, NRC. ‘Damn,’ thought the Captain, ‘rummage me topsail, we’ll never get off the ground like this. I had better read up on this new chaos theory.’ In fact the Captain did not do this, but instead took advice from an old sea dog who had just been awarded a grant of Pounds sterling 2 million from the emergency drawbridge fund to research the theory of gravity. NRC Application no MOD2391, January 1763 (to be considered by the Law and Order subcommittee). Applicant J. Cook. Aim: Feasibility study for the transportation of rascals, rogues and vagabonds to a remote continent on the other side of the world. Initial pilot studies involve a three-year return journey to a yet undiscovered land mass called Australia . Requirements: Supplies for a three-year voyage. Total cost: Pounds sterling 21 7s 8d (again adjusted for inflation). Dear Mr Coko, We are pleased to inform you that your application MOD2391 has been successful. Unfortunately, due to the financial crisis at the moment, the subcommittee has recommended that funding for your three-year round trip to Australia be cut to 18 months. Yours sincerely, NRC. |
Follow-up
A letter to Mr Darwin A correspondent draws my attention to another lovely piece from the EMBO Reports Journal )Vol 10, 2009), by Frank Gannon. [Download the pdf.]
Dear Dr Darwin
” . . . . In a further comment, referee three decries your descriptive approach, which leaves the task of explaining the ‘how’ to others. His/her view is that any publications resulting from your work will inevitably be acceptable only to lowimpact specialist journals; even worse, they might be publishable only as a monograph. As our agency is judged by the quality of the work that we support—measured by the average impact factor of the papers that result from our funding—this is a strongly negative comment”
Rather sadly, this excellent editorial had to accompanied by a pusillanimous disclaimer
This Editorial represents the personal views of Frank Gannon and not those of Science Foundation Ireland or the European Molecular Biology Organization.
Today the front page story in Times Higher Education concerned a letter, Modest revolt to save research from red tape a letter signed by 20 people (including me).
My reason for signing the letter was that I am interested in how to get good science, and I am concerned that the government, and many vice-chancellors, are getting it wrong. Read also the wonderful essay. The Mismeasurement of Science, by Peter Lawrence (who is one of the signatories of the letter).
The letter had an accompanying article by Zoe Corbyn, Scientists call for a revolt against grant rule they claim will end blue-skies research, “Letter blames research councils’ policies for fall in number of UK Nobel laureates, reports Zoë Corbyn”.
It also had a leading article by Ann Mroz, Leader: Short-term outlook, no blue skies, with the subtitele “By forcing academics to do R&D for industry, ministers stifle the type of curiosity-driven work that delivers the biggest bangs”.
The “revolt” proposed in the letter is very modest indeed. We propose merely that those who are asked to referee grants will confine themselves to the part of the application that they are able to judge. In most cases that will not include the two pages of arm-waving guesswork about how the results might save the nation twenty years hence.
Everyone loves the great discoveries, but great discoveries are very rare If you want them, you must hire the best people you can, let them get on with it, and accept failure. Lots and lots of failure.
It’s a pity, but nobody knows any better way to do it. But at least please don’t try to impose worse ways.
The writers point to the fact that since 1980, roughly the time when bureaucracy started to run rampant,
” . . . the changes have resulted in almost a tenfold decrease in the rate at which researchers at UK universities win Nobel prizes.”
OK, it’s true that we have no way to be sure that the undoubted increase in top-down control has actually been the cause of the smaller number of Nobel prizes, but there is no doubt that it has wasted a great deal of time that could have been spent more profitably. It’s intriguing that 1980 is also the time that Francis Wheen identified as being roughly the start of what I like to call the age of endarkenment.
“But what is the point of having a second-to-none academic sector if its commitment to innovation is not matched by commerce and business? British industry’s relatively low investment in science and technology has long been a serious problem that our proxies and governments have consistently ignored. Academics are a much easier target.
While understanding that our proxies are often in a difficult situation, they must become more courageous in dealing with Government or they won’t have an enterprise worth protecting. In recent years, they have acquiesced in subjecting academics to withering barrages of control, and researchers’ lives have become bureaucratic nightmares.
The latest turn of the screw inflicts yet another distracting burden, namely that of requiring prospective researchers to write “a two-page impact plan in addition to the case for support”.
We the undersigned suggest that it is time for a modest revolt. We would urge that reviewers for grant applications decline invitations to take these additional pages into consideration and confine their assessments to matters in which they are demonstrably competent.
Indeed, in research worthy of the name, we are not aware of anyone who would be competent at foretelling specific future benefits and therefore in complying with the request in any meaningful manner.”
I cannot speak for the other signatories, but I expect they, like I, have nothing against industry, and nothing against applied research. The problem. of course, is is that the really big developments are never obvious at the time they are made, and still less at the time you are writing a grant to do the work.
The government, and far less excusably, the Research Councils, often appear to be populated by the sort of people who would have told Michael Faraday to stop wasting his time with those wires and coils, and get to work on something really useful like designing better leather washers for steam pumps.
Certainly there can be no more inappropriate time than this to be urging universities to act more like businesses. The sight of a bunch of very highly paid bankers who proved incapable of distinguishing good investments from worthless ones is pitiful. That was their job and they failed to do it on a grand scale, The fact that the bastions of conservatism, who have spent most of their lives telling us how wicked it is to bail out failing businesses are now taking many thousands of pounds from me, inspires feelings that are perhaps better left unwritten.
It is all part of a much more general move to trust nobody to do a job properly, and, worse, to then construct a hierarchy of regulators that consists largely of people who have themselves failed at the job they are supposed to be regulating. They then lay down the law about how to do the job to those who have succeeded.
Sadly, this process is evident not only in government, and not only in Research Councils, but also within universities themselves. We have the managerial culture that disempowers anyone who does real research and teaching. We have vice-chancellors who support degrees in various sorts of quackery which they can’t possibly believe to be true, like Professor Petts of Westminster (where Amethysts emit high Yin energy), and Michael Pittilo of Robert Gordon’s University Aberdeen. We have vice chancellors who defend corrupt practices by the pharmaceutical industry, as in Sheffield. We have lunatic assessment targets, as at Imperial Medical School. And of course we have some curious “Morals in High Places“. And we have the vice-chancellors’ Trades Union, UUK, that does nothing to stop these things.
It really is time that the Research Councils, and university vice-chancellors resisted a bit more strongly government pressure to do things in a way that can only harm the economy (not to mention the universities) in the long run.
They have nothing to lose but their knighthoods.
Follow-up
One of the signatories of the letter mentioned this beautiful quotation..
“Every important discovery is by definition unpredictable. If it were predictable, it would not be important.”
Freeman Dyson (speech when receiving the Britannica Award for excellence in the dissemination of learning for the benefit of mankind, part reported in the Independent, March 12th 1990)
He added
“I intend forthwith to copy and paste this into the section of any grant application which asks for impact etc. No more and no less.”
There has already been some criticism of our letter. It has come, though, not from scientists, but from “science policy professionals”, a new breed of hangers-on who mostly seem to be too far from the coal-front to know how science works.
Testing hypotheses is written by the head the Research Councils UK Strategy Unit. He chooses to concentrate on the secondary matter of Nobel prizes, rather than answer the real questions. Fortunately Philip Moriarty has tried to keep him to the point, though without any very satisfactory reply.
The Prometheus blog from the USA has an article by Ryan Meyer that has given rise to some comments and questions. I have asked him to explain to me what he means by the “social context” of work on single ion channels.
I’m a bit late on this one, but better late than never.
| The opinionated and ill-informed actress turned talk show host, Jeni Barnett, spent an hour or so endangering your children (and hers) with what most surely be one of the worst ever accounts of measles vaccination.
|
Chart from BBC report |
She was abominably rude to a well-informed nurse who phoned in to try to inject some sense into the conversation.
The LBC tried to stop Ben Goldacre from publicising this horrific show by legal action.
Blogs are the new journalism. The response has been wonderful. People of all ages sat up late into the night transcribing the entire broadcast. Unlike the doubtless highly-paid actress, they did it as a public service. They were not paid by anyone. It is all rather beautiful.. Within a day of the legal notice being sent to Goldacre, the offensive broadcast has spread like wildfire over the web.
The result of all this hard work is that if you type ‘Jeni Barnett MMR’ into Google, every item but one on the first page links to the sites that are highly critical of Barnett’s irresponsible and ill-mannered rant (at 7 am on 7 Feb).
You can listen to the entire broadcast here. Or read the entire transcript here.
The many people who have put work into this effort are listed, for example, on Ben Goldacre’s own site.
Holfordwatch lists many links, and also lists previous attempts of lawyers to suppress science.
When will people learn that lawyers are not the proper way to settle matters of truth and falsehood.
Dice, n . Small polka-dotted cubes of ivory, constructed like a lawyer to lie on any side, but commonly the wrong one. [ Bierce, Ambrose , The Enlarged Devil’s Dictionary , 1967]
Follow-up
The list of commentators, on Holfordwatch, grows by the minute. The story rapidly spread to the USA: for example the excellent Orac has spoken eloquently.
The condemnation extends far beyond the usual bad medicine writers. Anyone who wants to speak the truth as they see it sees legal actions like these as a threat to freedom of speech. A side effect is that I learned about several new blogs.
One, with a name as good as its content is A Somewhat Old, But Capacious Handbag, written by (you guessed it) Miss Prism, has Today’s irresponsible tripe courtesy of Jeni Barnett.”
Another one that was new to me is the Black Triangle blog, written by Dr Anthony Cox (a pharmacovigilance pharmacist). He writes in Conspiracy?
Anti-vaccinators have exploited the internet for years. Websites, blogs, and forums are widely used by activists to promote their wrong-headed cause. However, when the pro-science pro-vaccine lobby use similar methods a common accusation is leveled at them. Here it is posted at JABS, the UK’s leading anti-vaccine website.
“There is no way all of this could have happened so quickly without Pharma backing.” “
That is really priceless. These anti-vaccination fanatics just don’t seem to be able to grasp that there is a big army of people who care so much about the public interest that they do all this for no money and a considerable cost to themselves in time and lost sleep.
Besides which, anyone who thinks that a big corporation could whip up so much support and activity in 24 hours obviously has a rather better opinion of the efficiency of big companies than I do. They’d need 25 meetings and an awayday in Majorca before anything happened . Even a university can do better than that (perhaps only 20 meetings and an awayday in Uxbridge). One does wonder why, then, universities are always being told to be more like businesses. But that is another story.
Anthony Cox also deals with another of my favourite topics in The Today Programme’s irresponsible MMR interview. I listen to the Today Programme, I listen every morning. But I do wish they could bring their medical reporting up to the same standard as their political reporting. Their policy of ‘equal time for the flat earth society” is not my idea of impartiality.
The Sunday Times for 8th February, by coincidence, has a major article by the excellent investigative reporter, Brian Deer.
An excellent summary has appeared already Dynutrix on Holfordwatch.
Part 1.MMR doctor Andrew Wakefield fixed data on autism .
“However, our investigation, confirmed by evidence presented to the General Medical Council (GMC), reveals that: In most of the 12 cases, the children’s ailments as described in The Lancet were different from their hospital and GP records. Although the research paper claimed that problems came on within days of the jab, in only one case did medical records suggest this was true, and in many of the cases medical concerns had been raised before the children were vaccinated. Hospital pathologists, looking for inflammatory bowel disease, reported in the majority of cases that the gut was normal. This was then reviewed and the Lancet paper showed them as abnormal. “
Part 2. MMR: Key Dates in the Crisis .
Part 3. Most shockingly Hidden records show MMR truth
“A Sunday Times investigation has found that altered data was behind the decade-long scare over vaccination ”
Part 4. How the MMR scare led to the return of measles.
Let’s hope that some of the original documents appear on-line soon.
The Times on 10 February carried a beautifully hard-hitting column by David Aaronovitch: The preposterous prejudice of the anti-MMR lobby
“Last week there was a bust-up in blogland.”
“Last week, justifying herself on her blog, Barnett invoked the spirit of the insurgent ignoramus. Yes, she said, she should have been ready with facts and figures on MMR.”
“That’s why I’m passionately for Goldacre, and why I find myself wondering whether we can file a class action against LBC for permitting a presenter to inflict her preposterous prejudices on her listeners, to the detriment of someone else’s kids.”
Jeni Barnett: have you lost something?. Well well, first Jeni Barnett removed the critical comments from her blog. Then she removed the blog altogether. Seems she isn’t interested in debate at all.
Neither does she understand the internet. You can read the missing blog here, and the invaluable Quackometer has reproduced the whole blog post and all the missing comments. Great work Andy,
Stephen Fry left a comment (#223) on Goldacre’s site.
“The fatuity of the Jeni Barnett woman’s manner – her blend of self-righteousness and stupidity, her simply quite staggering inability to grasp, pursue or appreciate a sequence of logical steps – all these are signature characteristics of Britain these days. The lamentable truth is that most of the population wouldn’t really understand why we get so angry at this assault on reason, logic and sense. But we have to keep hammering away at these people and their superstitious inanities. We have to. Well done you and well done all you supporting. I’ve tweeted this site to my followers. I hope they all do their best to support you. Publish and be damned. We’ll fight them and fight them and fight them in the name of empricism, reason, double blind random testing and all that matters.”
London Evening Standard on 11 February. Nick Cohen on How my friends fell for the MMR panic.
Press Gazette covered the start of the srory on 6th February, here.
MSNBC TV broadcast by Keith Olberman votes Andrew Wakefield as “today’s worst person in the world” on February 10th. Click in video “Vaccine lie puts kids at risk”.
Write to your MP to ask him/her to sign Early Day Motion 754, MMR Vaccine and the Media
David Aaronovitch writes again in the Times, February 14th, “We need an inquiry into how Andrew Wakefield got away with it“.