LOB-vs
Download Lectures on Biostatistics (1971).
Corrected and searchable version of Google books edition

Download review of Lectures on Biostatistics (THES, 1973).

Latest Tweets
Categories
Archives

regulators

Back to my old interest in medicine that doesn’t work. And the fact that there is such a huge market for medicine that doesn’t work is, in large part, a result of regulators who fail to regulate.

One example of a regulator that fails to protect the public from health fraud is the Complementary and Natural Healthcare Council -the CNHC (known colloquially as OfQuack). I have history with them, having offered my services to them, and, astonishingly, been accepted. There I found a lot of well-meaning but poorly-educated people: they had no idea about what constituted evidence, and very little interest in finding out. Of course, I was asked to resign and when I declined to do so, I was fired.

AI reports on health claims

That’s one reason that I was happy to play a small part in a recent study of the health claims made by people who are registered with the CNHC. [link to the paper -it’s open access]. Registration is voluntary: anyone can practise as a naturopath or reflexologist with no need for qualifications and no check on their activities. But if they do choose to register with the CNHC they’ll appear on their website, and that confers on them a (spurious) sort of respectablity.

In the study, artificial intelligence was asked to identify false or misleading claims made on the web sites of practitioners who were registered with the CNHC. Only the practitioners of the most obviously pseudoscientific subjects were included (Alexander Technique, Aromatherapy, Bowen Therapy, Colon Hydrotherapy, Craniosacral Therapy, Healing, Kinesiology, Microsystems Acupuncture, Naturopathy, Reflexology, Reiki and Shiatsu).

The AI assessed text from 11,771 web pages, scraped from 725 web sites. It identified false or misleading claims in 704 (97%) of the websites. The complete list of 23,307 claims identified by AI, with the reasons that AI provided, can be downloaded at https://osf.io/hnuqs (better not try printing it out -it has 2370 pages).

A few of the web pages were read by four humans, and the humans identified a comparable number of false or misleading claims to that found by the AI.

What does this mean for AI?

It seems that the AI did a good job in identifying false and misleading claims. That surprised me because AI has to learn from what it finds on the web. And the information on the web about reflexology, craniosacral therapy and naturopathy etc, is almost all nonsense -just sales talk. The fact that AI wasn’t fooled by this shows that the information that was used to train the AI came, almost entirely, from reliable sources, The sources of training material are a human choice. So the finding that AI can judge that reflexology is nonsense is actually a result of the human intelligence that decided which sources were reliable and which were not.

I’ve tried asking AI several real science questions and I’ve been very disappointed by the answers. The AI showed no ability to judge between different, but legitimate, points of view. For example there is a lot of good material on the web about statistical inference (but also quite a lot of mediocre stuff).

When I asked about the interpretation of p values, the first answers were quite accurate but entirely frequentist. When I pointed out that likelihood and Bayesian approaches are often critical of the frequentist interpretation, the AI apologised and tried again (the mock humility of the apology was particularly horrid). After a few more prompts, the AI came to a view of the problem which was quite close to mine. That doesn’t mean that my view is right. It means that, with helpful prompts, AI can be led to almost any conclusion you wish.

Once again, the biggest influence is the human intelligence that guides the AI. The computer algorithm showed NO understanding of the problems. It spat out a version of the material that was used to train it. It was at the level of a mediocre undergraduate. The only way to detect that it was AI was that it answered in better English, with fewer spelling mistakes than most undergraduates can manage. That makes its use doubly dangerous.

Implications for regulation of alternative medicine

It’s been obvious for a long time that the regulators of alternative medicine fail totally to protect the public from false and misleading claims being made about the products that are being sold to the public. That is shown abundantly by this paper. The number of false or misleading claims that were found on the web is inevitably an underestimate of the real number. AI could count only claims that were made in public, on the web. It couldn’t count the number of false claims that are made in private conversations, nor the number made in printed advertisements.

It’s clear from the huge number of false claims made by CNHC registrants that it would take a lot of time and effort for the CNHC to fulfil its duty to protect the public from them. The benefit of using AI is not that it’s any better than humans at detecting false and misleading claims. The benefit lies entirely in the speed with which it can do the job. That’s why the senior author of the paper, Simon Perry, offered the use of his methods to the CNHC. So far they have shown little interest in accepting that help. All that I can infer from this is that the CNHC has little interest in the false claims made by its registrants, as long as they keep paying the registration fees. It hasn’t changed much since I was part if it, 14 years ago.

Conclusions

AI looks quite promising for some problems where the task is limited and the rules are clear. Learning to spot cancerous tissue in radiographs, when trained on enough radiographs in which the human diagnosis turned out to be accurate. AI also did a good job in detecting false or misleading claims by practitioners of various sorts of alternative medicine, though this reflects the (human) choice of reliable sources of information. The speed with which this can be done should be useful to regulators who are interested in stopping the spread of misinformation. Sadly most regulators show very little interest in doing that part of their job.

Ai seems to have done a good job in predicting protein folding. That is a very restricted task, and there is a lot of (mostly) reliable information to train on. There is a lot of hype about its use for drug discovery, but the training data for that job is often crude and inaccurate.

But general-AI has a long way to go. It has no understanding of what it says. At the moment it’s mainly a tool for crooks, fraudsters and charlatans whose intent is to deceive.

Jump to follow-up

gphc

The General Pharmaceutical Council (GPhC) has been the statutory body responsible for the regulation of pharmacy since 2010. It’s status is similar to that of the GMC and. heaven help us, the GCC. Before that the regulator was the same as the professional body, the Royal Pharmaceutical Society of Great Britain (RPS). The RPS proved to be as useless as most other regulators, as documented in detail in my 2008 post, At around the time it stopped being a regulator, the RPS started to condemn quackery more effectively, but by then it had lost the power to do much about it (I hope the latter wasn’t the cause of the former). The body that could do something, the GPhC has done essentially nothing. as described in this post.

I did a 2 year apprenticeship in Timothy White’s and Taylor’s Homeopathic (yes, really) Chemists in the 1950s.
My first degree was in pharmacy. I got my interest in pharmacology from reading Martindale’s Extra Pharmacopoeia in the shop. I soon decided that I didn’t really want to spend the rest of my life selling lipstick and Durex. The latter was quite a big seller because the Boots across the road didn’t sell contraceptives (they changed their minds in the 1960s).

In those days, we spent quite a lot of time making up (almost entirely ineffective) ‘tonics’ and ‘cough mixtures’. Now the job consists largely of counting pills. This has exacerbated the ‘chip on the shoulder’ attitude that was present even in the 1950s. For a long time now, pharmacists have wanted to become the a ‘third tier’ in the NHS, alongside GP practices and hospitals., after hospitals and doctors". . Here are a few comments on this proposition.

First let me say that I’ve met some very good and ethical pharmacists. I did a vacation job in a hospital pharmacy where the boss had an encyclopaedic knowledge of the effects and side effects of drugs, and of their dosage. His advice was often sought by doctors, and rightly so. He had no way of knowing at the time that his advice to replace barbiturates with thalidomide would lead to such a tragedy, because the evidence had been concealed by the manufacturer. Some of the problems alluded to here have already been highlighted by two excellent pharmacists, Anthony Cox and @SparkleWildfire, neither of whom work in pharmacists shops. They are absolutely spot on but they seem to be in a minority among pharmacists.

The problems seem to lie mostly in retail shops. Their shelves are laden with ineffective pills and potions. And the pharmacist has every incentive to sell them. His/her income depends on it directly if it’s a privately owned pharmacy. And his/her standing with head office depends on it in chain store pharmacies. This conflict of financial interest is the prime reason why pharmacists are not qualified to form a third tier of healthcare. The avoidance of conflicts of interest among doctors was one of the great accomplishments of the NHS. In the USA there are huge scandals when, as happens repeatedly, doctors order expensive and unnecessary treatments from which they profit. It’s no consolation that such problems are creeping back in the UK as a result of the government’s vigorous efforts to sell it off.

Here are few examples of things that have gone wrong, and who is to blame. Then I’ll consider what can be done.

Ineffective medicines

In any pharmacy you can see ineffective ‘tonics’ and ‘cough medicines’, unnecessary supplements with dishonest claims and even, heaven help us, the ultimate scam, homeopathic pills.

What’s worse, if you ask a pharmacist for advice, it’s quite likely that they’ll recommend you to buy them.

I was amazed to discover that a number of old-fashioned ‘tonics’ and ‘cough medicines’ still have full marketing authorisation. That’s the fault of the Medicines and Healthcare Regulatory Auhority (MHRA) who are supposed to assess efficacy and totally failed to do so, Read about that in “Some medicines that don’t work. Why doesn’t the MHRA tell us honestly?” . It’s hard to blame a pharmacist for the bad advice given by the MHRA, but a good one would tell patients to save their money.

benylin

Big corporate pharmacies

Companies like Boots seem to have no interest whatsoever in ethical behaviour. All that matters is sales. They provide “(mis)educational” materials that promote nonsense They advertise ridiculous made-up claims in the newspapers, which get shot down regularly by the Advertising Standards Authority, but by that time the promotion is over so they don’t give a damn. See for example, CoQ10 scam and the ASA verdict on it. And "Lactium: more rubbish from Boots the Chemists. And a more serious problem". And "The Vitamin B scam. Don’t trust Boots"

Recently the consumer magazine Which? checked 122 High Street pharmacies. They got unsatisfactory advice from 43% of them, a disastrously bad performance for people who want to be the third tier of healthcare.

Even that’s probably better than my own experience. Recently, when I asked a Sainsbury’s pharmacist about a herbal treatment for prostate problems, he pointed to the MHRA’s kite mark and said it must work because the MHRA approved it -he was quite unaware that you get the THR kite mark without having to present any evidence at all about efficacy.

mhra

Of course that is partly the fault of the MHRA for allowing misleading labels, but nevertheless, he should have known. See “Why does the MHRA refuse to label herbal products honestly? Kent Woods and Richard Woodfield tell me” for more on how the MHRA has betrayed its own standards.

When I’ve asked Boots’ pharmacists advice about persistent diarrhoea in an infant, saying I wanted a natural remedy, I’ve usually been guided to the homeopathic display. Only once was I told firmly that I should use rehydration not homeopathy (something every good parent knows) and when I asked that good pharmacist where she’d been educated, she said in Germany (mildly surprising given the amount of junk available in German pharmacies)

Regulators

Anthony Cox, a pharmacist who has been consistently on the side of right, says

"This is something that needs to be dealt with at a regulatory and professional body by the whole profession, and I am certain we have the majority of the UK pharmacy profession on side."

But the regulator has done nothing, and it isn’t even clear that there is a majority on his side.

At a 2009 meeting of Branch Representatives of the RPS a motion was proposed:

“…registration as a pharmacist and practice as a homeopath are not compatible, and that premises registered with the Society should not be used for the promotion of homeopathy”

Although that is obviously sensible to most people, the proposal was followed by a speaker from Leicester who thought it right to keep an open mind about Avogadro’s number and the motion was defeated. So much for the "scientists on the High Street" aspiration.

There have been two major scandals surrounding homeopathy recently. Both were revealed first by bloggers, and both came to wide notice through television programs. None were noticed by the regulators, and when they were brought to the attention of the regulator, nothing effective was done.

The malaria scandal

A lot has been wriitten about this here and on other blogs e.g. here and here. The idea that sugar pills can prevent or cure malaria is so mind-bogglingly dangerous that it was condemned by the Queen’s Homeopathic Physician, Peter Fisher. It was exposed on a BBC Newsnight programme in 2006. Watch the video.

The Gentle Art of Homeopathic Killing was an article that originally appeared on the excellent Quackometer blog produced by Andy Lewis. "The Society of Homeopaths were so outraged about one of their members flouting the code of ethics so blatantly that they took immediate action. That action was, as expected, not to do anything about the ethics breach but to threaten Andy and his hosting ISP with legal action for defamation. The article is reproduced here as a public service".

Some of the people involved in this bad advice were pharmacists, Very properly they were referred to the RPS in 2006 qnd 2009, the regulator at that time. They sat on the complaint so long that eventually the RPS was replaced by the GPhC as regulator. Nothing much has happened.

The GPhC did precisely nothing. Read their pathetic response.

Homeopathy for meningitis

An equally murderous fraud, "homeopathic vaccines" by Ainsworth’s has long been targeted by bloggers. In January 2013, Samantha Smith made an excellent BBC South West programme about it. Watch it and get angry.

Anthony Pinkus, pharmacist at Ainsworths, has been referred to the then regulator, the RPS, in 2006 and 2009. It’s said that he took "remedial action", though there is little obvious change judged by the video above. No doubt some of the most incriminating stuff has been removed from his web site to hide it from the ASA. It’s safer to mislead people by word of mouth. Since the last video more complaints have been made to the GPhC. So far, nothing but silence.

Why doesn’t the regulator regulate?

This pamphlet is reproduced from the July 2011 Quackometer post, “Ainsworths Pharmacy: Casual Disregard for the Law

ainsworth

It’s almost as though those royal warrants, enlarged on right, acted as a talisman that puts this dangerous company outside the grasp of regulators. I hope that the GPhC Council , and Duncan Rudkin (its chief executive and registrar), are not so worried about their knighthoods that they won’t risk upsetting the royal family, just to save patients from malaria and meningitis. Their CEO, Robert Nicholls is only a CBE so far.

warrants

Another reason for their inaction might be that the GPhC Council members, and Duncan Rudkin (its chief executive and registrar), lack critical faculties. Perhaps they have not been very well educated? Many of them aren’t even pharmacists, but that curious breed of professional administrators who inhabit the vast number of quangos, tick their boxes and do harm. Or perhaps they are just more interested in protecting the income of pharmacists than in protecting their customers?

Education

The solution to most problems is education. But there is no real knowledge of how many pharmacists in the UK are educated in the critical assessment of evidence. A recent paper from the USA did not give cause for optimism. It’s discussed by the excellent US pharmacist, Scott Gavura, at Science-based medicine. The results are truly horrifying.

“Few students disagreed with any CAM therapy. There was the greatest support for vitamins and minerals (94%, mean 4.29) which could include the science-based use these products. But there was strong support for demonstrably ineffective treatments like acupuncture, with 64% agreeing it was acceptable. Even homeopathy, which any pharmacy student with basic medicinal chemistry skills ought to know is absurd, was supported by over 40% of students.”

If the numbers are similar in the UK, the results of the Which? magazine survey are not so surprising. And if they are held by the GPhC Council. their inaction is to be expected. We just don’t know, and perhaps someone should find out.

I suspect that sympathy for quackery may sometimes creep in through that old-fashioned discipline known as pharmacognosy. It is about the botany of medicinal plants, and it’s still taught, despite the fact that very few drugs are now extracted from plants. At times, it gets dangerously close to herbalism. For example, at the School of Pharmacy (now part of UCL) a book is used Fundamentals of Pharmacognosy and Phytotherapy  by Michael Heinrich, Joanne Barnes, Simon Gibbons and Elizabeth M. Williamson, ot the Centre for Pharmacognosy and Phytotherapy at the School of Pharmacy. The introductory chapter says.

“TRADITIONAL CHINESE MEDICINE (TCM) The study of TCM is a mixture of myth and fact, stretching back well over 5000 years. At the time, none of the knowledge was written down, apart from primitive inscriptions of prayers for the sick on pieces of tortoise carapace and animal bones, so a mixture of superstition, symbolism and fact was passed down by word of mouth for centuries. TCM still contains very many remedies, which were selected by their symbolic significance rather than proven effects; however, this does not necessarily mean that they are all ‘quack’ remedies!”

Well, not necessarily. But as in most such books, there are good descriptions of the botany, more or less good accounts of the chemical constituents followed by uncritical lists of things that the herb might (or might not) do. The fact that even the US National Institutes of Health quackery branch, NCCAM, doesn’t claim that a single herbal treatment is useful tells you all you need to know.

Joanna Barnes is Associate Professor in Herbal Medicines, School of Pharmacy, University of Auckland, New Zealand. She has written a book, Herbal Medicines (“A useful book for learning holistic medicine”) that is desperately uncritical about the alleged theraputic effectiveness of plants. ("Simon Gibbons is on the editorial board of The Chinese Journal of Natural Medicine. Elizabeth Williamson is editor of the Journal of Phytotherapy Research, a journal that has a strong flavour of herbalism (take the infamous snoring remedy). These people aren’t quacks but they are dangerously tolerant of quacks.

The warning is in the title. "Phytotherapy" is the current euphemism for herbalism. It’s one of those red light words that tells you that what follows is unlikely to be critical. Exeter’s fantasy herbalist, Simon Mills, now describes himself as a phytotherapist. What more warning could you need?

Perhaps this explains why so many pharmacists are unworried by selling things that don’t work. Pharmacy education seems not to include much about the critical assessment of evidence. It should do.

Chemist and Druggist magazine certainly doesn’t help. It continually reinforces the idea that there is a debate about homepathy. There isn’t. And in one of its CPD modules Katherine Gascoigne says

"Homeopathic remedies are available, but are best prescribed by a homeopath"

Ms Gascoigne must be living on another planet.

Conclusions

The main conclusion from all of this is that the General Pharmaceutical Council is almost criminally negligent. It continues to allow pharmacists, Anthony Pinkus among them, to endanger lives. It fails to apply its own declared principles. The members of its Council, and Duncan Rudkin (its chief executive and registrar), are not doing their job.

Individual pharmacists vary a lot, from the superb to those who believe in quackery. Some, perhaps many, are embarrassed by the fact that their employer compels them to sell rubbish. It’s too much to expect that they’ll endanger their mortgage payments by speaking out about it, but the best ones will take you aside and explain that they can’t recommend it.

The GPhC itself is regulated by the Professional Standards Authority, the subject of my last post. We can’t expect anything sensible from them.

In the USA there is a shocking number of pharmacists who seem to believe in quackery. In the UK. nobody knows, though judging by their failure to vote against the daftest of all scams, homeopathy, there is no cause for complacency here.

It seems that there will have to be big improvements in pharmacy education before you can have much confidence in the quality of the advice that you get in a pharmacy.

Follow-up

Yesterday a talk was given at the School of Pharmacy, organised by the “The Centre for Homeopathic Education” (an oxymoron if there ever was one). The flyer had all the usual nonsense. Its mention of “Remedies & Tonics for Cancer Recovery” might well have breached the Cancer Act (1939). When I asked whether the amount received in room rental was sufficient to offest the damage to the reputation of the School of Pharmacy resulting from hosting a nutty (and possible illegal) event, I had the greatest difficulty in extracting any sort of response from the school’s director, Duncan Craig. I’m told that he considers “the policy on space rental to be a UCL management issue, rather than a matter of discussion on scientific ethics with a colleague”. Oh dear.