Jump to follow-up

Peter Dixon is a chiropractor. He is chair of the General Chiropractic Council (GCC). He was also a member of the hotly-disputed NICE low back pain guidance group that endorsed (you guessed it) the use of chiropractic, a decision that has led to enormous criticism of the standards of the National Institute of health and Clinical Excellence (NICE).

As a consequence largely of the decision of the British Chiropractic Association (BCA) to sue Simon Singh for defamation, there has been an unprecedented interest taken in the claims made by chiropractors in general.

Peter Dixon has a problem because something like 600 individual complaints about unjustified health claims have been sent to the GCC. Even when a web site does not claim to be able to benefit things like asthma and colic, a phone call may reveal that claims are made in private (one of the many complaints to the GCC concerns such behaviour by two practices belonging to, ahem, Peter Dixon Associates).

The crucial question is, as always, one of evidence. The BCA claim to have a plethora of evidence for their claims, but they have been strangely reluctant to produce it. In fact evidence is cited on the “Your first visit” page on Dixon’s site.

At the bottom we see “How effective is Chiropractic?”.

That sounds very impressive indeed: . ” . . . patients who received chiropractic treatment improved by 70% more than those given hospital out-patient.”

But hang on. If we look at the paper, Meade et al., 1990 [download reprint], we see that Figure 2 looks like this.

Several things jump out. First, the Oswestry disability index scale runs for 0 to 100, but scores are plotted only from 0 to 35, so the size of the effects are exaggerated. Second, there are no error bars on the points. Third there is essentially no advantage for chiropractic at all when all patients are taken together (top graph). Fourth, and most important, the patients who were followed up for two years (bottom graph) seem to show a slight advantage for chiropractic but on average, the effect is 7 percent (on the 100 point scale, NOT 70 percent as claimed on the web site of Peter Dixon Associates.

What sort of mistake was made?

The abstract of the paper itself says “A benefit of about 7% points on the Oswestry scale was seen at two years.” How did this become “improved by 70% more”?

It could have been simple a typographical error, but that seems unlikely, Who’d boast about a 7% improvement?

Perhaps it is a question of relative versus absolute change. The Figure does not show the actual scores on the 100 point scale, but rather the change in score, relative to a questionnaire given just before starting treatment. If we look at the lower part of the Figure, restricted to those patients who stayed with the trail for 2 years (by this time 28% of the patients had dropped out), we see that there is a reduction in score (improvement) of about 10 points on the 100 point scale with hospital treatment (not a very impressive response). The improvement with those sent to private chiropractic clinics was about seven points bigger. So a change from 10 to 17 is a 70 percent change. What’s wrong with that?

What’s wrong is that it is highly misleading, as relative changes often are. Imagine that the hospital number had been 7 points and the chiropractic number had been 14 (both out of 100). That would mean that both treatments had provided very modest benefits to the patients. Would it then be fair to describe the chiropractic patients as have improved by 100 percent more than the hospital patients, when in fact neither got much benefit? Of course it would not. To present the results in this way would be highly deceptive.

Put another way, a 70% increase in a trivial effect is still pretty trivial.

That isn’t all either. The paper has been analysed in some detail on the ebm-first site. The seven point difference on a 100 point scale, though it may be real, is too small to be ‘clinically significant’ In other words the patient would scarcely notice such a small change. Another problem lies in the nature of the comparison. Patients were, quite properly, allocated at random to chiropractic or to to hospital treatment. BUT the comparion was very from blind. one group was treated in hospital. The other group was sent to private chiropractic clinics. The trivial 7 point difference could easily be as much to do with the thickness of the carpets rather than any effect of spinal manipulation.

What this paper really tells you is that neither treatment is very effective and that there is little to choose between them.

It is really most unfortunate that the chairman of the GCC should show himself to be so careless about evidence at a time when the evidence for the claims of chiropractors is under inspection as never before. It does not add to their case for criticising Simon Singh and it does not add to one’s confidence in the judgement of the NICE guidance group.

Follow-up

The Pain Society revolt. A letter has been sent from several distinguished members of the British Pain Society to its President and Council.

“We, the undersigned, call upon the President of the British Pain Society to issue a statement to NICE and to the press condemning outright the conclusions of the recent UK National Institute for Health and Clinical Excellence (NICE) guidelines . . .”

The sigificance of this letter is that the present president of the British Pain Society is Professor Paul Watson who was a member of the NICE guidance group that produced the recommendations which have engendered such criticism. He was clinical advisor to the guidance group. There is a video of Paul Watson talking about back pain, that seems to me to illustrate very well the problem with the guidance. He says it is a huge problem (everyone knows that) and that something must be done, but he doesn’t say what. There is no admission that, in very many cases, nobody knows what to do. It is exactly this sort of hubris that that makes the NICE report so bad,

One caustic comment on the letter says

“We are led by a physiotherapist! A Professor who cannot even interpret straight forward evidence when it is presented to him on a plate.
Who’s going to be the next BPS President? A Hospital Porter?”