Testimonials and Research

If you visit the website of any pseudo-scientific practitioner one thing almost always displayed is a list of testimonials – effusive endorsements from previous clients. For example, here’s an extract from the website of Cynde Van Vleet, a Tellington T Touch practitioner from California:

My experience with Cynde and her TTouch work has been nothing but wonderful, rewarding, and informative.  Cynde is a most gentle  and intuitive animal lover who conducts herself professionally at all times, yet exudes warmth and kindness as well.  I have great respect and admiration for Cynde and the work she does with animals. Her knowledge of TTouch and the world of dogs is impressive, and I have learned a lot from her in our brief history together…

Similarly, here’s an example from alleged psychic Philena Bruce’s website:

“I came to Philena when I was in a very dark place. Through her patience, guidance and gentleness, I genuinely left feeling hopeful. She held me together emotionally and spiritually throughout this time. She lit the candle in my mind and let my spirit guide me to light. She has a wonderful personality and in my heart I know our paths were meant to cross.”
K.K., London, Project Manager

In most cases testimonials are intended to establish the credibility of the practitioner. In Philena’s case, the long list she provides also pushes the “entertainment only” disclaimer – which all psychics in the UK are now legally obliged to put on their literature – far out of the view of the casual visitor.

But are testimonials useful? Can we use them to work out how effective a treatment of practitioner is? Should we as consumers pay any attention to testimonials?

The most obvious problem with most testimonials, especially those you might find on websites, is with authenticity. It doesn’t take a huge amount of imagination to make up your own testimonials and in most cases it would be difficult to prove that they were fake. However, I doubt many of the testimonials I have seen have been faked. It seems more likely that they came from customers who were genuinely pleased with the service. I’ve no doubt that they have plenty of satisfied customers – I’ve met some of them.

However, we should keep in mind that testimonials are highly selective. They give us no indication of the percentage of customers who were satisfied with the service. Those who were not pleased with the service may not have given any report. They may have felt foolish for having tried it, so rather than write a negative testimonial, they may forget about it and get on with their lives. Even if they did write to complain, surely no one attempting to establish their credibility is going to publish their views. I’ve visited the personal websites of quite a few spiritual healers, fortune tellers and other sellers of pseudo-science, but I have yet to see a single negative testimonial. I think it unlikely that no customer has ever been dissatisfied with the services any of them provided.

How about if we independently collected testimonials from every customer who visited a particular therapist or fortune teller? Even then, we still wouldn’t be able to say with any certainty whether on not they had genuine abilities. We might be able to tell that they were friendly, professional, helpful and so on, but judging whether the treatment worked is something else. It has been well documented that people are more likely to judge something to have been a success once they have invested substantially in it – whether emotionally or financially. I suspect this is why payment or gift-giving always has to be part of the process of Reiki. Perhaps on some subconscious level people say to themselves:

“I’ve paid more than my weekly grocery bill for this treatment. Only a fool would do that for something which doesn’t work. I know I’m not a fool, so it must have worked!”

This effect is brilliantly explained in Carol Tarvis and Elliot Aronson’s book, Mistakes Were Made.

It seems testimonials are a pretty unreliable guide to whether a medical treatment is effective or whether a psychic prediction is accurate. Certainly they are a very poor substitute for research. They are a selective form of anecdotal evidence.

So why do they get used at all? I think there are several reasons.

Firstly, there are times when use of a testimonial is valid. It’s not just treatments with dubious efficacy that print praise from former clients. Testimonials are commonplace in all sorts of marketing – child carers, home builders, restaurants and so on.

In any situation where the customer is in a position to judge whether the service or treatment was effective a testimonial can be useful. For example, if the testimonial relates to a novel, film or restaurant, then the aim is entertainment and the customer is best qualified to say whether they have been entertained.

Secondly, testimonials are easy to understand. A clinical trial is not something that most people are willing to wade through. Doing so takes considerable effort and even an intelligent reader might not be able to judge the strength of the research.

Thirdly, testimonials are easy. All you need is a few satisfied customers to say you’re warm and friendly. A full research project is beyond the means of most independent therapists.

Lastly, testimonials are human. What other people tell us is how we gain most of our information, so it feels quite natural to read what other people say about a service or therapy.

None of these are especially good reasons to trust testimonials.

Mircoscope and clipboardPharmaceutical companies have to put all their products through clinical trials before they can be marketed. I don’t think that other treatments should be subjected to any lesser scrutiny.

In an ideal world each therapists’ techniques would be independently evaluated with something akin to clinical trials, before an official regulator’s endorsement could be given. The next best thing would be for their methods to be independently reviewed and researched to discover whether they can work. However, the money to do this research has to come from somewhere, perhaps a jointly-funded regulatory body whose research and methods are made public.

Alternative therapists have told me that clinical trials are biased and corrupt due to the companies pursuit of profit above all else. There have certainly been some cases where this is true and there’s a case for improving accountability and publication of medical research in general. However, I imagine the situation would be much worse if the pharmaceutical industry could validate new medicines simply by gathering testimonials.

I’ve also been told that using testimonials instead of research is acceptable for alternative treatments because laying your hands on someone or reading their palm doesn’t have harmful side-effects that ingested medicines can. However, the purpose of clinical trials is twofold; to ensure there are no dangerous side-effects and to ensure the treatment is effective. Giving someone an ineffective treatment – even for free – is irresponsible and dangerous in itself.

[BPSDB]

When woo works

[BPSDB]

I recently became aware of Tellington T Touch therapy which is an animal healing technique, partly influenced by the Feldenkrais Method. It is principally used on ailing horses, cats and dogs, although it can be applied to many species. I’ve even heard reports of it being used on snakes and tarantulas – apparently they “touched” the snake with sticks. According to the official spiel, instructors and practitioners can be found in 27 countries. The inventor was Linda Tellington-Jones; her website explains how it works:

The intent of the TTouch is to activate the function of the cells and awaken cellular intelligence – a little like “turning on the electric lights of the body.”

Predictably, no explanation of “cellular intelligence” is given, but I’m guessing it has nothing to do with the cellular neural network parallel computing paradigm. Further reading reveals that Ms Tellington Jones received an honorary doctorate degree from the dubiously accredited and pompously-named Wisdom University.

None of which means that Tellington T Touch therapy won’t work; it just sets off the woo-alarm.

So what is Tellington T Touch therapy?

Using a combination of specific touches, lifts, and movement exercises, TTouch helps to release tension and increase body awareness.

Which I sounds a lot like massage and exercise and I think it’s quite reasonable that massage and exercise should be good for animals. The physical and mental benefits of exercise for humans are well-known. Ordinary, non-magical massage – with no ineffable cellular intelligence – shows signs of being beneficial to elderly people suffering loneliness or depression amongst others. It makes sense that social creatures, be they humans or dogs, would benefit from physical contact and connection to those around them.

So I can really see Tellington T Touch working. I expect that a well-intentioned practitioner really could improve a pet’s behaviour and to some extent its general health. But I seriously doubt it could perform significantly better than a combination of loving attention and regular “walkies”.

Which rather clouds the issue. Practitioners vary in how much they market the pseudo-scientific side of their animal therapy; many of the UK ones steer clear of the vague explanations and simply use Tellington T Touch as an extra skill on their CV – no need to scare off the skeptics when the rest of what they do is fairly “normal”.

However, there are plenty who do claim that some holistic, cellular level awakening of energies or similar is behind the effects. Is such an animal therapist a con artist or not? If they can achieve the results they claim, then what does the reasoning behind it matter? After all, there are a number of medicines in common usage whose exact mechanisms are poorly-understood, but they are still rigorously tested before being administered to the general public.

Which is exactly the point – all alternative therapies should be tested, like any other medical treatment. That means randomised, double-blind, placebo-controlled trials.

It’s also worth noting that it doesn’t matter what the claimed reasoning behind the treatment might be. Whether it’s “cellular intelligence” or “a certain energy” or just “a mystery” – whether it works or not can still be tested. As long as you can compare it against a “fake” version without the patients or those measuring the results knowing, you have a double-blind trial. I think it’s safe to say that the animals won’t be able to read the Tellington T Touch practitioner’s certificates on the wall, so will be unaware of which is a real treatment and which is the control. As for the measurers, there’s no need for them to see the therapy or control being carried out and they could assess the results over the next few days or weeks. As long as it’s done on a statistically signficant number of animals the study can be fair and informative.

I think testing is important and I see no reason that alternative therapies should be able to shirk this responsibility. Few people would be willing to take a pharmaceutical company’s products if for example, they said “Yes it works, although it can’t be tested, but they’ve been using it in the far East for generations and my grandmother swears by it” (that’s probably more akin to Reiki than T Touch, but the implications are similar). I’d like all “medical” treatments to be held to the same high standards. If “Fairy dust” or “cellular intelligence” is an unreasonable explanation for GlaxoSmithKline, then it’s unreasonable for everyone.

Granted, a bit of ineffective massage is unlikely to make your eyes drop out or to cause sterility (unless it’s very clumsy). However, even if you were to give the therapy for free, I still don’t think it’s entirely harmless. An ailing person (or animal) only has a certain amount of time and energy to spend trying out alternatives. Confusing the issue with ill-defined treatments of dubious efficacy is at best an irresponsible waste of people’s time and at worst cruel and deceitful.

As regards Tellington T Touch, I think it sits on a borderline. Most of the claims it makes sound reasonable for simple massage and exercise (except perhaps the treatment of snakes and spiders), so why add all the magical nonsense? Perhaps it means they can charge more or take on cases where traditional therapy has failed. In any case, I think they should undergo independent clinical trials – preferably by an institution which doesn’t have “Truth” or “Wisdom” in the title. If their techniques show significantly better results than plain massage and exercise, then they’ve earned the right to be respected medical practitioners. If not, then they should call a spade a spade and admit it’s just massge and exercise.