Testimonials and Research

If you visit the website of any pseudo-scientific practitioner one thing almost always displayed is a list of testimonials – effusive endorsements from previous clients. For example, here’s an extract from the website of Cynde Van Vleet, a Tellington T Touch practitioner from California:

My experience with Cynde and her TTouch work has been nothing but wonderful, rewarding, and informative.  Cynde is a most gentle  and intuitive animal lover who conducts herself professionally at all times, yet exudes warmth and kindness as well.  I have great respect and admiration for Cynde and the work she does with animals. Her knowledge of TTouch and the world of dogs is impressive, and I have learned a lot from her in our brief history together…

Similarly, here’s an example from alleged psychic Philena Bruce’s website:

“I came to Philena when I was in a very dark place. Through her patience, guidance and gentleness, I genuinely left feeling hopeful. She held me together emotionally and spiritually throughout this time. She lit the candle in my mind and let my spirit guide me to light. She has a wonderful personality and in my heart I know our paths were meant to cross.”
K.K., London, Project Manager

In most cases testimonials are intended to establish the credibility of the practitioner. In Philena’s case, the long list she provides also pushes the “entertainment only” disclaimer – which all psychics in the UK are now legally obliged to put on their literature – far out of the view of the casual visitor.

But are testimonials useful? Can we use them to work out how effective a treatment of practitioner is? Should we as consumers pay any attention to testimonials?

The most obvious problem with most testimonials, especially those you might find on websites, is with authenticity. It doesn’t take a huge amount of imagination to make up your own testimonials and in most cases it would be difficult to prove that they were fake. However, I doubt many of the testimonials I have seen have been faked. It seems more likely that they came from customers who were genuinely pleased with the service. I’ve no doubt that they have plenty of satisfied customers – I’ve met some of them.

However, we should keep in mind that testimonials are highly selective. They give us no indication of the percentage of customers who were satisfied with the service. Those who were not pleased with the service may not have given any report. They may have felt foolish for having tried it, so rather than write a negative testimonial, they may forget about it and get on with their lives. Even if they did write to complain, surely no one attempting to establish their credibility is going to publish their views. I’ve visited the personal websites of quite a few spiritual healers, fortune tellers and other sellers of pseudo-science, but I have yet to see a single negative testimonial. I think it unlikely that no customer has ever been dissatisfied with the services any of them provided.

How about if we independently collected testimonials from every customer who visited a particular therapist or fortune teller? Even then, we still wouldn’t be able to say with any certainty whether on not they had genuine abilities. We might be able to tell that they were friendly, professional, helpful and so on, but judging whether the treatment worked is something else. It has been well documented that people are more likely to judge something to have been a success once they have invested substantially in it – whether emotionally or financially. I suspect this is why payment or gift-giving always has to be part of the process of Reiki. Perhaps on some subconscious level people say to themselves:

“I’ve paid more than my weekly grocery bill for this treatment. Only a fool would do that for something which doesn’t work. I know I’m not a fool, so it must have worked!”

This effect is brilliantly explained in Carol Tarvis and Elliot Aronson’s book, Mistakes Were Made.

It seems testimonials are a pretty unreliable guide to whether a medical treatment is effective or whether a psychic prediction is accurate. Certainly they are a very poor substitute for research. They are a selective form of anecdotal evidence.

So why do they get used at all? I think there are several reasons.

Firstly, there are times when use of a testimonial is valid. It’s not just treatments with dubious efficacy that print praise from former clients. Testimonials are commonplace in all sorts of marketing – child carers, home builders, restaurants and so on.

In any situation where the customer is in a position to judge whether the service or treatment was effective a testimonial can be useful. For example, if the testimonial relates to a novel, film or restaurant, then the aim is entertainment and the customer is best qualified to say whether they have been entertained.

Secondly, testimonials are easy to understand. A clinical trial is not something that most people are willing to wade through. Doing so takes considerable effort and even an intelligent reader might not be able to judge the strength of the research.

Thirdly, testimonials are easy. All you need is a few satisfied customers to say you’re warm and friendly. A full research project is beyond the means of most independent therapists.

Lastly, testimonials are human. What other people tell us is how we gain most of our information, so it feels quite natural to read what other people say about a service or therapy.

None of these are especially good reasons to trust testimonials.

Mircoscope and clipboardPharmaceutical companies have to put all their products through clinical trials before they can be marketed. I don’t think that other treatments should be subjected to any lesser scrutiny.

In an ideal world each therapists’ techniques would be independently evaluated with something akin to clinical trials, before an official regulator’s endorsement could be given. The next best thing would be for their methods to be independently reviewed and researched to discover whether they can work. However, the money to do this research has to come from somewhere, perhaps a jointly-funded regulatory body whose research and methods are made public.

Alternative therapists have told me that clinical trials are biased and corrupt due to the companies pursuit of profit above all else. There have certainly been some cases where this is true and there’s a case for improving accountability and publication of medical research in general. However, I imagine the situation would be much worse if the pharmaceutical industry could validate new medicines simply by gathering testimonials.

I’ve also been told that using testimonials instead of research is acceptable for alternative treatments because laying your hands on someone or reading their palm doesn’t have harmful side-effects that ingested medicines can. However, the purpose of clinical trials is twofold; to ensure there are no dangerous side-effects and to ensure the treatment is effective. Giving someone an ineffective treatment – even for free – is irresponsible and dangerous in itself.

[BPSDB]