Comments on: Testimonials and Research

By: Daylight Atheism > How to Think Critically: Testimonials

Daylight Atheism > How to Think Critically: Testimonials — Fri, 01 Jan 2010 17:21:07 +0000

[...] from true believers. Eshu of Bridging Schisms gives many more examples, in his post “Testimonials and Research“, like this gem from a satisfied client singing the praises of a psychic claimant: “I [...]

By: Eshu

Eshu — Mon, 22 Dec 2008 11:20:39 +0000

Thanks LadyFlasheart. There’s plenty more homework I could do on the psychology of this, which could be an interesting project for another day.

This makes me think that there are two slightly different properties of testimonials here:

1. Effectiveness – How likely it is to influence people’s decisions. As Chappie pointed out, testimonials from friends and family are likely to be more “effective” in this respect.
2. Genuine usefulness – How likely the testimonial is to contain accurate information pointing towards the truth. As Yunshui said, testimonials (whether from friends and family or not) are less likely to point towards the truth than independent evidence.

I’m sure there should be better terms for those two.

By: LadyFlasheart

LadyFlasheart — Sun, 21 Dec 2008 22:36:59 +0000

There’s been quite a bit of psychology research how the psychology of the consumer affects the effectiveness of testimonials. Testimonials are particularly effective when used on people with a high SNI (susceptibility to normative influence) i.e. people with a strong willingness to conform to others expectations and desire to fit in.

By: Eshu

Eshu — Wed, 17 Dec 2008 22:52:03 +0000

Well that’s one to me… and about 110 to you! I guess you had a head start.

I’ll look forward to your version.

By: Ebonmuse

Ebonmuse — Wed, 17 Dec 2008 21:18:59 +0000

You stole my thunder, Eshu – I was planning a very similar post for later this month. But now I have a good source to use as a reference. :)

By: yunshui

yunshui — Wed, 17 Dec 2008 11:55:07 +0000

If a friend recommends me a book, a foodstuff, a new way of doing the laundry or a board game they think I’d like, I’ll almost always check it out. None of these things (apart maybe from the laundry idea) are assessed on anything other than subjective grounds. Testimonials – even from one’s mates – on issues that can be assessed objectively, using the scientific method, are less valid to me than the real-world objective valuation.

To put it another way, if I tell you a book is “good”, you’ll assume that I mean I like the plot, or the language, or the cover… You may have a different opinion to me on these issues, but you can accept that by my standards it’s a “good” book. If I say a therapy is “good”, however, you take that to mean it works, it made me better, it reduced my symptoms. Those are testable, scientific claims, and they aren’t decided by opinion.

I’m sticking with my position. My best mate may credit his continued existence to an invisible man in the sky, but his testimony is meaningless next to our observations of reality.

By: Eshu

Eshu — Tue, 16 Dec 2008 22:25:20 +0000

yunshui,

For the examples you give, I completely agree that it makes no difference to how trustworthy the information is. However, I’d trust a book recommendation from a friend (or fellow blogger I knew) more than one on Amazon. Perhaps this is the sort of thing the chaplain means?

By: yunshui

yunshui — Mon, 15 Dec 2008 13:11:27 +0000

I’d go one step further, Chappy – why are testimonials from close friends and family more valid than any other? Just because they may have your best interests at heart doesn’t make them right. My grandfather loves me and wants the best for me, no doubt, but when he testifies about how I should head on back to Jebus I smile, nod and think “bullshit”. One of my closest friends is a kinesiologist and naturopath – she’s convinced that kinesiology has cured her of all kinds of ills, and offers it to me every time we meet – but that doesn’t mean I’m going to discount the clinical trials which show it to be a fraud.

Testimonals and personal recommendations both suffer from the same criticism – the sample size is way to small for any results to be statistically significant. Just as I wouldn’t trust a clinical trial with only one participant, so I’m not going to trust my mate’s recommendation of homeopathy, unless he’s a qulified medical researcher with a stack of RCT results on hand.

By: the chaplain

the chaplain — Mon, 15 Dec 2008 00:49:29 +0000

Testimonials are most effective if they come from known friends and acquaintances. I’ll take the word of a trusted family member, friend or co-worker over that of a televised talking head any day. Testimonials from people I don’t know are mostly useless, unless they are backed up by other kinds of evidence, such as documented studies, etc.