What Does Evidence-based Therapy Really Mean?
Evidence-based therapies are getting very popular. But what does “evidence-based” really mean? A closer look at what we do and don’t know about effective psychotherapy.
There is something very appealing about the idea of evidence-based psychotherapies, also known as empirically supported treatments (ESTs). If you are looking around for a therapist, finding someone who is using a therapy that’s been rigorously evaluated and found to be effective (i.e., “empirically supported”) seems like a great idea. As a quick Google search will show, there a quite a few evidence-based therapies out there. Some have been shown to work well for a wide variety of problems, while others seem to excel at treating specific problems such as phobias, anxiety, depression, or psychological trauma.
The process for determining that a therapy is effective is pretty straightforward, at first glance. Researchers conduct a randomized controlled trial (RCT) in which participants are randomized into one group that receives the therapy being evaluated, or a control/comparison group that a participates in a different sort of therapy or intervention, or receives nothing at all until the study is over (a waitlist control group). It’s also possible to have more than two groups; one could compare three different therapies, or Therapy A vs Therapy B vs no treatment at all. The randomization is meant to ensure that the groups are highly similar, with differences evenly distributed (randomized) across the groups. This allows us to conclude that any differences between the groups at the end of the study are the result of the different treatments or therapies and not due to baseline differences between the groups. If the therapy being evaluated turns out to be effective relative to the control conditions, and if this finding can be replicated in several studies, the therapy is generally considered to be empirically supported.
In fact, the process of establishing that a psychotherapy is genuinely effective, or that it’s superior to another form of therapy, is more complicated than initially meets the eye.
In a bitingly serious article with a tongue and cheek title (“ How To Prove That Your Therapy is Effective, Even When It Is Not: A Guideline “), the eminent researcher Pim Cuijpers and his colleague Ioana Aline Cristea illustrate the many ways that therapies are determined to be empirically supported when in fact, the research supporting their effectiveness is flawed or seriously limited. Here is a sampling of the flaws and limitations they discuss:
- The researchers’ own preference for the therapy they are evaluating tends to influence the results of their study. This happens a lot, though exactly how it happens is unclear. But once we control for (take into account statistically) the preference of researchers for the therapy they are evaluating, the superiority of that therapy often diminishes or even disappears. The superiority of cognitive behavioral therapy (CBT) relative to other therapies, for example, generally decreases substantially once this sort of researcher bias is controlled for. This is true for other approaches as well. (To be clear, CBT appears to work well for a variety of problems, but its superiority to other therapies has been overstated).
- Historically, researchers have often chosen not to publish studies that failed to find the effects they were looking for. This means we mostly read about the studies that did find that Therapy X was effective, and not the numerous studies that failed to find this. This creates a so-called publication bias, and leads to a distorted perception of just how effective Therapy X actually is. Imagine if you were considering having heart surgery, and the two published studies on the procedure showed it to be highly effective. How would your thinking change if you learned that an additional 10 studies, none of them published, found no benefit from the surgery or found it to be no more effective than safer and less invasive treatments such as medication and dietary changes?
In addition to the concerns raised by Cuijpers & Cristea, there are other questions we should ask in order to better understand the evidence behind empirically supported treatments.
When we hear that a therapy has been shown to be effective, we should always ask, “effective for whom?”
What are we to do? How should we choose a therapist and a type of therapy best suited for whatever we are struggling with, and who we are individually? We can take comfort and find guidance in a couple of reasonably well-established guidelines:
1. Effective therapists seem to share in common a set of common factors or characteristics. These include, for example, a high level of empathy, good interpersonal skills, a lack of defensiveness, a comfort with strong emotions, and a focus on helping clients develop new coping skills. I discuss these common factors in another post, What are the Qualities of Effective Therapists?
An initial consultation, or word of mouth from people you trust, can help you assess whether a prospective therapist seems high on these attributes and thus is likely to be helpful. Oh, and what about the letters that follow a therapist’s name, the particular degrees they hold? Turns out that’s far less important than those key attributes that effective helpers share in common.
2. Most forms of therapy are similarly effective for most problems. Despite claims of superiority made by many schools of psychotherapy, research continues to show that most forms of therapy are effective, and for most problems. To be sure, there are exceptions. Complex trauma may require expertise in working with fragmentation of the self (see, for example, the outstanding work of Janina Fisher) and the use of mindfulness-based, experiential approaches. Phobias respond particularly well to exposure-based therapies, obsessive compulsive disorder (OCD) often responds well to CBT (often in conjunction with SSRI medication), and uncomplicated PTSD frequently responds well to CBT, cognitive processing therapy (CPT), and EMDR as well as other exposure-based treatments. But for problems such as depression, anxiety, and low self-esteem, and for finding one’s way through many of life’s challenges, most therapies are quite helpful. It doesn’t mean they all work the same way, only that they all have a pretty good chance of helping you feel better and create the changes you desire in your life.
