Quality of Research
Review Date: October 2010
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Carlson, J. G., Chemtob, C. M., Rusnak, K., Hedlund, N. L., & Muraoka, M. Y. (1998). Eye Movement Desensitization and Reprocessing (EMDR) treatment for combat-related posttraumatic stress disorder. Journal of Traumatic Stress, 11(1), 3-24. Study 2Wilson, S. A., Becker, L. A., & Tinker, R. H. (1995). Eye Movement Desensitization and Reprocessing (EMDR) treatment for psychologically traumatized individuals. Journal of Consulting and Clinical Psychology, 63(6), 928-937.
Wilson, S. A., Becker, L. A., & Tinker, R. H. (1997). Fifteen-month follow-up of Eye Movement Desensitization and Reprocessing (EMDR) treatment for posttraumatic stress disorder and psychological trauma. Journal of Consulting and Clinical Psychology, 65(6), 1047-1056. Study 3Marcus, S. V., Marquis, P., & Sakai, C. (1997). Controlled study of treatment of PTSD using EMDR in an HMO setting. Psychotherapy, 34(3), 307-315.
Marcus, S., Marquis, P., & Sakai, C. (2004). Three- and 6-month follow-up of EMDR treatment of PTSD in an HMO setting. International Journal of Stress Management, 11(3), 195-208.
Supplementary Materials Falsetti, S. A. (1997, November). A review of the Modified PTSD Symptom Scale. Paper presented at the 13th annual meeting of the International Society for Traumatic Stress Studies, Montreal, Quebec, Canada.
Keane, T. M., Caddell, J. M., & Taylor, K. L. (1988). Mississippi Scale for Combat-Related PTSD: Three studies in reliability and validity. Journal of Consulting and Clinical Psychology, 56(1), 85-90.
Kim, D., Bae, H., & Park, Y. C. (2008). Validity of the Subjective Units of Disturbance Scale in EMDR. Journal of EMDR Practice and Research, 2, 57-62.
Sundin, E. C., & Horowitz, M. J. (2002). Impact of Event Scale: Psychometric properties. British Journal of Psychiatry, 180, 205-209.
Thyer, B. A., Papsdorf, J. D., Davis, R., & Vallecorsa, S. (1984). Autonomic correlates of the Subjective Anxiety Scale. Journal of Behavior Therapy and Experimental Psychiatry, 15(1), 3-7.
Weathers, F. W., Keane, T. M., & Davidson, J. R. T. (2001). Clinician-Administered PTSD Scale: A review of the first ten years of research. Depression and Anxiety, 13(3), 132-156.
Outcomes
Outcome 1: PTSD symptoms |
Description of Measures
|
PTSD symptoms were measured by at least two of the following instruments in each of three studies:
- Mississippi Scale for Combat-Related PTSD (M-PTSD). The M-PTSD is a 35-item self-report instrument derived from DSM-III criteria for PTSD symptoms and includes items for frequently observed features specific to combat veterans (e.g., substance abuse, suicidality, depression). Items are rated on a 5-point Likert scale and summed to provide a continuous measure of PTSD symptom severity. Scores range from 35 to 175, and higher scores indicate more severe symptoms.
- Clinician-Administered PTSD Scale (CAPS-1). The CAPS-1 is a 30-item structured interview for measuring current (i.e., over the past month) and lifetime (i.e., for the worst month since the trauma) PTSD diagnostic status and symptom severity. The CAPS-1 has 17 items that correspond to DSM-IV criteria for PTSD symptoms; 8 items that measure associated features, such as guilt, hopelessness, and memory impairment; and 5 items that measure response validity, global severity, global improvement, and social and occupational impairment. Each of the 17 PTSD symptom items is rated on a separate 5-point scale for frequency (ranging from 0 [never] to 4 [daily or almost every day]) and intensity (ranging from 0 [none] to 4 [extreme, incapacitating distress, cannot dismiss memories, unable to continue activities]). The frequency and intensity scores can be combined for an overall symptom severity score; higher scores indicate symptoms that are more frequent, intense, or severe, depending on the scoring focus.
- PTSD Symptoms Scale. The PTSD Symptoms Scale, ranging from 0 to 10, was developed by the investigators to permit client self-rating of overall symptom status. Higher scores indicate worse symptoms, with 10 being the worst.
- Impact of Event Scale (IES). The IES is a 15-item self-report measure of posttraumatic stress symptoms that occurred recently (in the past 1-2 weeks) in response to an identified traumatic event. The scale generates two symptom frequency subscales--intrusion (intrusive thoughts, feelings, and images) and avoidance (avoidance of thoughts, emotion, and reminders)--which are summed for a total scale score. Higher scores indicate a greater degree of distress.
- Somatization dimension of the Symptom Checklist-90-Revised (SCL-90-R). The SCL-90-R is a 90-item self-report checklist that measures current psychiatric symptoms in nine dimensions, including somatization. Clients rate each item for the prior 7 days using a 5-point scale that ranges from 0 (not at all) to 4 (extremely). The SCL-90-R also provides several global indices. Higher scores indicate more symptoms with greater intensity.
- Modified PTSD Symptom Scale (MPSS-SR). The MPSS-SR is a 17-item self-report instrument that measures frequency and severity of PTSD symptomatology, according to DSM-III-R criteria, for the prior 2-week period. Frequency is measured on a 4-point scale that ranges from 0 (not at all) to 3 (≥5 times per week/very much/almost always). Scores range from 0 to 51 for frequency, and higher scores are associated with more frequent symptoms.
|
Key Findings
|
In a randomized clinical trial (RCT), combat veterans with a DSM-IV diagnosis of PTSD were randomly assigned to one of three conditions: 12 biweekly 60- to 75-minute sessions of EMDR, 12 biweekly 40-minute sessions of biofeedback-assisted relaxation, or a 6-week wait-list control condition. Assessments occurred at baseline (pretreatment), at 6 weeks after baseline (posttreatment), and at 3 months after posttreatment assessment (follow-up). Only the EMDR and relaxation conditions were contrasted at follow-up, since veterans in the wait-list control condition were offered treatment after the first 6 weeks of the study. Findings from this study included the following:
- Veterans receiving EMDR had less PTSD symptom severity (M-PTSD) relative to veterans in the control group (p < .05) and relaxation group (p < .05) at posttreatment and relative to veterans in the relaxation group (p < .05) at follow-up. These group differences were associated with large effect sizes (Cohen's d = 1.01, 1.07, and 1.01, respectively).
- Veterans receiving EMDR had less frequent PTSD symptoms (frequency scale of the CAPS-1; p < .0004) with less intensity (intensity scale of the CAPS-1; p < .002) relative to veterans in the relaxation group at follow-up. These group differences were associated with large effect sizes (Cohen's d = 2.10 and 1.82, respectively).
- Veterans receiving EMDR had greater improvement in overall PTSD symptom status (PTSD Symptoms Scale) relative to veterans in the control group (p < .01) from pre- to posttreatment and relative to veterans in the relaxation group (p < .005) from pretreatment to follow-up. These group differences were associated with large effect sizes (Cohen's d = 1.26 and 1.55, respectively).
In another RCT, adults experiencing traumatic memories were recruited from the community and randomly assigned to one of two conditions: three 90-minute EMDR sessions or a 6-week delayed-treatment control (followed by EMDR). Ninety-four percent of the participants met at least three DSM-IV criteria for PTSD in the 30 days prior to study entry. Assessments occurred at baseline (pretreatment) and at 1, 2, 3, 6, and 18 months after baseline (follow-ups). Between-condition contrasts were limited to relative change at the 1-month follow-up compared with the pretreatment assessment. From pretreatment to the 1-month follow-up, participants receiving EMDR had greater reductions in intrusive thoughts (intrusion subscale of the IES; p < .006), avoidance symptoms (avoidance subscale of the IES; p < .006), and somatic symptoms (somatization dimension of the SCL-90-R; p < .006) relative to participants in the control group. These group differences were associated with medium and large effect sizes (Cohen's d = 0.66, 1.03, and 1.35, respectively).
In a third RCT, which continued for 2 years, participants presenting to a health maintenance organization's (HMO's) psychiatry clinic with PTSD (DSM-III-R criteria) were randomly assigned to one of two conditions: 50-minute sessions of EMDR or standard HMO care for PTSD. The number of treatment sessions was not fixed for either condition, and treatment continued until participants no longer met DSM-III-R criteria for PTSD (typically between 6 and 12 months) or until the study ended. Standard HMO care for PTSD consisted of one or more of the following: individual psychotherapy sessions (cognitive, psychodynamic, or behavioral), medication (antidepressants or anxiolytics), group therapy (relaxation training, panic and anxiety reduction, and medication stabilization groups), and brief inpatient hospitalization and/or day treatment. Assessments were carried out at baseline (pretreatment), after three treatment sessions, at posttreatment, and at 3 and 6 months posttreatment (follow-ups). Findings from this study included the following:
- Relative to participants receiving standard HMO care, those receiving EMDR had fewer PTSD symptoms after three treatment sessions (IES total score and frequency scale of the MPSS-SR; p = .001 and p = .001, respectively).
- Relative to participants receiving standard HMO care, those receiving EMDR continued to have fewer PTSD symptoms (frequency scale of the MPSS-SR) at posttreatment (p = .005) and at the 3- and 6-month follow-ups (p = .002 and p = .004, respectively).
- Relative to participants receiving standard HMO care, those receiving EMDR had fewer PTSD symptoms (IES total score) at the 3- and 6-month follow-ups (p < .001 and p = .010, respectively).
- Relative to participants receiving standard HMO care, those receiving EMDR had a larger decrease in PTSD symptoms (IES total score) from pre- to posttreatment (p = .009).
- Fewer participants receiving EMDR than those receiving standard HMO care met DSM-III-R criteria for PTSD after three treatment sessions (50% vs. 80%; p = .034) and at posttreatment (23% vs. 50%; p = .025).
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
Study Designs
|
Experimental
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Outcome 2: Anxiety symptoms |
Description of Measures
|
Anxiety symptoms were measured by at least one of the following instruments in each of three studies:
- State-Trait Anxiety Inventory (STAI). The STAI is a 40-item self-report instrument with 20 items to measure current (or state) anxiety and 20 items to measure dispositional (or trait) anxiety. Each item is a statement about feelings or general tendencies, which respondents rate in regard to how they are feeling at the moment (state anxiety subscale) and more generally (trait anxiety subscale), using a 4-point scale ranging from 1 (not at all) to 4 (very much so). Scores are calculated separately for the state and trait anxiety subscales, and each subscale score ranges from 20 to 80, with higher scores indicating a higher level of anxiety.
- Subjective Units of Disturbance Scale (SUDS). The SUDS, extracted from the Subjective Anxiety Scale, is a single-item self-report measure of the anxiety disturbance experienced while thinking about a specific traumatic event. The rating ranges from 0 (neutral) to 10 (highest level of disturbance imaginable).
- Anxiety dimension of the SCL-90-R. The SCL-90-R is a 90-item self-report checklist that measures current psychiatric symptoms in nine dimensions, including anxiety. Clients rate each item for the prior 7 days using a 5-point scale that ranges from 0 (not at all) to 4 (extremely). The SCL-90-R also provides several global indices. Higher scores indicate more symptoms with greater intensity.
|
Key Findings
|
In an RCT, combat veterans with a DSM-IV diagnosis of PTSD were randomly assigned to one of three conditions: 12 biweekly 60- to 75-minute sessions of EMDR, 12 biweekly 40-minute sessions of biofeedback-assisted relaxation, or a 6-week wait-list control condition. Assessments occurred at baseline (pretreatment), at 6 weeks after baseline (posttreatment), and at 3 months after posttreatment assessment (follow-up). Only the EMDR and relaxation conditions were contrasted at follow-up, since veterans in the wait-list control condition were offered treatment after the first 6 weeks of the study. Veterans receiving EMDR had less dispositional anxiety (trait anxiety subscale of the STAI) relative to veterans in the control group (p < .001) and relaxation group (p < .001) at posttreatment and relative to veterans in the relaxation group (p < .01) at follow-up. These group differences were associated with large effect sizes (Cohen's d = 1.62, 1.15, and 1.38, respectively).
In another RCT, adults experiencing traumatic memories were recruited from the community and randomly assigned to one of two conditions: three 90-minute EMDR sessions or a 6-week delayed-treatment control (followed by EMDR). Ninety-four percent of the participants met at least three DSM-IV criteria for PTSD in the 30 days prior to study entry. Assessments occurred at baseline (pretreatment) and at 1, 2, 3, 6, and 18 months after baseline (follow-ups). Between-condition contrasts were limited to relative change at the 1-month follow-up compared with the pretreatment assessment. From pretreatment to 1-month follow-up, participants receiving EMDR had greater reductions in anxiety symptoms (SUDS, state and trait anxiety subscales of the STAI, and anxiety dimension of the SCL-90-R; p < .006 for all four scales) relative to participants in the control group. These group differences were associated with effect sizes ranging from small to large (Cohen's d = 2.07, 0.63, 0.44, and 0.49, respectively).
In a third RCT, which continued for 2 years, participants presenting to an HMO's psychiatry clinic with PTSD (DSM-III-R criteria) were randomly assigned to one of two conditions: 50-minute sessions of EMDR or standard HMO care for PTSD. The number of treatment sessions was not fixed for either condition, and treatment continued until participants no longer met DSM-III-R criteria for PTSD (typically between 6 and 12 months) or until the study ended. Standard HMO care for PTSD consisted of one or more of the following: individual psychotherapy sessions (cognitive, psychodynamic, or behavioral), medication (antidepressants or anxiolytics), group therapy (relaxation training, panic and anxiety reduction, and medication stabilization groups), and brief inpatient hospitalization and/or day treatment. Assessments were carried out at baseline (pretreatment), after three treatment sessions, at posttreatment, and at 3 and 6 months posttreatment (follow-ups). Findings from this study included the following:
- Relative to participants receiving standard HMO care, those receiving EMDR had fewer current (state anxiety subscale of the STAI; p < .05) and dispositional (trait anxiety subscale of the STAI; p = .013) anxiety symptoms and less anxiety disturbance while thinking of the traumatic event (SUDS; p = .001) after three treatment sessions.
- Relative to participants receiving standard HMO care, those receiving EMDR had a lower current anxiety level (state anxiety subscale of the STAI) at the 6-month follow-up (p = .017).
- Relative to participants receiving standard HMO care, those receiving EMDR had fewer dispositional anxiety symptoms (trait anxiety subscale of the STAI) at posttreatment (p = .005) and at the 3- and 6-month follow-ups (p = .023 and p = .007, respectively).
- Relative to participants receiving standard HMO care, those receiving EMDR had less anxiety disturbance while thinking of the traumatic event (SUDS) at posttreatment (p = .001) and at the 3- and 6-month follow-ups (p = .003 and p = .006, respectively).
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
Study Designs
|
Experimental
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Outcome 3: Depression symptoms |
Description of Measures
|
Depression symptoms were measured by one of the following instruments in each of three studies:
- Beck Depression Inventory (BDI). The BDI is a 21-item self-report instrument that assesses the severity of depression symptoms over the past week. Scores range from 0 to 63, with higher scores indicating greater severity of depression symptoms.
- Depression dimension of the SCL-90-R. The SCL-90-R is a 90-item self-report checklist that measures current psychiatric symptoms in nine dimensions, including depression. Clients rate each item for the prior 7 days using a 5-point scale that ranges from 0 (not at all) to 4 (extremely). The SCL-90-R also provides several global indices. Higher scores indicate more symptoms with greater intensity.
|
Key Findings
|
In an RCT, combat veterans with a DSM-IV diagnosis of PTSD were randomly assigned to one of three conditions: 12 biweekly 60- to 75-minute sessions of EMDR, 12 biweekly 40-minute sessions of biofeedback-assisted relaxation, or a 6-week wait-list control condition. Assessments occurred at baseline (pretreatment), at 6 weeks after baseline (posttreatment), and at 3 months after posttreatment assessment (follow-up). Only the EMDR and relaxation conditions were contrasted at follow-up, since veterans in the wait-list control condition were offered treatment after the first 6 weeks of the study. Veterans receiving EMDR had less severe depression symptoms (BDI) relative to veterans in the control group (p < .01) at posttreatment. This group difference was associated with a large effect size (Cohen's d = 1.48).
In another RCT, adults experiencing traumatic memories were recruited from the community and randomly assigned to one of two conditions: three 90-minute EMDR sessions or a 6-week delayed-treatment control (followed by EMDR). Ninety-four percent of the participants met at least three DSM-IV criteria for PTSD in the 30 days prior to study entry. Assessments occurred at baseline (pretreatment) and at 1, 2, 3, 6, and 18 months after baseline (follow-ups). Between-condition contrasts were limited to relative change at the 1-month follow-up compared with the pretreatment assessment. From pretreatment to 1-month follow-up, participants receiving EMDR had greater reductions in depression symptoms (depression dimension of the SCL-90-R; p < .006) relative to participants in the control group. This group difference was associated with a medium effect size (Cohen's d = 0.62).
In a third RCT, which continued for 2 years, participants presenting to an HMO's psychiatry clinic with PTSD (DSM-III-R criteria) were randomly assigned to one of two conditions: 50-minute sessions of EMDR or standard HMO care for PTSD. The number of treatment sessions was not fixed for either condition, and treatment continued until participants no longer met DSM-III-R criteria for PTSD (typically between 6 and 12 months) or until the study ended. Standard HMO care for PTSD consisted of one or more of the following: individual psychotherapy sessions (cognitive, psychodynamic, or behavioral), medication (antidepressants or anxiolytics), group therapy (relaxation training, panic and anxiety reduction, and medication stabilization groups), and brief inpatient hospitalization and/or day treatment. Assessments were carried out at baseline (pretreatment), after three treatment sessions, at posttreatment, and at 3 and 6 months posttreatment (follow-ups). Relative to participants receiving standard HMO care, those receiving EMDR had less severe depression symptoms (BDI) after three treatment sessions (p = .005), at posttreatment (p = .020), and at the 6-month follow-up (p = .012). Also at the 6-month follow-up, relative to participants receiving standard HMO care, more participants receiving EMDR had BDI scores that were no longer in the clinical depression range (defined by investigators as a BDI score of ≥12; p = .012).
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
Study Designs
|
Experimental
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Outcome 4: Global mental health functioning |
Description of Measures
|
Global mental health functioning was measured by the Global Severity Index and the Positive Symptom Distress Index of the SCL-90-R. The SCL-90-R is a 90-item self-report checklist that measures current psychiatric symptoms in nine dimensions: somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. Clients rate each item for the prior 7 days using a 5-point scale that ranges from 0 (not at all) to 4 (extremely). The SCL-90-R also provides several global indices, including the Global Severity Index, which is a measure of overall psychological distress and can be used as a summary measure for the instrument, and the Positive Symptom Distress Index, which is a measure of symptom intensity. Higher scores indicate more symptoms with greater intensity.
|
Key Findings
|
In an RCT that continued for 2 years, participants presenting to an HMO's psychiatry clinic with PTSD (DSM-III-R criteria) were randomly assigned to one of two conditions: 50-minute sessions of EMDR or standard HMO care for PTSD. The number of treatment sessions was not fixed for either condition, and treatment continued until participants no longer met DSM-III-R criteria for PTSD (typically between 6 and 12 months) or until the study ended. Standard HMO care for PTSD consisted of one or more of the following: individual psychotherapy sessions (cognitive, psychodynamic, or behavioral), medication (antidepressants or anxiolytics), group therapy (relaxation training, panic and anxiety reduction, and medication stabilization groups), and brief inpatient hospitalization and/or day treatment. Assessments were carried out at baseline (pretreatment), after three treatment sessions, at posttreatment, and at 3 and 6 months posttreatment (follow-ups). Findings from this study included the following:
- Relative to participants receiving standard HMO care, those receiving EMDR had less overall psychological distress (Global Severity Index of the SCL-90-R) after three treatment sessions (p = .016), at posttreatment (p = .022), and at the 3- and 6-month follow-ups (p = .002 and p = .037, respectively).
- Relative to participants receiving standard HMO care, those receiving EMDR reported less symptom intensity (Positive Symptom Distress Index of the SCL-90-R) after three treatment sessions (p = .001), at posttreatment (p = .017), and at the 3- and 6-month follow-ups (p = .005 and p = .022, respectively).
|
Studies Measuring Outcome
|
Study 3
|
Study Designs
|
Experimental
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
Study
|
Age
|
Gender
|
Race/Ethnicity
|
Study 1
|
26-55 (Adult) 55+ (Older adult)
|
100% Male
|
54.3% White 45.7% Race/ethnicity unspecified
|
Study 2
|
18-25 (Young adult) 26-55 (Adult) 55+ (Older adult)
|
50% Female 50% Male
|
96% White 4% Hispanic or Latino
|
Study 3
|
18-25 (Young adult) 26-55 (Adult) 55+ (Older adult)
|
79.1% Female 20.9% Male
|
66% White 13% Black or African American 12% Hispanic or Latino 7% Race/ethnicity unspecified 2% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
1: PTSD symptoms
|
3.5
|
3.8
|
3.3
|
2.3
|
3.0
|
3.3
|
3.2
|
2: Anxiety symptoms
|
3.5
|
4.0
|
3.0
|
2.3
|
3.0
|
3.3
|
3.2
|
3: Depression symptoms
|
3.5
|
4.0
|
3.3
|
2.3
|
3.0
|
3.3
|
3.2
|
4: Global mental health functioning
|
3.5
|
4.0
|
2.5
|
2.3
|
2.5
|
3.0
|
3.0
|
Study Strengths The measurement scales for each outcome are well known in the field and have strong psychometric properties. Convergent results across multiple scales for the same outcome suggest strong construct validity. In all three studies reviewed, the intervention was delivered by trained therapists according to a standardized treatment session protocol, isolation of the target population was achieved by extensive individual interviews prior to study enrollment, and randomized control designs were used to minimize potential confounds. The analytic approaches were appropriate for the limited sample size in each of the three studies.
Study Weaknesses Study sample reliability statistics for the measurement scales were not provided. Fidelity was not assessed using a review of audio- or videotaped treatment sessions or measurement instruments. Participant attrition was not addressed statistically, despite one study having moderate attrition (26% before randomized group assignment) and another study having high attrition at the 3-month (34%) and 6-month (46%) follow-up points. Each of the three studies had a limited sample size that restricted the use of more sophisticated data modeling approaches.
|
|