Impact of Structured Clinical Interviews on Physicians' Practices in Community Mental Health Settings

In an era of burgeoning scientific discoveries and evidence-based care, clinical practice guidelines have become an important tool for clinicians to improve the quality of care for persons with mental illness (1,2,3,4,5) as well as for managed care organizations to reduce health care costs (6,7,8). In response to declining resources, publicly supported health care systems, such as the Department of Veterans Affairs (9), and recently state agencies, such as the Texas Department of Mental Health and Mental Retardation (10), have begun to embrace clinical algorithms as potentially cost-effective additions to traditional mental health care. These algorithms are designed to aid clinical decision making by organizing strategic (what to do) and tactical (how to do) decisions into sequential stages (11,12). Presented as flowcharts or decision trees, algorithms provide guidance for prioritizing treatment options on the basis of patient status and therapeutic responses.

However, treatment algorithms presume that a correct diagnosis has been made, which can itself be a difficult task, especially when patients have multiple and interrelated problems. Structured clinical interviews provide reliable mental health diagnoses (13,14,15,16,17,18,19,20). Examples are the Diagnostic Interview Schedule (21), a lay-administered, computer-scored instrument designed for epidemiological surveys; the Schedule for Affective Disorders and Schizophrenia (22), an instrument designed to describe psychopathology, disease severity, and diagnoses; and the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I)—Clinician Version (23), a clinician-driven, patient-oriented interview that covers diagnostic criteria for major mental illnesses. However, resource constraints often limit practicing psychiatrists from using labor-intensive interviews. It has been suggested that psychiatric nurses or other nonphysician mental health professionals should administer structured diagnostic interviews to patients and report their findings to the practitioners making the final diagnosis (24). Such a plan has not been tested.

This study assessed the impact of psychiatric nurse-administered SCIDs on physicians' evaluation, diagnostic, and prescription practices in a publicly funded outpatient mental health setting.

Methods

Patients

The patients came from two outpatient clinics operated by Dallas County Mental Health and Mental Retardation (DCMHMR) between December 15, 1995, and June 13, 1997. Eligible participants were those who were eligible for DCMHMR services; signed informed consent as approved by the institutional review board for the University of Texas Southwestern Medical Center; had not visited DCMHMR during the previous year; received an initial evaluation, diagnosis, and medication treatment at a study clinic; were physically and mentally suitable for an interview; had permission to enter the study from the attending physician; and returned to the clinic for care at least once.

Consistent with usual practice, all new DCMHMR patients—that is, those who had not been seen in more than a year—began with an initial visit that included assessments by a licensed professional health care associate, a master's in social work, or a licensed professional counselor to determine DCMHMR service eligibility and service needs (30 minutes); a psychiatric evaluation (60 minutes); and indicated laboratory tests, casework (psychosocial history, appointment schedule, treatment plan, return to work goals), and service agency referrals (60 minutes) and closed with the signing of a treatment plan by the attending psychiatrist, the caseworker, and the patient.

After the routine clinical intake, a nurse coordinator enrolled eligible patients at the conclusion of their initial visit during 15-minute intake interviews during which study eligibility was checked, demographic information was collected, and participants were randomly assigned to either the SCID group or the control group.

Randomization

During study intake, an off-site investigator used random-number tables to assign consenting participants to either the SCID group or the control group. Sampling ratios of three patients in the SCID group for every two in the control group provided sufficient sample sizes for within-SCID group analyses. Randomization was conducted after the psychiatric evaluation to avoid influencing the initial clinician-rendered diagnoses.

SCID

Patients who were assigned to receive SCIDs were scheduled to return to the clinic to receive their interview within two weeks after intake. Nurses entered SCID results into charts and provided five- to ten-minute consultations with attending psychiatrists who, in turn, could reevaluate patients and change diagnoses, medications, or dosages.

Registered nurses with psychiatric and mental health certification from the American Nurse Credentialing Center administered the SCID after participating in a ten-week training course (25) that was conducted by a doctoral-level clinical psychologist trained by the author of the SCID. After two-day didactic training, the nurses participated in two role-play interviews, two observed interviews, four supervised interviews, and 12 to 18 practice interviews. Certification was awarded when the nurse's and the corater's results agreed on two consecutive SCID interviews and the nurse passed a four-item interviewing style test and a 14-item diagnostic information skills test (25).

Assessments

Patients' use of DCMHMR services, chart diagnoses, and prescription information came from DCMHMR clinical databases as confirmed by chart reviews. DCMHMR outpatient mental health visits were counted and classified as those that included an evaluation procedure (psychiatric, psychological, neurological, medical, behavioral, or psychosocial); treatment procedure (medication management, family therapy, education and counseling, individual and group therapy, or prescription); update of diagnosis whenever a three- to five-digit DSM-IV (26) axis I code was entered into the chart after intake; changes in diagnosis involving a deletion of all previously listed disorders from a single category or an addition of at least one disorder from a previously unlisted category; and any new or refill prescription noted in the chart, including those that added a category of drugs not previously prescribed for the patient or deleted all previously prescribed medications from at least one drug category; and increased or decreased daily dosage for currently prescribed medication. Daily dosages were measured as a percentage of an average DCMHMR dosage calculated for each named drug and across prescriptions. Clinic stops that patients made to receive SCIDs were not counted as outpatient visits.

Statistical methods

For discrete variables, group differences were tested with a two-tailed Yates' continuity-corrected χ². For continuous variables, group differences were tested with two-tailed t tests where equal variances between groups were not assumed and with the dependent variable normalized by using log transformations, whenever appropriate. Use of clinic services was described by user rates (percentage of patients with at least one encounter), volume (number of visits per user), and content (proportion of visits that included a given procedure or characteristic, such as an evaluation taken, diagnosis changed, or medication prescribed). Clinic visits are nested within subjects, so between-group content differences were expressed as odds ratios computed from hierarchical logistic models with level-1 visits and level-2 patients classified as either SCID or control (27). To adjust for chance agreement, diagnostic reliability was assessed by using the kappa statistic (28), with a kappa below .4 indicating poor agreement (29).

Results

During the study period, 26 psychiatrists saw 5,569 adult patients during 33,949 encounters, logging a mean± SD number of patients of 3.2±1.3 per working hour (range, .1 to 12.5), for a total of 8.8±5.1 patients per working day (range, 1 to 31). Among 1,300 referred patients, some were ineligible for the study because they had had a DCMHMR encounter in the previous year (48 patients), were rejected for SCID administration by the attending physician (18 patients), had language barriers (23 patients), did not show up for the interview (52 patients), or were ineligible for DCMHMR services (two patients). Among 1,157 eligible patients, 121 (9.3 percent) refused to participate, 564 (43.4 percent) failed to show up for the study intake interview, and 68 (5.2 percent) did not complete the intake interview. The remaining 404 eligible patients were randomly assigned to a study group at intake. Of these, 108 failed to return for care after their initial visit.

The mean age of the 296 patients who participated in the study was 38.6±10.9 years (range, 18 to 80), and the patients were demographically mixed: 32 (11 percent) were African American, 17 (6 percent) were Hispanic, 204 (69 percent) were women, and 47 (16 percent) were married. The mean monthly income, in 1997 dollars, was $458±512. A total of 38 patients (13 percent) owned homes, 92 (31 percent) were employed, 184 (62 percent) had graduated from high school, and 77 (26 percent) received food stamps. The impact of refusals and no-shows on internal validity is likely to be small, because no statistically significant demographic or health-related difference was found between the SCID group (N=158) and the control group (N=138).

It is possible that there was some impact on external validity. Compared with all DCMHMR adult mental health patients treated during the study period who had had a previous clinic visit (N=5,569), the study participants included fewer males (31 percent compared with 41 percent), African Americans (11 percent compared with 23 percent), and Hispanics (6 percent compared with 8 percent) and were slightly older when DCMHMR treatment began (39 compared with 37 years). Compared with nonreturning but eligible patients (N=108), the study participants reported poorer physical functioning at intake on the basis of the SF-36 (30,31) (mean score of 71.3±27.1 compared with 78.4±26.6; t=2.28, df=393, p<.023) but otherwise were not significantly different by income, gender, ethnicity, age, or mental and social functioning.

Feasibility

All SCID-assigned patients who returned to the clinic (N=158) received SCIDs within a mean 13.6±13.8 days after intake (range, 0 to 85). Results were entered into charts and attending psychiatrists consulted within two days of administration of the SCIDs. For each patient, nurses spent an average of 122.6±53.3 minutes (range, 30 to 280) conducting SCID interviews and 34.3±17.4 minutes (range, 0 to 135) on administrative matters, such as scheduling interviews and assisting with physician consultations.

Use of care

During the first 90 days after the initial visit, all patients made at least one visit. Those who received SCIDs averaged two more visits than their counterparts in the control group (8.41 compared with 6.36; t=2.41, df=286, p<.017). During the second quarter, 133 patients in the SCID group (84.2 percent) and 121 in the control group (87.7 percent) had at least one visit, with comparable mean numbers of visits between groups who used services.

Table 1 lists the proportions of patients who made one visit and the percentage of visits, by type, patient group, and quarter. During the first quarter, during any given visit, physicians were 3.8 times as likely to provide an evaluation procedure but half as likely to provide a treatment procedure for patients who received SCIDs compared with control patients. These differences were short lived, with no significant differences observed by the second quarter.

Diagnoses

During the first quarter, the patients who received SCIDs were 110 times as likely to have at least one visit during which a diagnosis was updated and 14 times as likely to have at least one visit during which a diagnosis changed as the control patients (Table 1). For any given visit, patients receiving SCIDs were twice as likely to have an update in diagnosis and four times as likely to have a change in diagnosis as control patients. These differences did not persist through the second quarter.

Changes in clinical diagnoses were assessed as the proportion of patients for whom a disorder category was deleted or added between intake and week 26. A higher percentage of patients in the SCID group than in the control group had a diagnosis deleted or added. The six most frequently reported disorders on the basis of the SCID are listed in Table 2. For patients with an initial diagnosis of schizophrenia, three of 18 patients in the SCID group (17 percent) had their diagnosis changed to bipolar disorder, compared with zero in the control group, and an additional three (17 percent) in the SCID group had their diagnosis changed to major depressive disorder, compared with zero in the control group.

For bipolar disorder, two (6 percent) of 35 patients in the SCID group and one (4 percent) of 27 in the control group had their diagnosis changed to schizophrenia; five (14 percent) in the SCID group and zero in the control group had their diagnosis changed to major depressive disorder. For major depressive disorder, five (5 percent) of 101 patients in the SCID group and two (2 percent) of 86 in the control group had their diagnosis changed to schizophrenia; 13 patients in the SCID group (13 percent) and one (1 percent) in the control group had their diagnosis changed to bipolar disorder (χ²=7.58, df=1, p<.006).

Overall, 31 patients in the SCID group (20 percent) and three (2 percent) in the control group had their diagnosis changed from one severe disorder to another (χ²=20.52, df=1, p<.001) during the 180-day period. In addition, for 81 patients in the SCID group (51 percent) a diagnosis of substance use disorder, anxiety, or eating disorder was added, compared with seven (5 percent) patients in the control group (χ²=73.04, df=1, p<.001).

SCID versus chart diagnosis

Table 3 shows agreement between the SCID and chart diagnoses abstracted at intake and at eight, 16, and 26 weeks. Adjusting for chance, we found that agreement between the SCID and chart diagnoses improved with time, ranging at intake from .52 to .60 (severe mental illness) and .20 to .30 (other disorders) to between .80 and .90 for all disorders after 16 weeks. Agreement improved because uncorroborated rates for both the SCID and chart diagnoses declined. For example, the kappa statistic for major depressive disorders increased from .56 at intake to .90 by the 16th week, with declines in the number of patients identified in the chart but not in the SCID and in the SCID but not in the chart. The evidence suggests that physicians updated charts so that they would be consistent with SCID results. Furthermore, 27 (87 percent) of the 31 patients in the SCID group whose diagnoses were changed by the 26th week had a new diagnosis consistent with their SCID results.

Prescriptions

Table 1 lists the proportions of patients and treatment visits at which medications were prescribed, by SCID group. Information from the SCID had little impact during the first 90 days of care, although patients who received SCIDs were less likely to have their dosage increased for any given follow-up visit when a previously prescribed medication was reviewed by the physician. By the second quarter, patients who received SCIDs were more likely to have a prescription stopped and, visit for visit, were more likely to have a prescription deleted (by a factor of 2.3), changed (by a factor of 2.8), or otherwise noted (by a factor of 1.6).

Generally, few differences were noted between the SCID group and the control group in the number of individual medications or classes of drugs prescribed during the 180 days of care. The average dosage (as a percentage of average dosage based on the weight of the active ingredient over all DCMHMR prescriptions with the same medication name) did not differ significantly between the two groups during days 1 to 19 (92 percent in the SCID group compared with 92 percent in the control group), days 20 to 89 (92 percent compared with 101 percent), or days 90 to 180 (102 percent compared with 101 percent). However, initial dosages were lower among patients in the SCID group during the second quarter: 82 percent compared with 89 percent for days 1 to 89 and 71 percent compared with 99 percent for days 90 to 180 (t=2.38. df=82, p<.017).

Overall, 19 patients in the SCID group (12 percent) and one (1 percent) in the control group had both a change in diagnosis and a new prescription for an agent not previously prescribed: antipsychotics for a new diagnosis of schizophrenia, antimanics for bipolar disorder, and antidepressants for major depressive disorder (χ²=10.95, df=1, p<.001). This comparison is important for revealing whether differences in chart diagnoses between patients in the SCID group and those in the control group reflect real changes in diagnoses or merely differences in record keeping.

Discussion and conclusions

Inaccurate diagnoses are common in community clinics, often preventing physicians from selecting proper treatments (32) and leaving patients and their families uncertain. Nurse-administered SCID interviews may improve the accuracy of diagnoses (24). This study found that nurse-administered SCIDs led physicians in routine practice at two urban, publicly supported clinics to reevaluate patients, change diagnoses, and change medication prescriptions consistent with SCID findings and to offer medications at initially lower dosages to patients who received SCIDs compared with patients in a control group.

Although dramatic, these findings are not out of step with those reported in the literature. In a meta-analysis of randomized controlled trials (33), physician feedback from patients' self-reported depression ratings—the Zung Self-Rating Depression Scale (34) and the Beck Depression Inventory (35)—was associated with changes in diagnoses (seven of 13 trials) or treatment (three of eight trials). The larger effect sizes we found for the patients who received SCIDs may reflect a focus on psychiatrists rather than generalists in primary care settings; use of a clinical interview rather than self-reported questionnaires as feedback instruments; provision of face-to-face consultations by a professional nurse rather than summary reports; and primary rather than specialty mental health care settings in which changes in diagnoses are likely to lead to changes in type of medication.

Although the patients who participated in this study were not assessed for outcome—which is a limitation of the study—a correct diagnosis is important in its own right. Incorrect diagnoses can result in the prescription of incorrect medication, causing unnecessary side effects, and can also obfuscate the education of patients and family members about mental disorders, erode patient confidence in the attending psychiatrist, and reduce patients' acceptance of clinical advice. Correct and timely diagnoses enable psychiatrists to begin appropriate therapy early and to reduce the number of medications as well as dosages that must be tried before the patient obtains relief from unwanted symptoms. In fact, the greater number of psychiatric diagnoses reported for the patients in the SCID group did not appear to be associated with any increase in the number or dosage of prescribed medications. On the contrary, the patients in the control group were more likely to have their dosages increased during the first quarter. This finding may indicate that in the case of patients who do not respond to initial treatment, having the SCID findings may help physicians to reconsider the patient's diagnosis before increasing the medication dosage. Further study in this area is recommended.

Our findings are limited to one service system, and many eligible patients were lost because they did not show up for interviews, placing some doubt on the representativeness of the sample. Because participating physicians treated patients in both the SCID group and the control group, between-group differences may have been understated to the extent that nurse consultations provided an education effect. On the other hand, parallel changes in both diagnosis and prescribed treatments consistent with SCID results suggest an impact on physicians' thinking and not merely an impact on the quality of medical record keeping. When queried at the end of the study, many participating physicians claimed that they would order SCIDs for their more difficult patients.

The important conclusion from this study is that having a well-trained psychiatric nurse administer a structured clinical interview and provide results to psychiatrists appears to be both feasible and clinically useful. Future studies should consider targeting patients who may be at a higher risk of receiving a misdiagnosis, such as children who communicate through a guardian and persons from ethnic minorities who may face cultural barriers to obtaining care; disseminating SCID results to the physician before the initial psychiatric evaluation; assessing changes in primary symptoms and side-effect burden; and expanding time allotted for intake psychiatric evaluations.

Acknowledgments

This project was supported in part by Mental Health Connections, a partnership between Dallas County Mental Health and Mental Retardation and the department of psychiatry at the University of Texas Southwestern Medical Center, with funding from the Texas State Legislature and the Dallas County Hospital District. The project was also supported by the Department of Veterans Affairs, Health Services Research and Development Research Career Scientist Award (RCS 92-403) and by the National Institute of Mental Health (5-R24-MH53799).

Dr. Kashner, Dr. Rush, Dr. Surís, Dr. Biggs, Dr. Shoaf, and Dr. Altshuler are affiliated with the department of psychiatry at the University of Texas Southwestern Medical Center at Dallas, 5323 Harry Hines Boulevard, Dallas, Texas 75390-9086 (e-mail, [email protected]). Dr. Kashner and Dr. Surís are also with the Department of Veterans Affairs North Texas Health Care System in Dallas, with which Ms. Gajewski is affiliated. Ms. Hooker is with Dallas MetroCare Services. A version of this paper was presented at the annual meeting of the Academy for Health Services Research and Health Policy held June 25 to 28, 2000, in Los Angeles.

Enlarge table

Enlarge table

Enlarge table