A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

Rinad S Beidas; Nathaniel J Williams; Emily M Becker-Haimes; Gregory A Aarons; Frances K Barg; Arthur C Evans; Kamilah Jackson; David Jones; Trevor Hadley; Kimberly Hoagwood; Steven C Marcus; Geoffrey Neimark; Ronnie M Rubin; Sonja K Schoenwald; Danielle R Adams; Lucia M Walsh; Kelly Zentgraf; David S Mandell

doi:10.1186/s13012-019-0912-4

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

Implement Sci. 2019 Jun 21;14(1):67. doi: 10.1186/s13012-019-0912-4.

Authors

Rinad S Beidas^{1

2

3}, Nathaniel J Williams⁴, Emily M Becker-Haimes^{5

6}, Gregory A Aarons⁷, Frances K Barg⁸, Arthur C Evans⁹, Kamilah Jackson¹⁰, David Jones¹¹, Trevor Hadley⁵, Kimberly Hoagwood¹², Steven C Marcus¹³, Geoffrey Neimark¹⁰, Ronnie M Rubin^{10

14}, Sonja K Schoenwald¹⁵, Danielle R Adams¹⁶, Lucia M Walsh¹⁷, Kelly Zentgraf⁵, David S Mandell^{5

18}

Affiliations

¹ Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. rbeidas@upenn.edu.
² Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. rbeidas@upenn.edu.
³ Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Hall- Mercer Community Mental Health Center, Philadelphia, PA, USA. rbeidas@upenn.edu.
⁴ School of Social Work, Boise State University, Boise, ID, USA.
⁵ Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
⁶ Hall- Mercer Community Mental Health Center, Philadelphia, FL, USA.
⁷ Department of Psychiatry, University of California San Diego, San Diego, CA, USA.
⁸ Department of Family Medicine and Community Health, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA.
⁹ American Psychological Association, Washington, DC, USA.
¹⁰ Community Behavioral Health, Impact Reach, LLC, Philadelphia, PA, USA.
¹¹ Department of Behavioral Health, Philadelphia, PA, USA.
¹² Department of Child and Adolescent Psychiatry, New York University Langone Health, New York, NY, USA.
¹³ School of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA.
¹⁴ Impact Reach, LLC, Philadelphia, PA, USA.
¹⁵ Oregon Social Learning Center, Eugene, OR, USA.
¹⁶ School of Social Service Administration, University of Chicago, Chicago, IL, USA.
¹⁷ Department of Psychology, University of Miami, Miami, FL, USA.
¹⁸ Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Hall- Mercer Community Mental Health Center, Philadelphia, PA, USA.

Abstract

Background: Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static.

Method: Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised.

Results: Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048).

Conclusions: System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

Keywords: Cognitive-behavioral therapy; Organizational factors; System-level implementation; Targets.

Publication types

Research Support, N.I.H., Extramural

MeSH terms

Adult
Community Mental Health Services / organization & administration*
Cross-Sectional Studies
Evidence-Based Practice*
Family Therapy*
Female
Humans
Male
Organizational Culture
Philadelphia
Practice Patterns, Physicians' / statistics & numerical data*

Abstract

Publication types

MeSH terms

Grants and funding