Research Article| Volume 64, ISSUE 4, P535-542, April 2023

Impact of an Urban Project ECHO: Safety-Net Clinician Self-Efficacy Across Conditions

Published:January 10, 2023DOI:


      To mitigate the lack of specialty healthcare, Project ECHO (Extension for Community Health Outcomes) trains community-based primary care clinicians to better prevent the progression of, manage, and treat common health conditions. ECHO-Chicago launched in 2010 as the first urban-centered ECHO program, focusing on safety-net clinicians, and has trained over 5,175 community clinicians across 34 topic areas. This paper examines self-efficacy among ECHO-Chicago participants across 11 clinical series, including a novel use of qualitative themes from self-efficacy questions.


      Five years of baseline and postseries survey data were collected from 2014 to 2019, resulting in 951 participants. Paired t-tests assessed change from baseline survey to postsurvey, and Cohen's d determined effect size. Change was assessed by individual series, adult or pediatric focus, participants’ prescription privilege status, and across series by qualitative question theme. Metrics included total change, any improvement, a 10% target, and a clinical competency threshold. Analysis occurred from July 2020 to January 2022.


      All clinical series achieved statistically significant improvement in self-efficacy, and most had a large effect size. A total of 88% had any improvement, 65% met the 10% target of 0.7 points, and 52% met the competency threshold of 5.0 in the postsurvey. Prescribers had a significantly greater increase in their self-efficacy scores than nonprescribers. With a comparison across series, each theme achieved statistical significance, with most reaching large effect sizes.


      ECHO-Chicago successfully increased participants’ self-efficacy. This inquiry adds an urban focus, years of data, multiple series, and a novel qualitative theme component to enable comparisons across rather than solely within the ECHO series.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to American Journal of Preventive Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • IHS Markit Ltd
        The complexities of physician supply and demand: projections from 2018 to 2033.
        Association of American Medical Colleges, Washington, DC2020 (Published June. Accessed December 20, 2021)
        • Ezeonwu MC.
        Specialty-care access for community health clinic patients: processes and barriers.
        J Multidiscip Healthc. 2018; 11: 109-119
        • Lee MS
        • Nambudiri V.
        Integrating telemedicine into training: adding value to graduate medical education through electronic consultations.
        J Grad Med Educ. 2019; 11: 251-254
        • Patrick G
        • Bisgaier J
        • Hasham I
        • Navarra T
        • Hickner J.
        Specialty care referral patterns for the underserved: a study of community health centers on the South Side of Chicago.
        J Health Care Poor Underserved. 2011; 22: 1302-1314
        • Arora S
        • Thornton K
        • Murata G
        • et al.
        Outcomes of treatment for hepatitis C virus infection by primary care providers.
        N Engl J Med. 2011; 364: 2199-2207
      1. Cook NL, Hicks LS, O'Malley AJ, Keegan T, Guadagnoli E, Landon BE. Access to specialty care and medical services in community health centers. Health Aff (Millwood). 2007;26(5):1459–1468.

        • Reschovsky JD
        • O'Malley AS
        Do primary care physicians treating minority patients report problems delivering high-quality care?.
        Health Aff (Millwood). 2008; 27: w222-w231
        • Gruen RL
        • Weeramanthri TS
        • Knight SE
        • Bailie RS
        Specialist outreach clinics in primary care and rural hospital settings.
        Cochrane Database Syst Rev. 2004; 2003CD003798
        • Doykos PM.
        Striving for equity in access to and use of specialty care.
        HealthAffairs, Bethesda, MD2016 (Published August 1, 2016. Accessed July 9, 2019)
        • Cha AE
        • Cohen RA.
        Demographic variation in health insurance coverage: United States, 2021.
        Centers for Disease Control and Prevention, Atlanta, GA2022 (Published November 3)
        • Institute of Medicine (U.S.) Committee on the Changing Market
        Managed care, and the future viability of safety net providers.
        in: Ein Lewin M Altman S Americas's Health Care Safety Net: Intact but Endangered. U.S.: National Academies Press, Washington, DC2000 (Accessed February 16, 2021)
        • Skinner AC
        • Mayer ML.
        Effects of insurance status on children's access to specialty care: a systematic review of the literature.
        BMC Health Serv Res. 2007; 7: 194
        • Arora S
        • Thornton K
        • Jenkusky SM
        • Parish B
        • Scaletti JV.
        Project ECHO: linking university specialists with rural and prison-based clinicians to improve care for people with chronic hepatitis C in New Mexico.
        Public Health Rep. 2007; 122: 74-77
        • Socolovsky C
        • Masi C
        • Hamlish T
        • et al.
        Evaluating the role of key learning theories in ECHO: A telehealth educational program for primary care providers.
        Prog Community Health Partnersh. 2013; 7: 361-368
        • Loftus J
        • Allen EM
        • Call KT
        • Everson-Rose SA
        Rural-urban differences in access to preventive health care among publicly insured minnesotans.
        J Rural Health. 2018; 34: s48-s55
        • Lippke S.
        Self-efficacy theory.
        in: Zeigler-Hill V Shackelford TK Encyclopedia of Personality and Individual Differences. Springer International Publishing, Berlin, Germany2017: 1-6
        • Marzillier J
        • Eastman C.
        Continuing problems with self-efficacy theory: a reply to Bandura.
        Cognit Ther Res. 1984; 8: 257-262
        • Stajkovic AD
        • Luthans F.
        Self-efficacy and work-related performance: a meta-analysis.
        Psychol Bull. 1998; 124: 240-261
        • Strecher VJ
        • DeVellis BM
        • Becker MH
        • Rosenstock IM
        The role of self-efficacy in achieving health behavior change.
        Health Educ Q. 1986; 13: 73-92
        • Bikinesi L
        • O'Bryan G
        • Roscoe C
        • et al.
        Implementation and evaluation of a Project ECHO telementoring program for the Namibian HIV workforce.
        Hum Resour Health. 2020; 18: 61
        • Hostutler CA
        • Valleru J
        • Maciejewski HM
        • Hess A
        • Gleeson SP
        • Ramtekkar UP.
        Improving pediatrician's behavioral health competencies through the project ECHO teleconsultation model.
        Clin Pediatr (Phila). 2020; 59: 1049-1057
        • Mazurek MO
        • Parker RA
        • Chan J
        • Kuhlthau K
        • Sohl K
        • Autism Collaborative ECHO
        Effectiveness of the extension for community health outcomes model as applied to primary care for autism: a partial stepped-wedge randomized clinical trial.
        JAMA Pediatr. 2020; 174e196306
        • Sockalingam S
        • Clarkin C
        • Serhal E
        • Pereira C
        • Crawford A.
        Responding to health care professionals’ mental health needs during COVID-19 through the rapid implementation of project ECHO.
        J Contin Educ Health Prof. 2020; 40: 211-214
        • Giachetto G
        • Casuriaga AL
        • Santoro A
        • et al.
        Extension for community healthcare outcomes Uruguay: a new strategy to promote best primary care practice for autism.
        Glob Pediatr Health. 2019; 6 (2333794X19833734)
        • Komaromy M
        • Madden EF
        • Hager B
        • et al.
        Improvement in behavioral health symptoms and functioning among rural patients cared for by primary care teams using the extension for community health care outcomes model.
        J Rural Ment Health. 2019; 43: 73-80
        • Lewiecki EM
        • Rochelle R.
        Project ECHO: telehealth to expand capacity to deliver best practice medical care.
        Rheum Dis Clin North Am. 2019; 45: 303-314
        • Momin B
        • Mera J
        • Essex W
        • et al.
        Implementation of liver cancer education among health care providers and community coalitions in the Cherokee nation.
        Prev Chronic Dis. 2019; 16: E112
        • Tofighi B
        • Isaacs N
        • Byrnes-Enoch H
        • et al.
        Expanding treatment for opioid use disorder in publicly funded primary care clinics: exploratory evaluation of the NYC health + hospitals buprenorphine ECHO program.
        J Subst Abuse Treat. 2019; 106: 1-3
        • Yennurajalingam S
        • Amos Jr, CE
        • Weru J
        • et al.
        Extension for community healthcare outcomes-palliative care in Africa program: improving access to quality palliative care.
        J Glob Oncol. 2019; 5: 1-8
        • Flynn DM
        • Eaton LH
        • McQuinn H
        • et al.
        TelePain: primary care chronic pain management through weekly didactic and case-based telementoring.
        Contemp Clin Trials Commun. 2017; 8: 162-166
        • Eaton LH
        • Gordon DB
        • Wyant S
        • et al.
        Development and implementation of a telehealth-enhanced intervention for pain and symptom management.
        Contemp Clin Trials. 2014; 38: 213-220
        • Harris PA
        • Taylor R
        • Thielke R
        • Payne J
        • Gonzalez N
        • Conde JG.
        Research electronic data capture (REDCap)–A metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42: 377-381
        • Harris PA
        • Taylor R
        • Minor BL
        • et al.
        The REDCap consortium: building an international community of software platform partners.
        J Biomed Inform. 2019; 95103208
        • Arora S
        • Kalishman S
        • Thornton K
        • et al.
        Expanding access to hepatitis C virus treatment–Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care.
        Hepatology. 2010; 52: 1124-1133
      2. Metropolitan and micropolitan statistical area reference files. U.S. Updated 2020. Accessed July 19, 2022.

        • Estrada E
        • Ferrer E
        • Pardo A.
        Statistics for evaluating pre-post change: relation between change in the distribution center and change in the individual scores.
        Front Psychol. 2018; 9: 2696
        • Sawilowsky SS.
        New effect size rules of thumb.
        J Mod App Stat Meth. 2009; 8: 597-599
      3. R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Published December 2019. Accessed November 2020.

        • Ggpubr KA.
        “ggplot2” Based publication ready plots. Vol R package version 0.4.0.
        R Foundation for Statistical Computing, Vienna, Austria2020 (Published June 27. Accessed December 1, 2021)
        • Elo S
        • Kyngäs H.
        The qualitative content analysis process.
        J Adv Nurs. 2008; 62: 107-115
      4. Core competencies. Stanford Medicine Graduate Medical Education. Updated 2020. Accessed January 7, 2022.

      5. Diversity in Medicine: facts and figures 2019. AAMC. Updated 2019. Accessed September 20, 2022.

        • Abraham TH
        • Stewart GL
        • Solimeo SL.
        The importance of soft skills development in a hard data world: learning from interviews with healthcare leaders.
        BMC Med Educ. 2021; 21: 147
        • Ratka A.
        Empathy and the development of affective skills.
        Am J Pharm Educ. 2018; 82: 7192
        • Caruana CJ
        • Damilakis J
        Being an excellent scientist is not enough to succeed! Soft skills for medical physicists.
        Eur J Radiol. 2022; 155110108
        • Mercer SW
        • Reynolds WJ.
        Empathy and quality of care.
        Br J Gen Pract. 2002; 52: S9-12
      6. Learning objectives for medical student education–guidelines for medical schools: report I of the Medical School Objectives Project.
        Acad Med. 1999; 74: 13-18
        • Epstein RM
        • Street Jr, RL
        The values and value of patient-centered care.
        Ann Fam Med. 2011; 9: 100-103
        • Becevic M
        • Nair P
        • Wallach E
        • Hoffman K
        • Sohl K.
        ECHO autism: evaluation of participants’ perceptions of collaborative telementoring network.
        J Patient Exp. 2021; 823743735211065292
        • Agley J
        • Delong J
        • Janota A
        • Carson A
        • Roberts J
        • Maupome G.
        Reflections on project ECHO: qualitative findings from five different ECHO programs.
        Med Educ Online. 2021; 261936435
      7. Ball S, Stryczek K, Stevenson L, et al. A qualitative evaluation of the pain management VA-ECHO program using the RE-AIM framework: the participant's perspective. Front Public Health. 2020;8:169.

        • Howard GS
        • Dailey PR.
        Response-shift bias: a source of contamination of self-report measures.
        J Appl Psychol. 1979; 64: 144-150
        • Geldhof GJ
        • Warner DA
        • Finders JK
        • Thogmartin AA
        • Clark A
        • Longway KA.
        Revisiting the utility of retrospective pre-post designs: the need for mixed-method pilot data.
        Eval Program Plann. 2018; 70: 83-89