Advertisement

Cardiovascular Disease Guideline Adherence: An RCT Using Practice Facilitation

Published:February 14, 2020DOI:https://doi.org/10.1016/j.amepre.2019.12.013

      Introduction

      Practice facilitation is a promising practice transformation strategy, but further examination of its effectiveness in improving adoption of guidelines for multiple cardiovascular disease risk factors is needed. The objective of the study is to determine whether practice facilitation is effective in increasing the proportion of patients meeting the Million Hearts ABCS outcomes: (A) aspirin when indicated, (B) blood pressure control, (C) cholesterol management, and (S) smoking screening and cessation intervention.

      Study design

      The study used a stepped-wedge cluster RCT design with 4 intervention waves. Data were extracted for 13 quarters between January 1, 2015 and March 31, 2018, which encompassed the control, intervention, and follow-up periods for all waves, and analyzed in 2019.

      Setting/participants

      A total of 257 small independent primary care practices in New York City were randomized into 1 of 4 waves.

      Intervention

      The intervention consisted of practice facilitators conducting at least 13 practice visits over 1 year, focused on capacity building and implementing system and workflow changes to meet cardiovascular disease care guidelines.

      Main outcome measures

      The main outcomes were the Million Hearts’ ABCS measures. Two additional measures were created: (1) proportion of tobacco users who received a cessation intervention (smokers counseled) and (2) a composite measure that assessed the proportion of patients meeting treatment targets for A, B, and C (ABC composite).

      Results

      The S measure improved when comparing follow-up with the control period (incidence rate ratio=1.152, 95% CI=1.072, 1.238, p<0.001) and when comparing follow-up with intervention (incidence rate ratio=1.060, 95% CI=1.013, 1.109, p=0.007). Smokers counseled improved when comparing the intervention period with control (incidence rate ratio=1.121, 95% CI=1.037, 1.211, p=0.002).

      Conclusions

      Increasing the impact of practice facilitation programs that target multiple risk factors may require a longer, more intense intervention and greater attention to external policy and practice context.

      Trial registration

      This study is registered at www.clinicaltrials.gov NCT02646488.

      INTRODUCTION

      Cardiovascular disease (CVD) remains the leading cause of death in the U.S.
      • Van Dyke M
      • Greer S
      • Odom E
      • et al.
      Heart disease death rates among blacks and whites aged ≥35 years—United States, 1968–2015.
      In 2012, the Department of Health and Human Services launched the Million Hearts initiative to reduce CVD-related deaths through clinical and community-based interventions. Million Hearts set specific priorities for improving ABCS outcomes for CVD prevention: (A) appropriate aspirin use, (B) blood pressure (BP) control, (C) cholesterol management, and (S) tobacco use screening and cessation interventions.
      CDC
      Million hearts: strategies to reduce the prevalence of leading cardiovascular disease risk factors–United States, 2011.
      Practice changes that optimize the use of health information technology and care processes can facilitate implementation of evidence-based guidelines as defined by Million Hearts ABCS outcomes.

      Million Hearts HHS. Framework. https://millionhearts.hhs.gov/files/MH-Framework.pdf. Accessed January 10, 2018, 2022.

      However, redesigning clinical care is complex and requires expertise and resources that practices, especially small ones, may lack. Small independent practices (SIPs) often face distinct practice transformation challenges including a lack of health information technology expertise and support staff infrastructure, financial and resource constraints, inflexible organizational structures, and smaller panel sizes.
      • Landon BE
      • Normand SL.
      Performance measurement in the small office practice: challenges and potential solutions.
      • Wolfson D
      • Bernabeo E
      • Leas B
      • Sofaer S
      • Pawlson G
      • Pillittere D
      Quality improvement in small office settings: an examination of successful practices.
      • Nutting PA
      • Crabtree BF
      • McDaniel RR
      Small primary care practices face four hurdles–including a physician-centric mind-set–in becoming medical homes.
      • Rittenhouse DR
      • Casalino LP
      • Shortell SM
      • et al.
      Small and medium-size physician practices use few patient-centered medical home processes.
      Practice facilitation (PF) is an implementation strategy that may effectively address these barriers. Facilitators provide external expertise and support to help clinicians and healthcare delivery systems make meaningful practice changes, tailored to local context. They also build capacity for ongoing practice improvement.
      • Laferriere D
      • Liddy C
      • Nash K
      • Hogg W
      Navigating change: how outreach facilitators can help clinicians improve patient outcomes.
      • Baskerville NB
      • Liddy C
      • Hogg W
      Systematic review and meta-analysis of practice facilitation within primary care settings.
      • Hemler JR
      • Hall JD
      • Cholan RA
      • et al.
      Practice facilitator strategies for addressing electronic health record data challenges for quality improvement: EvidenceNOW.
      • Harvey G
      • Lynch E.
      Enabling continuous quality improvement in practice: the role and contribution of facilitation.
      • Stetler CB
      • Legro MW
      • Rycroft-Malone J
      • et al.
      Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration.
      • Berta W
      • Cranley L
      • Dearing JW
      • Dogherty EJ
      • Squires JE
      • Estabrooks CA
      Why (we think) facilitation works: insights from organizational learning theory.
      • Kitson AL
      • Rycroft-Malone J
      • Harvey G
      • McCormack B
      • Seers K
      • Titchen A
      Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges.
      • Liddy C
      • Rowan M
      • Valiquette-Tessier SC
      • Drosinis P
      • Crowe L
      • Hogg W
      Improved delivery of cardiovascular care (IDOCC): findings from narrative reports by practice facilitators.
      • Dogherty EJ
      • Harrison MB
      • Graham ID
      Facilitation as a role and process in achieving evidence-based practice in nursing: a focused review of concept and meaning.
      • Rycroft-Malone J
      • Seers K
      • Chandler J
      • et al.
      The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework.
      • Lessard S
      • Bareil C
      • Lalonde L
      • et al.
      External facilitators and interprofessional facilitation teams: a qualitative study of their roles in supporting practice change.
      • Harvey G
      • Loftus-Hills A
      • Rycroft-Malone J
      • et al.
      Getting evidence into practice: the role and function of facilitation.
      Several studies and a meta-analysis have found that PF is effective in improving preventive care delivery and guideline adoption.
      • Baskerville NB
      • Liddy C
      • Hogg W
      Systematic review and meta-analysis of practice facilitation within primary care settings.
      ,
      • Alagoz E
      • Chih MY
      • Hitchcock M
      • Brown R
      • Quanbeck A
      The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review.
      ,
      • Wang A
      • Pollack T
      • Kadziel LA
      • et al.
      Impact of practice facilitation in primary care on chronic disease care processes and outcomes: a systematic review.
      However, these studies were largely focused on a single disease. It is less clear whether PF is effective in assisting primary care practices to simultaneously adopt multiple guidelines (e.g., for secondary prevention of multiple CVD risk factors).
      • Liddy C
      • Hogg W
      • Singh J
      • et al.
      A real-world stepped wedge cluster randomized trial of practice facilitation to improve cardiovascular care.
      Further, PF remains largely untested in SIPs. Additional research is needed to clarify the impact of PF on CVD risk management and related clinical outcomes.
      HealthyHearts New York City (HHNYC) was 1 of 7 cooperatives nationwide funded through the Agency for Health Care Research and Quality's EvidenceNOW initiative. The mixed methods study was designed to evaluate the impact of PF compared with usual practice on implementation of the ABCS treatment guidelines in small-to-medium-sized primary care practices.

      HHS. EvidenceNOW: Advancing Heart Health in Primary Care. www.ahrq.gov/evidencenow/index.html. Accessed August 27, 2018.

      This paper presents the primary quantitative outcomes of the HHNYC study.

      METHODS

      The HHNYC study used a stepped-wedge cluster RCT design in which sites were randomized into 1 of 4 waves that determined when they would start receiving the 1-year PF intervention (Appendix Figure 1, available online, shows a diagram of stepped-wedge design).
      • Shelley D
      • Blechter B
      • Siman N
      • et al.
      Quality of cardiovascular disease care in small urban practices.
      ,
      • Shelley DR
      • Ogedegbe G
      • Anane S
      • et al.
      Testing the use of practice facilitation in a cluster randomized stepped-wedge design trial to improve adherence to cardiovascular disease prevention guidelines: HealthyHearts NYC.
      The intervention was informed by the Chronic Care Model and the patient-centered medical home and included 13 onsite visits over 12 months, in which practice facilitators supported practices to (1) optimize use of their electronic health records (EHRs) to monitor and drive change (e.g., data validation, using data for quality reporting, previsit planning, and goal setting), (2) provide intervention updates and patient materials related to preventing and managing CVD risk factors, and (3) redesign workflows to facilitate integration of evidence-based guidelines into routine care.
      • Wagner EH
      • Austin BT
      • Davis C
      • Hindmarsh M
      • Schaefer J
      • Bonomi A
      Improving chronic illness care: translating evidence into action.
      ,

      National Committee for Quality Assurance. Patient-centered medical home (PCMH). www.ncqa.org/programs/health-care-providers-practices/patient-centered-medical-home-pcmh/. Accessed April 23, 2019.

      Facilitator activities were guided by the ABCS Toolkit for the Practice Facilitator, created by the New York City Department of Health and Mental Hygiene, which included checklists and action plans. The toolkit is available at www.ahrq.gov/evidencenow/heart-health/overall/dashboard.html. Most practices (94%) received at least 13 visits.

      Shelley D, Cuthel A, Corwin M, Siman N, Cleland C, Berry C. Measuring fidelity in HealthyHearts NYC: a complex intervention using practice facilitation in primary care. 11th Annual Conference on the Science of Dissemination and Implementation in Health. Washington, DC; 2018.

      There were no expected harms in this prevention-focused intervention. The study received IRB approval from the New York University School of Medicine.
      The SIPs were members of the New York City (NYC) Department of Health and Mental Hygiene's Primary Care Information Project (PCIP) practice network. PCIP is a bureau in the Division of Prevention and Primary Care and serves as NYC's Regional Extension Center.

      Study Sample

      Study sites were small practices (10 or fewer full-time equivalent clinicians) in NYC. Sites were required to have been using 1 of 2 EHR systems—eClinicalWorks or MDLand—for ≥1 year and to have no plans to change EHRs in the next 18 months. Sites could not have any immediate plans to participate in any CVD-related quality improvement projects. Eligibility criteria are described in detail in a previous publication (i.e., study protocol).
      • Shelley DR
      • Ogedegbe G
      • Anane S
      • et al.
      Testing the use of practice facilitation in a cluster randomized stepped-wedge design trial to improve adherence to cardiovascular disease prevention guidelines: HealthyHearts NYC.
      PCIP recruited sites in 2015, and of those eligible, 291 were randomized by NYU into an intervention wave; 34 withdrew during the study (Figure 1 and Appendix Table 1, available online). The remaining 257 were included in this analysis.
      Figure 1
      Figure 1CONSORT diagram. EHR, electronic health record; HHNYC, HealthyHearts New York City; PCIP, Primary Care Information Project.

      Measures

      The ABCS data were collected in the control, intervention, and follow-up periods. ABCS measures were based on Million Hearts definitions (Appendix Table 2, available online) and are consistent with national quality measures (e.g., National Quality Forum, Uniform Data System, and Centers for Medicare and Medicaid Services Quality Payment Program).

      HHS. Clinical quality measure alignment. https://millionhearts.hhs.gov/data-reports/cqm/measures.html. Accessed April 2, 2019.

      The individual measures were defined as the number of at-risk patients who reached clinical goals for each of the 4 ABCS guidelines: (A) aspirin use when indicated, (B) BP control, (C) cholesterol management, and (S), also referred to as smoking composite, screened for tobacco use and received counseling or cessation intervention if identified as a tobacco user. An additional tobacco measure was created that removed the screening component of the composite to include only the proportion of tobacco users who received a cessation counseling or intervention, referred to as smokers counseled. Finally, a composite measure was designed that assessed the number of patients with a history of ischemic vascular disease who met treatment targets for aspirin, BP, and cholesterol management (ABC composite).
      For practices using the eClinicalWorks EHR system, the ABCS data were extracted through electronic queries using PCIP's ad hoc distributed query network, the Hub Population Health System.
      • Buck MD
      • Anane S
      • Taverna J
      • Amirfar S
      • Stubbs-Dame R
      • Singer J
      The Hub Population Health System: distributed ad hoc queries and alerts.
      For practices using the MDLand EHR system, electronic queries were executed directly against the practices’ EHRs. Data were extracted for the 13 quarters between January 1, 2015 and March 31, 2018, which encompassed the control, intervention, and follow-up periods for all waves (Appendix Figure 1, available online), and analyzed in 2019.

      Statistical Analysis

      Descriptive statistics were used to summarize the median proportion of patients receiving guideline-recommended care by study period (control, intervention, and follow-up) and each outcome (A, B, C, S, smokers counseled, and ABC composite). To estimate the impact of the PF intervention on outcomes, negative binomial mixed-effects models were used, with repeated measures nested within 257 small primary care practice sites and randomly varying intercepts across SIPs.
      For each outcome, the count of patients receiving guideline recommended care was regressed on the following: (1) a general time trend over the entire study period, (2) an indicator of which period the site was in (control, intervention, or follow-up), (3) a time trend during the intervention period, (4) a time trend during the follow-up period, and (5) an offset using the natural logarithm of the number of patients eligible for care (i.e., the denominator). For the BP outcome, to take BP seasonality into account,
      • Amoah AO
      • Angell SY
      • Byrnes-Enoch H
      • Amirfar S
      • Maa P
      • Wang JJ
      Bridging the gap between clinical practice and public health: using EHR data to assess trends in the seasonality of blood-pressure control.
      • Brennan PJ
      • Greenberg G
      • Miall WE
      • Thompson SG
      Seasonal variation in arterial blood pressure.
      • Modesti PA
      Season, temperature and blood pressure: a complex interaction.
      • Rosenthal T.
      Seasonal variations in blood pressure.
      additional terms were included to separately contrast Quarters 1 (January–March) and 3 (July–September) with Quarters 2 (April–June) and 4 (October–December).
      Fixed-effects coefficients from these models were combined to compare (1) the rate of guideline recommended care in Months 7–9 of intervention versus the rate expected if the control period trend had continued, (2) the rate of guideline-recommended care in Months 7–9 of follow-up versus the rate expected if the control period trend had continued, and (3) the rate of guideline-recommended care in Months 7–9 of follow-up versus the rate expected if the intervention period trend had continued. Months 7–9 of the intervention period was used as the comparison point as this is when it was expected to see intervention effects, allowing time between the start of the intervention and the expectation of impact on ABCS outcomes, as well as to avoid extrapolating the control trend any further than needed. Months 7–9 of the follow-up period served as the comparison point because this is when it was expected to observe any deterioration in outcomes because of removal of the active PF intervention. These effects were estimated as incidence rate ratios (IRRs). The lme4 package of R, version 3.6.1, was used for negative binomial mixed-effects analysis, and the R multcomp package was used to test the significance of the contrasts of interest and calculate CIs.
      • Bates D
      • Mächler M
      • Bolker B
      • Walker S
      Fitting linear mixed-effects models using lme4.
      ,
      • Hothorn T
      • Bretz F
      • Westfall P
      Simultaneous inference in general parametric models.
      In a sensitivity analysis, these models and comparisons were repeated with sites contributing 4 or fewer quarters of data excluded (n=232–254, varying by measure). All tests of statistical significance were two-tailed, and p<0.05 was considered significant.

      RESULTS

      Table 1 shows the characteristics of the 257 practices. About two thirds of practices had only 1 physician, and >90% were owned independently. On average, 26.1% of patients at these practices were white non-Hispanic, and almost half (47.5%) were covered by Medicaid.
      Table 1Characteristics of Small Primary Care Practices (n=257)
      Characteristicsn (%)
      Practice characteristics
       Number of clinicians
        Solo clinician148 (65.5)
        ≥2 clinicians78 (34.5)
       Practice ownership
        Independent228 (92.7)
        Other18 (7.3)
       ACO status
        Part of ACO96 (37.4)
        Other161 (62.7)
       MUA designation
      Data extracted from HRSA website. ACO, accountable care organization; CPCQ, Change Process Capability Questionnaire; HRSA, Health Resources and Services Administration; MUA, medically underserved area; PCMH, patient-centered medical home.
        Yes121 (47.3)
        No135 (52.7)
       PCMH recognition
        Yes118 (46.9)
        No138 (53.9)
       Full-time equivalent of supporting staff, mean (SD)6.5 (13.9)
       Patient panel size, mean (SD)2,138 (2,161)
      Patient characteristics
       Race/ethnicity, mean (SD)
        Hispanic patient28.1 (33.1)
        Black, non-Hispanic patient21.5 (28.8)
        White, non-Hispanic patient17.0 (26.1)
        Asian patient17.5 (31.7)
       Medicaid payer, mean (SD)47.5 (26.9)
      Organizational capacity
       Adaptive reserve, mean (SD)0.8 (0.2)
       CPCQ, mean (SD)0.8 (0.2)
      Note: Missing data account for categorical variables where responses do not sum to 257. Among the 257 primary care practices, 214 reported their full-time equivalent of supporting staff; 228 provided data about their patient panel size and the percentage of white patients; 177 reported the percentage of Medicaid payers among their patients; and 152 completed data on the adaptive reserve.
      a Data extracted from HRSA website.ACO, accountable care organization; CPCQ, Change Process Capability Questionnaire; HRSA, Health Resources and Services Administration; MUA, medically underserved area; PCMH, patient-centered medical home.
      Practices varied in their delivery of guideline recommended care over time (Appendix Figure 2, available online), with more variation seen in the smoking measures and ABC composite and less variation in aspirin, BP, and cholesterol. On average, practices started out very close to or higher than the 70% individual targets for aspirin, BP, cholesterol, and the smoking composite (i.e., in the control period). Fixed-effects estimates for the ABC composite were consistently <50% over time, substantially lower than individual components of the ABC composite. The fixed-effects estimates of smokers counseled increased across study periods but never surpassed 50%.
      Table 2 shows the results of the negative binomial mixed-effects regression of ABCS and composite outcomes on the PF intervention. In 2 of the 3 pairwise comparisons described in Table 2, performance in the smoking composite measure improved over time. First, the smoking composite was significantly higher in the third quarter of the follow-up period compared with the expected rate if the control period trend had continued (IRR=1.152, 95% CI=1.072, 1.238, p<0.001). Second, when comparing the third quarter of the follow-up period to the expected rate if the intervention period trend had continued, the smoking composite measure again rose (IRR=1.060, 95% CI=1.013, 1.109, p=0.007). However, given the absence of an intervention effect for this measure (i.e., comparing the third quarter of the intervention period to the expected rate if the control period had continued was not statistically significant), it is possible that these 2 associations may reflect a secular trend. Implementation of the HHNYC intervention was also associated with an increase in the proportion of smokers counseled when comparing the third quarter of the intervention period with the expected rate if the control period trend had continued (IRR=1.121, 95% CI=1.037, 1.211, p=0.002).
      Table 2Negative Binomial Mixed-Effects Regression of Practice Facilitation Intervention Effects on ABCS and Composite Outcomes
      VariableIRR (95% CI)p-value
      Intervention vs control
       Aspirin1.009 (0.981, 1.038)0.747
       Blood pressure control0.996 (0.977, 1.015)0.904
       Cholesterol management1.001 (0.981, 1.022)0.994
       Smoking composite1.041 (0.995, 1.089)0.097
       Smokers counseled1.121 (1.037, 1.211)0.002
       ABC composite1.023 (0.984, 1.064)0.369
      Follow-up vs control
       Aspirin1.010 (0.964, 1.059)0.883
       Blood pressure control0.981 (0.950, 1.013)0.344
       Cholesterol management1.015 (0.982, 1.049)0.570
       Smoking composite1.152 (1.072, 1.238)<0.001
       Smokers counseled1.076 (0.951, 1.217)0.347
       ABC composite1.009 (0.943, 1.079)0.967
      Follow-up vs intervention
       Aspirin0.981 (0.944, 1.020)0.484
       Blood pressure control0.995 (0.972, 1.019)0.910
       Cholesterol management1.002 (0.975, 1.029)0.994
       Smoking composite1.060 (1.013, 1.109)0.007
       Smokers counseled0.933 (0.866, 1.004)0.069
       ABC composite0.948 (0.902, 0.996)0.031
      Note: Boldface indicates statistical significance (p<0.05).
      ABCS, aspirin, blood pressure, cholesterol, smoking; IRR, incidence rate ratio.
      When comparing the third quarter of the follow-up period to the expected intervention trend, the ABC composite measure declined (IRR=0.948, 95% CI=0.902, 0.996, p=0.031). However, given the absence of an intervention effect for this measure, this association may reflect a secular trend, rather than the loss of an intervention effect. Intervention implementation did not change the rate of guideline recommended care for aspirin, BP, cholesterol, or for the other 2 comparisons of the ABC composite measure.
      In a sensitivity analysis, which excluded sites that contributed 4 or fewer quarters of data for a specific ABCS outcome, there were no substantial changes in the pattern of findings. There were no interaction effects between the intervention, wave, and practice characteristics.

      DISCUSSION

      In this study, PF was associated with improvements in the 2 smoking-related outcomes only, the smoking composite and smokers counseled. Numerous studies have demonstrated the effectiveness of PF; however, these studies have focused on a single guideline.
      • Baskerville NB
      • Liddy C
      • Hogg W
      Systematic review and meta-analysis of practice facilitation within primary care settings.
      ,
      • Alagoz E
      • Chih MY
      • Hitchcock M
      • Brown R
      • Quanbeck A
      The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review.
      Only 1 other study has evaluated PF as a strategy for implementing multiple guidelines for chronic disease prevention, the Improved Delivery of Cardiovascular Care study, which found that PF did not improve adherence to guidelines for CVD prevention in Canadian primary care practices.
      • Liddy C
      • Hogg W
      • Singh J
      • et al.
      A real-world stepped wedge cluster randomized trial of practice facilitation to improve cardiovascular care.
      Their study, and this one, highlight the complexity of simultaneously changing care processes to meet guideline-recommended care for a range of CVD risk factors. This study is, to the authors’ knowledge, the first to examine PF impact on multiple risk factors in SIPs.
      The intervention focused on promoting system changes (e.g., clinical alerts, registry reports, and templates) and workflow redesign that were consistent with current models for practice transformation.
      • Wagner EH
      • Austin BT
      • Davis C
      • Hindmarsh M
      • Schaefer J
      • Bonomi A
      Improving chronic illness care: translating evidence into action.
      ,

      National Committee for Quality Assurance. Patient-centered medical home (PCMH). www.ncqa.org/programs/health-care-providers-practices/patient-centered-medical-home-pcmh/. Accessed April 23, 2019.

      In theory, these practice changes facilitate improvements in guideline-concordant care. In practice, however, operationalizing the large number of changes required to concurrently improve guideline adoption across a broad range of chronic disease indicators is challenging. Complexity also arises from contextual factors that may hinder the facilitation process and practice improvement goals.
      • Tomoaia-Cotisel A
      • Scammon DL
      • Waitzman NJ
      • et al.
      Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
      In the qualitative analyses of PF interviews from HHNYC, it was found that practice facilitators described several contextual factors that created challenges, including competing demands, varying and low levels of readiness to change in terms of workforce capacity and expertise in health information technology, and inconsistent leadership engagement. Practice facilitators also reported that small practices frequently requested assistance with practice issues unrelated to the study. Responding to these requests reduced the time they could spend on the project goals but was a critically important strategy for building trust and demonstrating value to busy practices.
      • Nguyen AM
      • Cuthel A
      • Padgett DK
      • et al.
      How practice facilitation strategies differ by practice context.
      The Improved Delivery of Cardiovascular Care study likewise reported that organizational and structural aspects of primary care practices (e.g., lack of time, minimal staff engagement, and resistance to change) may have reduced the impact of the PF intervention.
      • Liddy C
      • Rowan M
      • Valiquette-Tessier SC
      • Drosinis P
      • Crowe L
      • Hogg W
      Improved delivery of cardiovascular care (IDOCC): findings from narrative reports by practice facilitators.
      A longer period of practice engagement often not feasible within the constraints of research trial protocols and intensity is likely needed to be able to attain improvements in guideline adherence across multiple risk factors. This would allow facilitators the time to build relationships, build capacity, and develop, in partnership with site staff and leadership, tailored approaches that fit local context and practice preferences.
      • Tomoaia-Cotisel A
      • Scammon DL
      • Waitzman NJ
      • et al.
      Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
      • Nguyen AM
      • Cuthel A
      • Padgett DK
      • et al.
      How practice facilitation strategies differ by practice context.
      • Cohen DJ
      • Balasubramanian BA
      • Davis M
      • et al.
      Understanding care integration from the ground up: five organizing constructs that shape integrated practices.
      Suboptimal intensity of the intervention (i.e., not attaining the proposed intervention dose) has been cited as a reason for not being able to demonstrate an impact for PF interventions that address multiple risk factors or chronic conditions.
      • Liddy C
      • Hogg W
      • Singh J
      • et al.
      A real-world stepped wedge cluster randomized trial of practice facilitation to improve cardiovascular care.
      Practice facilitators in HHNYC achieved high rates of fidelity to the site visit schedule, however, and there were no clear differences between practices that met the site visit schedule and the few that did not.

      Shelley D, Cuthel A, Corwin M, Siman N, Cleland C, Berry C. Measuring fidelity in HealthyHearts NYC: a complex intervention using practice facilitation in primary care. 11th Annual Conference on the Science of Dissemination and Implementation in Health. Washington, DC; 2018.

      Dose alone does not elucidate the quality of the visit or the adaptive process that is an inherent characteristic of this implementation strategy. Moreover, when viewed through the lens of quantitative data alone, the full value proposition of PF may be underestimated. The HHNYC qualitative study of enrolled clinicians found that clinicians viewed facilitation as an important resource particularly in terms of optimizing their use of the EHR for quality improvement and creating awareness of quality gaps; furthermore, clinicians in SIPs of different sizes (i.e., number of staff members) perceived different benefits from facilitators.
      • Rogers ES
      • Cuthel AM
      • Berry CA
      • Kaplan SA
      • Shelley DR
      Clinician perspectives on the benefits of practice facilitation for small primary care practices.
      Mixed methods studies are needed to obtain data that can further define the PF process for implementing multiple guidelines, from both the facilitator and practice perspective, to inform strategies for optimizing the impact of PF for chronic disease prevention and management in primary care.
      • Cohen DJ
      • Balasubramanian BA
      • Gordon L
      • et al.
      A national evaluation of a dissemination and implementation initiative to enhance primary care practice capacity and improve cardiovascular disease care: the ESCALATES study protocol.
      With the noted challenges, why did the smoking-related outcomes improve? First, the finding that smoking counseling improved demonstrates the importance of measuring this outcome in addition to the Million Hearts recommended composite measure. Rates of tobacco use screening are high in NY and nationally, largely because of Meaningful Use Stage 1 that included this measure, but cessation counseling remains low.
      • Bailey SR
      • Heintzman JD
      • Marino M
      • et al.
      Smoking cessation assistance: before and after Stage 1 meaningful use implementation.
      ,
      • Huo J
      • Yang M
      • Tina Shih YC
      Sensitivity of claims-based algorithms to ascertain smoking status more than doubled with meaningful use.
      Combining screening and counseling therefore masks poor performance on the treatment measure. Second, less than half of sites were adhering to the smoking-cessation counseling guideline during the baseline period, which meant there was room for improvement. This was in contrast to the other measures for which, on average, practices were at the Million Hearts target of 70% at baseline. Third, the policy environment provided a financial incentive to improve tobacco use treatment. In 2011, New York implemented a Medicaid policy that reimbursed eligible providers (e.g., physicians and nurse practitioners) for up to 6 sessions of smoking-cessation counseling. Facilitators raised awareness about this policy and assisted practices to meet the clinical guidelines and state requirements for reimbursement through a simple system change (i.e., clinical decision support smartform). This system both prompted and facilitated prescribing, counseling, and electronic referrals to the state quitline, as well as documentation, and triggered the billing code for behavioral counseling. Thus, the improvement in cessation counseling demonstrated the importance of choosing clinically meaningful measures to capture change and how PF-supported system changes and policy alignment can support the practice improvement process.
      As noted, the inability to demonstrate improvement in the other outcome measures may be in part because of a ceiling effect. There is evidence that it is difficult to achieve improvement in BP, for example, beyond a certain BP threshold, even with extensive use of concomitant therapies.
      • Zanchetti A
      Bottom blood pressure or bottom cardiovascular risk? How far can cardiovascular risk be reduced?.
      The SIPs in this study, on average, were meeting or very close to meeting Million Hearts individual clinical goals for ABCS and smoking composite before initiating the PF intervention. PCIP's mission is to support SIPs to improve quality of care in NYC through PF and other services with a particular focus on CVD prevention. This may have contributed to SIPs’ strong performance on these quality measures and suggests that ongoing and sustainable support offered through a PF program can lead to sustained practice improvement. It is also possible that higher-performing practices were more likely to participate in and complete a complex intervention.
      In addition, documentation may have played a role in why the aspirin measure did not change because of the intervention. As part of the intervention, facilitators did remind clinicians to document aspirin in the EHR, and there was a clinical alert. However, aspirin is an over-the-counter drug, which clinicians in the HHNYC study reported in the qualitative data as being something they did not normally document.
      • Rogers ES
      • Cuthel AM
      • Berry CA
      • Kaplan SA
      • Shelley DR
      Clinician perspectives on the benefits of practice facilitation for small primary care practices.
      For statins, the American College of Cardiology/American Heart Association guideline on the primary prevention of CVD was relatively new (2013), and despite training on the new algorithm for prescribing, it may have been too soon, and the intervention too short, to observe guideline adoption.
      In contrast to the individual ABC measures, adherence to the ABC composite never exceeded 50%, and there was no intervention effect on the ABC composite measure. The PF approach in this study may have contributed to the findings. Facilitators took a sequential, rather than comprehensive, approach to addressing each of the risk factors and related guideline implementation. In part, this was related to limitations in the reporting function of the EHRs, which made it difficult to generate a performance report that would present multiple quality measures for a given patient. However, the types of system-level redesign (e.g., team-based care) needed to accommodate multiple chronic diseases are challenging in SIPs.
      • Schuttner L
      • Parchman M.
      Team-based primary care for the multimorbid patient: matching complexity with complexity.
      Studies of PF intervention that address this specific patient population are needed to fill this gap in care and outcomes, particularly in SIPs. Alternatively, it is possible no intervention effect was detected owing to the extrapolation of the control period; there may have been a modest improvement in the ABC composite during the intervention period but one not large enough to be significantly different from the control trend, which was not sustained into the follow-up period.

      Limitations

      The main limitations included that the outcome data were not available from the 34 practices that withdrew after randomization. Other practices were missing outcome data for some, but not all, quarters. However, sensitivity analyses excluding those with missing data did not show any differences, so it is doubtful that a full data set would have shown anything different. In addition, EHR outcome data were dependent on consistent documentation at the practice level. However, PCIP implemented a quality assurance process, and practice facilitators focused early in the intervention on optimizing documentation. Next, the study did not have individual-level data for BP measures, which precluded the study from capturing improvements in BP among those who started with high BP levels and improved but did not reach the 140/90 mmHg threshold. Finally, comparing rates of guideline-recommended care at follow-up to rates expected if the control period trend had continued required extrapolation of control trends across multiple quarters. The uncertainty attached to these comparisons were captured in part by increases in CI width, but the analyses relied on the strong, untestable assumption that the control period trends would have continued for more than a year.

      CONCLUSIONS

      Small practices, having fewer staff and less dedicated onsite expertise to implement complex practice changes, may particularly benefit from PF. Increasing the impact of PF programs and interventions that target multiple risk factors may require greater attention to external policy and practice context and a longer timeline and intensity to meet multiple objectives. The findings related to the smoking measures support ongoing review of clinical quality measures to ensure that they accurately reflect changes in practice.

      ACKNOWLEDGMENTS

      This project was supported by grant number 1R18HS023922-01 from the Agency for Health care Research and Quality. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The authors thank Sarah Shih, the Assistant Commissioner of the Primary Care Information Project at the New York City Department of Health and Mental Hygiene, for her insightful comments on the manuscript draft.
      No financial disclosures were reported by the authors of this paper.

      Appendix. SUPPLEMENTAL MATERIAL

      REFERENCES

        • Van Dyke M
        • Greer S
        • Odom E
        • et al.
        Heart disease death rates among blacks and whites aged ≥35 years—United States, 1968–2015.
        MMWR Surveill Summ. 2018; 67: 1-11
        • CDC
        Million hearts: strategies to reduce the prevalence of leading cardiovascular disease risk factors–United States, 2011.
        MMWR Morb Mortal Wkly Rep. 2011; 60: 1248-1251
      1. Million Hearts HHS. Framework. https://millionhearts.hhs.gov/files/MH-Framework.pdf. Accessed January 10, 2018, 2022.

        • Landon BE
        • Normand SL.
        Performance measurement in the small office practice: challenges and potential solutions.
        Ann Intern Med. 2008; 148: 353-357
        • Wolfson D
        • Bernabeo E
        • Leas B
        • Sofaer S
        • Pawlson G
        • Pillittere D
        Quality improvement in small office settings: an examination of successful practices.
        BMC Fam Pract. 2009; 10: 14
        • Nutting PA
        • Crabtree BF
        • McDaniel RR
        Small primary care practices face four hurdles–including a physician-centric mind-set–in becoming medical homes.
        Health Aff (Millwood). 2012; 31: 2417-2422
        • Rittenhouse DR
        • Casalino LP
        • Shortell SM
        • et al.
        Small and medium-size physician practices use few patient-centered medical home processes.
        Health Aff (Millwood). 2011; 30: 1575-1584
        • Laferriere D
        • Liddy C
        • Nash K
        • Hogg W
        Navigating change: how outreach facilitators can help clinicians improve patient outcomes.
        J Am Board Fam Med. 2012; 25: 232-237
        • Baskerville NB
        • Liddy C
        • Hogg W
        Systematic review and meta-analysis of practice facilitation within primary care settings.
        Ann Fam Med. 2012; 10: 63-74
        • Hemler JR
        • Hall JD
        • Cholan RA
        • et al.
        Practice facilitator strategies for addressing electronic health record data challenges for quality improvement: EvidenceNOW.
        J Am Board Fam Med. 2018; 31: 398-409
        • Harvey G
        • Lynch E.
        Enabling continuous quality improvement in practice: the role and contribution of facilitation.
        Front Public Health. 2017; 5: 27
        • Stetler CB
        • Legro MW
        • Rycroft-Malone J
        • et al.
        Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration.
        Implement Sci. 2006; 1: 23
        • Berta W
        • Cranley L
        • Dearing JW
        • Dogherty EJ
        • Squires JE
        • Estabrooks CA
        Why (we think) facilitation works: insights from organizational learning theory.
        Implement Sci. 2015; 10: 141
        • Kitson AL
        • Rycroft-Malone J
        • Harvey G
        • McCormack B
        • Seers K
        • Titchen A
        Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges.
        Implement Sci. 2008; 3: 1
        • Liddy C
        • Rowan M
        • Valiquette-Tessier SC
        • Drosinis P
        • Crowe L
        • Hogg W
        Improved delivery of cardiovascular care (IDOCC): findings from narrative reports by practice facilitators.
        Prev Med Rep. 2017; 5: 214-219
        • Dogherty EJ
        • Harrison MB
        • Graham ID
        Facilitation as a role and process in achieving evidence-based practice in nursing: a focused review of concept and meaning.
        Worldviews Evid Based Nurs. 2010; 7: 76-89
        • Rycroft-Malone J
        • Seers K
        • Chandler J
        • et al.
        The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework.
        Implement Sci. 2013; 8: 28
        • Lessard S
        • Bareil C
        • Lalonde L
        • et al.
        External facilitators and interprofessional facilitation teams: a qualitative study of their roles in supporting practice change.
        Implement Sci. 2016; 11: 97
        • Harvey G
        • Loftus-Hills A
        • Rycroft-Malone J
        • et al.
        Getting evidence into practice: the role and function of facilitation.
        J Adv Nurs. 2002; 37: 577-588
        • Alagoz E
        • Chih MY
        • Hitchcock M
        • Brown R
        • Quanbeck A
        The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review.
        BMC Health Serv Res. 2018; 18: 42
        • Wang A
        • Pollack T
        • Kadziel LA
        • et al.
        Impact of practice facilitation in primary care on chronic disease care processes and outcomes: a systematic review.
        J Gen Intern Med. 2018; 33: 1968-1977
        • Liddy C
        • Hogg W
        • Singh J
        • et al.
        A real-world stepped wedge cluster randomized trial of practice facilitation to improve cardiovascular care.
        Implement Sci. 2015; 10: 150
      2. HHS. EvidenceNOW: Advancing Heart Health in Primary Care. www.ahrq.gov/evidencenow/index.html. Accessed August 27, 2018.

        • Shelley D
        • Blechter B
        • Siman N
        • et al.
        Quality of cardiovascular disease care in small urban practices.
        Ann Fam Med. 2018; 16 (suppl 1): S21-S28
        • Shelley DR
        • Ogedegbe G
        • Anane S
        • et al.
        Testing the use of practice facilitation in a cluster randomized stepped-wedge design trial to improve adherence to cardiovascular disease prevention guidelines: HealthyHearts NYC.
        Implement Sci. 2016; 11: 88
        • Wagner EH
        • Austin BT
        • Davis C
        • Hindmarsh M
        • Schaefer J
        • Bonomi A
        Improving chronic illness care: translating evidence into action.
        Health Aff (Millwood). 2001; 20: 64-78
      3. National Committee for Quality Assurance. Patient-centered medical home (PCMH). www.ncqa.org/programs/health-care-providers-practices/patient-centered-medical-home-pcmh/. Accessed April 23, 2019.

      4. Shelley D, Cuthel A, Corwin M, Siman N, Cleland C, Berry C. Measuring fidelity in HealthyHearts NYC: a complex intervention using practice facilitation in primary care. 11th Annual Conference on the Science of Dissemination and Implementation in Health. Washington, DC; 2018.

      5. HHS. Clinical quality measure alignment. https://millionhearts.hhs.gov/data-reports/cqm/measures.html. Accessed April 2, 2019.

        • Buck MD
        • Anane S
        • Taverna J
        • Amirfar S
        • Stubbs-Dame R
        • Singer J
        The Hub Population Health System: distributed ad hoc queries and alerts.
        J Am Med Inform Assoc. 2012; 19: e46-e50
        • Amoah AO
        • Angell SY
        • Byrnes-Enoch H
        • Amirfar S
        • Maa P
        • Wang JJ
        Bridging the gap between clinical practice and public health: using EHR data to assess trends in the seasonality of blood-pressure control.
        Prev Med Rep. 2017; 6: 369-375
        • Brennan PJ
        • Greenberg G
        • Miall WE
        • Thompson SG
        Seasonal variation in arterial blood pressure.
        BMJ. 1982; 285: 919-923
        • Modesti PA
        Season, temperature and blood pressure: a complex interaction.
        Eur J Intern Med. 2013; 24: 604-607
        • Rosenthal T.
        Seasonal variations in blood pressure.
        Am J Geriatr Cardiol. 2004; 13: 267-272
        • Bates D
        • Mächler M
        • Bolker B
        • Walker S
        Fitting linear mixed-effects models using lme4.
        J Stat Soft. 2015; 67: 1-48
        • Hothorn T
        • Bretz F
        • Westfall P
        Simultaneous inference in general parametric models.
        Biom J. 2008; 50: 346-363
        • Tomoaia-Cotisel A
        • Scammon DL
        • Waitzman NJ
        • et al.
        Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
        Ann Fam Med. 2013; 11: S115-S123
        • Nguyen AM
        • Cuthel A
        • Padgett DK
        • et al.
        How practice facilitation strategies differ by practice context.
        J Gen Intern Med. 2019;
        • Cohen DJ
        • Balasubramanian BA
        • Davis M
        • et al.
        Understanding care integration from the ground up: five organizing constructs that shape integrated practices.
        J Am Board Fam Med. 2015; 28: S7-S20
        • Rogers ES
        • Cuthel AM
        • Berry CA
        • Kaplan SA
        • Shelley DR
        Clinician perspectives on the benefits of practice facilitation for small primary care practices.
        Ann Fam Med. 2019; 17: S17-S23
        • Cohen DJ
        • Balasubramanian BA
        • Gordon L
        • et al.
        A national evaluation of a dissemination and implementation initiative to enhance primary care practice capacity and improve cardiovascular disease care: the ESCALATES study protocol.
        Implement Sci. 2016; 11: 86
        • Bailey SR
        • Heintzman JD
        • Marino M
        • et al.
        Smoking cessation assistance: before and after Stage 1 meaningful use implementation.
        Am J Prev Med. 2017; 53: 192-200
        • Huo J
        • Yang M
        • Tina Shih YC
        Sensitivity of claims-based algorithms to ascertain smoking status more than doubled with meaningful use.
        Value Health. 2018; 21: 334-340
        • Zanchetti A
        Bottom blood pressure or bottom cardiovascular risk? How far can cardiovascular risk be reduced?.
        J Hypertens. 2009; 27: 1509-1520
        • Schuttner L
        • Parchman M.
        Team-based primary care for the multimorbid patient: matching complexity with complexity.
        Am J Med. 2019; 132: 404-406