If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to: Timothy B. Baker, PhD, Center for Tobacco Research and Intervention, University of Wisconsin School of Medicine and Public Health, 1930 Monroe Street, Madison WI 53711.
Center for Tobacco Research and Intervention, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinDepartment of Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
Center for Tobacco Research and Intervention, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinDepartment of Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinUniversity of Wisconsin Hospitals and Clinics, Madison, Wisconsin
University of Wisconsin Hospitals and Clinics, Madison, WisconsinDepartment of Family Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
Center for Tobacco Research and Intervention, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinDepartment of Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinUniversity of Wisconsin Hospitals and Clinics, Madison, Wisconsin
Center for Tobacco Research and Intervention, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinDepartment of Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WisconsinUniversity of Wisconsin Hospitals and Clinics, Madison, Wisconsin
Patients who use tobacco are too rarely connected with tobacco use treatment during healthcare visits. Electronic health record enhancements may increase such referrals in primary care settings. This project used the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework to assess the implementation of a healthcare system change carried out in an externally valid manner (executed by the healthcare system).
Methods
The healthcare system used their standard, computer-based training approach to implement the electronic health record and clinic workflow changes for electronic referral in 30 primary care clinics that previously used faxed quitline referral. Electronic health record data captured rates of assessment of readiness to quit and quitline referral 4 months before implementation and 8 months (May–December 2017) after implementation. Data, analyzed from October 2018 to June 2019, also reflected intervention reach, adoption, and maintenance.
Results
For reach and effectiveness, from before to after implementation for electronic referral, among adult patients who smoked, assessment of readiness to quit increased from 24.8% (2,126 of 8,569) to 93.2% (11,163 of 11,977), quitline referrals increased from 1.7% (143 of 8,569) to 11.3% (1,351 of 11,977), and 3.6% were connected with the quitline after implementation. For representativeness of reach, electronic referral rates were especially high for women, African Americans, and Medicaid patients. For adoption, 52.6% of staff who roomed at least 1 patient who smoked referred to the quitline. For maintenance, electronic referral rates fell by approximately 60% over 8 months but remained higher than pre-implementation rates.
Conclusions
Real-world implementation of an electronic health record–based electronic referral system markedly increased readiness to quit assessment and quitline referral rates in primary care patients. Future research should focus on implementation methods that produce more consistent implementation and better maintenance of electronic referral.
Fiore MC, Jaén CR, Baker TB, et al. Treating tobacco use and dependence: 2008 update. Rockville, MD: HHS, U.S. Public Health Service.https://www.ncbi.nlm.nih.gov/books/NBK63952/. Published May 2008. Accessed September 11, 2020.
However, it has proven difficult to meaningfully increase the rates at which primary care patients are offered and provided evidence-based smoking cessation treatment.
Centers for Disease Control and Prevention (CDC) Tobacco use screening and counseling during physician office visits among adults–National Ambulatory Medical Care Survey and National Health Interview Survey, United States, 2005-2009.
; perhaps, the most promising and cost effective are the ones that use electronic health records (EHRs) to help identify smokers, prompt cessation treatment offers, and facilitate treatment referral.
Using the electronic health record to connect primary care patients to evidence-based telephonic tobacco quitline services: a closed-loop demonstration project.
evaluated EHR enhancements designed to prompt smoker identification, assessment of interest in cessation treatment, and assistance with cessation treatment referral. Referral was accomplished with a secure, HIPAA-compliant, EHR-based electronic referral (eReferral) order to the Wisconsin Tobacco Quitline (WTQL). On receiving the referral, the WTQL attempted to contact the patient and ultimately provided secure, closed-loop feedback on the referral outcome, for example, whether the patient was contacted, accepted, or declined WTQL services and the service provided (e.g., counseling and pharmacotherapy). This feedback was automatically transmitted into the patient's EHR.
Using the electronic health record to connect primary care patients to evidence-based telephonic tobacco quitline services: a closed-loop demonstration project.
were implemented in 2 primary care clinics that had been using a paper fax-to-quit method of cessation treatment referral. Before the EHR enhancements, about 0.3% of adult tobacco users were referred to the WTQL through fax; this rate climbed to 14% after the eReferral enhancements. Other observational studies suggest that eReferral can boost quitline referral rates in comparison with pre-existing fax referral or other methods.
A total of 23 primary care clinics across 2 healthcare systems were randomized to either eReferral resources or continued use of paper fax-to-quit referral; eReferral increased quitline referral (System A=17.9% vs 3.8%; System B=18.9% vs 5.2%) and connection rates (System A=5.4% vs 1.3%; System B=5.3% vs 2%) for eReferral and fax-to-quit, respectively.
An electronic health record-based interoperable eReferral system to enhance smoking quitline treatment in primary care [published correction appears in J Am Med Inform Assoc. 2019;26(10):1159].
Intervention implementation in real-world settings is often extremely difficult, and poor implementation meaningfully reduces intervention effectiveness.
even with EHR enhancements, only 54% of patients had their smoking status documented. In this same study, the majority of the clinicians in the participating clinics (56%) never used a smoking cessation treatment order set for over a 9-month period.
Using the electronic health record to connect primary care patients to evidence-based telephonic tobacco quitline services: a closed-loop demonstration project.
Recruiting and engaging smokers in treatment in a primary care setting: developing a chronic care model implemented through a modified electronic health record.
An electronic health record-based interoperable eReferral system to enhance smoking quitline treatment in primary care [published correction appears in J Am Med Inform Assoc. 2019;26(10):1159].
raise questions concerning how well EHR-based smoking treatment systems will work if implemented with less intensive methods, that is, those likely to be used in real-world applications. The healthcare system, not the research team, managed implementation in this study, enhancing the real-world relevance of this implementation analysis. This pragmatic application of the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework
focused on the RE-AIM constructs most relevant to health systems.
Implementation success and effects were assessed across 30 primary care clinics in a large health system (University of Wisconsin [UW] Health) in central Wisconsin to address unmet needs in tobacco use treatment within UW Health; for example, the healthcare system ranked poorly among state healthcare systems on an assessment of willingness to quit. This investigation sought to examine the implementation success of a healthcare system–implemented, EHR-based, interoperable, and closed-loop eReferral mechanism for smoking treatment in primary care.
METHODS
Study Sample
Located in the Upper Midwest, UW Health is an integrated healthcare delivery system that treats >600,000 patients each year, with approximately 1,500 physicians and 16,500 other staff at 6 hospitals and more than 80 outpatient clinic sites. UW Health leadership, UW Center for Tobacco Research and Intervention research and outreach teams, and WTQL collaboratively designed this intervention. Over the course of 10 months (July 2016–May 2017), the intervention was conceptualized, designed, pilot tested in 1 large clinic, and launched in all UW Health primary and urgent care clinics. The UW School of Medicine and Public Health IRB deemed this project to be a program evaluation activity.
Before eReferral implementation, the UW Health policy was to determine and document every patient's smoking status within the EHR, typically during the rooming process (roomers, typically medical assistants or licensed practical nurses). Primary care clinicians could then initiate a conversation with the patient about willingness to quit and provide a cessation intervention or fax a referral for the patient to the WTQL; neither action was completed and documented on a regular basis. This protocol led to high rates of screening for tobacco use (approximately 99%) but relatively low rates of documented assessment of interest in quitting or referrals to the WTQL.
The new EHR-based eReferral system was designed to increase both assessment of willingness to quit and linkage with the WTQL for treatment. In the new workflow, disseminated throughout outpatient primary and urgent care clinics (Figure 1), roomers were to ask patients about their current tobacco use status and update in the EHR as needed, assess and document readiness to quit within 30 days in all patients who used tobacco, and offer WTQL eReferral to individuals ready to quit smoking, which accords with the Public Health Service Guideline‒recommended 5 As.
Fiore MC, Jaén CR, Baker TB, et al. Treating tobacco use and dependence: 2008 update. Rockville, MD: HHS, U.S. Public Health Service.https://www.ncbi.nlm.nih.gov/books/NBK63952/. Published May 2008. Accessed September 11, 2020.
Note: This workflow documents decision points and possible outcomes of eReferral workflow, beginning with the identification of a patient as currently smoking during a primary care encounter.
EHR, electronic health record; eReferral, electronic referral; HL7v2, High Level Seven International Messaging Version 2; NRT, nicotine replacement therapy; ORM, object relational mapping; ORU, object results mapping; WTQL, Wisconsin Tobacco Quitline.
If roomers attempted to leave the vital sign assessment module in the EHR of patients who used tobacco without documenting the patient's readiness to quit, a pop-up reminder prompted them: The most important thing you can do to improve your health is to quit tobacco, and we can help. Are you interested in trying to cut back or quit in the next 30 days? For adult patients who were ready to quit and had a Wisconsin address on file, an EHR alert and order set prompted the eReferral offer. The alert provided suggested language: Could we have the Wisconsin Quit Line call you to discuss free coaching and nicotine replacement? and 3 talking points to encourage enrollment in quitline services: (1) the increased likelihood of success with quitline support, (2) quit coaching is free, and (3) free nicotine-replacement medicines. This alert was suppressed for 90 days after eReferral to prevent duplicate referrals.
For patients who consented to WTQL referral, rooming staff entered the eReferral order within the EHR for clinician review. The order prompted clinicians to address smoking cessation with the patient, sign the order for quitline referral, and discuss other smoking cessation treatments, including cessation medications as appropriate. WTQL eReferral orders electronically signed by clinicians were then securely transmitted to the WTQL, where the staff made up to 5 attempts to reach patients by phone at the preferred number listed in the EHR, starting within 3 days of eReferral. Patients who accepted a WTQL call and set a quit day within 30 days were provided quit-smoking coaching and, if medically eligible according to a WTQL standing protocol, a 2-week starter kit of either nicotine patch, gum, or lozenge as selected by the patient.
The WTQL electronically returned results of the referral to the EHR within 2 weeks, both to the patient's chart and to the referring clinician's inbox. The EHR eReferral tools were also available during telephone, allied health, and orders-only encounters, but clinic staff had to navigate to the order set to access these tools during such encounters.
Staff received intervention training through online UW Health computer-based training (CBT), the standard practice for all new clinical practice initiatives in this system. Separate CBT resources were developed for clinicians and rooming staff. CBT components included an online video, slides, and screenshots detailing step-by-step instructions regarding the 2 new EHR alerts; detailed workflows; and information about WTQL services. The 12-minute video and the 11-slide set presented information on (1) intervention goals, (2) targeted patients and involved staff, (3) specific workflow changes in consecutive steps, (4) screenshots of EHR pages illustrating specific steps and locations, (5) how to send the referral to the quitline, (6) what to convey to the patient, and (7) how to get further information (e.g., frequently asked questions) (Appendix Text, available online). No in-person training occurred. One month before the eReferral launch, clinic managers received training materials and distributed them to their rooming staff and clinicians. Clinic managers were responsible for documenting CBT completion before eReferral went live. The CBT was included in new employee training after the eReferral launch. After implementation, clinic managers received monthly feedback regarding the rates of each roomer's assessment of willingness to quit and WTQL eReferral, which they were to share with rooming staff and clinicians.
Measures
In RE-AIM, reach and effectiveness were assessed at the individual level. Reach was assessed by examining the percentages of patients who smoked who were (1) asked whether they were willing to make an aided quit attempt and (2) referred. Representativeness of reach was indexed by the reach of these intervention outcomes, including referral rates, across key population dimensions (e.g., race, sex, insurance status). Effectiveness was assessed by the percentage of patients who smoked and were connected with the WTQL (i.e., accepted WTQL treatment).
Adoption and maintenance were assessed at the setting and staff levels. Adoption was assessed by overall percentages and variance across roomers and clinicians in (1) assessing interest in making an aided quit attempt, (2) identifying patients who smoked and were willing to make a quit attempt in the next 30 days, and (3) referring to treatment (WTQL). Finally, maintenance was examined by trends in the execution of the 3 smoking treatment elements over the course of this study.
Statistical Analysis
The authors gathered deidentified EHR data from UW Health to evaluate the eReferral initiative. Only visits from adult patients (aged ≥18 years) who were listed as current tobacco users in the 4 months pre-implementation and 8 months postimplementation launch (May 9, 2017) were extracted. The following variables were captured for every adult tobacco user visit between January and December 2017: month of visit, patient age (truncated at 90 years), sex, race, ethnicity, insurance (coded as commercial/private, Medicare, Medicaid, or uninsured, which could overlap), clinic of encounter, tobacco products used, patient willingness to quit (yes, no, or not assessed), and WTQL referral. Administrative data on the size of clinic adult patient panels and the number of clinicians on staff at each clinic were also gathered.
Only data from adult patients who reported smoking were analyzed (versus exclusive users of other forms of nicotine such as smokeless or e-cigarettes). Data from pediatric, specialty, and urgent care clinics and from the pilot clinic were also excluded, leaving 30 clinics. The reach of quit readiness assessment and WTQL eReferral was analyzed at the patient level. The denominators reflect the total number of adult patients who smoked and were seen in a target clinic in the relevant period (i.e., rates of readiness to quit reflect all patients who smoked and made clinic visits, not just those assessed, and rates of eReferral reflect all patients who smoked and were seen, not just those ready to quit). Investigators conducted bivariate chi-square tests and t-tests to examine the representativeness of reach across demographics, insurance, tobacco use groups, and visit counts. Adoption and implementation were assessed by descriptive data examining the rates and variance in the assessment of quit readiness and eReferral at the visit level; overall; and by rooming staff, clinician, and clinic. The denominator for these rates was the total number of visits by adult patients who smoked in the target period.
RESULTS
The characteristics of the adult patients who smoked and were seen in the 30 target clinics in the 4 months before launch and 8 months after launch are shown in Table 1. Among those seen at the participating clinics during the relevant time periods (a patient-level of analysis), the rate of assessing readiness to quit increased from 24.8% before launch to 93.2% after launch, quit readiness rates increased from 9.5% to 31.9%, and the WTQL referral rate increased from 1.7% (143 of 8,569; this estimate is not shown in Table 1) to 11.3%. The pre-launch WTQL referral rate is likely an overestimate; it included fax referrals from all the UW Health system, not just the target 30 clinics. Of those electronically referred (eReferred) before implementation, 28.7% (3.6% of all patients who smoked and were seen at a clinic) accepted WTQL services. Clinic-specific results are shown in Appendix Table 1 (available online).
Table 1Pre- and Post-Launch Patient-Level Descriptions by Patient Characteristics Among Patients Who Smoked
Race subgroups with different superscript letters differ significantly in rate assessed, ready to quit, or eReferred; e.g., for post-implementation eReferral rates, Whites with a “g” superscript differ from African Americans with an “h” superscript, but not from other minority group, which also has a “g” superscript. M, mean.
Race subgroups with different superscript letters differ significantly in rate assessed, ready to quit, or eReferred; e.g., for post-implementation eReferral rates, Whites with a “g” superscript differ from African Americans with an “h” superscript, but not from other minority group, which also has a “g” superscript. M, mean.
93 (29.2)g
38 (11.9)g
Ethnicity, n (%)
Hispanic
275 (3.2)
60 (21.8)
30 (10.9)
364 (3.1)
326 (89.6)
112 (30.8)
34 (9.3)
Not Hispanic
8,222 (95.9)
2,046 (24.9)
775 (9.4)
11,500 (96.0)
10,735 (93.3)
3,667 (31.9)
1,304 (11.3)
Insurance type, n (%)
Commercial (yes versus no)
4,752 (55.5)
1,146 (24.1)
461 (9.7)
6,870 (57.4)
6,387 (93.0)
2,206 (32.1)
754 (11.0)
Medicare (yes versus no)
1,870 (21.8)
470 (25.1)
152 (8.1)
2,640 (22.0)
2,484 (94.1)
808 (30.6)
293 (11.1)
Medicaid (yes versus no)
1,534 (17.9)
402 (26.2)
170 (11.1)
2,088 (17.4)
1,956 (93.7)
732 (35.1)
271 (13.0)
Uninsured (yes versus no)
789 (9.2)
235 (29.8)
84 (10.6)
922 (7.7)
874 (94.8)
310 (33.6)
109 (11.8)
Other tobacco, n (%)
Yes
416 (4.9)
75 (18.0)
16 (3.8)
598 (5.0)
531 (88.8)
95 (15.9)
22 (3.7%)
No
8,153 (95.1)
2,051 (25.2)
799 (9.8)
11,379 (95.0)
10,632 (93.4)
3,728 (32.8)
1,329 (11.7)
Total
—
2,126 (24.8)
815 (9.5)
—
11,163 (93.2)
3,823 (31.9)
1,351 (11.3)
Note: Boldface indicates statistical significance (p<0.05) by chi-square test.
a Those ready to quit were significantly younger than those not ready to quit; mean difference=1.31, SE=0.51.
b Those assessed were significantly older than those not assessed after launch; mean difference= −2.29, SE=0.58.
c Those ready to quit were significantly younger than those not ready to quit; mean difference=0.62, SE=0.28.
d Those assessed had significantly fewer visits than those who were not assessed, mean difference= −0.88, SE=0.05.
e Those ready to quit had significantly more visits than those who were not ready to quit; mean difference= −0.57, SE=0.03.
f Those eReferred had significantly more visits than those who were not eReferred; mean difference= −0.56, SE=0.04.
g,h Race subgroups with different superscript letters differ significantly in rate assessed, ready to quit, or eReferred; e.g., for post-implementation eReferral rates, Whites with a “g” superscript differ from African Americans with an “h” superscript, but not from other minority group, which also has a “g” superscript.M, mean.
To assess the representativeness of reach, bivariate relations between available patient-level variables (demographics, insurance, and tobacco type) and rates of assessment, quit readiness, and eReferral are shown in Table 1. All the 3 outcomes tended to vary significantly as a function of race, insurance type, and use of other (noncigarette) tobacco products. eReferral was especially likely for women, African American patients, those receiving Medicaid, and those who smoked exclusively versus those who used multiple forms of tobacco. Age was related to whether the patient was assessed for readiness to quit and found to be ready to quit, but age differences were small. Having had more clinic visits was associated with lower rates of readiness assessment but higher rates of readiness to quit and of eReferral.
The mean number of visits with patients who smoked for the 411 roomers (who had at least 1 encounter) was 55.7 (SD=57.0; median=43, range=1−260). The mean percentage of patients assessed for readiness to quit smoking across roomers was 72.8% (SD=35.3%; median=89.4%, range=0%–100%), the mean percentage of patients found ready to quit across roomers was 20.2% (SD=22.0%; median=22.0%, range=0%–100%), and the mean percentage of patients eReferred to the quitline across roomers was 5.2% (SD=10.6; median=1.5%, range=0%–100%). A total of 64 of 411 (15.6%) roomers never assessed readiness to quit. These appeared to be staff who roomed infrequently because their mean number of smoker visits was only 1.4 (SD=1.3), much lower than that of those who assessed readiness to quit at least once (mean=65.7, SD=56.6; t= −9.07, p<0.001). More than a quarter of roomers (114, 27.7%) never noted that a patient was ready to quit during the 8-month postlaunch period. The mean number of smoker visits for such roomers was 3.0 (SD=7.8) vs 75.9 (SD=54.8; t= −14.1, p<0.001) for roomers who identified at least 1 patient as ready to quit. Nearly half of the roomers (212, 47.4%) never eReferred a patient; the mean number of smoker visits for such roomers was 15.6 (SD=30.8) vs 88.9 (SD=51.6; t= −17.1, p<0.001) for roomers who eReferred at least 1 patient. Thus, roomers who saw very few patients who smoked tended to significantly underperform across all outcomes. Figure 2 (Panel A) shows box-and-whisker plots depicting the mean, median, quartiles, and range in roomer-level assessment, readiness to quit, and eReferral rates among roomers who had at least 10 visits with patients who smoke after implementation.
Figure 2Box-and-whisker plots of rates of assessing patient interest in quitting, patient readiness to quit within 30 days, and eReferral to the WTQL.
Note: Means are marked with an X, medians are marked with a horizontal black line, and quartiles (25th and 75th percentiles) are marked by the lower and upper borders of the colored boxes, respectively. Whiskers depict range within 1.5 times the interquartile range, and dots indicate outliers that are more than 1.5 times the interquartile range outside the box. Panel A shows distributions of implementation rates across roomers (MAs/LPN) primarily responsible for the implementation of assessment and eReferral. Panel B shows the distributions of implementation rates across clinicians responsible for approving eReferral orders. Only roomers and clinicians who saw at least 10 patients who smoked during the 8-month implementation period were included in these analyses.
indicated that visits occurring with individual roomers were more similar than were visits across different roomers, especially for assessment rates (ICC=0.15) and readiness to quit (ICC=0.09). The intraroomer correlation coefficient was lower for eReferral (ICC=0.03). These analyses were restricted to roomers with at least 2 visits with smokers (330 of 411 roomers, 80.3%).
Of 413 clinicians, 49 (11.9%) saw no patients identified as ready to quit and 146 (35.4%) never eReferred a patient to the quitline. Clinicians with no patients ready to quit had fewer visits with smokers (mean=4.4, SD=5.6) than those with 1 or more patients ready to quit (mean=62.3, SD=54.9; t= −7.38, p<0.001). Clinicians with no eReferred patients also had fewer visits with patients who smoked than clinicians with eReferred patients (mean=18.8, SD=24.7 vs mean=74.6, SD=55.9; t= −11.4, p<0.001). Figure 2 (Panel B) shows box-and-whisker plots of assessment, readiness to quit, and eReferral rates among clinicians who had at least 10 visits with patients who smoke after implementation.
Rates of visit-level assessment, readiness to quit, and WTQL referral are shown by month from launch in Figure 3 to illustrate maintenance of assessment and intervention over time. Figure 3 (Panel A) depicts the dramatic increases in the rate at which readiness to quit was assessed at smoker visits after implementation. Assessment rates reached a peak in Month 1 and did not change significantly after Month 2. The rates of reported readiness to quit at visits also increased, peaking during the launch month with only small declines thereafter. Figure 3 (Panel B) shows that fax referral rates declined slightly (but not to 0) once eReferral was introduced and that eReferral rates were consistently higher than pre-implementation fax referral rates. The percentage of all adult smoker visits that resulted in patients receiving WTQL services was consistently higher after launch than before launch. Despite this, the rates of eReferral declined over the postlaunch period with significant declines every month through Month 5. This is unlikely to be due solely to repeat visits during the implementation period of the study because the mean number of visits per patient was 2. WTQL service acceptance rates also declined significantly in the first 2 months after launch, with a more modest decline after Month 4.
Figure 3Plots of implementation maintenance across time.
Note: Panel A shows the rates of roomer documentation of assessment of patient readiness to quit within the next 30 days (solid line) and documentation of patient reports of readiness to quit within 30 days (dashed line) in the 4 months preceding and 8 months following eReferral launch. Panel B shows the rates of eReferral to the WTQL during implementation (solid line), fax referral to the WTQL before implementation and during implementation (dotted line; in months −4 to −1, this includes referrals from all clinics in the healthcare system; in months 0–7, this includes fax referrals from only the 30 primary care clinics of interest in this study), and acceptance of WTQL services among all referred (regardless of referral method, dashed line).
aThe rate for this outcome at this time point is significantly higher than the rate for the same outcome at all subsequent time points combined at p<0.05 in a chi-square test.
bThe rate for this outcome at this time point is significantly lower than the rate for the same outcome at all subsequent time points combined at p<0.05 in a chi-square test.
Fiore MC, Jaén CR, Baker TB, et al. Treating tobacco use and dependence: 2008 update. Rockville, MD: HHS, U.S. Public Health Service.https://www.ncbi.nlm.nih.gov/books/NBK63952/. Published May 2008. Accessed September 11, 2020.
However, a failure to consistently implement smoking interventions in healthcare settings is a major impediment to reducing smoking prevalence in clinic populations.
In this study, implementation and maintenance strategies were designed and executed by the healthcare system, enhancing external validity. Implementation strategies included online videos, detailed instructions, and screenshots to guide medical assistants and clinicians. Research staff delivered no implementation training. Clinic managers were responsible for ensuring that staff completed training and for providing performance feedback after implementation.
The real-world implementation/training strategy appeared to increase the reach of quitline eReferral. Among adult patients who smoked, the rate of assessing readiness to quit increased from 24.8% before launch to 93.2% after launch, the rate of readiness to quit increased from 9.5% to 31.9%, and the WTQL referral rate increased from 1.7% to 11.3%. Thus, the rate of assessment of smoker readiness to quit increased more than 3 fold, whereas the rate of referral to the quitline increased more than 6 fold. There was evidence of maintenance of rooming staff documentation of the assessment of readiness to quit and patient reports of readiness. Although eReferral rates declined significantly across months, they remained at least twice as high as pre-implementation fax referral rates.
Examination of the representativeness of reach showed that the smoking-related activities were delivered fairly equitably across patient groups; some differences were significant but typically small. One meaningfully large difference was that African American patients who smoked were more likely to be eReferred to the quitline than were White patients who smoked (15.5% vs 10.7%). In addition, those using other forms of tobacco in addition to cigarettes were less likely to be eReferred than were those who smoked cigarettes only (3.7% vs 11.1%). This is consistent with the finding that those who used multiple forms of tobacco were significantly less likely to express interest in quitting smoking than those who smoked exclusively (Table 1). In summary, there was a good reach across diverse smoker populations (e.g., women, those on Medicaid).
Only about one third of patients who said that they were willing to quit actually accepted quitline referral, which is similar to other quitline referral results.
The cause of this drop-off is unknown; some of these patients may have wanted to quit on their own or wanted to quit with a different form of assistance, or some may have merely responded to perceived social demand. A quitline connection rate of 3.6% of smokers may seem modest, but this compares with an estimated connection rate of 0.6 before launch. An estimated 10% of 600,000 patients making annual visits in this healthcare system smoked. This translates into 60,000 smokers. Thus, real-world implementation of the EHR-based system increased quitline connection from about 336 to about 2,160 per year, a difference of public health significance (although future erosion of eReferral could reduce this).
The performance of intervention steps varied markedly across rooming staff and clinicians. The rates of assessing readiness to quit and quitline eReferral ranged from 0% to 100% across roomers (who were responsible for most of the workflow steps) and clinicians (who signed off on eReferral orders). However, the large majority of roomers assessed quitting interest in more than 80% of their patients, and very few fell below 60% (Figure 2, Panel A).
Variation in assessment is likely due to roomer rather than patient factors. This view is supported by the ICCs that show that assessment rates, in particular, were more similar within roomers (across visits) than across rooming staff. In this study, the roomers seeing relatively few patients had especially low assessment rates; thus, the range of implementation rates might unduly reflect such staff. Variation in the rates of identifying those ready to quit and willing to be eReferred appears to be less roomer-dependent and may reflect patient motivation level.
The rates of documenting the assessment of readiness to quit and smokers ready to quit were very stable across the postimplementation period. Thus, eReferral, even with low-intensity implementation, appears to result in the consistent assessment of smoker readiness over time. However, although the percentage who smoked and said that they were ready to quit did not drop meaningfully in Months 2–7, eReferral of such individuals dropped significantly during this time for unknown reasons. Roomers may have stopped using or promoting eReferral or patients may have declined repeat referrals; low rates of multiple visits cast doubt on the latter account. The drop in eReferral rates was steep (falling by more than 50%) and warrants further research. Perhaps more intensive implementation training (e.g., extended or in-person training or training that provides roomers with more skills such as motivational intervention) might have improved maintenance.
Limitations
Limitations include a lack of follow-up data on smoking outcomes (e.g., abstinence). In addition, this pragmatic
health system–led project comprised a limited set of implementation measures. Thus, some targeted RE-AIM constructs were not assessed (e.g., cost and adaptation
). This evaluation also lacks intensive qualitative evidence that might shed light on the sources and determinants of variation in staff and clinician performance. Finally, the extent to which the EHR-based system change altered actual intervention delivery versus reporting of delivery is uncertain.
CONCLUSIONS
Low-intensity CBT of an EHR-based system change yielded marked and persisting improvements in rates of documenting readiness to quit among adult primary care patients who smoke. It also more than doubled the rate that patients who smoked were referred to the tobacco quitline. Although eReferral rates declined over the first several months of implementation, they remained much higher than pre-implementation fax referral rates. The reach of WTQL services improved after the implementation of eReferral, and reach was fairly equitable across patient characteristics available for analysis. Reach was particularly high among African American patients and Medicaid recipients.
There was considerable variability in the performance of roomers. About half of roomers never eReferred a smoker to quitline treatment. Absolute eReferral and WTQL service reach rates remained low, suggesting the need to explore more intensive implementation strategies (e.g., in-person training) or motivational strategies to increase smoker willingness to accept treatment, such as incentives
A randomized controlled trial of financial incentives to low income pregnant women to engage in smoking cessation treatment: effects on post-birth abstinence.
This research identified factors that are associated with relatively poor eReferral implementation: (1) patients who use other forms of tobacco in addition to cigarettes, (2) staff or clinicians who do not regularly perform rooming duties or who see relatively few patients who smoke, and (3) the passage of time after implementation. Thus, this research suggests how implementation efforts may be targeted—which patients, which staff, and when.
ACKNOWLEDGMENTS
The authors thank Wendy Theobald for her invaluable assistance with background research and manuscript preparation.
This work was supported by National Cancer Institute Grants, PO1 CA180945 and R35CA197573 (TBB and MCF). The funder had no role in the study design; collection, analysis, and interpretation of data; writing the report; or the decision to submit for publication.
All authors contributed to manuscript preparation. MCF and TBB secured funding. MCF, DEM, and RG contributed to the design. All authors assisted with developing methods and conducting the research. All authors contributed to the data analysis and interpretation of the analytic results.
The material in this manuscript has not been presented elsewhere.
TBB has served as a consultant to ICF International regarding their involvement in the work for the National Cancer Institute. No other financial disclosures were reported.
This article is part of a supplement entitled The Role of Quitlines in Tobacco Cessation, which is sponsored by the U.S. Centers for Disease Control and Prevention (CDC), an agency of the U.S. Department of Health and Human Services (HHS), with support from RTI International under contract # 200-2014-61263.
REFERENCES
Fiore MC, Jaén CR, Baker TB, et al. Treating tobacco use and dependence: 2008 update. Rockville, MD: HHS, U.S. Public Health Service.https://www.ncbi.nlm.nih.gov/books/NBK63952/. Published May 2008. Accessed September 11, 2020.
Tobacco use screening and counseling during physician office visits among adults–National Ambulatory Medical Care Survey and National Health Interview Survey, United States, 2005-2009.
Using the electronic health record to connect primary care patients to evidence-based telephonic tobacco quitline services: a closed-loop demonstration project.
An electronic health record-based interoperable eReferral system to enhance smoking quitline treatment in primary care [published correction appears in J Am Med Inform Assoc. 2019;26(10):1159].
Recruiting and engaging smokers in treatment in a primary care setting: developing a chronic care model implemented through a modified electronic health record.
A randomized controlled trial of financial incentives to low income pregnant women to engage in smoking cessation treatment: effects on post-birth abstinence.
This article is part of a supplement entitled The Role of Quitlines in Tobacco Cessation, which is sponsored by the U.S. Centers for Disease Control and Prevention (CDC), an agency of the U.S. Department of Health and Human Services (HHS), with support from RTI International.
30327-5&id=gr1.jpg){kind=link}
30327-5&id=gr2.jpg){kind=link}
30327-5&id=gr3.jpg){kind=link}