Advertisement

Learning About and Using Research Evidence Among Public Health Practitioners

      Introduction

      Funders and accreditation standards increasingly call on state and local public health agencies to use the best available science. Using research evidence is a key process in practicing evidence-based decision making (EBDM). This study explored preferences for and uses of research evidence, and examined correlates regarding frequency of use.

      Methods

      In 2014, eligible staff from 12 state health departments and their partnering agencies were invited to complete an online self-report questionnaire and achieved an 82% response rate (1,237/1,509). The cross-sectional data analyzed in 2015 were baseline to a study on enhancing EBDM capacity and supports.

      Results

      Webinars/workshops was the most frequently selected method to learn public health findings among those in state and local health departments, whereas academic journals was the top selection by those in universities and healthcare facilities (p<0.001). Several modifiable EBDM practices were associated with more frequent use of research evidence, including direct supervisor expectations for EBDM use and performance evaluation based partially on EBDM use (AOR=2.5, 95% CI=1.9, 3.2 and AOR=2.5, 95% CI=2.1, 2.9, respectively). Increased numbers of EBDM practices were associated with higher odds of frequent research evidence use. Participant characteristics associated with higher research evidence use and adjusted for were job role, education attainment, and gender.

      Conclusions

      To translate research into public health practice, researchers can tailor evidence on intervention implementation and effectiveness and disease burden to accessible and preferred formats for public health workers and partners. Management practices to support evidence-based disease prevention can be instituted and fostered in public health and partnering agencies.

      Introduction

      Evidence-based decision making (EBDM) in public health integrates best available scientific findings into making decisions at all stages of public health programing (planning, implementing, evaluating, and disseminating) and can lead to more effective use of scarce funds and improved community health outcomes.
      • Kohatsu N.D.
      • Robinson J.G.
      • Torner J.C.
      Evidence-based public health: an evolving concept.
      • Brownson R.C.
      Evidence-Based Public Health.
      To maintain robust EBDM processes within organizations, scientific findings must be regularly consumed and applied by staff to day-to-day public health tasks. Retrieval and synthesis of evidence is a core public health competency.

      The Council on Linkages Between Academia and Public Health Practice. Core competencies for public health professionals. June 2014. www.phf.org/resourcestools/pages/core_public_health_competencies.aspx.

      Although diversity in source and format preferences for receiving evidence has been documented,
      • Revere D.
      • Turner A.M.
      • Madhavan A.
      • et al.
      Understanding the information needs of public health practitioners: a literature review to inform design of an interactive digital knowledge management system.
      • Fields R.P.
      • Stamatakis K.A.
      • Duggan K.
      • Brownson R.C.
      Importance of scientific resources among local public health practitioners.
      • Harris J.K.
      • Allen P.
      • Jacob R.R.
      • Elliott L.
      • Brownson R.C.
      Information-seeking among chronic disease prevention staff in state health departments: use of academic journals.
      • Dobbins M.
      • Jack S.
      • Thomas H.
      • Kothari A.
      Public health decision-makersʼ informational needs and preferences for receiving research evidence.
      few studies explore factors influencing frequency of research evidence use. These studies suggest that frequency of research evidence use and preferences are contextual to the individual user, including time, access, resources, organizational setting and culture, perceptions of relevance, and position or role within an agency.
      • Orton L.
      • Lloyd-Williams F.
      • Taylor-Robinson D.
      • O’Flaherty M.
      • Capewell S.
      The use of research evidence in public health decision making processes: systematic review.
      • Zardo P.
      • Collie A.
      Predicting research use in a public health policy environment: results of a logistic regression analysis.
      • Zardo P.
      • Collie A.
      Type, frequency and purpose of information used to inform public health policy and program decision-making.
      Little is documented regarding research evidence use or preferences for non-governmental agency partners.
      • Brownson R.C.
      Evidence-Based Public Health.
      This study examined preferences for receiving evidence, and explored correlates of research evidence use in public health programming among a sample of 12 U.S. state health department disease prevention and health promotion staff and partner health organizations to understand possible touch points for tailoring dissemination practices.

      Methods

      As baseline to an ongoing study
      • Allen P.
      • Sequeira S.
      • Jacob R.R.
      • et al.
      Promoting state health department evidence-based cancer and chronic disease prevention: a multi-phase dissemination study with a cluster randomized trial component.
      to support state health departments in enhancing capacity for EBDM, a staggered enrollment survey was conducted with staff in 12 randomly selected state health departments and partners between January and November 2014. State agency staff supplied the authors with health department and partner organization contacts working in disease prevention and health promotion (e.g., local coalition staff from universities, community-based organizations). Data presented here are a cross-sectional snapshot. The Washington University in St. Louis IRB granted human subject exempt approval.
      Study staff sent e-mail invitations to participate in the online survey. Overall, 1,237 of 1,509 eligible health department staff and their partners (82.0%) participated. Of the 1,509, a total of 45.4% were from state health departments.

      Measures

      The 63-item survey was developed from the authors’ previous survey tools assessing EBDM in governmental health departments.
      • Brownson R.C.
      • Reis R.S.
      • Allen P.
      • et al.
      Understanding administrative evidence-based practices: findings from a survey of local health department leaders.
      Most survey items have been tested for reliability.
      • Allen P.
      • Sequeira S.
      • Jacob R.R.
      • et al.
      Promoting state health department evidence-based cancer and chronic disease prevention: a multi-phase dissemination study with a cluster randomized trial component.
      • Reis R.S.
      • Duggan K.
      • Allen P.
      • et al.
      Developing a tool to assess administrative evidence-based practices in local health departments.
      Agreement with modifiable organizational supports for EBDM (e.g., work unit access to research)
      • Brownson R.C.
      • Allen P.
      • Duggan K.
      • Stamatakis K.A.
      • Erwin P.C.
      Fostering more-effective public health by identifying administrative evidence-based practices: a review of the literature.
      was assessed with a 7-point Likert scale. Participants indicated their top three preferred methods to learn about public health findings from a list. A 4-point Likert scale assessed frequency of research evidence use for each of six job functions, such as planning and implementing programs.

      Statistical Analysis

      In 2015, descriptive and bivariate statistics were conducted to explore participant characteristics. For frequency of research evidence use, each Likert point was assigned a score and averaged across the six functions for each participant. Individual averages were arranged into three frequency tertiles based on distribution.
      The study sought to examine the relationship between modifiable organizational practices
      • Brownson R.C.
      • Allen P.
      • Duggan K.
      • Stamatakis K.A.
      • Erwin P.C.
      Fostering more-effective public health by identifying administrative evidence-based practices: a review of the literature.
      and frequency of research evidence use. Each EBDM practice was entered separately using generalized estimating equations with robust estimators to allow for state clustering and adjusted for participant characteristics associated with higher research evidence use (gender, job position, and education level).

      Results

      Nearly half (48.3%) of participants were state health department staff, with others representing a variety of organizations (Table 1). Most (80.3%) were women and 65.2% held a master’s degree or higher in any field. The majority of participants worked in chronic disease prevention (59.5%). Other non-communicable areas, such as injury prevention, were represented (12.9%), as well as cross-cutting positions (e.g., health equity; 25.4%). Job positions are highlighted in Table 2. Participants reported working in public health for 14.8 (SD=9.5) years.
      Table 1Preferred Methods to Learn Research Evidence Among State Health Department Staff and Partners (n=1,237)
      Selected methodState health department, % (95% CI) (n=596 [48.3%])Local health department, % (95% CI) (n=147 [11.9%])Community-based organization, % (95% CI) (n=225 [18.2%])University/ academic institution, % (95% CI) (n=107; 8.7%)Healthcare facility/health plan, % (95% CI) (n=86; 7.0%)Other state/local/tribal government agencies, % (95% CI) (n=51; 4.1%)p-value
      Seminars or workshops (phone, webinars, or in-person)59.2 (55.3, 63.2)50.4 (42.0, 58.7)45.0 (38.4, 51.6)39.6 (30.2, 49.1)50.6 (39.7, 61.4)39.6 (25.2, 53.9)0.000
      Academic journals50.3 (46.3, 54.4)34.8 (26.8, 42.7)40.1 (33.6, 46.6)68.9 (59.9, 77.8)64.7 (54.3, 75.1)41.7 (27.2, 56.1)0.000
      E-mail alerts40.4 (36.4, 44.4)46.1 (37.8, 54.4)37.4 (31.0, 43.8)29.2 (20.4, 38.0)37.6 (27.1, 48.2)31.3 (17.6, 44.9)0.100
      Policy briefs30.1 (26.4, 33.9)24.1 (17.0, 31.3)30.2 (24.1, 36.3)26.4 (17.9, 34.9)11.8 (4.8, 18.8)22.9 (10.6, 35.3)0.011
      Professional associations24.5 (21.0, 28.0)29.8 (22.1, 37.4)23.4 (17.8, 29.0)29.2 (20.4, 38.0)36.5 (26.0, 46.9)22.9 (10.6, 35.3)0.149
      Newsletters20.4 (17.1, 23.7)31.2 (23.5, 38.9)26.1 (20.3, 32.0)17.0 (9.7, 24.2)17.6 (9.4, 25.9)33.3 (19.5, 47.2)0.009
      Academic conferences17.5 (14.4, 20.6)24.1 (17.0, 31.3)15.8 (10.9, 20.6)36.8 (27.5, 46.1)27.1 (17.4, 36.7)16.7 (5.7, 27.6)0.000
      Face-to-face meetings with stakeholders
      Other commonly selected options not shown are: other conferences, press releases, targeted mailings, social media (Facebook, Twitter), reports to funders, and media interviews.
      14.7 (11.8, 17.6)14.9 (8.9, 20.8)25.2 (19.5, 31.0)15.1 (8.2, 22.0)18.8 (10.3, 27.3)27.1 (14.0, 40.1)0.006
      Note: Boldface indicates statistical significance (p<0.05).
      a Other commonly selected options not shown are: other conferences, press releases, targeted mailings, social media (Facebook, Twitter), reports to funders, and media interviews.
      Table 2AORs for Research Evidence Use
      Dependent variable was research evidence use (highest tertile versus other).
      Among State Health Department Staff and Partners
      Independent variablesPercentage of respondentsAll, AOR
      Separate generalized estimating equation estimates for each EBDM practice and EBDM practices sum score, after adjusting for participant characteristics and clustering by state, and corresponding 95% CIs. EBDM, evidence-based decision making.
      (95% CI) (n=1,237)
      Administrative practices for EBDM support and use (strongly agree or agree)
       My direct supervisor expects me to use EBDM58.52.5 (1.9, 3.2)
       Performance is partially evaluated on EBDM use28.82.5 (2.1, 2.9)
       Work unit has access to current research evidence56.41.8 (1.4, 2.3)
       Informational resources (academic journals, guidelines, toolkits) are available to my work unit51.31.5 (1.2, 1.9)
       Information is widely shared in work unit so that everyone who makes decisions has access48.71.5 (1.2, 1.8)
       Work unit engages in a diverse external network of partners that share resources for EBDM44.91.6 (1.3, 2.0)
       Top leadership in my agency encourages use of EBDM53.31.8 (1.4, 2.2)
       EBDM practices sum score
        All 7 EBDM practices listed above11.84.3 (3.1, 5.9)
        5–6 EBDM practices24.42.2 (1.8, 2.7)
        3–4 EBDM practices26.31.5 (1.2, 1.9)
        0–2 EBDM practices37.51.0 (reference)
      Participant characteristics
       Job position type
        Program manager or coordinator46.71.0 (reference)
        Agency director or bureau head18.71.3 (1.1, 1.7)
        Specialist (epidemiologist, statistician, evaluator)30.10.8 (0.6, 0.9)
        Other position4.51.4 (0.7, 2.8)
       At least master’s in any field65.21.9 (1.6, 2.3)
       Female80.31.6 (1.3, 2.1)
      Note: Boldface indicates statistical significance (p<0.05).
      a Dependent variable was research evidence use (highest tertile versus other).
      b Separate generalized estimating equation estimates for each EBDM practice and EBDM practices sum score, after adjusting for participant characteristics and clustering by state, and corresponding 95% CIs.EBDM, evidence-based decision making.
      Commonly preferred methods for receiving research evidence varied significantly by agency type (Table 1). Webinars/workshops was the most frequently selected method to learn public health findings among those in state and local health departments, whereas academic journals was the top selection by those in universities and healthcare facilities (p<0.001).
      The majority of participants reported often or always using research evidence to: select policies, programs, or other types of interventions (72.1%); justify the selection of interventions to funders, agency leadership, or external partners (69.9%); evaluate interventions (62.8%); write a grant application (60.6%); or plan or conduct a needs assessment (58.4%).
      Table 2 shows correlates of the highest tertile of frequency of evidence use. Having at least a master’s degree in any field, being female, and working in leadership positions were associated with frequent research evidence use. Several EBDM administrative practices were associated with higher frequency of research evidence use. Findings were similar when stratified by state population or program area or accreditation status (data not shown). A direct relationship was observed, with the likelihood of frequent research evidence use increasing as the number of reported EBDM administrative practices increased.

      Discussion

      Preferences for how to receive research evidence varied by agency type. Several modifiable and relatively low-cost EBDM practices, such as supervisory expectations of EBDM use, were associated with higher frequency of research evidence use in public health job functions among participants (regardless of organization type).
      Preference to receive research evidence through academic journals was higher among those with access to full-text journal articles through university library subscriptions, whereas full-text access remained a barrier for others, as found previously.
      • Fields R.P.
      • Stamatakis K.A.
      • Duggan K.
      • Brownson R.C.
      Importance of scientific resources among local public health practitioners.
      • Harris J.K.
      • Allen P.
      • Jacob R.R.
      • Elliott L.
      • Brownson R.C.
      Information-seeking among chronic disease prevention staff in state health departments: use of academic journals.
      • LaPelle N.R.
      • Dahlen K.
      • Gabella B.A.
      • Juhl A.L.
      • Martin E.
      Overcoming inertia: increasing public health departments’ access to evidence-based information and promoting usage to inform practice.
      Seminars or workshops was a top choice among staff from governmental agencies here and in a recent national study,
      • Fields R.P.
      • Stamatakis K.A.
      • Duggan K.
      • Brownson R.C.
      Importance of scientific resources among local public health practitioners.
      and may relate to lack of journal access, time constraints, or information overload.
      • Orton L.
      • Lloyd-Williams F.
      • Taylor-Robinson D.
      • O’Flaherty M.
      • Capewell S.
      The use of research evidence in public health decision making processes: systematic review.
      E-mail alerts, policy briefs, and professional associations are examples of active dissemination methods preferred by many participants that can be tailored to various issues and settings.
      Similar to Zardo and colleagues,
      • Zardo P.
      • Collie A.
      Predicting research use in a public health policy environment: results of a logistic regression analysis.
      participants in leadership or middle management were likely to use research evidence more frequently than program managers. Likewise, specialists used research evidence less often than program managers, suggesting a need for more frequent reference to evidence in daily planning positions and those overseeing the overall management of the programming process. The study also found having a master’s or doctoral degree in any field was positively associated with frequent research evidence use, also reported elsewhere.
      • Zardo P.
      • Collie A.
      Predicting research use in a public health policy environment: results of a logistic regression analysis.
      Dissimilarly,
      • Zardo P.
      • Collie A.
      Predicting research use in a public health policy environment: results of a logistic regression analysis.
      being female was positively associated with frequent evidence use, suggesting more information may be needed to understand how gender plays a role in research evidence use.
      Several perceptions of organizational support for EBDM were found to be associated with more frequent use of research evidence in program planning, implementation, evaluation, and dissemination. Moreover, findings suggest that relatively low-cost administrative practices may increase the likelihood for more frequent research evidence use by staff. Consistent familiarity with the latest evidence-based approaches, especially within leadership teams, may create an environment for communication about systematic decisions to fund (or potentially retool or defund) various programs to implement. Individuals without advanced degrees in public health, and/or local health departments lacking staff capacity, may especially benefit from an organizational norm of valuing research evidence as part of EBDM planning and evaluation processes and from on-the-job trainings, supervisory expectations to use EBDM, and ready access to evidence sources to gain more skills, familiarity, and support for regular research evidence use.
      • Brownson R.C.
      • Reis R.S.
      • Allen P.
      • et al.
      Understanding administrative evidence-based practices: findings from a survey of local health department leaders.

      Limitations

      The sample involves staff working in non-communicable disease prevention and health promotion. Future research in other public health areas, such as infectious disease and environmental health, is needed. National-level estimates are needed to provide additional insight into research evidence use and preferences. As these data are cross-sectional, causality cannot be inferred.

      Conclusions

      Ongoing use of research evidence is key in building robust EBDM processes within governmental public health and partnering agencies. More tailoring of research evidence dissemination to reach partnering organizations is needed. Additional efforts to build organizational cultures and structures that support routine research evidence use (e.g., ongoing training) are needed, especially within partner organizations/health departments lacking staff with advanced public health training.

      Acknowledgments

      This publication is a product of the Prevention Research Centers Program at the Centers for Disease Control and Prevention. The findings and conclusions in this publication are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or NIH.
      We appreciate the survey development guidance from Jon Kerner, Canadian Partnership Against Cancer; Maureen Dobbins, McMaster University in Ontario, Canada; Jenine K. Harris and Kathleen Duggan, Brown School, Washington University in St. Louis; Elizabeth A. Baker and Katherine A. Stamatakis, College of Public Health and Social Justice, St. Louis University; Vicki Benard and Sonia Sequeira, Centers for Disease Control and Prevention; and Leslie Best and Ellen Jones, National Association of Chronic Disease Directors. We are grateful for the reliability test−retest guidance and statistical analyses provided by Rodrigo Reis and Akira Hino, Pontifical Catholic University of Parana, School of Health and Biosciences and Federal University of Parana, Department of Physical Education, Curitiba, Brazil. We appreciate Lindsay Elliott for data collection and other contributions while a graduate student at the Brown School, Washington University in St. Louis.
      This study is funded by the National Cancer Institute (Award Number 5R01CA160327). The Washington University in St. Louis IRB approved exempt E-2 status (IRB number 201111105). Author contributions included study design (RCB), survey development (RCB, PA, RRJ), data collection (RRJ, PA), data management (RRJ), data analyses (RRJ, PA), and substantive manuscript content and revisions (RRJ, PA, RCB, LJA). This study is registered with ClinicalTrials.gov as a cluster randomized trial (NCT01978054) with state health departments.
      No financial disclosures were reported by the authors of this paper.

      References

        • Kohatsu N.D.
        • Robinson J.G.
        • Torner J.C.
        Evidence-based public health: an evolving concept.
        Am J Prev Med. 2004; 27: 417-421https://doi.org/10.1016/j.amepre.2004.07.019
        • Brownson R.C.
        Evidence-Based Public Health.
        2nd ed. Oxford University Press, New York2011
      1. The Council on Linkages Between Academia and Public Health Practice. Core competencies for public health professionals. June 2014. www.phf.org/resourcestools/pages/core_public_health_competencies.aspx.

        • Revere D.
        • Turner A.M.
        • Madhavan A.
        • et al.
        Understanding the information needs of public health practitioners: a literature review to inform design of an interactive digital knowledge management system.
        J Biomed Inform. 2007; 40: 410-421https://doi.org/10.1016/j.jbi.2006.12.008
        • Fields R.P.
        • Stamatakis K.A.
        • Duggan K.
        • Brownson R.C.
        Importance of scientific resources among local public health practitioners.
        Am J Public Health. 2015; 105: S288-S294https://doi.org/10.2105/AJPH.2014.302323
        • Harris J.K.
        • Allen P.
        • Jacob R.R.
        • Elliott L.
        • Brownson R.C.
        Information-seeking among chronic disease prevention staff in state health departments: use of academic journals.
        Prev Chronic Dis. 2014; 11: E138https://doi.org/10.5888/pcd11.140201
        • Dobbins M.
        • Jack S.
        • Thomas H.
        • Kothari A.
        Public health decision-makersʼ informational needs and preferences for receiving research evidence.
        Worldviews Evid Based Nur. 2007; 4: 156-163https://doi.org/10.1111/j.1741-6787.2007.00089.x
        • Orton L.
        • Lloyd-Williams F.
        • Taylor-Robinson D.
        • O’Flaherty M.
        • Capewell S.
        The use of research evidence in public health decision making processes: systematic review.
        PLoS One. 2011; 6: e21704https://doi.org/10.1371/journal.pone.0021704
        • Zardo P.
        • Collie A.
        Predicting research use in a public health policy environment: results of a logistic regression analysis.
        Implement Sci. 2014; 9: 142https://doi.org/10.1186/s13012-014-0142-8
        • Zardo P.
        • Collie A.
        Type, frequency and purpose of information used to inform public health policy and program decision-making.
        BMC Public Health. 2015; 15: 381https://doi.org/10.1186/s12889-015-1581-0
        • Allen P.
        • Sequeira S.
        • Jacob R.R.
        • et al.
        Promoting state health department evidence-based cancer and chronic disease prevention: a multi-phase dissemination study with a cluster randomized trial component.
        Implement Sci. 2013; 8: 141https://doi.org/10.1186/1748-5908-8-141
        • Brownson R.C.
        • Reis R.S.
        • Allen P.
        • et al.
        Understanding administrative evidence-based practices: findings from a survey of local health department leaders.
        Am J Prev Med. 2014; 46: 49-57https://doi.org/10.1016/j.amepre.2013.08.013
        • Reis R.S.
        • Duggan K.
        • Allen P.
        • et al.
        Developing a tool to assess administrative evidence-based practices in local health departments.
        Front Public Health Serv Syst Res. 2014; 3 (Article 2)https://doi.org/10.13023/FPHSSR.0303.02
        • Brownson R.C.
        • Allen P.
        • Duggan K.
        • Stamatakis K.A.
        • Erwin P.C.
        Fostering more-effective public health by identifying administrative evidence-based practices: a review of the literature.
        Am J Prev Med. 2012; 43: 309-319https://doi.org/10.1016/j.amepre.2012.06.006
        • LaPelle N.R.
        • Dahlen K.
        • Gabella B.A.
        • Juhl A.L.
        • Martin E.
        Overcoming inertia: increasing public health departments’ access to evidence-based information and promoting usage to inform practice.
        Am J Public Health. 2014; 104: 77-80https://doi.org/10.2105/AJPH.2013.301404