If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to: Amelie G. Ramirez, DrPH, MPH, Institute for Health Promotion Research, The University of Texas Health Science Center at San Antonio, 7411 John Smith, Suite 1000, San Antonio TX 78229
Increasingly, funders expect that public health researchers will include policy contributions as outcomes. Lack of agreement as to what constitutes a policy contribution of research provides little conceptual or implementation guidance to researchers who lack policy training, as well as to evaluators called on to assess “good” policy contribution.
This study applies a previously developed policy framework to explore potential policy contributions from research conducted by 20 principal investigators of Salud America!, the Robert Wood Johnson Foundation's (RWJF) Research Network to Prevent Obesity Among Latino Children.
The literature-driven “Policy Contribution Spectra” served as the conceptual framework to jointly develop 20 cases of potential policy contribution. Data collection included document reviews and interviews. Data analysis included within- and cross-case analyses, member checking, data triangulation, and expert reviews.
Plotting all 20 projects on the Policy Contribution Spectra showed projects have the potential to contribute to policy across intervention types (e.g., needs assessment or applied research); levels (e.g., local or state); timing (e.g., before or after policy enactment); and outcomes (e.g., process action or health benefits). Potential policy contributions on the Spectra framework were shown as multidirectional; multilayered (e.g., simultaneous state and local action); and multidimensional (e.g., multiple strategies aimed at multiple stakeholders).
The Policy Contribution Spectra adds a useful policy lens to existing public health practice by enabling researchers, funders, advocates, and evaluators to visualize, reframe, discuss, and communicate with policymakers and the public to resolve important public health issues.
Policy and environmental approaches must be part of the mix in public health, as many ecologic frameworks have long advocated, and to which the Healthy People 2020 objectives for disease prevention and health promotion have now committed the DHHS and its partners.
The literature on health promotion supports giving more weight to policy interventions and environmental change and is turning to the evidence on bringing about policy change or making policy contributions.
A growing literature on the importance of local policy change, especially in underserved populations, has been short on guidance from research on points of intervention in the policy-making process. Policy can be served by research on the delineation of the stages and levels of policy influence.
Such research also would contribute to the evaluation challenge of determining what counts as successful policy contributions.
Researchers, program managers, advocates, funders, and evaluators who contribute to policy and environmental solutions lack understanding and agreement as to what counts as a policy contribution. When questioned about policy contributions, key informants in a national obesity control research program answered: “passing of legislation, the raising of community awareness, or something else.”
These definitions were simultaneously diverse, specific, and vague. To pick one of these definitions over another, such as the passage of legislation, discounts other potential policy-related contributions, including raising awareness. If stakeholders hold such disparate views on policy contribution, is it wise or necessary to choose one definition over another? Is it possible to encompass and link these definitions conceptually?
The present study assesses the usefulness of a previously developed conceptual framework
—The Policy Contribution Spectra—to engage researchers in the policy process and to identify the potential policy contributions of their efforts. The Spectra framework was applied to 20 pilot research projects funded by the Robert Wood Johnson Foundation (RWJF) through its Salud America! program, which aims to reduce Latino childhood obesity (www.salud-america.org/research.html). Salud America! is a national network of researchers, advocates, and policymakers working to increase the number of researchers and the amount of research seeking environmental and policy solutions to Latino childhood obesity. The effort aligns with RWJF's national initiative to reverse the epidemic by 2015 (www.rwjf.org/en/about-rwjf/program-areas/childhood-obesity.html). Salud America! is headquartered at and directed by the Institute for Health Promotion Research at The University of Texas Health Science Center at San Antonio. A National Advisory Committee helps guide network decisions.
Salud America! funded 20 projects in 2009 for up to 24 months with budgets up to $75,000 each. Each project was rooted in an understanding of rising U.S. childhood obesity rates and the corresponding effects on Latino children, who have among the highest rates.
All projects were responsible for identifying the policy contributions of their efforts. Yet, few of the scientifically and professionally stellar principal investigators/researchers had any formal policy-related training, highlighting the need for a conceptual framework to help them see beyond hypothesis testing and reframe program effectiveness to include policy processes and contributions. The Policy Contribution Spectra framework was introduced to Salud America! principal investigators collectively and through individual case study development.
The case study method was used to identify and explore potential policy contributions by Salud America! research projects. Case studies are an appropriate strategy when research asks “how” and “why” questions about a contemporary phenomenon in uncontrolled contexts.
The unit of analysis, or case, was conceptualized as the potential policy contribution of each Salud America! project.
Policy contribution was not predefined for the investigators, but rather explored as multiple facets of the policy process as conceptualized in the Spectra framework. The use of a conceptual framework or map, such as the Policy Contribution Spectra (Figure 1), helps make study propositions and assumptions explicit.
to identify multiple policy stages: pre-policy, developing policy, policy enactment, and post-policy enactment. Within these stages are multiple policy signposts around which the work of policy coalesces, such as problem described, awareness, advocacy, and implementation.
The Science/Evaluation Spectrum. This spectrum (top of Figure 1) adds public health influences on the policy process from the work of basic research to evaluation, including the effort to determine problem-related factors; assess needs; create knowledge by testing interventions (e.g., hypotheses testing or replication); and/or evaluation (e.g., determine intervention value to inform decision making.)
It is anticipated that each spectrum interacts with the other two in policy contribution. The Spectra framework served as a guide for determining data collection methods and analyses.
Within-case data collection and analysis for each of the 20 projects began with document reviews of logic models, program descriptions, grant applications, and other project-related documents. Some principal investigators demonstrated intended policy contributions by plotting them to the Spectra framework. Follow-up interviews that lasted up to 1 hour with each principal investigator and some research teams were conducted in 2010. During the interviews, a researcher familiar with the Spectra framework probed each project for potential policy contributions. Additional questions were asked about project context, including constituencies, intent, strategies, and Latino-specific policy issues.
Using findings from the document reviews and interviews, a one-page case study of potential policy contribution was written for each of the 20 Salud America! projects in collaboration with the principal investigator. Each case was then plotted to the Spectra framework, showing potential policy signposts on the policy spectrum, project focus on the science/evaluation spectrum, and considered public interest on the public spectrum. A summary of these initial case descriptions and plots were presented to Salud America! leadership and all principal investigators for review and comment at the network's annual conference in September 2010.
In October 2010, the individual case description and plot on the Spectra framework were sent to each principal investigator. Between October 2010 and September 2011, principal investigators and their research teams honed their own cases through edits, individual or project-team phone consultation, individual meetings, taped interviews, project presentations, or e-mails. Member checking—which tests data with members of a group from whom the data were originally obtained—confirmed the face validity of each case and provided deeper insight into potential policy contribution.
Cross-case analyses explored emerging dimensions of potential policy contributions of Salud America! research, including level of contribution (e.g., organizational, community, state, or national); type of policy contribution (e.g., describe the problem, create awareness, or implement); temporal order of policy contributions; and the interaction within the Spectra framework (e.g., public spectrum influence on the policy spectrum). Comparisons were made between potential policy contributions identified by principal investigators and the Spectra framework propositions.
To address objectivity/confirmability standards, the researcher sought to make bias explicit through open critique of the Spectra framework. Reliability/dependability/auditability was addressed through consistent and transparent detail of methods and analysis. Internal validity/credibility/authenticity was addressed through data triangulation, member checking with principal investigators as previously noted,
External validity/transfer ability/fittingness was addressed through review and comments by National Advisory Committee members of Salud America!, who represented related stakeholders and organizations. Regarding utilization/application/action orientation, a detailed report of findings was provided to all principal investigators and stakeholders. Even with many efforts to address these standards, a cautionary note is offered regarding the degree to which the use of a conceptual framework sometimes constricts, rather than informs, findings. Further, the limited scope of these cases might limit their transferability to other program types.
Individual Salud America! projects plotted to the Policy Contribution Spectra are shown in Figure 2 (Projects 1–10) and Figure 3 (Projects 11–20). Projects are referred to by number only because they were offered confidentiality for participation in the study. Examples of individual cases of potential policy contribution are discussed below, followed by collective findings on each spectrum.
Individual Cases of Potential Policy Contribution
Three cases demonstrate different paths to potential policy contribution and the usefulness of the Spectra framework in capturing them. A relatively straightforward example, Case 7 (Figure 2) shows a project determining factors associated with Latino childhood obesity (science/evaluation spectrum). The project holds itself accountable for potential policy contributions starting with problem description in the pre-policy stage and moving to increased awareness about obesity in the developing policy stage (policy spectrum). The project intends (dashed line) that these contributions lead to community mobilization in the developing policy stage (policy spectrum), but does not judge its success by that outcome. The public was estimated to be in the pre-problem stage (public spectrum).
As a policy evaluation (science/evaluation spectrum), Case 16 (Figure 3) starts potential policy contribution in the post-policy enactment stage by assessing policy implementation (16a on the policy spectrum) and moves “back” to the pre-policy stage (16b) to determine the impact of policy on the described problem. The project holds itself accountable for moving “forward” on the Spectra framework toward developing policy by using evaluation findings to mobilize the community (policy spectrum); Project 16 intends (dashed line) that findings be used to reframe and pass future policy. Public interest is estimated to be at the pre-problem stage (public spectrum).
Project 6 (Figure 2) demonstrates a multidirectional and multidimensional intent to contribute to policy. This project starts potential policy contribution by raising awareness of obesity (policy spectrum) and assessing needs (science/evaluation spectrum) through the use of grassroots social action. The findings are then used to inform the pre-policy problem description and identify multiple strategies in the developing policy stage (policy spectrum). Although the research project intends (dashed lines) that its efforts lead to the passage of legislation, it does not judge its success by such outcomes. The public is estimated initially to be at the pre-problem stage.
Individually, these cases show the multiple potential paths to policy contribution taken by Salud America! principal investigators (Figure 2, Figure 3). Taken together, these multiple paths also show the potential policy contribution of the whole Salud America! network. The discussion now turns to potential policy contributions on each spectrum.
The science/evaluation spectrum (Figure 2, Figure 3) shows a range of projects aiming to determine factors, assess needs, test an intervention, or evaluate. Seven projects determined factors associated with childhood obesity at one or more levels including the individual, family, or community. Two projects focused on community-based needs assessment. Seven projects tested an intervention aimed at reducing childhood obesity for Latino children (e.g., lifestyle intervention, educational intervention). Four projects focused on evaluation of a whole program or its implementation.
All 20 projects initiated potential policy contributions at the pre-policy, developing policy, or post-policy enactment stages; none initiated policy contribution at the policy-enactment stage (Figure 2, Figure 3), policy spectrum. Fourteen projects initiated potential policy contributions in the pre-policy stage where they described the obesity-related problem (e.g., analysis of secondary child development data); studied the problem's causes and consequences (e.g., relationship between activity and academic performance); and/or described the trend or spread of the problem (e.g., lack of physical activity among Latinos). Two projects began potential policy contributions in the developing policy stage, aiming to increase awareness of childhood obesity. Four projects initiated potential policy contributions in the post-policy enactment stage through evaluation of existing policies and programs.
All projects worked to advance potential policy contribution from one stage to another. For example, all 14 projects initiated in the pre-policy stage worked to move potential policy contribution to the developing policy stage. In this later stage, all sought to increase awareness about Latino childhood obesity within and/or beyond the Latino community; some projects took their efforts further to reframe childhood obesity in the Latino context or to mobilize the community toward action. Of these 14 projects, eight ended at the developing policy stage, and six projects had strategies in place or strategies intended (dashed lines) to contribute to the next stage of policy enactment.
Strategies used to move potential policy contribution from one stage to another were common, multiple, deliberate, and iterative. Common strategies included educational programs, mass media, websites, coalitions, reports, publications, meetings, policy briefs, photography, networking, and/or community organizing. All projects used strategies in some combination with each other at multiple points across the policy spectrum. For example, using mass media helps describe the trend or spread of Latino childhood obesity in the pre-policy stage and raises awareness in the developing policy stage.
Projects proposed varying actions toward the policy enactment stage. Although six projects intended to influence the passage of legislation or policy, none held themselves accountable for meeting this criterion as the sole measure of their success. The evidence produced by research was most often viewed as a contribution, not a causal certainty, to passing legislation, thus reinforcing the revised moniker for the Policy Contribution Spectra from its original publication.
Further, that contribution of evidence influenced the policy process at multiple points as precursors to policy passage that might predispose, enable, or reinforce changes in policy promotion.
Among policy enactment precursors, advocacy was the most controversial, with some researchers who saw their role as producing knowledge, not promoting it. Of the 12 projects intending to contribute to policy enactment, five had advocacy strategies in place, six had intent but no strategies, and one project stated that their institution did not allow advocacy.
Seventeen of 20 projects identified the public at the pre-problem stage (Figure 2, Figure 3, public spectrum). For many projects, the public did not share researcher or community partner concerns that Latino childhood obesity was a problem. In fact, researchers noted that cultural influences in some communities led adults to perceive bigger children as healthier children. Among some economically disadvantaged Latino communities in the Salud America! projects, using community-based participatory research strategies to engage the public on obesity was over-ridden or challenged by more-basic survival concerns about immigration status, racial profiling, or migratory movement.
To operationalize and ultimately evaluate policy contribution, it needs to be defined and described.
The Salud America! cases give a taste of what might be metaphorically thought of as a “policy contribution sandwich.” That is, the meat of the policy process (policy spectrum) is sandwiched between the work of researchers (science/evaluation spectrum) and the awareness and actions of the public (public spectrum). Although each component can be experienced independently, to make a contribution to policy, they need to be sandwiched together, as described below.
This is the spectrum at which researchers are most knowledgeable and comfortable by virtue of their educational preparation, scientific interest, and experience. In interviews or presentations, more than one Salud America! principal investigator described himself or herself as “not a policy person.” Instead, they design and conduct research; they strive for credible results; and most work in organizations that reward academic excellence. This was especially true for early-career researchers whose academic survival depended on numbers of publications and amounts of funded research—not on policies affected or legislation passed. Researchers debated how far beyond producing credible evidence they should be held accountable.
Rather than posing policy contributions as something beyond what they do, a policy lens like the Policy Contribution Spectra can help researchers see policy contribution as something they may already be doing. For example, one researcher talked with a state legislator about the issues and intents of her Salud America! project as the project began. From a health promotion perspective, she viewed her actions as networking. Without expecting her to do anything differently, her same actions could be viewed from a policy contribution perspective as creating awareness about Latino childhood obesity in the developing policy stage on the public spectrum (Figure 1). Further, creating awareness with a policymaker can spur anticipation and use of research or evaluation findings.
For health promotion researchers familiar with concepts such as awareness, actions such as mobilization, and responsibilities such as community involvement (public spectrum), changing language is not even necessary for policy contribution. What it takes is an added policy lens through which researchers can connect their work to the vision of policymakers and the public. Through policy training and working with the Spectra framework over time, the policy lens sharpened the focus for many Salud America! principal investigators. One investigator said, “[The Spectra model] got me out of my box and thinking about policy. I was further along than I thought!”
As a process, not a time point, policy contribution is multidirectional, multifaceted, and multileveled. Although most projects in this case study started in pre-policy, some started policy contribution efforts at other stages. Policy contribution is not a one-way, left-to-right march across the Spectra framework; rather, it can be multidirectional and iterative.
The directionality of one's policy contribution may be less important than the terrain crossed and policy signposts observed. For example, the three examples previously discussed include awareness as a signpost in policy contribution: Case 6 starts at awareness (Figure 2), Case 7 ends at awareness (Figure 2), and Case 16 moves beyond awareness to community mobilization (Figure 3). It is not clear that the timing of creating awareness was as potentially influential on policy as simply addressing awareness.
Potential policy contribution occurs at multiple levels. For example, principal investigators worked within schools, communities, states, and/or national networks. Although most worked in public settings, a few operated within private or nonprofit agencies. For some researchers, policy at the local level was perceived as small “p” policy and they were uncertain whether it counted as much as big “P” policy at the national level. Anyone familiar with the changes in tobacco policy over time knows that local policy change paved the way for national changes.
The contribution of layers of policy adds up because there are policymakers—and thus policy contribution opportunities—at multiple levels. Although the Spectra framework (Figure 1) may not capture the layering of policy contributions, the authors acknowledge that policy contribution is not a two-dimensional plane. Adding a policy lens to research makes it possible not only to broaden the policy picture, but also to deepen it.
Most projects determined the public in the pre-problem stage. In some communities, adults did not view childhood obesity as a problem. In some Latino communities where portions of the public are flying under the radar to avoid prosecution or persecution, this spectrum is challenged. Is it reasonable, wise, or even safe to move the public to “alarm and enthusiasm?” Community-based participatory research approaches can help to identify ways to engage, not expose, the public.
General issues should be considered in interpreting these findings. First, the Policy Contribution Spectra remains a work in progress. More descriptive work on policy contribution is needed in other health issues and contexts. Second, the cases are limited in number and generalizability. Third, the authors' roles in plotting research to the Spectra framework and writing initial case descriptions make unclear the extent to which Salud America! principal investigators owned their particular case or bought into the larger Spectra framework.
Researchers, advocates, and program managers can contribute to policy at multiple points in time, through varied actions, and at multiple levels. By adding a policy lens with multiple spectra of potential influence to existing public health practice, they can engage both policymakers and the public in understanding health issues, implementing solutions, and evaluating their effectiveness. The Spectra framework serves as one tool in helping researchers look beyond hypothesis testing and helping managers to look beyond program efficiency toward policy contribution. Policy stakeholders also should consider the fact that some level of consensus needs to be reached on what constitutes policy contribution before evaluation efforts are initiated to judge “good” policy contribution.
There is no stereotypic research focus that defines policy contribution; rather, it is the intentional links among the science/evaluation, policy, and public spectra that enable policy contribution. To make a policy contribution, researchers do not need to do everything suggested by the Spectra framework. Where one focuses is not “good” or “bad,” but rather depends on where the community is in relation to needs, public awareness and concern, political mobilization and policy experience, and what research skills the researcher can offer to address these needs.
Policy contribution is not a straight line in time or tasks.
It is definitely not limited to handing over published results. Instead, it may require multiple strategies, at multiple levels, and with multiple stakeholders at the same point in time (e.g., working with families on describing the problem, meeting with legislators on obesity awareness, community mobilization).
Advocacy is vital to policy contribution, but problematic or prohibited for some researchers. Even if these researchers want to avoid the perceived murky realm of advocacy, they certainly can lay the groundwork for others to do so. Translating policy and research into a framework, such as the Policy Contribution Spectra, makes it accessible to multiple stakeholders.
Policy contribution is increasingly required, consistently complex, and used to varying degrees by public health researchers and practitioners. To make policy contributions requires multiple policy lenses; to evaluate it takes an understanding of both what it is and which criteria comprise a successful policy contribution. The Policy Contribution Spectra served as a useful tool toward these ends in this application.
Publication of this article was supported by the Robert Wood Johnson Foundation.
This study was funded by the Robert Wood Johnson Foundation through its national program, Salud America! The RWJF Research Network to Prevent Obesity Among Latino Children (www.salud-america.org). Salud America!, led by the Institute for Health Promotion Research at The University of Texas Health Science Center at San Antonio, Texas, unites Latino researchers and advocates seeking environmental and policy solutions to the epidemic.
The authors thank RWJF, the 20 Salud America! principal investigators, the Salud America! National Advisory Committee members, and other internal scientific reviewers. Note that personnel from the Robert Wood Johnson Foundation were not involved in this analysis in any way.
No financial disclosures were reported by the authors of this paper.