IRB approval was obtained from Kaiser Permanente Washington (formerly Group Health) for evaluation of this Initiative. Interviews were conducted with 11 coordinators of LiveWell Colorado Communities. Invitations were sent to 15 of 30 currently active coordinators, including coordinators who had been in their position >1 year, who represented a broad selection of communities in terms of rurality, size, and SES. Three coordinators were unable to participate, one because she was on leave, and two because of scheduling conflicts; no one refused to participate. The 11 participating coordinators had all been involved with LiveWell Colorado for >2 years and ≤8 years. They represented four urban, four suburban, and three rural areas. Because the CHI was rolled out in several cohorts, four coordinators had used dose for early planning, four during middle-stage implementation, and three in the last 2 years of implementation only.
Coordinators lived in the communities where they were working. One coordinator role was helping organize community coalitions, which consisted of business leaders, healthcare leaders, school administration, and interested community members. The coordinator facilitated coalition meetings and provided technical assistance as coalitions created community action plans (CAPs). The coordinator then worked with the coalition and community partners to implement that plan, with review and revision. Coordinators were deeply involved in planning and implementing strategies, with coalitions having final say in implementation.
Data collection involved telephone interviews with the 11 coordinators; two were group interviews of four people each from two of the regions, and then three individual interviews were conducted with members from the third region rather than as a group because of scheduling difficulties. Interviews were designed to capture participant perceptions of how dose methods were utilized by them and by the coalitions, and how dose impacted the trajectory of their work. Trained interviewers used a semi-structured interview guide with probes to ensure that all coordinators were asked similar questions; at the end, the discussion was opened up for general comments.
The interviews were recorded, transcribed, and then coded by two analysts to gather themes, patterns, and lift-up quotes. The team drafted an initial code list based on representative transcript review. The two coders analyzed the transcripts in Excel, met to review codes and clarify patterns, and then finalized the analysis. A final coding memo was created with key findings and example quotes, which became the basis for this manuscript.
An interview was conducted with a coordinator from a fourth region that had used dose methods from the beginning of their work, and the authors present that information as a case study in how dose can work in practice if integrated from the beginning into planning and implementation work. This interview was also recorded and transcribed, and then analyzed for themes and quotes.
The dose framework provides a lens through which strategies can be assessed and prioritized in light of overarching initiative goals. For example, one coordinator from Northern California reported that dose is helpful for early decision making, with the coalition deciding to include strategies in their CAP based on contribution to overall dose in order to get “more bang for our buck.” The coordinator said, “Every strategy we looked at, we looked at with the lens of dose… . If it doesn’t have all the ingredients of dose, we ask is it worthwhile to move in this direction.”
Dose planning helped coordinators decide the value of conducting population-level surveys and what questions to include. If potential dose was not high enough to realistically expect to see population-level behavior changes in 3 years, they instead would focus evaluation efforts and population survey questions on proximal measures of behavior change and strategy-level evaluations that would be more informative for assessing impact. For example, they focused on measuring changes in knowledge, attitudes, and beliefs in the short term, healthier food consumption in the intermediate term, and did not focus on longer-term BMI changes, which were unlikely to move based on dose estimates of impact. For example, one coordinator from Southern California indicated the following:
When we did that first round of project planning, the first “aha” across all six projects was there was no way … we were going to … see certain population-level changes in just 1 or 2 years. It helped us face reality and not pursue an evaluation approach that was going to be foolhardy, but rather look for intermediate outcomes to measure progress in the short term while we built towards longer-term behavior and biometric changes.
Another interviewee from Colorado described how the dose framework helped to clarify the objectives of the funder at the outset:
The concept of dose helped clarify Kaiser’s objective for the grant. There are all kinds of criteria that we can use to prioritize strategies, such as sustainability or cost effectiveness, best practices. But the concept of dose was helpful in understanding what Kaiser is going to use to measure success for us.
Coordinators whose coalitions began using the dose framework after they had already implemented their strategies reflected on how the tool would have been helpful at an earlier stage. Another interviewee from Colorado indicated: “We would likely not have funded as many small isolated strategies and focused on strategies in clusters where there was momentum and greater potential for impact.”
Dose was also used as a common framework that allowed the coalitions to build consensus among their members during early stage planning. Eight coordinators described instances where dose language was helpful when explaining why a certain strategy should or should not be pursued. For example, one coordinator described how when community members supported a low-dose, high-cost strategy, discussing the concepts of reach and strength gave them a way to explain in a quantitative way, what the potential for impact was. One coordinator said,
There was a lot of pressure from the community to use dollars to develop recreational trails outside of town. I used the dose formula to demonstrate that working on bike paths and pedestrian crosswalks in town reached more of the population, which therefore increased the overall dose of the active transportation strategy, and was maybe a better investment.
When strategies that did not meet dose criteria were still backed by some, members could then ponder whether other justifications for continuing to pursue the work made sense, such as it being something the community really wants, it having broad support and backing from other funders, building capacity, or creating buy in for future more impactful work. For example, as one coordinator described,
Our coalition was really interested in school garden work. It’s expensive, doesn’t reach that many students, and doesn’t impact fruit and vegetable consumption from a dose perspective. But there was a lot of momentum here, and another partner was willing to pay half. Our hope that the school garden became the catalyst for adding other strategies down the road paid off—they added a fairly robust education curriculum to it and are getting parents involved. The garden was just the beginning of a long-term investment in healthy eating for those kids that will hopefully be higher dose and really impact their food choices.
Others described how discussing dose helped community members recognize the value of implementing strategies at multiple levels, not just at the individual level:
We were having a hard time reaching consensus with the residents and partners about activities and strategies. They wanted certain things that did not have a high dose, it didn’t have policy or environmental change. So explaining the concept of dose to them, supported why we had some activities and not others. It was a way to justify the CAP and build consensus around it.
Five interviewees noted that although the concept of population dose is helpful in building consensus, it is sometimes confusing for community partners who are not familiar with evaluation. They recommended keeping explanations as simple as possible and providing relevant examples. They also stress the importance of constantly revisiting dose concepts and reminding partners of the importance of creating sustained impact.
The concept of dose was also used by all interviewees in ongoing planning throughout the implementation of the CAP strategies. A dose lens was used to revise strategies by taking steps to increase a single strategy’s reach or strength, or by adding or removing strategies to increase the potential for overall population-level change in a target behavior. Table 1
provides the Checklist for Increasing Dose, created based on this feedback: “We needed to constantly ask ourselves: Are we touching enough lives, and is that touch strong enough to move population-level health?”
Table 1Planning Checklist for Increasing Dose
Communities often needed to expand or cluster strategies throughout the initiative to increase impact, and let go of strategies that were not impactful to focus resources elsewhere.
Nine coordinators described increasing the reach of strategies by expanding them to larger audiences. In some cases, this was accomplished by leveraging the reach of other existing programs. For example, one coalition expanded the reach of a community garden by partnering with an after-school program: The garden used to provide individual garden plots for a limited number of families; now students in the after-school program come to the garden to learn about growing food, and fresh produce from the garden is distributed to families of the students in the program twice a week. Another coalition moved the location of their farmers market: “There is a lot of momentum around a farmers’ market in town, but we are struggling to get enough shoppers to sustain the market at the first location. We are moving the market to the hospital where more of the target population will have access to the stand.”
Nine coordinators cited increasing strategy strength through changes in implementation to increase frequency, intensity, or duration of a strategy. For example, one coordinator from Northern California described how they reviewed data with school partners to show them how better implementation would result in a stronger impact:
It’s particularly helpful when we review the data with principals each summer and plan for the next school year’s [HEAL] work. It helps them see where changes are occurring, and where they can put resources into encouraging change…we show the schools total minutes of [physical activity] increased as a result of their work, and decreased when teachers stop implementation as faithfully or didn’t have sufficient resources to keep it up.
Another coordinator described how their coalition added new components to a healthy eating strategy in order to increase strength. At first, they were only providing technical assistance to store owners to help them offer more fresh produce. They decided to add food demonstrations, tasting events, and marketing materials for healthy meal options to increase the likelihood that customers would purchase the healthy food. Additionally, to sustain these changes over time, they provided training to store employees on produce handling and storage.
Ten of 11 coordinators said that thinking about the concepts of reach and strength led them to add new strategies to clusters of existing strategies targeting the same behavioral outcome. For example, one coordinator described how during implementation, their coalition identified the elementary school as an additional intervention setting that needed to be included to better reach their target population:
[Engaging schools was] a better way to reach families than we had originally thought. When we were writing the grant, we did not have strong ties to the elementary school, but we realized we needed to work on that… . It was a way to clarify who we are targeting in the community, how we can increase some of our numbers but also look at the overlap in service and collective impact.
Another coordinator described how seeing strategies to increase active transportation to school encouraged them to build on the momentum and add more strategies:
We could see from our teacher tallies that kids were walking/biking to and from school more, and the biggest complaint was from kids who lived too far away for it to be practical. So we worked with the district and parents to add a walking school bus where bus riding kids are dropped off a half mile from school and walk with a parent chaperone the rest of the way. It was a fun activity that was a win–win for parents and kids.
An equally important implementation tactic, cited by all 11 interviewees, was removing strategies for reasons such as too resource intensive, lack of momentum, and very low dose without having other benefits. For example, one coordinator mentioned the following:
[We] purchased a building to host Local Food by Donation dinners. There was momentum in the beginning, but the strategy was not generating enough income to sustain the work. We will repurpose the space into an indoor farmers’ market … for two reasons: (1) the lack of data around Local Food by Donation dinner’s ability to change behavior and (2) the increased potential strength and sustainability.
Dropping strategies is extremely common as coalitions begin to implement their plans. Communities start with ten to 30 potential strategies and then typically drop more than half of them throughout their grant. These decisions free up resources to spend elsewhere on higher dose strategies. CAPs usually have six to ten sustained strategies by the end of their funding cycle.
One coordinator indicated: “We have primarily used [dose] when we update our CAP every year… . By eliminating strategies or components of strategies that aren’t being impactful we can focus more on strategies that can get us to a higher dose.”
In 2015, Kaiser Permanente funded six communities in southwest Washington and Oregon to implement place-based HEAL interventions. Local public health departments led four community initiatives, and nonprofit organizations led two others.
The population dose concept was used from the outset for planning purposes. In the request for proposals, prospective grantees were asked to consider implementing strategies across multiple levels and sectors using the dose lens. Proposal reviewers noted that although some grantees used dose terminology, the concept was neither well understood yet nor was dose being used meaningfully to develop strategies. Soon after funding was awarded, Kaiser Permanente established a HEAL Support Team composed of Kaiser Permanente subject experts, coordinators, and evaluators to support grantees implementing and evaluating their strategies. This Team supported the coalition by providing technical assistance, resources, and information regarding planning and implementation of strategies. The HEAL Support Team presented a high-level overview of dose to each community, including examples and the Center for Community Health and Evaluation’s online toolkit.
- Glasgow R.
- Vogt T.
- Boles S.
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
They also created an online repository of dose-related resources for community partners. Community partners worked with the HEAL Support Team to create a project plan that clarified target HEAL behaviors, and included the estimated reach and strength of multiple strategies.
Population dose was used for consensus building, particularly during the early stages. The ongoing dialogue between community partners and the HEAL Support Team improved understanding and meaningful use of the dose concept. For example, one community had originally proposed to implement a strategy promoting healthy eating for high school students. When reviewed through a dose lens, it became clear that the proposed strategy involved only an assessment, lacking strength to actually impact the desired outcome. This insight galvanized community members to brainstorm outside the box and pursue different strategies within and outside the school environments, and agree upon a final strategy that both worked for the community partners and had a good potential for impact.
Population dose was also used during implementation. After a year, the HEAL Support Team held strategic reflection sessions to understand progress, current reach/strength, and update plans for Year 2. The HEAL Support Team discussed implications of these findings with communities; in some cases, encouraging sites to make significant adjustments to their strategies based on dose estimations and projections. For example, one community decided to partner with the city planning department and supplement two programmatic strategies with broader environmental strategies in the interest of increasing overall dose and sustainability.
The HEAL Support Team has found that one of the biggest strengths of the dose framework is having language to help partners conceptualize the overall actual and potential impact of their work. Although all communities ground their work in a logic model approach, dose helps concretely quantify how to get from activities to outcomes, and what success would look like for outcomes. A coordinator said,
I have been working in place-based community initiative work for a long time, and I thought there is no way we are ever going to make population behavior changes because what we are doing is so weak. I had already been seeing that as an evaluator, and to have a way to talk about it and tools to deal with it, I think that’s really the beauty of dose … I think it is a really needed concept, helping people think about that during the design and implementation process.
The HEAL Support Team encountered some challenges in using dose with community partners. When dose was first introduced, some community partners were concerned that the framework would prevent consideration of community engagement and cultural appropriateness. The HEAL Support Team stressed that dose is one of several guiding principles in strategy design. Additionally, some community partners became discouraged when they discovered that many of their proposed strategies had a low dose. Community partners were encouraged to not only think about ways to increase the dose of individual strategies, but also to think about the synergistic effect of strategy clusters, and were also encouraged to think of dose as one of many lenses to view impact—as strategies often have impact above and beyond behavior change (social networking, environmental change, capacity building, and more).
Overall, the HEAL Support Team found the population dose framework to be an effective tool for planning and implementation with community-based initiatives. Accessible informational resources, project planning templates, and ongoing dialogue between the HEAL Support Team and community partners were key elements in this process.
Based on the data from interviews and lessons learned in the case study, a Dose for Planning Checklist was created (Table 1
, also freely available from the dose toolkit online).
This checklist can help guide community coordinators and evaluators in decision making throughout the development and implementation of their initiatives, to increase the reach, strength, and overall impact of their strategies.