If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to: Allen Cheadle, PhD, Center for Community Health and Evaluation, Kaiser Permanente Washington Health Research Institute, Seattle WA 98101.
A growing number of health systems are leading health promotion efforts in their wider communities. What impact are these efforts having on health behaviors and ultimately health status? This paper presents evaluation results from the place-based Kaiser Permanente Healthy Eating Active Living Zones obesity prevention initiative, implemented in 2011–2015 in 12 low-income communities in Kaiser Permanente’s Northern and Southern California Regions.
Methods
The Healthy Eating Active Living Zones design targeted places and people through policy, environmental, and programmatic strategies. Each Healthy Eating Active Living Zone is a small, low-income community of 10,000 to 20,000 residents with high obesity rates and other health disparities. Community coalitions planned and implemented strategies in each community. A population-dose approach and pre and post surveys were used to assess impact of policy, program, and environmental change strategies; the analysis was conducted in 2016. Population dose is the product of reach (number of people affected by a strategy divided by target population size) and strength (the effect size or relative change in behavior for each person exposed to the strategy).
Results
More than 230 community change strategies were implemented over 3 years, encompassing policy, environmental, and programmatic changes as well as efforts to build community capacity to sustain strategies and make changes in the future. Positive population-level results were seen for higher-dose strategies, particularly those targeting youth physical activity. Higher-dose strategies were more likely to be found in communities with the longest duration of investment.
Conclusions
These results demonstrate that strong (high-dose), community-based obesity prevention strategies can lead to improved health behaviors, particularly among youth in school settings.
Supplement information
This article is part of a supplement entitled Building Thriving Communities Through Comprehensive Community Health Initiatives, which is sponsored by Kaiser Permanente, Community Health.
Introduction
A growing number of health systems are leading efforts to promote health in the wider communities they serve.
and a recognition that there are potential savings in healthcare utilization and costs by promoting healthier environments and more health-promoting policies. The obesity epidemic is one such health issue that can benefit from a comprehensive, community-level approach, using a portfolio of strategies at multiple levels (e.g., individual, family, community) across multiple sectors (e.g., school, worksite, neighborhood) following the socioecologic model or some modification of that framework.
A comprehensive approach recognizes that the epidemic is driven by many complex and interrelated factors; food and physical activity behaviors and weight gain are influenced by cultural, economic, social, genetic, and environmental influences that are hard to disentangle and even harder to change.
Population-based prevention of obesity: the need for comprehensive promotion of healthful eating, physical activity, and energy balance: a scientific statement from American Heart Association Council on Epidemiology and Prevention, Interdisciplinary Committee for Prevention (formerly the expert panel on population and prevention science).
This emphasis on a comprehensive or multicomponent approach (i.e., multilevel, multisector) has shaped several obesity prevention initiatives, including the W.K. Kellogg Foundation’s Food and Fitness Initiative,
Published studies assessing the impact of comprehensive community approaches have been mixed. A recent “review of reviews” article found evidence of impact among young children and the school setting and inconclusive evidence among adults and the general community setting.
Other studies looking at multicomponent initiatives targeting entire communities (i.e., including both youth and adults) found some promising results for these approaches, but noted a difficulty in determining the specific strategies or combined effect of strategies that work.
A Cochrane review focusing on community-wide physical interventions found few credible studies showing impact at the population level, noting that this may be because of the difficulty reaching enough people in a community setting.
This article presents evaluation results from a multicomponent initiative targeting both youth and adults: the Kaiser Permanente (KP) Healthy Eating Active Living (HEAL) Zones initiative, implemented in 2011–2015 in 12 low-income communities in two KP regions: Northern California (NCAL) and Southern California (SCAL). The concept of population dose is used to assess the impact of the initiative on the food and physical activity behaviors of both adults and youth in the communities; the analysis was conducted in 2016.
Methods
Program Description
KP is a nonprofit integrated healthcare delivery system that serves ≅12 million members in eight states and the District of Columbia. In 2003, KP’s Community Benefit program created a multisector place-based program—the Community Health Initiative (CHI)—to promote obesity prevention policy and environmental change in communities served by KP.
To date, CHI programs have been implemented in nearly 60 communities in five KP regions. Overarching CHI principles include a place-based focus; an emphasis on change at multiple levels, particularly environmental and policy change; a multisectoral collaboration that involves health care, neighborhood, schools, and work sites; and community engagement and community ownership.
This paper focuses on CHI programs in two KP regions, NCAL and SCAL, whose design and evaluations were closely aligned. The two regional initiatives share some common features described in the next section along with the ways in which they differ.
For both regions, the HEAL Zones design targeted places and people through policy, environmental, and programmatic strategies. Examples of these strategies include implementing a new physical education curricula in kindergarten to 12th-grade schools or installing a lighted walking trail to provide access to safe physical activity. Each site or HEAL Zone is a small community of 10,000 to 20,000 residents with high obesity rates and other health disparities, such as in rates of diabetes and cardiovascular disease. Community coalitions planned and implemented strategies in each community; coalition members included schools and school districts, community-based organizations, employers, local businesses, local governmental agencies, faith-based organizations, healthcare providers, and residents. The initiative evaluator supported local evaluation efforts and provided guidance on the HEAL framework as well as strategy development and improvement.
The most important difference between the two regional initiatives was the duration of investment. That duration was significantly longer in NCAL where KP has implemented two large-scale initiatives since 2006: (1) the HEAL CHI initiative from 2006 to 2010, in three communities
; and (2) the HEAL Zones from 2011 to 2014, in six communities, including two of the original three HEAL CHI communities that received funding from 2006 to 2014. Although KP made some adjustments in approach between HEAL CHI and HEAL Zones, they kept core elements consistent across initiatives. By contrast, the SCAL initiative was launched more recently—in January 2012—with the first phase ending in March 2016 (a second 3-year phase is currently underway).
The other regional difference was that in SCAL, a more formal support structure was put in place to support the building of effective coalitions. In particular, an outside technical assistance provider supported coalition development and functioning, and facilitated connections to needed resources. They also supported cross-site learning through activities and events. In NCAL, a separate technical assistant provider was not included in the HEAL Zones initiative, and support for the coalitions was provided largely by KP staff.
Evaluation Design
The evaluation goals were to describe implementation progress, document population-level impact, and contribute to program improvement. The evaluators worked in close collaboration with program staff and technical assistance partners. IRB approval for the evaluation was obtained from KP Washington Health Research Institute (formerly Group Health Research Institute) and University of California Davis (and Berkeley) IRB. Key evaluation questions included the following: Were the HEAL Zones strategies implemented? What was the impact of implemented strategies on the people touched by them? Were strategies implemented with sufficient dose to expect population-level impact? Did population-level change occur and was it attributable to the initiative?
A logic model design was used to answer these questions, particularly those related to attribution of population-level change. Logic model designs
create indicators for each step in the process by which an initiative is intended to achieve its outcomes and then examine whether the temporal pattern of change is consistent with what is specified in the model. The key steps in the HEAL Zones logic model were creating coalitions and implementing environmental and policy changes in communities, which were then expected to lead to population-level outcomes (e.g., physical activity behaviors). The concept of population dose (or simply dose)
was used to examine the link in the logic model between strategy implementation and population-level change. Dose is defined as the product of reach (exposure or number of people touched divided by target population size) and strength (the effect size or relative change in behavior for each exposed person). The dose of clusters of environmental and policy change strategies in a given health target area were estimated (e.g., youth physical activity) and then the association between higher-dose strategies and measured population-level changes was examined (e.g., self-reported minutes of physical activity among youth).
Measures
Table 1 lists the primary data sources used in the HEAL Zones evaluation. Strategy status and implementation details were gathered during monthly check-in calls where the HEAL Zone project coordinator provided an update on the status of each strategy and described recent accomplishments and challenges. Additionally, key informant interviews and Photovoice
with a sample of community residents were conducted to gather the community perspective on community impacts. Results from the key informant interviews focusing on the role of the SCAL community coalitions in the initiative will be presented in a separate paper.
Table 1HEAL Zones Evaluation Data Sources
Method
Description
Strategy tracking database
•
Monthly check-in calls with site coordinators to gather information on strategy status, accomplishments, reach, potential sustainability
•
Information used to populate a strategy-tracking database: Documentation of Community Change (DOCC)
Adult surveys
•
Self-administered mail survey with phone follow-up of non-responders
•
Sample: 1,300 randomly selected HEAL Zones residents and a comparison group of 1,300 residents from throughout KP’s Southern California region
•
Survey questions: Food and physical activity behaviors; attitudes about, and social support for, healthy eating and active living; demographic information
Youth surveys
•
Self-administered survey
•
Sample: fifth- and seventh-grade students in elementary and middle schools where HEAL work was taking place
•
Survey questions: Attitudes and behaviors related to food and physical activity and perceptions of school and community environments
Key informant interviews
•
Semi-structured telephone interviews
•
Sample: Partner organizations and residents involved in HEAL Zones work
•
Questions: Project and coalition successes and challenges, perceptions of community changes, feedback on the support provided by technical assistance provider
Coalition member surveys
•
Online or in-person self-administered survey
•
Sample: All coalition members
•
Questions: Coalition functioning including items related to shared vision and planning, community participation, decision-making, leadership, sense of community, and sustainability
Teacher surveys
•
Online survey
•
Sample: All teachers in HEAL Zones schools
•
Questions: Status of nutrition and physical activity in their school and their own physical health
Photovoice
•
Participants used photos and captions to identify, from their perspective, the most significant accomplishments from the initiative at both baseline and follow-up
Strategy-level evaluations
•
Variety of program evaluation methods used to capture impact of individual strategies on health behavior, including surveys, observation, and school and clinic records (Table 4 provides details)
HEAL, Healthy Eating Active Living; KP, Kaiser Permanente.
Population-level impact was tracked using self-administered and phone surveys of youth and adults living in the HEAL Zones. Survey questions were drawn from existing validated instruments (e.g., the Youth Risk Behavior Surveillance Survey
), supplemented by questions about the school and neighborhood environment. NCAL schools were surveyed in 2012 and 2015 (2012: n=1,745, response rate=83%; and 2015: n=1,733, response rate=91%). SCAL schools were surveyed in 2013 and 2016 (2013: n=2,932, response rate=94%; and 2016: n=2,739, response rate=90%).
The adult survey instrument included questions drawn from the Behavioral Risk Factor Surveillance System Survey
related to food and physical activity, along with demographic information and questions about social support attitudes for HEAL. NCAL adults were surveyed in 2012 and 2015 (2012: n=1,867, response rate=19.5%; and 2015: n=1,394, response rate=14.5%). SCAL adults were surveyed in 2013 and 2016 (2013: n=1,685, response rate=18.1%; and 2016: n=1,436, response rate=15.4%).
Another key element in both regions was evaluating the impact of individual strategies on health behaviors of people exposed to each strategy. At the beginning of the initiative, evaluators met with coordinators from each site to choose two to three strategies to investigate more deeply based on evaluation feasibility/cost and impact strategies were likely to have on health behaviors if fully implemented. Strategy-level evaluation activities were typically done in collaboration with the sites; methods included surveys of people interacting with changing environments (e.g., shoppers in grocery stores), observation of activity levels in parks and schools, and records from clinics and cafeterias. NCAL strategy-level evaluation results in corner stores and early childhood settings are presented in separate papers in this issue.
Strategy implementation data were converted into reach, strength, and dose estimates and then combined with the population-level survey data to examine associations between dose and population-level behavior change. This paper presents a high-level summary of the analytic approach; more details, including examples of the dose rating process, are published in an online dose toolkit.
Dose is defined as the product of reach (exposure or number of people touched divided by target population size) and strength (the effect size or relative change in behavior for each exposed person). Reach is the proportion of people in a community who are touched by a strategy (e.g., the number of participants in an exercise program divided by the community population). For environmental changes, the community population (or numerator) is the number of people who encounter the improved environment on a regular basis and are assumed to be influenced by it (e.g., the number of residents living near a newly renovated park or playground).
Where possible, estimates of strategy strength were obtained from published research or from evaluations conducted of particular strategies; an example is using student surveys to assess the impact of Safe Routes to School programs that encourage walking and biking to school. When these sources were unavailable, strength ratings were based on frequency of exposure; intensity of exposure (e.g., the magnitude of an environmental change); and the degree to which the healthy choice is the only choice (e.g., removing all unhealthy snacks from a closed-campus school versus adding a few healthy snacks but leaving the unhealthy ones in place).
Strategies were grouped into four strength categories for dose calculations. Minimal-strength strategies included changes with low frequency/intensity and limited or no evidence of effectiveness (e.g., general health education/promotion, walking trails that were minimally used; based on observations). Low-strength strategies were also low-frequency/intensity but had some evaluation evidence showing effect sizes of about 2% (e.g., Safe Routes to Schools programs). Medium-strength strategies were programs and environmental strategies with greater intensity/frequency of exposure, with some evidence of effect sizes in the 5% range (e.g., active recess programs in elementary schools). A rating of high strength was given only to evidence-based strategies with demonstrated effect sizes in the range of ≥10% (e.g., evidence-based school physical education curricula). Multiple independent raters were used to standardize the ratings as much as possible (see dose toolkit for more details on the ratings of specific strategies
After computing the dose of the individual strategies, strategies among youth or adults targeting similar outcomes (e.g., minutes of physical activity) were combined into dose clusters approximating the combined impact of multiple strategies. Table 2 illustrates the dose calculations for youth physical activity strategies in one HEAL Zone community.
Table 2Example of a Community-Level Dose Cluster for Youth Physical Activity Strategies
Dose clusters were computed for each of the 12 HEAL Zone communities (six NCAL, six SCAL) across four health target areas and two populations (a total of 96 clusters). The health target areas were physical activity, sugar-sweetened beverage (SSB) consumption, fruit and vegetable consumption, and healthy food—a general category including examples such as reduced-calorie snacks in schools and junk food removal. The two populations were youth (school-aged children ages 5–18 years) and community (i.e., the community as a whole as measured by surveying adults >18 years). The populations overlap somewhat because some community strategies can impact the entire family (e.g., if healthier food is purchased). Overlap was avoided whenever possible; an example is that community strategies primarily targeting kindergarten to 12th-grade youth (e.g., recreational center after school physical activity programs) were assigned to youth.
Impact was attributed to the HEAL Zones if higher-dose clusters of strategies were associated with a greater frequency of positive population-level changes. Positive population-level change was defined as change in the desired direction that was either statistically significant (p<0.05) or trending toward significant (p<0.10) with an absolute percentage behavior change of >5% (note: trending criteria were applied because fifth-grade survey data had small sample sizes and corresponding limited power). State and national trends were adjusted for using either comparison data collected by the CHI evaluation team (for the adult survey) or other secondary data sources; for example, the Youth Risk Behavior Surveillance Survey.
For the two longer-duration sites in NCAL, there was concern about missing the baseline in the population-level analysis by using pre–post results from 2011 to 2014, when the sites had begun work in 2006. For physical activity only, information was added from a previous pre–post assessment of changes in physical activity
to approximate impact across the entire 8-year period. (Note: The evaluation of the 2006–2010 period used different survey instruments and methods so that data could not be directly combined.) For longer duration sites, a positive population change was assigned over the 8-year period if (1) there was a significant increase in physical activity from 2006 to 2010; (2) the baseline level of physical activity in 2011 was significantly greater than the overall baseline for all sites combined; and (3) the pre–post measure of physical activity showed improvement or stayed the same from 2011 to 2014.
If dose computations for a cluster of strategies are based on accurate reach and strength estimates, they will approximate the population-level changes expected to result from implementing those strategies. However, in practice it is difficult to accurately estimate strength, so rather than attempting to correlate dose with population change, the strategies were divided into two categories—higher and lower dose—and the proportion of positive population-level changes was examined within each group. A cutpoint of 2% was used that corresponds to reaching everyone in a community or school with a low-strength strategy. For population-level initiatives, a 2% change in key risk behaviors, such as diet and physical activity, can be clinically significant in that the long-term change in downstream conditions, such as diabetes and cardiovascular disease, can affect a significant number of community residents (see simulation models projecting significant long-term changes in health outcomes from relatively modest behavior change interventions: e.g., Gortmaker et al.
The results confirm the emphasis of the initiative on environmental and policy changes: 93 strategies (40% of the total) involved organizational policy change; 34 (15%) environmental change; and 14 (6%) public policy. Communities also implemented 29 (12%) programmatic strategies and 40 (17%) community capacity-building strategies.
Table 3HEAL Zones Strategies by Levels of the Socio-Ecological Model
Percentage of total and total number of strategies in that category are in parentheses; the total number of strategies across all 12 HEAL Zone communities was 233.
•
Adopt a policy removing flavored milk from school breakfast in elementary school
•
Make changes to worksite physical activity environments, including point of decision prompts in stairwells
•
Implement new physical education curricula in K–12 schools
•
Develop protocol for clinic staff to support patients with healthy eating and active living
Programs (12%; n=29)
•
Increase physical activity programs in community centers
•
Create a promotora program to do community HEAL education
Environmental change (15%; n=34)
•
Install a lighted walking trail to provide access to safe physical activity
•
Improve/build parks/physical activity areas that are safe and accessible
•
Implement the “Good For Kids” restaurant certification program
•
Increase purchase or distribution points for fresh fruits and vegetables in the community
Public policy (6%; n=14)
•
Work to implement a city soda tax initiative
•
Impact the urban planning via the city general plans
•
Implement city policies to promote school joint use agreements and Safe Routes to School programs
Media campaigns, promotion (10%; n=23)
•
Rethink Your Drink health education campaigns to discourage sugar-sweetened beverage consumption
These are more generic capacity building strategies (e.g., community building activities), not directly related to obesity policies and programs. HEAL, Healthy Eating Active Living.
(17%; n=40)
•
Implement a Resident Leadership and Advocacy Program
•
Establish active neighborhood groups and a youth council
•
Build worksite sector leadership and infrastructure
a Percentage of total and total number of strategies in that category are in parentheses; the total number of strategies across all 12 HEAL Zone communities was 233.
b These are more generic capacity building strategies (e.g., community building activities), not directly related to obesity policies and programs.HEAL, Healthy Eating Active Living.
Table 4 summarizes the SCAL strategy-level evaluations examining the impact of implemented strategies on those directly touched by them (i.e., participants in programs and people exposed to policy, program, and environmental changes). Strategies around active recess and SSBs showed the greatest impact on behavior; and clinic HEAL prescription strategies and environmental improvements in child care settings also showed promising results. For other strategies implemented in parks, grocery stores, and restaurants, the impact was mixed or negative. NCAL strategy-level evaluation results included in other papers in this issue showed mixed findings for corner store strategies
Table 4Summary of Southern California Strategy-Level Evaluation Results
Strategy
Description of activities
Evaluation methods
Results
Active recess
Parent volunteers, teachers, or YMCA staff were trained to help engage students in steady physical activity during daytime school recess periods
Pre–post observation of student activity levels during recess
•
Increases in moderate or vigorous physical activity (MVPA) during recess in 3 of 4 communities
•
Pre- to post-MVPA increases ranged from 17% to 19%
Flavored milk removal
Community 1 removed flavored milk during all meals at all 24 schools in the district. Community 2 removed flavored milk during breakfast at all 10 schools in their district
Cafeteria records showing consumption of milk
•
Children replaced the flavored milk with nonfat or 1% regular milk, with no overall decrease in milk consumption
Child care
In Community 1, 24 small early childhood care sites worked with a YMCA Health Educator to increase healthy offerings. In Community 2, one center serving 144 children was certified as a healthy site, by offering more fruits and vegetables and making other health-promoting changes
Pre–post self-assessment using the NAPSACC – Nutrition and Physical Activity Self-Assessment for Child Care instrument (Community B only)
Results for Community 1 showed modest improvements:
•
Mean number of healthy food items served per day increased by one
•
Percentage of sites providing more than 60 minutes of physical activity increased from 56% to 71%
•
Two sites were serving flavored milk at pre; none at post
Parks
Five of the six HEAL Zones added trails and new equipment to parks. Some sites began promoting the new equipment/trails through exercise programs
Pre–post observation of activity levels among park users
•
No overall increase in intensity of exercise among park users
•
15% of park users at one site were observed using new exercise equipment
Clinics
Clinics in three HEAL Zones developed and began using a HEAL prescription with patients. In several locations, additional educational and promotional strategies were provided along with the prescription
Clinic electronic health record data used to track uptake of BMI screening and follow-up discussions
Significant uptake in the 3 clinics:
•
Clinic 1: Increase from 42% to 82% of BMI screenings
•
Clinic 2: HEAL prescription given in 70% of encounters which included fruit and vegetable vouchers
•
Clinic 3: HEAL prescription written in 90% of encounters which included PA incentives (e.g., soccer balls)
Corner stores
Store environmental changes included refrigeration units, promotional materials, healthy food labels
Pre–post shopper intercept surveys
•
Increases in awareness of healthy labeling
•
No significant increases in purchasing fruits and vegetables or other healthy items
Restaurants
In Community 1, ten restaurants created healthy menus for families and youth. In Community 2, three restaurants created healthy menus for youth
Tracking of healthy menus, restaurant participation
•
After 6 months, only three of ten Community 1 restaurants still made the healthy menu available. The strategy was discontinued in Community 2
HEAL, Healthy Eating Active Living; PA, physical activity.
Modest positive results were found for population changes from the youth and adult surveys across the entire initiative (data not shown). Of the 92 pre and post comparisons (Note: one community in SCAL did not do the school surveys, so there were 44 comparisons in SCAL, not 48), just over one fifth (n=20, 22%) were positive—either statistically significant (p<0.05, n=11) or trending (p<0.10, n=9)—and 7 (8%) were negative and significant. Thus, there were only slightly more statistically significant (p<0.05) positive pre and post changes versus negative (11 vs 7).
Given the logic model approach, the key question was, Were positive population-level changes associated with higher-dose strategies? As shown in Figure 1, among the higher-dose strategy clusters, there were positive population changes in six of nine cases (67%) versus in only 14 of 83 cases (17%) for lower-dose strategy clusters (p<0.01 for the difference in proportions). All of these six dose-aligned improvements occurred among youth in schools and four of those were in the area of physical activity.
Figure 1Distribution of HEAL Zone strategy clusters by dose and population-level change.
Note: Percentages show the percent of strategies in each dose category (higher or lower dose) with positive population-level change and negative or non-significant population-level change. Total number of strategy clusters=92 (9 higher-dose, 83 lower-dose).
As the counts in Figure 1 show, just 10% (9/92) of the strategy clusters were of sufficient dose to reach the 2% higher-dose threshold (all were in schools). Notably, the two longer-duration communities in NCAL had the highest proportion of higher-dose clusters: six of 16 (38%) of their strategies were higher dose compared with just three of 76 (4%) for the other ten communities.
Discussion
A population dose approach was used to assess the impact of the HEAL Zones initiative, a multicomponent obesity prevention initiative implemented by KP in its NCAL and SCAL regions. More than 230 community change strategies were implemented over 3 years, encompassing policy, environmental, and programmatic changes as well as efforts to build community capacity to sustain strategies and make changes in the future. Positive population-level results were seen for higher-dose strategies, particularly those targeting youth physical activity.
Of the 14 (of 20) positive population-level changes that were not accompanied by higher-dose strategies, seven were in SSB consumption. It is likely that at least some of this decrease in consumption can be attributed to the significant downward time trends both at the state level and nationally during the period of the initiative.
suggests that SSB consumption may be more sensitive than other health behaviors to modest, educational messaging. In the end, given the uncertainty about the relative contribution of the HEAL Zones strategies, the conclusion drawn was that the strategies may have played a role in the observed changes in SSB consumption.
Despite the relatively low threshold for higher dose (2%), only 10% of the HEAL Zone strategy clusters met or exceeded this threshold. Furthermore, of the 83 low-dose clusters 78 (94%) were less than 1% and 60 (72%) were less than 0.5%. The community strategies were particularly low dose, with 88% of all community strategy clusters having less than 0.5% dose (compared with 38% in schools). One explanation for these modest implementation results is the relatively short (3-year) time frame for most of the communities participating in the initiative. It typically took 1 to 2 years to implement some of the institutional changes required by the action plans, and reach may increase over time as institutional changes become more established. This may explain why the longer-duration NCAL sites had significantly more strategy clusters that were higher dose than the other ten communities (38% vs 4%).
The findings showing the greatest impact among youth, particularly with school-based strategies, are broadly consistent with the existing literature. As noted in the introduction, multicomponent initiatives targeting youth have been the most frequently reported as successful, beginning with the school and community initiative in Somerville, Massachussetts.
The evaluation of the HEAL CHI initiative—the predecessor to HEAL Zones—showed positive results only for youth physical activity using a similar dose methodology.
The finding on physical activity is consistent with a recent systematic review that rated physical activity interventions in schools as having a high strength of evidence.
showing that implementing nutrition standards in schools is both evidence-based and cost effective.
There are a number of good underlying reasons for both targeting school-aged children and using school-focused strategies. Children, especially in elementary school, are a captive population with more limited food choices while in school and there is greater opportunity for in-class and recess physical activity. It is also relatively easier to make changes in school building policies, practices, and the environment that can impact all or most children.
The results showing low-dose strategies and little population-level change for community-level strategies targeting adults are also consistent with the existing literature.
It is challenging to implement community-wide interventions for many reasons. Community settings are more fragmented and diverse, making it harder to reach large segments of the population through single strategies and harder to implement high-strength strategies. Adult residents typically interact with environments outside the community setting (transit to work, shop across town for groceries). For example, community physical activity programs may significantly increase activity among participants, but it is challenging to get widespread, consistent, and frequent participation in these programs by a large number of residents from a community. Finally, many higher-reach environmental strategies, such as in corner stores, have a modest effect on those exposed because so many alternative food options exist within and outside the HEAL Zone.
In addition to providing a lens for assessing initiative impact, the dose concept proved useful in the program planning and implementation. During their planning phase, many of the HEAL Zone community collaboratives initially proposed more programmatic strategies that reached small segments of their population. Through conversations with evaluators, funders, and community coordinators and their coalition members, many incorporated the concept of dose and revised their plans with strategies that were higher reach and likely to have a more significant impact (strength) on those reached. For example, communities began to partner with institutions (e.g., schools, worksites, grocery stores) that have broad reach in their communities and significant daily interactions with residents (e.g., youth spending 6 hours per day in schools).
Limitations
Implementation tracking relied largely on progress reporting from the community collaboratives and other institutions involved (e.g., schools, worksites). These self-reported accomplishments may have been biased in favor of making changes appear to be more comprehensive and sustainable than was true in practice. Where possible, progress reporting was corroborated with secondary data, such as strategy-level evaluations involving direct observation and environmental assessments.
Ratings of the strength component of population dose were often necessarily subjective given the lack of information in the scientific literature or from strategy-level evaluations about effect sizes for CHI environmental and policy interventions. Multiple independent raters were used to attempt to standardize the ratings as much as possible, and a sensitivity analysis was conducted to explore the impact of potential misclassification on the final results. In addition, a number of strategy-level evaluations were conducted to estimate effect sizes and these are being used to further refine and validate the ratings.
Response rates to the mail survey of adults were relatively low—in the 15%–20% range. Because rates were comparable pre- and post-survey it is plausible that roughly the same biases applied on both occasions and that the changes were estimated accurately. Finally, a true baseline was not available for two of the NCAL communities that began work in 2006, and triangulation of several data sources was used to estimate the impact.
Quantifying the cumulative dose of strategies targeting the same outcomes requires assumptions about how overlapping strategies interact within a population. For example, is interaction occurring and are the combined effects of healthier food availability in corner stores, restaurants, and worksites resulting in a greater healthy eating increase than expected from the effect of each strategy alone? Simply adding up dose may not account for a potential larger impact as strategies reinforce each other. Conversely, does adding up dose overestimate the potential impact of multiple strategies that exceeds the level of behavior change expected to reasonably achieve in a population?
A final set of limitations concerns the population surveys, particularly the youth surveys in elementary schools (fifth grade). These data were collected in elementary schools because most of the school strategies were taking place in these schools, but some young children may report inaccurate recall responses. Furthermore, fifth-grade classes had relatively small sample sizes compared with seventh and ninth grades. These smaller sample sizes limited the power to detect small changes in population outcomes that would be reasonably expected given the dose of the strategies.
Despite the limitations, the dose methodology provides a useful lens for interpreting population-level findings and determining attribution to the initiative. In particular, the approach can rule out chance findings in population-level change by requiring that observed positive changes are accompanied by high-dose strategies.
Conclusions
The HEAL Zones evaluation results show that the initiatives in the two KP regions were generally successful in achieving their goals of implementing policy and environmental change strategies that have the potential to change behaviors and impact long-term obesity rates. The population-level results showed that where higher-dose strategies were present, population-level improvements occurred. The HEAL Zones initiative represents a successful effort by a large health care system to influence the health-promoting conditions in the communities they serve. The initiative addressed a need prioritized in many of the KP community health needs assessments, and has the potential to reduce long-term health care costs for both members and non-members living in the communities served by KP.
Acknowledgments
This project was funded by Kaiser Permanente. The authors acknowledge and thank Kaiser Permanente for financial support.
Cheadle contributed to the design and implementation of the Healthy Eating Active Living (HEAL) Zones evaluation and wrote the first draft of the paper. Atiedu, Keene, and Molina designed and carried out the Southern California HEAL Zones evaluation, contributed to sections of the paper, and reviewed and made edits on draft versions. Davoudi, Lee, Steimberg, Tinajero, and Azuma designed and implemented the Southern California HEAL Zones intervention, contributed to the design of the evaluation, and reviewed and made edits on draft versions of the paper. Spring contributed to the design and carried out the evaluation of the Northern California HEAL Zones intervention, and made edits on draft versions of the paper. Boyle, Williamson, Ravel, and Nudelman designed and implemented the Northern California HEAL Zones intervention, contributed to the design of the evaluation, and reviewed and made edits on draft versions of the paper. Rauzon, Schwartz, Kuo, and Solomon contributed to the design of the evaluation, and reviewed and made edits on draft versions of the paper.
No financial disclosures were reported by the authors of this paper.
Supplement Note
This article is part of a supplement entitled Building Thriving Communities Through Comprehensive Community Health Initiatives: Evaluations from 10 Years of Kaiser Permanente's Community Health Initiative to Promote Healthy Eating and Active Living, which is sponsored by Kaiser Permanente, Community Health.
References
Prybil L.
Scutchfield F.
Killian R.
et al.
Improving Community Health Through Hospital-Public Health Collaboration.
Commonwealth Center for Governance Studies,
Lexington, KY2014
Population-based prevention of obesity: the need for comprehensive promotion of healthful eating, physical activity, and energy balance: a scientific statement from American Heart Association Council on Epidemiology and Prevention, Interdisciplinary Committee for Prevention (formerly the expert panel on population and prevention science).
30056-4&id=gr1.jpg){kind=link}