Advertisement

Effectiveness of a Multimodal Online Well-Being Intervention

A Randomized Controlled Trial

      Background

      Well-being encompasses physical, psychological, and social aspects of health and predicts healthcare utilization and expenditures. Despite their potential clinical impact, interventions that leverage social network effects to target well-being are uncommon.

      Purpose

      Using a pragmatic design, to evaluate the effectiveness of an online well-being intervention as part of ongoing program development.

      Design

      Randomized, placebo-controlled, parallel-group trial with longitudinal outcome measurements at baseline, 30 days, and 90 days.

      Participants/setting

      A total of 1503 U.S.-based adults were enrolled. Recruitment, eligibility verification, and baseline data collection were conducted entirely online; follow-up took place online or by phone. The study was conducted in 2012.

      Intervention

      A multimodal e-mail-, web-, and mobile-based intervention (Daily Challenge), in which participants receive daily suggestions of small health actions that they complete in a social environment. A traditional weekly health newsletter served as control.

      Main outcome measure

      Overall well-being as measured by the Individual-level Well-Being Assessment and Scoring Method (scale: 0 to 100).

      Results

      Follow-up rates reached 68.7% (n=1032) at 30 days and 62.6% (n=940) at 90 days. Overall, 84.6% of treatment group participants visited the website, and 76.5% opened program e-mails (vs 51.1% in the control group). Daily Challenge improved well-being significantly more than control at 30 days (2.27 points, p=0.004) and at 90 days (2.35 points, p=0.004). A dose response for intensity of use was observed at 30 days (p=0.001) and 90 days (p=0.003). Well-being improvement was greater in participants with than without social ties in the program (at 30 days: p=0.02; at 90 days: p=0.003).

      Conclusions

      A multimodal online intervention leveraging social network effects significantly improved well-being over control. Higher levels of participation as well as increasing levels of social integration were associated with greater improvement in well-being.

      Trial registration

      Introduction

      The WHO defines health as a “state of complete physical, mental and social well-being and not merely the absence of disease or infirmity.”

      World Health Organization. Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June 1946, and entered into force on 7 April 1948.

      The concept of overall well-being is more expansive, capturing traditional physical, psychological, and social aspects of health, while incorporating two subjective dimensions: evaluative (cognitive reflections on one’s life and work satisfaction, relationships, and personal health, among others) and experienced (momentary affective states and feelings about experiences).
      • Kahneman D.
      • Riis J.
      Living, and thinking about it: two perspectives on life.
      • Diener E.
      Guidelines for national indicators of subjective well-being and ill-being.

      Gallup-Healthways Well-Being IndexTM: Methodology Report for Indexes. 2009. Gallup. well-beingindex.com/files/Gallup-Healthways%20Index%20Methodology%20Report%20FINAL%203-25-08.pdf.

      Metrics of overall well-being have been shown to predict healthcare utilization, medical and prescription expenditures, health risk factors, short-term disability, and work productivity.
      • Harrison P.L.
      • Pope J.E.
      • Coberley C.R.
      • Rula E.Y.
      Evaluation of the relationship between individual well-being and future health care utilization and cost.
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      • Prochaska J.O.
      • Evers K.E.
      • Castle P.H.
      • et al.
      Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
      • Sears L.E.
      • Shi Y.
      • Coberley C.R.
      • Pope J.E.
      Overall well-being as a predictor of health care, productivity, and retention outcomes in a large employer.
      Higher overall well-being is associated with positive health outcomes. For instance, for every 1-point increase on a 100-point scale, individuals are 2.2% less likely to have a hospital admission, 1.7% less likely to have an emergency room visit, 1% less likely to incur any healthcare costs and, if they did, incurred 1% fewer costs.
      • Harrison P.L.
      • Pope J.E.
      • Coberley C.R.
      • Rula E.Y.
      Evaluation of the relationship between individual well-being and future health care utilization and cost.
      Despite the evidence that improving well-being is a desirable goal, interventions that target overall well-being as a primary outcome are uncommon.
      • Prochaska J.O.
      • Evers K.E.
      • Castle P.H.
      • et al.
      Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
      Even a small effect (e.g., a 1-point increase) could potentially yield a significant impact if combined with large reach (impact = reach X effectiveness).
      • Abrams D.B.
      • Orleans T.
      • Niaura R.
      • Goldstein M.
      • Prochaska J.
      • Velicer W.
      Integrating individual and public health perspectives for treatment.
      Reaching population scale with traditional interventions can be difficult; web-based interventions offer the potential reach to large populations but often suffer from rapid decline in program usage and high dropout rates.
      • Eysenbach G.
      The law of attrition.
      Nonadherence and attrition reduce individuals’ exposure to the program, which in turn jeopardizes treatment effect.
      • Eysenbach G.
      The law of attrition.
      • Couper M.P.
      • Alexander G.L.
      • Zhang N.
      • et al.
      Engagement and retention: measuring breadth and depth of participant use of an online intervention.
      • Wanner M.
      • Martin-Diener E.
      • Bauer G.
      • Braun-Fahrländer C.
      • Martin B.W.
      Comparison of trial participants and open access users of a web-based physical activity intervention regarding adherence, attrition, and repeated participation.
      Given this, we theorized that an intervention designed to sustain participant engagement over time could increase overall program exposure and, potentially, the effect at a population level.
      A multimodal (web, e-mail, and mobile) intervention was developed to improve overall well-being. The program uses a small-steps approach,
      • Hill J.O.
      • Wyatt H.R.
      • Reed G.W.
      • Peters J.C.
      Obesity and the environment: where do we go from here?.
      game mechanics, and social networks to stimulate engagement and retention. As part of an iterative development protocol, the intervention was subjected to a pragmatic, randomized controlled trial. The primary aim of the trial presented herein was to evaluate the intervention’s real-world effectiveness early in its life cycle. A secondary aim was to explore the relationship between social support and well-being improvement.

      Methods

      Intervention

      Daily Challenge is a freely accessible e-mail-, web-, and mobile-based intervention. Participants receive a daily e-mail and/or text message suggesting a small health-related action (a “challenge”) that they can usually complete in a few minutes, along with information about how to complete the challenge and its relationship to well-being (Figure 1, left).
      Figure thumbnail gr1
      Figure 1Challenge-delivering e-mail and participant’s homepage on website (treatment)
      Reprinted with permission from MeYou Health
      A multidisciplinary team of writers, a behavioral scientist, and subject matter experts created challenges that pertain to all domains of well-being: Healthy Behaviors, Physical Health, Emotional Health, Life Evaluation, Work Environment, and Basic Access.

      Gallup-Healthways Well-Being IndexTM: Methodology Report for Indexes. 2009. Gallup. well-beingindex.com/files/Gallup-Healthways%20Index%20Methodology%20Report%20FINAL%203-25-08.pdf.

      The challenges cover topics such as healthy eating, physical activity, stress management, financial matters, relationships, life satisfaction, and sleep, among others. By default, a participant receives challenges covering all domains of well-being; at any point, they may opt to focus on an area of their choice.
      Intervention design was informed by social cognitive theory stressing a reciprocal learning process (social proof, verbal persuasion, self-monitoring, and frequent feedback) along with social influence and support within social networks to affect self-efficacy. Members report having completed the challenge (by e-mail, text message, or on the website) and collect virtual rewards. The challenge mechanism is built on top of a web-based social network (Figure 1, right).
      • Poirier J.
      • Cobb N.K.
      Social influence as a driver of engagement in a web-based health intervention.
      • Bandura A.
      Social foundations of thought and action: a social cognitive theory.
      Members are encouraged to recruit individuals from their real-life social network and connect with them within Daily Challenge. Additionally, members may interact and establish “friend” connections with people they meet through the intervention site. These connections are explicit and must be acknowledged (reciprocated) to be activated. Members can form pacts to complete challenges together, encourage one another, cheer each other on via “smiles,” and comment on each other’s challenge completion stories. Engagement is rewarded with points, badges, gradual revealing of graphic-level images, and other virtual elements drawn from game design work.

      Study Design

      The trial was a randomized, placebo-controlled, parallel-group trial with longitudinal outcome measurements at baseline, 30 days, and 90 days. The study protocol was approved by Independent IRB, Inc. All data were collected between April and September 2012; analysis was performed in October 2012.
      Treatment participants received the Daily Challenge intervention as designed, with no modifications. Control participants received a generic health newsletter by e-mail once a week (no social interactivity or calls to action) and otherwise had no access to the Daily Challenge system. The newsletter reported four current news stories that had been published no more than 7 days prior and covered each of the six domains of well-being over a 2-week period.

      Recruitment

      Recruitment, eligibility verification, and baseline data collection took place online. Advertisements were placed within Facebook, running 82 different ads over 47 days. Individuals clicking ads run specifically for the trial (e.g., Do one small action every day to improve well-being. Sign-up is easy and no cost!) were taken to the intervention website. Advertisements did not mention the trial itself. Individuals registered for the product using Facebook-enabled authentication, which provided their name, e-mail address, and information on their Facebook friends. Participants were recruited to take part in the trial during product registration. Candidates provided informed consent, completed a demographic questionnaire and baseline assessments, and verified their mobile phone number. Phone numbers were verified by response to a text message or an automated call. Eligible candidates were randomized; randomization was automated and gender-stratified (permutation within strata). Investigators were blinded to group assignments.
      Participants received a $20 Amazon.com Gift Card via e-mail upon enrollment and after completing each follow-up assessment. Treatment group participants went on to use the intervention whereas control group participants awaited their first newsletter.

      Participants

      Adults living in the U.S. who were age 18 or older (19 or older if residing in Alabama or Nebraska) were eligible for inclusion. Candidates were excluded if they did not provide a valid e-mail address; failed to provide informed consent; did not complete enrollment in the allotted time (45 minutes); or had a Facebook friend enrolled in the trial (having a Facebook friend in the intervention was not an exclusion criterion). Gender-based oversampling was conducted to reach a minimum 30% male population. Baseline sociodemographic data collected included gender, age, ZIP code, ethnicity, and race as well as a complete depiction of the participant’s social network within Facebook. ZIP codes served to estimate educational level (percentage of college graduates or higher) and median income based on U.S. Census 2000 data.

      Power Analysis

      Power calculations and sample size were based on an expected 2-point change in well-being score coupled with evidence indicating that a 1-point change is correlated with significant economic outcomes.
      • Harrison P.L.
      • Pope J.E.
      • Coberley C.R.
      • Rula E.Y.
      Evaluation of the relationship between individual well-being and future health care utilization and cost.
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      • Prochaska J.O.
      • Evers K.E.
      • Castle P.H.
      • et al.
      Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
      • Sears L.E.
      • Shi Y.
      • Coberley C.R.
      • Pope J.E.
      Overall well-being as a predictor of health care, productivity, and retention outcomes in a large employer.
      Budgetary constraints set an upper limit on recruitment at 1500 participants, which allowed the detection of a 2.2-point effect in a two-tailed t-test with 80% power, at a 5% significance level with a 20% dropout rate (alpha Bonferroni-corrected to 0.025; estimated SD=16.7).

      Data Collection

      The primary outcome (well-being) was assessed using the Individual-level Well-Being Assessment and Scoring Method (IWBS).
      • Evers K.
      • Prochaska J.O.
      • Castle P.H.
      • et al.
      Development of an individual well-being scores assessment.
      A score (range=0–100) is calculated for each of the six well-being domains: Healthy Behaviors, Emotional Health, Physical Health, Work Environment, Basic Access, and Life Evaluation. Domain scores are averaged to compute an overall well-being score. This instrument has been used for outcome measurement in interventions
      • Prochaska J.O.
      • Evers K.E.
      • Castle P.H.
      • et al.
      Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
      • Evers K.
      • Prochaska J.O.
      • Castle P.H.
      • et al.
      Development of an individual well-being scores assessment.
      and has led to insights into the relationship between well-being and healthcare costs and utilization (see introduction)
      • Harrison P.L.
      • Pope J.E.
      • Coberley C.R.
      • Rula E.Y.
      Evaluation of the relationship between individual well-being and future health care utilization and cost.
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      ; short-term disability
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      ; and work productivity.
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      Study participants received their well-being score after completing the assessment.
      Social support was assessed using the Interpersonal Support Evaluation List (ISEL, 12-item version).
      • Cohen S.
      • Mermelstein R.
      • Kamarck T.
      • Hoberman H.
      Measuring the functional components of social support.
      As pilot work indicated a relationship between social ties and engagement,
      • Poirier J.
      • Cobb N.K.
      Social influence as a driver of engagement in a web-based health intervention.
      connections within Daily Challenge were used as a secondary metric of social support.
      For the intervention group, all interactions were tracked automatically and recorded in real time in a relational database. Participants were uniquely tracked by identifiers embedded in URLs in e-mail messages, long-term cookies, or, if needed, via a login screen. A site visit was counted when the participant logged in and/or a page was seen after a period of at least 30 minutes without participant-generated activity while the participant was logged in. Self-reported challenge completions, friend connections to other intervention members (“social ties”), as well as every action (e.g., sharing challenge completion stories) and social interaction (“smiles,” replies to stories) on the website were archived. For both groups, an e-mail was marked opened if the participant’s e-mail program downloaded an embedded graphic image in the message; not all e-mail programs do so automatically, leading to an expected undercount using this metric.

      Follow-Up

      Participants were contacted over a 7-day window at 30 and 90 days post-enrollment. A multi-modal communication strategy (e-mail, text messaging, Facebook private messages, and on-site prompts) directed participants to a secure, online assessment interface. If they failed to complete the assessment within 5 (30-day follow-up) or 3 days (90-day follow-up), participants were called for manual data collection. Participants not reached after 7 days were considered lost to follow-up.

      Analysis

      Well-being and social support scores were analyzed in mixed-model regressions with maximum-likelihood estimation. Response variables were modeled with time, group, and their interaction as predictors. Participants’ age, gender, income, and education level were included as controlling variables. The model was first fitted to all available data, excluding 14 participants for whom no estimate of income and education level could be computed. Multiple imputation methods were used to create forty replicates of the data set with all participant data (including income and education) made complete,
      • Royston P.
      • White I.R.
      Multiple Imputation by Chained Equations (MICE): implementation in Stata.
      using Rubin’s rules to combine the results.
      • Schafer J.L.
      Analysis of incomplete multivariate data.
      Finally, the analysis was replicated for the sample with complete cases (i.e., restricted to participants with both 30- and 90-day follow-up data).
      Intensity of use (composite metric of e-mail opens, site visits, and challenge completions) was tested as a predictor of well-being change from baseline using linear regression. The relationship between social tie formation (i.e., “friend” connection to another intervention member) and well-being was evaluated with Student’s t-test. All p-values were two-sided (one-sided for the t-test), with significance set at 5% and corrected to 2.5% for the well-being and social support outcomes.

      Results

      Enrollment and study participation are shown in Figure 2. A total of 1503 eligible participants (17.2%) were included in the trial. Further, 752 candidates (50.0%) were assigned to the treatment group and 751 (50.0%) to the control group. One participant was erroneously assigned to both conditions and excluded from analysis.
      The final sample comprised 452 (30.1%) men and 1050 (69.9%) women. The mean age was 46.7 years. Mean median income was estimated at $75,227 (inflation-adjusted to 2012), and 29.8% of the sample was estimated to have reached college or higher education levels (Table 1). There was no association between group allocation and gender, age, ethnicity, race, income, education level, baseline well-being, or baseline social support.
      Table 1Participant characteristics at baseline
      Treatment, n=752Control, n=750
      Gender (% [n])
       Female35.0 (525)35.0 (525)
       Male15.1 (227)15.1 (225)
      Age (years; M [SD], IQR)42.4 (14.0), 2442.6 (14.0), 23
      Estimated median income (dollars; M [SD], IQR)74,697 (25,913), 33,29075,717 (30,043), 33,676
      Estimated education level (%)
       High school graduates27.928.3
       Some college, no degree21.421.2
       College graduates or higher30.329.2
      Ethnicity (% [n])
       Hispanic or Latino10.2 (77)8.0 (60)
       Not Hispanic or Latino89.8 (675)92.0 (690)
      Race (% [n])
       White87.6 (659)87.9 (659)
       African American8.4 (63)8.9 (67)
       Native American4.5 (34)2.7 (20)
       Asian2.4 (18)3.3 (25)
       Pacific Islander1.1 (8)0.9 (7)
      Well-being (M [SD], IQR)58.9 (17.5), 25.158.1 (16.8), 24.8
      Social support (M [SD], IQR)36.2 (8.1), 1235.9 (7.5), 11
      Note: Demographic and socioeconomic characteristics, well-being and social support at baseline are reported. For race and ethnicity, the values represent the percentage of study participants who identified with the category. Forty-five participants (3%) identified with more than one race. Income and educational attainment estimates derived from 2000 U.S. Census data, adjusted for inflation; estimates could not be computed for 14 participants. No difference between groups was significant.
      IQR, interquartile range
      Across groups, 75.0% (n=1126) of participants were reached for at least one follow-up, and 56.3% (n=846) of participants were reached at both follow-ups. At 30 days, 68.7% (n=1032) of the sample completed follow-up; 62.6% (n=940) of participants did so at 90 days. No evidence of differential study retention between groups was found at either time point (p>0.05).
      Process data on program participation (Table 2) indicate that 76.5% (n=575) of treatment participants and 51.1% (n=383) of control participants opened at least one program e-mail. In the treatment group, 84.6% (n=636) of participants visited the site at least once and 92.4% (n=695) completed at least one challenge. More than 50% of participants continued to complete challenges at 60 days.
      Table 2Use of treatment program, M (SD) [range], IQR
      1–30 days31–60 days61–90 days
      E-mail opens16.3 (24.3) [0–208], 2512.4 (20.8) [0–150], 199.9 (19.5) [0–170], 12
      Site visits12.7 (17.8) [0–231], 199.3 (18.9) [0–239], 138.1 (18.1) [0–193], 7
      Challenge completions13.0 (10.8) [0–31], 2210.7 (11.6) [0–34], 229.2 (11.4) [0–31], 20
      How I did it posts6.3 (8.7) [0–31], 104.4 (8.3) [0–33], 43.7 (8.1) [0–31], 2
      Social connectionsAt 30 daysAt 90 days
       In treatment only0.7 (2.9) [0–40], 01.9 (8.7) [0–147], 1
       In treatment and in Facebook0.3 (0.7) [0–6], 00.3 (0.8) [0–6], 0
      Note: Program usage patterns for treatment participants. Confirmed e-mail opens, site visits, challenge completions, social communications (posting “How I did it”), and social connections are reported. (Participants can complete a challenge up to 7 days after its assignment, which allows them to complete and post on more than 30 challenges in 30 days.) Social connections are divided into two groups: friendships that existed in participants’ Facebook accounts at study enrollment and that were re-established in Daily Challenge (“In treatment and in Facebook”), and ties between study participants and other Daily Challenge members that were not Facebook friends at study enrollment (“In treatment”).
      IQR, interquartile range
      Mean well-being scores over time are depicted for each group in Figure 3 (all available data). A time-by-group interaction was observed, indicating that changes in well-being over time depended on condition assignment (χ2=11.32, df=2, p=0.0035). Treatment increased well-being by 2.27 points over control at 30 days (95% CI=0.71, 3.83, p=0.004), and by 2.35 points at 90 days (95% CI=0.73, 3.96, p=0.004).
      Figure thumbnail gr3
      Figure 3Well-being scores at baseline, 30 days, and 90 days
      These results were consistent across analyses (Table 3). In the sample completed with multiple imputation, time interacted with group (F2,606=5.38, p=0.005); treatment increased well-being by 2.20 points over control at 30 days (p=0.006) and by 2.56 points by 90 days (p=0.004). In the complete case sample, the time-by-group interaction was marginally significant (χ2=5.17, df=2, p=0.0754), whereas the effect decreased to 1.71 at 30 days (p=0.057) and to 1.82 at 90 days (p=0.042).
      Table 3Change in well-being at 30 days and 90 days
      30 days90 days
      All available data (n=1488)2.27 p=0.004 (0.71, 3.83)2.35 p=0.004 (0.73, 3.96)
      Multiple imputation (n=1502)2.20 p=0.006 (0.63, 3.76)2.56 p=0.004 (0.84, 4.29)
      Complete case (n=840)1.71 p=0.057 (–0.05, 3.47)1.82 p=0.042 (0.06, 3.59)
      Note: Individual time-by-group interactions for all available data, data made complete by multiple imputation (which included imputed estimates of income and educational attainment for 14 participants), and complete case only. Values in parentheses are 95% CIs.
      Change in each of the well-being domains was analyzed. Significant time-by-group interactions were observed in the Healthy Behaviors (HB) and Emotional Health (EH) domains only. Treatment increased HB domain scores by 5.28 points over control at 30 days (95% CI=1.11, 9.45, p=0.013), and by 4.60 points at 90 days (95% CI=0.29, 8.92, p=0.036). EH domain scores in the treatment group increased more than in the control group by 5.34 points at 30 days (95% CI=0.95, 9.72, p=0.017), and by 5.60 points at 90 days (95% CI=1.07, 10.13, p=0.015).
      In the treatment group, intensity of use (composite metric of e-mail opens, site visits, and challenge completions) was a significant predictor of overall well-being change at 30 days (β=2.30, p=0.001, 95% CI=0.95, 3.65) and at 90 days (β=2.43, p=0.003, 95% CI=0.81, 4.04). Overall well-being change was compared between participants who had formed social ties (friend connections) in the intervention and those who had not. Further, t-tests revealed that social participants improved their well-being more significantly than nonsocial participants (at 30 days: 9.4- vs 7.0-point increase from baseline, p=0.02). A similar but more pronounced advantage for social participants was observed at 90 days (11.3- vs 7.3-point increase from baseline, p=0.003).
      Across groups, social support scores did not appear to be affected by condition assignment (change for treatment and control groups, respectively: 0.92 vs 0.77 at 30 days; 1.81 vs 1.16 at 90 days) as no significant time-by-group interactions were noted for social support scores in any model (p>0.05).

      Discussion

      A multi-modal online intervention (Daily Challenge) was effective in improving well-being by 2.27 points (from a baseline of 58.9) at 30 days and by 2.35 points at 90 days. Participants who opened more e-mails, visited the site more often, and completed more challenges showed greater improvement, consistent with a dose response. Well-being improvement was greater in participants who had formed social ties within the intervention, despite the fact that no change in perceived social support was detected.
      The effect on well-being found in this trial, although small in absolute terms, could have public health significance if it is shown to affect healthcare outcomes or spending. Changes of this magnitude have been associated with measurable healthcare expenditures, hospital utilization, short-term disability frequency, and work productivity in observational studies.
      • Harrison P.L.
      • Pope J.E.
      • Coberley C.R.
      • Rula E.Y.
      Evaluation of the relationship between individual well-being and future health care utilization and cost.
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      Combined with the large reach afforded by its multimodal, electronic delivery, the potential population-level impact (reach X effectiveness)
      • Abrams D.B.
      • Orleans T.
      • Niaura R.
      • Goldstein M.
      • Prochaska J.
      • Velicer W.
      Integrating individual and public health perspectives for treatment.
      of interventions with similar targets and approach is promising. Interestingly, the baseline well-being in the population was lower than observed in employee populations,
      • Shi Y.
      • Sears L.
      • Coberley C.
      • Pope J.E.
      Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
      but nearly identical to that reported in a previous intervention study that drew from a national survey database of 1.5 million participants.
      • Prochaska J.O.
      • Evers K.E.
      • Castle P.H.
      • et al.
      Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
      In both trials, a notable and similar increase in average well-being scores of the control condition was observed (current study: 58.1–66.1. Prochaska et al. study
      • Prochaska J.O.
      • Evers K.E.
      • Castle P.H.
      • et al.
      Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
      : 58.9–65.3). Such a change could indicate assessment reactivity, the “mere measurement” phenomenon (where simply asking about a behavior causes it to change)
      • Godin G.
      • Sheeran P.
      • Conner M.
      • et al.
      Which survey questions change behavior? Randomized controlled trial of mere measurement interventions.
      or another undetected process. Although Daily Challenge is designed to affect all six domains of well-being as measured by the IWBS, three of those domains would not be expected to change within the relatively short window of this study: Physical Health, Work Environment, and Basic Access. Of the remaining three, Life Evaluation, Emotional Health, and Healthy Behaviors, the benefit of the intervention at 30 and 90 days was primarily attributable to the latter two. These findings argue against the present results being secondary to assessment reactivity or the mere measurement phenomenon.
      The intervention benefited from less non-usage attrition than typically observed in online interventions,
      • Eysenbach G.
      The law of attrition.
      • Mathieu E.
      • McGeechan K.
      • Barratt A.
      • Herbert R.
      Internet-based randomized controlled trials: a systematic review.
      and a larger effect was observed in participants who used the program more extensively. This is likely the result of design decisions stressing engagement, including the implementation of a “gamification” strategy mated with a small-steps approach,
      • Hill J.O.
      • Wyatt H.R.
      • Reed G.W.
      • Peters J.C.
      Obesity and the environment: where do we go from here?.
      where small, individual day-by-day behaviors can be rewarded with points, badges, and social recognition. These findings may be useful for intervention design, but confirming this hypothesis will require rigorous manipulation of program elements in a future controlled trial.
      Recruitment was deliberately limited to Facebook advertising in an effort to measure and control attrition. This strategy afforded the benefit of detailing the denominators throughout the process, starting at the number of individuals that viewed the advertisements, a capacity not present in most trials using more traditional recruitment methods.
      • Graham A.L.
      • Bock B.C.
      • Cobb N.K.
      • Niaura R.
      • Abrams D.B.
      Characteristics of smokers reached and recruited to an internet smoking cessation trial: a case of denominators.
      More importantly, individuals clicking through on a vague advertisement and transitioning out of the Facebook site may be poorly primed to take action around a behavioral or well-being intervention, resulting in the large drop off between reaching the site, registering and eventually enrolling in a trial, as seen in the CONSORT diagram. As more trials present this sort of data, the evaluation of recruitment methods and the assessment of the generalizability of enrolled populations will be feasible.
      Having formed social ties in the intervention was associated with larger improvement in well-being despite the absence of a significant increase in social support scores. It is possible that forms of social influence, signaling, or social norms that were not measured may be equally powerful as traditional metrics of social support but more difficult to detect or disentangle. Parallel to this problem is the fact that translation of observational social network research into active interventions has proven difficult,
      • Christakis N.
      Social networks and collateral health effects.
      • Valente T.W.
      Network interventions.
      • Cobb N.K.
      • Graham A.L.
      • Byron M.J.
      • Abrams D.B.
      Online social networks and smoking cessation: a scientific research agenda.
      even with the potential of online interventions to create network ties and sophisticated informatics tools for evaluation. Further research on determining active social mediators, and how they can be efficiently and effectively applied in behavioral health interventions, is acutely needed for ongoing and future intervention design.
      Several limitations to the present findings should be noted. The lack of change in social support scores may represent a true null effect, or may be attributable to the choice of assessment tool. An abbreviated form of the ISEL (12 vs 40 items in full version) was used to reduce attrition at enrollment and follow-up. More than half of the questions in the shorter assessment refer to the accessibility of physically local support (e.g., If I were sick, I could easily find someone to help me with my daily chores.). It is possible that the instrument is not sensitive to the sorts of changes in perceived social support that might occur online. Future work would benefit from instruments specifically validated and refined for use in online environments.
      Loss to follow-up remains a major challenge and problem in trials that are conducted entirely online. The follow-up rates (68.7% at 30 days and 62.6% at 90 days) obtained in this trial compare favorably with other real-world effectiveness trials of online interventions,
      • Graham A.L.
      • Bock B.C.
      • Cobb N.K.
      • Niaura R.
      • Abrams D.B.
      Characteristics of smokers reached and recruited to an internet smoking cessation trial: a case of denominators.
      but stress the difficulty of achieving universal follow-up in these settings, even with an aggressive contact protocol. The pragmatic nature of this trial deliberately restricted its time frame, notably limiting the ability to observe impact in domains of well-being that would be slower to change, such as Work Environment or Basic Access. Further work will be needed to validate the present findings at more distal time points as well as assess for changes in additional domains of well-being. Ultimately, though well-being has been tied to diverse economic, behavioral, and medical outcomes, future research will need to demonstrate a causation effect, where an intervention can be shown to improve not only the metrics but also the more distal outcomes themselves.
      Finally, the described pragmatic trial follows an alternative model of research and development of behavioral health interventions derived from engineering approaches.
      • Cobb N.K.C.
      • Poirier J.
      Implementation of an online pragmatic randomized controlled trial: a methodological case study.
      • Collins L.M.
      • Murphy S.A.
      • Nair V.N.
      • Strecher V.J.
      A strategy for optimizing and evaluating behavioral interventions.
      In this model of iterative development, successive variants of increasingly complex and sophisticated interventions are piloted in the intended target population. Collected data are used immediately to feed back into the design cycle. This randomized controlled trial thus represents a waypoint rather than an endpoint—indeed, in this model an intervention may never be considered to be truly complete until there is no viable or cost-effective path to improving either effect or reach. Although this model is difficult to mesh with traditional grant funding mechanisms,
      • Riley W.T.
      • Glasgow R.E.
      • Etheredge L.
      • Rapid Abernethy A.P.
      Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise.
      we strongly believe that this sort of design methodology will be needed to build sufficiently effective interventions to close this loop, as well as tackle related issues such as obesity or tobacco use.

      Acknowledgments

      Financial support was provided by MeYou Health LLC, a wholly owned subsidiary of Healthways Inc. The authors gratefully acknowledge E. Paul Wileyto for his contribution to analysis and James Pope, Carter Coberley, Kerry Evers, and James Prochaska for their valuable feedback on earlier versions of the manuscript. The authors also thank the MeYou Health team for their work implementing and conducting the trial.
      Drs. Cobb and Poirier are employees of MeYou Health LLC (a wholly owned subsidiary of Healthways, Inc.), which developed and markets the intervention Daily Challenge. Both authors have owned stock and/or stock options in Healthways.
      Institutional Review Board: Independent IRB, Inc., approved this protocol on March 15, 2012.
      No other financial disclosures have been reported by the authors of this paper.

      References

      1. World Health Organization. Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June 1946, and entered into force on 7 April 1948.

        • Kahneman D.
        • Riis J.
        Living, and thinking about it: two perspectives on life.
        in: Huppert F.A. Baylis N. Keverne B. The science of well-being. Oxford University Press, Oxford2005
        • Diener E.
        Guidelines for national indicators of subjective well-being and ill-being.
        Appl Res Qual Life. 2006; 1: 151-157
      2. Gallup-Healthways Well-Being IndexTM: Methodology Report for Indexes. 2009. Gallup. well-beingindex.com/files/Gallup-Healthways%20Index%20Methodology%20Report%20FINAL%203-25-08.pdf.

        • Harrison P.L.
        • Pope J.E.
        • Coberley C.R.
        • Rula E.Y.
        Evaluation of the relationship between individual well-being and future health care utilization and cost.
        Popul Health Manag. 2012; 15: 325-330
        • Shi Y.
        • Sears L.
        • Coberley C.
        • Pope J.E.
        Classification of individual well-being scores for the determination of adverse health and productivity outcomes in employee population.
        Popul Health Manag. 2013; 16: 90-98
        • Prochaska J.O.
        • Evers K.E.
        • Castle P.H.
        • et al.
        Enhancing multiple domains of well-being by decreasing multiple health risk behaviors: a randomized clinical trial.
        Popul Health Manag. 2012; 15: 276-286
        • Sears L.E.
        • Shi Y.
        • Coberley C.R.
        • Pope J.E.
        Overall well-being as a predictor of health care, productivity, and retention outcomes in a large employer.
        Popul Health Manag. 2013; https://doi.org/10.1089/pop.2012.0114
        • Abrams D.B.
        • Orleans T.
        • Niaura R.
        • Goldstein M.
        • Prochaska J.
        • Velicer W.
        Integrating individual and public health perspectives for treatment.
        Ann Behav Med. 1996; 18: 290-304
        • Eysenbach G.
        The law of attrition.
        J Med Internet Res. 2005; 7: e11
        • Couper M.P.
        • Alexander G.L.
        • Zhang N.
        • et al.
        Engagement and retention: measuring breadth and depth of participant use of an online intervention.
        J Med Internet Res. 2010; 12: e52
        • Wanner M.
        • Martin-Diener E.
        • Bauer G.
        • Braun-Fahrländer C.
        • Martin B.W.
        Comparison of trial participants and open access users of a web-based physical activity intervention regarding adherence, attrition, and repeated participation.
        J Med Internet Res. 2010; 12: e3
        • Hill J.O.
        • Wyatt H.R.
        • Reed G.W.
        • Peters J.C.
        Obesity and the environment: where do we go from here?.
        Science. 2003; 299: 853-855
        • Poirier J.
        • Cobb N.K.
        Social influence as a driver of engagement in a web-based health intervention.
        J Med Internet Res. 2012; 14: e36
        • Bandura A.
        Social foundations of thought and action: a social cognitive theory.
        Prentice-Hall, Englewood Cliffs NJ1986
        • Evers K.
        • Prochaska J.O.
        • Castle P.H.
        • et al.
        Development of an individual well-being scores assessment.
        Psychol Well Being. 2013; 2: (2)
        • Cohen S.
        • Mermelstein R.
        • Kamarck T.
        • Hoberman H.
        Measuring the functional components of social support.
        in: Sarason I.G. Sarason B.R. Social support: theory, research and application. The Hague. Martinus Nijhoff, The Netherlands1985
        • Royston P.
        • White I.R.
        Multiple Imputation by Chained Equations (MICE): implementation in Stata.
        J Stat Soft. 2011; 45: 1-20
        • Schafer J.L.
        Analysis of incomplete multivariate data.
        Chapman & Hall, CRC, New York2002
        • Godin G.
        • Sheeran P.
        • Conner M.
        • et al.
        Which survey questions change behavior? Randomized controlled trial of mere measurement interventions.
        Health Psychol. 2010; 29: 636-644
        • Mathieu E.
        • McGeechan K.
        • Barratt A.
        • Herbert R.
        Internet-based randomized controlled trials: a systematic review.
        J Am Med Inform Assoc. 2013; 20: 568-576
        • Graham A.L.
        • Bock B.C.
        • Cobb N.K.
        • Niaura R.
        • Abrams D.B.
        Characteristics of smokers reached and recruited to an internet smoking cessation trial: a case of denominators.
        Nicotine Tob Res. 2006; 8: S43-S48
        • Christakis N.
        Social networks and collateral health effects.
        BMJ. 2004; 329: 184-185
        • Valente T.W.
        Network interventions.
        Science. 2012; 337: 49-53
        • Cobb N.K.
        • Graham A.L.
        • Byron M.J.
        • Abrams D.B.
        Online social networks and smoking cessation: a scientific research agenda.
        J Med Internet Res. 2011; 13: e119
        • Cobb N.K.C.
        • Poirier J.
        Implementation of an online pragmatic randomized controlled trial: a methodological case study.
        Transl Behav Med. 2013; 3: 295-303
        • Collins L.M.
        • Murphy S.A.
        • Nair V.N.
        • Strecher V.J.
        A strategy for optimizing and evaluating behavioral interventions.
        Ann Behav Med. 2005; 30: 65-73
        • Riley W.T.
        • Glasgow R.E.
        • Etheredge L.
        • Rapid Abernethy A.P.
        Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise.
        Clin Transl Med. 2013; 2: 10