Designing and Undertaking a Health Economics Study of Digital Health Interventions

      This paper introduces and discusses key issues in the economic evaluation of digital health interventions. The purpose is to stimulate debate so that existing economic techniques may be refined or new methods developed. The paper does not seek to provide definitive guidance on appropriate methods of economic analysis for digital health interventions.
      This paper describes existing guides and analytic frameworks that have been suggested for the economic evaluation of healthcare interventions. Using selected examples of digital health interventions, it assesses how well existing guides and frameworks align to digital health interventions. It shows that digital health interventions may be best characterized as complex interventions in complex systems. Key features of complexity relate to intervention complexity, outcome complexity, and causal pathway complexity, with much of this driven by iterative intervention development over time and uncertainty regarding likely reach of the interventions among the relevant population. These characteristics imply that more-complex methods of economic evaluation are likely to be better able to capture fully the impact of the intervention on costs and benefits over the appropriate time horizon. This complexity includes wider measurement of costs and benefits, and a modeling framework that is able to capture dynamic interactions among the intervention, the population of interest, and the environment. The authors recommend that future research should develop and apply more-flexible modeling techniques to allow better prediction of the interdependency between interventions and important environmental influences.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to American Journal of Preventive Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ramsey S.
        • Willke R.
        • Glick H.
        Cost-effectiveness analysis alongside clinical trials II—an ISPOR good research practices task force report.
        Value Health. 2015; 18: 161-172
        • Ekeland A.
        • Bowes A.
        • Flottorp S.
        Effectiveness of telemedicine: a systematic review of reviews.
        Int J Med Inform. 2010; 79: 736-771
        • Mistry H.
        Systematic review of studies of the cost-effectiveness of telemedicine and telecare: changes in the economic evidence over twenty years.
        J Telemed Telecare. 2012; 18: 1-6
        • Mistry H.
        • Garnvwa H.
        • Oppong R.
        Critical appraisal of published systematic reviews assessing the cost-effectiveness of telemedicine studies.
        Telemed J E Health. 2014; 20: 609-618
        • Murray E.
        • Hekler E.
        • Andersson G.
        • et al.
        Evaluating digital health interventions: key questions and approaches.
        Am J Prev Med. 2016;
        • Kelly M.
        • Morgan A.
        • Ellis S.
        • et al.
        Evidence based public health: a review of the experience of the national institute of health and clinical excellence (NICE) of developing public health guidance in England.
        Soc Sci Med. 2010; 71: 1056-1062
        • Rychetnik L.
        • Frommer M.
        • Hawe P.
        • Shiell A.
        Criteria for evaluating evidence on public health interventions.
        J Epidemiol Community Health. 2002; 56: 119-127
        • Vitora C.G.
        • Habichtl J.P.
        • Bryce J.
        Evidence-based public health: moving beyond randomized trials.
        Am J Public Health. 2004; 94: 400-405
        • Petticrew M.
        Time to rethink the systematic review catechism? Moving from “what works” to “what happens.”.
        Syst Rev. 2015; 4: 36
        • Hawe P.
        • Shiell A.
        • Riley T.
        Theorising interventions as events in systems.
        Am J Community Psychol. 2009; 43: 267-276
        • Hawe P.
        • Shiell A.
        • Riley T.
        • Gold L.
        Methods for exploring implementation variation and local context within a cluster randomised community intervention trial.
        J Epidemiol Community Health. 2004; 58: 788-793
        • Shiell A.
        • Hawe P.
        • Gold L.
        Complex interventions or complex systems? Implications for health economic evaluation.
        BMJ. 2008; 336: 1281-1283
        • Craig P.
        • Dieppe P.
        • Macintyre S.
        • et al.
        Developing and evaluating complex interventions: the new MRC guidance.
        BMJ. 2008; 337: a1655
        • Petticrew M.
        • Anderson L.
        • Elder R.
        • et al.
        Complex interventions and their implications for systematic reviews: a pragmatic approach.
        J Clin Epidemiol. 2013; 66: 1209-1214
        • Zhang D.
        • Giabbanelli P.J.
        • Arah O.
        • Zimmerman F.J.
        Impact of different policies on unhealthy dietary behaviors in an urban adult population: an agent-based simulation model.
        Am J Public Health. 2014; 104: 1217-1222
        • Zimmerman F.J.
        Habit, custom, and power: a multi-level theory of population health.
        Soc Sci Med. 2013; 80: 47-56
        • Sanson-Fisher R.W.
        • D’Este C.A.
        • Carey M.L.
        • Noble N.
        • Paul C.L.
        Evaluation of systems oriented public health interventions: alternative research designs.
        Annu Rev Public Health. 2014; 35: 9-27
      1. NHS England. Real world testing of “combinatorial interventions”—a global invitation to innovators. The AHSN Network. Published 2015. Accessed December 7, 2015.

        • Fischer A.J.
        • Threlfall A.
        • Meah S.
        • et al.
        The appraisal of public health interventions: an overview.
        J Public Health. 2013; 35: 488-494
        • Threlfall A.G.
        • Meah S.
        • Fischer A.J.
        • et al.
        The appraisal of public health interventions: the use of theory.
        J Public Health. 2015; 37: 166-171
        • Little P.
        • Stuart B.
        • Hobbs F.
        • et al.
        An internet-delivered handwashing intervention to modify influenza-like illness and respiratory infection transmission (PRIMIT): a primary care randomised trial.
        Lancet. 2015; 386: 1631-1639
        • Christakis N.A.
        • Fowler J.H.
        The spread of obesity in a large social network over 32 years.
        N Engl J Med. 2007; 357: 370-379
      2. Get The World Moving Limited. Global corporate challenge. Published 2015. Accessed December 7, 2015.

        • Christakis N.A.
        • Fowler J.H.
        The collective dynamics of smoking in a large social network.
        N Engl J Med. 2008; 358: 2249-2258
        • Christakis N.A.
        • Fowler J.H.
        Connected: The Amazing Power of Social Networks and How They Shape Our Lives.
        1st ed. Harper, London2009
        • Powell K.
        • Wilcox J.
        • Clonan A.
        • et al.
        The role of social networks in the development of overweight and obesity among adults: a scoping review.
        BMC Public Health. 2015; 15: 996
        • Leroux J.
        • Moore S.
        • Dubé L.
        Beyond the "I" in the obesity epidemic: a review of social relational and network interventions on obesity.
        J Obes. 2013; 2013: 348249
        • Frerichs L.
        • Araz O.
        • Huang T.
        Modeling social transmission dynamics of unhealthy behaviors for evaluating prevention and treatment interventions on childhood obesity.
        PLoS One. 2013; 8: e82887
        • El-Sayed A.
        • Scarborough P.
        • Seman L.
        • Galea S.
        Social network analysis and agent-based modelling in social epidemiology.
        Epidemiol Perspect Innov. 2012; 9
        • Centola D.
        An experimental study of homophily in the adoption of health behavior.
        Science. 2011; 334: 1269-1272
        • Siebers P.O.
        • Macal C.M.
        • Garnett J.
        • Buxton D.
        • Pidd M.
        Discrete-event simulation is dead, long live agent-based simulation!.
        J Simulat. 2010; 4: 204-210
        • Little P.
        • Stuart B.
        • Francis N.
        • et al.
        Effects of internet-based training on antibiotic prescribing rates for acute respiratory-tract infections: a multinational, cluster, randomised, factorial, controlled trial.
        Lancet. 2013; 382: 1175-1182
        • Yardley L.
        • Ware L.J.
        • Smith E.R.
        • et al.
        Randomised controlled feasibility trial of a web-based weight management intervention with nurse support for obese patients in primary care.
        Int J Behav Nutr Phys Act. 2014; 11: 1-11
        • Yardley L.
        • Morrison L.
        • Bradbury K.
        • Muller I.
        The person-based approach to intervention development: application to digital health-related behavior change Interventions.
        J Med Internet Res. 2015; 17: e30
        • McCambridge J.
        • OʼDonnell O.
        • Godfrey C.
        • et al.
        How big is the elephant in the room? Estimated and actual IT costs in an online behaviour change trial.
        BMC Res Notes. 2010; 3: 172
        • Yardley L.
        • Spring B.J.
        • Riper H.
        • et al.
        Understanding and promoting engagement with digital behaviour change interventions.
        Am J Prev Med. 2016;
        • Saddichha S.
        • Al-Desouki M.
        • Lamia A.
        • Linden I.A.
        • Krausz M.
        Online interventions for depression and anxiety—a systematic review.
        Health Psychol Behav Med. 2014; 2: 841-888
        • Dennison L.
        • Morrison L.
        • Lloyd S.
        • et al.
        Does brief telephone support improve engagement with a web-based weight management intervention? Randomized controlled trial.
        J Med Internet Res. 2014; 16: e95
        • Ling T.
        Evaluating complex and unfolding interventions in real time.
        Evaluation. 2012; 18: 79-91
        • Al-Janabi H.
        • Flynn T.
        • Coast J.
        Development of a self-report measure of capability wellbeing for adults: the ICECAP-A.
        Qual Life Res. 2012; 21: 167-176
      3. InTechnology plc. New digital healthcare service improves patient quality of life. Published 2014. Accessed December 7, 2015.

        • Weaver E.R.
        • Horyniak D.R.
        • Jenkinson R.
        • Dietze P.
        • Lim M.S.
        “Let’s get wasted!” and other apps: characteristics, acceptability, and use of alcohol-related smartphone applications.
        J Med Internet Res. 2013; 1: e9
        • Hofmann M.
        • Dack C.
        • Barker C.
        • Murray E.
        The impact of an Internet-based self-management intervention (HeLP-diabetes) on the psychological well-being of adults with type 2 diabetes: a mixed-method cohort study.
        J Diabetes Res. 2016; 2016: 1-13
        • Kahneman D.
        Thinking, Fast and Slow. 1st ed. Penguin, London2012
        • Squires H.
        A Methodological Framework for Developing the Structure of Public Health Economic Models [PhD thesis].
        University of Sheffield, Sheffield, UK2014
        • Chalabi Z.
        • Lorenc T.
        Using agent-based models to inform evaluation of complex interventions: examples from the built environment.
        Prev Med. 2013; 57: 434-435
        • Maglio P.
        • Mabry P.
        Agent-based models and systems science approaches to public health.
        Am J Prev Med. 2011; 40: 384-392
      4. INTEGRATE-HTA. 2015. Accessed December 7, 2015.