Advertisement

Data Needs in Opioid Systems Modeling: Challenges and Future Directions

Open AccessPublished:December 01, 2020DOI:https://doi.org/10.1016/j.amepre.2020.08.017

      Introduction

      The opioid crisis is a pervasive public health threat in the U.S. Simulation modeling approaches that integrate a systems perspective are used to understand the complexity of this crisis and analyze what policy interventions can best address it. However, limitations in currently available data sources can hamper the quantification of these models.

      Methods

      To understand and discuss data needs and challenges for opioid systems modeling, a meeting of federal partners, modeling teams, and data experts was held at the U.S. Food and Drug Administration in April 2019. This paper synthesizes the meeting discussions and interprets them in the context of ongoing simulation modeling work.

      Results

      The current landscape of national-level quantitative data sources of potential use in opioid systems modeling is identified, and significant issues within data sources are discussed. Major recommendations on how to improve data sources are to: maintain close collaboration among modeling teams, enhance data collection to better fit modeling needs, focus on bridging the most crucial information gaps, engage in direct and regular interaction between modelers and data experts, and gain a clearer definition of policymakers’ research questions and policy goals.

      Conclusions

      This article provides an important step in identifying and discussing data challenges in opioid research generally and opioid systems modeling specifically. It also identifies opportunities for systems modelers and government agencies to improve opioid systems models.

      INTRODUCTION

      Opioid-related deaths have risen dramatically since the late 1990s. In 2018, a total of 120 people died every day from an overdose involving opioids.
      Opioid data analysis and resources
      Centers for Disease Control and Prevention.
      ,
      Drug overdose deaths in the United States, 1999–2018.
      Opioid overdose is now the leading cause of accidental death among Americans aged <50 years and has contributed to the first decrease in U.S. life expectancy in a century. Consequently, opioid misuse has seized the attention of scholars, researchers, health professionals, and politicians,
      HHS, Office of the Surgeon General
      Facing addiction in America: the Surgeon General's report on alcohol, drugs, and health.
      resulting in calls for a broad public health approach to reducing opioid misuse, disorder, and fatal and nonfatal overdoses.
      • Saloner B
      • McGinty EE
      • Beletsky L
      • et al.
      A public health strategy for the opioid crisis.
      The opioid crisis is multifaceted and dynamic,
      • Jalali MS
      • Botticelli M
      • Hwang RC
      • Koh HK
      • McHugh RK
      The opioid crisis: a contextual, social-ecological framework.
      ,
      • DeWeerdt S.
      Tracing the U.S. opioid crisis to its roots.
      involving a wide range of societal entities including pharmaceutical companies
      • Haffajee RL
      • Mello MM.
      Drug companies’ liability for the opioid epidemic.
      ; healthcare systems

      Leslie DL, Ba DM, Agbese E, Xing X, Liu G. The economic burden of the opioid epidemic on states: the case of Medicaid. Am J Manag Care. 2019;25(13 suppl):S243–S249. https://www.ajmc.com/view/the-economic-burden-opioid-epidemic-on-states-case-of-medicaid. Accessed June 18, 2002.

      ; insurance companies
      • Schatman ME
      • Webster LR.
      The health insurance industry: perpetuating the opioid crisis through policies of cost-containment and profitability.
      ; law enforcement agencies and criminal justice systems
      • Brinkley-Rubinstein L
      • Zaller N
      • Martino S
      • et al.
      Criminal justice continuum for opioid users at risk of overdose.
      ; social welfare and foster care systems

      Crowley DM, Connell CM, Jones D, Donovan MW. Considering the child welfare system burden from opioid misuse: research priorities for estimating public costs. Am J Manag Care. 2019;25(13 suppl):S256–S263. https://pubmed.ncbi.nlm.nih.gov/31361428/. Accessed June 18, 2020.

      ; employers, individuals, and communities

      Hagemeier NE. Introduction to the opioid epidemic: the economic burden on the healthcare system and impact on quality of life. Am J Manag Care. 2018;24(10 suppl):S200–S206. https://www.ajmc.com/view/intro-opioid-epidemic-economic-burden-on-healthcare-system-impact-quality-of-life. Accessed June 17, 2020.

      ; and local, state, and federal public health agencies.
      • Soelberg CD
      • Brown Jr, RE
      • Du Vivier D
      • Meyer JE
      • Ramachandran BK
      The U.S. opioid crisis: current federal and state legal issues.
      ,
      • Dayer LE
      • Painter JT
      • McCain K
      • King J
      • Cullen J
      • Foster HR
      A recent history of opioid use in the U.S.: three decades of change.
      The rapidly changing dynamics of this crisis make it a particularly challenging problem to understand in its entirety.
      • Volkow ND
      • Jones EB
      • Einstein EB
      • Wargo EM
      Prevention and treatment of opioid misuse and addiction: a review.
      The uncertainty of how policies or interventions might impact behaviors and harms, as well as the lack of comprehensive data to fully describe the heterogeneous agents involved,
      • Smart R
      • Kase CA
      • Taylor EA
      • Lumsden S
      • Smith SR
      • Stein BD
      Strengths and weaknesses of existing data sources to support research to address the opioids crisis.
      necessitate the development of simulation approaches that project and explore policy outcomes before such policies are tested in expensive and resource-intensive clinical trials or ad hoc policy implementation. Simulation modeling accomplishes this by drawing on disparate data sources, including qualitative structural data on system functioning, to help answer “what if” policy questions through policy projections.
      • Jalali MS
      • Botticelli M
      • Hwang RC
      • Koh HK
      • McHugh RK
      The opioid crisis: need for systems science research.
      Simulation models that integrate a systems perspective and whose model outcomes result from the inter-relationship of risk factors as a function of time (e.g., system dynamics, agent-based, dynamic compartment—hereafter, systems modeling or systems models) provide further insight by accounting for the dynamic interactions of complex systems, which can produce unintended consequences, policy resistance, and “worse before better” scenarios.
      • Benneyan J
      • Garrahan J
      • Ilieş I
      • Duan X
      Modeling approaches, challenges, and preliminary results for the opioid and heroin co-epidemic crisis.
      Systems modeling is unique in its ability to support a trade-off analysis of prospective policy decisions where time-varying outcomes are critical to consider. It can assist researchers and policymakers in surfacing counterintuitive insights and potential interactive and synergistic effects of policies
      • Jalali MS
      • Botticelli M
      • Hwang RC
      • Koh HK
      • McHugh RK
      The opioid crisis: need for systems science research.
      (both negative and positive) and conducting economic evaluation (E Beaulieu, MA, unpublished data, May 2020) (e.g., cost-effectiveness analysis) before policy implementation in the real world. For instance, opioid systems models

      Homer J, Wakeland W. A dynamic model of the opioid drug epidemic with implications for policy. Am J Drug Alcohol Abuse. In press. Online June 9, 2020. https://doi.org/10.1080/00952990.2020.1755677.

      have been developed to test the simultaneous expansion of different evidence-based interventions that reduce overdose mortality, such as widespread naloxone distribution and increased access to medications for opioid use disorder (OUD).
      • Volkow ND.
      Medications for opioid use disorder: bridging the gap in care.
      Other interventions that have been proposed to address the opioid overdose crisis include decreasing chronic opioid prescribing. However, this could have the unintended consequence of increased untreated chronic pain and overall reduced quality of life.
      • Foglia MB.
      When people you love are the unintended consequences of opioid policy.
      Policy changes like this are arguably the result of having (justified) anxiety about the implications of broader structural changes and too few tools to consider what those implications might be. Systems models are a key tool to support decision makers in exploring policies and their implications. However, as with all models, decision makers must use the tools’ projections in combination with value judgments to consider the complexities of trade-offs. More discussion about the benefits, weaknesses, and uses of different modeling methods, as well as the type of policy questions they can answer, can be found elsewhere.
      • Jalali MS
      • Botticelli M
      • Hwang RC
      • Koh HK
      • McHugh RK
      The opioid crisis: need for systems science research.
      ,
      • Sharareh N
      • Sabounchi SS
      • McFarland M
      • Hess R
      Evidence of modeling impact in development of policies for controlling the opioid epidemic and improving public health: a scoping review.
      ,

      OUD Modeling writing group. How simulation modeling can support the public health response to the opioid crisis in North America: setting priorities and assessing value. Int J Drug Policy. In press. Online April 28, 2020. https://doi.org/10.1016/j.drugpo.2020.102726.

      Systems modeling has been applied widely to public health issues and epidemics for which insufficient data or high levels of uncertainty make more traditional analytic approaches infeasible.
      • Leischow SJ
      • Best A
      • Trochim WM
      • et al.
      Systems thinking to improve the public's health.
      • Peters DH.
      The application of systems thinking in health: why use systems thinking?.
      • Sterman JD.
      Learning from evidence in a complex world.
      Examples include large-scale models developed by multiple teams such as the Cancer Intervention and Surveillance Modeling Network for cancer,
      • van Ravesteyn NT
      • van den Broek JJ
      • Li X
      • et al.
      Modeling ductal carcinoma in situ (DCIS): an overview of CISNET model approaches.
      Framework for Reconstructing Epidemiological Dynamics for infectious diseases,
      • Grefenstette JJ
      • Brown ST
      • Rosenfeld R
      • et al.
      FRED (a framework for reconstructing epidemic dynamics): an open-source software system for modeling infectious diseases and control strategies using census-based populations.
      CommunityFlu for influenza,

      Atkins CY, Meltzer MI, Haber MJ. Community flu: software to estimate the impact of an influenza pandemic on a simulated community with and without interventions. Atlanta, GA: Centers for Disease Control and Prevention. https://stacks.cdc.gov/view/cdc/49287. Published August 10, 2009. Accessed 1 August 2020.

      and projects that have developed models for obesity
      • Hammond RA.
      Complex systems modeling for obesity research.
      and post-traumatic stress disorder.
      • Ghaffarzadegan N
      • Larson RC
      • Fingerhut H
      • et al.
      Model-based policy analysis to mitigate post-traumatic stress disorder.
      These efforts are based on cross-disciplinary collaborations to better collect available data, identify data gaps, and employ comparative modeling approaches. For instance, the Cancer Intervention and Surveillance Modeling Network's breast cancer modeling group has assembled a nationally representative collection of common model inputs, which has enabled direct comparison of results across models, strengthening inferences about their findings.
      • Mandelblatt JS
      • Near AM
      • Miglioretti DL
      • et al.
      Common model inputs used in CISNET collaborative breast cancer modeling.
      Recently, several research teams have begun to develop systems models to tackle the opioid overdose and addiction crisis.
      • Wakeland W
      • Nielsen A
      • Schmidt TD
      • et al.
      Modeling the impact of simulated educational interventions on the use and abuse of pharmaceutical opioids in the United States: a report on initial efforts.
      • Wakeland W
      • Nielsen A
      • Geissert P
      Dynamic model of nonmedical opioid use trajectories and potential policy interventions.
      • Chen Q
      • Larochelle MR
      • Weaver DT
      • et al.
      Prevention of prescription opioid misuse and projected overdose deaths in the United States.
      • Pitt AL
      • Humphreys K
      • Brandeau ML
      Modeling health benefits and harms of public policy responses to the U.S. opioid epidemic.
      These teams face the same data challenges that all opioid researchers do and that have been well documented in the literature.
      • Smart R
      • Kase CA
      • Taylor EA
      • Lumsden S
      • Smith SR
      • Stein BD
      Strengths and weaknesses of existing data sources to support research to address the opioids crisis.
      ,
      • Smart R
      • Kase CA
      • Meyer A
      • Stein BD
      Data sources and data-linking strategies to support research to address the opioid crisis.
      • Nuamah JK
      • Sasangohar F
      • Erraguntla M
      • Mehta RK
      The past, present and future of opioid withdrawal assessment: a scoping review of scales and technologies.
      • Kim SC
      • Bateman BT.
      Methodological challenges in conducting large-scale real-world data analyses on opioid use in musculoskeletal disorders.

      Weiner J, Bao Y, Meisel Z. Prescription drug monitoring programs: evolution and evidence. LDI/CHERISH Issue Brief. Philadelphia, PA: Penn LDI. https://psnet.ahrq.gov/issue/prescription-drug-monitoring-programs-evolution-and-evidence. Published June 2017. Accessed 8 September 2020.

      For instance, some of the data systems that have been used to monitor opioid misuse have been terminated or significantly modified in recent years (e.g., the Arrestee Drug Abuse Monitoring System and the Drug Abuse Warning Network), and national data on the number of people with OUD who access evidence-based treatment such as medications for OUD are nonexistent, let alone data on treatment outcomes.
      National Academies of Sciences, Engineering, and Medicine
      Medications for Opioid Use Disorder Save Lives.
      Notwithstanding these significant shared challenges among all opioid researchers, data needs for simulation models in particular have not been documented systematically. This lack of research attention necessitates beginning conversations among stakeholders to identify the best opportunities for collaboration and improvement of the opioid data landscape.
      To better guide future opioid crisis research, this paper introduces some of the data issues facing several teams of researchers who are actively developing national systems models of the opioid crisis. Using a convening meeting among stakeholders as the basis, this paper provides a description of quantitative data sources that might also be used for such efforts, describes the challenges involved with using each source, and provides some recommendations for researchers interested in moving existing modeling efforts forward, in the hopes of evaluating policies to reduce the negative social, economic, and health impacts of the crisis.

      METHODS

      To foster the development of a broader community of opioid modeling expertise, the U.S. Food and Drug Administration, in collaboration with the National Institute on Drug Abuse and Centers for Disease Control and Prevention, hosted an interagency meeting on April 29, 2019 that initiated an information exchange of key national data sources, data needs, and data considerations for the development of systems models; Appendix Section S1, available online, describes 3 models developed by these organizations. Notwithstanding that qualitative data are critical to systems modeling, the meeting focused on national-level quantitative data and alignment around the use of available sources. The discussions took place in a 1-day event involving a diverse group of 79 experts from 8 federal agencies and 16 research institutions. Of the participants, 32% were addiction and opioids experts, 22% were modeling experts, 19% were data experts, 13% were government public health experts, and 7% each were decision scientists and health economists. A total of 25% of attendees hold professorships. Appendix Section S2, available online, provides information about the attendees.
      The meeting was designed to generate a variety of viewpoints. During the first half of the meeting, the modeling teams provided an overview of their approaches, and a panel of data experts presented on potential data sources. In the second half, attendees participated in breakout sessions to identify data needs and possible strategies to address data challenges. Finally, the group discussed priority data needs for opioid systems modeling and strategies for continued collaboration; Appendix Section S3, available online, presents the meeting agenda.
      This synthesis highlights key takeaway messages from the meeting notes in the context of ongoing opioid systems modeling work, in which data challenges continue to arise. Agreement with individual points was not explicitly tracked, and points may not reflect consensus among group participants, but no participants voiced significant dissent to any of these points.

      RESULTS

      Table 1 summarizes the main data sets discussed, along with some of the key considerations associated with each. This table is presented as a summary of meeting discussions and as a resource for other opioid systems modelers who are exploring new data sources; the information presented in the table represents years of cumulative experience with these data sets. Changes in data collection methods (e.g., change of questions in surveys) and which states contribute their data to the databases in a given year introduce trend breaks that pose a further limitation. Appendix Table S1, available online, provides additional information about databases with descriptions from their publishers.
      Table 1Key Considerations in Data Sources
      ConsiderationSourceYears
      The years of availability in Table 1 reflect the years data are available in any form. AHRQ, Agency for Healthcare Research and Quality; CDC, Centers for Disease Control and Prevention; CMS, Centers for Medicare and Medicaid Services; DEA, Drug Enforcement Administration; DOT, Department of Transportation; EMS, emergency medical services; MME, morphine milligram equivalent; MOUD, medication for opioid use disorder; NIS, Nationwide Inpatient Sample; NPA, National Prescription Audit; NSP, National Sales Perspective; OUD, opioid use disorder; SAMHSA, Substance Abuse and Mental Health Services Administration.
      AvailabilityExample parametersKey considerations
      Prescription opioid utilization data
      Medicare Part D

      CMS
      2013–2017Reports, public-release research data setTotal opioid prescriptions paid for by Medicare Part D per year.Limited lag, includes misclassifications. Excludes prescriptions paid for by other insurers or with cash. Linking prescription and treatment data is not feasible, except for data by CMS, but these were impacted by CMS redaction (2013–2017).
      Symphony Health

      PRA Health Sciences, Inc.
      2009–2016Proprietary data set, accessible with feeTotal individuals receiving ≥5 opioid prescriptions in a given year.Reported with limited lag and contain filled prescriptions regardless of the source of payment. Cannot be linked to other medical treatment, including behavioral therapies for OUD.
      NPA, NSP

      IQVIA, Inc.
      1990s–2020Proprietary data set, accessible with feeTotal opioid prescriptions filled by mail order pharmacies. Total MOUD prescriptions filled by retail pharmacies.Reported with limited lag time and contain filled prescriptions regardless of the source of payment. Cannot be linked to other medical treatment, including behavioral therapies for OUD. Some issues with unit measurement and overlap between what NPA/NSP cover.
      ARCOS: Automation of Reports and Consolidated Orders System

      DEA
      2006–2019Reports, selected restricted use data filesTotal prescription opioids manufactured (by class). Locations to which these prescriptions were distributed.Excludes transfer of substances from retailer to patient. Many distributors submit written forms which contain errors. ARCOS was formerly not publicly available at a localized geographic level (i.e., ZIP code) and information by drug is not public, just by drug class, so cannot standardize by MME.
      MEPS-HC: Medical Expenditure Panel Household Component

      AHRQ
      1996–2017Reports, public-release research data setTotal number of individuals with an opioid prescription. Average number of days’ supply in an opioid prescription.Does not include information on inpatient drugs, institutionalized populations, or misuse of prescription drugs. Drugs for brief acute conditions are more likely to be underreported by patient reporters.
      Illicit use of prescription opioids and heroin data
      NSDUH: the National Survey on Drug Use and Health

      SAMHSA
      2002–2017Reports, Public-Release research data set. Restricted data set accessible with special application at federal facilitiesTotal number of people who misuse prescription opioids or heroin. Total number of people who have an opioid or heroin-use disorder.Restricted NSDUH includes location and dates for better calculation of substance use initiation. Nonmedical prescription opioid use definition changed in 2015, introducing a trend break. Underreporting likely based on numerous factors.
      NFLIS: National Forensic Laboratory Information System

      DEA
      2007–2018Reports, selected restricted use data filesPercentage of illicit substances, including opioids, identified and secured in law enforcement operations that contain fentanyl.Details identified in the field are not included unless confirmed in the lab. Reporting schedules and level of detail vary across labs. Most labs report with significant delays.
      Opioid addiction treatment data
      TEDS: The Treatment Episode Data Set

      SAMHSA
      2006–2017 (discharge)

      1992–2017 (admission)
      Reports, public-release research data setPercentage of opioid use treatment episodes where heroin is the primary substance of abuse. Percentage of treatment episodes for heroin where the individual also indicated problematic use of benzodiazepines.Length of Stay data are a mix of continuous and categorical, which limits analysis of treatment duration. Captures treatment episodes, so does not reflect repeat visits. Only some states include private facilities and individual practitioners; details of this are not included in the data set.
      NSDUH: the National Survey on Drug Use and Health

      SAMHSA
      2002–2017Reports, Public-Release research data set. Restricted data set accessible with special application at federal facilitiesTotal number of people whose last treatment episode was for their use of heroin.Does not include detailed information on MOUDs received during treatment or treatment duration.
      N-SSATS: National Survey of Substance Abuse Treatment Services

      SAMHSA
      2000–2018Reports, public-release research data setTotal individuals receiving buprenorphine, methadone, or naltrexone per year (reported in most years).Does not distinguish repeat visits. Data include categorical numbers of patients per center (e.g., “25–50”), which requires the use of annual reports to find total persons. Census of all treatment facilities, but response rates range year to year (60%–90%). Size of clinics missed is unknown, thus estimating underreporting is impossible.
      Overdose, hospitalizations, mortality data
      NIS component of HCUP: Healthcare Cost and Utilization Project

      AHRQ
      1988–2017Reports, public-release research data setTotal number of hospitalizations involving opioid poisoning/overdoses per year. Total number of hospitalizations involving opioid poisoning/overdoses per year resulting in a fatality.Forty-eight states and the District of Columbia report to HCUP, but states report different levels of information and data are not state representative. HCUP also includes the State Inpatient Database (SID), but not all states contribute to this.
      NEMSIS: National Emergency Medical Services Information System

      University of Utah and DOT
      2006–2018Reports, public-release research data setFraction of overdose events involving EMS that are fatal versus nonfatal.NEMSIS does not distinguish between individuals so shows 2 entries for transport and inpatient care of the same individual. Reports the care provided, not the diagnoses.
      Drug Overdose Mortality Data – NVSS: National Vital Statistics System

      CDC
      1968–2018Reports, public-release research data set as well as restricted use data setFraction of fatal overdoses involving fentanyl alone versus fentanyl and benzodiazepines.Coded in ICD-10, which provides categories, not specific drugs. Information about intent (misuse, suicide, etc.) difficult to reflect. Multiple substances are included if present, but most lethal substance is not specified. Reporting detail and frequency vary by locality.
      a The years of availability in Table 1 reflect the years data are available in any form.AHRQ, Agency for Healthcare Research and Quality; CDC, Centers for Disease Control and Prevention; CMS, Centers for Medicare and Medicaid Services; DEA, Drug Enforcement Administration; DOT, Department of Transportation; EMS, emergency medical services; MME, morphine milligram equivalent; MOUD, medication for opioid use disorder; NIS, Nationwide Inpatient Sample; NPA, National Prescription Audit; NSP, National Sales Perspective; OUD, opioid use disorder; SAMHSA, Substance Abuse and Mental Health Services Administration.
      Time constraints limited the sources presented during the meeting, so Table 1 is not an exhaustive list of relevant data sources; notable omissions include data from the Bureau of Criminal Investigation and the Poison Control Centers. Also, Table 1 provides national (U.S.)-level data sources; however, several meeting participants noted the need for similar data on state and regional levels. Pennsylvania, Illinois, Minnesota, and Delaware are among the states working to develop opioid dashboards to better inform public health and public safety officials of the needs of local communities. These states have used resources to collect the raw data underlying these dashboards (e.g., estimates of nonfatal overdoses through police or emergency medical services reports, emergency department diagnoses).
      Further discussion addressed 4 aspects of the opioid crisis, which are outlined in Table 2: (1) use, misuse, and use disorder; (2) nonfatal overdose; (3) illicit opioid supply and demand; and (4) treatment utilization, outcomes, and relapse. The discussion focused on national data available to model the respective aspect of the crisis, challenges associated with the available data, additional data that are needed, and priority actions that may help address challenges and needs.
      Table 2Key Discussion Points and Potential Paths Forward
      AreaMain discussion pointsPotential paths forward
      Use, misuse, and use disorder
      Inconsistency in definition and measurement is a significant problem for system modelers, who need to identify parameter values that correspond to a single construct.

      Available data are difficult to interpret, especially contextually. The practice in systems modeling of triangulating data sources surfaces contradictions that threaten the internal and external validity of the model.

      Details about which individuals were included in data sources and why are often unclear or missing. This is challenging for systems models that rely on defining mutually exclusive groups or “types” of people.

      In addition to prevalence data, modelers need incidence data. Understanding incidence rates is critical to capturing and validating dynamics over time.

      Data on the outcomes of marginalized individuals who use opioids, including criminal justice–involved, poor and homeless, and those with co-occurring mental illness, are particularly difficult to find. The lack of data reduces the ability to test targeted policies and address inequities.

      There is a great need for longitudinal data collected on individuals to give insight into interrelationships of variables over time and their unintended outcomes. In addition, establishing the validity of models depends in part on being able to replicate historical trends.
      Standard list of definitions for prioritized terms and variables.

      Explore innovative efforts to triangulate data sources, harness new and/or novel data sources like social media, and employ synthetic datasets.
      Nonfatal overdose
      The incidence of nonfatal overdose is a key data need. Policy interventions tested in a model meant to reduce fatal overdose are compared to a (highly uncertain) baseline level of overdose and survival; reducing uncertainty here is critical.

      Existing data collection practices limit the accuracy of available proxies for overdose. The size of the underreporting and overreporting margins is unclear.
      Text mining through qualitative EMS reports may identify overdoses.

      Incentives that encourage providers to report the identification of overdose and submit the data frequently.

      Implementation of Good Samaritan laws to encourage the use of EMS.

      Improved collaboration between modelers with data systems like PCORnet and Enhanced State Opioid Overdose Surveillance (ESOOS) that are developing overdose metrics.
      Illicit opioid supply and demand
      There is limited data available regarding sources of illicit opioids (e.g., laboratories, drug trafficking, or diversion). Such data would allow for more targeted testing of interventions in systems models.

      There is limited data available regarding the volume and price of illicit opioids, which affects availability of drugs and thus key transition rates (e.g., initiation, escalation) that systems models use to model dynamics.

      Estimating opioid diversion nationally should be a priority. In systems modeling, these underlying market dynamics are critical to understand shifts in behavior over time.

      Opioid data alone are insufficient; it is critical to note that heroin-use interventions may have unintended consequences for other substances (e.g., cocaine).
      Programs that collect information about purity, street price, and volume of illicit substances.

      Making computable forms of data publicly accessible by data providers.
      Treatment utilization, outcomes, and relapse
      Definitions and measurement of key treatment variables are inconsistent; modelers should agree upon these to facilitate quantification and communication.

      Efforts to define nebulous variables like recovery and appropriate duration of medications for opioid use disorder (MOUD) should keep in mind the potential effects of terms and definitions on stigma. Systems modelers should seek partnerships with OUD experts who are aware of these issues.

      There is limited and interspersed data on treatment history, relapse, and long-term remission outcomes. This makes defining transition rates between states nearly impossible, adding uncertainty to systems models.

      Available data are limited by the caveats of conducting small controlled trials with specific populations. Modelers need data that can be reasonably generalized to larger populations. Propensity scores can be used to generalize data, but this process requires detailed patient data.
      Explore treatment outcomes piecemeal, starting with claims data.

      Use limited treatment utilization data to fill other information gaps.

      Utilize state treatment administrative databases that track unique individuals to obtain information on relapse and length of recovery.

      Develop a process to incorporate and triangulate claims data and clinical trial data.
      EMS, emergency medical services; OUD, opioid use disorder.
      In summary, the main issues raised revolved around definitional inconsistencies; general lack of data; and issues with overestimation, underestimation, and potential overlap between data sources. In each case, the path forward included the development of mutually exclusive and exhaustive definitions, which are required for systems models where individuals or their aggregates drive the key dynamics, rather than analysis of behavior. There was significant discussion around the need for incentives and support for expanded data collection and access to that data (Table 2).
      The summary of perspectives in Table 2 includes points raised by meeting attendees present at each discussion group. In addition, a key point was raised that the reliance on data to build models can inadvertently lead to neglecting marginalized subpopulations, including people who are incarcerated or homeless or who have co-occurring mental illnesses and polysubstance use, who are not represented well in data sources. Although difficult to incorporate owing to the lack of data and their nuanced role in the system, these factors have important and dynamic effects on treatment decisions, initiation, durability, and recovery. Supplementary modeling projects and conceptual models may be helpful tools for considering data-scarce topics of interest without compromising the quality of system-level models.

      DISCUSSION

      This meeting provided a foundation for understanding the landscape of relevant quantitative, national-level data sources for the opioid crisis in the U.S., identifying existing and anticipated data obstacles to modeling efforts, and establishing relationships to foster future communication among key stakeholders.
      Ideally, opioid systems models would utilize data that are geographically and demographically representative of the modeled populations and consistently collected over time. When these data exist, a model can better reflect the complexity of the underlying phenomena and reproduce historical system behavior without the need for too many assumptions or caveats. Model performance and validity can then be assessed on whether and to what degree the model reproduces these historical trends in a theoretically justifiable way; guidelines surrounding evaluation of model performance are further explored elsewhere.
      • Sterman JD.
      Business Dynamics: Systems Thinking and Modeling for a Complex World.
      ,
      • Railsback SF
      • Grimm V.
      Agent-Based and Individual-Based Modeling: A Practical Introduction.
      This allows for more complete analyses of potential outcomes of interventions.
      However, modelers encounter common challenges in accessing such data. First, no single data source offers all the qualities described here, particularly when looking at impacts on chronic disease that progress at varying rates depending on individual behavior. Second, definitions and methods used for measuring relevant aspects of the phenomena often vary across data sources. Third, available data often provide only approximations and are prone to errors and biases. Fourth, differences in data collection practices, as well as changes in those practices over time, make it difficult to distinguish true historical trends from reporting changes, which complicates model calibration.
      Meeting participants identified several opportunities to enhance their collective efforts to better understand and address the opioids crisis. The next steps summarized hereafter pertain to modelers, data owners, and policymakers, and it is at the discretion of those stakeholders to implement them. In addition, public health prevention experts can play an important role in advocating for the development or funding of each initiative or convening the relevant experts to help make them happen.

      Maintain Close, Direct, and Regular Collaboration Among Modeling Teams

      Doing so is critical to ensuring the alignment, efficiency, and collective value of the models. Modelers are already communicating to a limited extent via informal channels to share insights and build upon each other's work. As specific discussion topics or needs arise, the community should consider the value of more formal communication via regular webinars, workgroups, or other avenues. As this effort matures, this may entail mechanisms more broadly for the research community that foster collaboration directly supporting the goals of the opioid systems models.

      Enhance Data Collection to Better Fit Modeling and Research Needs

      There would be value in federal, state, and local governments continuing to develop mechanisms to: (1) better capture data surrounding the opioid crisis in its entirety, including marginalized or excluded populations as well as populations exposed through medical channels; (2) better measure opioid-related events and phenomena (e.g., nonfatal overdoses, average duration of medication receipt for successful treatment) that are insufficiently reflected in current data sources; (3) develop data collection mechanisms that would capture at-risk populations longitudinally; (4) increase the timeliness of data dissemination; (5) provide the data in formats that allow for efficient incorporation into models; and (6) increase the accessibility of existing available data (e.g., National Survey on Drug Use and Health geocoded data; Centers for Medicare and Medicaid Services redacted data [2013–2017]; System to Retrieve Information from Drug Evidence, National Forensic Laboratory Information System, and Automation of Reports and Consolidated Order System data; linked admissions/discharge data from Treatment Episode Data Set; and linked facility/treatment data from National Survey of Substance Abuse Treatment Services and Treatment Episode Data Set). Furthermore, developing a standard list of definitions for prioritized terms and variables through a dedicated, expert working group could provide a foundation for increased consistency across state and local governments. Although the authors are not the first to call for standardized language,

      National Center on Addiction and Substance Abuse at Columbia University. Addiction medicine: closing the gap between science and practice. New York, NY: Partnership to End Addiction. https://drugfree.org/reports/addiction-medicine-closing-the-gap-between-science-and-practice. Published June 2012. Accessed 1 August 2020.

      the need is particularly urgent for systems modeling because model performance and validation is predicated on reducing the amount of uncertainty and number of assumptions made. Unclear and nonstandardized definitions of common phenomena represent an unnecessary source of uncertainty.

      Focus on Ways to Bridge the Most Crucial Information Gaps

      Obtaining, revising, cleaning, and adjusting data to the particular needs of a specific model can be very resource intensive. In addition, information contained in many available data sources does not precisely align with the measured constructs in models, so modelers will need to move forward with imperfect or incomplete data. The triangulation of current with new data sets, along with using text-mining technology, will be essential for this process.

      Engage in Direct and Regular Interaction Among Modelers, Data Experts, and Public Health Experts

      Increased communication about modeler priorities and data collection challenges could facilitate the availability or awareness of better data in the future. Collaboration could also help modelers develop innovative ways to apply the data currently available. Priority issues for collaboration include: (1) data that are standardized across reporting parties and highly granular; (2) data that are as close to real-time as possible and data publication schedules; and (3) data that are easy to extract, search, and manipulate.

      Gain a Clearer Definition of Policymakers’ Research Questions and Policy Goals

      Research questions and overarching goals, along with federal, state, and local governments’ policy priorities, shape the scope, variables of interest, and simulation scenarios of models. Modelers at the April 2019 meeting commented that a second meeting focused on identifying, articulating, and clarifying specific research questions would improve the value of models to federal public health agencies and further help identify data needs for those research questions. Such a meeting is currently being planned for the fall of 2020.
      Achieving this is not an easy task. Realistically, despite best efforts to address data challenges, there will never be perfect data for all areas relevant to the opioid crisis. Even with the best data from sources listed in Table 1, there remain many mechanisms for which longitudinal data are lacking, such as transitions through the stages of OUD development. As one of the biggest strengths of systems modeling is its ability to bridge information from multiple imperfect data sources to tell a comprehensive story about a given phenomenon, there is a tremendous opportunity for modeling efforts to provide better insights into the opioid crisis. To do so will require assumptions just as is true of any other modeling effort. Therefore, careful sensitivity analyses to investigate the effects of those assumptions, based on constant communication and discussions between modelers, data experts, public health experts, and government agencies, will be particularly valuable for guaranteeing products that generate useful information for decision makers.

      Limitations

      This report has limitations. First, the discussions took place in a 1-day event and focused on national-level data; not all data sources and challenges in opioid research were covered. Important data sets that were not mentioned during the meeting include, among others, the National Violent Death Reporting System or the Overdose Detection and Mapping Application Program from the High Intensity Drug Trafficking Areas Program. In addition, any data sets that were local, state, or regional were excluded, such as the Massachusetts Chapter 55 Overdose Report, which has been useful at the state level for guiding research and policy discussions but is not representative of the national opioid system. Also, syringe exchanges may be out of scope of the modeling projects, which made the discussion of harm reduction limited to naloxone. Second, although this summary is derived from comments from a diverse group of 79 experts from the federal government and research institutions, there are many other groups working in this area and continued discussion about how best to develop and utilize data is needed. Finally, bringing attention to data challenges in opioids systems modeling is only a first step. To ensure that these suggestions are implemented, future meetings will need to discuss how to address the data challenges. Overall, given that the authors only reported a summary of meeting discussions, future research can aggregate and enhance the discussion points with findings in other studies.

      CONCLUSIONS

      Systems modeling can be used to address the opioid crisis and prevent overdose deaths. Reliable and valid data are needed to develop these models, but such data are not always available. Data experts, particularly those knowledgeable of public health systems, can play a critical role in systems modeling by lending their expertise and collaborating with modelers and policymakers. This meeting summary provides an important step in increasing the awareness of data challenges and dialogue within and between prevention research communities and government agencies.

      ACKNOWLEDGMENTS

      We thank all participants who attended the meeting and contributed to the discussions. We are also grateful to researchers at the Centers for Disease Control and Prevention who provided Appendix Table S1. Research reported in this article was supported in part by the U.S. Food and Drug Administration ( U01FD006868 ). This article reflects the views of the authors and should not be construed to represent the views or policies of the U.S. Food and Drug Administration or HHS .
      No financial disclosures were reported by the authors of this paper.

      Appendix. SUPPLEMENTAL MATERIAL

      REFERENCES

        • Opioid data analysis and resources
        Centers for Disease Control and Prevention.
        2020 (Updated March 19)
      1. Drug overdose deaths in the United States, 1999–2018.
        Centers for Disease Control and Prevention, 2020 (Updated January 30)
      2. CDC director's media statement on U.S. life expectancy.
        2018 (November 29)
        • HHS, Office of the Surgeon General
        Facing addiction in America: the Surgeon General's report on alcohol, drugs, and health.
        HHS, Washington, DCNovember 2016 (Published)
        • Saloner B
        • McGinty EE
        • Beletsky L
        • et al.
        A public health strategy for the opioid crisis.
        Public Health Rep. 2018; 133: 24S-34S
        • Jalali MS
        • Botticelli M
        • Hwang RC
        • Koh HK
        • McHugh RK
        The opioid crisis: a contextual, social-ecological framework.
        Health Res Policy Syst. 2020; 18: 87
        • DeWeerdt S.
        Tracing the U.S. opioid crisis to its roots.
        Nature. 2019; 573: S10-S12
        • Haffajee RL
        • Mello MM.
        Drug companies’ liability for the opioid epidemic.
        N Engl J Med. 2017; 377: 2301-2305
      3. Leslie DL, Ba DM, Agbese E, Xing X, Liu G. The economic burden of the opioid epidemic on states: the case of Medicaid. Am J Manag Care. 2019;25(13 suppl):S243–S249. https://www.ajmc.com/view/the-economic-burden-opioid-epidemic-on-states-case-of-medicaid. Accessed June 18, 2002.

        • Schatman ME
        • Webster LR.
        The health insurance industry: perpetuating the opioid crisis through policies of cost-containment and profitability.
        J Pain Res. 2015; 8: 153-158
        • Brinkley-Rubinstein L
        • Zaller N
        • Martino S
        • et al.
        Criminal justice continuum for opioid users at risk of overdose.
        Addict Behav. 2018; 86: 104-110
      4. Crowley DM, Connell CM, Jones D, Donovan MW. Considering the child welfare system burden from opioid misuse: research priorities for estimating public costs. Am J Manag Care. 2019;25(13 suppl):S256–S263. https://pubmed.ncbi.nlm.nih.gov/31361428/. Accessed June 18, 2020.

      5. Hagemeier NE. Introduction to the opioid epidemic: the economic burden on the healthcare system and impact on quality of life. Am J Manag Care. 2018;24(10 suppl):S200–S206. https://www.ajmc.com/view/intro-opioid-epidemic-economic-burden-on-healthcare-system-impact-quality-of-life. Accessed June 17, 2020.

        • Soelberg CD
        • Brown Jr, RE
        • Du Vivier D
        • Meyer JE
        • Ramachandran BK
        The U.S. opioid crisis: current federal and state legal issues.
        Anesth Analg. 2017; 125: 1675-1681
        • Dayer LE
        • Painter JT
        • McCain K
        • King J
        • Cullen J
        • Foster HR
        A recent history of opioid use in the U.S.: three decades of change.
        Subst Use Misuse. 2019; 54: 331-339
        • Volkow ND
        • Jones EB
        • Einstein EB
        • Wargo EM
        Prevention and treatment of opioid misuse and addiction: a review.
        JAMA Psychiatry. 2019; 76: 208-216
        • Smart R
        • Kase CA
        • Taylor EA
        • Lumsden S
        • Smith SR
        • Stein BD
        Strengths and weaknesses of existing data sources to support research to address the opioids crisis.
        Prev Med Rep. 2019; 17101015
        • Jalali MS
        • Botticelli M
        • Hwang RC
        • Koh HK
        • McHugh RK
        The opioid crisis: need for systems science research.
        Health Res Policy Syst. 2020; 18: 88
        • Benneyan J
        • Garrahan J
        • Ilieş I
        • Duan X
        Modeling approaches, challenges, and preliminary results for the opioid and heroin co-epidemic crisis.
        in: Paper presented at: 2017 Winter Simulation Conference; December 3–6, Las Vegas, NV2017.
      6. Homer J, Wakeland W. A dynamic model of the opioid drug epidemic with implications for policy. Am J Drug Alcohol Abuse. In press. Online June 9, 2020. https://doi.org/10.1080/00952990.2020.1755677.

        • Volkow ND.
        Medications for opioid use disorder: bridging the gap in care.
        Lancet. 2018; 391: 285-287
        • Foglia MB.
        When people you love are the unintended consequences of opioid policy.
        Health Aff (Millwood). 2019; 38: 2105-2108
        • Sharareh N
        • Sabounchi SS
        • McFarland M
        • Hess R
        Evidence of modeling impact in development of policies for controlling the opioid epidemic and improving public health: a scoping review.
        Subst Abuse. 2019; 131178221819866211
      7. OUD Modeling writing group. How simulation modeling can support the public health response to the opioid crisis in North America: setting priorities and assessing value. Int J Drug Policy. In press. Online April 28, 2020. https://doi.org/10.1016/j.drugpo.2020.102726.

        • Leischow SJ
        • Best A
        • Trochim WM
        • et al.
        Systems thinking to improve the public's health.
        Am J Prev Med. 2008; 35: S196-S203
        • Peters DH.
        The application of systems thinking in health: why use systems thinking?.
        Health Res Policy Syst. 2014; 12: 51
        • Sterman JD.
        Learning from evidence in a complex world.
        Am J Public Health. 2006; 96: 505-514
        • van Ravesteyn NT
        • van den Broek JJ
        • Li X
        • et al.
        Modeling ductal carcinoma in situ (DCIS): an overview of CISNET model approaches.
        Med Decis Making. 2018; 38: 126S-139S
        • Grefenstette JJ
        • Brown ST
        • Rosenfeld R
        • et al.
        FRED (a framework for reconstructing epidemic dynamics): an open-source software system for modeling infectious diseases and control strategies using census-based populations.
        BMC Public Health. 2013; 13: 940
      8. Atkins CY, Meltzer MI, Haber MJ. Community flu: software to estimate the impact of an influenza pandemic on a simulated community with and without interventions. Atlanta, GA: Centers for Disease Control and Prevention. https://stacks.cdc.gov/view/cdc/49287. Published August 10, 2009. Accessed 1 August 2020.

        • Hammond RA.
        Complex systems modeling for obesity research.
        Prev Chronic Dis. 2009; 6: A97
        • Ghaffarzadegan N
        • Larson RC
        • Fingerhut H
        • et al.
        Model-based policy analysis to mitigate post-traumatic stress disorder.
        in: Gil-Garcia J Pardo T Luna Reyes L Policy Analytics, Modelling, and Informatics. Public Administration and Information Technology, vol. 25. Springer, Cham, Switzerland2018: 387-406
        • Mandelblatt JS
        • Near AM
        • Miglioretti DL
        • et al.
        Common model inputs used in CISNET collaborative breast cancer modeling.
        Med Decis Making. 2018; 38: 9S-23S
        • Wakeland W
        • Nielsen A
        • Schmidt TD
        • et al.
        Modeling the impact of simulated educational interventions on the use and abuse of pharmaceutical opioids in the United States: a report on initial efforts.
        Health Educ Behav. 2013; 40: 74S-86S
        • Wakeland W
        • Nielsen A
        • Geissert P
        Dynamic model of nonmedical opioid use trajectories and potential policy interventions.
        Am J Drug Alcohol Abuse. 2015; 41: 508-518
        • Chen Q
        • Larochelle MR
        • Weaver DT
        • et al.
        Prevention of prescription opioid misuse and projected overdose deaths in the United States.
        JAMA Netw Open. 2019; 2e187621
        • Pitt AL
        • Humphreys K
        • Brandeau ML
        Modeling health benefits and harms of public policy responses to the U.S. opioid epidemic.
        Am J Public Health. 2018; 108: 1394-1400
        • Smart R
        • Kase CA
        • Meyer A
        • Stein BD
        Data sources and data-linking strategies to support research to address the opioid crisis.
        HHS, Washington, DCOctober 2018 (Published)
        • Nuamah JK
        • Sasangohar F
        • Erraguntla M
        • Mehta RK
        The past, present and future of opioid withdrawal assessment: a scoping review of scales and technologies.
        BMC Med Inform Decis Mak. 2019; 19: 113
        • Kim SC
        • Bateman BT.
        Methodological challenges in conducting large-scale real-world data analyses on opioid use in musculoskeletal disorders.
        J Bone Joint Surg Am. 2020; 102: S10-S14
      9. Weiner J, Bao Y, Meisel Z. Prescription drug monitoring programs: evolution and evidence. LDI/CHERISH Issue Brief. Philadelphia, PA: Penn LDI. https://psnet.ahrq.gov/issue/prescription-drug-monitoring-programs-evolution-and-evidence. Published June 2017. Accessed 8 September 2020.

        • National Academies of Sciences, Engineering, and Medicine
        Medications for Opioid Use Disorder Save Lives.
        The National Academies Press, Washington, DC2019
        • Sterman JD.
        Business Dynamics: Systems Thinking and Modeling for a Complex World.
        Irwin/McGraw-Hill, Boston2000
        • Railsback SF
        • Grimm V.
        Agent-Based and Individual-Based Modeling: A Practical Introduction.
        Princeton University Press, Princeton, NJ2011
      10. National Center on Addiction and Substance Abuse at Columbia University. Addiction medicine: closing the gap between science and practice. New York, NY: Partnership to End Addiction. https://drugfree.org/reports/addiction-medicine-closing-the-gap-between-science-and-practice. Published June 2012. Accessed 1 August 2020.