Advertisement

An Economic Evaluation of PulseNet

A Network for Foodborne Disease Surveillance
Open AccessPublished:March 15, 2016DOI:https://doi.org/10.1016/j.amepre.2015.09.018
      The PulseNet surveillance system is a molecular subtyping network of public health and food regulatory agency laboratories designed to identify and facilitate investigation of foodborne illness outbreaks. This study estimates health and economic impacts associated with PulseNet. The staggered adoption of PulseNet across the states offers a natural experiment to evaluate its effectiveness, which is measured as reduction of reported illnesses due to improved information, enhanced industry accountability, and more-rapid recalls. Economic impacts attributable to PulseNet include medical costs and productivity losses averted due to reduced illness. Program costs are also reported. Better information and accountability from enhanced surveillance is associated with large reductions of reported illnesses. Data collected between 1994 and 2009 were assembled and analyzed between 2010 and 2015. Conservatively, accounting for underreporting and underdiagnosis, 266,522 illnesses from Salmonella, 9,489 illnesses from Escherichia coli (E. coli), and 56 illnesses due to Listeria monocytogenes are avoided annually. This reduces medical and productivity costs by $507 million. Additionally, direct effects from improved recalls reduce illnesses from E. coli by 2,819 and Salmonella by 16,994, leading to $37 million in costs averted. Annual costs to public health agencies are $7.3 million. The PulseNet system makes possible the identification of food safety risks by detecting widespread or non-focal outbreaks. This gives stakeholders information for informed decision making and provides a powerful incentive for industry. Furthermore, PulseNet enhances the focus of regulatory agencies and limits the impact of outbreaks. The health and economic benefits from PulseNet and the foodborne disease surveillance system are substantial.

      Introduction

      Foodborne illness continues to be an important source of morbidity and mortality in the U.S. Annually, approximately 48 million illnesses resulting in 128,000 hospitalizations and 3,000 deaths are caused by contaminated food.
      • Scallan E.
      • Hoekstra R.M.
      • Angulo F.J.
      • et al.
      Foodborne illness acquired in the United States - major pathogens.
      • Scallan E.
      • Griffin P.M.
      • Angulo F.J.
      • et al.
      Foodborne illness acquired in the United States - unspecified agents.
      In response to this problem, the Centers for Disease Control and Prevention (CDC) created PulseNet: a molecular subtyping network of federal, state, and local public health laboratories designed to facilitate the identification of and response to outbreaks caused by bacterial foodborne pathogens. The development of a national database of molecular “fingerprints” for clinical isolates of foodborne pathogens utilizing uniform standards and protocols has allowed for seemingly sporadic cases of foodborne illness to be linked to previously unidentifiable outbreaks. The specific objectives of PulseNet are to detect foodborne disease case clusters through comparison of pulsed field gel electrophoresis (PFGE) patterns; to allow for real-time communication between federal, state, and local health departments; to facilitate early identification of common source outbreaks; and to help food regulatory agencies identify areas where implementation of new measures is likely to improve the safety of the food supply.

      CDC. PulseNet. Accessed September 28, 2015. www.cdc.gov/pulsenet/

      The goals of this study were to estimate illness reduction attributable to PulseNet and assess the costs and benefits associated with the PulseNet program. The study focuses on program effectiveness for Salmonella, shiga toxin–producing Escherichia coli (STEC) and Listeria monocytogenes— a subset of important pathogens that have been a part of PulseNet for an amount of time sufficient to allow for empirical analysis. Results from this analysis will help public health officials, policymakers, and the public assess the health and economic returns from maintaining, and potentially even expanding, the PulseNet program.

      Methods

      Two approaches were used to assess the number of illnesses averted due to PulseNet. First, a “Recall” model was developed to assess the direct effects of faster identification of outbreaks on consumption of contaminated product and the resultant illness reduction. Second, a “Process Change” model was designed to capture the indirect effects from enhanced outbreak identification on illnesses averted due to new incentives and information used by industry and government. A comprehensive Monte Carlo analysis using @Risk, version 5.7.1, was performed to account for uncertainty in both the illness and economic models. Credible intervals are reported for terminal values across alternative models. A conservative cost of illness analysis is then used to produce summary measures for burden of illness averted. Data for this study were assembled and analyzed between 2010 and 2015. An enhanced description of the methodology is available in the Appendix (available online).

      The Recall Model

      The primary direct effect from PulseNet is earlier identification and recall of contaminated foods, leading to a reduced number of illnesses associated with the contaminated product. Specifically, contaminated food products in interstate commerce may cause illnesses across multiple states. Aggregation of cases by PulseNet reduces the time until the outbreak is recognized and increases the likelihood of preventing additional exposures. To measure this, two Monte Carlo (probabilistic) models were built to estimate the number of cases prevented through recalling contaminated ground beef products associated with E. coli O157:H7 and multistate Salmonella outbreaks detected by the PulseNet system for the period 2007–2008, which were the most current years with data available at the start of the project. Separate models were developed for E. coli O157:H7 and Salmonella outbreaks owing to differences in the data available from which to build the models. The E. coli O157 model specifically looked at ground beef because it was implicated in 19 of 29 (66%) E. coli O157 outbreaks with a confirmed food vehicle and because the second most common vehicle was leafy greens (eight outbreaks), which are not typically identified early enough to prevent cases by recalls.
      • Painter J.A.
      • Hoekstra R.M.
      • Ayers T.
      • et al.
      Attribution of foodborne illnesses, hospitalizations, and deaths to food commodities by using outbreak data, United States, 1998-2008.
      For the first model, U.S. Department of Agriculture Food Safety and Inspection Service recall records were reviewed to identify recalls related to E. coli O157:H7 outbreaks in which ground beef products were implicated as a vehicle.
      • Seys S.A.
      • Sampedro F.
      • Hedberg C.W.
      Assessment of shiga toxin-producing Escherichia coli O157 illnesses prevented by recalls of beef products.
      For each recall, the number of pounds of consumed ground beef was calculated by subtracting the amount of product recovered from the amount of recalled product, reduced by 32% to account for product loss at the consumer level.

      USDA-ERS. Loss-adjusted food availability. www.ers.usda.gov/Data/FoodConsumption/FoodGuideSpreadsheets.htm. Accessed September 28, 2015.

      The number of outbreak-associated cases was determined by reviewing published outbreak summaries, the CDC Foodborne Outbreak Response and Surveillance database, and the PulseNet database to identify isolates with PFGE patterns matching the outbreak strain. This allowed for the determination of the number of cases that were reported, confirmed, and subtyped by PFGE for each outbreak. The rate of E. coli O157:H7 outbreak–associated cases per 1,000 pounds of product consumed was calculated, and this rate was multiplied by the amount of product recovered to determine the number of E. coli O157:H7 cases averted in each outbreak. Although each outbreak-specific attack rate was treated as a constant over the course of the outbreak, because the value is derived from outbreak-specific parameters, it functionally averages variability in attack rate differences that would have occurred over the course of the outbreak. Beta distributions were fit to each of the input variables for which uncertainty existed: pounds of distributed product, pounds of recovered product, fraction of consumed product, and the number of reported outbreak cases. Ninety percent ranges for the number of cases averted per outbreak were determined by Monte Carlo simulation with 10,000 iterations per outbreak using @Risk, version 5.7.1.
      In contrast to Food Safety and Inspection Service recalls, the U.S. Food and Drug Administration does not routinely collect or report information that would allow an estimation of the amount of distributed or consumed product. Therefore, the number of Salmonella cases was calculated by modeling the difference between observed and expected cases reported following initiation of the product recall. The analysis of U.S. Food and Drug Administration–recorded recalls was limited to those in which the recalled food item was identified as a result of an outbreak investigation.
      To estimate the number of cases expected to occur, the number of reported cases was counted during each week of the outbreak. The duration of the outbreak was determined on the basis of the number of weeks between onset of the first and last outbreak-associated cases. Outbreak-specific negative binomial distribution parameters were used to generate an expected range of cases per week for the remainder of the outbreak period, with the ranges bounded by zero and the highest weekly count of reported cases.
      • Scallan E.
      • Hoekstra R.M.
      • Angulo F.J.
      • et al.
      Foodborne illness acquired in the United States - major pathogens.
      Salmonella cases averted in each outbreak were calculated as ∑ (expected cases, week i – reported cases, week i) for all weeks from initiation of the recall to the end of the outbreak.
      Ninety percent ranges for the number of cases averted in each outbreak were determined by Monte Carlo simulation with 10,000 iterations per outbreak using @Risk, version 5.7.1.

      The Process Change Model

      The process change model was developed to capture indirect effects of the PulseNet system on incidence of illness. Specifically, better identification of the source of outbreaks provides information to regulators and industry, while creating an enhanced incentive for industry members to implement practices that reduce the risks associated with their foods. The fact that PulseNet was adopted by states to different degrees in different years creates ideal conditions for evaluating PulseNet as a natural experiment. The result of this process change is expected to be fewer foodborne illnesses experienced and reported.
      The following three specifications were used to test the effect of PulseNet on reported illnesses:
      Illnessesist=β1Pulseist+β2Isoist+β3Isoist1+δXst+εiIllnessesist=αist+β1Pulseist+β2Isoist+β3Isoist1+δXst+εistProb(Illnesses=Illnessesist)=exp(λist)(λist)illnessesistillnessesist!,y=1,2,...;wherelogλist=β1Pulseist+β2Isoist+β3Isoist1+δXst+εi
      (1)


      In equation 1, Illnessesist = number of illnesses recorded by National Notifiable Diseases Surveillance System (NNDSS) by pathogen i in state s at time t, Isoist = number of isolates tested in the state lab at time t, Isoist-1 = number of isolates tested in the state lab at time t – 1, Pulseist = PulseNet implemented for pathogen i in state s at time t × population, and Xst = a set of potentially confounding variables (Table 1).
      Table 1Descriptive Statistics of Sample Used
      VariableDescriptionMeanSD
      Illnesses Reported to NNDSS
      Salmonella illnessesSalmonella illnesses at time t in state s869.2921,034.520
      E. coli illnesses
      E. coli O157:H7 estimates based on 561 observations (1994–2005).
      E. coli O157 illnesses at time t in state s64.20775.240
      Listeria illnesess
      Listeriosis estimates based on 501 observations (2000–2009).
      Listeria illnesses at time t in state s15.32921.934
      Tests Reported by Labs
      Salmonella isolatesSalmonella isolates at time t in state s304.538488.820
       STEC isolates
      E. coli O157:H7 estimates based on 561 observations (1994–2005).
      STEC isolates at time t in state s31.13755.840
      Listeria isolates
      Listeriosis estimates based on 501 observations (2000–2009).
      Listeria isolates at time t in state s10.44520.641
      State Participation in PulseNet
      0–1 dummy variable equals 1 if state has implemented PulseNet at time t0.6310.483
      Controls
       State PopulationPopulation at time t in state s5,570,4996,206,105
       Year (t)(Calendar year – 1993)8.5004.613
       PCIState per capita personal income at time t36,1496,331
      Regions
       New England(0–1 dummy variable)0.1180.322
       Middle Atlantic(0–1 dummy variable)0.0590.235
       East North Central(0–1 dummy variable)0.0980.298
       West North Central(0–1 dummy variable)0.1370.344
       South Atlantic(0–1 dummy variable)0.1570.364
       East South Central(0–1 dummy variable)0.0780.269
       West South Central(0–1 dummy variable)0.0780.269
       Mountain(0–1 dummy variable)0.1570.364
       Pacific(0–1 dummy variable)0.0980.298
      Number of Observations = 816
      STEC, shiga toxin–producing Escherichia coli; PCI, per capita income; NNDSS, National Notifiable Diseases Surveillance System.
      a E. coli O157:H7 estimates based on 561 observations (1994–2005).
      b Listeriosis estimates based on 501 observations (2000–2009).
      Reported illnesses are a function of the existence of PulseNet, the number of tested isolates, both past and present, and a set of variables which may impact adoption and effectiveness of changes in food handling and production.

      U.S. Census Bureau. Table 1: Annual estimates of the population for the United States, regions, and states and for Puerto Rico: April 1, 2000 to July 1, 2009 (NST-EST2009-01). 2009. www.census.gov/popest/data/historical/2000s/vintage_2009/. Accessed September 28, 2015.

      Bureau of Economic Analysis, U.S. Department of Commerce. State per capita personal income 2010. www.bea.gov/newsreleases/regional/spi/2011/pdf/spi0911pc_fax.pdf. Accessed September 28, 2015.

      In the first equation, no intercept is specified because it was assumed that the pre-PulseNet number of illnesses in a state was essentially independent of population size. The second equation includes an intercept, allowing for baseline (pre-PulseNet) differences in number of cases. The third equation is a maximum likelihood Poisson model, designed for estimating count data models (such as the number of illnesses in a given state at a given time). An alternative (spillover effects) specification of this model includes a variable, Iso-ist, to measure the reduction of pathogen i owing to process changes resulting from identification of other pathogens –i. Essentially, this model asserts that efforts aimed at reducing one pathogen will also reduce risks from other pathogens. For example, a facility’s improved sanitation measures in response to a highly publicized Listeria outbreak will also reduce the risk in that facility from Salmonella and E. Coli.
      The PulseNet system currently collects data for six pathogens (Salmonella, STEC, Listeria, Campylobacter, Shigella, and Yersinia). This study focused, however, on Salmonella, STEC, and Listeria because these pathogens were the first to be adopted by PulseNet states and, as a result, have a sufficient number of observations to allow for empirical analysis.
      To populate the model, data were collected from a number of sources for the period 1994–2009. The number of reported illnesses from each pathogen in each state was obtained from the NNDSS.

      CDC. National Notifiable Diseases Surveillance System. 2015. http://wwwn.cdc.gov/nndss/. Accessed September 28, 2015.

      MMWR: Summary of Notifiable Diseases. CDC. http://www.cdc.gov/mmwr/mmwr_nd/index.html. Updated September 8, 2015.

      Salmonella data were available for each state for all years of surveillance (1994–2009). E. coli O157 data for the years 1994–2005 were used in the analysis. In 2006, E. coli O157 was replaced in the NNDSS by STEC, which could not be used because it is a broader category of pathogens. Reports for illnesses caused by Listeria were available for most states from 2000 to 2009. Isolates tested and reported to PulseNet from 1993 to 2009 for each pathogen were obtained through personal correspondence (P. Gerner-Smidt, CDC, written communication, 2010). For each of these variables, a “lagged isolate” variable was created. The number of usable observations ranged from 501 (for Listeria) to 816 (for Salmonella). The three models detailed in Equation 1 were estimated using LIMDEP, version 9.0 econometric software.
      National illness reduction was calculated as “PulseNet Implemented” × 51 (the marginal effect of PulseNet implementation for the average state times the number of states in which PulseNet has been implemented) + “Lagged Isolates Tested” × number of illnesses for pathogen i reported in the U.S. in 2009. For the spillover model, this calculation included the marginal effects of lagged isolate testing for each of the three pathogens.
      For each pathogen, the number of reported illnesses is an underestimate of the true number of experienced illnesses. Alternative benefits estimates that are adjusted to account for underreporting and underdiagnosis are provided. A multiplier for reported cases is taken from Scallan et al. (29.3 for Salmonella, 26.1 for E. coli O157, and 2.1 for Listeria).

      Economic Impacts of PulseNet: Program Costs and Costs Averted Due to Reduced Foodborne Illnesses

      Program costs associated with implementing PulseNet are generated as a result of laboratory setup, isolate testing, and outbreak response, borne both by government and industry. Costs averted due to PulseNet include medical costs and productivity losses associated with reduced mortality and morbidity from foodborne illnesses. These direct and indirect costs averted for each illness are reported in 2010 dollars.

      U.S. Department of Labor, Bureau of Labor Statistics. May 2010 national occupational employment and wage estimates: United States. www.bls.gov/oes/2010/may/oes_nat.htm. Accessed September 28, 2015.

      U.S. Department of Labor, Bureau of Labor Statistics. Consumer Price Index - all urban consumers. 2010. www.bls.gov/cpi/. Accessed December 20, 2010.

      A complete explanation of the methodology used to estimate these costs is found in the Appendix (available online).

      Results

      The Recall Model

      The recall model demonstrated that more-rapid outbreak response led to significant reductions in reported illnesses associated with Salmonella and E. Coli O157:H7 outbreaks.
      During 2007–2008, 15 recalls of Food Safety and Inspection Service–regulated ground beef products were initiated owing to foodborne illness outbreaks of E. coli O157:H7 (Appendix Table A1, available online). The percentage of recalled meat recovered ranged from 0% to 66%, with a median of 5%. Thus, for most outbreak-associated recalls, only a small percentage of the recalled meat was recovered. Other outbreak-associated parameters were similarly skewed toward the low end of distributions observed across the 15 recalls. The number of outbreak-associated cases ranged from 0 to 49 with a median of 11. The estimated rate of illness per 1,000 pounds of consumed meat ranged from 0 to 7.1 with a median of 0.054. The estimated number of prevented cases ranged from 0 to 49 with a median of 1.2. For the 15 outbreak-associated recalls, the final estimate for the number of reported prevented cases was 108, with a 90% credible interval (CI) from 95 to 266 cases (Table 2). Based on the CDC multiplier for underdiagnosis of 26.1 cases occurring for each reported case, the estimated total number of prevented cases was 2,819, with a 90% CI from 2,480 to 6,943.
      • Scallan E.
      • Hoekstra R.M.
      • Angulo F.J.
      • et al.
      Foodborne illness acquired in the United States - major pathogens.
      Table 2Estimated Numbers of Cases Prevented by Outbreak-Associated Recalls of E. coli O157:H7 and Salmonella Infections, 2007–2008
      AgentVehicleNo. outbreaksNo. outbreak cases reported
      Total number of outbreak-associated cases reported.
      No. reportable cases prevented
      Reportable cases prevented based on expected number of reported cases minus actual number of reported cases for Salmonella, and the estimated attack rate of cases per 1,000 pounds consumed times the pounds of product recovered for E. coli O157:H7.
      Cases prevented, 90% credible interval
      90% credible intervals based on Monte Carlo simulations (10,000 iterations) with distributions of possible values substituted for specific values.
      E. coli O157:H7Ground beef1527610895, 266
      Salmonella outbreaks
       Wandsworth, TyphimuriumVeggie Booty1874932, 67
       I 4,[5],12:i:-Pot pies14017219, 132
       LitchfieldCantaloupe15100
       St. PaulJalapeno and serrano peppers11,50034550, 714
       TyphimuriumPeanut butter and peanut-containing products171411427, 214
       Total SalmonellaMultiple52,753580128, 1,127
      a Total number of outbreak-associated cases reported.
      b Reportable cases prevented based on expected number of reported cases minus actual number of reported cases for Salmonella, and the estimated attack rate of cases per 1,000 pounds consumed times the pounds of product recovered for E. coli O157:H7.
      c 90% credible intervals based on Monte Carlo simulations (10,000 iterations) with distributions of possible values substituted for specific values.
      During 2007–2008, five major multistate Salmonella outbreaks of foodborne illness that led to recalls of food products were investigated. For these outbreaks, the numbers of prevented cases ranged from 0 to 345, with an estimated total of 580 and a 90% CI of 128 to 1,127 (Table 2). Based on the CDC multiplier for underdiagnosis of 29.3 cases occurring for each reported case, the estimated total number of prevented cases was 16,994 with a 90% CI from 3,750 to 33,021.
      • Scallan E.
      • Hoekstra R.M.
      • Angulo F.J.
      • et al.
      Foodborne illness acquired in the United States - major pathogens.

      The Process Change Model

      The process change empirical model tested the effect of PulseNet implementation and testing intensity on reported illnesses, finding that both implementation and intensity were negatively related to illness reporting.
      The descriptive statistics for the variables used in the process change model are illustrated in Table 1. Over the studied periods, states reported an average of 869 Salmonella illnesses, 64 E. coli O157 illnesses, and 15 Listeria illnesses. Over the respective time periods for which NNDSS illnesses were recorded, states tested an average of 305 Salmonella isolates, 31 STEC isolates, and ten Listeria isolates.
      The effect of PulseNet isolate testing on reported illnesses is illustrated in Table 3. In the upper panel, the marginal effects from the base model, as specified in Equation 1, are presented for each pathogen and each model type. For Salmonella, there was a strong negative effect on illnesses from both the existence of PulseNet and the intensity of testing within each state. For other pathogens, only intensity of testing is significant. The intensity variable (lagged isolate testing) measures the effect of past isolate testing on current illnesses reported. This was negative and significant for all specifications except the random effects E. coli model and Poisson Listeria model. Given that more testing leads to better identification and traceback of outbreaks, this is consistent with a reduction in future illness due to enhanced industry accountability. The models’ predictive power (as demonstrated by R2 values) was weakest for E. coli, which makes sense given that the dependent variable measures E. coli O157 whereas isolates were recorded for the more general category, STEC.
      Table 3Effect of PulseNet on Reported Foodborne Illnesses: Process Change Model
      Independent variables
      Variables not reported, are: for the random effects models: population, year, PCI, and eight census division dummy variables; for the fixed effects models (state and year being the fixed effects): population and PCI; for the Poisson models: population, PCI, 15 year dummy variables, and 50 state dummy variables.
      SalmonellaE. ColiListeria
      ABCABCABC
      N809809809561561561501501501
      PulseNet implemented–22.647**–37.294**–13.573**–0.1992.894*0.0080.1970.1460.013
      Isolate tests0.227**0.160**0.053**0.461**0.150**0.057**0.288**0.301**0.102**
      Lagged isolate tests–0.292**–0.321**–0.166**–0.068–0.231**–0.092**–0.118**–0.109**–0. 023
      Adjusted R-squared0.8590.9480.9660.4840.6760.8090.9050.9350.798
      Predicted reduction in reported illnesses
      Base model
       Illnesses15,784**19,758**9,096**310489*364**113**113*27
       90% CI11,948, 19,62315,871, 23,6628,504, 9,686–95, 71748, 939274, 45339, 18731, 195–38, 92
      Adjusted462,487578,947266,5228,09612,7509,48923823756
       Hospitalizations29737217111171210410325
       90% CI140, 516180, 63686, 286–3, 291, 395, 2333, 19026, 197–34, 88
       Deaths673<1<1<118184
       90% CI0, 160, 190, 90, 10, 10, 10, 550, 55–6, 21
      Spillover effects model
       Illnesses21,249**25,181**11,291**2,673**1,597**670**151**7573
       90% CI16,863, 25,63220,747, 29,5955,628, 16,9481,718, 3,627609, 2,589451, 88946, 256–46, 196–26, 172
      Adjusted622,614737,845330,84069,75541,68417,475316157153
       Hospitalizations4004742129054231386867
       90% CI192, 687231, 80577, 40731, 17313, 1148, 4338, 259–39, 189–22, 167
       Deaths89411<1241212
       90% CI0, 210, 250, 120, 50, 30, 10, 74–6, 48–3, 43
      Note: A = Random effects model; B = Fixed effects model; C = Poisson model. Boldface indicates statistical significance (*p<0.05; **p<0.01).
      a Variables not reported, are: for the random effects models: population, year, PCI, and eight census division dummy variables; for the fixed effects models (state and year being the fixed effects): population and PCI; for the Poisson models: population, PCI, 15 year dummy variables, and 50 state dummy variables.
      The lower panel of Table 3 presents the predicted illness reduction associated with PulseNet for each pathogen. The inclusion of spillover effects led to larger declines in Salmonella and E. coli cases, but not in Listeria cases. The reduction in Salmonella illnesses associated with PulseNet ranged from 9,096 (90% CI=8,504, 9,686) in the Poisson base model to 25,181 (90% CI=20,747, 29,595) in the fixed effects spillover model. For E. coli, illness reduction ranged from 310 (90% CI= –99, 717) to 2,673 (90% CI=1,718, 3,627). For Listeria, reported illnesses declined by 27 (90% CI= –38, 92) to 151 (90% CI=46, 256).
      Adjusted for underdiagnosis and underreporting (in italics in Table 3), mean estimates of 266,522–737,845 Salmonella illnesses, 8,096–69,755 E. coli illnesses, and 56–316 Listeria illnesses were predicted to be avoided as a result of PulseNet in 2009.

      Economic Impacts of PulseNet

      The economic impact of PulseNet was calculated as the product of the expected cost per illness for each pathogen and the number of averted illnesses, as estimated above.
      The cost per illness was estimated to be $1,792 (90% CI=$1,461, $2,295) for Salmonella, $2,154 (90% CI=$1,464, $3,435) for E. coli O157, and $156,019 (90% CI=$81,003, $254,934) for Listeria, based on data from Scharff
      • Scharff R.L.
      The economic burden from health losses due to foodborne illness in the United States.
      and Grosse and colleagues.
      • Grosse S.D.
      • Krueger K.V.
      • Mvundura M.
      Economic productivity by age and sex.
      The total program costs and healthcare costs averted from PulseNet are presented in Table 4. Results from the Poisson model, which generally yielded the most conservative illness reduction estimates, are illustrated. Estimates are presented for both the reported NNDSS number of illnesses and the number of illnesses adjusted with the underdiagnosis and underreporting multiplier.
      Table 4Means and Medians. Economic Impact of PulseNet (in Millions, 2010 U.S.$)
      Costs averted,Program costs for public health agencies,Net cost averted,Net cost averted per public health $ invested,
      Mean, MedianMean, MedianMean, MedianMean, Median
      ModelCI (5%–95%)CI (5%–95%)CI (5%–95%)CI (5%–95%)
      Simple lag (Process change model)
       Reported
      • 21.3, 20.7
      • (11–34)
      • 7.3, 7.3
      • (7.1–7.6)
      • 14.0, 13.5
      • (3–26)
      • 2.9. 2.9
      • (1–5)
       Adjusted
      Adjusted for underreporting and underdiagnosis, based on estimates from Scallan et al.1
      • 506.8, 491.3
      • (410–646)
      • 7.3, 7.3
      • (7.1–7.6)
      • 499.5, 484.0
      • (402–639)
      • 69.7, 67.6
      • (56–89)
      Spillover effects (Process change model)
       Reported
      • 33.1, 32.0
      • (14–55)
      • 7.3, 7.3
      • (7.1–7.6)
      • 25.8, 24.7
      • (7–48)
      • 4.6, 4.4
      • (2–8)
       Adjusted
      Adjusted for underreporting and underdiagnosis, based on estimates from Scallan et al.1
      • 654.4, 638.5
      • (340–1,002)
      • 7.3, 7.3
      • (7.1–7.6)
      • 647.1, 631.2
      • (333–995)
      • 90.0, 87.7
      • (47–138)
      Recall model
       Reported
      • 1.3, 1.2
      • (0.5–2)
      • 7.3, 7.3
      • (7.1–7.6)
      • –6.0, –6.1
      • (–7– –5)
      • 0.2, 0.2
      • (0.1–0.3)
       Adjusted
      Adjusted for underreporting and underdiagnosis, based on estimates from Scallan et al.1
      • 36.5, 35.1
      • (13–65)
      • 7.3, 7.3
      • (7.1–7.6)
      • 29.2, 27.8
      • (6–58)
      • 5.0, 4.8
      • (2–9)
      a Adjusted for underreporting and underdiagnosis, based on estimates from Scallan et al.
      • Scallan E.
      • Hoekstra R.M.
      • Angulo F.J.
      • et al.
      Foodborne illness acquired in the United States - major pathogens.
      The economic returns from PulseNet were substantial. For the process change models using reported illnesses, total median costs averted ranged from $21 to $33 million, depending on the model used. When estimates were adjusted for underreporting and underdiagnosis factors, the range became $491–$654 million. The direct effect of removing tainted product from the market (because of faster recalls) added $1–$37 million.
      The measurable PulseNet costs borne by public health agencies were approximately $7.3 million. These costs included $1.3 million for the amortized annual laboratory setup costs associated with the three PulseNet pathogens examined in this study, $4.3 million in costs for testing samples and investigating clusters, and $1.65 million for CDC’s internal cost of running the program.
      Annually, averted illnesses attributable to improved industry processes resulted in $14–$647 million in median reduced direct and indirect costs, yielding a societal return of $3–$90 for every $1 invested by public health agencies. The direct effects from improved recalls yielded up to $29 million, with a societal return of up to $5 for every $1 invested by public health agencies.

      Discussion

      This evaluation of PulseNet demonstrated significant economic and public health benefits from the system. These benefits are driven by improvements in outbreak detection, which provide industry and government with valuable information, while exposing food producers to increased threat from litigation and reputation losses. This ultimately leads to adjustments in processes that reduce foodborne illness. The measurable costs of the program, in contrast, are very modest.

      Limitations

      One limitation of this study is that it does not include all economic costs. Welfare losses from premature death and reduced quality of life due to illness are not monetized here, but could be potentially large. Also missing are indirect costs from PulseNet, including costs to other government entities, administrative costs of litigation, costs associated with recalls, and costs of implementing and managing process changes. The largest of these costs, however—the industry cost of implementing and managing process changes—is primarily driven by the expression of consumer preferences in the marketplace (the avoidance of lost sales owing to outbreaks). As such, this cost essentially represents actions to correct a market failure, thereby restricting process changes to be efficiency-enhancing.
      Another potential limitation is that, for Salmonella, the predicted illness reduction numbers are high relative to the baseline number of illnesses and are not apparent in NNDSS or FoodNet trend data (www.cdc.gov/foodnet/index.html), which have shown relatively stable incidence rates over the past decade. Nevertheless, Appendix Figure A1 (available online) makes it clear that the numbers estimated in this study are plausible. Although Salmonella cases increased more than 20% in states that had a low ratio of tests to reported illnesses (bottom ten adopters), the states that analyzed samples from the highest proportion of reported illnesses experienced a reduction in illnesses of almost 10% (top ten adopters). Thus, PulseNet may well have helped prevent a substantial national increase in the occurrence of Salmonella over the past decade.
      Also complicating the analysis is the fact that outbreak identification and the media attention surrounding it leads to enhanced illness reporting by the public. Theoretically, this will affect the analysis in two ways. First, because reported illnesses are used as a dependent variable in one part of the analysis, enhanced reporting without a control for media effects introduces omitted variable bias into the model. Properly controlled, the process model would likely yield larger reductions in foodborne illness than it currently predicts. Second, enhanced reporting will affect the degree to which illnesses are underreported and underdiagnosed. As a result, in the adjusted model only, the effects of PulseNet may be overstated. The net effects of these biases are unclear, but do not undermine the principal conclusion.

      Conclusions

      The PulseNet system exposes food safety risks to the public, thereby giving consumers, industry, and the government valuable information that can be used to reduce foodborne illness. The reduction in healthcare costs and other economic benefits from the resulting actions are substantial. Improving public health surveillance for foodborne disease is a prerequisite for improving food safety in the U.S.

      Acknowledgments

      Publication of this article was supported by the Centers for Disease Control and Prevention (CDC), Office of the Associate Director for Policy . The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of CDC.
      Many people outside of the core group of authors made important contributions to this paper. In particular, we would like to thank Kakoli Roy and Martin Meltzer at CDC and Tanya Roberts at the U.S. Department of Agriculture (USDA) Economic Research Service for their guidance along the way. We would also like to thank Emilio Esteban and his team at USDA for detailed comments on an early version of the paper that helped us substantially improve the paper. Finally, we thank other members of the PulseNet Cost Benefit Study Work Group, Hugh Maguire, Bela Matyas, David Zorn, Shari Shea, Heather Green, Lauren Rosenberg, Kara Watarida and Richard Skibicki. This Work Group was organized by Kristy Kubota and other staff at the Association of Public Health Laboratories (APHL) under the auspices of the Council to Improve Foodborne Outbreak Response (CIFOR) .
      This study was supported by the Association of Public Health Laboratories and CDC under Cooperative Agreement U60/CD303019 .
      No financial disclosures were reported by the authors of this paper.

      Appendix A. Supplementary Materials

      References

        • Scallan E.
        • Hoekstra R.M.
        • Angulo F.J.
        • et al.
        Foodborne illness acquired in the United States - major pathogens.
        Emerg Infect Dis. 2011; 17: 7-15https://doi.org/10.3201/eid1701.P11101
        • Scallan E.
        • Griffin P.M.
        • Angulo F.J.
        • et al.
        Foodborne illness acquired in the United States - unspecified agents.
        Emerg Infect Dis. 2011; 17: 16-22https://doi.org/10.3201/eid1701.P21101
      1. CDC. PulseNet. Accessed September 28, 2015. www.cdc.gov/pulsenet/

        • Painter J.A.
        • Hoekstra R.M.
        • Ayers T.
        • et al.
        Attribution of foodborne illnesses, hospitalizations, and deaths to food commodities by using outbreak data, United States, 1998-2008.
        Emerg Infect Dis. 2013; 19: 407-415https://doi.org/10.3201/eid1903.111866
        • Seys S.A.
        • Sampedro F.
        • Hedberg C.W.
        Assessment of shiga toxin-producing Escherichia coli O157 illnesses prevented by recalls of beef products.
        Foodborne Pathog Dis. 2015; 12: 800-805https://doi.org/10.1089/fpd.2015.1968
      2. USDA-FSIS. Recall case archive. www.fsis.usda.gov/wps/portal/fsis/topics/recalls-and-public-health-alerts/recall-case-archive. Accessed September 28, 2015.

      3. USDA-ERS. Loss-adjusted food availability. www.ers.usda.gov/Data/FoodConsumption/FoodGuideSpreadsheets.htm. Accessed September 28, 2015.

      4. U.S. Census Bureau. Table 1: Annual estimates of the population for the United States, regions, and states and for Puerto Rico: April 1, 2000 to July 1, 2009 (NST-EST2009-01). 2009. www.census.gov/popest/data/historical/2000s/vintage_2009/. Accessed September 28, 2015.

      5. Bureau of Economic Analysis, U.S. Department of Commerce. State per capita personal income 2010. www.bea.gov/newsreleases/regional/spi/2011/pdf/spi0911pc_fax.pdf. Accessed September 28, 2015.

      6. CDC. National Notifiable Diseases Surveillance System. 2015. http://wwwn.cdc.gov/nndss/. Accessed September 28, 2015.

      7. MMWR: Summary of Notifiable Diseases. CDC. http://www.cdc.gov/mmwr/mmwr_nd/index.html. Updated September 8, 2015.

      8. U.S. Department of Labor, Bureau of Labor Statistics. May 2010 national occupational employment and wage estimates: United States. www.bls.gov/oes/2010/may/oes_nat.htm. Accessed September 28, 2015.

      9. U.S. Department of Labor, Bureau of Labor Statistics. Consumer Price Index - all urban consumers. 2010. www.bls.gov/cpi/. Accessed December 20, 2010.

        • Scharff R.L.
        The economic burden from health losses due to foodborne illness in the United States.
        J Food Prot. 2012; 75: 123-131https://doi.org/10.4315/0362-028X.JFP-11-058
        • Grosse S.D.
        • Krueger K.V.
        • Mvundura M.
        Economic productivity by age and sex.
        Med Care. 2009; 47: S94-S103https://doi.org/10.1097/MLR.0b013e31819c9571