H1N1 and Seasonal Influenza Vaccine Safety in the Vaccine Safety Datalink Project

      Background

      The emergence of pandemic H1N1 influenza virus in early 2009 prompted the rapid licensure and use of H1N1 monovalent inactivated (MIV) and live, attenuated (LAMV) vaccines separate from seasonal trivalent inactivated (TIV) and live, attenuated (LAIV) influenza vaccines. A robust influenza immunization program in the U.S. requires ongoing monitoring of potential adverse events associated with vaccination.

      Purpose

      To prospectively conduct safety monitoring of H1N1 and seasonal influenza vaccines during the 2009–2010 season.

      Methods

      The Vaccine Safety Datalink (VSD) Project monitors ∼9.2 million members in eight U.S. medical care organizations. Electronic data on vaccines and pre-specified adverse events were updated and analyzed weekly for signal detection from November 2009 to April 2010 using either a self-controlled design or a current versus historical comparison. Statistical signals were further evaluated using alternative approaches to identify temporal clusters and to control for time-varying confounders.

      Results

      As of May 1, 2010, a total of 1,345,663 MIV, 267,715 LAMV, 2,741,150 TIV, and 157,838 LAIV doses were administered in VSD. No significant associations were noted during sequential analyses for Guillain–Barré syndrome, most other neurologic outcomes, and allergic and cardiac events. For MIV, a statistical signal was observed for Bell's palsy for adults aged ≥25 years on March 31, 2010, using the self-controlled approach. Subsequent analyses revealed no significant temporal cluster. Case-centered logistic regression adjusting for seasonality demonstrated an OR for Bell's palsy of 1.26 (95% CI=0.97, 1.63).

      Conclusions

      No major safety problems following H1N1 or seasonal influenza vaccines were detected in the 2009–2010 season in weekly sequential analyses. Seasonality likely contributed to the Bell's palsy signal following MIV. Prospective safety monitoring followed by rigorous signal refinement is critical to inform decision-making by regulatory and public health agencies.

      Background

      The emergence of a novel H1N1 influenza virus of swine origin in North America in April 2009 prompted the rapid development of new H1N1 vaccines, which became available for use in October 2009. While the initial uncertainty of how the novel H1N1 pandemic would evolve led to heightened concerns about disease prevention, public anxiety about the safety of novel H1N1 vaccines also became apparent.
      • Maurer J.
      • Uscher-Pines L.
      • Harris K.M.
      Perceived seriousness of seasonal and A(H1N1) influenzas, attitudes toward vaccination, and vaccine uptake among U.S. adults: does the source of information matter?.
      Given the prior experience with the 1976 swine influenza vaccination program, where an excess risk of Guillain–Barré syndrome (GBS)
      • Retailliau H.F.
      • Curtis A.C.
      • Storr G.
      • Caesar G.
      • Eddins D.L.
      • Hattwick M.A.
      Illness after influenza vaccination reported through a nationwide surveillance system, 1976–1977.
      was identified, policymakers sought to ensure that robust systems of monitoring vaccine safety were in place for the 2009–2010 season.

      Salmon D, Akhtar A, Mergler M, et al. Monitoring immunization safety for the pandemic 2009–2010 H1N1 influenza vaccination program. Pediatrics; In press.

      The Vaccine Safety Datalink Project (VSD), initiated by the CDC, is widely acknowledged as the backbone of active surveillance for vaccine safety in the U.S.
      Federal Immunization Safety Task Force
      Federal plans to monitor immunization safety for the pandemic 2009 H1N1 influenza vaccination program.
      Methodologic approaches to influenza safety surveillance within the VSD were previously evaluated.
      • Greene S.K.
      • Kulldorff M.
      • Lewis E.M.
      • et al.
      Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.
      At the beginning of the 2009–2010 season, near–real time safety surveillance was initiated for both H1N1 and seasonal influenza vaccines within the VSD Project. Findings regarding the safety of influenza vaccines based on sequential analyses during the 2009–2010 season are presented, and the subsequent evaluation of a signal (i.e., a potential adverse event occurring more frequently than anticipated following vaccination
      Federal Immunization Safety Task Force
      Federal plans to monitor immunization safety for the pandemic 2009 H1N1 influenza vaccination program.
      ) detected in the surveillance system is described.

      Methods

      Study Population

      The Vaccine Safety Datalink Project is a collaboration among eight medical care organizations and CDC, which has performed population-based research on vaccine safety in the U.S. since 1990.
      • DeStefano F.
      The Vaccine Safety Datalink project.
      • Chen R.T.
      • DeStefano F.
      • Davis R.L.
      • et al.
      The Vaccine Safety Datalink: immunization research in health maintenance organizations in the USA.
      These organizations include Group Health Cooperative (Washington); Harvard Vanguard Medical Associates & Harvard Pilgrim Health Care (Massachusetts); HealthPartners Research Foundation (Minnesota); Kaiser Permanente of Colorado (Colorado); Kaiser Permanente of Northern California (California); Kaiser Permanente of Southern California (California); Marshfield Clinic Research Foundation (Wisconsin); and Northwest Kaiser Permanente (Oregon). Since 2005, VSD has conducted prospective weekly surveillance, or rapid cycle analysis (RCA), to monitor the safety of newly licensed vaccines and new vaccine recommendations.
      • Lieu T.A.
      • Kulldorff M.
      • Davis R.L.
      • et al.
      Real-time vaccine safety surveillance for the early detection of adverse events.
      At the start of the 2009–2010 influenza vaccination season, the VSD population included approximately 9.2 million children and adults, representing close to 3% of the U.S. population.

      Data Sources

      The Vaccine Safety Datalink Project uses a distributed data network model in which standardized data files are created and maintained by each site. Every week, each of the eight VSD sites updates data files with all immunizations (including manufacturer, lot number, injection site, and concurrent immunizations); hospital admissions; emergency department visits; outpatient visits; and ICD-9 codes for all patient encounters. Potential adverse events were identified using ICD-9 code diagnosis data from healthcare encounters (Appendix A, available online at www.ajpmonline.org). Other files contain demographic and enrollment data for each patient. For weekly analyses, aggregate, de-identified data sets were created that included counts of individuals in different strata by VSD site; age; gender; vaccine type (inactivated or live, attenuated H1N1 and seasonal influenza vaccines); week of vaccine administration; and type, setting, and timing of potential adverse events.

      Potential Adverse Events

      Prospective weekly surveillance was conducted from November 2009 to April 2010 on pre-specified potential adverse events selected for being clinically well defined, serious, and potentially associated with influenza vaccines (Appendix A, available online at www.ajpmonline.org). Potential adverse events identified by ICD-9 code data and occurring in pre-specified risk intervals after each patient's first dose of H1N1 or seasonal influenza vaccine monitored in the 2009–2010 season included GBS,
      • Schonberger L.B.
      • Bregman D.J.
      • Sullivan-Bolyai J.Z.
      • et al.
      Guillain-Barré syndrome following vaccination in the National Influenza Immunization Program, U.S., 1976-1977.
      • Haber P.
      • DeStefano F.
      Influenza vaccination and Guillain Barré syndrome.
      • Haber P.
      • DeStefano F.
      • Angulo F.J.
      • et al.
      Guillain-Barré syndrome following influenza vaccination.
      • Juurlink D.N.
      • Stukel T.A.
      • Kwong J.
      • et al.
      Guillain-Barré syndrome after influenza vaccination in adults: a population-based study.
      • Souayah N.
      • Nasar A.
      • Suri M.F.
      • Qureshi A.I.
      Guillain-Barré syndrome after vaccination in U.S.: a report from the CDC/FDA vaccine adverse event reporting system.
      • Langmuir A.D.
      • Bregman D.J.
      • Kurland L.T.
      • Nathanson N.
      • Victor M.
      An epidemiologic and clinical evaluation of Guillain-Barré syndrome reported in association with the administration of swine influenza vaccines.
      demyelinating disease of the central nervous system,
      • Haber P.
      Influenza vaccine and neurological adverse events: VAERS 7/1990 - 1/2003.
      • Brostoff J.M.
      • Beitverda Y.
      • Birns J.
      Post-influenza vaccine chronic inflammatory demyelinating polyneuropathy.
      • Hull T.P.
      • Bates J.H.
      Optic neuritis after influenza vaccination.
      • Nakamura N.
      • Nokura K.
      • Zettsu T.
      • et al.
      Neurologic complications associated with influenza vaccination: two adult cases.
      disorders of the peripheral nervous system and neuropathy,
      • Haber P.
      Influenza vaccine and neurological adverse events: VAERS 7/1990 - 1/2003.
      • Hull J.H.
      • Mead S.H.
      • Foster O.J.
      • Modarres-Sadeghi H.
      Severe vasculitic neuropathy following influenza vaccination.
      seizures,
      • Nakamura N.
      • Nokura K.
      • Zettsu T.
      • et al.
      Neurologic complications associated with influenza vaccination: two adult cases.
      • McMahon A.W.
      • Iskander J.
      • Haber P.
      • et al.
      Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
      encephalomyelitis,
      • Haber P.
      Influenza vaccine and neurological adverse events: VAERS 7/1990 - 1/2003.
      • Nakamura N.
      • Nokura K.
      • Zettsu T.
      • et al.
      Neurologic complications associated with influenza vaccination: two adult cases.
      • Gross W.L.
      • Ravens K.G.
      • Hansen H.W.
      Meningoencephalitis syndrome following influenza vaccination.
      Bell's palsy,
      • Zhou W.
      • Pool V.
      • DeStefano F.
      • Iskander J.K.
      • Haber P.
      • Chen R.T.
      A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
      • Chou C.H.
      • Liou W.P.
      • Hu K.I.
      • Loh C.H.
      • Chou C.C.
      • Chen Y.H.
      Bell's palsy associated with influenza vaccination: two case reports.
      • Mutsch M.
      • Zhou W.
      • Rhodes P.
      • et al.
      Use of the inactivated intranasal influenza vaccine and the risk of Bell's palsy in Switzerland.
      other cranial nerve disorders, ataxia,
      • Saito H.
      • Yanagisawa T.
      Acute cerebellar ataxia after influenza vaccination with recurrence and marked cerebellar atrophy.
      anaphylaxis,
      • McMahon A.W.
      • Iskander J.
      • Haber P.
      • et al.
      Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
      • Bohlke K.
      • Davis R.L.
      • Marcy S.M.
      • et al.
      Vaccine Safety Datalink Team
      Risk of anaphylaxis after vaccination of children and adolescents.
      • Coop C.A.
      • Balanon S.K.
      • White K.M.
      • Whisman B.A.
      • Rathkopf M.M.
      Anaphylaxis from the influenza virus vaccine.
      • Izurieta H.S.
      • Haber P.
      • Wise R.P.
      • et al.
      Adverse events reported following live, cold-adapted, intranasal influenza vaccine.
      and allergic reaction other than anaphylaxis (including angioneurotic edema and urticaria).
      • Greene S.K.
      • Kulldorff M.
      • Lewis E.M.
      • et al.
      Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.
      • McMahon A.W.
      • Iskander J.
      • Haber P.
      • et al.
      Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
      • Horowitz L.
      Urticaria subsequent to administration of influenza vaccine.
      Myocarditis/pericarditis was also monitored after live, attenuated vaccines.
      • Guarner J.
      • Paddock C.D.
      • Shieh W.J.
      • et al.
      Histopathologic and immunohistochemical features of fatal influenza virus infection in children during the 2003–2004 season.
      Risk-interval durations were based on published studies as well as biologic plausibility.
      • McMahon A.W.
      • Iskander J.
      • Haber P.
      • et al.
      Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
      • Zhou W.
      • Pool V.
      • DeStefano F.
      • Iskander J.K.
      • Haber P.
      • Chen R.T.
      A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
      • Zhou W.
      • Pool V.
      • Iskander J.K.
      • et al.
      Surveillance for safety after immunization: Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991-2001.
      • Nichol K.L.
      • Margolis K.L.
      • Lind A.
      • et al.
      Side effects associated with influenza vaccination in healthy working adults A randomized, placebo-controlled trial.
      • Hambidge S.J.
      • Glanz J.M.
      • France E.K.
      • et al.
      Vaccine Safety Datalink Team
      Safety of trivalent inactivated influenza vaccine in children 6 to 23 months old.
      To optimize specificity, adverse events were included only if they were the first event in either a 6-month or 1-year period.

      Study Design and Analyses

      Self-controlled risk interval design for signal detection—binomial-based MaxSPRT

      The self-controlled risk interval design was the primary and preferred approach.
      • Greene S.K.
      • Kulldorff M.
      • Lewis E.M.
      • et al.
      Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.
      • Klein N.P.
      • Hansen J.
      • Lewis E.
      • et al.
      Post-marketing safety evaluation of a tetanus toxoid, reduced diptheria toxoid and 3-component acellular pertussis vaccine administered to a cohort of adolescents in a United States health maintenance organization.
      This approach was used for more common adverse events (≥50 adverse events anticipated in an analytic stratum during the season) to adjust for known and unknown time-invariant confounders.
      • Greene S.K.
      • Kulldorff M.
      • Lewis E.M.
      • et al.
      Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.
      The self-controlled risk interval design examined whether the risk of adverse events in the risk interval after vaccination was greater than in a comparison interval in the same individual. For most adverse events, a pre-vaccination comparison interval was used to facilitate timely analyses. Because some patients may delay or forgo routine influenza vaccination shortly after experiencing an illness, otherwise known as the healthy vaccinee effect, the pre-comparison interval excluded the 14 days prior to vaccination.
      • Virtanen M.
      • Peltola H.
      • Paunio M.
      • Heinonen O.P.
      Day-to-day reactogenicity and the healthy vaccinee effect of measles-mumps-rubella vaccination.
      For certain adverse events with short risk intervals (i.e., ≤15 days) where (1) a recent prior adverse event might preclude vaccination (i.e., allergic reaction and anaphylaxis) or (2) individuals might have an underlying condition that is also an indication for vaccination (i.e., seizure disorder), a comparison interval occurring after the risk interval was used. The self-controlled risk interval approach could not be used for timely surveillance of adverse events with longer risk intervals (i.e., 42 days), such as GBS, where using a comparison interval after the risk interval would be most appropriate. Selected outcomes, such as demyelinating disease, were divided into strata by age to evaluate the possibility of effect modification. All ages were combined for allergic adverse events because effect modification was less likely.
      Because there is wide variability in the timeliness of inpatient data, depending on whether data are derived from electronic medical record systems or medical claims data, inpatient data were included for each site only after an estimated 95% of data accrued, to ensure that comparisons between risk and comparison intervals were valid.
      • Greene S.K.
      • Kulldorff M.
      • Yin R.
      • et al.
      Near real-time vaccine safety surveillance with partially accrued data.
      Additionally, in order to conduct timely analyses, intervals were scaled appropriately. For example, if only 1 week of a 42-day risk interval had elapsed since vaccination, these data were compared to the corresponding first week of the 42-day comparison interval.
      To account for multiple testing, sequential statistical methods were used. The maximized sequential probability ratio test (MaxSPRT) uses a one-sided composite alternative hypothesis of excess risk in the exposed postvaccination interval.
      • Lieu T.A.
      • Kulldorff M.
      • Davis R.L.
      • et al.
      Real-time vaccine safety surveillance for the early detection of adverse events.
      • Kulldorff M.
      • Davis R.L.
      • Kolczak M.
      • Lewis E.M.
      • Lieu T.A.
      • Platt R.
      A maximized sequential probability ratio test for drug and vaccine safety surveillance.
      For the self-controlled risk interval design, the MaxSPRT was used based on a binomial probability model.
      • Kulldorff M.
      • Davis R.L.
      • Kolczak M.
      • Lewis E.M.
      • Lieu T.A.
      • Platt R.
      A maximized sequential probability ratio test for drug and vaccine safety surveillance.
      The null hypothesis assumes that the risk of an adverse event occurring during the risk interval is the same as the risk of it occurring during the comparison interval, scaled by the length of the intervals. For each adverse event and age group, “upper limits” and “critical values” using a one-tailed test and stringent alpha level of 0.01 were obtained.
      • Greene S.K.
      • Kulldorff M.
      • Lewis E.M.
      • et al.
      Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.
      • Kulldorff M.
      • Davis R.L.
      • Kolczak M.
      • Lewis E.M.
      • Lieu T.A.
      • Platt R.
      A maximized sequential probability ratio test for drug and vaccine safety surveillance.
      Surveillance for the 2009–2010 season ended when (1) the log-likelihood ratio (LLR) test statistic exceeded the critical value, generating a “statistical signal”; or (2) the total number of adverse events reached a pre-specified upper limit; or (3) the vaccination season ended on April 30, 2010.
      • Greene S.K.
      • Kulldorff M.
      • Lewis E.M.
      • et al.
      Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.

      Current versus historical comparison for signal detection—Poisson-based and conditional MaxSPRT

      The current versus historical approach was used for extremely rare adverse events (<50 adverse events anticipated based on the historical VSD rates) to improve power and timeliness to detect a signal. In particular, this approach was used for prospective, timely surveillance of GBS. This approach compared the rate of adverse events in the risk interval following vaccination with a historical background rate. Historical rates were calculated from data available in VSD using the rate of adverse events in the risk interval following seasonal TIV during the 2000–2001 through 2008–2009 seasons, or the overall rate of adverse events in the entire VSD population. Current MIV or TIV vaccinees were compared to historical TIV vaccinees. Because LAMV and LAIV vaccinees were less likely to have high-risk conditions, they were compared to historical all-person time rates. Site and age adjustments were performed to estimate the expected number of adverse events for the current season.
      Either the Poisson-based MaxSPRT or the Poisson-based conditional MaxSPRT (CMaxSPRT) was used.
      • Kulldorff M.
      • Davis R.L.
      • Kolczak M.
      • Lewis E.M.
      • Lieu T.A.
      • Platt R.
      A maximized sequential probability ratio test for drug and vaccine safety surveillance.
      • Li L.
      • Kulldorff M.
      A conditional maximized sequential probability ratio test for pharmacovigilance.
      Upper limits and critical values were based on the estimated number of doses expected to be administered using a one-tailed test and an alpha level of 0.05 for GBS and anaphylaxis and a more stringent alpha level of 0.01 for all other adverse events.
      • Lieu T.A.
      • Kulldorff M.
      • Davis R.L.
      • et al.
      Real-time vaccine safety surveillance for the early detection of adverse events.
      • Kulldorff M.
      • Davis R.L.
      • Kolczak M.
      • Lewis E.M.
      • Lieu T.A.
      • Platt R.
      A maximized sequential probability ratio test for drug and vaccine safety surveillance.
      The proportion of adverse events expected each week was adjusted downward to account for lag in the arrival of inpatient data and the proportion of the risk interval that had elapsed. The CMaxSPRT approach was used when the ratio of the number of observed historical adverse events to the upper limit was ≤2.5, in order to account for the uncertainty in the estimates of the expected number of adverse events under the null hypothesis.
      • Li L.
      • Kulldorff M.
      A conditional maximized sequential probability ratio test for pharmacovigilance.

      Medical Record Reviews

      Medical record reviews were prospectively conducted on all GBS cases identified in the risk interval after H1N1 vaccination using electronic medical records available at each VSD site. Medical record reviews identified whether GBS cases were new-onset, follow-up visits for a previous condition, or incorrectly coded diagnoses, within a 1–2-week period after identification in claims data. In order to estimate comparable historical rates, medical record reviews on GBS cases that occurred after TIV during the 2000–2001 to 2008–2009 seasons in the VSD population were also conducted to understand whether the risk for GBS was higher in the current season following MIV compared to historical seasons following TIV. All cases were adjudicated by at least two neurologists with expertise in demyelinating disease using the GBS and Fisher syndrome definitions provided by the Brighton Collaboration, an international group that develops standardized criteria for potential vaccine-associated adverse events.
      Brighton Collaboration
      Guillain-Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data.

      Signal Refinement

      “Signals” occur when the rate of an adverse event following influenza vaccination is significantly greater than the anticipated adverse event rate in a comparison group. Statistical signals do not necessarily represent true associations. Rather, they indicate the need for further study to determine if a true association exists. In order to detect true associations rapidly, safety surveillance systems must allow a reasonable amount of type I error to ensure early identification and investigation.
      To further refine statistical signals generated by sequential analyses,

      Brookings Institution. Paper presented at Methods for Signal Refinement in Active Medical Product Surveillance; 2010; Washington DC.

      the presence of temporal clusters was assessed using a temporal scan statistic, which adjusts for multiple testing inherent when overlapping time intervals of different length were evaluated.
      • Kulldorff M.
      A spatial scan statistic.
      SaTScan(TM) v8.1.1: Software for the spatial and space–time scan statistics [computer program].
      Under the null hypothesis of no relationship between the potential adverse event and the vaccine, adverse events are assumed to be uniformly and independently distributed during the 120 days after vaccination. To determine potential clusters, all time intervals that started within the risk interval (i.e., 42 days for Bell's Palsy
      • Zhou W.
      • Pool V.
      • DeStefano F.
      • Iskander J.K.
      • Haber P.
      • Chen R.T.
      A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
      • Chou C.H.
      • Liou W.P.
      • Hu K.I.
      • Loh C.H.
      • Chou C.C.
      • Chen Y.H.
      Bell's palsy associated with influenza vaccination: two case reports.
      • Mutsch M.
      • Zhou W.
      • Rhodes P.
      • et al.
      Use of the inactivated intranasal influenza vaccine and the risk of Bell's palsy in Switzerland.
      ) were evaluated.
      Additional signal refinement depends on the vaccine and adverse event for which the signal was generated. In order to minimize bias caused by seasonality of vaccination and disease, the case-centered method was used.
      • Fireman B.
      • Lee J.
      • Lewis N.
      • Bembom O.
      • van der Laan M.
      • Baxter R.
      Influenza vaccination and mortality: differentiating vaccine effects from bias.
      This approach focuses on all cases with an adverse event that occur anytime after vaccination and uses logistic regression analysis to assess the number of cases that were vaccinated inside versus outside a pre-specified risk interval of 42 days prior to the adverse event. For each stratum (defined by case date, age group, gender, VSD site) in which there was a case, all vaccinees in that stratum were examined, including those without adverse events, and the total number of vaccinations given inside versus outside the risk interval was calculated as the offset term.

      Results

      Vaccine Doses

      As of May 1, 2010, there were 1,345,663 individuals vaccinated with MIV, 2,741,150 with TIV, 267,715 with LAMV, and 157,838 with LAIV in the VSD population. The percentages of children and adults vaccinated by age group for H1N1 and for seasonal influenza are shown in Figure 1. Coverage rates by age group varied from 10% to 54% for H1N1 and 15% to 81% for seasonal vaccines. Seasonal influenza vaccination rates were higher than H1N1 vaccination rates in every age group. Seasonal influenza vaccines were administered much earlier than H1N1 vaccines (Figure 2), although the timing of vaccinations varied by site according to local distribution of both types of vaccines.
      Figure thumbnail gr1
      Figure 1Proportion of Vaccine Safety Datalink population vaccinated by age group with H1N1 MIV, H1N1 LAMV, TIV, and LAIV
      LAIV, seasonal live, attenuated influenza vaccine; LAMV, H1N1 live, attenuated monovalent vaccine; MIV, H1N1 monovalent inactivated vaccine; mos, months; TIV, seasonal trivalent inactivated vaccine; yrs, years
      Figure thumbnail gr2
      Figure 2Timing of administration of 1st dose MIV, LAMV, TIV, and MIV in the 2009–2010 season
      LAIV, live, attenuated influenza vaccine; LAMV, H1N1 live, attenuated monovalent vaccine; MIV, H1N1 monovalent inactivated vaccine; TIV, trivalent inactivated vaccine

      Rapid Cycle Analysis of Adverse Events Following Immunization

      Among 11 potential neurologic, allergic, and cardiac adverse events being monitored in different age groups in weekly sequential analyses during the 2009–2010 season (Appendix B, available online at www.ajpmonline.org), there was only one with a statistical signal. For MIV, there was a signal for Bell's palsy for adults aged ≥25 years on March 31, 2010, with 141 observed cases compared to 88 expected, for a relative risk of 1.60 (Figure 3). By May 1, 2010, there were 157 cases of Bell's palsy identified by ICD-9 code 351.0 in the risk interval and 94 in the comparison interval among MIV vaccinees (Appendix B, available online at www.ajpmonline.org), for a relative risk of 1.67. No statistical signals were observed for LAMV, TIV, or LAIV vaccination.
      Figure thumbnail gr3
      Figure 3Log-likelihood ratio and relative risk following MIV for Bell's palsy for adults aged ≥25 years
      Note: The critical value threshold above which a statistical signal is detected is shown by the dashed line.
      MIV, monovalent inactivated vaccine
      Fifteen cases of GBS were identified by ICD-9 codes following MIV, one case after LAMV, 23 cases after TIV, and zero cases after LAIV. Medical record review of GBS cases following H1N1 vaccines identified only eight confirmed cases following MIV (Appendix B, available online at www.ajpmonline.org) and zero cases following LAMV. In current versus historical sequential analyses, the LLR did not exceed the critical value threshold for GBS cases identified either by ICD-9 code alone (LLR 1.56, critical value 3.68) or by medical record review (LLR 2.21, critical value 3.47) during the surveillance season.

      Signal Refinement for Bell's Palsy

      The temporal scan statistic did not find any significant clusters (p=0.19). A case-centered analysis was then conducted and adjusted for case date, age group, gender, and site. There was no significant association between Bell's palsy and being vaccinated in versus out of the risk interval with MIV (OR=1.21, 95% CI=0.93, 1.57). No association between Bell's palsy and TIV (OR=1.10, 95% CI=0.89, 1.37) was observed using the case-centered approach.

      Discussion

      The VSD successfully conducted near–real time, prospective surveillance for adverse events following seasonal and H1N1 influenza vaccines. This served as a key component of public health efforts to protect the health of children and adults in the U.S. during the 2009–2010 influenza season. Prospective surveillance did not detect statistical signals for GBS, demyelinating disease, peripheral nervous system disorders, seizures, encephalomyelitis, ataxia, anaphylaxis, allergic reactions, cranial nerve disorders other than Bell's palsy, or myocarditis for four separate influenza vaccines—MIV, TIV, LAMV and LAIV—in a cohort of approximately 9.2 million children and adults in the U.S. The use of rapid cycle analysis methods strengthened the ability to provide vaccine safety information of high relevance and timeliness to public health policymakers.
      The only statistical signal detected was for Bell's palsy in adults aged ≥25 years receiving MIV and further assessment of this potential signal pointed away from it being a true association. Prior studies have yielded mixed findings regarding the potential association between inactivated influenza vaccines and Bell's palsy.
      • Zhou W.
      • Pool V.
      • DeStefano F.
      • Iskander J.K.
      • Haber P.
      • Chen R.T.
      A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
      • Mutsch M.
      • Zhou W.
      • Rhodes P.
      • et al.
      Use of the inactivated intranasal influenza vaccine and the risk of Bell's palsy in Switzerland.
      • Stowe J.
      • Andrews N.
      • Wise L.
      • Miller E.
      Bell's palsy and parenteral inactivated influenza vaccine.
      Seasonality and arid climates have been associated with Bell's palsy in the U.S. military population, with a significantly higher rate during cold versus warm months (adjusted rate ratio 1.31) and arid climates (adjusted rate ratio 1.34).
      • Campbell K.E.
      • Brundage J.F.
      Effects of climate, latitude, and season on the incidence of Bell's palsy in the U.S. Armed Forces, October 1997 to September 1999.
      By evaluating for temporal clustering and conducting a case-centered logistic regression analysis that controlled for seasonality, further evidence of a causal association between MIV and Bell's palsy was not demonstrated.
      Prospective surveillance for adverse events following vaccination requires the ability to rapidly respond to potential signals that may arise, with a broad range of approaches that allow for a rigorous yet tailored evaluation depending on the adverse event of interest. In our prior experience in RCA, statistical signals often do not lead to the identification of true associations, but rather are a result of type I error, imprecision in estimated background rates, changes in true incidence or coding over time, inappropriate comparison groups, other confounding, or miscoding of outcomes in electronic medical records.

      Yih WK, Kulldorff M, Fireman BH, et al. Active surveillance for adverse events: the experience of the Vaccine Safety Datalink project. Pediatrics; In press.

      Studying the safety of influenza vaccine in near real time is particularly challenging because of confounding and the seasonality of vaccination and certain adverse events. Although the self-controlled design is preferred for monitoring influenza vaccine safety because of its ability to adjust for fixed confounders, it was not feasible to adjust for time-varying confounders, such as seasonality in prospective weekly surveillance.
      In any surveillance system using electronic data, several key issues need to be considered to ensure the ability to detect true signals in a timely fashion while minimizing the risk for false positive signals. First and foremost, the quality of patient-level and aggregate-level data requires ongoing assessment on a weekly basis to determine which sites can be included in analyses. Without rigorous assessment of data quality, false signals may arise. Second, the positive predictive value of ICD-9 code data for potential adverse events is important to consider. In prior work in those aged ≥18 years within the VSD, 1995–2000, 72% of potential Bell's palsy cases were confirmed after chart review (JB, unpublished observations, December 2010).
      Third, the choice of study design using prospective weekly surveillance methods depends on balancing the need for scalability, the potential for bias depending on the adverse event, and the frequency of the adverse event, which may affect signaling power. Because data on influenza vaccination status may be incomplete as a result of use of nontraditional delivery sites, methods that focus on vaccinated populations are preferred.
      • Greene S.K.
      • Shi P.
      • Dutta-Linn M.M.
      • et al.
      Accuracy of data on influenza vaccination status at four Vaccine Safety Datalink sites.
      • Sy L.S.
      • Liu I.L.
      • Solano Z.
      • et al.
      Accuracy of influenza vaccination status in a computer-based immunization tracking system of a managed care organization.
      For example, in the VSD population, 35% received seasonal influenza vaccines (42% of children, 32% of adults) and 19% received H1N1 vaccines (32% of children, 16% of adults) compared to national survey estimates of 41% for seasonal (44% of children, 40% of adults) and 27% (41% of children, 23% of adults) for H1N1 vaccines, which may be due to patients' receiving vaccines outside these medical care organizations (e.g., retail-based clinics, pharmacies, schools, community clinics).
      • Greene S.K.
      • Shi P.
      • Dutta-Linn M.M.
      • et al.
      Accuracy of data on influenza vaccination status at four Vaccine Safety Datalink sites.
      • Sy L.S.
      • Liu I.L.
      • Solano Z.
      • et al.
      Accuracy of influenza vaccination status in a computer-based immunization tracking system of a managed care organization.
      CDC
      Final Estimates for 2009–10 Seasonal Influenza and Influenza A (H1N1) 2009 Monovalent Vaccination Coverage – U.S., August 2009 through May, 2010.
      Some statistical signals may be anticipated because of the inability to adjust for all relevant confounders for each unique adverse event on a weekly basis, particularly if individuals vaccinated in the current season are different from those vaccinated in past seasons.
      • Fiore A.E.
      • Shay D.K.
      • Broder K.
      • et al.
      Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP)
      Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008.
      Further, seasonality of influenza vaccine administration and the seasonality of adverse events (independent of vaccination status) may confound the interpretation of the safety of influenza vaccines. Additional refinement of statistical signals is needed to determine whether a causal association truly exists. Finally, the analytic approach should take into consideration multiple testing over time as well as testing of multiple outcomes if such broad surveillance is needed. For example, five analyses at alpha=0.05 and 28 analyses at alpha=0.01 were performed for inactivated vaccines. Thus, it is not surprising to have observed one signal during surveillance during the 2009–2010 season.
      Because of the need to conduct timely analyses, this study had additional limitations. Adverse events were included only if they were amenable to monitoring in near real time. Thus, pregnancy outcomes following inactivated vaccines or wheezing following live, attenuated vaccines were not included, which are both being monitored in separate analyses at the end of the season when complete data are available. Overlapping risk intervals between H1N1 and seasonal influenza vaccines were not identified as this would require waiting for all risk intervals for both vaccines to elapse fully before conducting analyses, which would have precluded providing timely results. An assumption was also made that individuals remained insured throughout the season because enrollment data were not available in real time. Finally, an insured population was used in order to capture any medically attended adverse events following vaccination; thus, findings may not be generalizable to children and adults who are uninsured or publicly insured.
      In summary, the VSD was able to conduct prospective weekly surveillance of H1N1 and seasonal influenza vaccine safety for the 2009–2010 season. No evidence for a substantial safety concern was identified in sequential analyses among the pre-specified outcomes under surveillance. The ability to conduct timely analyses of safety is critical to safeguard public health and thereby ensure the success of vaccination programs in the U.S. As future vaccines, including new formulations of influenza vaccines, become available for use, VSD can continue to provide national policymakers, such as the National Vaccine Advisory Committee and Advisory Committee on Immunization Practices, with critical information needed for decision making, as it did within the 2009–2010 season.
      We gratefully acknowledge our epidemiologists and programmer analysts at the Harvard Pilgrim Health Care Institute VSD RCA Coordinating Center, including W. Katherine Yih, Irene Shui, Rong Li, Rich Fox, and Robert Jin for support of the VSD Influenza RCA; project managers and programmer analysts from participating VSD sites, including Melisa Rett (Harvard Pilgrim Health Care Institute [HPHCI]), Patti Benson (Group Health Cooperative [GHC]), Leslie Kuckler (HealthPartners Research Foundation [HPM]), Jill Mesa (Kaiser Permanente Northwest [NWK]), Lois Drew (NWK), Deanna Cole (Marshfield Clinic Research Foundation [MFC]), JoAnne Shoup (Kaiser Permanente Colorado [KPC]), Lina Sy (SCK), Zendi Solano Kaiser Permanente Southern California [SCK], Lei Qian (SCK), Paula Ray (Kaiser Permanente Northern California [NCK]); and colleagues from Brigham and Women's Department of Neurology for adjudicating all GBS cases (Anthony Amato, Sarah Sheikh, Doreen Ho). We sincerely thank our CDC and VSD colleagues Jerry Tokars (CDC), Karen Broder (CDC), Richard Platt (HPHCI), Nicola Klein (NCK), Edward Belongia (MFC), Lisa Jackson (GHC), David McClure (KPC), Jason Glanz (KPC), and Simon Hambidge (KPC) for their valuable advice throughout influenza RCA surveillance.
      This work was supported by a subcontract with America's Health Insurance Plans (AHIP) under contract 200-2002-00732 from the CDC.
      The findings and conclusions in this study are those of the authors and do not necessarily represent the official position of the CDC.
      RB has research grants from Sanofi-Pasteur , Novartis , GSK and MedImmune . SJ had research grants from and served as an unpaid consultant to Merck Research Labs .
      No other financial disclosures were reported by the authors of this paper.

      Supplementary data

      References

        • Maurer J.
        • Uscher-Pines L.
        • Harris K.M.
        Perceived seriousness of seasonal and A(H1N1) influenzas, attitudes toward vaccination, and vaccine uptake among U.S. adults: does the source of information matter?.
        Prev Med. 2010; 51: 185-187
        • Retailliau H.F.
        • Curtis A.C.
        • Storr G.
        • Caesar G.
        • Eddins D.L.
        • Hattwick M.A.
        Illness after influenza vaccination reported through a nationwide surveillance system, 1976–1977.
        Am J Epidemiol. 1980; 111: 270-278
      1. Salmon D, Akhtar A, Mergler M, et al. Monitoring immunization safety for the pandemic 2009–2010 H1N1 influenza vaccination program. Pediatrics; In press.

        • Federal Immunization Safety Task Force
        Federal plans to monitor immunization safety for the pandemic 2009 H1N1 influenza vaccination program.
        • Greene S.K.
        • Kulldorff M.
        • Lewis E.M.
        • et al.
        Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink project.
        Am J Epidemiol. 2010; 171: 177-188
        • DeStefano F.
        The Vaccine Safety Datalink project.
        Pharmacoepidemiol Drug Saf. 2001; 10: 403-406
        • Chen R.T.
        • DeStefano F.
        • Davis R.L.
        • et al.
        The Vaccine Safety Datalink: immunization research in health maintenance organizations in the USA.
        Bull World Health Organ. 2000; 78: 186-194
        • Lieu T.A.
        • Kulldorff M.
        • Davis R.L.
        • et al.
        Real-time vaccine safety surveillance for the early detection of adverse events.
        Med Care. 2007; 45: S89-S95
        • Schonberger L.B.
        • Bregman D.J.
        • Sullivan-Bolyai J.Z.
        • et al.
        Guillain-Barré syndrome following vaccination in the National Influenza Immunization Program, U.S., 1976-1977.
        Am J Epidemiol. 1979; 110: 105-123
        • Haber P.
        • DeStefano F.
        Influenza vaccination and Guillain Barré syndrome.
        Clin Immunol. 2003; 109 (author reply 360–51): 359
        • Haber P.
        • DeStefano F.
        • Angulo F.J.
        • et al.
        Guillain-Barré syndrome following influenza vaccination.
        JAMA. 2004; 292: 2478-2481
        • Juurlink D.N.
        • Stukel T.A.
        • Kwong J.
        • et al.
        Guillain-Barré syndrome after influenza vaccination in adults: a population-based study.
        Arch Intern Med. 2006; 166: 2217-2221
        • Souayah N.
        • Nasar A.
        • Suri M.F.
        • Qureshi A.I.
        Guillain-Barré syndrome after vaccination in U.S.: a report from the CDC/FDA vaccine adverse event reporting system.
        Vaccine. 2007; 25: 5253-5255
        • Langmuir A.D.
        • Bregman D.J.
        • Kurland L.T.
        • Nathanson N.
        • Victor M.
        An epidemiologic and clinical evaluation of Guillain-Barré syndrome reported in association with the administration of swine influenza vaccines.
        Am J Epidemiol. 1984; 119: 841-879
        • Haber P.
        Influenza vaccine and neurological adverse events: VAERS 7/1990 - 1/2003.
        (Paper presented at Immunization Safety Review: Influenza Vaccine and Possible Neurological Complications; 2003; Washington DC)
        • Brostoff J.M.
        • Beitverda Y.
        • Birns J.
        Post-influenza vaccine chronic inflammatory demyelinating polyneuropathy.
        Age Ageing. 2008; 37: 229-230
        • Hull T.P.
        • Bates J.H.
        Optic neuritis after influenza vaccination.
        Am J Ophthalmol. 1997; 124: 703-704
        • Nakamura N.
        • Nokura K.
        • Zettsu T.
        • et al.
        Neurologic complications associated with influenza vaccination: two adult cases.
        Intern Med. 2003; 42: 191-194
        • Hull J.H.
        • Mead S.H.
        • Foster O.J.
        • Modarres-Sadeghi H.
        Severe vasculitic neuropathy following influenza vaccination.
        J Neurol Neurosurg Psychiatry. 2004; 75: 1507-1508
        • McMahon A.W.
        • Iskander J.
        • Haber P.
        • et al.
        Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
        Pediatrics. 2005; 115: 453-460
        • Gross W.L.
        • Ravens K.G.
        • Hansen H.W.
        Meningoencephalitis syndrome following influenza vaccination.
        J Neurol. 1978; 217: 219-222
        • Zhou W.
        • Pool V.
        • DeStefano F.
        • Iskander J.K.
        • Haber P.
        • Chen R.T.
        A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
        Pharmacoepidemiol Drug Saf. 2004; 13: 505-510
        • Chou C.H.
        • Liou W.P.
        • Hu K.I.
        • Loh C.H.
        • Chou C.C.
        • Chen Y.H.
        Bell's palsy associated with influenza vaccination: two case reports.
        Vaccine. 2007; 25: 2839-2841
        • Mutsch M.
        • Zhou W.
        • Rhodes P.
        • et al.
        Use of the inactivated intranasal influenza vaccine and the risk of Bell's palsy in Switzerland.
        N Engl J Med. 2004; 350: 896-903
        • Saito H.
        • Yanagisawa T.
        Acute cerebellar ataxia after influenza vaccination with recurrence and marked cerebellar atrophy.
        Tohoku J Exp Med. 1989; 158: 95-103
        • Bohlke K.
        • Davis R.L.
        • Marcy S.M.
        • et al.
        • Vaccine Safety Datalink Team
        Risk of anaphylaxis after vaccination of children and adolescents.
        Pediatrics. 2003; 112: 815-820
        • Coop C.A.
        • Balanon S.K.
        • White K.M.
        • Whisman B.A.
        • Rathkopf M.M.
        Anaphylaxis from the influenza virus vaccine.
        Int Arch Allergy Immunol. 2008; 146: 85-88
        • Izurieta H.S.
        • Haber P.
        • Wise R.P.
        • et al.
        Adverse events reported following live, cold-adapted, intranasal influenza vaccine.
        JAMA. 2005; 294: 2720-2725
        • Horowitz L.
        Urticaria subsequent to administration of influenza vaccine.
        South Med J. 1979; 72: 1334-1335
        • Guarner J.
        • Paddock C.D.
        • Shieh W.J.
        • et al.
        Histopathologic and immunohistochemical features of fatal influenza virus infection in children during the 2003–2004 season.
        Clin Infect Dis. 2006; 43: 132-140
        • Zhou W.
        • Pool V.
        • Iskander J.K.
        • et al.
        Surveillance for safety after immunization: Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991-2001.
        MMWR Surveill Summ. 2003; 52: 1-24
        • Nichol K.L.
        • Margolis K.L.
        • Lind A.
        • et al.
        Side effects associated with influenza vaccination in healthy working adults.
        Arch Intern Med. 1996; 156: 1546-1550
        • Hambidge S.J.
        • Glanz J.M.
        • France E.K.
        • et al.
        • Vaccine Safety Datalink Team
        Safety of trivalent inactivated influenza vaccine in children 6 to 23 months old.
        JAMA. 2006; 296: 1990-1997
        • Klein N.P.
        • Hansen J.
        • Lewis E.
        • et al.
        Post-marketing safety evaluation of a tetanus toxoid, reduced diptheria toxoid and 3-component acellular pertussis vaccine administered to a cohort of adolescents in a United States health maintenance organization.
        Pediatr Infect Dis J. 2010; 29: 613-617
        • Virtanen M.
        • Peltola H.
        • Paunio M.
        • Heinonen O.P.
        Day-to-day reactogenicity and the healthy vaccinee effect of measles-mumps-rubella vaccination.
        Pediatrics. 2000; 106: E62
        • Greene S.K.
        • Kulldorff M.
        • Yin R.
        • et al.
        Near real-time vaccine safety surveillance with partially accrued data.
        Pharmacoepidemiol Drug Saf. 2011; (In press)
        • Kulldorff M.
        • Davis R.L.
        • Kolczak M.
        • Lewis E.M.
        • Lieu T.A.
        • Platt R.
        A maximized sequential probability ratio test for drug and vaccine safety surveillance.
        Seq Anal. 2011; 30: 58-78
        • Li L.
        • Kulldorff M.
        A conditional maximized sequential probability ratio test for pharmacovigilance.
        Stat Med. 2010; 29: 284-295
        • Brighton Collaboration
        Guillain-Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data.
      2. Brookings Institution. Paper presented at Methods for Signal Refinement in Active Medical Product Surveillance; 2010; Washington DC.

        • Kulldorff M.
        A spatial scan statistic.
        Commun Stat Theory Methods. 1997; 6: 1481-1496
      3. SaTScan(TM) v8.1.1: Software for the spatial and space–time scan statistics [computer program].
        Information Management Services [RH3]Inc, Silver Spring MD2010
        • Fireman B.
        • Lee J.
        • Lewis N.
        • Bembom O.
        • van der Laan M.
        • Baxter R.
        Influenza vaccination and mortality: differentiating vaccine effects from bias.
        Am J Epidemiol. 2009; 170: 650-656
        • Stowe J.
        • Andrews N.
        • Wise L.
        • Miller E.
        Bell's palsy and parenteral inactivated influenza vaccine.
        Hum Vaccin. 2006; 2: 110-112
        • Campbell K.E.
        • Brundage J.F.
        Effects of climate, latitude, and season on the incidence of Bell's palsy in the U.S. Armed Forces, October 1997 to September 1999.
        Am J Epidemiol. 2002; 156: 32-39
      4. Yih WK, Kulldorff M, Fireman BH, et al. Active surveillance for adverse events: the experience of the Vaccine Safety Datalink project. Pediatrics; In press.

        • Greene S.K.
        • Shi P.
        • Dutta-Linn M.M.
        • et al.
        Accuracy of data on influenza vaccination status at four Vaccine Safety Datalink sites.
        Am J Prev Med. 2009; 37: 552-555
        • Sy L.S.
        • Liu I.L.
        • Solano Z.
        • et al.
        Accuracy of influenza vaccination status in a computer-based immunization tracking system of a managed care organization.
        Vaccine. 2010; 28: 5254-5259
        • CDC
        Final Estimates for 2009–10 Seasonal Influenza and Influenza A (H1N1) 2009 Monovalent Vaccination Coverage – U.S., August 2009 through May, 2010.
        • Fiore A.E.
        • Shay D.K.
        • Broder K.
        • et al.
        • Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP)
        Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008.
        MMWR Recomm Rep. 2008; 57: 1-60