If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Center for Child Health Care Studies, Department of Population Medicine, Children's Hospital Boston, Boston, MassachusettsHarvard Pilgrim Health Care Institute and Harvard Medical School, Division of Infectious Diseases and Department of Laboratory Medicine, Children's Hospital Boston, Boston, Massachusetts
Center for Child Health Care Studies, Department of Population Medicine, Children's Hospital Boston, Boston, MassachusettsDivision of General Pediatrics, Children's Hospital Boston, Boston, Massachusetts
The emergence of pandemic H1N1 influenza virus in early 2009 prompted the rapid licensure and use of H1N1 monovalent inactivated (MIV) and live, attenuated (LAMV) vaccines separate from seasonal trivalent inactivated (TIV) and live, attenuated (LAIV) influenza vaccines. A robust influenza immunization program in the U.S. requires ongoing monitoring of potential adverse events associated with vaccination.
Purpose
To prospectively conduct safety monitoring of H1N1 and seasonal influenza vaccines during the 2009–2010 season.
Methods
The Vaccine Safety Datalink (VSD) Project monitors ∼9.2 million members in eight U.S. medical care organizations. Electronic data on vaccines and pre-specified adverse events were updated and analyzed weekly for signal detection from November 2009 to April 2010 using either a self-controlled design or a current versus historical comparison. Statistical signals were further evaluated using alternative approaches to identify temporal clusters and to control for time-varying confounders.
Results
As of May 1, 2010, a total of 1,345,663 MIV, 267,715 LAMV, 2,741,150 TIV, and 157,838 LAIV doses were administered in VSD. No significant associations were noted during sequential analyses for Guillain–Barré syndrome, most other neurologic outcomes, and allergic and cardiac events. For MIV, a statistical signal was observed for Bell's palsy for adults aged ≥25 years on March 31, 2010, using the self-controlled approach. Subsequent analyses revealed no significant temporal cluster. Case-centered logistic regression adjusting for seasonality demonstrated an OR for Bell's palsy of 1.26 (95% CI=0.97, 1.63).
Conclusions
No major safety problems following H1N1 or seasonal influenza vaccines were detected in the 2009–2010 season in weekly sequential analyses. Seasonality likely contributed to the Bell's palsy signal following MIV. Prospective safety monitoring followed by rigorous signal refinement is critical to inform decision-making by regulatory and public health agencies.
Background
The emergence of a novel H1N1 influenza virus of swine origin in North America in April 2009 prompted the rapid development of new H1N1 vaccines, which became available for use in October 2009. While the initial uncertainty of how the novel H1N1 pandemic would evolve led to heightened concerns about disease prevention, public anxiety about the safety of novel H1N1 vaccines also became apparent.
Perceived seriousness of seasonal and A(H1N1) influenzas, attitudes toward vaccination, and vaccine uptake among U.S. adults: does the source of information matter?.
Salmon D, Akhtar A, Mergler M, et al. Monitoring immunization safety for the pandemic 2009–2010 H1N1 influenza vaccination program. Pediatrics; In press.
The Vaccine Safety Datalink Project (VSD), initiated by the CDC, is widely acknowledged as the backbone of active surveillance for vaccine safety in the U.S.
At the beginning of the 2009–2010 season, near–real time safety surveillance was initiated for both H1N1 and seasonal influenza vaccines within the VSD Project. Findings regarding the safety of influenza vaccines based on sequential analyses during the 2009–2010 season are presented, and the subsequent evaluation of a signal (i.e., a potential adverse event occurring more frequently than anticipated following vaccination
) detected in the surveillance system is described.
Methods
Study Population
The Vaccine Safety Datalink Project is a collaboration among eight medical care organizations and CDC, which has performed population-based research on vaccine safety in the U.S. since 1990.
These organizations include Group Health Cooperative (Washington); Harvard Vanguard Medical Associates & Harvard Pilgrim Health Care (Massachusetts); HealthPartners Research Foundation (Minnesota); Kaiser Permanente of Colorado (Colorado); Kaiser Permanente of Northern California (California); Kaiser Permanente of Southern California (California); Marshfield Clinic Research Foundation (Wisconsin); and Northwest Kaiser Permanente (Oregon). Since 2005, VSD has conducted prospective weekly surveillance, or rapid cycle analysis (RCA), to monitor the safety of newly licensed vaccines and new vaccine recommendations.
At the start of the 2009–2010 influenza vaccination season, the VSD population included approximately 9.2 million children and adults, representing close to 3% of the U.S. population.
Data Sources
The Vaccine Safety Datalink Project uses a distributed data network model in which standardized data files are created and maintained by each site. Every week, each of the eight VSD sites updates data files with all immunizations (including manufacturer, lot number, injection site, and concurrent immunizations); hospital admissions; emergency department visits; outpatient visits; and ICD-9 codes for all patient encounters. Potential adverse events were identified using ICD-9 code diagnosis data from healthcare encounters (Appendix A, available online at www.ajpmonline.org). Other files contain demographic and enrollment data for each patient. For weekly analyses, aggregate, de-identified data sets were created that included counts of individuals in different strata by VSD site; age; gender; vaccine type (inactivated or live, attenuated H1N1 and seasonal influenza vaccines); week of vaccine administration; and type, setting, and timing of potential adverse events.
Potential Adverse Events
Prospective weekly surveillance was conducted from November 2009 to April 2010 on pre-specified potential adverse events selected for being clinically well defined, serious, and potentially associated with influenza vaccines (Appendix A, available online at www.ajpmonline.org). Potential adverse events identified by ICD-9 code data and occurring in pre-specified risk intervals after each patient's first dose of H1N1 or seasonal influenza vaccine monitored in the 2009–2010 season included GBS,
Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
Post-marketing safety evaluation of a tetanus toxoid, reduced diptheria toxoid and 3-component acellular pertussis vaccine administered to a cohort of adolescents in a United States health maintenance organization.
This approach was used for more common adverse events (≥50 adverse events anticipated in an analytic stratum during the season) to adjust for known and unknown time-invariant confounders.
The self-controlled risk interval design examined whether the risk of adverse events in the risk interval after vaccination was greater than in a comparison interval in the same individual. For most adverse events, a pre-vaccination comparison interval was used to facilitate timely analyses. Because some patients may delay or forgo routine influenza vaccination shortly after experiencing an illness, otherwise known as the healthy vaccinee effect, the pre-comparison interval excluded the 14 days prior to vaccination.
For certain adverse events with short risk intervals (i.e., ≤15 days) where (1) a recent prior adverse event might preclude vaccination (i.e., allergic reaction and anaphylaxis) or (2) individuals might have an underlying condition that is also an indication for vaccination (i.e., seizure disorder), a comparison interval occurring after the risk interval was used. The self-controlled risk interval approach could not be used for timely surveillance of adverse events with longer risk intervals (i.e., 42 days), such as GBS, where using a comparison interval after the risk interval would be most appropriate. Selected outcomes, such as demyelinating disease, were divided into strata by age to evaluate the possibility of effect modification. All ages were combined for allergic adverse events because effect modification was less likely.
Because there is wide variability in the timeliness of inpatient data, depending on whether data are derived from electronic medical record systems or medical claims data, inpatient data were included for each site only after an estimated 95% of data accrued, to ensure that comparisons between risk and comparison intervals were valid.
Additionally, in order to conduct timely analyses, intervals were scaled appropriately. For example, if only 1 week of a 42-day risk interval had elapsed since vaccination, these data were compared to the corresponding first week of the 42-day comparison interval.
To account for multiple testing, sequential statistical methods were used. The maximized sequential probability ratio test (MaxSPRT) uses a one-sided composite alternative hypothesis of excess risk in the exposed postvaccination interval.
The null hypothesis assumes that the risk of an adverse event occurring during the risk interval is the same as the risk of it occurring during the comparison interval, scaled by the length of the intervals. For each adverse event and age group, “upper limits” and “critical values” using a one-tailed test and stringent alpha level of 0.01 were obtained.
Surveillance for the 2009–2010 season ended when (1) the log-likelihood ratio (LLR) test statistic exceeded the critical value, generating a “statistical signal”; or (2) the total number of adverse events reached a pre-specified upper limit; or (3) the vaccination season ended on April 30, 2010.
Current versus historical comparison for signal detection—Poisson-based and conditional MaxSPRT
The current versus historical approach was used for extremely rare adverse events (<50 adverse events anticipated based on the historical VSD rates) to improve power and timeliness to detect a signal. In particular, this approach was used for prospective, timely surveillance of GBS. This approach compared the rate of adverse events in the risk interval following vaccination with a historical background rate. Historical rates were calculated from data available in VSD using the rate of adverse events in the risk interval following seasonal TIV during the 2000–2001 through 2008–2009 seasons, or the overall rate of adverse events in the entire VSD population. Current MIV or TIV vaccinees were compared to historical TIV vaccinees. Because LAMV and LAIV vaccinees were less likely to have high-risk conditions, they were compared to historical all-person time rates. Site and age adjustments were performed to estimate the expected number of adverse events for the current season.
Either the Poisson-based MaxSPRT or the Poisson-based conditional MaxSPRT (CMaxSPRT) was used.
Upper limits and critical values were based on the estimated number of doses expected to be administered using a one-tailed test and an alpha level of 0.05 for GBS and anaphylaxis and a more stringent alpha level of 0.01 for all other adverse events.
The proportion of adverse events expected each week was adjusted downward to account for lag in the arrival of inpatient data and the proportion of the risk interval that had elapsed. The CMaxSPRT approach was used when the ratio of the number of observed historical adverse events to the upper limit was ≤2.5, in order to account for the uncertainty in the estimates of the expected number of adverse events under the null hypothesis.
Medical record reviews were prospectively conducted on all GBS cases identified in the risk interval after H1N1 vaccination using electronic medical records available at each VSD site. Medical record reviews identified whether GBS cases were new-onset, follow-up visits for a previous condition, or incorrectly coded diagnoses, within a 1–2-week period after identification in claims data. In order to estimate comparable historical rates, medical record reviews on GBS cases that occurred after TIV during the 2000–2001 to 2008–2009 seasons in the VSD population were also conducted to understand whether the risk for GBS was higher in the current season following MIV compared to historical seasons following TIV. All cases were adjudicated by at least two neurologists with expertise in demyelinating disease using the GBS and Fisher syndrome definitions provided by the Brighton Collaboration, an international group that develops standardized criteria for potential vaccine-associated adverse events.
Brighton Collaboration Guillain-Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data.
“Signals” occur when the rate of an adverse event following influenza vaccination is significantly greater than the anticipated adverse event rate in a comparison group. Statistical signals do not necessarily represent true associations. Rather, they indicate the need for further study to determine if a true association exists. In order to detect true associations rapidly, safety surveillance systems must allow a reasonable amount of type I error to ensure early identification and investigation.
To further refine statistical signals generated by sequential analyses,
the presence of temporal clusters was assessed using a temporal scan statistic, which adjusts for multiple testing inherent when overlapping time intervals of different length were evaluated.
Under the null hypothesis of no relationship between the potential adverse event and the vaccine, adverse events are assumed to be uniformly and independently distributed during the 120 days after vaccination. To determine potential clusters, all time intervals that started within the risk interval (i.e., 42 days for Bell's Palsy
A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
Additional signal refinement depends on the vaccine and adverse event for which the signal was generated. In order to minimize bias caused by seasonality of vaccination and disease, the case-centered method was used.
This approach focuses on all cases with an adverse event that occur anytime after vaccination and uses logistic regression analysis to assess the number of cases that were vaccinated inside versus outside a pre-specified risk interval of 42 days prior to the adverse event. For each stratum (defined by case date, age group, gender, VSD site) in which there was a case, all vaccinees in that stratum were examined, including those without adverse events, and the total number of vaccinations given inside versus outside the risk interval was calculated as the offset term.
Results
Vaccine Doses
As of May 1, 2010, there were 1,345,663 individuals vaccinated with MIV, 2,741,150 with TIV, 267,715 with LAMV, and 157,838 with LAIV in the VSD population. The percentages of children and adults vaccinated by age group for H1N1 and for seasonal influenza are shown in Figure 1. Coverage rates by age group varied from 10% to 54% for H1N1 and 15% to 81% for seasonal vaccines. Seasonal influenza vaccination rates were higher than H1N1 vaccination rates in every age group. Seasonal influenza vaccines were administered much earlier than H1N1 vaccines (Figure 2), although the timing of vaccinations varied by site according to local distribution of both types of vaccines.
Figure 1Proportion of Vaccine Safety Datalink population vaccinated by age group with H1N1 MIV, H1N1 LAMV, TIV, and LAIV
Rapid Cycle Analysis of Adverse Events Following Immunization
Among 11 potential neurologic, allergic, and cardiac adverse events being monitored in different age groups in weekly sequential analyses during the 2009–2010 season (Appendix B, available online at www.ajpmonline.org), there was only one with a statistical signal. For MIV, there was a signal for Bell's palsy for adults aged ≥25 years on March 31, 2010, with 141 observed cases compared to 88 expected, for a relative risk of 1.60 (Figure 3). By May 1, 2010, there were 157 cases of Bell's palsy identified by ICD-9 code 351.0 in the risk interval and 94 in the comparison interval among MIV vaccinees (Appendix B, available online at www.ajpmonline.org), for a relative risk of 1.67. No statistical signals were observed for LAMV, TIV, or LAIV vaccination.
Figure 3Log-likelihood ratio and relative risk following MIV for Bell's palsy for adults aged ≥25 years
Fifteen cases of GBS were identified by ICD-9 codes following MIV, one case after LAMV, 23 cases after TIV, and zero cases after LAIV. Medical record review of GBS cases following H1N1 vaccines identified only eight confirmed cases following MIV (Appendix B, available online at www.ajpmonline.org) and zero cases following LAMV. In current versus historical sequential analyses, the LLR did not exceed the critical value threshold for GBS cases identified either by ICD-9 code alone (LLR 1.56, critical value 3.68) or by medical record review (LLR 2.21, critical value 3.47) during the surveillance season.
Signal Refinement for Bell's Palsy
The temporal scan statistic did not find any significant clusters (p=0.19). A case-centered analysis was then conducted and adjusted for case date, age group, gender, and site. There was no significant association between Bell's palsy and being vaccinated in versus out of the risk interval with MIV (OR=1.21, 95% CI=0.93, 1.57). No association between Bell's palsy and TIV (OR=1.10, 95% CI=0.89, 1.37) was observed using the case-centered approach.
Discussion
The VSD successfully conducted near–real time, prospective surveillance for adverse events following seasonal and H1N1 influenza vaccines. This served as a key component of public health efforts to protect the health of children and adults in the U.S. during the 2009–2010 influenza season. Prospective surveillance did not detect statistical signals for GBS, demyelinating disease, peripheral nervous system disorders, seizures, encephalomyelitis, ataxia, anaphylaxis, allergic reactions, cranial nerve disorders other than Bell's palsy, or myocarditis for four separate influenza vaccines—MIV, TIV, LAMV and LAIV—in a cohort of approximately 9.2 million children and adults in the U.S. The use of rapid cycle analysis methods strengthened the ability to provide vaccine safety information of high relevance and timeliness to public health policymakers.
The only statistical signal detected was for Bell's palsy in adults aged ≥25 years receiving MIV and further assessment of this potential signal pointed away from it being a true association. Prior studies have yielded mixed findings regarding the potential association between inactivated influenza vaccines and Bell's palsy.
A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
Seasonality and arid climates have been associated with Bell's palsy in the U.S. military population, with a significantly higher rate during cold versus warm months (adjusted rate ratio 1.31) and arid climates (adjusted rate ratio 1.34).
By evaluating for temporal clustering and conducting a case-centered logistic regression analysis that controlled for seasonality, further evidence of a causal association between MIV and Bell's palsy was not demonstrated.
Prospective surveillance for adverse events following vaccination requires the ability to rapidly respond to potential signals that may arise, with a broad range of approaches that allow for a rigorous yet tailored evaluation depending on the adverse event of interest. In our prior experience in RCA, statistical signals often do not lead to the identification of true associations, but rather are a result of type I error, imprecision in estimated background rates, changes in true incidence or coding over time, inappropriate comparison groups, other confounding, or miscoding of outcomes in electronic medical records.
Yih WK, Kulldorff M, Fireman BH, et al. Active surveillance for adverse events: the experience of the Vaccine Safety Datalink project. Pediatrics; In press.
Studying the safety of influenza vaccine in near real time is particularly challenging because of confounding and the seasonality of vaccination and certain adverse events. Although the self-controlled design is preferred for monitoring influenza vaccine safety because of its ability to adjust for fixed confounders, it was not feasible to adjust for time-varying confounders, such as seasonality in prospective weekly surveillance.
In any surveillance system using electronic data, several key issues need to be considered to ensure the ability to detect true signals in a timely fashion while minimizing the risk for false positive signals. First and foremost, the quality of patient-level and aggregate-level data requires ongoing assessment on a weekly basis to determine which sites can be included in analyses. Without rigorous assessment of data quality, false signals may arise. Second, the positive predictive value of ICD-9 code data for potential adverse events is important to consider. In prior work in those aged ≥18 years within the VSD, 1995–2000, 72% of potential Bell's palsy cases were confirmed after chart review (JB, unpublished observations, December 2010).
Third, the choice of study design using prospective weekly surveillance methods depends on balancing the need for scalability, the potential for bias depending on the adverse event, and the frequency of the adverse event, which may affect signaling power. Because data on influenza vaccination status may be incomplete as a result of use of nontraditional delivery sites, methods that focus on vaccinated populations are preferred.
For example, in the VSD population, 35% received seasonal influenza vaccines (42% of children, 32% of adults) and 19% received H1N1 vaccines (32% of children, 16% of adults) compared to national survey estimates of 41% for seasonal (44% of children, 40% of adults) and 27% (41% of children, 23% of adults) for H1N1 vaccines, which may be due to patients' receiving vaccines outside these medical care organizations (e.g., retail-based clinics, pharmacies, schools, community clinics).
Some statistical signals may be anticipated because of the inability to adjust for all relevant confounders for each unique adverse event on a weekly basis, particularly if individuals vaccinated in the current season are different from those vaccinated in past seasons.
Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP) Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008.
Further, seasonality of influenza vaccine administration and the seasonality of adverse events (independent of vaccination status) may confound the interpretation of the safety of influenza vaccines. Additional refinement of statistical signals is needed to determine whether a causal association truly exists. Finally, the analytic approach should take into consideration multiple testing over time as well as testing of multiple outcomes if such broad surveillance is needed. For example, five analyses at alpha=0.05 and 28 analyses at alpha=0.01 were performed for inactivated vaccines. Thus, it is not surprising to have observed one signal during surveillance during the 2009–2010 season.
Because of the need to conduct timely analyses, this study had additional limitations. Adverse events were included only if they were amenable to monitoring in near real time. Thus, pregnancy outcomes following inactivated vaccines or wheezing following live, attenuated vaccines were not included, which are both being monitored in separate analyses at the end of the season when complete data are available. Overlapping risk intervals between H1N1 and seasonal influenza vaccines were not identified as this would require waiting for all risk intervals for both vaccines to elapse fully before conducting analyses, which would have precluded providing timely results. An assumption was also made that individuals remained insured throughout the season because enrollment data were not available in real time. Finally, an insured population was used in order to capture any medically attended adverse events following vaccination; thus, findings may not be generalizable to children and adults who are uninsured or publicly insured.
In summary, the VSD was able to conduct prospective weekly surveillance of H1N1 and seasonal influenza vaccine safety for the 2009–2010 season. No evidence for a substantial safety concern was identified in sequential analyses among the pre-specified outcomes under surveillance. The ability to conduct timely analyses of safety is critical to safeguard public health and thereby ensure the success of vaccination programs in the U.S. As future vaccines, including new formulations of influenza vaccines, become available for use, VSD can continue to provide national policymakers, such as the National Vaccine Advisory Committee and Advisory Committee on Immunization Practices, with critical information needed for decision making, as it did within the 2009–2010 season.
We gratefully acknowledge our epidemiologists and programmer analysts at the Harvard Pilgrim Health Care Institute VSD RCA Coordinating Center, including W. Katherine Yih, Irene Shui, Rong Li, Rich Fox, and Robert Jin for support of the VSD Influenza RCA; project managers and programmer analysts from participating VSD sites, including Melisa Rett (Harvard Pilgrim Health Care Institute [HPHCI]), Patti Benson (Group Health Cooperative [GHC]), Leslie Kuckler (HealthPartners Research Foundation [HPM]), Jill Mesa (Kaiser Permanente Northwest [NWK]), Lois Drew (NWK), Deanna Cole (Marshfield Clinic Research Foundation [MFC]), JoAnne Shoup (Kaiser Permanente Colorado [KPC]), Lina Sy (SCK), Zendi Solano Kaiser Permanente Southern California [SCK], Lei Qian (SCK), Paula Ray (Kaiser Permanente Northern California [NCK]); and colleagues from Brigham and Women's Department of Neurology for adjudicating all GBS cases (Anthony Amato, Sarah Sheikh, Doreen Ho). We sincerely thank our CDC and VSD colleagues Jerry Tokars (CDC), Karen Broder (CDC), Richard Platt (HPHCI), Nicola Klein (NCK), Edward Belongia (MFC), Lisa Jackson (GHC), David McClure (KPC), Jason Glanz (KPC), and Simon Hambidge (KPC) for their valuable advice throughout influenza RCA surveillance.
This work was supported by a subcontract with America's Health Insurance Plans (AHIP) under contract 200-2002-00732 from the CDC.
The findings and conclusions in this study are those of the authors and do not necessarily represent the official position of the CDC.
RB has research grants from Sanofi-Pasteur , Novartis , GSK and MedImmune . SJ had research grants from and served as an unpaid consultant to Merck Research Labs .
No other financial disclosures were reported by the authors of this paper.
Perceived seriousness of seasonal and A(H1N1) influenzas, attitudes toward vaccination, and vaccine uptake among U.S. adults: does the source of information matter?.
Salmon D, Akhtar A, Mergler M, et al. Monitoring immunization safety for the pandemic 2009–2010 H1N1 influenza vaccination program. Pediatrics; In press.
Adverse events after inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990–2003.
A potential signal of Bell's palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS)—U.S., 1991–2001.
Post-marketing safety evaluation of a tetanus toxoid, reduced diptheria toxoid and 3-component acellular pertussis vaccine administered to a cohort of adolescents in a United States health maintenance organization.
Yih WK, Kulldorff M, Fireman BH, et al. Active surveillance for adverse events: the experience of the Vaccine Safety Datalink project. Pediatrics; In press.
00260-1&id=gr1.jpg){kind=link}
00260-1&id=gr2.jpg){kind=link}
00260-1&id=gr3.jpg){kind=link}