| BackgroundSyndromic surveillance uses both clinical and non-clinical data that are discernable before diagnosis is confirmed and activities prompted by the onset of symptoms, as an alert to changes in disease activity. These may include emergency department complaints, over-the-counter medication sales and absenteeism information. Syndromic surveillance has been widely used for the early detection of outbreaks, to follow the size, spread, and tempo of outbreaks, to monitor disease trends, and to provide reassurance that an outbreak has not occurred. Studies on syndromic surveillance in China have been mostly aimed at understanding the overall incidence trends and seasonal models of a given disease. Research on outbreak detection methods using syndromic surveillance data has not been published yet. With its high sensitivity and timely results, syndromic surveillance has been applied to mass gatherings in different countries and areas to detect infectious and bioterrorism-related disease outbreaks. The 41st World Expo was opened in Shanghai on May 1,2010. This event brought 70 million visitors to Shanghai and prompted the need to establish an effective and enhanced surveillance system to provide data for public-health protection. As there have been few such experiences before, urgency was required in selecting a surveillance index and choosing an effective method to detect potential public-health threats.ObjectiveTo select suitable syndromic surveillance indicators for a mass gathering, and compare different early warning models based on real surveillance data and simulated outbreaks.Methods1. Surveillance index selectionAfter studying the book "Infectious Disease" and other references, we combined the related research experiences within China and from overseas, and listed the more concerning diseases, and their typical symptoms, that have been observed in syndromic surveillance during mass gatherings in China. We chose the Delphi method to consult with 18 domestic experts in the related areas twice and scored the disease symptoms based on typicality.On the basis of the scores gathered from the experts, we drew-up a disease-risk matrix for syndromic surveillance during mass gatherings to define the surveillance objectives and risk probabilities By combining the scores and the five syndromic indices suggested in the five syndromics pathogen-spectrum surveillance sub-project of the National Science and Technology Key Projects, we screened for the research symptoms and syndromes.2. Data collectionBased on the chosen surveillance indices, the syndromic surveillance system was established and was run in 21 medical institutions in Pudong, Shanghai during the World Expo period (May 1 to October 31,2010). We considered Sanlin and Lujiazui communities as the most two important surveillance regions, and chose two upper hospitals, five general middle hospitals and 14 community sanitary service centers as the surveillance sites. All the surveillance data were collected electronically by clinic doctors in the Hospital Information System (HIS), stored in the front-end processors of hospitals, and automatically transferred to the Pudong Syndromic Surveillance platform at 24:00 every day. Personnel were assigned to check the quality of the data reports, and to correct data leakages and wrongly reported data.3. Data descriptionWe calculated the daily reported cases, and the time series (184 days) of each syndrome. We then observed the incidence levels, case distributions, the weekend effect and others items in order to supply evidence for choosing and designing the outbreak detection models.4. Outbreak detection modelsAccording to the surveillance data characteristics, we chose five popular time-series models for outbreak detection. These five models included the cumulative sum control model (CUSUM), the C1, C2 and C3 models in Early Aberration Reporting System(EARS), and the exponentially weighted moving average (EWMA) model. These five models were then compared for their efficacy.5. Outbreak simulation During the surveillance period, there were no reported infectious disease outbreaks or biological terror threats. However, using our research, we added various types of simulated outbreaks to the actual surveillance data in order to evaluate and compare the outbreak detection effect of the five models.To comprehensively test the five outbreak detection models, we simulated outbreaks in three dimensions, including size, duration and distribution. Five outbreaks factors, at different levels from each syndrome were chosen to simulate the different outbreak sizes. For the incubation periods we chose outbreak periods of 5,10 and 15 days, and for distribution characteristics and main transmission routes, we chose outbreaks shapes of normal and poisson distributions. There were a total of 30 (5×3×2) kinds of outbreaks when the three dimensions were combined.We used each kind of outbreak in each of the five syndromic time-series models. Other than the time set aside at the baseline for calculations at the beginning of the series, we ensured that each outbreak cycle was complete by the end of series, and each day in the series was considered the the beginning of one outbreak; that is to say each type of outbreak was simulated N times (N=the length of the series-the length of the baseline-duration of outbreak+1). As soon as each simulation began, the detection results for each of the five models were recorded, including data on whether the model detected the outbreak at all and the time to detection after the start of the outbreak.6. Evaluation of model performanceIn order to increase the comparability of the detective efficacies of the five models, we chose three specificities of 95%,90% and 85%, and compared the sensitivities and time-to-detection of the five models at each of the specificities. We detected 30 kinds of simulated outbreaks and then calculated the sensitivities and time-to-detection to compare the detective efficacies between the five models using the paired t test.Results1. Target disease and risk levelsWe identified 55 target diseases for syndrome surveillance in mass gatherings through literature reviews, expert consultations and the risk matrix. These diseases included eight digestive system diseases,11 respiratory system diseases, seven animal-borne diseases, eight insect-borne diseases, two parasitic diseases, five blood- and sexually transmitted diseases,14 bioterrorism-related diseases and diseases that have not yet been detected in China.The 55 target diseases were divided into four risk levels. There were no extremely high-risk diseases, but ten of the diseases were deemed to be high risk, such as cholera, plague, botulism, meningitis, and seasonal influenza. Forty of the diseases were medium risk, such as hand, foot and mouth disease (HFMD), chicken pox, anthrax, encephalitis, schistosomiasis, and tick-borne viral encephalitis, and five were low-risk diseases, such as brucellosis, leptospirosis, and echinococcosis.2. Surveillance indicatorsWe chose seven syndromes as the syndromic surveillance indicators during mass gatherings, which included the 25 symptoms listed below:2.1. Respiratory syndrome-fever with at least one of the following:cough, sputum, hemoptysis, chest pain, breathing difficulties2.2. Gastrointestinal syndrome-fever with at least one of the following:vomiting, diarrhea, pus and blood in mucous2.3. Rash with fever syndrome-fever with at least one of the following:herpes, maculopapular rash2.4. Neurological syndrome-fever with at least one of the following:headache, projectile vomiting, shock, altered consciousness2.5. Fever with hemorrhagic syndrome-fever with at least one of the following:skin and mucous congestion, petechiae, bleeding, bloody stool2.6. Botulinic syndrome-sudden vision disturbances or difficulty swallowing2.7. Acute viral hepatitis syndrome-hepatosplenomegaly or acute jaundice3. Data descriptionThe incidence levels varied among the seven syndromes. The respiratory and gastrointestinal syndromes had the most reported cases at 325 cases/day and 248 cases/day, respectively. Fever with hemorrhagic and botulinic syndromes had the lowest incidence levels, which were around 1 case/day each.Using a histogram we found that the data collected for the respiratory and gastrointestinal syndromes followed a normal distribution, while the data for the other five syndromes followed a partly normal distribution. Using box charts we found that the weekend effect existed in the data collected for the gastrointestinal and acute viral hepatitis syndromes, with the average number of daily reported cases on weekdays larger than that on the weekends. The weekend effect was not obvious for the other five syndromes.4. Model performance4.1. Respiratory syndromeThe sensitivities of the five models were >80% when the specificity was set at 95%, and the median detection time was about 2 days. The C2 model and CUSUM had the highest sensitivities (p<0.005), at 86.48% and 86.70%, respectively.The sensitivities of all the models were >90% when the specificity was set at 90%, and median detection time was 1 day. The sensitivity of CUSUM was 93.39%, the highest (p<0.001) among the five models.The sensitivities of the five models were >95% when the specificity was set at 85%, and the median detection time was 1 day. The EWMA model had the highest sensitivity (0.01<p<0.05), at 97.10%.4.2. Gastrointestinal syndromeThe sensitivities of the five models were between 85% and 90% when the specificity was set at 95%, and the median detection time was 1-2 days. CUSUM, and the C1 and C3 models had the highest sensitivities (p<0.01), at 89.93%,88.97% and 88.97%, respectively.The sensitivities of the five models were between 90% and 95% when the specificity was set at 90%, and the median detection time was 1 day. The C2 model had the highest sensitivity (p<0.05) at 94.6%.The sensitivities of the five models were >95% when the specificity was set at 85%, and the median detection time was 1 day. The C2 model had the highest sensitivity (p<0.05) at 96.75%.4.3. Rash with fever syndrome The sensitivities of the five models were between 85% and 90% when the specificity was set at 95%, and the median detection time was 1-2 days. CUSUM, and the C1 and C3 models had the highest sensitivities (p<0.01) at 89.93%,88.97% and 88.97%, respectively.The sensitivities of the five models were between 90% and 95% when the specificity was set at 90%, and the median detection time was 1 day. The C2 model had the highest sensitivity (p<0.05) at 94.6%.The sensitivities of the five models were>95% when the specificity was set at 85%, and the median detection time was 1 day. The C2 model had the highest sensitivity (p<0.05) at 96.75%.4.4. Neurological syndromeThe sensitivity of the C3 model was 95%, which was significantly higher than the other four models (p<0.01) when the specificity was set at 95%. The median detection time for all five models was 0 days.The sensitivities of the five models were >95% when the specificity was set at 90%. The sensitivities of CUSUM and the C3 model were the highest (p<0.05) at 98.08% and 97.51%, respectively. The median detection time for all five models was 0 days. The sensitivities of the five models were >98% when the specificity was set at 85%, and the median detection time was 0 days. There are no differences in sensitivity between the models (p>0.05).4.5. Fever with hemorrhagic syndromeThe sensitivities of the five models were between 40% and 60% when the specificity was set at 95%, and the median detection time for all five models was 3.5-5 days. The detection efficacy of the C2 model was the best (p<0.001) at 58.60%, and the median detection time for all the outbreaks was 3.5 days.The sensitivities of the five models were between 75% and 80% when the specificity was set at 90%, and the median detection time was 1 day. The C3 model had the highest sensitivity (p<0.01) at 79.76%.The sensitivities of the five models were >80% when the specificity was set at 85%, and the median detection time was 0 days. The C2 and C3 models had the highest sensitivity (p<0.001), both at 84.57%.4.6. Botulinic syndromeThe sensitivity of the C3 model was 66.65% when the specificity was set at 95%, and the median detection time was 1 day, which makes its detection efficacy the best (p<0.005).The sensitivities of the five models were close to 80% when the specificity was set at 90%, and the median detection time was 0 days. The C3 model and CUSUM had the highest sensitivity (p<0.05) at 79.43% and 79.41%, respectively.The sensitivities of the five models were >85% and <90% when the specificity was set at 85%, and the median detection time was 0 days. There are no differences in sensitivity between the models.4.7. Acute viral hepatitis syndromeThe C2 and C3 models had the highest sensitivities (p<0.001) at 82.20% and 81.67%, respectively, which was significantly more than the other three models when the specificity was set at 95%. The median detection time for all five models was 0 days.The sensitivities of the five models were between 80% and 85% when the specificity was set at 90%, and the median detection time was 0 days. The C3 and EWMA models had significantly higher sensitivities (p<0.05) than the other three models.The sensitivities of the five models were >90% when the specificity was set at 85%, and the median detection time was 0 days. There are no differences in sensitivity between the models (p>0.05).Conclusions1. The five selected models detected outbreaks in respiratory, gastrointestinal and neurological syndromes. Under different conditions, the sensitivities and time-to-detection were high. CUSUM and the C1, C2, C3 and EWMA models can be used effectively for outbreak detection and early warning on symptoms with a normal distribution and a high level of morbidity.2. When used for the detection of two types of syndrome (botulinic syndromes and fever with hemorrhagic syndromes) and with three levels of specificity, the detection results of all five models were not satisfactory. Even with the specificity set at 85%, none of the models had more than a 90% sensitivity. For this type of surveillance, it is suggested that another methodology should be chosen for early warning outbreak detection.3. Under different specificities, the sensitivities and time-to-detection of all the models vary widely. During syndromic surveillance, a full assessment of relevant public-health resources and the workload caused by the standby signal should be undertaken based on specific job requirements and the response should be determined by an appropriate level of specificity. The optimal sensitivity and time-to-detection of the model should then be selected under these conditions. |
PDF Full Text Request