传染病时空模型预警技术评价研究
|
摘要
研究背景
2004年我国传染病网络直报系统的启动,使我国传染病疫情报告发生了质的飞跃。为了充分利用监测信息资源,实现病例空间聚集性探测预警,中国疾病预防控制中心在时间模型预警技术研究基础上,进一步开展传染病时空模型预警技术研究,并于2008年4月21日在全国20个省份的221个试点县(区)启动了为期1年的传染病自动预警(时空模型)试点工作。因此,为修改和完善时空模型预警技术,目前急需建立成套的传染病时空模型预警系统评价指标体系,对试点效果开展评价,从而为传染病时空预警技术的改进提供科学依据和建议。
研究目的
1.探索建立传染病自动预警(时空模型)系统评价指标体系。
2.通过时空模型试点结果分析和用户问卷调查,综合评价时空模型的预警效果,为时空模型预警技术的完善提供科学依据和建议。
资料与方法
1.试点数据收集
研究收集了试点地区的时空模型预警数据、传染病监测数据和相关突发事件信息以及有关技术材料。
2.预警评价指标体系的构建
通过文献回顾和预警系统数据特征分析,初步构建预警评价框架；再通过专家咨询的方法,由专家对评价框架和指标内容进行评判并提出修改意见和建议,从而建立起传染病预警系统评价指标体系。
3.预警用户问卷调查
对试点地区疾控机构预警系统用户开展问卷调查,分析用户对预警模型使用情况,对预警及时性、灵敏度、假阳性率等时空模型实际应用效果的评价,并收集相关问题和建议。
4.时空模型预警效果评价
在了解试点地区相关突发事件报告情况基础上,利用研究构建的传染病时空模型预警技术评价指标体系和用户调查数据,描述分析预警信号数量、响应处理情况及其结果,重点开展时空模型预警功效分析和评价,通过与时间模型比较,进一步评价时空模型的预警效果。
研究结果和结论
1.时空模型试点运行情况
大部分试点地区及时进行预警信号的响应处理。经过试点运行,传染病预警(时空模型)系统的稳定性得到较大提高,但尚待进一步改进。
2.时空模型预警功效
(1)预警及时性
时空模型总体上预警及时性较好,并且优于时间模型,尤其对于通过监测数据分析发现的事件,具有很好的及时性,可以有利于促进疾控机构较早发现暴发/流行事件,但对于医疗机构、学校等单位直接报告的事件,预警及时性有待进一步提高。
(2)预警准确性
总体上,时空模型具有较好预警准确性。预警灵敏度非常高,对评价的67起事件全部进行了预警,灵敏度高于时间模型；对总体上具有很高的特异度,但需要进一步提高个别传染病的特异度；尽管灵敏度和特异度均高,但受现有评价事件较少的影响,时空模型的阳性预测值较低,但优于时间模型。需要在下一步研究中分不同病种评价模型预警效果和参数设置。
(3)空间聚集探测能力
时空模型通过空间探测到的病例聚集性区域与突发事件实际发病区域具有很高一致性,并且空间探测模型的地理信息有助于预警信号的分析。
(4)预警百分位数分析
预警信号的最高预警百分位数主要在P80及以上。突发事件的首次预警均没有P5。以下的预警信号,说明不进行P5。以下探测预警,不会降低预警灵敏度。
(5)连续预警情况分析
目前的时空模型连续预警信号剔除方法发挥了一定作用,但仍然存在较大比例的重复预警,须进一步改进连续预警剔除方案,减少重复预警信号数量。
3.预警系统评价指标体系
研究建立的时空模型预警系统评价指标体系,通过在时空模型试点效果分析和评价的使用,说明具有可行性；本指标体系得出的评价结果,与用户问卷调查结果较为一致,对于时空模型完善与推广应用提供了科学的依据。
综上,试点期间时空模型预警信号得到了及时响应处理,总体上体现出较好的预警功效,具有良好的预警及时性、灵敏度、特异度和空间探测能力,有助于早期发现传染病暴发、流行事件,但预警模型的阳性预测值较低。在下一步研究中,可利用研究建立的传染病预警系统评价指标体系,研究和细化不同地区、不同传染病的预警参数设置及其预警功效,深入分析阳性预测值低的原因,改进连续预警剔除方案,不断提高预警系统的效能,实现时空模型推广应用。
Background
Resurgence of infectious diseases, emerging infectious diseases and diseases caused by bioterrorism attacks require early detection of infectious disease outbreaks, which can timely trigger disease control and prevent, to reduce the hazards of communicable diseases.
The report quality of infectious diseases surveillance has greatly been improved since 2004. In order to make full use of surveillance data resources and early detecting the spatial aggregation of cases, China CDC further studies the spatial-temporal model of early detection for infectious diseases outbreak. In April 21, a pilot project of spatial-temporal early detection model had been launched in 20 provinces of China. The effectiveness of model is urgently needed to analysis and evaluates, so as to provide a scientific basis and recommendations for revising and improving the technology.
Objective
1. Establishment evaluation methods and indicators for early detection technology of infectious disease.
2. Through analysis data of spatial-temporal model in pilot study and user questionnaire, comprehensively evaluate the effect of model, so as to provide a scientific basis and recommendations for revising and improving the technology.
Data and Methods
1. Data collection
Research collected results of spatial-temporal model, infectious disease surveillance data, related events in pilot areas, and technical material.
2. Establishment evaluation methods
Through literature review, early warning data analysis and expert advice, research on the evaluation framework and indicators in order to establish evaluation guildline for early warning systems.
3. Warning effect evaluation
By calculating evaluation indicator, such as timeliness, sensitivity, specificity, positive predictive value, and analysis of the capacity of space exploration, early warning percentile, and comparing with the temporal model for comprehensive evaluation of spatial-temporal model's effectivities.
4. Users questionnaire
Carry out user questionnaire in Pilot areas, analyse the user's evaluation of warning timeliness, sensitivity, false-positive rate and so on.
Results and conclusions
1. Pilot performance
Most of the pilot areas for early warning have timely responsed to the signals. The stability of early warning system has been greatly improved, but has yet to be further improved.
2. Effectiveness of Spatio-temporal early warning model
(1) Timeliness:the Spatio-temporal model has better timeliness than temporal model, in particular those events discovered by suveillance data analysis, and this technology can help to promote disease control agency to earlier detect outbreak.But for the events directly reported by medical institutions, schools and other units, timeliness need to be further improved.
(2) The accuracy of early warning
In general, the accuracy of Spatio-temporal model is better than temporal model. Spatio-temporal model has high sensitivity and specificity. Spatio-temporal model is superior to the positive predictive value of time model. Because of the events are rare, the positive predictive values of infectious diarrheal diseases, dysentery, hepatitis C are still low. Further studies need to be carried out for evaluating the effects of separate different types of disease and parameters.
(3) Ability of detecting spatial aggregation
The cluster areas of cases detected by Spatio-temporal model have high coherence with the actual incidence of events. The geographic imformation of Space exploration is helpful for signal analysis.
(4) Analysis of early warning percentile
the first signals of all event detected by Spatio-temporal model are above P50.Therefore, it is not necessary to detect the cluster below P50, which will not improve the detection sensitivity and timeliness of early warning.
(5) Continious warning analysis
Current exclusion method for continious early warning signals has played an important role, but there are still a large proportion of repeat signals, exclusion method must be further improved in programs to reduce the number of warning signals.
3. Evaluation framework of early warning systems.
Through using in evaluation of pilot results, the evaluation framework is feasible. The evaluation results are consistent with the user questionair results, which provids scientific evidences and advice for model and system improvement in next step of the research.
Above all, after the pilot running of Spatio-temporal model has better effectiveness than temporal model, which has a good early warning timeliness, accuracy and capacity of space exploration. Spatio-temporal model will be helpful for disease control agencies to abnormal data analysis for detecting epidemic and outbreak. It can be used in disease control agencies of all levels in China, after improving the model and paraments.
2004年我国传染病网络直报系统的启动,使我国传染病疫情报告发生了质的飞跃。为了充分利用监测信息资源,实现病例空间聚集性探测预警,中国疾病预防控制中心在时间模型预警技术研究基础上,进一步开展传染病时空模型预警技术研究,并于2008年4月21日在全国20个省份的221个试点县(区)启动了为期1年的传染病自动预警(时空模型)试点工作。因此,为修改和完善时空模型预警技术,目前急需建立成套的传染病时空模型预警系统评价指标体系,对试点效果开展评价,从而为传染病时空预警技术的改进提供科学依据和建议。
研究目的
1.探索建立传染病自动预警(时空模型)系统评价指标体系。
2.通过时空模型试点结果分析和用户问卷调查,综合评价时空模型的预警效果,为时空模型预警技术的完善提供科学依据和建议。
资料与方法
1.试点数据收集
研究收集了试点地区的时空模型预警数据、传染病监测数据和相关突发事件信息以及有关技术材料。
2.预警评价指标体系的构建
通过文献回顾和预警系统数据特征分析,初步构建预警评价框架；再通过专家咨询的方法,由专家对评价框架和指标内容进行评判并提出修改意见和建议,从而建立起传染病预警系统评价指标体系。
3.预警用户问卷调查
对试点地区疾控机构预警系统用户开展问卷调查,分析用户对预警模型使用情况,对预警及时性、灵敏度、假阳性率等时空模型实际应用效果的评价,并收集相关问题和建议。
4.时空模型预警效果评价
在了解试点地区相关突发事件报告情况基础上,利用研究构建的传染病时空模型预警技术评价指标体系和用户调查数据,描述分析预警信号数量、响应处理情况及其结果,重点开展时空模型预警功效分析和评价,通过与时间模型比较,进一步评价时空模型的预警效果。
研究结果和结论
1.时空模型试点运行情况
大部分试点地区及时进行预警信号的响应处理。经过试点运行,传染病预警(时空模型)系统的稳定性得到较大提高,但尚待进一步改进。
2.时空模型预警功效
(1)预警及时性
时空模型总体上预警及时性较好,并且优于时间模型,尤其对于通过监测数据分析发现的事件,具有很好的及时性,可以有利于促进疾控机构较早发现暴发/流行事件,但对于医疗机构、学校等单位直接报告的事件,预警及时性有待进一步提高。
(2)预警准确性
总体上,时空模型具有较好预警准确性。预警灵敏度非常高,对评价的67起事件全部进行了预警,灵敏度高于时间模型；对总体上具有很高的特异度,但需要进一步提高个别传染病的特异度；尽管灵敏度和特异度均高,但受现有评价事件较少的影响,时空模型的阳性预测值较低,但优于时间模型。需要在下一步研究中分不同病种评价模型预警效果和参数设置。
(3)空间聚集探测能力
时空模型通过空间探测到的病例聚集性区域与突发事件实际发病区域具有很高一致性,并且空间探测模型的地理信息有助于预警信号的分析。
(4)预警百分位数分析
预警信号的最高预警百分位数主要在P80及以上。突发事件的首次预警均没有P5。以下的预警信号,说明不进行P5。以下探测预警,不会降低预警灵敏度。
(5)连续预警情况分析
目前的时空模型连续预警信号剔除方法发挥了一定作用,但仍然存在较大比例的重复预警,须进一步改进连续预警剔除方案,减少重复预警信号数量。
3.预警系统评价指标体系
研究建立的时空模型预警系统评价指标体系,通过在时空模型试点效果分析和评价的使用,说明具有可行性；本指标体系得出的评价结果,与用户问卷调查结果较为一致,对于时空模型完善与推广应用提供了科学的依据。
综上,试点期间时空模型预警信号得到了及时响应处理,总体上体现出较好的预警功效,具有良好的预警及时性、灵敏度、特异度和空间探测能力,有助于早期发现传染病暴发、流行事件,但预警模型的阳性预测值较低。在下一步研究中,可利用研究建立的传染病预警系统评价指标体系,研究和细化不同地区、不同传染病的预警参数设置及其预警功效,深入分析阳性预测值低的原因,改进连续预警剔除方案,不断提高预警系统的效能,实现时空模型推广应用。
Background
Resurgence of infectious diseases, emerging infectious diseases and diseases caused by bioterrorism attacks require early detection of infectious disease outbreaks, which can timely trigger disease control and prevent, to reduce the hazards of communicable diseases.
The report quality of infectious diseases surveillance has greatly been improved since 2004. In order to make full use of surveillance data resources and early detecting the spatial aggregation of cases, China CDC further studies the spatial-temporal model of early detection for infectious diseases outbreak. In April 21, a pilot project of spatial-temporal early detection model had been launched in 20 provinces of China. The effectiveness of model is urgently needed to analysis and evaluates, so as to provide a scientific basis and recommendations for revising and improving the technology.
Objective
1. Establishment evaluation methods and indicators for early detection technology of infectious disease.
2. Through analysis data of spatial-temporal model in pilot study and user questionnaire, comprehensively evaluate the effect of model, so as to provide a scientific basis and recommendations for revising and improving the technology.
Data and Methods
1. Data collection
Research collected results of spatial-temporal model, infectious disease surveillance data, related events in pilot areas, and technical material.
2. Establishment evaluation methods
Through literature review, early warning data analysis and expert advice, research on the evaluation framework and indicators in order to establish evaluation guildline for early warning systems.
3. Warning effect evaluation
By calculating evaluation indicator, such as timeliness, sensitivity, specificity, positive predictive value, and analysis of the capacity of space exploration, early warning percentile, and comparing with the temporal model for comprehensive evaluation of spatial-temporal model's effectivities.
4. Users questionnaire
Carry out user questionnaire in Pilot areas, analyse the user's evaluation of warning timeliness, sensitivity, false-positive rate and so on.
Results and conclusions
1. Pilot performance
Most of the pilot areas for early warning have timely responsed to the signals. The stability of early warning system has been greatly improved, but has yet to be further improved.
2. Effectiveness of Spatio-temporal early warning model
(1) Timeliness:the Spatio-temporal model has better timeliness than temporal model, in particular those events discovered by suveillance data analysis, and this technology can help to promote disease control agency to earlier detect outbreak.But for the events directly reported by medical institutions, schools and other units, timeliness need to be further improved.
(2) The accuracy of early warning
In general, the accuracy of Spatio-temporal model is better than temporal model. Spatio-temporal model has high sensitivity and specificity. Spatio-temporal model is superior to the positive predictive value of time model. Because of the events are rare, the positive predictive values of infectious diarrheal diseases, dysentery, hepatitis C are still low. Further studies need to be carried out for evaluating the effects of separate different types of disease and parameters.
(3) Ability of detecting spatial aggregation
The cluster areas of cases detected by Spatio-temporal model have high coherence with the actual incidence of events. The geographic imformation of Space exploration is helpful for signal analysis.
(4) Analysis of early warning percentile
the first signals of all event detected by Spatio-temporal model are above P50.Therefore, it is not necessary to detect the cluster below P50, which will not improve the detection sensitivity and timeliness of early warning.
(5) Continious warning analysis
Current exclusion method for continious early warning signals has played an important role, but there are still a large proportion of repeat signals, exclusion method must be further improved in programs to reduce the number of warning signals.
3. Evaluation framework of early warning systems.
Through using in evaluation of pilot results, the evaluation framework is feasible. The evaluation results are consistent with the user questionair results, which provids scientific evidences and advice for model and system improvement in next step of the research.
Above all, after the pilot running of Spatio-temporal model has better effectiveness than temporal model, which has a good early warning timeliness, accuracy and capacity of space exploration. Spatio-temporal model will be helpful for disease control agencies to abnormal data analysis for detecting epidemic and outbreak. It can be used in disease control agencies of all levels in China, after improving the model and paraments.
引文
[1]李立明.流行病学[M].北京：人民卫生出版社,2003.575.
[2]突发公共卫生事件应急条例.北京：中国法制出版社,2003.16.
[3]中华人民共和国传染病防治法.中国卫生法制,2004,12(5)：38-44.
[4]中华人民共和国突发事件应对法.中华人民共和国国务院公报,2007：16-23.
[5]Henning K J. What is syndromic surveillance. MMWR,2004,53 Suppl:5-11.
[6]Tsui F C, Espino J U, Dato V M, et al. Technical description of RODS:a real-time public health surveillance system. J Am Med Inform Assoc,2003,10(5):399-408.
[7]Valenciano M, Bergeri I, Jankovic D, et al. Strengthening early warning function of surveillance in the Republic of Serbia:lessons learned after a year of implementation. Euro Surveill,2004,9(5):24-26.
[8]Pavlin JA. Investigation of disease outbreaks detectied by "syndromic" surveillance systems. Journal urban health.2003 June; 80 (2 Suppl 1):107-14.
[9]Fu-Chiang Tsui, Jeremy U. Technical descriptin of RODS:a real-time public health surveillance system. Journal of the American Informatics Association.2003; 10(5):399-408.
[10]Public health agency of Canada. November 7,2004. http://www.phac-aspc.gc.ca/ media/nr-rp/2004_gphin-rmisp_e.html.
[11]Early warning systems. Communicable disease surveillance & response (CSR).2004. http://www.who.int/csr/labepidemiology/projects/earlywarnsystem.
[12]M Valenciano,I Bergeri,D Jankovic.Strengthening the early warning function of surveillance in the Republic of Serbia:lessons learnt one year after implementation.Weekly epidemiological recoud.16 April 2004, vol.79, no.16.
[13]胡明,赵春暖.灰色模型在传染病预测中的应用[J].铁道医学,1991,3(19)：167-168.
[14]曾光,丁雁鹏,程颖恺.传染病流行中存在“Z-D现象”的论证[J].中华流行病学杂志,1997,14：146-148.
[15]程颖恺,曾光.麻疹、猩红热流行事存在“”Z-D现象”进一步的论证及其预测价值研究[J].中华流行病学杂志,1999,20：200-203.
[16]杨维中,邢慧娴,王汉章,等.七种传染病控制图法预警技术研究.中华流行病学杂志,2004,25：1039-1041.
[17]金连梅,杨维中.我国传染病预警工作研究现况分析.中国公共卫生,2008,24(7)：548-549.
[18]中国疾病预防控制中心.全国传染病自动预警(时间模型)试运行和传染病自动预警(时空模型)试点工作方案.北京：中国疾病预防控制中心,2008.
[19]German R R, Lee L M, Horan J M, et al. Updated guidelines for evaluating public health surveillance systems:recommendations from the Guidelines Working Group. MMWR Recomm Rep,2001,50(RR-13):1-351.
[20]Sosin D M. Draft framework for evaluating syndromic surveillance systems. J Urban Health,2003,80(2 Suppl 1):8-13.
[21]Buehler J W, Hopkins R S, Overhage J M, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks:recommendations from the CDC Working Group. MMWR Recomm Rep,2004,53(RR-5):1-11.
[22]Jefferson H, Dupuy B, Chaudet H, et al. Evaluation of a syndromic surveillance for the early detection of outbreaks among military personnel in a tropical country. J Public Health (Oxf),2008.30(4):375-383.
[23]杨维中,祖荣强.突发公共卫生事件预警.中华预防医学杂志.2005,39(6)：427-429.
[24]Wagner M M, Wallstrom G. Methods for Algorithm Evaluation. In:Wagner M M, Moore A W, Aryel R M. Hand book of Biosurveillance. Burlington:Elsevier,2006:301-310.
[25]Kulldorff M, Zhang Z, Hartman J, et al. Benchmark data and power calculations for evaluating disease outbreak detection methods. MMWR,2004,53 Suppl:144-151.
[26]Buckeridge D L, Burkom H, Moore A, et al. Evaluation of syndromic surveillance systems-design of an epidemic simulation model. MMWR,2004,53 Suppl:137-143.
[27]Fawcett T, Provost F. Activity monitoring:noticing interesting changes in behavior. The Fifth ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,1999. San Diego (CA):ACM Press,1999:53-62.
[28]Straetemans M, Altmann D, Eckmanns T, et al. Automatic outbreak detection algorithm versus electronic reporting system. Emerg Infect Dis,2008,14(10):1610-1612.
[29]Buehler J W, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks. MMWR Recomm Rep,2004,53(RR-5):1-11.
Straetemans M, Altmann D, Eckmanns T, et al. Automatic outbreak detection algorithm versus electronic reporting system. Emerg Infect Dis,2008,14(10):1610-1612.
[1]Henning K J. What is syndromic surveillance. MMWR.,2004,53 Suppl:5-11.
[2]Tsui F C, Espino J U, Dato V M, et al. Technical description of RODS:a real-time public health surveillance system. J Am Med Inform Assoc,2003,10(5):399-408.
[3]Valenciano M, Bergen I, Jankovic D, et al. Strengthening early warning function of surveillance in the Republic of Serbia:lessons learned after a year of implementation. Euro Surveill,2004,9(5):24-26.
[4]German R R, Lee L M, Horan J M, et al. Updated guidelines for evaluating public health surveillance systems:recommendations from the Guidelines Working Group. MMWR Recomm Rep,2001,50(RR-13):1-351.
[5]Sosin D M. Draft framework for evaluating syndromic surveillance systems. J Urban Health,2003,80(2 Suppl 1):8-13.
[6]Buehler J W, Hopkins R S, Overhage J M, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks:recommendations from the CDC Working Group. MMWR Recomm Rep,2004,53(RR-5):1-11.
[7]Jefferson H, Dupuy B, Chaudet H, et al. Evaluation of a syndromic surveillance for the early detection of outbreaks among military personnel in a tropical country. J Public Health (Oxf),2008.30(4):375-383.
[8]杨维中,邢慧娴,王汉章,等.七种传染病控制图法预警技术研究.中华流行病学杂志,2004,25(12)：1039-1041.
[9]金连梅,杨维中.我国传染病预警工作研究现况分析.中国公共卫生,2008,24(7)：548-549
[10]中国疾病预防控制中心.全国传染病自动预警(时间模型)试运行和传染病自动预警(时空模型)试点上作方案.北京：中国疾病预防控制中心,2008.
[11]Buehler J W, Sosin D M, Platt R. Evaluation of surveillance systems for early epidemic detection. In:M'ikanatha N M, Lynfield R, Beneden C A, eds. Infectious Disease Surveillance.1st ed. Massachusetts:Blackwell,2007:432-442.
[12]Wagner M M, Wallstrom G. Methods for Algorithm Evaluation. In:Wagner M M, Moore A W, Aryel R M. Hand book of Biosurveillance. Burlington:Elsevier,2006:301-310.
[13]Hashimoto S, Murakami Y, Taniguchi K, et al. Detection of epidemics in their early stage through infectious disease surveillance. Int J Epidemiol,2000,29(5):905-910.
[14]Fawcett T, Provost F. Activity monitoring:noticing interesting changes in behavior. The Fifth ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,1999. San Diego (CA):ACM Press,1999:53-62.
[15]Straetemans M, Altmann D, Eckmanns T, et al. Automatic outbreak detection algorithm versus electronic reporting system. Emerg Infect Dis,2008,14(10):1610-1612.
[16]Sosin D M, Dethomasis J. Evaluation challenges for syndromic surveillance--making incremental progress. MMWR,2004,53 Suppl:125-129.
[17]Kulldorff M, Zhang Z, Hartman J, et al. Benchmark data and power calculations for evaluating disease outbreak detection methods. MMWR,2004,53 Suppl:144-151.
[18]Buckeridge D L, Burkom H, Moore A, et al. Evaluation of syndromic surveillance systems-design of an epidemic simulation model. MMWR,2004,53 Suppl:137-143.
[2]突发公共卫生事件应急条例.北京：中国法制出版社,2003.16.
[3]中华人民共和国传染病防治法.中国卫生法制,2004,12(5)：38-44.
[4]中华人民共和国突发事件应对法.中华人民共和国国务院公报,2007：16-23.
[5]Henning K J. What is syndromic surveillance. MMWR,2004,53 Suppl:5-11.
[6]Tsui F C, Espino J U, Dato V M, et al. Technical description of RODS:a real-time public health surveillance system. J Am Med Inform Assoc,2003,10(5):399-408.
[7]Valenciano M, Bergeri I, Jankovic D, et al. Strengthening early warning function of surveillance in the Republic of Serbia:lessons learned after a year of implementation. Euro Surveill,2004,9(5):24-26.
[8]Pavlin JA. Investigation of disease outbreaks detectied by "syndromic" surveillance systems. Journal urban health.2003 June; 80 (2 Suppl 1):107-14.
[9]Fu-Chiang Tsui, Jeremy U. Technical descriptin of RODS:a real-time public health surveillance system. Journal of the American Informatics Association.2003; 10(5):399-408.
[10]Public health agency of Canada. November 7,2004. http://www.phac-aspc.gc.ca/ media/nr-rp/2004_gphin-rmisp_e.html.
[11]Early warning systems. Communicable disease surveillance & response (CSR).2004. http://www.who.int/csr/labepidemiology/projects/earlywarnsystem.
[12]M Valenciano,I Bergeri,D Jankovic.Strengthening the early warning function of surveillance in the Republic of Serbia:lessons learnt one year after implementation.Weekly epidemiological recoud.16 April 2004, vol.79, no.16.
[13]胡明,赵春暖.灰色模型在传染病预测中的应用[J].铁道医学,1991,3(19)：167-168.
[14]曾光,丁雁鹏,程颖恺.传染病流行中存在“Z-D现象”的论证[J].中华流行病学杂志,1997,14：146-148.
[15]程颖恺,曾光.麻疹、猩红热流行事存在“”Z-D现象”进一步的论证及其预测价值研究[J].中华流行病学杂志,1999,20：200-203.
[16]杨维中,邢慧娴,王汉章,等.七种传染病控制图法预警技术研究.中华流行病学杂志,2004,25：1039-1041.
[17]金连梅,杨维中.我国传染病预警工作研究现况分析.中国公共卫生,2008,24(7)：548-549.
[18]中国疾病预防控制中心.全国传染病自动预警(时间模型)试运行和传染病自动预警(时空模型)试点工作方案.北京：中国疾病预防控制中心,2008.
[19]German R R, Lee L M, Horan J M, et al. Updated guidelines for evaluating public health surveillance systems:recommendations from the Guidelines Working Group. MMWR Recomm Rep,2001,50(RR-13):1-351.
[20]Sosin D M. Draft framework for evaluating syndromic surveillance systems. J Urban Health,2003,80(2 Suppl 1):8-13.
[21]Buehler J W, Hopkins R S, Overhage J M, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks:recommendations from the CDC Working Group. MMWR Recomm Rep,2004,53(RR-5):1-11.
[22]Jefferson H, Dupuy B, Chaudet H, et al. Evaluation of a syndromic surveillance for the early detection of outbreaks among military personnel in a tropical country. J Public Health (Oxf),2008.30(4):375-383.
[23]杨维中,祖荣强.突发公共卫生事件预警.中华预防医学杂志.2005,39(6)：427-429.
[24]Wagner M M, Wallstrom G. Methods for Algorithm Evaluation. In:Wagner M M, Moore A W, Aryel R M. Hand book of Biosurveillance. Burlington:Elsevier,2006:301-310.
[25]Kulldorff M, Zhang Z, Hartman J, et al. Benchmark data and power calculations for evaluating disease outbreak detection methods. MMWR,2004,53 Suppl:144-151.
[26]Buckeridge D L, Burkom H, Moore A, et al. Evaluation of syndromic surveillance systems-design of an epidemic simulation model. MMWR,2004,53 Suppl:137-143.
[27]Fawcett T, Provost F. Activity monitoring:noticing interesting changes in behavior. The Fifth ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,1999. San Diego (CA):ACM Press,1999:53-62.
[28]Straetemans M, Altmann D, Eckmanns T, et al. Automatic outbreak detection algorithm versus electronic reporting system. Emerg Infect Dis,2008,14(10):1610-1612.
[29]Buehler J W, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks. MMWR Recomm Rep,2004,53(RR-5):1-11.
Straetemans M, Altmann D, Eckmanns T, et al. Automatic outbreak detection algorithm versus electronic reporting system. Emerg Infect Dis,2008,14(10):1610-1612.
[1]Henning K J. What is syndromic surveillance. MMWR.,2004,53 Suppl:5-11.
[2]Tsui F C, Espino J U, Dato V M, et al. Technical description of RODS:a real-time public health surveillance system. J Am Med Inform Assoc,2003,10(5):399-408.
[3]Valenciano M, Bergen I, Jankovic D, et al. Strengthening early warning function of surveillance in the Republic of Serbia:lessons learned after a year of implementation. Euro Surveill,2004,9(5):24-26.
[4]German R R, Lee L M, Horan J M, et al. Updated guidelines for evaluating public health surveillance systems:recommendations from the Guidelines Working Group. MMWR Recomm Rep,2001,50(RR-13):1-351.
[5]Sosin D M. Draft framework for evaluating syndromic surveillance systems. J Urban Health,2003,80(2 Suppl 1):8-13.
[6]Buehler J W, Hopkins R S, Overhage J M, et al. Framework for evaluating public health surveillance systems for early detection of outbreaks:recommendations from the CDC Working Group. MMWR Recomm Rep,2004,53(RR-5):1-11.
[7]Jefferson H, Dupuy B, Chaudet H, et al. Evaluation of a syndromic surveillance for the early detection of outbreaks among military personnel in a tropical country. J Public Health (Oxf),2008.30(4):375-383.
[8]杨维中,邢慧娴,王汉章,等.七种传染病控制图法预警技术研究.中华流行病学杂志,2004,25(12)：1039-1041.
[9]金连梅,杨维中.我国传染病预警工作研究现况分析.中国公共卫生,2008,24(7)：548-549
[10]中国疾病预防控制中心.全国传染病自动预警(时间模型)试运行和传染病自动预警(时空模型)试点上作方案.北京：中国疾病预防控制中心,2008.
[11]Buehler J W, Sosin D M, Platt R. Evaluation of surveillance systems for early epidemic detection. In:M'ikanatha N M, Lynfield R, Beneden C A, eds. Infectious Disease Surveillance.1st ed. Massachusetts:Blackwell,2007:432-442.
[12]Wagner M M, Wallstrom G. Methods for Algorithm Evaluation. In:Wagner M M, Moore A W, Aryel R M. Hand book of Biosurveillance. Burlington:Elsevier,2006:301-310.
[13]Hashimoto S, Murakami Y, Taniguchi K, et al. Detection of epidemics in their early stage through infectious disease surveillance. Int J Epidemiol,2000,29(5):905-910.
[14]Fawcett T, Provost F. Activity monitoring:noticing interesting changes in behavior. The Fifth ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,1999. San Diego (CA):ACM Press,1999:53-62.
[15]Straetemans M, Altmann D, Eckmanns T, et al. Automatic outbreak detection algorithm versus electronic reporting system. Emerg Infect Dis,2008,14(10):1610-1612.
[16]Sosin D M, Dethomasis J. Evaluation challenges for syndromic surveillance--making incremental progress. MMWR,2004,53 Suppl:125-129.
[17]Kulldorff M, Zhang Z, Hartman J, et al. Benchmark data and power calculations for evaluating disease outbreak detection methods. MMWR,2004,53 Suppl:144-151.
[18]Buckeridge D L, Burkom H, Moore A, et al. Evaluation of syndromic surveillance systems-design of an epidemic simulation model. MMWR,2004,53 Suppl:137-143.