How well is surveillance actually working?
Surveillance systems collect enormous amounts of information, but how often do we stop to evaluate whether they are doing what we need them to?
Are we detecting cases early enough? Is the information complete and representative of the population? Are reporting and investigation processes unnecessarily complicated? And is the information being used to improve public health?
The US Centers for Disease Control and Prevention (CDC) has published guidelines for evaluating public health surveillance systems. Sentinel proposes to incorporate these principles into a framework that brings evaluation closer to routine surveillance work.
Evaluating the surveillance system
The CDC guidelines describe nine attributes of a surveillance system:
- Simplicity: How straightforward is it to report, manage and investigate cases?
- Flexibility: How easily can surveillance adapt to changing requirements?
- Data quality: How complete, valid and reliable is the information collected?
- Acceptability: Are people and organisations willing to participate?
- Sensitivity: How many of the cases occurring in the population are being detected?
- Predictive value positive: How many reported cases genuinely meet the relevant case definition?
- Representativeness: Does surveillance adequately reflect disease occurrence across the population?
- Timeliness: How quickly does information move from disease occurrence to public health action?
- Stability: How reliably does the surveillance system operate?
The guidelines also consider usefulness, including whether surveillance achieves its objectives and informs public health decisions and actions.
Making evaluation part of routine surveillance
Many useful performance measures can be derived from information already collected through public health work. Sentinel records notifications, investigations, collected information and the actions undertaken. These records could help calculate notification delays, data completeness and investigation timeliness.
Other attributes need more than routine data. Understanding sensitivity may require external comparison data or dedicated epidemiological studies. Acceptability may require speaking directly with the people and organisations participating in surveillance.
The proposed framework would bring these sources of evidence together, supporting automated indicators alongside structured qualitative assessments. It would record definitions, data sources, assumptions and limitations so the findings can be understood and repeated.
Understanding what needs to improve
Sentinel is one part of the wider surveillance system. Poor timeliness may reflect delays in healthcare, testing, notification or investigation. Missing cases may reflect barriers to diagnosis or differences in access to healthcare.
Evaluations could cover a disease, reporting pathway, jurisdiction or whole surveillance program. Repeated assessments could show whether changes to reporting processes, investigation workflows or policy have improved performance.
Proposed development
An initial version could provide configurable indicators, automated calculations where appropriate, qualitative assessments and evaluation reports. Later development could support external reference datasets, comparisons over time and tracking recommendations for improvement.
The aim is to make surveillance evaluation easier to undertake and more useful to the people responsible for improving public health surveillance.
Status: Proposed, not yet implemented.
Reference
Centers for Disease Control and Prevention. Updated guidelines for evaluating public health surveillance systems. MMWR Recomm Rep. 2001;50(RR-13):1–35.