Scholarly article on topic 'Dengue disease outbreak definitions are implicitly variable'

Dengue disease outbreak definitions are implicitly variable Academic research paper on "History and archaeology"

Share paper
Academic journal
OECD Field of science
{Outbreak / Response / Dengue / Decision-making / Policy}

Abstract of research paper on History and archaeology, author of scientific article — Oliver J. Brady, David L. Smith, Thomas W. Scott, Simon I. Hay

Abstract Infectious diseases rarely exhibit simple dynamics. Outbreaks (defined as excess cases beyond response capabilities) have the potential to cause a disproportionately high burden due to overwhelming health care systems. The recommendations of international policy guidelines and research agendas are based on a perceived standardised definition of an outbreak characterised by a prolonged, high-caseload, extra-seasonal surge. In this analysis we apply multiple candidate outbreak definitions to reported dengue case data from Brazil to test this assumption. The methods identify highly heterogeneous outbreak characteristics in terms of frequency, duration and case burden. All definitions identify outbreaks with characteristics that vary over time and space. Further, definitions differ in their timeliness of outbreak onset, and thus may be more or less suitable for early intervention. This raises concerns about the application of current outbreak guidelines for early warning/identification systems. It is clear that quantitatively defining the characteristics of an outbreak is an essential prerequisite for effective reactive response. More work is needed so that definitions of disease outbreaks can take into account the baseline capacities of treatment, surveillance and control. This is essential if outbreak guidelines are to be effective and generalisable across a range of epidemiologically different settings.

Academic research paper on topic "Dengue disease outbreak definitions are implicitly variable"


Epidemics xxx (2015) xxx-xxx

Contents lists available at ScienceDirect



journal homepage

1 Dengue disease outbreak definitions are implicitly variable

2 qi Oliver J. Brady3*, David L. Smithabc, Thomas W. Scottbd, Simon I. Haybe

3 a Spatial Ecology and Epidemiology Group, Tinbergen Building, Department of Zoology, University of Oxford, South Parks Road, Oxford, UK

4 b Fogarty International Center, National Institutes of Health, Bethesda, MD, USA

5 c Sanaria Institute for Global Health and Tropical Medicine, Rockville, MD, USA

6 d Department of Entomology and Nematology, University of California, Davis, CA, USA

7 e The Wellcome Trust Centre for Human Genetics, Roosevelt Drive, Oxford OX3 7BN, UK



20 21 22

Article history:

Received 23 October 2014

Received in revised form 12 March 2015

Accepted 13 March 2015

Available online xxx

Keywords: Outbreak Response Dengue

Decision-making Policy

Infectious diseases rarely exhibit simple dynamics. Outbreaks (defined as excess cases beyond response capabilities) have the potential to cause a disproportionately high burden due to overwhelming health care systems. The recommendations of international policy guidelines and research agendas are based on a perceived standardised outbreak defined as a prolonged, high-caseload, extra-seasonal surge. In this analysis we apply multiple candidate outbreak definitions to reported dengue case data from Brazil to test this assumption. The methods identify highly heterogeneous outbreak characteristics in terms of frequency, duration and case burden. All definitions identify outbreaks with characteristics that vary over time and space. Further, definitions differ in their timeliness of outbreak onset, and thus may be more or less suitable for early intervention. This raises concerns about the application of current outbreak guidelines for early warning/identification systems. It is clear that quantitatively defining the characteristics of disease is an essential prerequisite for outbreak response. More work is needed so that definitions of disease outbreaks can take into account the baseline capacities of treatment, surveillance and control. This is essential if outbreak guidelines are to be effective and generalisable across a range of epidemiological^ different settings.

© 2015 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license


24 1. Introduction

25q2 While much progress has been made in our ability to treat

26 and reduce the long-term burden of many infectious diseases (Lim

27 et al., 2013), unexpected surges in case numbers above the season-

28 ally expected mean can frequently derail progress or push already

29 stretched healthcare resources to breaking point (Cotter et al.,

30 2013; Garget al., 2008; Hay etal., 2003a, 2003b). Disease outbreaks

31 often develop rapidly, are difficult or impossible to predict and

32 cause a disproportionately high burden due to the lack of response

33 capabilities (Garg et al., 2008; Grais et al., 2007; Najera, 1999; WHO


35 As a result of the clear importance of disease outbreaks to

36 wider control efforts, research agendas and subsequent policy

37 guidelines have heavily focussed on methods to predict outbreaks

38 (early warning), how to identify them once they are occurring

* Corresponding author. Tel.: +44 7814561078. E-mail addresses:, (O.J. Brady), (D.L. Smith), (T.W. Scott), (S.I. Hay).

(early detection), how to respond to them appropriately (outbreak response protocols) and how to better plan for future outbreak occurrences (effective healthcare, surveillance and control resource allocation) (WHO, 2009, 2005, 1999; Farrar et al., 2007; Myers et al., 2000; Hutwagner et al., 2003). Optimisation of each of these individual goals is dependent on an unambiguous quantitative definition of exactly what the term "outbreak" refers to in terms of frequency, duration, amplitude and burden.

One common method for defining outbreaks is to use epidemi-ological criteria, where any temporal anomaly from the expected number of cases is classified as an outbreak (Wagner et al., 2001; Stroup et al., 1993). Distinguishing the expected number of cases (seasonal variation in incidence) from excessive case numbers (outbreaks) can be difficult for many communicable diseases that exhibit complicated transmission dynamics that are imperfectly sampled by health systems. Dengue, for example, is composed of four serotypes with complex patterns of cross-immunity in humans (Simmons et al., 2012; Wearing and Rohani, 2006) that are further complicated by highly heterogeneous environmentally-driven variations in each serotype's distribution and force of infection (Messina et al., 2014; Reiner et al., 2014). In addition to this, treatment-seeking, diagnosis and reporting of dengue is highly

1755-4365/© 2015 Published by Elsevier B.V. This is an open access article underthe CC BY-NC-ND license (>/).


2 O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

variable (Simmons et al., 2012; Endy et al., 2011; Brady et al., 2014), making interpretation of the seasonal signals in reported case data difficult (Hay et al., 2013).

Despite the complex heterogeneities in transmission of dengue virus (DENV) and many other infectious diseases, methods to distinguish baseline transmission from outbreaks remain simple, often attributed to the capacity of health systems personnel to implement them. Many methods restrict their analysis to intra-annual trends by calculating a monthly or seasonal mean to define the baseline, often referred to as an endemic channel (Cullen et al., 1984). Outbreak thresholds are then often arbitrarily set at two standard deviations in excess of the endemic channel (Wagner et al., 2001; Stroup et al., 1993; WHO, 2009; Hutwagner et al., 2003). For many diseases, the remaining variation not accounted for by these methods means that (i) determining the endemic channel is highly uncertain and (ii) that the outbreak threshold line is exceeded frequently, briefly and sporadically (Badurdeen et al., 2013). These brief outbreaks may lead to considerable outbreak response measures being deployed, despite minimal excess cases. In public health or operational terms, such an occurrence would be considered as a false alarm for an outbreak.

The inappropriateness of some outbreak response plans to the original case data-based outbreak definitions for dengue has led many to adapt the endemic channel plus two standard deviations method to increase or decrease sensitivity and specificity, for example by requiring two successive weeks above the threshold before response activity is triggered (Harrington et al., 2013). This has led to many different dengue outbreak definitions being employed in different countries and regions (Harrington et al., 2013; Badurdeen et al., 2013). These individual definitions are at odds with international efforts to produce standardised, evidence-based outbreak response strategies (WHO, 2009; Pilger et al., 2010; Farrar et al., 2007; Hutwagner et al., 2003) which are focussed on optimising responses to an idealised extra-seasonal surge-type outbreak that may or may not be relevant to the types of outbreaks identified by these locally adapted definitions.

In this paper we use a dataset of reported dengue cases from Brazil (Fig. 1) to test a wide range of 102 existing outbreak definitions based on five endemic channels and their various parameterisations. This allows us to assess their comparability, consistency over a range of DENV transmission settings and timeliness in outbreak detection.

2. Methods

2.1. Dengue case data

We chose dengue in Brazil as a case study for testing outbreak definitions for a number of reasons. First, Brazil has one of the most comprehensive dengue surveillance systems of any country, with over 200 million people under observation and data disaggregated monthly over ten years or more across 5570 municipalities. This data is also readily and freely available over long time periods from the Brazilian Ministry of Health surveillance system SINAN (Ministerio Dda Saude, 2014a, 2014b). Second, Brazil is also epidemiologically representative of a wide range of DENV transmission settings: In the tropical urban northeast, year-round transmission enables relatively consistent transmission levels with some seasonal patterns (Fig. 1); in the densely populated cities of the southeast much of the case burden is concentrated in outbreaks that occur only once every few years and in the interior western regions, dynamics are dominated by rare large outbreaks that may only occur once every 10 years. Third, Brazil already dedicates significant resources towards vector control, including dengue, with over one billion USD being spent in 2008 (WHO, 2012), as well

as, having an active research community interested in optimising how this money is spent. Brazil, therefore, is one of the most suitable countries to test and evaluate the usefulness of current dengue outbreak policy with a view to informing international policy guidelines on dengue outbreaks.

Monthly total dengue cases were extracted from January 2001 to December 2013 for 27 states (admin1 level (Food and Agriculture Organization of the United Nations, 2008)) from the Brazilian Ministry of Health surveillance system SINAN (Ministerio Dda Saude, 2014a, 2014b). Total dengue cases comprised of hospital reported, suspected and confirmed cases of dengue fever, dengue haemor-rhagic fever and dengue shock syndrome. We chose to aggregate the municipality-level data to state-level as, in endemic settings, a more reliable seasonal signal can be obtained over this scale, meaning it is the more frequent scale at which outbreak identification for strategic planning occurs (Harrington et al., 2013). The robustness of our results to our choice of spatial and temporal scale is examined in the Supplementary information with evaluation at the municipality level, in the state of Sao Paulo, and using simulated weekly dengue cases.

2.2. Outbreak definitions

Existing outbreak definitions are composed of two components, (i) an endemic channel which aims to replicate a historical trend of expected cases and (ii) a set of criteria that determines what level of variation above this endemic channel is classified as an outbreak. In this analysis we aim to test the comparability of all endemic channels and their various parameterisations that make up the variety of outbreak definitions that are currently in use (Badurdeen et al., 2013).

There are five main methods used to calculate an endemic channel: recent mean, monthly mean, moving mean, cumulative mean and fixed incidence threshold, the calculation of which is summarised in Table 1 (Wagner et al., 2001; Stroup et al., 1993; Cullen et al., 1984). These methods are included in core policy documents published by the World Health Organization (WHO) (WHO, 2009) and Centers for Disease Control (CDC) (Hutwagner et al., 2003) and are widely used across a range of different diseases at different levels of transmission intensity (Hay et al., 2002; Badurdeen et al., 2013; Harrington et al., 2013). Calculation of each definition and its typical uses are given in Table 1. For our analysis, the fixed incidence threshold was set at 100 or 300 cases per 100,000 individuals (Lowe et al., 2014; Rigau-Perez et al., 1999; Badurdeen et al., 2013) with population data for each state obtained from the Instituto Brasileiro de Geografia e Estatistica (IBGE) (IBGE, 2010). For all other methods the following variations on parameters used to define the base dataset for endemic channel calculation were explored: (i) the number of historical years to include: five (current) or all available (long-term); and (ii) the inclusion of outbreak years in the base dataset, yes or no. Outbreak years within the base dataset were defined by the total annual number of cases exceeding two standard deviations of any combination of three or more historical years. This is a practical approach to "trimming", an approach that minimises the effect of long-tailed distributions on the historical mean and is often implemented by excluding arbitrarily or quasi-quantitatively determined "outbreak years" to increase sensitivity of the contemporary outbreak definition. While "outbreak years" and "outbreaks" do not necessarily have to overlap, there may be some circularity in defining outbreaks using a dataset that has already had outbreak years arbitrarily defined and removed.

The following parameters that determine the level of variation above the endemic channel that defines an outbreak were also explored: (i) the number of standard deviations above the historical mean: 1 or 2 and (ii) the number of monthly observations above the threshold that would trigger the start of an outbreak: 1, 2 or


O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

Fig. 1. Reported dengue case data in Brazil. Each bar chart displays monthly reported dengue cases (suspected and confirmed) at a state level (n = 27) between the start of 2001 and the end of 2013. Map (a) shows the long-term average probability of dengue occurrence as determined by Bhatt et al. (2013). Map (b) shows the division of the 27 states into epidemiologically defined groups based on the epidemiological characteristics of their time series. States are divided as follows: Amazonas (A1), Acre (A2), Rondônia (A3), Matto Grosso (A4), Districto Federal (A5), Matto Grosso do Sul (A6), Santa Caterina (A7), Paraná (A8), Rio Grande do Sul (A9), Pernambuco (B1), Alagoas (b2), Sergipe (B3), Bahia (B4), Goiás (B5), Minas Gerais (B6), Espirito Santo (B7), Rio de Janeiro (B8), Säo Paulo (B9), Roraima (C1), Pará (C2), Amapá(C3), Maranhäo (C4), Tocantins (C5), Piaui (C6), Ceará (C7), Rio Grande do Norte (C8), Paraiba (C9).

3. The four dynamic endemic channel methods, combined with the four base dataset parameters and the six outbreak threshold parameters gave a total of 96 (=4 x 2 x 2 x 2 x 3) definitions which combined with the six definition variations of the fixed incidence method (two thresholds that just vary by the number of observations above the threshold to trigger an outbreak), gave a total of 102 different outbreak definitions.

Each outbreak definition was calculated annually at the beginning of years 2006-2013, then its performance was evaluated over the following year, with the exception of the recent mean method which was evaluated on a rolling monthly basis. The following measures were collected: the number of outbreaks identified and the proportion of total cases in the time series that exceeded the outbreak threshold.

To quantify the relative variation in outbreak characteristics identified by each definition, a kernel estimation method was used. The two variables, number of outbreaks identified and the proportion of total cases in the time series that exceeded the outbreak threshold, were first scaled to equivalent maximum ranges (0-10). Using the hdrcde package in R, a two dimensional highest density region method was used to estimate the area of the kernel that contained the most concentrated 50% of the density of the combined distribution (Hyndman, 1996). This allowed us to measure relative

variation in outbreak characteristics, between for example two dif- 210

ferent states, by taking into account the spread of the most central 211

results. A dispersal index, D, quantified this relative dispersal with 212

higher values indicating more varied outbreak characteristics. 213

3. Results 214

3.1. Different outbreak definitions are incomparable even when 215

applied to the same data 216

To compare the outbreak characteristics identified by each dif- 217

ferent outbreak definition we applied every definition to each state 218

in Brazil. To compare outbreak characteristics we measured the 219

total number of unique outbreaks identified by each definition and 220

the percentage of total cases across the time series that were clas- 221

sified as outbreak cases. The distribution of these results for each 222

state is shown grouped by the three epidemiological categories in 223

Fig. 2a-c and the aggregated results showing variation around the 224

mean for each state is shown in Fig. 2d. 225

Overall outbreak characteristics are highly variable (Fig. 2d). 226

Among the majority of definitions (95%), the total number of out- 227 breaks identified by any given pair of definitions could differ by 228 as many as 10 outbreaks and the total proportion of outbreak 229


O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

Fig. 2. Variability between all outbreak definitions applied to grouped states. Parts (a-c) show the distribution of the number of outbreaks and percentage of outbreak characteristics identified (the two axes) when each definition (102 same coloured points) is applied to the same state (different colours, grouped three states per plot). This variance around the mean for each state is aggregated to give the distribution at the national level in d. Dotted black lines in d show the mean and 95% confidence intervals. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

''"'IM^^^B Olllill.E IN PRESS

O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

Table 1

Endemic channel definitions. Equations calculate the mean standard deviation (a) and critical threshold (Tc) for observations at day i for each of the five methods. Selected methods can be modified by changing the number of years (b) in the baseline dataset, or by altering the number of standard deviations (k) that define the critical threshold.

Recent mean (EARS CI and C2)

Monthly mean (historical limits method)

Moving mean (smoothed mean)

Cumulative mean

Fixed incidence threshold

Countries using method

Typical uses

Method of calculation


USA for respiratory illnesses (Hutwagner et al., 2003). Similar methods for dengue in Indonesia (Badurdeen et al., 2013)

Diseases with little seasonal pattern and limited surveillance data

The overall mean of a small set of recent observations (Hutwagneret al., 2003)


Tci = № + kffj where the data window is defined by: s = 1 and t =7 forCl and s=3 and t =9 for C2

Colombia, Dominican Republic, Peru and Vietnam for dengue (Badurdeen et al., 2013)

Diseases with a consistent seasonal cycle

The mean of the corresponding months in the base dataset (Cullenet al., 1984)

E:=: b=i(x('-'2b))

,)-Mi >2

(:-i) Tc,i = Mi + k^i

Brazil, Malaysia and China for dengue (Zhang et al., 2014; Badurdeen et al., 2013). USA for multiple diseases including dengue (Rigau-Perez et al., 1999)

Diseases with a seasonal cycle, the timing of which shifts year on year

The mean of the corresponding months and three months either side in the base dataset (Cullen et al., 1984)

< I Em^-":-!)-^2 V (:-i)

Tci = Mi + k^i

USA for Salmonella (Hutwagneret al., 2003). Proposed for malaria in Thailand (Cullenetal., 1984). Ros River virus in Australia (Pelecanos et al., 20i0)

Diseases with sporadic outbreaks diseases where response capacity is set to a particular fixed level of incidence The mean of the corresponding months yearly cumulative case count (Hutwagner et al., 2003)


Tci = max(0, [ffi((k + 0.5) - Tc,(i_i)) + Mi])

Puerto Rico and Brazil for dengue (Rigau-Perez et al., 1999; Badurdeen et al., 2013; Lowe et al., 2014)

A chosen fixed value of cases per 100,000 individuals in the population (Lowe et al., 2014)

Tci = 0.001 or 0.003

230 cases could differ by as much as 65% over just a nine-year time

231 series. This variability is highest in the urban coastal regions (aver-

232 age dispersion metric D = 7.6), still high in the tropical north

233 regions (D = 6.4), and lowest in the interior region (D = 3.6) (Fig. 2).

234 For the interior states variability mainly occurs in the number

235 of outbreaks identified (Fig. 2c) due to definition sensitivity in

236 already low transmission settings, while variation in the trop-

237 ical north mainly occurs in the proportion of cases identified

238 as outbreaks (Fig. 2a) due to the similarities between seasonal

239 trends and the small outbreaks they experience (Fig. 1). In the

240 urban coastal region we see high variation in both of these axes

241 (Fig. 2b) highlighting the difficulty in distinguishing frequent large

242 outbreaks from seasonal trend, such as in Espirito Santo (B7,

243 Fig. 1)

3.2. Even if outbreak definitions are standardised, outbreak 244

characteristics are inconsistent in different states 245

While outbreak characteristics are clearly variable between 246

different outbreak definitions, thus hampering inter-definition 247

comparability, in many cases a particular single definition will 248

be chosen and applied over a wide number of areas, such as a 249

nationally standardised definition. To test the consistency and com- 250

parability of such an approach we measured variation in outbreak 251

characteristics when applying a single definition to all 27 Brazilian 252

states (Fig. 3). 253

No one single definition identified consistent outbreak charac- 254

teristics when applied nationally (Fig. 3). While some outbreak 255

definitions clearly identified outbreaks with more variable 256

Recent mean Moving mean Cumulative mean Monthly mean Fixed threshold

Vo of outbreak cases 75 50 25 0 ' . ' • • • 75 so 25 0 • : : • . £ I ' 100 75 50 25 •* • ■ • •— / 75 50 25 0 • I I' S. » • • 1 - .if ?» • It A • c " • 75 50 25 0 • . • CÎI - ' .

0.0 2.5 5.0 7.5 10.0 012345 0 5 10 0 1 2 3 4 5

Number of outbreaks

(0.9), (1.4), (2.1) (0.1), (1.2), (5.5) (2.5), (4.1), (9.0) (1.5), (2.8), (6.8) (0.1), (0.5), (1.8)

0 Most consistent parameterisation Average parameterisation Least consistent parameterisation

Fig. 3. Variability of outbreak characteristics when one outbreak definition is applied to all 27 states. The scatterplots show the outbreak characteristics of each state (shown by 27 same coloured dots) when the most consistent (green), least consistent (orange) and an average (yellow) defintion is chosen from each endemic channel. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)


O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

Table 2

Parameter details of the most and least consistent definitions shown in Fig. 3. Parameters are explained in the methods section. Fixed threshold methods use fixed values of incidence (100 or 300 cases per 10,000) instead of standard deviations above the mean.

Endemic channel


Years of baseline data

Number of consecutive observations above threshold

Outbreak years included

Standard deviations above mean



Recent mean




All All

Yes Yes

1.4 2.1

Moving mean




Cumulative mean




All All

Yes Yes

Monthly mean

Fixed threshold



consistent Most




All All

No Yes

0.01 fixed

0.01 fixed 0.01 fixed

2.8 6.8

0.5 1.8

Fig. 4. Outbreak characteristicvariability by endemic channel parameterisation. (a) Shows the mean outbreak characteristics across all states of each endemic channel parameter-isation(n = 24 except fixed threshold where n = 6 unique coloured dots). Representative examples of each of the endemic channel definition types applied to low transmission (Roraima, C1, upper panel) and high transmission (Sao Paulo, B9, lower panel) environments are shown in(b-f). Grey bars indicate monthly case numbers 2006-2013, dotted lines show the endemic channel for each year and red background indicates outbreak months identified by the given definition. The percentage figure in the top right shows the percentage of total cases that are identified as outbreak cases. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version ofthis article.)


O.J. Brady et al. / Epic

257 characteristics than others, the general trend was that even the

258 most consistent outbreak definition offered little improvement in

259 consistency over the average definition and there were no common

260 themes among the parameters that made definitions more consis-

261 tent (Table 2). Where outbreak definitions did reach consistency,

262 such as the most consistent moving mean definition (Fig. 3), this

263 was often due to identifying no outbreaks, or outbreaks of limited

264 duration. As the choice of outbreak definition parameterisation is

265 often justified by improved geographic relevance, this result sug-

266 gests little improvement in consistency is gained by minor changes

267 in definition parameterisation.

268 Outbreak definitions based on cumulative mean and monthly

269 mean endemic channels tended to identify a greater propor-

270 tion of total cases as outbreaks, while monthly mean and recent

271 mean methods tended to identify the highest number of out-

272 breaks (Fig. 4). Fixed mean and moving mean endemic channels

273 tended to identify outbreaks of smaller magnitude, but also fewer

274 outbreaks (Fig. 4). Recent mean, cumulative mean, and monthly

s xxx (2015) xxx-xxx 7

mean-based definitions appeared to be the most flexible and 275

could be tweaked to identify outbreaks of different magnitudes 276

and frequencies (D = 1.6, 0.8 and 0.7, respectively). Repre- 277

sentative examples from each endemic channel type are explored 278

in more detail in Fig. 4b-f where each definition is applied to 279

a low incidence (Roraima, C1, upper panel) and high incidence 280

(Sao Paulo, B9, lower panel) setting. The types of outbreaks 281

identified in these two contrasting DENV transmission settings 282

vary considerably in terms of (i) the characteristics of each out- 283

break (how many cases and over how long) and (ii) outbreak 284

frequency and intra-annual timing. This suggests that even if a 285

standardised definition of an outbreak was adopted, the actual 286

characteristics of an outbreak, and thus approaches and resources 287

needed to respond, would be significantly different in different 288

states. 289

Finally, it is also worth noting that even when one defini- 290

tion is applied consistently in one state, the characteristics of the 291

outbreaks identified across time are inconsistent (Fig. 4b-f). The 292

Fig. 5. The difference in time of onset and overall outbreak size for different outbreak definitions applied to different example outbreaks. The longitudinal plot (left) shows the monthly reported case number in the four years (shaded yellow) building up to an extra-seasonal surge in cases (unshaded) in three different states with differing DENV transmission dynamics. For each outbreak, the graph on the right shows the variability in timeliness of detection (number of months since outbreak onset, x-axis) and outbreak size as a proportion of total cases (y-axis) when fitted to data in the yellow shaded region and applied to the unshaded region. Only definitions that identified an outbreak are shown. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)


8 O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

293 outbreaks identified vary in size, length and in the times of year in time despite only subtle changes in DENV transmission dynamics 353

294 which they occur. over the same time period. 354

As an additional limitation, we showed that currently-used out- 355

break definitions vary widely in their capacity to enable effective 356

295 3.3. The timing of onset of an outbreak varies significantly preventative outbreak measures such as early detection and early 357

296 depending on outbreak definition response (Lowe et al., 2014; Chowell et al., 2008). This may make 358

the recommendation of such activities obsolete under certain con- 359

297 For the purpose of early detection and rapid response to disease ditions with particular outbreak definitions, particularly those with 360

298 outbreaks, the definitions need to identify the onset of an out- reduced sensitivity that require more than one observation above 361

299 break, and trigger appropriate response activities before the bulk a given threshold to trigger an outbreak. In such a situation, pre- 362

300 of cases occur. The earlier response measures are enacted during ventative control may be more heavily reliant on early warning 363

301 the onset of an outbreak, the greater the chance interventions will systems that predict outbreaks based on temporal anomalies in 364

302 have at reducing the number of excess cases. To test the variation in epidemiological and environmental warning signals (Lowe et al., 365

303 sensitivity of timeliness of outbreak detection, we analysed the pre- 2014; Degallier et al., 2010; Teklehaimanot et al., 2004a, 2004c). 366

304 dictions of all 102 definition combinations against outbreaks in an Detecting these temporal anomalies in different kinds of data may 367

305 interior state (Matto Grosso do Sul, A6, 2010), an urban south east- well encounter similar issues with inconsistency and variability 368

306 ern state (Rio de Janeiro, B8, 2011) and a northern state (Maranhao, that have been highlighted here. We would therefore recommend 369

307 C4, 2007) all with differing DENV transmission dynamics (Fig. 5). the use of techniques that are more complex than simple inter- 370

308 When focussing on a single outbreak there is considerable varia- pretations of historical means or fixed thresholds to detect these 371

309 tion in timeliness of detection and resultant outbreak size between temporal anomalies. Particular care must be taken in making arbi- 372

310 different outbreak definitions. In all three states different defini- trary decisions about the data used to define the historical baseline, 373

311 tions identified outbreaks that varied in their timing of outbreak such as excluding "outbreak years". Even with the use of more flex- 374

312 onset by as much as seven or eight months, with the majority of ible statistical frameworks (Box et al., 2013) and a wealth of new 375

313 outbreaks still varying by as much as four months (Fig. 5). Gener- data (Ministério Dda Saúde, 2014a, 2014b; PAHO, 2014), the iden- 376

314 ally, monthly mean and recent mean methods proved the timeliest, tified associations may well be highly dependent on how disease 377

315 while moving mean methods were consistently the least timely outbreaks are defined and a clear definition of a disease outbreak 378

316 of all methods tested. Over all three outbreaks it can be observed is an essential prerequisite for the evaluation of the sensitivity and 379

317 that even definitions that lag behind those that identify outbreaks specificity of forecasting, early warning, or early detection. Without 380

318 early, can still identify the bulk of cases in a given outbreak. If a meaningful and operationally relevant fixed definition, instead of 381

319 outbreak response is primarily reactive, these less sensitive defi- just arbitrary thresholds (Lowe et al., 2014, 2013), there is no way 382

320 nitions would be more suitable, however if outbreak response is to compare the performance of predictive models, nor is it possible 383

321 preventative, the additional lead time afforded by more sensitive to construct guidelines based on their outputs. These criteria need 384

322 definitions may be of benefit. As a final point, it should be noted to be rigorously assessed before any predictive or early warning 385

323 that at least the majority of definitions did identify an outbreak system can become a practical tool for outbreak identification and 386

324 in these three selected examples and while the timing and scale response. 387

325 of responses would likely be very different depending on which The recommendations presented here come from the analysis 388

326 outbreak definition was used, at least a response would have been of reported, suspected and confirmed dengue cases in Brazil. This 389

327 triggered. was done to maximise the available data and to avoid the effects 390

of discrepancies in laboratory diagnostic capacities between dif- 391

ferent states. While spatial and temporal variation in misreporting 392

328 4. Conclusions and discussion are likely to affect the ability of outbreak definitions to identify 393

consistent outbreaks, it is likely that the common propensity for 394

329 While the concept of a disease outbreak is undoubtedly impor- over-reporting of dengue cases during outbreaks and underreport- 395

330 tant, it is clear there is no consensus on how to define such a term. ing at other times would only reinforce the consistent distinction 396

331 Moreover, the range of existing definitions is incomparable, incon- between outbreak periods and non-outbreak periods (Teutsch and 397

332 sistent and highly variable in their ability to permit effective early Churchill, 2000). It is possible, however, that mis-reporting could 398

333 response. The magnitude of this incomparability also increases affect the timelines of outbreak detection, which was not tested 399

334 with outbreak size, and thus disease burden. These findings present here. 400

335 a real concern for existing and planned policy guidelines, as well We chose a subset of the wide variety of outbreak definitions 401

336 as operational early warning and early detection systems, that available (Debin et al., 2013; Pelecanos et al., 2010; Jafarpour 402

337 have given insufficient consideration to the heterogeneous out- et al., 2013; Dórea et al., 2013; Polanco et al., 2013; Chen and 403

338 break characteristics identified in real time in a variety of different Chang, 2013; Hay et al., 2003a). Our selection was based upon 404

339 DENV transmission settings (WHO, 2009, 2005,1999; Farrar et al., the principle types of definitions recommended in current out- 405

340 2007; Myers et al., 2000; Hutwagner et al., 2003; Harrington et al., break guidelines (WHO, 2009; Hutwagner et al., 2003), with proven 406

341 2013). uptake in selected countries (Table 1) and some claims of consis- 407

342 In this analysis we have shown that even if a single outbreak tent results (Zhang etal., 2014). Additional definitions may identify 408

343 definition was adopted from among the range of different defi- more consistent outbreak characteristics, but their performance 409

344 nitions currently available, exactly what constitutes an outbreak over wide-scale datasets have yet to be tested. We chose to evalu- 410

345 would still be inconsistent both across space and through time. ate outbreak definitions at the state scale as it is the most common 411

346 This has implications for national level outbreak response guide- level for strategic wide-scale outbreak response recommendations 412

347 lines as it does not present a standardised measure with which to be made (Harrington et al., 2013). Given the general increase 413

348 to compare sub-national healthcare and control needs, nor does it in heterogeneity at lower spatial scales and over shorter time 414

349 allow the monitoring of progress towards long-term reductions in periods, we would not expect existing outbreak definitions to be 415

350 disease burden. Furthermore, even using a standardised definition any more comparable at lower spatial scales or with more fre- 416

351 within a single state to optimise outbreak response strategies may quent reporting intervals. This is supported by the results of our 417

352 be inadequate as the characteristics of outbreaks may change over further analysis presented in the Supplementary information. It 418


O.J. Brady et al. / Epia

is possible that outbreak definitions in transmission-free settings (where any occurrence of cases typically triggers outbreak response (Harrington et al., 2013)) may be consistent and appropriate; this definition has not been tested in this analysis. Despite this, given the diverse nature of global DENV transmission (Bhatt et al., 2013; Messina et al., 2014), of which Brazil represents a small subset, we would not expect outbreak definitions to be any more comparable in different countries or regions. We also confine our analysis of disease outbreaks to dengue. While the four serotypes of dengue and their associated immune responses in humans mean patterns of DENV transmission are highly spatially and temporally heterogeneous, various intricacies of transmission in many other diseases (Reiner et al., 2013; Smith et al., 2014; Anderson and May, 1991) are likely to lead to similar complex reported case dynamics and equally unclear definitions of what comprises a disease outbreak. Similar analyses for other diseases should be conducted to test this hypothesis and to evaluate the usefulness of more established international guidelines for outbreak response.

For dengue, and many other diseases, information other than reported and suspected cases is available which may be more reliable for defining outbreaks due to the inherent temporal biases of reporting and diagnosis associated with simple reported case data (Klaucke, 1994). Data on the percentage of positive diagnostic tests, entomological indices and environmental signals (Harrington et al., 2013; Teklehaimanot et al., 2004b) can be incorporated alongside reported cases in defining disease outbreaks or pre-emptive alert phases. Following an alert phase, an outbreak investigation is typically triggered which may involve forms of sentinel or enhanced passive surveillance. Despite these recommendations, what defines an outbreak following the results of these investigations remains unclear in many cases (Harrington et al., 2013). Irrespective of their reliability, data additions are often logistically and financially costly to make timely and outbreak-relevant, making them counter-productive to the goal of internationally standardised outbreak response and less useful in resource-constrained settings. Furthermore, this does not represent outbreak identification in the majority of settings for dengue (Harrington et al., 2013; Badurdeen et al., 2013). It is also likely that many of these additional data types are closely correlated with reported dengue cases, in which case the benefit of improved outbreak detection consistency needs to be weighed up against the cost of collecting the additional data. If such additional data sources are found to be useful in defining a dengue outbreak, it should be emphasised that they should only be considered in addition to, and not at the expense of, routine passive surveillance of suspected and confirmed dengue cases. This basic measure has core value in identifying dengue seasonality (Hay et al., 2013) and detecting dengue outbreaks as it is the most direct measure of the number of people placing a burden on the healthcare system, in addition to being the most abundant data source for pattern recognition.

Given that dengue and likely many other diseases have frequent, varied and unpredictable deviations from seasonal mean case numbers, there is a rationale to reconsider what the terms "disease outbreak" and "outbreak response" refer to. In epidemiological terms, outbreaks occur at many different levels of intensity, duration and velocity in deviations from seasonal means, driven by a range of factors that determine transmission suitability (Brady et al., 2013; Wearing and Rohani, 2006; Johansson et al., 2009). From a public health perspective, however, the term "outbreak" refers to a situation where routine surveillance, treatment and control capacities are exceeded and exceptional interventions are required. When we use epidemiological data to differentiate routine from excessive caseloads we assume that routinely deployed public health resources can be, and are, matched to the identified baseline transmission levels. For a number of logistical, financial and practical reasons this is unlikely to be the case, especially with

xxx (2015) xxx-xxx 9

some of the more variable outbreak definitions. Furthermore, many 485

of these resources, such as patient beds in a hospital, are required 486

for many different diseases and conditions and therefore have vary- 487

ing degrees of flexibility. 488

If the purpose of an outbreak definition is to identify periods 489

of excess burden, a thorough assessment of baseline healthcare 490

surveillance, control, and treatment capacities is needed. Measures 491

such as the number of hospital beds, insecticide stockpiles and 492

the quantity of surveillance teams will all be important in defin- 493

ing the baseline capacity limits. Additional data on actions that are 494

taken in outbreak times, such as staff surge capacities, resources for 495

additional mass fogging, rapid sentinel surveillance and the speed 496

of deployment of each of these, will then be useful for defining 497

what numbers of cases per month should fall under the remit of 498

routine activities and what number of cases should trigger excep- 499

tional measures. Given this data, modelling approaches can be used 500

to optimally allocate resources between routine and exceptional 501

responses given the logistical constraints of each resource. Such a 502

model could be evaluated periodically and have a role in healthcare 503

budget allocation as well as providing recommendations for spe- 504

cific actions to be undertaken in outbreak times. Any integration of 505

early warning or early detection systems needs to be critically eval- 506

uated against investments in improvements of resource allocation 507

or improved disease surveillance. 508

This alternative method of defining an outbreak may require 509

unconventional and additional data sources, but has the potential 510

to add much-needed clarity to the neglected issue of what is a prac- 511

tical definition of an outbreak. Only then is it feasible to undertake 512

quantitative modelling approaches that aim to optimise preven- 513

tion or amelioration activities that minimise the burden of dengue 514

outbreaks. The scale and extent of Brazil's dengue surveillance sys- 515

tem means it is well-placed to advance dengue outbreak research 516

and could be a first adopter of a new generation of evidence-based 517

dengue outbreak policy. 518

Author contributions 519

OJB, DLS, TWS and SIH designed the experiment. OJB, wrote the 520

manuscript and collected and analysed the data. All authors helped 521

with data and results interpretation and were involved in drafting, 522

revising and final approval of the manuscript. 523

Competing financial interests 524

The authors declared no financial interests. 525

Acknowledgements 526

OJB is funded by a BBSRC studentship. SIH is funded by a Senior q4 527

Research Fellowship from the Wellcome Trust (095066). SIH, DLS 528

and TWS acknowledge funding support from the Research and Pol- 529

icy in Infectious Disease Dynamics (RAPIDD) program of the Science 530

& Technology Directorate, Department of Homeland Security, and 531

the Fogarty International Center, National Institutes of Health. DLS 532

acknowledges funding from the Bill and Melinda Gates Foundation 533

(#OPP1110495) 534

Appendix A. Supplementary data 535

Supplementary data associated with this article can be found, in 536

the online version, at doi:10.1016/j.epidem.2015.03.002. 537

References 538

Anderson, R.M., May, R.M., 1991. Infectious Diseases of Humans. Oxford University 539

Press, Oxford. 540


10 O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx

541 Badurdeen, S., Valladares, D.B., Farrar, J., Gozzer, E., Kroeger, A., Kuswara, N., Lim, S.S., Vos, T., Flaxman, A.D., Danaei, G., Shibuya, K., Adair-Rohani, H., Almazroa, 627

542 Ranzinger, S.R., Tinh, H.T., Leite, P., Mahendradhata, Y., Skewes, R., Verrall, A., M.A., Amann, M., Anderson, H.R., Andrews, K.G., 2013. A comparative risk assess- 628

543 2013. Sharing experiences: towards an evidence based model of dengue surveil- ment of burden of disease and injury attributable to 67 risk factors and risk factor 629

544 lance and outbreak response in Latin America and Asia. BMC Public Health 13, clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden 630

545 607. ofDisease Study 2010. Lancet 380,2224-2260. 63i

546 Bhatt, S., Gething, P.W., Brady, O.J., Messina, J.P., Farlow, A.W., Moyes, C.L., Drake, Lowe, R., Bailey, T.C., Stephenson, D.B.,Jupp, T.E., Graham, R.J., Barcellos, C., Carvalho, 632

547 J.M., Brownstein, J.S., Hoen, A.G., Sankoh, O., Myers, M.F., George, D.B., Jaenisch, M.S., 2013. The development of an early warning system for climate-sensitive 633

548 T., Wint, G.R.W., Simmons, C.P., Scott, T.W., Farrar, J.J., Hay, S.I., 2013. The global disease risk with a focus on dengue epidemics in Southeast Brazil. Stat. Med. 32, 634

549 distribution and burden of dengue. Nature 496,504-507. 864-883. 635

550 Box, G.E., Jenkins, G.M., Reinsel, G.C., 2013. Time Series Analysis: Forecasting and Lowe, R., Barcellos, C., Coelho, C.A., Bailey, T.C., Coelho, G.E., Graham, R., Jupp, T., 636

551 Control. John Wiley & Sons, Hoboken, NJ. Ramalho, W.M., Carvalho, M.S., Stephenson, D.B., 2014. Dengue outlook forthe 637

552 Brady,O.J.,Johansson, M.A.,Guerra,C.A., Bhatt,S.,Golding,N.,Pigott,D.M.,Delatte, H., World Cup in Brazil: an early warning model framework driven by real-time 638

553 Grech, M.G., Leisnham, P.T., Maciel-De-Freitas, R., Styer, L.M., Smith, D.L., Scott, seasonal climate forecasts. Lancet Infect. Dis. 14,619-626. 639

554 T.W., Gething, P.W., Hay, S.I., 2013. Modelling adult Aedes aegypti and Aedes Messina, J.P., Brady, O.J., Scott, T.W., Zou, C., Pigott, D.M., Duda, K.A., Bhatt, S., Katzel- 640

555 albopictus survival at different temperatures in laboratory and field settings. nick, L., Howes, R.E., Battle, K.E., 2014. Global spread of dengue virus types: 64i

556 Parasites Vectors 6,351. mapping the 70 year history. Trends Microbiol. 22,138-146. 642

557 Brady, O.J., Messina, J.P., Scott, T.W., Hay, S.I., 2014. Mapping the epidemiology of Ministerio Dda Saude, 2014a. Dengue: Notificacoes registradas no Sistema 643

558 dengue. In: Gubler, D.J., Ooi, E.E., Vasudevan, S.G., Farrar, J. (Eds.), Dengue and de Informacäo de Agravos de Notificacäo-SINAN, Brazil, 2001-2006 644

559 Dengue Hemorrhagic Fever., second ed. CABI, New York, NY. [Online], Available ( 645

560 Chen, C.-C., Chang, H.-C., 2013. Predicting dengue outbreaks using approximate dengue/bases/denguebr.def) (accessed 27-Jun 2014). 646

561 entropy algorithm and pattern recognition. J. Infect. 67,65-71. Ministerio Dda Saude, 2014b. Dengue: Notificacoes registradas no Sistema de 647

562 Chowell, G., Torre, C., Munayco-Escate, C., Suarez-Ognio, L., Lopez-Cruz, R., Hyman, Informacäo de Agravos de Notificacäo—SINAN, Brazil, 2007-2012 [Online], 648

563 J., Castillo-Chavez, C., 2008. Spatial and temporal dynamics of dengue fever in Available ( 649

564 Peru: 1994-2006. Epidemiol. Infect. 136,1667-1677. bases/denguebrnet.def) (accessed 27-Jun 2014). 650

565 Cotter, C., Sturrock, H.J., Hsiang, M.S., Liu, J., Phillips, A.A., Hwang, J., Gueye, C.S., Myers, M., Rogers, D., Cox, J., Flahault, A., Hay, S., 2000. Forecasting disease risk for 65i

566 Fullman, N., Gosling, R.D., Feachem, R.G., 2013. The changing epidemiology of increased epidemic preparedness in public health. Adv. Parasitol. 47,309-330. 652

567 malaria elimination: new strategies for new challenges. Lancet 382,900-911. Najera, J., 1999. Prevention and control of malaria epidemics. Parassitologia 41, 653

568 Cullen,J., Chitprarop, U., Doberstyn, E., Sombatwattanangkul, K., 1984. Epidemiolog- 339-347. 654

569 ical early warning system for malaria control in northern Thailand. Bull. World PAHO, 2014. Dengue Regional Information Number of Cases: 2014. Epidemiological 655

570 Health Organ. 62,107-114. Weekly reports. Pan American Health Organization, Washington, DC. 656

571 Debin, M., Souty, C., Turbelin, C., Blanchon, T., Boelle, P.-Y., Hanslik, T., Hejblum, Pelecanos, A.M., Ryan, P.A., Gatton, M.L., 2010. Outbreak detection algorithms for 657

572 G., Le Strat, Y., Quintus, F., Falchi, A., 2013. Determination of French influenza seasonal disease data: a case study using ross river virus disease. BMC Med. Inf. 658

573 outbreaksperiodsbetween1985and2011throughaweb-basedDelphimethod. Decis. Making 10,74. 659

574 BMC Med. Inf. Decis. Making 13,138. Pilger, D., De Maesschalck, M., Horstick, O., San Martin, J.L., 2010. Dengue outbreak 660

575 Degallier, N., Favier, C., Menkes, C., Lengaigne, M., Ramalho, W.M., Souza, R., Servain, response: documented effective interventions and evidence gaps. TropIKA 1,1. 66i

576 J., Boulanger,J.-P., 2010. Toward an early warning system for dengue prevention: Polanco, C., Castanon-Gonzalez, J.A., Macias, A.E., Samaniego, J.L., Buhse, 662

577 modeling climate impact on dengue transmission. Clim. Change 98,581-592. T., Villanueva-Martinez, S., 2013. Detection of severe respiratory 663

578 Dorea, F.C., Mcewen, B.J., Mcnab, W.B., Sanchez, J., Revie, C.W., 2013. Syndromic disease epidemic outbreaks by CUSUM-based overcrowd-severe- 664

579 surveillance using veterinary laboratory data: algorithm combination and cus- respiratory-disease-index model. Comput. Math. Methods Med. 2013, 665

580 tomization of alerts. PLoS ONE 8, e82183. 666

581 Endy, T.P., Anderson, K.B., Nisalak, A., Yoon, I.K., Green, S., Rothman, A.L., Thomas, S.J., Reiner, R.C., Perkins, T.A., Barker, C.M., Niu, T., Chaves, L.F., Ellis, A.M., George, D.B., 667

582 Jarman, R.G., Libraty, D.H., Gibbons, R.V., 2011. Determinants of inapparent and Le Menach, A., Pulliam, J.R., Bisanzio, D., Buckee, C., Chiyaka, C., Cummings, D.A., 668

583 symptomatic dengue infection in a prospective study of primary school children Garcia, A.J., Gatton, M.L., Gething, P.W., Hartley, D.M., Johnston, G., Klein, E.Y., 669

584 in Kamphaeng Phet, Thailand. PLoS Negl. Trop. Dis. 5, e975. Michael, E., Lindsay, S.W., Lloyd, A.L., Pigott, D.M., Reisen, W.K., Ruktanonchai, N., 670

585 Farrar, J., Focks, D., Gubler, D., Barrera, R., Guzman, M., Simmons, C., Kalayanarooj, Singh,B.K,Tatem,A.J.,Kitron,U.,Hay,S.L,Scott,T.W.,Smith,D.L.,2013.Asystem- 67i

586 S., Lum, L., Mccall, P., Lloyd, L., 2007. Towards a global dengue research agenda. atic review of mathematical models of mosquito-borne pathogen transmission: 672

587 Trop. Med. Int. Health 12,695-699. 1970-2010. J. R. Soc. Interface 10, 20120921. 673

588 Food and Agriculture Organization of the United Nations, 2008. The Global Admin- Reiner, R.C., Stoddard, S.T., Forshey, B.M., King, A.A., Ellis, A.M., Lloyd, A.L., Long, 674

589 istrative Unit Layers (GAUL): Technical Aspects. In: EC-FAO Food Security K.C., Rocha, C., Vilcarromero, S., Astete, H., 2014. Time-varying, serotype- 675

590 Programme (ESTG). Food and Agriculture Organization of the United Nations. specific force of infection of dengue virus. Proc. Natl. Acad. Sci. U.S.A. 111, 676

591 Garg, P., Nagpal, J., Khairnar, P., Seneviratne, S.L., 2008. Economic burden ofdengue E2694-E2702. 677

592 infections in India. Trans. R. Soc. Trop. Med. Hyg. 102,570-577. Rigau-Perez, J.G., Millard, P.S., Walker, D.R., Deseda, C.C., Casta-Velez, A., 1999. A 678

593 Grais, R.F., Dubray, C., Gerstl, S., Guthmann, J.-P., Djibo, A., Nargaye, K., Coker, J., deviation bar chart for detecting dengue outbreaks in Puerto Rico. Am. J. Public 679

594 Alberti, K., Cochet, A., Ihekweazu, C., 2007. Unacceptably high mortality related Health 89,374-378. 680

595 to measles epidemics in Niger, Nigeria, and Chad. PLoS Med. 4, e16. Simmons, C., Farrar, J., Chau, N., Wills, B., 2012. Dengue. N. Engl. J. Med. 366, 68i

596 Harrington, J., Kroeger, A., Runge-Ranzinger, S., O'Dempsey, T., 2013. Detecting and 1423-1432. 682

597 responding to a dengue outbreak: evaluation of existing strategies in country Smith, D.L., Perkins, T.A., Reiner, J.R.R.C., Barker, C.M., Niu, T., Chaves, L.F., Ellis, A.M., 683

598 outbreak response planning. J. Trop. Med. 2013,756832. George, D.B., Le Menach, A., Pulliam, J.R.C., Bisanzio, D., Buckee, C., Chiyaka, 684

599 Hay, S.I., Brady, O.J., Messina, J.P., Farrar, J., 2013. Football fever could be a dose of C., Cummings, D.A.T., Garcia, A.J., Gatton, M.L., Gething, P.W., Hartley, D.M., 685

600 dengue. Nature 503, 439. Johnston, G., Klein, E.Y., Michael, E., Lloyd, A.L., Pigott, D.M., Reisen, W.K., Ruk- 686

601 Hay, S.I., Renshaw, M., Ochola, S.A., Noor, A.M., Snow, R.W., 2003a. Performance tanonchai, N., Singh, B.K., Stroller, J., Tatem, A.J., Kitron, U., Godfray, H.C.J., 687

602 of forecasting, warning and detection of malaria epidemics in the highlands of Cohen, J.M., Hay, S.I.A., Scott, T.W., 2014. Recasting the theory of mosquito-borne 688

603 western Kenya. Trends Parasitol. 19,394-399. pathogen transmission dynamics and control. Trans. R. Soc. Trop. Med. Hyg. 108, 689

604 Hay, S.I., Simba, M., Busolo, M., Noor, A.M., Guyatt, H.L., Ochola, S.A., Snow, R.W., 185-197. 690

605 2002. Defining and detecting malaria epidemics in the highlands of western Stroup, D.F., Wharton, M., Kafadar, K., Dean, A.G., 1993. Evaluation of a method for 69i

606 Kenya. Emerg. Infect. Dis. 8,555-562. detecting aberrations in public health surveillance data. Am. J. Epidemiol. 137, 692

607 Hay, S.I., Were, E.C., Renshaw, M., Noor, A.M., Ochola, S.A., Olusanmi, I., Alipui, N., 373-380. 693

608 Snow, R.W., 2003b. Forecasting, warning, and detection of malaria epidemics: a Teklehaimanot, H.D., Lipsitch, M., Teklehaimanot, A., Schwartz, J., 2004a. Weather- 694

609 case study. Lancet 361,1705-1706. based prediction of Plasmodium falciparum malaria in epidemic-prone regions 695

610 Hutwagner, M.L., Thompson, M.W., Seeman, G.M., Treadwell, T., 2003. The bioter- of Ethiopia. I. Patterns of lagged weather effects reflect biological mechanisms. 696

611 rorism preparedness and response early aberration reporting system (EARS). J. Malar. J. 3,41. 697

612 Urban Health 80, i89-i96. Teklehaimanot, H.D., Schwartz, J., Teklehaimanot, A., Lipsitch, M., 2004b. Alert 698

613 Hyndman, R.J., 1996. Computing and graphing highest density regions. Am. Stat. 50, threshold algorithms and malaria epidemic detection. Emerg. Infect. Dis. 10, 699

614 120-126. 1220. 700

615 IBGE, IInstituto Brasileiro de Geografia e Estatistica, 2010. Population Teklehaimanot, H.D., Schwartz, J., Teklehaimanot, A., Lipsitch, M., 2004c. Weather- 70i

616 Census Results [Online]. IBGE, Available at ( based prediction of Plasmodium falciparum malaria in epidemic-prone regions 702

617 english/estatistica/populacao/censo2010/default.shtm) (accessed 27-Jun of Ethiopia. II. Weather-based prediction systems perform comparably to early 703

618 2014). detection systems in identifying times for interventions. Malar. J. 3, 44. 704

619 Jafarpour, N., Precup, D., Izadi, M., Buckeridge, D., 2013. Using hierarchical mixture Teutsch, S.M., Churchill, R.E., 2000. Principles and Practice of Public Health Surveil- 705

620 of experts model for fusion of outbreak detection methods. AMIA Ann. Symp. lance. Oxford University Press, Oxford. 706

621 Proc. 1,663. Wagner, M.M.,Tsui, F.-C., Espino,J.U., Dato, V.M., Sitting, D.F., Caruana, R.A., Mcginnis, 707

622 Johansson, M.A., Dominici, F., Glass, G.E., 2009. Local and global effects of climate on L.F., Deerfield, D.W., Druzdzel, M.J., Fridsma, D.B., 2001. The emerging science 708

623 dengue transmission in Puerto Rico. PLoS Negl. Trop. Dis. 3, e382. of very early detection of disease outbreaks. J. Public Health Manage. Pract. 7, 709

624 Klaucke, D.N., 1994. Evaluating public health surveillance. In: Teutsch, S.M., 51-59. 7i0

625 Churchill, R.E. (Eds.), Principles and Practice of Public Health Surveillance. Oxford Wearing, H.J., Rohani, P., 2006. Ecological and immunological determinants of 7ii

626 University Press, New York,NY. dengue epidemics. Proc. Natl. Acad. Sci. U.S.A. 103,11802-11807. 7i2


O.J. Brady et al. / Epidemics xxx (2015) xxx-xxx 11

713 WHO, 1999. WHO Guidelines for Epidemic Prepardness and Response to Measles WHO EBOLA RESPONSE TEAM, 2014. Ebola virus disease in West Africa—the first 719

714 Outbreaks. World Health Organization, Geneva. 9 months of the epidemic and forward projections. N. Engl. J. Med. 371, 720

715 WHO, 2005. Malaria Control Today. World Health Organization, Geneva. 1481-1495. 721

716 WHO, 2009. Dengue: Guidelines for Diagnosis, Treatment, Prevention and Control. Zhang, H., Li, Z., Lai, S., Clements, A.C.A., Wang, L., Yin, W., Zhou, H., Yu, H., Hu, W., 722

717 World Health Organization, Geneva. Yang, W., 2014. Evaluation of the performance of a dengue outbreak detection 723

718 WHO, 2012. Global Strategy for Dengue Prevention and Control 2012-2020. World tool for China. PLoS ONE 9, e106144. 724

Health Organization, Geneva.