Scholarly article on topic 'An 11,000-isolate same plate/same day comparison of the 3 most widely used platforms for analyzing multidrug-resistant clinical pathogens'

An 11,000-isolate same plate/same day comparison of the 3 most widely used platforms for analyzing multidrug-resistant clinical pathogens Academic research paper on "Biological sciences"

CC BY-NC-ND
0
0
Share paper
OECD Field of science
Keywords
{Antibiotics / Carbapenemase / Vitek-2 / Phoenix / MicroScan}

Abstract of research paper on Biological sciences, author of scientific article — L.E. Nielsen, R.J. Clifford, Y. Kwak, L. Preston, C. Argyros, et al.

Abstract Stewardship of the dwindling number of effective antibiotics relies on accurate phenotyping. We sought to conduct the first large-scale, same plate and day comparison of the 3 most widely used bacterial analyzers. A total of 11,020 multidrug-resistant clinical isolates corresponding to more than 485,000 data points were used to compare the 3 major identification and antibiotic susceptibility testing (AST) platforms. Bacterial suspensions, prepared from a single plate, were simultaneously tested on all platforms in the same laboratory. Discrepancies were derived from MIC values using 2014 interpretive guidelines. Molecular methods and manual microbroth dilution were reference standards. Most discrepancies were due to drug–organism–AST platform combination instead of individual factors. MicroScan misidentified Acinetobacter baumannii (P <0.001) and underestimated carbapenem susceptibility in Klebsiella pneumoniae. Vitek-2 and Phoenix had higher discrepancies for bla KPC-containing Enterobacteriaceae (P <0.05) and reported false susceptibilities more often. While all platforms performed according to standards, each had strengths and weaknesses for organism identification, assaying specific drug–organism combinations and inferring carbapenemase production.

Academic research paper on topic "An 11,000-isolate same plate/same day comparison of the 3 most widely used platforms for analyzing multidrug-resistant clinical pathogens"

Contents lists available at ScienceDirect

Diagnostic Microbiology and Infectious Disease

journal homepage: www.elsevier.com/locate/diagmicrobio

An 11,000-isolate same plate/same day comparison of the 3 most widely used platforms for analyzing multidrug-resistant clinical pathogens^

L.E. Nielsen a'b'*, R.J. Clifford a, Y. Kwak a, L. Preston a, C. Argyros c, R. Rabinowitz d, P. Waterman a'b, E. Lesho aAd

a Multidrug-Resistant organism Repository and Surveillance Network (MRSN), Walter Reed Army Institute of Research, Silver Spring, MD, USA b Uniformed Services University, Bethesda, MD USA c Department of Biological Sciences, Boston College, Chestnut Hill MA 02467

d R. Adams Cowley Shock Trauma Center, Program in Trauma Infectious Diseases, University of Maryland School of Medicine, Baltimore, MD, USA

article info abstract

Stewardship of the dwindling number of effective antibiotics relies on accurate phenotyping. We sought to conduct the first large-scale, same plate and day comparison of the 3 most widely used bacterial analyzers. A total of 11,020 multidrug-resistant clinical isolates corresponding to more than 485,000 data points were used to compare the 3 major identification and antibiotic susceptibility testing (AST) platforms. Bacterial suspensions, prepared from a single plate, were simultaneously tested on all platforms in the same laboratory. Discrepancies were derived from MIC values using 2014 interpretive guidelines. Molecular methods and manual microbroth dilution were reference standards. Most discrepancies were due to drug-organism-AST platform combination instead of individual factors. MicroScan misidentified Acinetobacter baumannii (P < 0.001) and underestimated carbapenem susceptibility in Klebsiella pneumoniae. Vitek-2 and Phoenix had higher discrepancies for blaKPC-containing Enterobacteriaceae (P < 0.05) and reported false susceptibilities more often. While all platforms performed according to standards, each had strengths and weaknesses for organism identification, assaying specific drug-organism combinations and inferring carbapenemase production.

Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

CrossMark

Article history:

Received 9 February 2015

Received in revised form 22 May 2015

Accepted 24 May 2015

Available online 11 June 2015

Keywords:

Antibiotics

Carbapenemase

Vitek-2

Phoenix

MicroScan

1. Introduction

Clinicians and infection preventionists rely on accurate laboratory results to direct therapy and support infection control or antibiotic stewardship (Bartlett et al., 2013; Boucher et al., 2009; Center for Disease Control, 2013a; Hoang et al., 2013; Pfeiffer and Beldavs, 2014; Talbot et al., 2006; WHO, 2014). Comparative effectiveness research is key to quality and cost in healthcare and considered a priority by the Institute of Medicine and the Agency on Healthcare Research and Quality (Sox and Greenfield, 2009; Agency for Healthcare Research and Quality, 2012). Furthermore, the College of American Pathologists (CAP) requires laboratories seeking accreditation to conduct comparison studies when they use multiple platforms for the same test (i.e., organism identification [ID] and antibiotic susceptibility testing [AST]). Earlier comparison studies indicated that the Phoenix (BD Diagnostics, Sparks, MD, USA) had the highest sensitivity for detecting extended-spectrum (3-lactamases (ESBLs) and carbapenemase producers in

☆ Disclaimer: The views expressed in this paper are solely those of the authors and not to be construed as official or representing those of the U.S. Department of State, the Department of Defense, or the U.S. Army. » Corresponding author. Tel.: +1-301-319-3968; fax: +1-301-319-9801. E-mail address: Lindsey.e.nielsen2.mil@mail.mil (L.E. Nielsen).

Enterobacteriaceae relative to the Vitek-2 and MicroScan systems (Wiegand et al., 2007; Woodford et al., 2010). However, those studies looked at relatively small numbers of locally acquired isolates and relied on outside reference laboratories when comparing 2 or more platforms. This limits generalizability and introduces variance such as changes in inoculum densities, growth conditions, or sample handling (Bratu et al., 2005; Thomson and Moland, 2001).

To our knowledge, there are no large-scale studies that assessed the results of the 3 most widely used platforms after simultaneous testing and included over 200 confirmed carbapenemase producers. Such data would be useful for baseline accreditation efforts and future benchmarking.

In our study, the Phoenix, Vitek-2 (bioMerieux, Durham, NC, USA) and MicroScan (Seimens, Deerfield, IL, USA) platforms were evaluated for their ability to accurately characterize over 11,000 genetically diverse multidrug-resistant organisms (MDROs) including 1323 Acinetobacter baumannii, 547 Klebsiella pneumoniae, 678 Pseudomonas aeruginosa, 2072 Escherichia coli, and 6400 methicillin-resistant Staphylococcus aureus (MRSA) isolates grown on the same plate, with the same set-up time on each platform by the same accredited laboratory. Furthermore, AST and identification discrepancy rates of >200 isolates confirmed to contain blaKPC, blaNDM, blaIMP, or blaVIM were compared to noncarbapenemase producers. Matrix-assisted laser desorption/

http://dx.doi.org/10.1016/j.diagmicrobio.2015.05.018

0732-8893/Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

ionization-time of flight (MALDI-TOF), sequencing, PCR, manual broth dilution, and/or the results of the submitting hospital laboratory were used as reference standards or to resolve discordances.

2. Methods

This study was undertaken as a quality improvement, infection control initiative authorized by policy memoranda 09-050,11-035, and 13016 and IRB protocol number HB-00050924-2.

2.1. Bacterial isolates

A. baumannii, K. pneumoniae, P. aeruginosa, E. coli, and MRSA isolates from medical treatment facilities were grown on blood agar plates (Difco, Detroit, MI, USA) for individual colonies. MDRO classification was based on previously published methods (Magiorakos et al., 2012). Isolates were collected from 2002 to 2014 from hospital laboratories across the United States, including Alaska and Hawaii, as well as Europe, Central and South America, Asia and the Middle East; they came from various anatomical sites, clinical settings (intensive care unit, ward, and outpatient clinics), and patient population representing both genders and all ages. Before submission to the central network laboratory where this study was conducted, isolates were characterized by the accredited laboratory of the submitting hospital (Waterman et al., 2012).

2.2. Strain evaluation

At the central laboratory, all isolates undergo concurrent testing on the 3 AST platforms according to CLSI guidelines and CAP standards as previously described (Lesho et al., 2014). Characterization by pulsed-field gel electrophoresis, multilocus sequence typing, PCR, and whole genome sequencing (WGS) are performed as described previously (Lesho et al., 2014; McGann et al., 2014). Suspected isolates carrying a carbapenemase gene were confirmed by the Carba NP assay, Real Time-Polymerase Chain Reaction (RT-PCR), or WGS (Lesho et al., 2013; McGann et al., 2013; Milillo et al., 2013). A large number (>300) of distinct clades of A. baumannii, K. pneumoniae, P. aeruginosa, E. coli, and MRSA were included (data not shown).

2.3. identification and antibiotic susceptibility testing

The following were used as controls: ATCC strain K. pneumoniae 700603, E. coli 35218 and 25922, P. aeruginosa 27853, Proteus vulgaris 49132, and Providencia stuartii 49809. Before testing, all analyzer panels were prevalidated according to CAP guidelines. All platforms were simultaneously inoculated from a single culture plate and analyzed using Phoenix panels NC44 or NC47 (Siemens, Deerfield, IL, USA), Vitek-2 cards GN30, GN59, or GN ID (bioMerieux, Durham, NC, USa), and MicroScan Walk Away panels NMIC/ID133 (BD Diagnostics). Technicians rotated between the AST analyzers to mitigate operator bias.

Raw MIC results were converted to their respective sensitive (S), intermediate (I), and resistant (R) categorical calls according to 2014 CLSI guidelines using a Perl script (CLSI, 2014). Only antibiotics reported by all platforms were considered. When derived categorical calls differed, these disagreements were classified into 3 groups: a minor discrepancy (mD) is an I call from 1 analyzer contrasted against 2 S or R calls from the other platforms; a major discrepancy (MD) is an R call contrasted against 2 S calls; and a very major discrepancy (VMD) is an S call contrasted against 2 R calls.

Analogous to the minor, major, and very major error lexicon, we used the term discrepancy for this comparison study as it is not feasible to determine the MIC on such a large number of organism-antibiotic combinations using manual broth or agar dilution methods. Hence, the analyzer results themselves were used for discrepancy calls with discrepancy types between instruments attributed to the platform reporting the outlier categorical call. In rare cases where the derived

calls were R, I, and S, the MIC values were determined based on manual microbroth dilutions (MBDs), per CLSI guidelines (CLSI, 2014), or the Sensititer AIM and Trek (Thermo Fisher Scientific, Waltham, MA, USA) system using plate GN2F. Controls for manual MBDs included at least 2 isolates from each species, one being sensitive and the other, resistant. When discordant organism identification was seen, the isolate was retested on each platform, and final adjudications were based on MALDI-TOF, 16S sequencing, or WGS as previously described (Carbonnelle et al., 2011; Center for Disease Control, 2013b). In addition to the reference standards described above, we could also refer to the ID and AST results of the submitting hospital laboratory (also CAP accredited) for further adjudication.

All statistics were calculated using Fisher's exact (P < 0.05) or Yate's X2 tests using the R software package (R Developmental Core Team, 2010). A P-value of less than 0.01 or 0.05 was considered significant for data analyzed by the Yate's X2 test or Fisher's exact, respectively.

3. Results

3.1. Organism identification

Organism identifications among the 3 platforms agreed at the species level for more than 99% of the 11,020 samples tested. MicroScan and the Phoenix misidentified 52 out of 11,149 organism identified, while the Vitek-2 misidentified only 5 (Table 1).

MicroScan misidentified A. baumannii significantly more often than Vitek-2 or Phoenix (P < 0.001, Yate's corrected x2), mainly due to reporting Shigella species in 16 of 1363 (1.2%) of cases. Vendor contacts were unable to either resolve or explain this occurrence. Likewise, Phoenix misidentified significantly more A. baumannii than did Vitek-2 but did not favor misidentification of any one genus The Phoenix and MicroScan instruments misidentified 9 isolates of P. aeruginosa. Overall misidentification of E. coli and K. pneumoniae were less than 2% on any platform. MRSA was the least likely to be misidentified by any platform, but when discrepancies occurred, they were identified as other Staphy-lococcus species. In all cases of discrepant identifications, MALDI-TOF, sequencing, and/or results of the submitting laboratory agreed with the majority decision, further supporting our conclusion that the outlier instrument is incorrect. Overall, Vitek-2 has the highest identification accuracy rate among all MDROs tested.

3.2. Antimicrobial susceptibility

Conflicting AST results among platforms were classified into mD, MD, or VMD (defined in Methods). Consistent with other reports, regardless of organism or drug, all instruments produced a significantly higher proportion of mDs than any other type (data not shown) (Kiyosuke et al., 2010; Markelz et al., 2012; Rybak et al., 2013). Overall, MicroScan had the highest number of discrepancies due to frequently reporting a 2-fold higher MIC yielding a categorical call of I or R relative to S or I on the other platforms, respectively. Full antibiograms or organisms with their respective RIS combinations can be found as supplemental files.

3.3. Gram-negative organisms

Occurrences of MD and VMD for A. baumannii, K. pneumoniae, P. aeruginosa, and E. coli MDRO isolates were summed (Table S1) and tallied by specific organism and drug (Table 2). All instrument/drug/organism combinations performed at or better than the manufacturer's specified error rate with exception to 2 notable combinations. The first exception was E. coli tested on the MicroScan against azteronam, which resulted in 101/1322 isolates reporting an MIC corresponding to resistance compared to a sensitive MIC interpretation on the other platforms (Table 2). To ensure the discrepancy rate was above 5%, samples were repeated, and an overall error rate of 7.95% was calculated.

Table 1

Identification discrepancies among the MicroScan, Phoenix, and Vitek-2 platforms.

Adjudicated identification No. of isolatesa MicroScan Phoenix Vitek-2

A. baumannii 1363 Empedobacter (F.) brevis (1b) Acinetobacter lwoffii (4) E. coli (1) Leminorella (2) Proteus oryzihabitans (1) Shigella (16) B. cepacia (1) Acinetobacter Iwoffii/haemolyticus (1) Burkholderia cepacia (2) K. pneumoniae (1) Pantoea agglomerans (2) Alcaligenes faecalis (2) Cupriavidus pauculus (1) D. acidovorans (1) Pseudomonas stutzeri (1)

E. coli 2091 Kluyvera ascorbata (7) A. lwoffii (1) Citrobacter amalonaticus (1) Vibrio vulnificus (1) C. freundii (4) P. agglomerans (1) Enterobacter cloacae (2) S. choleraesuis ssp. arizonae (2) K. ascorbata (4)

K. pneumoniae 568 Enterobacter aerogenes (3) K. oxytoca (4) Serratia odorifera (1) E. aerogenes (5) E. cloacae (3) K. oxytoca (2) Enterobacter sakazakii (1) P. aeruginosa (1) Klebsiella oxytoca (1)

P. aeruginosa 711 E. coli (1) Pseudomonas genus (5) E. cloacae complex (1)

Pseudomonas fluorescens (3) E. coli (1) P. fluorescens (1)

A. baumannii (1) P. fluorescens (1) Pseudomonas putida (1)

P. stutzeri (1) Staphylococcus hyicus (2)

P. oryzihabitans (3)

MRSA 6416 Staphylococcus intermedius (1) Staphylococcus xylosus (1) S. intermedius (1) Staphylococcus genus (7)

a Total isolates compared, including those with discrepant identifications. b Count of occurrence.

Overall, MicroScan had the greatest number of total MD than the other platforms for most of the organism/drug combinations with exception of nitrofurantoin/E. coli on the Vitek-2 and ciprofloxacin/P. aeruginosa

on the Phoenix. However, all other combinations resulting in an MD corresponded to less than a 5% discrepancy rate to the total number of isolates tested for a particular instrument/drug/organism combination.

Table 2

Occurrence of VMD and MD in select Gram-negative MDROs.

E. coli K. pneumoniae

Na MD VMD N MD VMD

Group Abx Mb P V M P V M P V M P V

ß-Lactam SAM 2071 15c 11 1 6 0 18 547 3 7 0 2 0 3

Penicillin AMP 2072 7 5 0 1 1 1 547 2 0 0 0 0 0

Monobactam ATM 855 68 0 0 0 7 41 300 5 0 0 0 9 1

Cephalosporins CFZ 1554 0 0 0 0 0 0 464 0 0 0 0 0 0

CAZ 740 13 3 0 0 1 21 206 0 0 0 0 1 2

CRO 1219 0 0 0 0 0 5 425 0 0 0 0 0 0

Carbapenems ERT 1635 30 0 0 0 0 2 140 9 0 0 0 0 1

IMP 1521 15 0 0 0 0 0 383 5 5 3 8 6 0

Aminoglycosides AMK 2071 18 0 0 0 3 1 547 6 0 1 0 0 1

GEN 2071 35 7 0 2 1 4 547 10 2 0 2 0 2

TOB 2071 22 10 0 1 1 5 547 3 0 0 0 0 1

Fluoroquinolones CIP 1715 24 2 0 0 1 0 488 4 2 0 2 0 8

LVX 741 30 0 0 0 0 0 206 3 0 0 0 0 0

Nitrofuran NIT 1795 18 5 21 1 0 0 393 0 1 2 4 0 0

Folate Inhibitor SXT 2068 40 16 5 13 7 7 546 12 4 0 1 7 11

P. aeruginosa A. baumannii

N MD VMD N MD VMD

Group Abx M P V M P V M P V M P V

Monobactam ATM 2 0 0 1 0 0 0 -d - - - - - -

Cephalosporins CAZ 314 4 3 2 0 2 2 982 8 0 1 0 0 1

CRO - - - - - - - 1323 0 0 0 2 0 2

Carbapenems IMP 532 3 5 0 2 7 3 11 1 0 0 0 0 1

Aminoglycosides AMK 676 15 2 6 1 9 2 1 0 0 0 0 0 0

GEN 676 13 8 1 1 3 3 1322 22 2 0 1 1 11

TOB 674 15 9 2 2 1 3 1322 47 3 0 1 0 101

Fluoroquinolones CIP 546 2 10 1 0 2 9 1062 13 0 0 0 0 0

LVX 314 6 2 3 1 3 0 981 6 0 0 0 0 0

Abx = antibiotic; AMK = amikacin; SAM = ampicillin/sulbactam; AMP = ampicillin; ATM = azteronam; CFZ = cefazolin; CAZ = ceftazidime; CRO = ceftriaxone; CIP = ciprofloxacin; ERT = ertapenem; GEN = gentamicin; IMP = imipenem; LVX = levofloxacin; NIT = nitrofuratoin; TOB = tobramycin; SXT = trimethoprim/sulfamethoxazole. a Total number of isolates reported.

b Commercial platform: M, MicroScan; P, Phoenix; V, Vitek-2.

c Gray shading indicates that there is a significant difference among the 3 analyzers based on a 3 x 2 Yate's x2 correction P < 0.01. d Dash represents no data.

The second exception was the combination of A. baumannii and tobramycin. Of 1322 isolates tested on the Vitek-2,101 (7.6%) reported an MIC of 4 |ag/mL or less relative to the other platforms, which reported an MIC of 16 |ag/mL or greater. This high VMD error rate was not seen in other drug/organism/platform combinations. Fewer VMDs were noted than MD for all instruments, and no combination gave a more than a 5% isolate testing error rate. However, significant differences between similar drug/organism combinations ran on different platforms occurred. Specifically, Vitek-2 reported a higher proportion ofVMD with E. coli against sulbactam/ampicillin, azteronam, ceftazidime, and ceftriaxone relative to the either the Phoenix or MicroScan. For K. pneumoniae, more VMDs were noted on the Phoenix and azteronam, the MicroScan and imipenem, and the Vitek-2 and ciprofloxacin or trimethoprim/sulfamethoxazole than other instrument/drug combinations. The only notable significant difference among P. aeruginosa VMD occurrences was found when testing amikacin on the Phoenix. A. baumannii testing revealed 2 drug/instrument combinations that gave higher VMD rates compared to the other platforms: gentamycin and tobramycin on the Vitek-2.

Cefepime was associated with the highest percentage of both MDs and VMDs on the MicroScan and Vitek-2 platforms, respectively, than any other drug/organism combination (Table S2). Many more discrepant results were seen for E. coli and K. pneumoniae than for P. aeruginosa or A. baumannii (data not shown). The MIC values of several isolates tested prior to the cefepime breakpoint change (CLSI, 2014) did not provide sufficient granularity to assign categorical calls. Therefore, cefe-pime was analyzed separately from other antibiotics by using raw MlCs for comparison (Table 3). Most often, MicroScan and Phoenix reported the same MIC values for all organisms, and Vitek-2 was the outlier, reporting an MIC 2-6 dilutions lower. Subsets of these isolates were tested by MBD. The MIC values determined by this method agreed with those reported by MicroScan and Phoenix if the MIC was greater than 16. This suggests the Vitek-2 often underestimated the MIC values of E. coli and K. pneumoniae (Table 3).

To understand how the CLSI breakpoint affected our data, we compared our cefepime data using 2013 and 2014 CLSI guidelines. Based on the 2013 CLSI guidelines, we found Vitek-2 reported 44% of E. coli and 35% of K. pneumoniae isolates to be susceptible to cefepime when both the Phoenix and MicroScan reported resistance. Analysis of the data that could be interpreted using the updated 2014 CLSI standards decreased the rate of E. coli VMD reported by Vitek-2 to 23% and K. pneumoniae isolates to 27% VMD (Table S2). The proportions of MDs attributed to the MicroScan also changed. Based on 2013 guidelines, E. coli and K. pneumoniae resulted in a 16% and 6%, respectively, MIC overesti-mation compared to the other platforms. However, reanalysis using the 2014 guidelines found a reduction of MicroScan MD rate to 10% for E. coli isolates but an increase to 15% for K. pneumoniae isolates. However, the number of isolates interpreted across all platforms using the 2013 versus 2014 guidelines showed an improvement in categorical call

Table 3

Common cefepime MIC patterns across AST platforms and MBD results in this study.

MIC value

Organism MicroScan Phoenix Vitek-2 Occurrences MBD MIC (n)a

E. coli >16 <1 <1 20 0.5 (3)

>16 8 2 46 16 (4), 8(1)

>16 >16 2 207 32(1)

>16 >16 4 124 32(5)

>16 >16 8 156 32 (2)

K. pneumoniae >16 >16 2 87 32 (2)

A. baumannii 16 8 32 8 16(2)

>16 8 >64 5 32 (1)

16 8 >64 7 16(2)

P. aeruginosa <8 >16 >64 2 32 (1)

<8 >16 2 2 4(1)

a MIC results of MBD by either manual or Trek Sensititer. The number in parentheses denotes the number of subset isolates tested resulting in the given MIC value.

agreement across all platforms including an increase of 26-29% of E. coli categorical calls and 36-43% of K. pneumoniae isolates.

3.4. Carbapenemase producers

Carbapenemase production was confirmed using the Carba NP test (Tijet et al., 2013), and the molecular mechanism was determined by real-time PCR or WGS. Carba NP-positive isolates contained blaNDM-1, blaIMP, blaKPC, blaOXA-48, or blaVIM. No significant difference between the numbers of discrepancies produced by any platform/drug combination was found in A. baumannii isolates producing an NDM-1 (n = 21) or IMP (n = 3) metallo-p-carbapenemase relative to non-carbapenemase-producing A. baumannii samples. In KPC-producing E. coli and K. pneumoniae isolates (n = 10 and 91, respectively), higher discrepancy rates compared to noncarbapenemase gene carriers were only seen for cefepime tested on Vitek-2 and imipenem assayed on Phoenix (Table 2). For cefepime, 9/10 E. coli KPC-positive organisms and 22/91 K. pneumoniae isolates were reported as susceptible (MIC = 2 ng/mL) on the Vitek-2, while an MIC >16 ^g/mL was reported by the MicroScan and Phoenix. As observed with non-KPC isolates, the Vitek-2 underestimated cefepime MIC values; however, inaccurate reports were more common and significant in those isolates producing a KPC carbapenemase (E. coli P < 0.001, K. pneumoniae P = 0.037, Fisher's exact test). Imipenem susceptibility was erroneously reported by the Phoenix platform in 4/10 E. coli and 10/91 K. pneumoniae KPC-carrying isolates, whereas in non-KPC producers, no discrepancies were found in a sample size of 510 and 133, respectively. Enterobacteriaceae carrying the blaNDM -1 or blaOXA-48 genes produced similar discrepancy rates to noncarbapenemase producers. Two P. aeruginosa isolates, one producing a KPC and the other a VIM type carbapenemase, had consistent MIC values across all platforms and drugs tested.

3.5. MRSA

No MRSA testing combinations resulted in a greater than 5% discrepancy rate (Table 4). Most errors were mDs and were drug, not platform, dependent. No confirmatory testing of mDs was performed. For MDs, MicroScan and Phoenix were statistically significantly more likely than the Vitek-2 to show discrepancies for tetracycline and vancomycin. MicroScan reported a greater proportion of MDs for rifampicin, moxifloxacin, levofloxacin, erythromycin, and clindamycin than the other platforms. Vitek-2 had a significantly higher MD rate reported for daptomycin testing than the other 2 instruments (Table 4). Erroneous reporting of vancomycin resistance by automated platforms has been previously reported (Rybak et al., 2013). When this occurred, isolates were retested and/or interrogated with PCR or WGS for resistance genes. There were no confirmed instances of vancomycin resistance. Few VMDs were found for MRSA isolates on any platform.

4. Discussion

In what is the largest reported comparison to date, we tested the antibiotic susceptibilities and identifications of 11,020 and 11,149 MDRO isolates, respectively, from the same culture plate on the Phoenix, MicroScan, and Vitek-2 platforms yielding a total of 418,521 data points for analysis. All platforms often agreed on assigning correct organism identification more than 99% of the time, with the Vitek-2 reporting the least identification discrepancies. For antibiotic susceptibility testing, we found all analyzers performed according to manufacturer's specifications for most antibiotic and organism combinations. Notable exceptions included cefepime and tobramycin. Using the 2014 CLSI breakpoints, cefepime sensitivity was overreported by the Vitek-2 for ESBL-positive E. coli and K. pneumoniae. Often, these MICs were 2- to 3-fold lower MIC values than reported by the other platforms leading to a high VMD rate (data not shown). Lower MICs were also reported by the MicroScan than MBD testing. Neither VMD nor MD errors were

MRSA discrepancy rates.

Discrepancy

typea Platform CLIb DAP ERY LEV LZD MOX PEN RIF TET VAN

mD MicroScan 92 0 62 57 0 413 0 30 33 17

Phoenix 3 0 47 30 0 234 0 0 21 1

Vitek-2 6 0 84 81 0 392 0 3 8 1

M D MicroScan 78 15 43 159 21 138 3 33 34 24

Phoenix 12 13 10 4 5 3 4 1 20 21

Vitek-2 3 245 1 2 12 3 1 0 4 4

VMD MicroScan 4 2 8 5 0 4 32 6 0 0

Phoenix 5 6 2 10 2 14 1 0 1 0

Vitek-2 9 0 13 4 0 3 6 0 6 0

No discrepancy 6182 6112 6127 6033 6360 5185 3966 6315 6258 6330

N isolates 6398 6393 6399 6393 6400 6399 4013 6399 6398 6399

CLI = clindamycin; DAP = daptomycin; ERY = erythomycin; LEV = levofloxacin; LZD = linezolid; MOX = moxifloxacin; PEN = penicillin; RIF = rifampicin; TET = tetracycline;

VAN = vancomycin.

aDiscrepancy type: mD, MD, and VMD.

bShading indicates significant difference between same discrepancy type relative to other platforms at P < 0.01, Yate's corrected x2.

confined to 1 type of antibiotic panel, lot number, technician, or timeframe, suggesting that this finding is an ongoing occurrence. Vendor inquiry did not resolve our findings.

A high number of A. baumannii isolates tested against tobramycin were reported as sensitive by the Vitek-2 and resistant by the other instruments. Samples were reran and confirmed to produce the same categorical call, although the MIC of individual isolates may have altered within 1 dilution. This result was exclusive to A. baumannii isolates, as MICs from the other tested organisms compared exceptionally well across all instruments. We have been unable to find similar literature reports suggesting that further investigation to determine if this is a common outcome of multidrug-resistant A. baumannii isolates or common to all is warranted.

Our study is the first to provide such comparisons without the need for multiple subcultures, testing centers, and days between testing of samples on each platform as all inoculum came from the same plate under the same growth conditions. It is also one of the first to use the newest 2014 CLSI breakpoints. Finally, it includes a large, genetically diverse collection of samples and the most number AST result comparisons of confirmed carbapenemase producers to date.

Thaden et al. (2014) found that hospitals using the Vitek-2 had significantly higher rates of carbapenem-resistant Enterobacteriaceae detection compared to those using MicroScan, but the Phoenix was not included in that study (Bobenchik et al., 2014). We found that the ability of an isolate to produce a carbapenemase did not result in more AST discrepancies than did those noncarbapenemase producers, with the exception of blaKPC-containing Enterobacteriaceae and cefepime on the Vitek-2 and imipenem on the Phoenix platform.

For cefepime results, we suggest using another validation method if clinically important, especially if reported as susceptible by Vitek-2. No susceptible isolates were reported resistant to imipenem on the Phoenix, suggesting that the Phoenix reports false negatives, but not false positives. Interestingly, higher discrepancy rates in both Vitek-2/ cefepime and Phoenix/imipenem combinations occurred only in En-terobacteriaceae containing a kpc gene. Woodford et al. (2010) concluded that blaOXA-48-containing organisms were poorly detected on any of the 3 AST platforms, while KPC producers and other metallo-p-carbapenemases were accurately detected. Using a genetically diverse and larger sample size, our study is consistent with this conclusion for blaNMD-1-, blaIMP-, and blaVIM-containing organisms but does not support the poor detection of blaOXA-48-containing isolates or the robust detection of KPC producers.

Gram-positive agreement/performance was worse than for Gram negatives, as several MD categorical discrepancies were detected. Overall, MicroScan reported MICs that were 2- to 4-fold higher than those reported by either the Vitek-2 or Phoenix. Most concerning is in the case of the quinolones levofloxacin and moxifloxacin. We did not accrue enough data to report on other quinolones, including ciprofloxacin and gatifloxacin, to determine if this outcome was specific to these drugs or the entire drug family. For vancomycin, all platforms were previously noted to report false positives, although this was seen more often on the Vitek-2 (Rybak et al., 2013). In our study, we found a higher false-positive rate using the MicroScan platform, although discrepancies still existed on the Vitek-2 and Phoenix suggesting that they produce the least number of false-positive discrepancies when reporting MRSA vancomycin results. Since laboratories are especially concerned with vancomycin-resistant MRSA, erroneous resistance reports increase workload and could needlessly eliminate an effective antibiotic.

In addition to accuracy, cost and throughput are relevant considerations. Various analyzer panels (ID and AST or AST only), contracting agreements, and pricing structures hamper cost comparisons. If only AST panels are used, Vitek-2 processes the most isolates per run (120) but the least (60) if also determining identification. The Phoenix and MicroScan process 98 sample per run, supplying both ID and AST results. For us, the MicroScan has the lowest cost per isolate and per panel. Vitek-2 and Phoenix were 1.54 and 1.68 times the cost of MicroScan, respectively.

Our study has important limitations. First, it was not financially or logistically feasible to perform manual broth dilution on all the isolates due to the prohibitively large number of isolate-antibiotic combinations. Second, we focused on only a few species or genera known to constitute the most problematic nosocomial pathogens. Third, only MDRO isolates were included. While these are not expected to perform differently for ID and AST testing than non-MDRO isolates, this possibility cannot be excluded.

In summary, these data from a large, same plate and day comparison of the 3 most utilized automated platforms suggest that while all platforms performed in overall accordance with the manufacturers' specifications, each had notable strengths and weaknesses for organism identification, specific drug-organism combinations, and inferring carbapenemase production.

Funding

This work was supported by the US Army Medical Command and the Armed Forces Health Surveillance Center's Global Emerging Infections and Response System.

Conflicts of interest

None to declare.

Acknowledgments

We thank B. David Goldman for critical review of the manuscript

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.diagmicrobio.2015.05.018.

References

Agency for Healthcare Research and Quality. Comparative effectiveness portfolio: comparing treatment options for different health conditions; 2012 [Rockville, MD, http://www.ahrq.gov/cpi/portfolios/comparative-effectiveness/index.html. Accessed January 23rd, 2015].

Bartlett JG, Gilbert DN, Spellberg B. Seven ways to preserve the miracle of antimicrobials.

Clin Infect Dis 2013;56:1445-50. Bobenchik AM, Hindler JA, Giltner CL, Saeki S, Humpries RM. Performance of Vitek 2 for antimicrobial susceptibility testing of Staphylococcus spp. and Enterococcus spp. J Clin Microbiol 2014;52:392-7. Boucher HW, Talbot GH, Bradley JS, Edwards JE, Gilbert D, Rice LB, et al. Bad bugs, no drugs: no ESKAPE! An update from the Infectious Diseases Society of America. Clin Infect Dis 2009;48:1-12. Bratu S, Landman D, Alam M, Tolentino E, Quale J. Detection of KPC carbapenem-hydrolyzing enzymes in Enterobacter spp. from Brooklyn, New York Antimicrob Agents Chemother 2005;49:776-8. Carbonnelle E, Mesquita C, Bille E, Day N, Dauphin B, Beretti JL, et al. MALDI-TOF mass spectrometry tools for bacterial identification in clinical microbiology laboratory. Clin Biochem 2011;44:104-9. Center for Disease Control. Antibiotic resistance threats in the United States. http://www.

cdc.gov/drugresistance/threat-report-2013/, 2013. [Accessed February 5,2015]. Center for Disease Control. PulseNet protcols. , vol. 2013Athens, GA. : CDC; 2013b. Clinical Laboratory Standards Institute (CLSI). Performance standards for antimicrobial

susceptibility testing., vol. M100-S24Wayne, PA. : CLSI; 2014. Hoang TH, Wertheim H, Minh NB, Duong TN, Anh DD, Phuong TTL, Masaki T, Yoshimura H, To H, et al. Carbapenem-resistant Escherichia coli and Klebsiella pneumoniae strains containing New Delhi metallo-beta-lactamase isolated from two patients in Vietnam. J Clin Microbiol 2013;51:373-4. Kiyosuke M, Nagasawa Z, Kusaba K, et al. Comparison of the antimicrobial susceptibility testing with three automated systems for MRSA, VISA, ESBL-producing Escherichia coli and Klebsiella pneumoniae. J Assoc Rapid Meth Auto Microb 2010;21:1-11. Lesho E, Yoon EJ, McGann P, Snesrud E, Kwak Y, Milillo M, et al. Emergence of colistin-resistance in extremely drug-resistant Acinetobacter baumannii containing a novel

pmrCAB operon during Colistin therapy of wound infections. J Infect Dis 2013;208: 1142-51.

Lesho E, Waterman P, Chukwuma U, McAuliffe K, Neumann C, Julius M, et al. The Antimicrobial Resistance Monitoring and Research (ARMoR) Program: the Department of Defense's Response to Escalating Antimicrobial Resistance. Clin Infect Dis 2014;59: 390-7.

Magiorakos AP, Srinivasan A, Carey RB, Carmeli Y, Falagas ME, Giske CG, et al. Multidrug-resistant, extensively drug-resistant and pandrug-resistant bacteria: an international expert proposal for interim standard definitions for acquired resistance. Clin Microbiol Infect 2012;18:268-81.

Markelz AE, Mende K, Murray CK, Yu X, Zera WC, Hospenthal DR, et al. Carbapenem susceptibility testing errors using three automated systems, disk diffusion, Etest, and broth microdilution and carbapenem resistance genes in isolates of Acinetobacter baumannii-calcoaceticus complex. Antimicrob Agents Chemother 2011;55:4704-11.

McGann P, Milillo M, Clifford RJ, Snesrud E, Stevenson L, Backlund MG, et al. Detection of New Delhi metallo-beta-lactamase (encoded by blaNDM-i) in Acinetobacter schindleri during routine surveillance. J Clin Microbiol 2013;51:1942-4.

McGann P, Courvalin P, Snesrud E, et al. Amplification of aminoglycoside resistance gene aphA1 in Acinetobacter baumannii results in tobramycin therapy failure. MBio 2014;5: e00915-14.

Milillo M, Kwak YI, Snesrud E, Waterman PE, McGann P. Rapid and simultaneous detection of blaKPC and blaNDM by use of multiplex real-time PCR. J Clin Microbiol 2013; 51:1247-9.

Pfeiffer CD, Beldavs ZG. Much to do about carbapenem-resistant Enterobacteriaceae: why supplementing surveillance may be the key to stopping spread. Infect Control Hosp Epidemiol 2014;35:984-5.

R Developmental Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statisical Computing; 2010.

Rybak MJ, Vidaillac C, Sader HS, Rhomberg PR, Rhomberg PR, Salimnia H, Briski LE, et al. Evaluation of vancomycin susceptibility testing for methicillin-resistant Staphylococ-cus aureus: comparison of Etest and three automated testing methods. J Clin Microbiol 2013;51:2077-81.

Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med 2009;151:203-5.

Talbot GH, Bradley J, Edwards Jr JE, Gibert D, Scheld M, Bartlett JG, et al. Bad bugs need drugs: an update on the development pipeline from the Antimicrobial Availability Task Force of the Infectious Diseases Society of America. Clin Infect Dis 2006;42:657-68.

Thaden JT, Lewis SS, Hazen KC, Huslage K, Fowler Jr VG, Moehring RW, et al. Rising rates of carbapenem-resistant Enterobacteriaceae in community hospitals: a mixed-methods review of epidemiology and microbiology practices in a network of community hospitals in the southeastern United States. Infect Control Hosp Epidemiol 2014; 35:978-83.

Thomson KS, Moland ES. Cefepime, piperacillin-tazobactam, and the inoculum effect in tests with extended-spectrum beta-lactamase-producing Enterobacteriaceae. Antimicrob Agents Chemother 2001;45:3548-54.

Tijet N, Boyd D, Patel SN, Mulvey MR, Melano RG. Evaluation of the Carba NP test for rapid detection of carbapenemase-producing Enterobacteriaceae and Pseudomonas aeruginosa. Antimicrob Agents Chemother 2013;57:4578-80.

Waterman P, Kwak Y, Clifford R, Julius M, Onmus-Leone F, Tsurgeon C, et al. A multidrug-resistance surveillance network: 1 year on. Lancet Infect Dis 2012;12:587-8.

Wiegand I, Geiss HK, Mack D, Sturenburg E, Seifert H. Detection of extended-spectrum beta-lactamases among Enterobacteriaceae by use of semiautomated microbiology systems and manual detection procedures. J Clin Microbiol 2007;45:1167-74.

Woodford N, Eastaway AT, Ford M, Leanord A, Keane C, Quayle RM, et al. Comparison of BD Phoenix, Vitek 2, and MicroScan automated systems for detection and inference of mechanisms responsible for carbapenem resistance in Enterobacteriaceae. J Clin Microbiol 2010;48:2999-3002.

World Health Organization (WHO). Antimicrobial resistance: global report on surveillance. http://www.who.int/drugresistance/documents/surveillancereport/en/, 2014. [Accessed February 1,2015].