Scholarly article on topic 'Performance of Quantification of Modified Hodge Test: An Evaluation with Klebsiella pneumoniae Carbapenemase-Producing Enterobacteriaceae Isolates'

Performance of Quantification of Modified Hodge Test: An Evaluation with Klebsiella pneumoniae Carbapenemase-Producing Enterobacteriaceae Isolates Academic research paper on "Biological sciences"

Share paper
Academic journal
BioMed Research International
OECD Field of science

Academic research paper on topic "Performance of Quantification of Modified Hodge Test: An Evaluation with Klebsiella pneumoniae Carbapenemase-Producing Enterobacteriaceae Isolates"

Hindawi Publishing Corporation BioMed Research International Volume 2014, Article ID 139305, 6 pages

Research Article

Performance of Quantification of Modified

Hodge Test: An Evaluation with Klebsiella pneumoniae

Carbapenemase-Producing Enterobacteriaceae Isolates

Vanessa Bley Ribeiro,1,2 Adriano Rostirolla Linhares,2 Alexandre P. Zavascki,3 and Afonso Luis Barth1,2

1 Faculdade de Farmacia, Universidade Federal do Rio Grande do Sul (UFRGS), 2752 Ipiranga Avenue, 90160-093 Porto Alegre, RS, Brazil

2 Laboratorio de Pesquisa em Resistencia Bacteriana (LABRESIS), Centro de Pesquisa Experimental, Hospital de Clínicas de Porto Alegre (HCPA), 2350 Ramiro Barcelos Street, 90035-903 Porto Alegre, RS, Brazil

3 Infectious Diseases Service, HCPA, 2350 Ramiro Barcelos Street, 90035-903 Porto Alegre, RS, Brazil

Correspondence should be addressed to Afonso Luis Barth; Received 1 December 2013; Accepted 18 February 2014; Published 26 March 2014 Academic Editor: Sanda Sardelic

Copyright © 2014 Vanessa Bley Ribeiro et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Modified Hodge Test (MHT) has been suggested as screening tests for carbapenemases, but concerns regarding its difficult interpretation and common false-positive results obtained in the presence of other ^-lactamases have been noted. This study aimed to quantify the enhanced growth formed by the indicator strain and thus evaluate the performance of a quantitative interpretation of MHT for KPC screening. MHT was performed in 50 KPC-producing isolates and 334 non-carbapenemase-producing isolates, using ertapenem (ETP) and meropenem (MEM) as substrates. The size of enhanced growth of indicator strain was measured for each isolate tested and for the positive control used, and a ratio was calculated. Our results revealed 17 different ETP and MEM ratios, with distinct sensitivity (SN) and specificity (SP). Higher SN combined to higher SP was achieved when ETP and MEM ratios were 0.45, with a SN value of 96% for both substrates and SP values of 99.4% and 100% for ETP and MEM, respectively. The quantification with both substrates increased SP of the test for KPC detection. Considering that MHT is the unique phenotypic test that is referred to by CLSI, a more accurate approach for its interpretation could be applied to make it a more useful tool.

1. Introduction

The global spread of Klebsiella pneumoniae carbapenemases (KPC) in Enterobacteriaceae in the last decade poses as a serious public health threat, since it leads to variable levels of carbapenem resistance and few therapeutic options remain available for treating such infections [1, 2]. Moreover, KPC codifying genes are harbored in genetically mobile elements allowing their rapid spread among gram-negative rods [3].

Molecular methods are the gold standard to detect the KPC genes; however, although no longer recommended neither by CLSI nor by EUCAST, screening test may be necessary to discriminate which isolates with reduced susceptibility to carbapenems are more likely to produce KPC, particularly in low-income settings. Many screening tests for Ambler

Class A carbapenemases have been presented in recent years, including the Modified Hodge Test (MHT) and tests based on boronic acid (BA) [4-6], as well as the combined use of ^-lactamase inhibitors, including BA, EDTA, dipicolinic acid, and cloxacillin in order to differentiate Class A from other carbapenemases, such as metallo-^-lactamases and AmpC ^-lactamases [7, 8]. Although MHT often presents high sensitivity (>90%) [4, 5, 9], its interpretation is usually difficult and also subjective [10,11]. Moreover, many studies have demonstrated positive results in MHT in the presence of other ^-lactamases, such as ESBL and AmpC coupled with porin loss [5, 10, 12, 13]. Although some new and more practical phenotypic methods have been proposed for carbapenemase detection [14, 15], MHT is inexpensive and accessible for virtually all clinical laboratories.

Since MHT positivity depends on the enzymatic activity, it should be expected that the enhanced growth intensity would be bigger with stronger hydrolytic activity of the enzyme. In this study, we evaluate the sensitivity (SN) and specificity (SP) of a quantitative interpretation of MHT for KPC detection.

2. Materials and Methods

2.1. Bacterial Isolates. A panel of distinct genera of Enter-obacteriaceae resistant or with reduced susceptibility to ertapenem (ETP) and/or meropenem (MEM) by discdiffusion test (zone of inhibition < 21 mm) was included in the study [16]. A total of 50 KPC-producing and 334 non-carbapenemase-producing clinical isolates were recovered from 13 Brazilian hospitals from 2009 to 2011. All of them were previously identified by conventional techniques and by use ofVITEK2 (bioMerieux, France). CarbapenemsMICs were performed according to CLSI [17] and the distribution of both MIC50 and MIC90 groups is shown in Table 1. Carbapenemases were first investigated by multiplex realtime PCR for detection of blaKPC, biaVlM, blaGES, blaNDM, blaOXA-A8, and blamP genes [18]. Of the carbapenemases evaluated by PCR, only the KPC enzyme was detected among the isolates. All isolates were double-checked for the presence of blaKPC by a different set of primers in a simplex PCR [19]. The positive control strains used in this study included a KPC-producing K. pneumoniae ATCC BAA-1705, a VIM-producing P. aeruginosa, a GES-producing K. pneumoniae, a NDM-producing K. pneumoniae, a OXA-48-producing K. pneumoniae, and a IMP-producing P aeruginosa.

2.2. MHT Performance and Quantification. The MHT was performed for all isolates as recommended by CLSI [17]. In order to investigate the performance of both substrates, ETP and MEM disks were placed on the agar plate seeded with E. coli ATCC 25922 (indicator strain). The isolates were inoculated in a straight line out from the edge of the disk to the edge of the plate. The plates were incubated at 35 ± 2°C during 20 hours. The KPC-producing K. pneumoniae ATCC BAA 1705 was used as positive control in all experiments. For both substrates, the size of enhanced growth of indicator strain was measured in millimeters (mm) with a ruler, for each isolate tested and for the positive control used in each experiment separately, as demonstrated in Figure 1. For data analysis, a ratio was calculated according to the formula below, and then an ETP enhanced growth ratio and a MEM enhanced growth ratio were established for all isolates evaluated:

R = (size of enhanced growth obtained

from isolate tested (mm)) x (size of the enhanced growth obtained from positive control strain (mm)) 1.

The use of a ratio instead of using only the measurement of the enhanced growth generated by each isolate was

Figure 1: Quantification of Modified Hodge Test (MHT). The enhanced growth ratio of ETP and MEM subtracts were obtained from the measure (mm) of the enhanced growth of E. coli ATCC 25922 for both the isolate and the KPC-producing positive control (K. pneumoniae ATCC BAA 1705) and a ratio was calculated. In this example, B is the positive control and the enhanced growth measured for ETP was 10 mm. The enhanced growth measured for isolates A and C was 5 and 2 mm, respectively, resulting in ETP enhanced ratios of 0.5 and 0.2, respectively.

chosen in order to minimize the variations inherent to the method, since it minimizes possible distinct growths that a same isolate could produce in a distinct day test and also discounts variations due to different agar media or disk suppliers, for example. The readings were done through the back of the plate in a dark background. To ensure the test reproducibility and the intra- and interobserver integrity, 30 isolates (15 KPC-producers and 15 KPC-nonproducers) were randomly selected to be tested again and the interpretation was performed by two independent investigators who were blinded for the carbapenemase results.

2.3. Statistical Analysis. The SN, SP, and receiver operating characteristic (ROC) curves and their respective 95% CI for ETP and MEM ratios were calculated using SPSS version 18.0. SN and SP were calculated using PCR as gold standard. The positive predicted values (PPV) and negative predictive values (NPV) of the test using both substrates were calculated according to estimated population prevalence of blaKPC-2. Agreement between intraobserver of two distinct investigators and interobserver readings was assessed by Bland-Altmann analysis, assuming the first reading as standard. Statistical significance was considered when the P value was <0.05.

3. Results

All isolates were submitted to MHT quantification and revealed 17 different ETP and MEM ratios by the ROC curves. The SN and SP of conventional reading of MHT (any enhanced growth, represented by lower ratios), with both ETP and MEM as substrates, were 100% and 28.4%. Depending on the enhanced growth ratio used for positive or negative MHT result, the SN and SP of ETP for KPC detection ranged from 58% to 100% and from 28.4% to 99.7%,

Table 1: Minimal inhibitory concentration (MIC) of carbapenems of the 384 isolates evaluated.

Isolates n MICb Imipenem MIC50 MIC90 Range MICb MIC (mg/L)'1 Meropenem MIC50 MIC90 Range MICb Ertapenem MIC50 MIC90 Range

KPC-producing isolates 50 — 16 128 1.0-256 — 32 128 4.0->256 — 64 256 4.0->256

Klebsiella spp. 36 — 16 64 — 32 64 — 64 128

Enterobacter spp. 10 — 8 64 — 16 128 — 32 256

S. marcescens 3 — 128 256 — 32 64 — 64 128

K. georgiana 1 256 - — 128 — — 128 — —

Non-carbapenemase-producing isolates 334 — 0.5 2.0 <0.5-32 0.5 4.0 <0.5-32 4.0 16 <0.5-128

Enterobacter spp. 220 — 0.5 2.0 0.5 2.0 4.0 8.0

Klebsiella spp. 89 — 1.0 4.0 1.0 8 4.0 32

E. coli 17 — 0.5 4.0 1.0 4.0 4.0 16

S. marcescens 5 — 1.0 4.0 1.0 4.0 2.0 8.0

P. mirabilis 1 2.0 — — 0.5 — — 0.5 _ —

M. morganii 1 4.0 - — 0.5 — — 0.5 — —

C. freundii 1 <0.5 — — 0.5 — — 0.5 — —

'Determined by broth microdilution [17]. Result of MIC of species with a single isolate.

Table 2: Sensitivity (SN) and specificity (SP) of distinct values of ertapenem (ETP) enhanced growth ratios.

ETP ratios SN (%) SP (%)

0.045 100.0 28.4

0.095 100.0 29.3

0.106 100.0 63.5

0.118 100.0 64.1

0.134 100.0 65.6

0.171 100.0 67.1

0.211 96.0 90.7

0.254 96.0 91.3

0.293 96.0 92.5

0.317 96.0 97.0

0.367 96.0 97.3

0.450 96.0 99.4

0.583 94.0 99.7

0.683 92.0 99.7

0.750 82.0 99.7

0.850 70.0 99.7

0.950 58.0 99.7

Table 3: Sensitivity (SN) and specificity (SP) of distinct values of meropenem (MEM) enhanced growth ratios.

MEM ratios SN (%) SP (%)

0.045 100.0 28.4

0.095 100.0 29.3

0.113 100.0 79.0

0.134 100.0 79.9

0.162 100.0 81.7

0.191 100.0 82.3

0.225 98.0 94.6

0.268 98.0 95.2

0.293 98.0 95.8

0.350 96.0 98.5

0.450 96.0 100.0

0.550 94.0 100.0

0.560 90.0 100.0

0.750 86.0 100.0

0.838 66.0 100.0

0.888 62.0 100.0

0.950 54.0 100.0

respectively (Table 2), and from 54% to 100% and 28.4% to 100% for MEM, respectively (Table 3). As both ETP and MEM ratios increase, the SP also increases. In contrast, as both increase, the SN decreases.

The highest combined SN and SP values obtained for both substrates were a ratio = 0.45 (Tables 2 and 3). If ratios <0.45 were assigned as a positive MHT, two KPC-producing isolates would be considered negative for both substrates: an E. cloacae (ETP ratio = 0.2 and MEM ratio = 0.3) and

a S. marcescens (ETP and MEM ratio = 0.2). Among non-carbapenemase-producing isolates, two E. cloacae showed a ratio > 0.45 only when ETP was used as substrate, with ratios of 0.5 and 1.0. For these two isolates, ESBL production was performed by phenotypic tests using combination-disk assay with clavulanic acid as inhibitors, as well as by PCR for the presence of blaCTX-M [17,20]. The presence of AmpC was also investigated, as described by Giske et al. [8]. Both isolates presented positive results with cloxacillin assay, indicating the presence of an AmpC, and the first one also showed the presence of a CTX-M.

The PPV and NPV for quantitative MHT are shown in Table 4. The area under the curve (AUC) calculated for ER was 0.990 (95% confidence interval [CI] 0.9771.0) (Figure 2(a)) and for MR 0.997 (95% [CI] 0.992-1.0) (Figure 2(b)). The Bland-Altmann analysis indicated that there was intra- and interobserver agreement in the readings. The comparisons included intraobserver agreement of ERfor two different investigators (P = 0.081 and P = 0.073), intraobserver agreement of MR also for both investigators (P = 0.962 and P = 0.078), and interobserver agreement for both substrates (P = 0.894 and P = 0.136). Most reading disagreements were <1 mm.

4. Discussion

KPC dissemination among Enterobacteriaceae isolates in recent years has challenged clinical laboratories to provide a rapid, inexpensive, and accurate result of which isolate with reduced susceptibility to carbapenems would more likely produce these enzymes. Although MHT fulfilled the first two characteristics, it is an inaccurate test, since false-positive results maybe found in isolates producing other ^-lactamases with some marginal carbapenemase activity, such as AmpC and ESBLs [5,10].

The results of the present study indicate that both SN and SP vary according to the size of the enhanced growth of the indicatorstrain. Forbothsubstratesanalyzed, ETPand MEM, our results clearly demonstrated that higher ratios originated higher SP values, whereas lower ratios originated higher SN. Therefore, as previously hypothesized, ETP and MEM ratios are directly proportional to SP and inversely proportional to SN in carbapenemase detection. It is interesting to note that, using the conventional qualitative MHT interpretation, the SN was 100% for both substrates; however, unacceptable low SP values (28.4%) were obtained. In our study, the SN of ETP and MEM ratios >0.45 (96%) was only slightly decreased in comparison with the SN (100%) of the conventional qualitative MHT reading, and it was similar to the SN values of MHT reported in other studies (>90%) [17, 21, 22]. The PPV and NPV found for both substrates indicated that the quantitative MHT is a useful and reliable test to be used in settings with low and high prevalence of KPC-2-producing bacteria. Regarding overall accuracy, MEM tended to have a better performance compared to ETP, with an AUC of 0.997 compared to 0.990 for the last. These results were driven by the better SP values when using MEM as substrate. The

Table 4: Positive predictive values (PPV) and negative predictive values (NPV) of the quantitative Modified Hodge Test, according to estimated population prevalence of blaKPC-2.

Population prevalence fui PPV of blaKPC 2 KPC-2 (95% CI) MEM NPV (95% CI) PPV (95% CI) ERT NPV (95% CI)

10% 97.3% 99.6% 94.7% 99.6%

(87.8-99.4%) (98.5-99.9%) (83.8-98.4%) (98.5-99.9%)

20% 98.8% 99.0% 97.6% 99.0%

(94.2-99.8%) (96.7-99.7%) (92.1-99.3%) (96.7-99.7%)

30% 99.3% 98.3% 98.6% 98.3%

(96.5-99.9%) (94.5-99.5%) (95.2-99.6%) (94.5-99.5%)

50% 99.7% 96.1% 99.4% 96.1%

(98.0-99.9%) (88.0-98.8%) (97.9-99.8%) (88.0-98.3%)

70% 99.9% 91.4% 99.7% 91.4%

(99.3-100%) (75.9-97.3%) (99.1-99.9%) (75.9-97.3%)

MEM: meropenem; ERT: ertapenem.

1.0 -,

ER ROC curve

I 0.4 A

AUC = 0.990 (95% [CI] 0.977-1.0)

0.4 0.6

1 - specificity Diagonal segments are produced by ties.

MR ROC curve

0.4 0.6

1 - specificity Diagonal segments are produced by ties.

Figure 2: Ertapenem enhanced growth ratio (ER) ROC curve (a) and meropenem enhanced growth ratio (MR) ROC curve (b). AUC = area under the curve; CI = confidence interval.

agreement analysis of intra- and interobserver indicated that the readings are reproducible and that the ratios generated on different days are reliable.

A limitation of our study was that only KPC carbapen-emase was assessed, and these findings may not be applied to other carbapenemases types. Considering that MHT is not specific for KPC detection, it could be expected that the SN and SP would vary in a similar way with other car-bapenemases, although further studies are needed to confirm this hypothesis. Quantification of MHT primarily aims to differentiate the presence of carbapenemase production from other enzymes with minimal carbapenemase activity. It is also known that the 0.45 cut-off must not be taken as definitive and its accuracy should be further investigated.

5. Conclusions

Our study showed that distinct sizes of enhanced growth of the indicator strain in MHT determine different SN and SP for the detection of carbapenemase-producing isolates. The quantification of MHT with either ETP or MEM as substrates increases SP of the test for the detection of carbapenemase activity in Enterobacteriaceae with minimal impairment in the SN, when compared with the conventional reading of the MHT. Although other phenotypic tests are currently available, the quantification of MHT may be an alternative screening test, particularly for clinical laboratories from low-income regions. The quantification of MHT must be further investigated with other non-KPC-type carbapenemases.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Authors' Contribution

Alexandre P. Zavascki and Afonso Luis Barth, both senior authors, contributed equally to the work.


The authors are most grateful to Dr. Jaime Rocha and Dr. Felipe Tuon who kindly provided some KPC isolates. This work was supported by Fundo de Incentivo a Pesquisa e Eventos do Hospital de Clínicas de Porto Alegre (FIPE—Project no. 09-270), Fundafáo de Amparo a Pesquisa do Estado do Rio Grande do Sul (FAPERGS—Process no. 10/0026-1), and Coordenafao de Aperfeifoamento de Pessoal de Nivel Superior (CAPES).


[1] J. B. Patel, J. K. Rasheed, and B. Kitchel, "Carbapenemases in enterobacteriaceae: activity, epidemiology, and laboratory detection," Clinical Microbiology Newsletter, vol. 31, no. 8, pp. 5562, 2009.

[2] J. Quale, "Global spread of carbapenemase-producing Klebsiella pneumoniae," Microbe, vol. 3, no. 11, pp. 516-520, 2008.

[3] P. Nordmann, G. Cuzon, and T. Naas, "The real threat of Klebsiella pneumoniae carbapenemase-producing bacteria," The Lancet Infectious Diseases, vol. 9, no. 4, pp. 228-236, 2009.

[4] F. Pasteran, T. Mendez, L. Guerriero, M. Rapoport, and A. Corso, "Sensitive screening tests for suspected class A carbapene-mase production in species of Enterobacteriaceae," Journal of Clinical Microbiology, vol. 47, no. 6, pp. 1631-1639, 2009.

[5] F. Pasteran, T. Mendez, M. Rapoport, L. Guerriero, and A. Corso, "Controlling false-positive results obtained with the hodge and masuda assays for detection of class a carbapenemase in species of enterobacteriaceae by incorporating boronic acid," Journal of Clinical Microbiology, vol. 48, no. 4, pp. 1323-1332, 2010.

[6] A. Tsakris, K. Themeli-Digalaki, A. Poulou et al., "Comparative evaluation of combined-disk tests using different boronic acid compounds for detection of Klebsiella pneumoniae carba-penemase-producing Enterobacteriaceae clinical isolates," Journal of Clinical Microbiology, vol. 49, no. 8, pp. 2804-2809, 2011.

[7] A. Birgy, P. Bidet, N. Genel et al., "Phenotypic screening of carbapenemases and associated ^-lactamases in carbapenem-resistant Enterobacteriaceae," Journal of Clinical Microbiology, vol. 50, no. 4, pp. 1295-1302, 2012.

[8] C. G. Giske, L. Gezelius, 0. Samuelsen, M. Warner, A. Sundsfjord, and N. Woodford, "A sensitive and specific phenotypic assay for detection of metallo-^-lactamases and KPC in Kleb-siella pneumoniae with the use of meropenem disks sup-ple-mented with aminophenylboronic acid, dipicolinic acid and cloxacillin," Clinical Microbiology and Infection, vol. 17, no. 4, pp. 552-556, 2011.

[9] K. F. Anderson, D. R. Lonsway, J. K. Rasheed et al., "Evaluation of methods to identify the Klebsiella pneumoniae carbapenemase in Enterobacteriaceae," Journal of Clinical Microbiology, vol. 45, no. 8, pp. 2723-2725, 2007.

[10] C. G. Carvalhaes, R. C. Picao, A. G. Nicoletti, D. E. Xavier, and A. C. Gales, "Cloverleaf test (modified Hodge test) for detecting carbapenemase production in Klebsiella pneumoniae: be aware of false positive results," The Journal of Antimicrobial Chemotherapy, vol. 65, no. 2, pp. 249-251, 2010.

[11] S. Pournaras, A. Poulou, and A. Tsakris, "Inhibitor-based methods for the detection of KPC carbapenemase- producing Enterobacteriaceae in clinical practice by using boronic acid compounds," Journal of Antimicrobial Chemotherapy, vol. 65, no. 7, Article ID dkq124, pp. 1319-1321, 2010.

[12] M. Doumith, M. J. Ellington, D. M. Livermore, and N. Woodford, "Molecular mechanisms disrupting porin expression in ertapenem-resistant Klebsiella and Enterobacter spp. clinical isolates from the UK," Journal of Antimicrobial Chemotherapy, vol. 63, no. 4, pp. 659-667, 2009.

[13] E. B. Hirsch and V. H. Tam, "Detection and treatment options for Klebsiella pneumoniae carbapenemases (KPCs): an emerging cause of multidrug-resistant infection," Journal of Antimicrobial Chemotherapy, vol. 65, no. 6, Article ID dkq108, pp. 11191125, 2010.

[14] P. Nordmann, D. Girlich, and L. Poirel :, "Detection of carbapenemase producers in Enterobacteriaceae by use of a novel screening medium," Journal of Clinical Microbiology, vol. 50, pp. 27612766, 2012.

[15] P. Nordmann, L. Poirel, and L. Dortet, "Rapid detection of carbapenemase-producing Enterobacteriaceae," Emerging Infectious Diseases Journal, vol. 18, pp. 1503-1507, 2012.

[16] Clinical and Laboratory Standard Institute, "Performance standards for antimicrobial susceptibility testing, 19th informational supplement," Approved Document M100-S19, CLSI, Wayne, Pa, USA, 2009.

[17] Clinical and Laboratory Standards Institute, "Performance standards for antimicrobial susceptibility testing: 22th informational supplement," Approved Document M100-S22, CLSI, Wayne, PA, 2012.

[18] J. Monteiro, R. H. Widen, A. C. C. Pignatari, C. Kubasek, and S. Silbert, "Rapid detection of carbapenemase genes by multiplex real-time PCR," Journal of Antimicrobial Chemotherapy, vol. 67, no. 4, Article ID dkr563, pp. 906-909, 2012.

[19] H. Yigit, A. M. Queenan, G. J. Anderson et al., "Novel carbape-nem-hydrolyzing ^-lactamase, KPC-1, from a carbapenem-re-sistant strain of Klebsiella pneumoniae," Antimicrobial Agents and Chemotherapy, vol. 45, no. 4, pp. 1151-1161, 2001.

[20] D. L. Paterson, K. M. Hujer, A. M. Hujer et al., "Extended-spectrum ^-lactamases in Klebsiella pneumoniae bloodstream isolates from seven countries: dominance and widespread prevalence of SHV- and CTX-M-type ^-lactamases," Antimicrobial Agents and Chemotherapy, vol. 47, no. 11, pp. 3554-3560, 2003.

[21] D. Doyle, G. Peirano, C. Lascols, T. Lloyd, D. L. Church, and J. D. Pitout, "Laboratory detection of enterobacteriaceae that produce carbapenemases," Journal of Clinical Microbiology, vol. 50, pp. 3877-3880, 2012.

[22] D. Girlich, L. Poirel, and P. Nordmann, "Value of the modified hodge test for detection of emerging carbapenemases in Enterobacteriaceae," Journal of Clinical Microbiology, vol. 50, no. 2, pp. 477-479, 2012.

Copyright of BioMed Research International is the property of Hindawi Publishing Corporation and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.