0BMJ Open Diagnosing malignant melanoma
in ambulatory care: a systematic review of clinical prediction rules
Emma Harrington,1 Barbara dyne,1 Nieneke Wesseling,2 Harkiran Sandhu,1 Laura Armstrong,1 Holly Bennett,1 Tom Fahey1
To cite: Harrington E, Clyne B, Wesseling N, etal. Diagnosing malignant melanoma in ambulatory care: a systematic review of clinical prediction rules. BMJ Open 2017;7:e014096. doi:10.1136/bmjopen-2016-014096
► Prepublication history and additional material is available. To view please visit the journal (http://dx.doi.org/ 10.1136/bmjopen-2016-014096).
Received 30 August 2016 Accepted 10 January 2017
CrossMark
1HRB Centre for Primary Care Research, Royal College of Surgeons in Ireland, Dublin 2, Ireland
2Medical School, Radboud University, Nijmegen, Netherlands
ABSTRACT
Objectives: Malignant melanoma has high morbidity and mortality rates. Early diagnosis improves prognosis. Clinical prediction rules (CPRs) can be used to stratify patients with symptoms of suspected malignant melanoma to improve early diagnosis. We conducted a systematic review of CPRs for melanoma diagnosis in ambulatory care. Design: Systematic review. Data sources: A comprehensive search of PubMed, EMBASE, PROSPERO, CINAHL, the Cochrane Library and SCOPUS was conducted in May 2015, using combinations of keywords and medical subject headings (MeSH) terms.
Study selection and data extraction: Studies deriving and validating, validating or assessing the impact of a CPR for predicting melanoma diagnosis in ambulatory care were included. Data extraction and methodological quality assessment were guided by the CHARMS checklist.
Results: From 16 334 studies reviewed, 51 were included, validating the performance of 24 unique CPRs. Three impact analysis studies were identified. Five studies were set in primary care. The most commonly evaluated CPRs were the ABCD, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter (ABCD) dermoscopy rule (at a cut-point of >4.75; 8 studies; pooled sensitivity 0.85, 95% CI 0.73 to 0.93, specificity 0.72, 95% CI 0.65 to 0.78) and the 7-point dermoscopy checklist (at a cut-point of >1 recommending ruling in melanoma; 11 studies; pooled sensitivity 0.77, 95% CI 0.61 to 0.88, specificity 0.80, 95% CI 0.59 to 0.92). The methodological quality of studies varied.
Conclusions: At their recommended cut-points, the ABCD dermoscopy rule is more useful for ruling out melanoma than the 7-point dermoscopy checklist. A focus on impact analysis will help translate melanoma risk prediction rules into useful tools for clinical practice.
Strengths and limitations of this study
■ The main strengths of this review are the use of broad inclusion criteria, the systematic search of multiple databases not limited by language, use of the CHARMS checklist to assess methodological quality, pooling data from a broad range of studies to enhance generalisability and the use of a broad definition of primary care to account for the variation in primary care services and access internationally. Quality assessment criteria were used to assess risk of bias and the majority of studies were at low risk in relation to the randomisation procedure and monitoring of loss to follow-up.
■ A large proportion of studies did not provide sufficient information and data to perform stratified meta-analysis according to different levels of risk.
■ Current research shows that dermoscopic clinical prediction rules (CPRs) may be a useful tool for primary care physicians prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients but future impact analysis research is necessary to establish their impact on patient outcomes.
Correspondence to
Dr Barbara Clyne; barbaraclyne@rcsi.ie
INTRODUCTION
The incidence of malignant melanoma in most developed countries has been steadily
rising (faster than other cancer types) in recent decades.1 2 Increases in the age-standardised incidence of at least 4—6% per annum have been reported internationally in many fair-skinned populations including Australia, the USA and most of Europe.3-5 Simultaneously, there has been a significant rise in overall 5-year survival in melanoma patients, largely attributable to earlier detection and diagnosis of thinner tumours.6 While the majority of patients may survive melanoma, the disease has a significant impact on patient quality of life7 and healthcare expenditure, with the average annual total treatment costs for melanoma in the USA increasing to US $3.3 billion in 2011. Melanoma is potentially preventable since a significant risk factor,
exposure to ultraviolet (UV) radiation, is modifiable. However, other risk factors (eg, number naevi, eye and hair colour, freckles, familial history and genetic predisposition) also play an important role in the risk of developing 10 11
melanoma.
Early detection followed by curative surgery greatly improves melanoma prognosis. However, early detection may be affected by the challenging natures of differential diagnosis of pigmented lesions. Particularly in primary care where the evaluation of suspected skin lesions is imposing an increasing burden due to rising incidences of skin cancer.12 It has been suggested that primary care practitioners' skills of diagnosing skin lesions could be improved.13 A number of clinical prediction pules (CPRs) and computer-assisted diagnostic tools have been developed to assist in distinguishing malignant melanoma from benign pigmented skin lesions. The UK National Institute for Clinical Excellence (NICE) guidelines advise against routine use of computer-assisted diagnostic tools in the initial evaluation of a pigmented skin lesion (NICE guidelines) and promote use of the weighted 7-point checklist in primary care to guide referral (NG12). When used by dermatologists for the diagnosis of melanoma, certain CPRs have demonstrated high sensitivity and specificity.6 Although each CPR has its own unique elements, there is significant overlap in terms of their content (see online supplementary appendix 1), and while their use is promoted, it is unclear which rules are most suitable for use in primary care.
CPRs may be for use in clinical (ie, naked eye) examination, or in conjunction with dermoscopy. Dermoscopy, dermatoscopy or epiluminescent microscopy refers to the examination of pigmented skin lesions using surface microscopy.14 15 The use of dermoscopy, primarily by dermatologists, has been found to increase diagnostic accuracy compared with naked-eye inspection, as it allows the visualisation of features that are not visible to the naked eye.14-16 However, the effectiveness of dermoscopy depends on clinical experience and training. Dermatologists with formal training in dermo-scopy have higher melanoma detection rates compared
with untrained dermatologists and primary care physi-16-18
cians.
As primary care or ambulatory care physicians are frequently and increasingly confronted with the care of skin lesions suspected of malignancy,12 it is essential to identify tools to aid primary care practitioners to differentiate patients with clinically significant lesions, requiring referral, from those who can be treated and monitored in primary care. The aim of this study was to perform a systematic review of CPRs for the diagnosis of malignant melanoma, to evaluate their diagnostic accuracy in primary care and specialist outpatient settings, among patients with a pigmented skin lesion. Secondary aims were to review studies that have examined the implementation of CPRs in clinical practice through impact analysis studies.
METHODS
The protocol for this systematic review was published on PROSPERO (CRD42015020898) and was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.19
Search strategy and data sources
A systematic literature search was conducted (May 2015) including the following databases: PubMed, EMBASE, PROSPERO, CINAHL, the Cochrane Library and SCOPUS, using combinations of the following keywords and MeSH terms: melanoma/diagnosis, melanoma, prediction, score, model, decision, sensitivity, specificity, validate, derived. Hand searches of references of retrieved full-text articles and key author searches supplemented the search. No date or language limits were imposed.
Study selection
All articles were initially screened for inclusion according to title and abstract by two reviewers (NW, EH). Full-text articles of studies considered eligible for inclusion were independently read by both reviewers, with any disagreements resolved by a third reviewer (BC).
Validation studies
Validation studies were eligible for inclusion if they met the following criteria;
1. Population: Adults (age >18 years) with a pigmented skin lesion in ambulatory care settings in general practice/family medicine, dermatology, plastic surgery and other relevant specialties.
2. Risk: Derivation and/or validation of a CPR for melanoma diagnosis to aid decision making about referral or investigation of a pigmented skin lesion. CPRs were defined as 'a clinical tool that quantifies the individual contributions that various components of the history, physical examination and investigations make toward the diagnosis, prognosis or likely response to treatment in a patient'.
3. Comparison: Usual clinical judgement for decision making about referral or investigation OR another CPR for melanoma diagnosis.
4. Primary outcome: Performance of a CPR for predicting diagnosis of malignant melanoma (in terms of sensitivity, specificity, negative predictive values and positive predictive values).
Observational study designs (eg, cohort, cross-sectional, case-control) were included. Studies were excluded where they had undergone derivation only, reported individual predictors only, or used computerassisted diagnostic tools, following the NICE guideline recommendation against the routine use of computerassisted diagnostic tools.
Impact analysis
The following study designs were included for impact analysis: (cluster) randomised controlled trials (RCTs), controlled before-after studies or interrupted time series
studies. We excluded uncontrolled study designs. We included studies where a melanoma CPR was used to predict melanoma compared with usual care in the clinical setting. The outcomes of interest included physician behaviour, process of care, patient outcomes and/or cost-effectiveness. A requirement for inclusion was that the CPR comprised the entire intervention. Studies where the CPR was implemented as part of a broader guideline, protocol or decision aid were excluded. Studies that used a CPR to determine eligibility for trial inclusion but were not part of the intervention were also excluded.
Data extraction
Data were extracted by four reviewers (LA, HB, HS, EH) using a data form based on the CHARMS checklist.21 Data extracted included study design and setting, patient demographics and inclusion criteria, CPR name, CPR type (clinical or dermoscopic), predictive accuracy of the CPR (sensitivity/specificity) and for impact analysis, the impact on the primary outcome.
Critical appraisal of studies
Two reviewers (EH, NW) critically appraised included studies using the CHARMS checklist, developed to provide guidance on data extraction and critical appraisal of prediction modelling studies.21 The checklist contains 11 domains of critical appraisal. The methodological quality of each study was independently evaluated by two reviewers and by a third reviewer if consensus was not reached. The methodological quality of each impact analysis study was also independently assessed, using an appropriate quality assessment checklist. RCTs were assessed using the Cochrane risk of bias tool and controlled before-after studies were evaluated using Cochrane criteria for these study designs.22
Statistical analysis
Statistical analysis was conducted using Stata V.12 (StataCorp, College Station, Texas, USA), in particular the metandi and midas commands. For each CPR, a standard cut-point was identified (table 1). From each included study we extracted (where available) the
Table 1 CPRs identified for inclusion with cut-points for identification of melanoma
Number of validation
Rule name Cut-point used studies
Clinical rule
ABCDE clinical rule >1 or >2 4
ABCD clinical rule >1 4
Revised 7-point checklist (clinical) >3 4
7-point checklist (clinical) >3 4
Dermoscopic rules
ABCD rule of dermoscopy* >4.75 15
>5.45 6
>4.2 1
Not reported 1
7-point checklist for dermoscopy >3 17
Menzies 1996 dermoscopy for melanoma >1, no negative features 8
3-point checklist for dermoscopy >1 6
Seven features for melanoma (7FFM) >2 5
CASH dermoscopy algorithm >8 3
ABCDE rule (dermoscopy) Not reported 2
The 3-colour dermoscopy test >3 2
Revised 7-point checklist for dermoscopy >1 1
Kreusch 1992 dermoscopy Not reported 1
Nilles 1994 dermoscopy Not reported 1
Menzies 2008 dermoscopy for melanoma >1 1
DynaMel algorithm >3 1
Menzies 2008 dermoscopy for skin cancer >0 (high sensitivity); >1 (high specificity) 1
Simplified ABC-point list for dermoscopy >4 1
AC rule for dermoscopy Not reported 1
Emery 2010 SIAscopy >6 1
Guitera RCM 2012 Not reported 1
Digital dermoscopy algorithms Multiple algorithms, different cut-offs 1
*Score = (A scorex1.3)+(B scorex0.1)+(C scorex0.5)+(D scorex0.5).
ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Colour; ABCD, Asymmetry, irregular Borders, more than one or
uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven
distribution of Colour, or a large (greater than 6mm) Diameter, Evolution of moles; Ac, asymmetry, colour variation; CASH, color, architecture,
symmetry, and homogeneity; CPR, clinical prediction rules RCM, reflectance confocal microscopy.
numbers of true positives, false positives, true negatives, false negatives, sensitivity and specificity and their corresponding 95% CIs. Where sensitivity/specificity for more than one observer was reported, the mean value was included in the analysis. Studies were grouped for analysis by CPR type (ie, clinical or dermoscopic). Summary estimates of sensitivity and specificity and their corresponding 95% CIs were calculated using the bivariate random effects model (midas). The bivariate model has the benefits of being easily interpretable, is technically straightforward to undertake and takes into account the sample and heterogeneity beyond chance between
studies.23
Individual and summary estimates of sensitivity and specificity were plotted on a hierarchical summary receiver operating characteristic (HSROC) graph. This approach incorporates sensitivity and specificity, while taking into account the correlation between the two.24 Sensitivity (true positive) was graphed on the y-axis and 1-specificity ( false negative) on the x-axis. The 95% confidence region and the 95% prediction region were also plotted around the pooled estimates in order to depict the precision with which the pooled estimates were determined (confidence ellipse around the mean value) and to illustrate the amount of between-study variation ( prediction ellipse).
RESULTS Study selection
The search strategy yielded a total of 25 816 articles. Of these 9481 were duplicates and 16 166 were deemed irrelevant based on title/abstract. The remaining 171 were reviewed in full with 51 meeting the inclusion criteria (see online supplementary appendix 2). From these, 24 unique melanoma CPRs were identified (table 1). Twelve papers reported derivation and validation studies, 36 were validation studies only and three were impact analyses.
Summary of studies
Table 2 summarises the characteristics of the included studies. The majority (11, 22%) were conducted in
Italy14 15 25-34 and ranged from an analysis of 40 lesions
to 1580 lesions. From 13 studies providing information, mean age of included patients ranged from 36.7 to 53.25 28 31 35-44 From the 14 studies that reported gender, the proportion of males ranged from 22% to 60%.25 31 33 35-45 In total, 31 of the 50 studies were published in/or after 20 00.14 25 28 29 31-37 42-44 46-62 Five
studies were set in primary care,36 44 49 62 63 with the
remainder undertaken in specialist outpatient settings.
Summary of CPRs identified
Of the 24 rules identified, four were clinical (ie, naked eye), 17 were dermoscopic and the remaining three used novel diagnostic technologies. The most commonly applied clinical CPR was the ABCDE rule ( five
studies),6 15 28 64 65 while for dermoscopy the most common were the Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter (ABCD) rule of dermo-
14 25 26 29 31 32 39 42 43 47-49 52 53 57 65-70
scopy (23 studies)
and the 7-point checklist for dermoscopy (17
studies) 14 25 26 29 35 37 42 43 46-50 52 56 57 59
Each of the elements included in the 24 rules identified are presented in table 3. All four clinical rules included the elements of diameter and colour variegation (table 3 and see online supplementary appendix 1). The most frequently included elements in the 17 dermo-scopic rules were multiple colours (13 rules), asymmetry (12 rules) and streaks (10 rules) (table 3 and see online supplementary appendix 1).
Methodological quality of validation studies
Based on the CHARMS checklist, the quality of included studies varied.21 All studies had weaknesses in study design and quality assessment was often hindered by poor reporting of methods. The studies had reasonable sample sizes and all provided adequate definitions of the outcome of interest. However, a number of important weaknesses were identified. None of the studies reported on missing data and key performance measures of model performance (eg, calibration) were often missing. Derivation studies typically reported information on model development, in terms of selection of candidate predictors, selection of predictors during modelling, and model evaluation. However, often the methods applied introduced a strong risk of bias, for example, a number of studies described splitting the original sample into a development and validation sample which is considered statistically inefficient and results in overfitting of the model.21 Full results of the quality assessment are shown in online supplementary appendix 3.
Predictive accuracy of melanoma CPRs
The results for the most commonly applied CPRs, the ABCD rule and the 7-point checklist are presented here. The sensitivity and specificity of all rules identified (including the ABCDE clinical rule, the seven features for melanoma rule and Menzies dermoscopy for melanoma rule) are summarised in table 4.
Clinical (naked eye) CPRs for melanoma diagnosis
Four studies validating the ABCDE clinical rule6 15 28 64 and one validating the ABCD clinical rule65 were included. There was insufficient data to conduct any meta-analysis. Rao et al reported a sensitivity of 0.84 and specificity of 0.78, for an unspecified cut-point.65
Six studies validating the original and revised 7-point checklist were included. There was insufficient data to conduct a meta-analysis. Of the four studies validating the original 7-point checklist (cut-point >3), three reported sensitivity (range 0.44-0.86, mean 0.70) and specificity (range 0.62-0.94, mean 0.74).40 41 44 Only one of the four studies validating the revised 7-point
Table 2 Characteristics of validation and impact analysis studies included
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Annessi Department of ABCD rule of 198 N=195 2 ABCD rule of dermoscopy
2007,25 Italy dermatology dermoscopy 96 melanomas, 102 54% male ELM-experienced (cut-point >4.75)
7-point checklist non-melanoma Mean age: 43 dermatologists Se: 84.4
for dermoscopy Sp: 74.5
7-point checklist for
dermoscopy (cut-point >3)
Se: 78.1
Sp: 64.7
Argenziano Department of 7-point checklist 342 NR 5 7-point checklist for
1998,26 Italy dermatology for dermoscopy 117 melanoma, 225 3 experienced dermoscopy (cut-point >3)
ABCD rule of non-melanoma 2 less experienced Expert user:
dermoscopy Se: 95.0
Argenziano 2003,14 9 countries
Department of dermatology
ABCD rule of dermoscopy 7-point checklist for dermoscopy Menzies 1996 dermoscopy for melanoma
Experienced
Sp: 75.0
Non-expert user (mean): Se: 89.0 Sp: 61.5
ABCD rule of dermoscopy (cut-point >4.75) Expert user: Se: 85.0 Sp: 66.0
Non-expert user (mean): Se: 91.5 Sp: 31.0
ABCD rule of dermoscopy (cut-point >4.75) Se: 82.6 Sp: 70.0
7-point checklist for dermoscopy Se: 85.7 Sp: 71.1
Menzies 1996 dermoscopy for melanoma Se: 85.7 Sp: 71.1
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Argenziano Department of 7-point checklist 300 NR 8 7-point checklist for
2011,27 Italy dermatology for dermoscopy 100 excised melanoma, Experienced dermoscopy (cut-point >3)
Revised 7-point 100 excised Se: 77.9
checklist for non-melanoma, 100 Sp: 85.6
dermoscopy non-excised Revised 7-point checklist for
non-melanoma dermoscopy (cut-point >1)
Se: 87.8
Sp: 74.5
Benelli Department of 7FFM (seven 401 NR 2 7FFM (seven features for
1999,15 Italy dermatology features for 60 melanomas, 341 Research team melanoma) dermoscopy
melanoma) non-melanoma (cut-point of >2)
dermoscopy Se: 80.0
ABCDE clinical Sp: 89.1
rule ABCDE clinical rule
(cut-point >2)
Se: 85.0
Sp: 44.5
Benelli Department of 7FFM (seven 600 Mean age: 53 3 7FFM (seven features for
2000,28 Italy dermatology features for 76 melanomas, 524 melanoma) dermoscopy
melanoma) non-melanoma (cut-point of >2)
dermoscopy Se: 68.8
ABCDE clinical Sp: 86.0
rule ABCDE clinical rule (cut-point
of >2)
Se: 47.3
Sp: 56.0
Binder Department of ABCD rule of 250 NR 17 ABCD rule of dermoscopy
1999,66 dermatology dermoscopy 12 experienced (cut-point >4.75)
Austria 5 trainee Se: 81.0
Sp: 77.0
ABCD rule of dermoscopy
(cut-point >5.45)
Se: 73.0
Sp: 90.0
Blum 2003,71 Department of The 3-colour 249 NR NR The 3-colour dermoscopy test
Germany dermatology dermoscopy test Se: 76.9
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Blum 2004,47 Department of ABCD rule of 269 NR NR ABCD rule of dermoscopy
Germany dermatology dermoscopy 84 melanomas, 185 Se: 90.5
7-point checklist non-melanoma Sp: 72.4
for dermoscopy 7-point checklist for
Menzies 1996 dermoscopy
dermoscopy for Se: 90.5
melanoma Sp: 87.0
Simplified Menzies 1996 dermoscopy
ABC-point list for for melanoma
dermoscopy Se: 95.2
7FFM (seven Sp: 77.8
features for 7FFM (seven features for
melanoma) melanoma) dermoscopy
dermoscopy Se: 94.0
Sp: 74.6
Simplified ABC-point list for
dermoscopy
Se: 90.5
Sp: 87.0
Blum 2004,48 Department of ABCD rule of 269 NR NR ABCD rule of dermoscopy
Germany dermatology dermoscopy 84 melanomas, 185 Se: 90.5
7-point checklist non-melanoma Sp: 72.4
for dermoscopy 7-point checklist for
Menzies 1996 dermoscopy
dermoscopy for Se: 90.5
melanoma Sp: 87.0
7FFM (seven Menzies 1996 dermoscopy
features for for melanoma
melanoma) Se: 95.2
dermoscopy Sp: 77.8
7FFM (seven features for
melanoma) dermoscopy
Se: 94.0
Buhl 2012,35 Sp: 74.6
Department of DynaMel Algorithm 675 N=688 Dermatology residents DynaMel Algorithm
Germany dermatology 7-point checklist 57% male Se: 77.1
for dermoscopy
Mean age: 42
Sp: 98.1
7-point checklist for dermoscopy (cut-point >3) Se: 47.5
Sp: 99.0_
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Carli 2002,29 Department of ABCD rule of 200 NR 5 ABCD rule of dermoscopy
Italy dermatology dermoscopy 44 melanomas, 156 Dermatology residents (cut-point >5.45)
7-point checklist non-melanoma Se: 88.1
for dermoscopy Sp: 45.7
7-point checklist for
dermoscopy (cut-point >3)
Se: 91.9
Sp: 35.2
Dal Pozzo Department of 7FFM (seven 713 NR 3 7FFM (seven features for
1999,30 Italy dermatology features for 168 melanomas, 545 melanoma) dermoscopy
melanoma) non-melanoma Se: 94.6
dermoscopy Sp: 85.5
Dolianitis Primary care 7-point checklist 40 NR 61 7-point checklist for
2005,49 and dermatology for dermoscopy 20 melanomas, 20 35 primary care dermoscopy
Australia department ABCD rule of non-melanoma physicians, 10 Se: 81.4
dermoscopy dermatologists, 16 Sp: 73.0
Menzies 1996 trainee dermatologists ABCD rule of dermoscopy
dermoscopy for (cut-point >5.45)
melanoma Se: 77.5
Sp: 80.4
Menzies 1996 dermoscopy
for melanoma
Se: 84.6
Sp: 77.7
Emery Family practice Emery 2010 1211 N=858 1 Emery 2010 SIAscopy in
2010,36 UK SIAscopy in 52% male SIAscopy expert primary care for melanoma
primary care for Mean age: 50 Se: 50.0
melanoma Sp: 84.0
Feldman Department of ABCD rule of 500 NR NR ABCD rule of dermoscopy
1998,67 dermatology dermoscopy 30 melanomas, 470 (cut-point >4.2)
Austria non-melanoma Se: 88.0
Gereli 2010,50 Sp: 64.0
Department of 7-point checklist 96 NR 3 7-point checklist for
Turkey dermatology for dermoscopy 48 melanoma, 48 2 experienced dermoscopy (cut-point >3)
3-point checklist non-melanoma 1 inexperienced Se: 87.5
for dermoscopy Sp: 16.2
3-point checklist for dermoscopy (cut-point >2) Se: 89.6 Sp: 31.2
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Guitera Skin cancer Guitera 2012 710 N=663 NR Guitera 2012 confocal
2012,51 clinic confocal 216 melanomas, 494 microscopy for melanoma
Multiple microscopy for non-melanoma Se: 87.6
melanoma Sp: 70.8
Haenssle Department of 7-point checklist 1219 N=688 Inexperienced 7-point checklist for
2010,37 dermatology for dermoscopy 127 melanomas, 1092 57% male dermoscopy (cut-point >3)
Germany non-melanoma Mean age: 42 Se: 62.0
Sp: 97.0
Healsmith Pigmented Revised 7-point 165 NR NR Revised 7-point checklist
1993,64 UK lesion clinic checklist (clinical) 65 melanoma, 100 (clinical)
ABCDE clinical non-melanoma Se: 100
rule Sp: NR
ABCDE clinical rule
Se: 92.3
Sp: NR
Henning Department of CASH 150 NR 2 CASH dermoscopy algorithm
2008,52 USA dermatology dermoscopy 50 melanoma, 100 Inexperienced Se: 87.0
algorithm non-melanoma Sp: 67.0
ABCD rule of ABCD rule of dermoscopy
dermoscopy Se: 86.0
7-point checklist Sp: 74.0
for dermoscopy 7-point checklist for
Menzies 1996 dermoscopy
dermoscopy for Se: 76.0
melanoma Sp: 57.0
Menzies 1996 dermoscopy
for melanoma
Se: 92.0
Sp: 38
Higgins Department of 7-point checklist 100 N=100 NR 7-point checklist (clinical)
1992,38 UK dermatology (clinical) 0 melanoma, 100 30% male revised
7-point checklist non-melanoma Mean age: 36.7 Se: NR
Kittler 1999,39 (clinical) revised Sp: 70.0
Department of ABCD rule of 356 N=352 NR NR
Austria dermatology dermoscopy 73 melanomas, 283 43% male
ABCDE rule non-melanoma Mean age: 52
(dermoscopy)
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Keefe 1989,40 Hospital 7-point checklist 222 N=195 Dermatologists 7-point checklist (clinical)
Scotland dermatology clinic (clinical) 22% male Mean age: 43 195 patients (cut-point >3) Dermatologists: Se: 85.7 Sp: 66.5 Patients: Se: 71.4 Sp: 66.2
Kreusch Department of Kreusch 1992 317 NR 2 Kreusch 1992 dermoscopy for
1992,84 dermatology dermoscopy for 96 melanomas, 221 1 experienced melanoma
Germany melanoma non-melanoma 1 inexperienced Experienced: Se: 98.9 Sp: 94.1 Inexperienced: Se: 97.0 Sp: 94.2
Lorentzen Department of ABCD rule of 232 NR 8 ABCD rule of dermoscopy
1999,68 dermatology dermoscopy 4 experienced (cut-point >4.75)
Denmark 4 inexperienced Se: 59.0 Sp: 92.0 ABCD rule of dermoscopy (cut-point >5.45) Se: 41.0 Sp: 98.0
Lorentzen Department of ABCD rule of 258 NR 3 ABCD rule of dermoscopy
2000,53 dermatology dermoscopy 64 melanoma, 194 Experienced (cut-point >4.75)
Denmark non-melanoma Se: 70.7 Sp: 88.0
Luttrell Department of AC rule for 200 NR 17 AC rule for dermoscopy
2012,54 dermatology dermoscopy 25 melanoma, 178 Lay persons Se: 91.2
Austria non-melanoma Sp: 94.0
Mackie Pigmented The 3-colour 126 NR 3 The 3-colour dermoscopy test
2002,55 lesion clinic dermoscopy test 69 melanoma 57 Experienced Se: 97.0
Scotland non-melanoma Sp: 55.0
McGovern Dermatology 7-point checklist 237 N=179 NR 7-point checklist (clinical)
1992,41 USA clinic (clinical) 16 malignant, 221 50% male Se: 0.44
ABCD clinical rule non-melanoma Mean age: 44 Sp: 0.94
Menzies Melanoma unit Menzies 1996 385 NR NR Menzies 1996 dermoscopy
1996,85 dermoscopy for 107 melanomas for melanoma
Australia melanoma Se: 92.0 Sp: 71.0
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Menzies 7-point checklist 497 NR 12 7-point checklist for
200856 for dermoscopy 105 melanomas, 392 Experienced dermoscopy
3-point checklist of non-melanoma Se: 41.0
dermoscopy Sp: 83.0
Menzies 1996 3-point checklist of
dermoscopy for dermoscopy
melanoma Se: 50.0
Menzies 2008 Sp: 71.0
dermoscopy for Menzies 1996 dermoscopy
melanoma for melanoma
Menzies 2008 Se: 54.0
dermoscopy for Sp: 76.0
skin cancer Menzies 2008 dermoscopy for melanoma Se: 70.0 Sp: 56.0 Menzies 2008 dermoscopy for skin cancer Se: 95.0 Sp: 80.0
Menzies ABCD rule of 465 NR 12 ABCD rule of dermoscopy
201357 dermoscopy 217 melanomas, 248 Se: 81.5
7-point checklist non-melanoma Sp: NR
for dermoscopy 7-point checklist for
3-point checklist of dermoscopy
dermoscopy Se: 94.4
Menzies 1996 Sp: NR
dermoscopy for 3-point checklist of
melanoma dermoscopy
CASH Se: 83.9
dermoscopy Sp: NR
algorithm Menzies 1996 dermoscopy
Menzies 2013 for melanoma
dermoscopy for Se: 98.4
nodular melanoma Sp: NR CASH dermoscopy algorithm
Se: 41.0 Sp: 83.0
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Menzies 2013 dermoscopy
for nodular melanoma
Se: 93.0
Sp: 70.0
Nachbar Department of ABCD rule of 194 NR NR ABCD rule of dermoscopy
1994,69 dermatology dermoscopy 69 melanomas (cut-point >5.45)
Germany Se: 92.8
Nilles 1994,86 Sp: 91.2
Department of Nilles 1994 260 NR NR Nilles 1994 dermoscopy for
Germany dermatology dermoscopy for 72 melanomas, 188 melanoma
melanoma non-melanoma Se: 90.0
Sp: 85.0
Osborne Department of Revised 7-Point 778 N=733 NR Revised 7-Point Checklist
1999,45 UK Dermatology Checklist (clinical) 778 melanomas, 0 35% male (clinical)
non-melanoma False negative rate: 18.5
Piccolo Department of ABCD rule of 165 N=165 4 ABCD rule of dermoscopy
2014,31 Italy dermatology dermoscopy 33 melanomas, 129 59% male 3 dermatologists 1 FP Se: 91.0
non-melanoma Mean age: 43.5 Sp: 52.0
Pizzichetta Department of ABCD rule of 129 N=123 2 ABCD rule of dermoscopy
2002,32 Italy oncology dermoscopy Experienced (cut-point >4.75)
Se: 90.0
Sp: 43.0
ABCD rule of dermoscopy
(cut-point >5.45)
Se: 90.0
Sp: 53.5
Rao 199765 Department of ABCD rule of 73 N=63 4 ABCD rule of dermoscopy
dermatology dermoscopy Experienced (cut-point >4.75)
ABCD clinical rule dermatologists Se: 90.0
Sp: 57.0
ABCD clinical rule
Se: 84.0
Sp: 78.0
Skvara Department of ABCD rule of 325 N=297 2 ABCD rule of dermoscopy
2005,42 dermatology dermoscopy 63 melanomas, 262 44% male Experienced (cut-point >4.75)
Austria 7-point checklist non-melanoma Mean age: 39 dermatologists Se: 31.7
for dermoscopy
Sp: 87.3
7-point checklist for dermoscopy Se: 11.1
Sp: 95.2_
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Soyer 2004,33 Department of 3-point checklist of 231 N=225 6 3-point checklist of
Italy dermatology dermoscopy 68 melanomas, 163 49% male Inexperienced dermoscopy
non-melanomas Se: 96.3
Sp: 32.8
Stolz 1994,70 Department of ABCD rule of 157 NR NR ABCD rule of dermoscopy
Germany dermatology dermoscopy (cut-point >5.45)
Se: 97.9
Sp: 90.3
Strumia Department of ABCD rule of 49 NR 2
2003,34 Italy dermatology dermoscopy
ABCDE rule
(dermoscopy)
Thomas Department of ABCDE clinical 1140 NR NR ABCDE clinical rule
1998,6 France dermatology rule (cut-point >2)
Se: 89.3
Sp: 65.3
Unlu 2014,43 Department of ABCD rule of 115 N=115 3 ABCD rule of dermoscopy
Turkey dermatology dermoscopy 24 melanomas, 91 49% male Experienced Se: 91.6
7-point checklist non-melanoma Mean age: 39 dermatoscopists Sp: 60.4
for dermoscopy 7-point checklist for
3-point checklist of dermoscopy
dermoscopy Se: 79.1
CASH Sp: 62.6
dermoscopy 3-point checklist of
algorithm dermoscopy
Se: 87.5
Sp: 65.9
CASH dermoscopy algorithm
Se: 91.6
Sp: 64.8
Wadhawan Images from 7-point checklist 347 NR NR 7-point checklist for
2011,59 USA library of skin for dermoscopy dermoscopy
cancer Se: 87.3
Sp: 71.3
Walter 2013,44 Family practice 7-point checklist 1436 N=1182 NR 7-point checklist (clinical)
UK (clinical) 36 melanomas, 1400 35.9% male Se: 80.6
Revised 7-point non-melanoma Mean age: 44.7 Sp: 61.7
checklist (clinical) Revised 7-point checklist
(clinical)
Se: 91.7
Sp: 33.1
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Zalaudek Pigmented 3-point checklist 150 NR 150 3-point checklist for
2006,60 29 lesion clinic for dermoscopy 44 malignant, 106 Varying levels of dermoscopy
Countries non-melanoma experience Se: 94.0
Sp: 71.9
Impact Analysis Studies
Author year, Study design Participant Lesions Intervention Control Outcomes
Country selection
Westerhoff Controlled before 74 FPs n=100 (50 melanoma, 50 Educational intervention. FPs Usual care Correct diagnosis of
2000,62 and after non-melanoma) given educational material on melanoma, percent (SD):
Australia Selected randomly from Menzies 1996 rule, followed by Intervention 75.9 (12)
Primary care the Sydney Melanoma a 1-hour Control 54.8 (22)
Unit image database Presentation on surface Correct diagnosis of
microscopy non-melanoma, percent (SD):
Intervention 57.8 (14)
Walter 2012,63 RCT Control 55.8(15)
15 FP practices 1580 from 1297 patients Patients assessed using the Best practice (clinical Primary, appropriateness of
England MoleMate system (SIAscopy history, naked eye referral (defined as the
Primary care with primary care scoring examination, 7-checklist proportion of referred lesions
algorithm) clinical) that secondary care experts
decided to biopsy or monitor):
no statistically significant
difference between
intervention and control;
56.8% vs 64.5%; difference
-8.1% (95% CI -18.0% to 1.8%). Secondary:
Appropriate management of benign lesions in primary care: no statistically significant difference between intervention and control (99.6% vs 99.2%, p=0.46). Agreement with an expert decision to biopsy or monitor: no statistically significant difference between intervention and control (98.5% vs control 95.7%, p=0.26).
Validation studies
Author year, Patient: n, sex, CPR applied by: n Reported sensitivity/
country Setting CPR used Lesions mean age Experience specificity
Argenziano 2006,72 Spain, Italy
Primary Care
73 FPs
2548 lesions from 2522 patients presenting to primary care with a pigmented skin lesion. 1203 lesions in dermoscopy group (six melanoma)
1345 lesions in control group (six melanoma)
Use of dermoscopy in addition to 'naked-eye' lesion screening.
Both groups received a 4 hours educational intervention incorporating clinical examination and use of the 3-point checklist (dermoscopy algorithm)
Naked-eye screening alone.
Patient satisfaction: more intervention patients ranked their consultation very good/ excellent for thoroughness than control (83.1% vs 71.2%, p<0.001).
Patient anxiety: no statistically significant difference between intervention and control in anxiety scores (32.56 vs 34.72, p=0.013) Primary outcome: Referral accuracy of PCPs (defined as the ability of the PCP to correctly determine a lesion may be malignant or benign, when the gold standard is diagnosis by a second expert clinician) reported as sensitivity, specificity, PPV, NPV. Significant difference in sensitivity (dermoscopy 79.2%, naked eye 54.1%, p=0.002) and negative predictive value (dermoscopy 9801%, naked eye 95.8%, p=0.004)
Secondary outcome: Number of malignant tumours missed by PCPs using naked-eye examination (n=23) and using dermoscopy (n=6) (p=0.002)_
ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Color; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6mm) Diameter, Evolution of moles; AC, asymmetry, colour variation; CASH, colour, architecture, symmetry, and homogeneity; CPR, clinical prediction rules, ELM, epiluminescence microscopy; FP, family physicians; PCP, primary care physicians; PPV, positive predictive value; NPV, negative predictive value; NR, Not reported; RCT, randomised controlled trials; Se, sensitivity; Sp, specificity.
Table 3 Comparison of elements in clinical prediction rules for malignant melanoma
(a) Clinical rules
Clinical CPR name
Elements ABCD ABCDE 7-point checklist Revised 7-point checklist
Asymmetry X X X
Border irregularity X X X
Colour variegation X X X X
Diameter (>6 mm) X X X (>7 mm) X (>7 mm)
Evolving (eg, size, shape, colour) X X (size) X
Altered sensation X X
Inflammation X X
Crusting, bleeding X X
Cut-point >1 >1 or >2 >3 >3
(b) Dermoscopic rules
CPR name
Menzies
Revised Menzies 2008—
7-point 7-point Menzies 3-point 3-colour Kreusch Nilles 2008— skin Simplified
Element ABCD checklist checklist 1996 checklist 7FFM CASH ABCDE test 1992 1994 melanoma cancer DynaMel ABC AC rule
Asymmetry X X X X X X X X X X X X
Multiple colours (light/dark X X X X XX X X X X X X X
brown, black, red, white, blue)
Architectural disorder (structures X X X X X X X X
and colours)
Atypical network XX XX X X XX X
Blue-white veil XX X X X X
Blue-white structures X X
Streaks/radial streaming/ X X X X X X X X X X
pseudopods
Dots, globules XX XX XX X X X
Regression structures or X X X X X X X X
erythema
Scarring X X
Blotches (structure less region X X X
Atypical vascular pattern X X X X X X X
Recognisable as benign X
Abrupt cut-off border pigment X X X X
Blue-grey dots X
Change X X X X
Cut-point >4.75 >3 >1 >1,no >1 >2 >2 Not >3 Not Not >1 >0 (high >3 >4 Not
>5.45 features reported reported reported sensitivity) reported
>1 (high
specificity)
ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Colour; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater
than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6mm)Diameter, Evolution of moles; AC, asymmetry, colour
variation; CASH, color, architecture, symmetry, and homogeneity; CPR, clinical prediction rules; FFM, features for melanoma.
Table 4 Sensitivity and specificity of all clinical and dermoscopy CPRs
Rule name Cut-point Sensitivity* Specificity*
Clinical rules
ABCDE >1 Two studies One study
0.47-0.92 (mean 0.70) 0.56
>2 0.85 0.44
7-point checklist >3 Three studies Three studies
0.44-0.86 (mean 0.70) 0.62-0.94 (mean 0.74)
Revised 7-point checklist >3 0.92 0.33
ABCD rule >1 0.84 0.78
Dermoscopic rules
ABCD rule >4.75 Meta-analysis (eight studies) Meta-analysis (eight studies)
0.85 (95% CI 0.73 to 0.93) 0.72 (95% CI 0.65 to 0.78)
>5.45 Four studies Four studies
0.73-0.98 (mean 0.85) 0.46-0.91 (mean 0.79)
>4.2 0.88 0.64
7-point checklist >3 Meta-analysis (11 studies) Meta-analysis (11 studies)
0.77 (95% CI 0.61 to 0.88) 0.80 (95% CI 0.59 to 0.92)
Menzies 1996 for melanoma >1 Six studies Six studies
0.85-0.95 (mean 0.91) 0.38-0.78 (mean 0.69)
3-point checklist >1 Five studies Four studies
0.50-0.96 (mean 0.84) 0.31 -0.72 (mean 0.55)
Seven features for melanoma (7FFM) >2 Five studies Five studies
0.69-0.95 (mean 0.86) 0.74-0.86 (mean 0.82)
CASH algorithm >8 Three studies Three studies
0.41-0.92 (mean 0.73) 0.65-0.97 (mean 0.82)
The 3-colour test >3 Two studies Two studies
0.77-0.97 (mean 0.87) 0.55-0.90 (mean 0.73)
Revised 7-point checklist >1 0.88 0.28
Kreusch 1992 Not reported 0.99 0.94
Nilles 1994 Not reported 0.90 0.85
Menzies 2008 for melanoma >1 0.70 0.56
DynaMel algorithm >3 0.77 0.98
Menzies 2008 for skin cancer >0 (high sensitivity); 0.95 0.80
>1 (high specificity)
Simplified ABC-point list >4 0.90 0.87
AC rule Not reported 0.91 0.94
Emery 2010 SIAscopy >6 0.50 0.84
Guitera RCM 2012 Not reported 0.88 0.71
ABCDE rule Not reported Not reported Not reported
*Where sensitivity and specificity are presented for more than one study, the range and mean are presented. Where meta-analysis was possible, values from meta-analysis are presented with 95% CIs.
ABC, Asymmetry, irregular Borders, more than one or uneven distribution of Colour; ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; ABCDE, Asymmetry, irregular Borders, more than one or uneven distribution of Color, or a large (greater than 6mm) Diameter, Evolution of moles; AC, asymmetry, colour variation; CASH, color, architecture, symmetry, and homogeneity; CPR, clinical prediction rules; RCM, reflectance confocal microscopy.
checklist (cut-point >1) reported sensitivity (0.92) and specificity (0.33) (table 4).44
Dermoscopic CPRs for melanoma diagnosis
ABCD rule of dermoscopy
The ABCD rule of dermoscopy (also described as the ABCD rule of Stolz), was validated in 23 studies, 15 of which applied a cut-point of >4.75 (indicating a suspicious lesion) and six studies a cut-point of 5.45 (highly suggestive for melanoma). At a cut-point of >4.75, eight studies provided sufficient information for
42 43 47 52 65 71
meta-analysis, resulting in a pooled sensi-
tivity of 0.85 (95% CI 0.73 to 0.93) and specificity of
0.72 (95% CI 0.65 to 0.78) (figure 1A, B). This indicates that at this cut-point, the dermoscopy CPR is more useful for ruling out rather than ruling in melanoma, with a higher pooled sensitivity than specificity. I2 were high (>70%), indicating a high degree of heterogeneity. Of the seven studies excluded from meta-analysis, sensitivity ranged from 0.71 to 0.91 (mean 0.79) and specificity ranged from 0.43 to 0.92 (mean 0.72). None of the six studies that applied a cut-point of 5.45 were suitable for meta-analysis. From four studies that presented the information, sensitivity ranged from 0.73 to 0.98 (mean 0.85) and specificity ranged from 0.46 to 0.91 (mean 0.79) (table 4).
Seven-point checklist for dermoscopy
The 7-point checklist for dermoscopy was validated in 18 studies, 17 of which applied a cut-point of 3. 11 studies provided sufficient information for meta-analysis, revealing a pooled sensitivity of 0.77 (95% CI 0.61 to 0.88) and pooled specificity of 0.80 (95% CI 0.59 to 0.92) (See figure 2A, B).25-27 35 37 42 43 47 50 52 71 There was a high degree of heterogeneity in the results (I2>90%). Removing two outliers27 50 made minimal difference to the pooled result. Only one study validated the revised 7-point checklist for dermoscopy and reported sensitivity 0.78 and specificity 0.65 for a cut-point of 3 (table 4).27
Impact analysis
We identified three unique studies that examined the impact of a melanoma CPR on processes of care (melanoma diagnosis and referrals), however, no patient outcomes were examined (table 2).62 63 The methodological quality of these studies is presented in online supplementary appendix 4.
Using a controlled before-after design, Westerhoff et al investigated the impact of an educational intervention about the Menzies 1996 rule on melanoma diagnosis by family physicians (FP). The control group did not receive the training. Postintervention, there was a significant improvement in melanoma diagnosis (75.9% vs 62.7%, p<0.001). No significant improvement was seen in the control group (54.8% vs 53.7%, p=0.59).62
Walter et al conducted a RCT to compare the use of a new imaging device, the MoleMate system (SIAscopy with a primary care scoring algorithm), to current best
practice (clinical history, naked-eye examination, 7-point checklist). The authors found no difference between these two approaches in terms of appropriate referrals (the proportion of referred lesions that secondary care experts biopsied or monitored) to urgent skin cancer clinics (intervention 56.8% vs control 64.5% p=0.11) or the proportion of benign lesions appropriately managed in primary care (intervention 99.6% vs control 99.2%, p=0.46).63
Argenziano et als RCT, involved primary care physicians first attending a 1-day training course describing the ABCD rule (cut-point unspecified) and the 3-point checklist. They were then randomly assigned to assess patients with skin lesions, either by clinical (ie, naked eye) examination, or by dermoscopy using the 3-point checklist. The referral assessments were checked for accuracy by dermatologists. The dermoscopy arm demonstrated a 25% improvement in the sensitivity of primary care referrals of pigmented lesions compared with the naked-eye examination (79.2% vs 54.1%, p=0.002), without a reduction in specificity (71.8% vs 71.3%, p=0.915).72
DISCUSSION Summary of findings
This systematic review identified 48 studies validating a total of 24 CPRs for melanoma. Overall, the majority of validation studies used dermoscopic CPRs, with very few studies validating clinical CPRs. Meta-analysis of the der-moscopic CPRs demonstrated relatively high pooled estimates of sensitivity (0.77-0.86). The clinical implication is that applying dermoscopy CPRs will enable low-risk
Figure 1 (A) Diagnostic accuracy ABCD rule with dermoscopy—pooled sensitivity and specificity (eight studies). (B) Summary receiver operating characteristic curves for ABCD rule of dermoscopy The circles represent individual studies and the size reflects the sample size. The red square represents the summary estimates of sensitivity and specificity and the dotted ellipses around this represent the 95% CI around the estimate. The 95% prediction region (amount of variation between studies) was wide, suggesting heterogeneity between studies. ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; HSROC, hierarchical summary receiver operating characteristic.
Figure 2 (A) Diagnostic accuracy of 7-point checklist with dermoscopy—pooled sensitivity and specificity (11 studies). (B) Summary receiver operating characteristic curves for ABCD rule of dermoscopy The circles represent individual studies and the size reflects the sample size. The red square represents the summary estimates of sensitivity and specificity and the dotted ellipses around this represent the 95% CI around the estimate. The 95% prediction region (amount of variation between studies) was wide, suggesting heterogeneity between studies. ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; HSROC, hierarchical summary receiver operating characteristic; ROC, receiver operating characteristic.
patients to be observed and kept under review in a primary care setting, without immediate referral for excision to secondary care. Meta-analysis was not possible for clinical CPRs but individual studies report variable sensitivity, ranging from 0.44 to 0.86. Three impact analysis studies were identified, with two reporting an improvement in melanoma diagnosis with the use of a CPR.
Context of previous research
The sensitivities and specificities we report indicate that currently available CPRs are reasonably good at ruling out melanoma. The pooled sensitivity of the ABCD rule for dermoscopy (cut-point of >4.75) was 0.85 (95% CI 0.73 to 0.93), higher than that of the 7-point checklist for dermoscopy (0.77, 95% CI 0.61 to 0.88). While this evidence would support the use of such rules in prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients, there are a number of important caveats that may prevent their adoption in primary care.
Melanoma is a high-stakes condition, one which doctors tend to be cautious in diagnosing, often preferring to excise a benign lesion rather than to miss a potentially fatal cancer.73 In such cases, a CPR with near perfect sensitivity would be desirable, however, it has been argued that a lower sensitivity should not prevent CPR use unless usual decisions, made without the rule,
are demonstrably better.74 Our results are comparable with previous systematic reviews focused on melanoma diagnosis across healthcare settings in highlighting that dermoscopic CPRs are demonstrably better in terms of
diagnostic accuracy in comparison with inspection by the naked eye.16 75 However, even a rule with almost 100% sensitivity may not be adopted. For instance, implementation of the Canadian CT Head Rule, despite 100% sensitivity in validation studies, did not result in a reduction in imaging rates, with clinicians' reporting unease with certain components of the rule and fear of missing a high-stakes diagnosis as reasons for not adopting the CPR.76
Before considering whether to use a CPR in clinical practice, it is essential that its performance be established through external validation (ie, in settings outside where it was derived). We identified a number of external validation studies in this review, however, in keeping with much CPR research, the reporting of these studies was often poor.77 78 In particular, the common issues of limited acknowledgement and handling of missing data and key performance measures of prediction models, that is, calibration, being omitted was encountered.77 The lack of available data in some papers meant not all studies could be combined in the meta-analysis, meaning the sensitivities and specificities reported here are not based on the totality of existing evidence. Furthermore, we were unable to assess diagnostic accuracy at different cut-point thresholds for respective CPRs. Improved reporting of CPRs at cut-point thresholds will enable pooling of diagnostic accuracy data, and will provide more robust measures of diagnostic accuracy. After validation, impact analysis studies are undertaken to determine the impact of the implementation of a CPR on processes and outcomes of care. Despite increasing
interest in developing and validating CPRs relevant to primary care, relatively few have undergone impact analysis.79 Despite the large number of CPRs identified in this review, we identified only three impact analysis studies, with only two studies reporting an improvement in correct melanoma diagnosis in primary care as a result. Arguably, the dearth of well conducted and clearly reported external validation and impact analysis studies undermines trust in the use of such rules in practice.77
Current NICE guidelines for melanoma detection and management recommend dermoscopy of any suspicious lesion, advising against using computer-assisted diagnostic tools (NG14) while promoting use of the weighted 7-point checklist in primary care to guide referral (NG12).20 Based on the findings of this review, the ABCD rule for dermoscopy had a higher sensitivity than the seven point for dermoscopy checklist at their respective cut-points, indicating its potential for use in primary care. Dermoscopy, however, requires training and equipment, and is less commonly performed in primary care. Evidence suggests that dermatologists have better diagnostic accuracy than primary care physi-cians.18 Three studies retrieved in our search assessed dermoscopy CPR performance when applied by nonexperts, with two studies reporting that the CPRs performed well overall when used by non-experts, mainly primary care physicians.49 66 72 Westerhoff et al62 and Blum et al80 demonstrated that training primary care physicians to use dermoscopy with CPRs showed significant improvement in the diagnosis of melanoma compared with naked eye inspection. Alongside the use of CPRs, training in dermoscopy would seem to be a strategy that will enhance diagnostic accuracy of melanoma in the future particularly in light of emerging evidence of differences in dermoscopic features of melanoma such as head and neck melanoma.81 It has also been highlighted that significant efforts are needed to standardise and improve dermoscopic terminology to more broadly promote the use of dermoscopy in the primary care setting.82 Of the 24 rules identified in this review, four were clinical (ie, naked eye) and 17 were dermo-scopic. Owing to the limited number of studies and available data, no meta-analysis of clinical CPRs could be conducted. The range of reported sensitivities from individual studies indicates that there is insufficient evidence to recommend their use in practice.
Strengths and limitations of our study
The main strengths of this review are the use of broad inclusion criteria, the systematic search of multiple databases not limited by language, use of the CHARMS checklist to assess methodological quality, pooling data from a broad range of studies to enhance generalisabil-ity and the use of a broad definition of primary care to account for the variation in primary care services and access internationally. However, the findings of this systematic review need to be interpreted in the context of the limitations of the original studies. The lack of
available data in some papers meant not all studies could be combined in the meta-analysis. A number of studies that validated CPRs and algorithms using novel diagnostic technologies which incorporated computerised image analysis and artificial intelligence were excluded from the review as routine use of these are not currently recommended in UK NICE clinical guidelines. Significant heterogeneity existed between the studies with respect to differences in the study populations and application of the CPR. Finally, individual patient data that enables pooling of risk scores at the different cutpoints would enable researchers to explore the clinical use of applying risk scores at different cut-points with the purpose of assessing the role of melanoma CPRs at the different diagnostic thresholds of 'ruling out' (using highest pooled sensitivity) or 'ruling in' (using highest pooled specificity) of respective melanoma CPRs.
Implications for practice and future research
Early detection followed by curative surgery greatly improves the prognosis of malignant melanoma. As the incidence of melanoma skin cancer increases, primary care physicians are increasingly required to screen for melanoma.12 Therefore, efforts to increase the early detection of melanoma must focus on supporting primary care physicians in performing skin cancer screenings with recent evidence highlighting the benefits of developing targeted screening strategies in high-risk patients in primary care.18 83 This systematic review identified 24 separate clinical (naked eye) and dermoscopic CPRs, with some overlap in the included the elements. Our analysis highlights that dermoscopic CPRs have reasonable sensitivity, with the ABCD rule for dermoscopy having better sensitivity than the 7-point checklist for der-moscopy. Further development of new rules is unlikely to benefit the field of research. An increased emphasis on better reporting of validation studies, particularly at different cut-point thresholds, would allow for the conduct of more robust diagnostic accuracy meta-analysis to inform decision making. Further methodologically robust RCTs are necessary also to examine the impact of implementing CPRs in clinical practice, in terms of patient outcomes, physician behaviour, processes of care and cost-effectiveness. Finally, while guidelines promote the use of dermoscopy in the assessment of pigmented skin lesions, there needs to be greater emphasis on training in primary care on this examination technique.
CONCLUSION
This systematic review and meta-analysis shows that der-moscopic CPRs have reasonably high pooled estimates of sensitivity and may be a useful tool for primary care physicians prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients. The ABCD rule of dermoscopy has higher pooled sensitivity than the 7-point checklist for dermoscopy, when consideration about ruling out
melanoma is being made. A focus on impact analysis may help translate melanoma CPRs into useful and effective triage tools for use in primary care.
Contributors EH, NW and BC drafted the manuscript. EH, NWand BC contributed to development of the selection criteria, the risk of bias assessment strategy and the data extraction criteria. EH developed the search strategy. HB, LA and HS contributed the data extraction and quality assessments. BC and TF read, provided feedback and approved the final manuscript.
Funding This systematic review is funded by the HRB Centre for Primary Care Research under grant number HRC/2014/1, Royal College of Surgeons in Ireland (RCSI), Dublin, Ireland.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Open Access This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http:// creativecommons.org/licenses/by-nc/4.0/
REFERENCES
1. Erdmann F, Lortet-Tieulent J, Schuz J, etal. International trends in the incidence of malignant melanoma 1953-2008—are recent generations at higher or lower risk? IntJ Cancer 2013;132: 385-400.
2. Arnold M, Holterhues C, Hollestein LM, etal. Trends in incidence and predictions of cutaneous melanoma across Europe up to 2015. J Eur Acad Dermatol Venereol 2014;28:1170-8.
3. Coory M, Baade P, Aitken J, et al. Trends for in situ and invasive melanoma in Queensland, Australia, 1982-2002. Cancer Causes Control 2006;17:21-7.
4. Jemal A, Saraiya M, Patel P, et al. Recent trends in cutaneous melanoma incidence and death rates in the United States, 19922006. J Am Acad Dermatol 2011;65(Suppl 1):S17-25.e11-13.
5. de Vries E, Bray FI, Coebergh JW, et al. Changing epidemiology of malignant cutaneous melanoma in Europe 1953-1997: rising trends in incidence and mortality but recent stabilizations in western Europe and decreases in Scandinavia. IntJ Cancer 2003;107:119-26.
6. Thomas L, Tranchand P, Berard F, et al. Semiological value of ABCDE criteria in the diagnosis of cutaneous pigmented tumors. Dermatology (Basel) 1998;197:11-17.
7. Cornish D, Holterhues C, van de Poll-Franse LV, etal. A systematic review of health-related quality of life in cutaneous melanoma. Annals Oncol 2009;20(Suppl 6):vi51-8.
8. Guy GP, Machlin SR, Ekwueme DU, etal. Prevalence and costs of skin cancer treatment in the U.S., 2002-2006 and 2007-2011. Am J PrevMea 2015;48:183-7.
9. Gandini S, Sera F, Cattaruzza MS, etal. Meta-analysis of risk factors for cutaneous melanoma: II. Sun exposure. Eur J Cancer 2005;41:45-60.
10. Gandini S, Sera F, Cattaruzza MS, etal. Meta-analysis of risk factors for cutaneous melanoma: I. Common and atypical naevi. Eur J Cancer 2005;41:28-44.
11. Gandini S, Sera F, Cattaruzza MS, et al. Meta-analysis of risk factors for cutaneous melanoma: III. Family history, actinic damage and phenotypic factors. Eur J Cancer 2005;41:2040-59.
12. Koelink CJL, Kollen BJ, Groenhof F, etal. Skin lesions suspected of malignancy: an increasing burden on general practice. BMC Fam Pract 2014;15:29.
13. Pockney P, Primrose J, George S, etal. Recognition of skin malignancy by general practitioners: observational study using data from a population-based randomised controlled trial. Br J Cancer 2009;100:24-7.
14. Argenziano G, Soyer HP, Chimenti S, etal. Dermoscopy of pigmented skin lesions: results of a consensus meeting via the internet. J Am Acad Dermatol 2003;48:679-93.
15. Benelli C, Roscetti E, Pozzo VD, et al. The dermoscopic versus the clinical diagnosis of melanoma. Eur J Dermatol 1999;9:470-6.
Kittler H, Pehamberger H, Wolff K, etal. Diagnostic accuracy of
dermoscopy. Lancet Oncol 2002;3:159-65.
Piccolo D, Ferrari A, Peris K, et al. Dermoscopic diagnosis by a
trained clinician vs. a clinician with minimal dermoscopy training vs.
computer-aided diagnosis of 341 pigmented skin lesions:
a comparative study. Br J Dermatol 2002;147:481-6.
Chen SC, Pennie ML, Kolm P, etal. Diagnosing and managing
cutaneous pigmented lesions: primary care physicians versus
dermatologists. J Gen Intern Mea 2006;21:678-82.
Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for
systematic review and meta-analysis protocols (PRISMA-P) 2015
statement. Syst Rev 2015;4:1-9.
Macbeth F, Newton-Bishop J, O'Connell S, etal. Melanoma: summary of NICE guidance. BMJ 2015;351:h3708. doi: 10.1136/ bmj.h3708.
Moons KGM, de Groot JA, Bouwmeester W, etal. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Medicine 2014;11:e1001744. Higgins JPT, Green S eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. http://handbook.cochrane.org. Reitsma JB, Glas AS, Rutjes AW, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58:982-90. Macaskill P, Gatsonis C, Deeks JJ, etal. Analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0. The Cochrane Collaboration, 2010. http://methods.cochrane.org/ sdt/handbook-dta-reviews/
Annessi G, Bono R, Sampogna F, etal. Sensitivity, specificity, and diagnostic accuracy of three dermoscopic algorithmic methods in the diagnosis of doubtful melanocytic lesions. The importance of light brown structureless areas in differentiating atypical melanocytic nevi from thin melanomas. J Am Acad Dermatol 2007;56:759-67. Argenziano G, Fabbrocini G, Carli P, etal. Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions. Comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Arch Dermatol 1998;134:1563-70.
Argenziano G, Catricala C, Ardigo M, etal. Seven-point checklist of dermoscopy revisited. Br J Dermatol 2011;164:785-90. Benellii C, Roscetti E, Dal Pozzo V. The dermoscopic (7FFM) versus the clinical (ABCDE) diagnosis of small diameter melanoma. Eur J Dermatol 2000;10:282-7.
Carli P, Quercioli E, Sestini S, etal. Pattern analysis, not simplified algorithms, is the most reliable method for teaching dermoscopy for melanoma diagnosis to residents in dermatology. Br J Dermatol 2003;148:981-4.
Dal Pozzo V, Benelli C, Roscetti E. The seven features for melanoma: a new dermoscopic algorithm for the diagnosis of malignant melanoma. Eur J Dermatol 1999;9:303-8. Piccolo D, Crisman G, Schoinas S, et al. Computer-automated abcd versus dermatologists with different degrees of experience in dermoscopy. Eur J Dermatol 2014;24:477-81. Pizzichetta MA, Talamini R, Piccolo D, etal. Interobserver agreement of the dermoscopic diagnosis of 129 small melanocytic skin lesions. Tumori2002;88:234-8.
Soyer HP, Argenziano G, Zalaudek I, etal. Three-point checklist of
dermoscopy. A new screening method for early detection of
melanoma. Dermatology (Basel) 2004;208:27-31.
Strumia R, Montanari A. Low positive predictive value of ABCD-E
rule for dermatoscopy of small melanocytic naevi. Melanoma Res
2003;13:631-2.
Buhl T, Hansen-Hagge C, Korpas B, et al. Integrating static and dynamic features of melanoma: the DynaMel algorithm. J Am Acad Dermatol 2012;66:27-36.
Emery JD, Hunter J, Hall PN, etal. Accuracy of SIAscopy for pigmented skin lesions encountered in primary care: development and validation of a new diagnostic algorithm. BMC Dermatol 2010;10:9-9.
Haenssle HA, Korpas B, Hansen-Hagge C, etal. Seven-point checklist for dermatoscopy: performance during 10 years of prospective surveillance of patients at increased melanoma risk. J Am Acad Dermatol 2010;62:785-93.
Higgins EM, Hall P, Todd P, etal. The application of the seven-point check-list in the assessment of benign pigmented lesions. Clin Exp Dermatol 1992;17:313-5.
Kittler H, Seltenheim M, Dawid M, etal. Morphologic changes of pigmented skin lesions: a useful extension of the ABCD rule for dermatoscopy. J Am Acad Dermatol 1999;40:558-62.
40. Keefe M, Dick DC, Wakeel RA. A study of the value of the seven-point checklist in distinguishing benign pigmented lesions
from melanoma. Clin Exp Dermatol 1990;15:167-71. 64.
41. McGovern TW, Litaker MS. Clinical predictors of malignant pigmented lesions. A comparison of the Glasgow seven-point
checklist and the American Cancer Society's ABCDs of pigmented 65. lesions. J Dermatol Surg Oncol 1992;18:22-6.
42. Skvara H, Teban L, Fiebiger M, etal. Limitations of dermoscopy in the recognition of melanoma. Arch Dermatol 2005;141:155-60.
43. Unlu E, Akay BN, Erdem C. Comparison of dermatoscopic 66. diagnostic algorithms based on calculation: the ABCD rule of dermatoscopy, the seven-point checklist, the three-point checklist
and the CaSh algorithm in dermatoscopic evaluation of melanocytic 67. lesions. J Dermatoi 2014;41:598-603.
44. Walter FM, Prevost AT, Vasconcelos J, etal. Using the 7-point
checklist as a diagnostic aid for pigmented skin lesions in general 68.
practice: a diagnostic validation study. Br J Gen Pract 2013;63:345-53.
45. Osborne JE, Bourke JF, Graham-Brown RAC, etal. False negative
clinical diagnoses of malignant melanoma. Br J Dermatol 69.
1999;140:902-8.
46. Argenziano G, Longo C, Cameron A, etal. Blue-black rule: a simple dermoscopic clue to recognize pigmented nodular melanoma. 70. Br J Dermatol 2011;165:1251-5.
47. Blum A, Rassner G, Garbe C. Modified ABC-point list of
dermoscopy: a simplified and highly accurate dermoscopic algorithm 71.
for the diagnosis of cutaneous melanocytic lesions. J Am Acad
Dermatol 2003;48:672-8. 72.
48. Blum A, Luedtke H, Ellwanger U, etal. Digital image analysis for diagnosis of cutaneous melanoma. Development of a highly
effective computer algorithm based on analysis of 837 melanocytic 73. lesions. Br J Dermatol 2004;151:1029-38.
49. Dolianitis C, Kelly J, Wolfe R, etal. Comparative performance of 4 dermoscopic algorithms by nonexperts for the diagnosis of 74. melanocytic lesions. Arch Dermatol 2005;141:1008-14.
50. Gereli MC, Onsun N, Atilganoglu U, etal. Comparison of two dermoscopic techniques in the diagnosis of clinically atypical 75. pigmented skin lesions and melanoma: seven-point and three-point checklists. Int J Dermatol 2010;49:33-8.
51. Guitera P, Menzies SW, Longo C, etal. In vivo confocal microscopy 76. for diagnosis of melanoma and basal cell carcinoma using a 2-step method: analysis of 710 consecutive clinically equivocal cases.
Australas J Dermatol 2012;53:7-8. 77.
52. Henning JS, Stein JA, Yeung J, etal. CASH algorithm for dermoscopy revisited. Arch Dermatol 2008;144:554-5.
53. Lorentzen H, Weismann K, Kenet RO, etal. Comparison of dermatoscopic ABCD rule and risk stratification in the diagnosis of 78. malignant melanoma. Acta Derm Venereol 2000;80:122-6.
54. Luttrell MJ, McClenahan P, Hofmann-Wellenhof R, etal. Laypersons' sensitivity for melanoma identification is higher with dermoscopy images than clinical photographs. Br J Dermatol 79. 2012;167:1037-41.
55. MacKie RM, Fleming C, McMahon AD, et al. The use of the dermatoscope to identify early melanoma using the three-colour test. 80. Br J Dermatol 2002;146:481-4.
56. Menzies SW, Kreusch J, Byth K, et al. Dermoscopic evaluation of amelanotic and hypomelanotic melanoma. Arch Dermatol 81. 2008;144:1120-7.
57. Menzies SW, Moloney FJ, Byth K, etal. Dermoscopic evaluation of nodular melanoma. JAMA Dermatol 2013;149:699-709. 82.
58. Argenziano G, Soyer HP, Chimenti S, et al. Impact of dermoscopy on the clinical management of pigmented skin lesions. Clin Dermatol 2002;20:200-2.
59. Wadhawan T, Situ N, Rui H, etal. Implementation of the 7-point 83. checklist for melanoma detection on smart handheld devices. Conf
Proc IEEE Eng Med Biol Soc 2011;2011:3180-3.
60. Zalaudek I, Argenziano G, Soyer HP, etal. Three-point checklist of dermoscopy: an open internet study. BrJDermatol 2006;154:431-7. 84.
61. Walter FM, Prevost AT, Vasconcelos J, et al. The diagnostic accuracy of the 7-point checklist to assess pigmented skin lesions in primary care. Asia Pac J Clin Oncol 2012;8:311. 85.
62. Westerhoff K, McCarthy WH, Menzies SW. Increase in the sensitivity for melanoma diagnosis by primary care physicians using skin
surface microscopy. Br J Dermatol 2000;143:1016-20. 86.
63. Walter FM, Morris HC, Humphrys E, etal. Effect of adding a diagnostic aid to best practice to manage suspicious pigmented
lesions in primary care: randomised controlled trial. BMJ (Online) 2012;345:e4110.
Healsmith MF, Bourke JF, Osborne JE, etal. An evaluation of the revised seven-point checklist for the early diagnosis of cutaneous malignant melanoma. Br J Dermatol 1994;130:48-50. Rao BK, Marghoob AA, Stolz W, et al. Can early malignant melanoma be differentiated from atypical melanocytic nevi by in vivo techniques? Part I. Clinical and dermoscopic characteristics. Skin Res Technol 1997;3:8-14.
Binder M, Kittler H, Steiner A, etal. Reevaluation of the ABCD rule for epiluminescence microscopy. J Am Acad Dermatol 1999;40I):171-6.
Feldmann R, Fellenz C, Gschnait F. The ABCD rule in dermatoscopy: analysis of 500 melanocytic lesions. Hautarzt 1998;49:473-6.
Lorentzen H, Weismann K, Secher L, et al. The dermatoscopic
ABCD rule does not improve diagnostic accuracy of malignant
melanoma. Acta Derm Venereol 1999;79:469-72.
Nachbar F, Stolz W, Merkle T, et al. The ABCD rule of
dermatoscopy. High prospective value in the diagnosis of doubtful
melanocytic skin lesions. J Am Acad Dermatol 1994;30:551-9.
Stolz W, Riemann A, Cognetta AB, etal. ABCD rule of
dermatoscopy: a new practical method for early recognition of
malignant melanoma. Eur J Dermatol 1994;4:521-7.
Blum A, Clemens J, Argenziano G. Three-colour test in dermoscopy:
a re-evaluation. Br J Dermatol 2004;150:1040.
Argenziano G, Puig S, Zalaudek I, etal. Dermoscopy improves
accuracy of primary care physicians to triage lesions Suggestive of
Skin Cancer. J Clin Oncol 2006;24:1877-82.
Carli P, Nardini P, Crocetti E, et al. Frequency and characteristics of
melanomas missed at a pigmented lesion clinic: a registry-based
study. Melanoma Res 2004;14:403-7.
Reilly BM, Evans AT. Translating clinical research into clinical
practice: impact of using prediction rules to make decisions.
Ann Intern Mea 2006;144:201-9.
Rajpara SM, Botello AP, Townend J, et al. Systematic review of dermoscopy and digital dermoscopy/ artificial intelligence for the diagnosis of melanoma. Br J Dermatol 2009;161:591-604. Stiell IG, Clement CM, Grimshaw JM, et al. A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments. CMAJ 2010;182:1527-32. Collins GS, de Groot JA, Dutton S, et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol 2014;14:40.
Riley RD, Ensor J, Snell KIE, etal. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ 2016;353: i3140.
Wallace E, Uijen MJM, Clyne B, etal. Impact analysis studies of clinical prediction rules relevant to primary care: a systematic review. BMJ Open 2016;6.
Blum A, Clemens J, Argenziano G. Modified dermoscopic algorithm for the differentiation between melanocytic and nonmelanocytic skin tumors. J Cutan Med Surg 2006;10:73-8.
Cengiz FP, Cengiz AB, Emiroglu N, etal. Dermoscopic and clinical features of head and neck melanoma. An Bras Dermatol 2015;90:488-93.
Carrera C, Marchetti MA, Dusza SW, etal. Validity and reliability of dermoscopic criteria used to differentiate nevi from melanoma: a web-based international dermoscopy society study. JAMA Dermatol 2016;152:798-806.
Rat C, Grimault C, Quereux G, etal. Proposal for an annual skin examination by a general practitioner for patients at high risk for melanoma: a French cohort study. BMJ OPEN 2015;5: e007471.
Kreusch J, Rassner G, Trahn C, etal. Epiluminescent microscopy: a score of morphological features to identify malignant melanoma. Pigment Cell Res 1992;(Suppl 2):295-8.
Menzies SW, Ingvar C, Crotty KA, et al. Frequency and morphologic characteristics of invasive melanomas lacking specific surface microscopic features. Arch Dermatol 1996;132:1178-82. Nilles M, Boedeker RH, Schill WB. Surface microscopy of naevi and melanomas—clues to melanoma. Br J Dermatol 1994;130:349-55.
