CC BY-NC-ND
0ELSEVIER
Contents lists available at ScienceDirect
Schizophrenia Research: Cognition
journal homepage: http://www.schizrescognition.com/
Utility of the UCSD Performance-based Skills Assessment-Brief Japanese version: discriminative ability and relation to neurocognition
Chika Sumiyoshia'*, Manbu Takakib, Yuko Okahisa b, Thomas L. Pattersonc, Philip D. Harvey d, Tomiki Sumiyoshie
a Faculty of Human Development and Culture, Fukushima University, 1 Kanayagawa, Fukushima, Fukushima, Japan, 960-1296 b Department of Neuropsychiatry, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences, Okayama, Japan c Department of Psychiatry School of Medicine, University of California San Diego, La Jolla, USA d Department of Psychiatry and Behavioral Sciences, University of Miami Miller School of Medicine, Miami, USA e Department of Clinical Research Promotion, National Center Hospital, National Center of Neurology and Psychiatry, Tokyo, Japan
ARTICLE INFO
ABSTRACT
Article history:
Received 29 June 2014
Received in revised form 24 August 2014
Accepted 29 August 2014
Available online 12 October 2014
Keywords: Functional capacity Neurocognition
Receiver Operating Characteristic (ROC) curve
analysis Co-primary measure
MATRICS Cognitive Consensus Battery (MCCB)
The UCSD Performance-based Skills Assessment Brief (the UPSA-B) has been widely used for evaluating functional capacity in patients with schizophrenia. The utility of the battery in a wide range of cultural contexts has been of concern among developers. The current study investigated the validity of the Japanese version of the UPSA-B as a measure of functional capacity and as a co-primary for neurocognion. Sixty-four Japanese patients with schizophrenia and 83 healthy adults entered the study. The Japanese version of the UPSA-B (UPSA-B Japanese version) and the MATRICS Cognitive Consensus Battery Japanese version (MCCB Japanese version) were administered. Normal controls performed significantly better than patients, with large effect sizes for the Total and the subscale scores of the UPSA-B. Receiver Operating Characteristic (ROC) curve analysis revealed that the optimal cut-off point for the UPSA-B Total score was estimated at around 80. The UPSA-B Total score was significantly correlated with the MCCB Composite score and several domain scores, indicating the relationship between this co-primary measure and overall cognitive functioning in Japanese patients with schizophrenia. The results obtained here suggest that the UPSA-B Japanese version is an effective tool for evaluating disturbances of daily-living skills linked to cognitive functioning in schizophrenia, providing an identifiable cut-off point and relationships to neurocognition. Further research is warranted to evaluate the psychometrical properties and response to treatment of the Japanese version of the UPSA-B.
© 2014 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/3.0/).
1. Introduction
There has been a growing concern over functional outcome in patients with schizophrenia (Burns and Patrick, 2007) since its inclusion in DSM-III or later editions. Functional outcome refers to a wide range of real-world functioning including residential independence, employment, daily-living skills (e.g. financial management, telephone communication), or social activities (Harvey and Bellack, 2009). The role of cognitive deficits in impaired functional outcome has been well-conceptualized with the advent of the Measurement and Treatment Research to Improve in Schizophrenia Consensus Cognitive Battery (Nuechterlein and Green, 2006; Nuechterlein et al., 2008). Although the initial purpose of the MCCB was to provide a comprehensive battery sensitive to neurocognitive improvement by drug treatment, co-primary measures, predictive of real-world functioning, were also
* Corresponding author. Tel./fax: +81 24 548 8161. E-mail address: sumiyoshi@educ.fukushima-u.ac.jp (C. Sumiyoshi).
requested to accommodate the development of cognitive enhancers (Buchanan et al., 2011).
Performance-based batteries such as the UCSD Performance-based Skills Assessment-Brief (Mausbach et al., 2007) have been shown to provide a potential co-primary measure, satisfying 1) test-retest reliability, 2) a moderate practice effect, 3) a high completion rate, 4) a good correlation with neurocognitive performance, and 5) a discriminability for residential status and social involvement, such as work (Leifker et al., 2009, 2010; Mausbach etal., 2007, 2008, 2011; Olssonetal., 2012).
As the name suggests, tasks in the UPSA-B are role-played using props (e.g. money, an invoice, a letter, and a telephone etc.) to evaluate functional capacity (competence) in daily-living contexts (Mausbach et al., 2007,2011). The battery consists of two subscales: Finances (e.g. counting money, bill payment) and Communication (e.g. using a phone). They were extracted from the full version of the original UPSA (Patterson et al., 2001) based on factor analysis (Mausbach et al., 2007). Due to its conciseness (approximately 10-15 min) and effectiveness as a co-primary measure, the battery has been widely used in the US, and has been introduced in Europe (Sweden: Harvey et al., 2009a; Olsson et al.,
http://dx.doi.org/10.1016/j.scog.2014.08.002
2215-0013/© 2014 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).
Table 1
The UPSA-B studies in the US and other countries.
Country Study Partiripantsa N UPSA- Bb Samplec
Total Finances Communication
US Bowie et al., 2010 SCZ BD 161 130 69.9 88.5 Patients with schizophrenia or schizoaffective disorder who were community living with Ashkenazi Jewish backgrounds. Results were obtained from the analyses of subsamples
Green et al., 2011 SCZ 162 73.0 - Recruited at the Validation of Intermediate Measure (VIM) study under the MATRICS initiative
Harvey et al., in press SCZ BD 3445 4624 74.0 83.0 - Enrolled in the genomic study based on a Veterans Administration initiative (CSP#572)
Keefe et al., 2011 SCZ 323 70.0 - Participants were in a large multi-site trial assessing the comparative effects of antipsychotic treatment with lurasidone or risperidone
Leifker et al., 2009 SCZ 194 72.2 Older patients with schizophrenia enrolled in longitudinal study of the course of cognitive and functional status. They were recruited at Mt. Sinai School of Medicine or other hospitals
Leifker et al., 2010 SCZ (Mt Sinai) SCZ (UCSD) NC 238 116 109 68.7d 56.5 84.5 Part of data was from Leifker et al., 2009 Recruited from Board and Care facilities in San Diego who enrolled in Functional Adaptation Skills Training Healthy subjects recruited at a naturally occurring retirement community (NORC) in Manhattan
Mausbach et al., 2007 Independent living SCZ Non-independent living SCZ 99 335 72.5 54.5 Recruited at the UCSD Advanced Center for Interventions and Services Research. A subset of participants was part of the Functional Adaption Skills Training (FAST) study (Patterson et al., 2006)
Mausbach et al., 2008 Assisted-living, SCZ, Schizoaffective Community-living SCZ, Schizoaffective 163 73 53.9 54.7 Part of data was from the FAST study
China Mcintosh et al., 2011 SCZ (Chinese) BD (Chinese) Major depression (Chinese) NC (Chinese) 272 61 50 284 37.6 55.2 47.9 64.3 Inpatients treated at a municipal psychiatric hospital in Beijing
India Velligan et al., in press SCZ (Indian) 160 67.6 - Recruited from 6 different sites recommended by the MATRICS Scientific Advisory Board
US Harvey et al., 2011 SCZ (Atlanta Skyland Trail) SCZ (Atlanta VA Medical Center) 55 40 75.5 81.7 40.1 Enrolled in the Validation of Everyday Real-World Outcomes (VALERO) study (Harvey et al., 2011,2013)
Harvey et al., 2013 SCZ (UCSD Outpatients Psychiatric Services) 100 75.3
Burton et al., 2013 SCZ, Schizoaffective 183 40.0 36.4 Data from the three different services referred in the VALERO study (Harvey et al., 2011,2013)
Harvey et al., 2009b SCZ, Schizoaffective 236 68.8 " " Schizophrenia or schizoaffective participants in a longitudinal study of cognitive and functional status. Outpatients recruited from several sites in New York and its suburbs
Harvey et al., 2009a SCZ 244 37.4 31.5 Data from the same general data set in Harvey et al., 2009a,b
Mausbach et al., 2010 SCZ BD 116 89 75.6 88.7 39.5 36.1 45.7 43.0 Data from the same resource in Bowie et al., 2010
Mausbach et al., 2011 SCZ 367 77.5 41.1 36.4
Sweden Harvey et al., 2009a SCZ (Swedish) 146 68.9 38.7 30.9 Outpatients recruited at a country council-founded clinic at NU Health Care Hospital
Olsson et al., 2012 SCZ, Schizoaffective, Delusional (Swedish) 211 69.4 38.8 30.6 Participants were in the study of Clinical Long-term Investigation of Psychosis in Sweden (CLIPS)
Denmark Vesterager et al., 2012 First episode SCZ (Danish) 117 77.5 40.2 37.3 Participants were in the multi-site randomized clinical trial for cognitive remediation program (the NEUROCOM trial)
a SCZ: schizophrenia, BD: bipolar disorder, NC: normal controls. b MAX score: Total = 100, Finances = 50, Communication = 50. c Patients were mostly outpatients except for Mcintosh et al., 2011. d A base line score.
2012; Denmark: Vesterager et al., 2012) and Asia (China: Mcintosh et al., Although the UPSA-B has promise for assessing functional capacity,
2011; India: Velligan et al. in press) (Table 1). Also, the Japanese version some issues remain under consideration in developing its Japanese ver-(Sumiyoshi et al., 2011) is under standardization. sion. First, the normative performance on the Japanese version needs to
be clarified. To date, several studies have presented data of a normal population (Leifker et al., 2010; McIntosh et al., 2011). According to these reports, the achievement of the normal samples ranges from 60 to 85 (Table 1), which may be affected by age and educational attainment. The study with the US samples (Leifker et al., 2010) presented scores of elderly healthy people (the mean age = 68.0) with large effect sizes between schizophrenia patients (d = 0.90-1.58 across multiple sites). On the other hand, Chinese control subjects who had relatively low levels of average educational attainment (Mean = 8 years) elicited a considerably lower score (64.3) (McIntosh et al., 2011), while the disassociation from normal controls (d = 1.08) was the same degree as that in the US study (Leifker et al., 2010).
The normative performance differentiating between normal and clinical samples can also be discussed from the view point of independency of living. The initial development study of the UPSA-B (Mausbach et al., 2007), examined a cut-off point classifying patients with schizophrenia into independent- versus non-independent living groups. A score of around 60 was estimated as the optimal cut-off, suggesting that patients above that have the capacity to live independently as a part of the normal population. Given the rather wide range of scores of the achievement related to the standard, an optimal cut-off point, differentiating normal subjects from patients, needs to be determined for the Japanese version of the UPSA-B. Specifically, data from relatively younger (30-50 years old) normal samples are of concern, which has not been addressed in previous studies.
Second, the possible influence of the cultural or socio-economical backgrounds needs to be considered. Cross-cultural adaptability of functional outcome measures including the UPSA-B has been discussed among its developers (Gonzalez et al., 2013; Harvey and Velligan, 2011; Velligan et al., 2012). Ratings by experts in different countries including Europe, Russia, and Asia resulted in a relatively poorer cultural adaptability of the UPSA-B compared to the case in the US, due probably, to the difference in daily-living standards. Specifically, Mexico, India, and China presented the greatest challenges in adaptation (Velligan et al., 2012). On the other hand, the UPSA-B may be well accommodated in countries with relatively uniformly westernized living environments like Japan.
Third, the Japanese version of the UPSA-B is expected to serve as an effective co-primary measure for standard neurocognitive batteries such as the MCCB, as shown by previous studies (Green et al., 2011). Accordingly, the correlation with cognitive functioning needs to be evaluated as part of the development of the UPSA-B_J.
The purposes of the current study were to address those issues in showing the utility of the UPSA-B in Japan. First, the performance on the UPSA-B was compared between young or middle-aged normal controls and patients with schizophrenia. A cut-off point differentiating normal controls from patients was also determined. In addition, profiles of
the task performance were produced to see if the domain- and task-specific difficulties exist in both groups. Finally, the relation to neurocognitive functioning, as assessed by the MCCB-J, was investigated to confirm its validity as a co-primary measure.
2. Methods
2.1. Participants
Sixty-four Japanese patients with schizophrenia and 83 healthy adults entered the study. Demographic and clinical profiles of the participants are summarized in Table 2. Patients were outpatients treated in Okayama University Hospital, and public or private hospitals in Toyama Prefecture. Diagnosis was established based on the DSM-IV-TR criteria by experienced psychiatrists using a structured interview, reference to medical history, and all available information. Patients known to be abusing alcohol or illicit drugs, or those with epilepsy, brain damage, or neurologic disorders, were excluded from the study. Psychiatric symptoms were assessed on the Brief Psychiatric Rating Scale (BPRS), 18-item version (Overall and Gorham, 1962).
Normal controls were recruited at Okayama University. The majority of them were office employees working in Okayama Prefecture. Written informed consent was obtained from all participants. The study protocol was approved by ethics committees at the respective study sites.
22. Measures
The Japanese versions ofUPSA (Sumiyoshi et al., 2011) and MCCB were administered to all participants. The UPSA-B Japanese version was developed based on the international version of the UPSA-B, with some modifications to adjust for differences in everyday functional demands in Japan. It has been approved by developers after conducting two independent forward and back translations, reconciliation, and pilot testing on patients. The MCCB Japanese version has been shown to have good psychometric properties and validity (Kaneda et al., 2013).
Subscale scores of the two domains of the UPSA-B (i.e. Finances, Communication) were converted into the standard score ranging from 0 to 50, and thus the maximum of the Total score was 100 (Mausbach et al., 2007). Raw scores for the 10 subtests of the MCCB were converted to T-scores (mean = 50, SD = 10), out of which the seven domain scores were produced (Nuechterlein and Green, 2006). The T-score of each task corresponds to the domain score except for Speed of Processing (TMT, BACS SC, and Fluency) and Working Memory (LNS and WMS-SS), for which the composite scores were calculated by summing to the T-scores of tests included in those domains. The overall composite score
Table 2
Characteristics of participants.
NC SCZ Effect sizea t/F (df) P Interpretations
N (M/F) 83 (71/12) 64 (34/30) -
Age 34.6 (9.4)b 35.2 (11.2) - t = -0.13 (143) 0.72 NC = SCZ
Education 16.6 (l.l) 13.6 (2.4) - t = 10.12 (143) <0.000 NC > SCZ
Duration - 9.7 (8.1) - -
Drug (mg)c - 444.8 (492.0) - -
BPRS_Positive - 10.0 (5.9) - -
BPRS_Negative - 7.5 (3.4) - -
BPRS_Total - 36.5 (12.8) - -
MCCB Composite 510.5 (47.3) 376.3 (76.0) 2.1 F = 67.30 (1,140) <0.000 NC > SCZ
UPSA-B Total 82.1 (8.6) 69.5 (13.7) 1.1 t = 6.80 (145) <0.000 NC > SCZ
Finances 48.7 (3.3) 43.8 (7.8) 0.8 F = 46.24 (1,145) <0.000 NC > SCZ
Communication 33.4 (7.6) 25.7 (9.5) 0.9
NC: normal controls, SCZ: patients with schizophrenia. a Cohen's d for normal controls vs. patients. b Mean (SD). c CPZ equivalent.
was the sum of the seven domain scores (Kern et al., 2008; Nuechterlein and Green, 2006).
The formal Japanese version of the MCCB was not released at that moment, and thus, the T-scores of the Japanese version were produced based on the data obtained in a preliminary study for the development of the Japanese version; the normative group consisted of 85 healthy adults (mean age =40.0, SD = 11.2, range 19-65) and the mean and the standard deviation of this group served as the reference for the T-score conversion. Age-correction was applied according to the regression method employed in the standardization study in the US (Kern et al., 2008). Although both age- and gender-corrections are recommended in the MCCB Manual (Nuechterlein and Green, 2006), only the former was applied due to limitations in the reference group at that moment.
2.3. Statistical analyses
SPSS ver. 17.0 (SPSS Inc.) was used for all the analyses except for the estimation of effect sizes.
2.4. Group comparisons
Demographic variables (age and education) and the UPSA-B Total score were compared by t-test. Two-way analysis of variance (Two-way ANOVA) was conducted for the group comparisons for the UPSA-B subscales with Group (normal controls vs. patients) as a between-subject factor while Subscale (Finances vs. Communication) as a within-subject factor. The MCCB composite score was compared by one-way analysis of covariance (ANCOVA) controlling education. Effect sizes (Cohen's d) were calculated by dividing the mean difference between normal controls and patients by a pooled SDs from the two groups.
2.5. ROC curve analyses
Receiver Operating Characteristic (ROC) curves analyses were conducted for the UPSA-B Total and subscale scores. Every possible cutoff point was specified at a specific sensitivity and 1 — specificity. Sensitivity corresponds to the 'hit' rate indicating the correct classification of normal subjects as a normal sample. 1 — specificity and specificity representthe 'false alarm (FA)' and 'correct rejection (CR)' rates, respectively. The former refers to the rate of misclassifying patients into the normal sample, while the latter means the rate of correctly determining patients. As for the measure of sensitivity, the area under curve (AUC) with the 95% confidential interval (95% CI) and d' (d-prime) (Gescheider, 1985) were calculated; larger values suggest better sensitivity. Optimal cut-off points were determined for the UPSA-B Total score, at which the sum of sensitivity (% of hit) and specificity (% of CR) was maximized (Youden's J: Youden, 1950); (Mausbach etal., 2011).
2.6. Profiles for the UPSA-B
Profiles were created to show domain- or task-specific performance in each group. The mean scores were calculated for each task (MAX = 1 except for one task with MAX = 2 in the Finance part), and were plotted on a horizontal axis scaled with task numbers.
2.7. Correlation analyses
Simple correlations (Pearson's r) were calculated between the UPSA-B Total score and the MCCB overall composite and seven domain scores.
3. Results
3.1. Group comparisons
Table 2 presents the statistical results for demographic variables and the performance on the UPSA-B and the MCCB. Age did not differ between groups (t = — 0.36, df = 143, n.s.) while Education was significantly higher for normal controls (t = 10.12, df = 143, p < 0.01). The UPSA-B Total score (t = 6.80, df = 145, p < 0.01) and the MCCB composite score (F = 67.30, df = 1,140, p < 0.01) were significantly higher for normal controls than Main effects of Group and Subscales of the UPSA-B were significant without an interaction effect; normal controls performed better than patients (F =46.24, df = 1, 145, p < 0.01) on both Finances and Communication subscales, and the Finances score was higher than Communication in both groups (F = 474.18, df = 1,145, p < 0.01).
Overall, relatively large ESs were obtained (d > 0.8), suggesting that substantial differences existed between normal controls and patients in measures of both functional capacity (UPSA-B) and neurocognition (MCCB) (Table 2).
3.2. ROC curve analyses
Fig. 1 illustrates ROC curves for the UPSA-B Total score. AUC of 0.77 (95%CI: 0.70-0.85) was significantly greater (p <0.001) than that of no information (0.50). d' associated with this curve was estimated as 1.26.
The optimal cut-off point for the Total score (MAX = 100) was estimated as 77.8, at which sensitivity (hit rate) and 1 — specificity (FA rate) were 0.67 and 0.21, respectively. The results suggest that of all normal controls, 67% had the UPSA Total score of 77.8 or above, while of all patients, 89 ( = 100 — 21)% scored below this score.
3.3. Profiles of the UPSA-B
Fig. 2 presents profiles of UPSA-B scores for normal controls (A) and patients with schizophrenia (B). Normal controls performed almost perfectly on the Finances part ( Fig. 2A, the left section). Likewise, the patient group showed better performance on the Finances part (Fig. 2B, the left section) than the Communication part (Fig. 2B, the right part).
Both normal controls and patients tended to perform poorly on Task 13 (speaking on the phone with a name and an address given by the
<U tß
0.0 0.2 0.4 0.6 0.8 1.0
1-Specificity
Fig. 1. ROC curves for UPSA-B Total score.
Total score (MAX=100)
Cut-off=77.8
d '= 1.26
AUC=0.77
■ 95%CI: 0.70-0.85 i i i
Finances Communication
À A* i^i-jiTjpi^ \fA
NC 16 13 19
SCZ 13 19
Fig. 2. UPSA-B profiles for normal controls (A) and patients with schizophrenia (B). Task 13: Speaking to the phone a name and an address given by a tester. Task 16: Leaving a voice-mail message to a reschedule of an appointment. Task 19: Remembering preparations designated by a medical appointment.
tester), Task 16 (leaving a voice-mail message to reschedule an appointment), and Task 19 (remembering preparations designated by a medical appointment) (Fig. 2 A, B).
3.4. Correlations between the UPSA-B and the MCCB
Correlations between the MCCB measures and the UPSA-B Total score scores were presented in Table 3. On the whole, the patient group yielded relatively larger rs than the normal control group
Table 3
Correlations between the MCCB and the UPSA-B.
MCCB domains and tests
Overall compositea Speed of Processing Compositeb Trail Making Test BACS Symbol Coding Fluency Working Memory Compositec WMS-II Spatial Span Letter Number Sequencing Verbal Learning HVLT-R
Reasoning and Problem solving
NAB Maze Visual Learning
BVMT-R Social Cognition
MSCEIT Managing Emotion Attention/Vigilance CPT-IP
0.27* 0.15
0.06 -0.01 0.19 0.16 0.23*
0.35** 0.25
0.36**
0.36** 0.20 0.24 - 0.10 0.27*
NC: normal controls, SCZ: patients with schizophrenia.
* p < 0.05.
** p < 0.01. a The sum of the T scores for seven domains. b The sum of TMT, BACS SC, and Fluency. c The sum of WMSIIISS and LNS.
(Table 3). The MCCB Overall composite, Working Memory composite, and Verbal Learning in the patient group showed moderate correlations (>0.3) reaching a statistical significance. Attention/Vigilance was also significantly correlated with the UPSA-B Total score. in both groups, although the rs were modest (0.23-0.27). The MSCEIT had the weakest relationship to the UPSA-B Total score, not revealing substantial correlations (<0.1) in either group (Table 3).
4. Discussion
The principal findings of the current study are summarized as follows: (1) the Japanese version of the UPSA-B has good discriminating power for diagnostically different groups yielding a large effect size (d > 0.8) and optimal cut-off point (around 80), (2) the Finance subscale of the UPSA-B was markedly better performed than the Communication subscale in both healthy adults and patients, showing poor performances on certain tasks in the Communication subscale, and (3) Performance on the UPSA-B Japanese version reflected neurocognitive functioning especially in patients with schizophrenia.
The average score of the UPSA-B Total score of the normal sample obtained in the current study was above 80, consistent with Leifker et al. (2010). Given that a majority of the studies from the US and Europe have reported that the averages of patients fell under 80 (Table 1), and that the optimal cut-off point in the current study was 78.8, it seems to be feasible to set the discriminative point at around 80 (less than 3-4 failures out of 20 tasks). In other words, the point around here is suggested to be an endpoint for patients with schizophrenia to live and function well in the community.
The disassociation between patients and normal controls found here was also in accord with a study in China (Mcintosh et al., 2011 ), yielding relatively larger effect sizes (d > 0.8). Although the scores of Chinese samples were lower than other studies (Table 1), this is due to a wide range of educational attainment in the research participants who were selected, in order to show that the UPSA-B was sensitive to education and that patients with low levels of education would have poor performance because of both educational and illness related variables (Mcintosh et al., 2011).
in addition to the diagnostically critical point noted above, identification of 'functional milestones' (e.g. goal lines for achievement in residential independence or employment) (Mausbach et al., 2011) would be informative for patients undergoing their rehabilitation. Previous studies exploring those cut-offs have reported that a score of 60 or above was able to be used to determine independent- vs. assisted-living status (Mausbach et al., 2007), and a score of 80 or above was able to be used to predict employment status (> 20 h/week) ( Mausbach et al., 2011). it would be of importance to investigate whether a similar degree of achievement is required for those functional milestones in westernized Asian countries like Japan.
As was suggested by the average score of subscales (Table 2) and the profile analyses (Fig. 2), the performance on the UPSA-B may vary depending on the domains; ANOVA results indicated that the Finances subscale was considerably higher than that of the Communication sub-scale in both the normal and the patient groups. in fact, our normal controls nearly gained the maximum score of 50, suggesting that healthy adults are expected to be almost perfect in this domain. Previous studies have also reported that the average of the Finances domain exceeded 40 even in people with schizophrenia (Burton et al., 2013; Mausbach et al., 2007; Vesterager et al., 2012) (Table 1). Given these discrepant figures between the subscales, the Total score on the UPSA-B is preferable for the purposes of general classification of functioning on the part of people with schizophrenia. Also, our supplementary analyses showed that discriminative ability of single subscales alone (Finances: AUC =0.68, d' = 1.0; Communication: AUC =0.73, d' = 1.3) was poorer than that oftheTotal score (AUC =0.77, d' = 1.3).
The profile analysis showed similar trends between normal controls and patients for the Communication subscale. Both groups presented a relatively low score at memory-demanding tasks (Tasks 13,16, and 19; Fig. 2). Our preliminary analysis including university students (N = 30, mean age = 20.6) performed well on those tasks, suggesting that age-and/or disease-based degradation of memory may account for poor performance.
The UPSA-B Total score showed a significant and moderately good correlation with the MCCB Composite score in patients with schizophrenia (r > 30), indicating sensitivity of the Japanese version of the UPSA-B to overall cognitive functioning in a clinical population. Moreover, in the patient group, Working memory and Verbal learning, in particular, showed relatively larger correlations compared to other domains, consistent with a previous report by Burton et al. (2013). Interestingly, the MSCEIT was excluded from the MCCB composite score in their study, assuming that social cognition would substantially differ from neurocognition in terms of the relationship to everyday functioning. Our result seems to support their assumption, as revealed by the weakest correlations between the MSCEIT and the UPSA-B measure in both normal controls and patients (Table 3).
Several issues are worth discussing in order to enhance the utility of the UPSA-B Japanese version. First, essential psychometric properties, including test-retest reliability, practice effect, and potential sensitivity in response to treatment need to be examined. Although previous studies (Leifker et al., 2010; Velligan et al. in press) have reported the UPSA-B has good qualities in these aspects, these issues should be replicated with the Japanese version of the UPSA-B.
Second, stratified cut-off points according to types of functional recovery (e.g. residential status, employment) (Mausbach et al., 2011) or demographic variables (e.g. age) would be worth investigating. Regarding the latter, as noted before, healthy adults have been shown to perform well (> 80) on the UPSA-B (Leifker et al., 2010). Similarly, a study with the full version of the UPSA (Harvey et al., 2010) has reported that basic daily-living activities were relatively unaffected by age, based on data from healthy elderly population. Despite those facts, our preliminary data indicate that younger subjects (university students) performed better than middle-aged workers for the Communication subscale (students = 38.5, workers = 33.4). This suggests that the execution of this battery might be age-dependent to a certain degree. Besides, the normal achievement might become less demanding as a person gets older; elderly people may not be expected to be as equally efficient as younger people who are generally more productive in work or social activities. Whether the performance on the UPSA-B is affected by age or socio-economical requirements warrants further investigations. Results from such studies may indicate a need for stratified standard scores or cut-offs.
Finally, as noted in a previous study (Vesterager et al., 2012), updated versions with or without supplemental tasks/props would facilitate the application to subjects with a wide range of demographic backgrounds.
To conclude, our study has presented data which confirmed the utility of the Japanese version of the UPSA-B. The battery has a good discriminative validity in differentiating patients with schizophrenia from a healthy middle-aged population, consistent with previous studies using elderly samples. Further, the responsiveness to cognitive functioning in patients with schizophrenia adds support to the utility of the UPSA-B as a co-primary measure in westernized countries like Japan.
Role of Funding Source
This work was supported by Japan Society for the Promotion of Science (JSPS) Grant-in-Aid for Scientific Research (C) No. 22530691 to CS and No. 26461761 to TS, as well as Health and Labour Sciences Research Grants for Comprehensive Research on Disability, Health, and Welfare (H24-Seishin-Ippan-002 and H26-Seishin-Ippan-011) to TS.
Contributors
CS designed the study, analyzed data, and wrote the draft. TS organized the international collaborative team for this study and supervised it. CS, MT, and OS collected data and supported the interpretation. TS, TP, and PH revised the draft critically for important intellectual content. All authors contributed to manuscript writing.
Conflict of Interest
The authors report no potential conflicts of interest. Acknowledgements
The authors thank Nao Imai, Mayumi Yabe, Shimako Nishiyama, Yuko Mizukami, and Miki Katou for organizing the data, and Alan Milby for his editorial assistance.
References
Bowie, C.R., Depp, C., McGrath, J.A., Wolyniec, P., Mausbach, B.T., Thornquist, M.H., et al., 2010. Prediction of real-world functional disability in chronic mental disorders: a comparison of schizophrenia and bipolar disorder. Am. J. Psychiatry 167 (9), 1116-1124.
Buchanan, R.W., Keefe, R.S., Umbricht, D., Green, M.F., Laughren, T., Marder, S.R., 2011. The FDA-NIMH-MATRICS guidelines for clinical trial design of cognitive-enhancing drugs: what do we know 5 years later? Schizophr. Bull. 37 (6), 1209-1217. Burns, T., Patrick D., 2007. Social functioning as an outcome measure in schizophrenia
studies. Acta Psychiatr. Scand. 116 ( 6), 403-418. Burton, C.Z., Vella, L., Harvey, P.D., Patterson, T.L., Heaton, R.K., Twamley, E.W., 2013. Factor structure of the MATRICS Consensus Cognitive Battery (MCCB) in schizophrenia. Schizophr. Res. 146 (1-3), 244-248. Gescheider, G.A., 1985. Psychophsics, 2nd ed. Lawrence Erlbaum Associates, New Jersey. Gonzalez, J.M., Rubin, M., Fredrick, M.M., Velligan, D.I., 2013. A qualitative assessment of cross-cultural adaptation of intermediate measures for schizophrenia in multisite international studies. Psychiatry Res. 206 (2-3), 166-172. Green, M.F., Schooler, N.R., Kern, R.S., Frese, F.J., Granberry, W., Harvey, P.D., et al., 2011. Evaluation of functionally meaningful measures for clinical trials of cognition enhancement in schizophrenia. Am. J. Psychiatry 168 (4), 400-407. Harvey, P.D., Bellack A.S., 2009. Toward a terminology for functional recovery in schizophrenia: is functional remission a viable concept? Schizophr. Bull. 35 (2), 300-306. Harvey, P.D., Velligan, D.I., 2011. International assessment of functional skills in people
with schizophrenia. Innov. Clin. Neurosci. 8 (1), 15-18. Harvey, P.D., Helldin, L., Bowie, C.R., Heaton, RK., Olsson, A.K., Hjarthag, F., et al., 2009a. Performance-based measurement of functional disability in schizophrenia: a cross-national study in the United States and Sweden. Am. J. Psychiatry 166 (7), 821-827. Harvey, P.D., Keefe, R.S., Patterson, T.L., Heaton, R.K., Bowie, C.R., 2009b. Abbreviated neuropsychological assessment in schizophrenia: prediction of different aspects of outcome. J. Clin. Exp. Neuropsychol. 31 (4), 462-471. Harvey, P.D., Reichenberg, A., Bowie, C.R., Patterson, T.L., Heaton, RK, 2010. The course of neuropsychological performance and functional capacity in older patients with schizophrenia: influences of previous history of long-term institutional stay. Biol. Psychiatry 67 (10), 933-939. Harvey, P.D., Raykov, T., Twamley, E.W., Vella, L., Heaton, RK, Patterson, T.L., 2011. Validating the measurement of real-world functional outcomes: phase I results of the VALERO study. Am. J. Psychiatry 168 (11), 1195-1201. Harvey, P.D., Stone, L., Lowenstein, D., Czaja, S.J., Heaton, RK, Twamley, E.W., et al., 2013. The convergence between self-reports and observer ratings of financial skills and direct assessment of financial capabilities in patients with schizophrenia: more detail is not always better. Schizophr. Res. 147 (1), 86-90. Harvey, P.D., Siever, L.J., Huang, G.D., Muralidhar, S., Zhao, H., Miller, P., Aslan, M., Mane, S., McNamara, M., Gleason, T., Brophy, M., Przygodszki, R., O'Leary, T.J., Gaziano, M., Concato, J., in press. The genetics of functional disability in schizophrenia and bipolar illness: methods and initial results for VA cooperative study #572. Am. J. Med. Genet. B Neuropsychiatr. Genet. Kaneda, Y., Ohmori, T., Okahisa, Y., Sumiyoshi, T., Pu, S., Ueoka, Y., et al., 2013. Measurement and treatment research to improve cognition in schizophrenia consensus cognitive battery: validation of the Japanese version. Psychiatry Clin. Neurosci. 67 (3), 182-188.
Keefe, R.S., Fox, K.H., Harvey, P.D., Cucchiaro, J., Siu, C., Loebel, A., 2011. Characteristics of the MATRICS Consensus Cognitive Battery in a 29-site antipsychotic schizophrenia clinical trial. Schizophr. Res. 125 (2-3), 161-168. Kern, R.S., Nuechterlein, K.H., Green, M.F., Baade, L.E., Fenton, W.S., Gold, J.M., et al., 2008. The MATRICS Consensus Cognitive Battery, part 2: co-norming and standardization. Am. J. Psychiatry 165 (2), 214-220. Leifker, F.R., Bowie, C.R., Harvey, P.D., 2009. Determinants of everyday outcomes in schizophrenia: the influences of cognitive impairment, functional capacity, and symptoms. Schizophr. Res. 115 (1), 82-87. Leifker, F.R., Patterson, T.L., Bowie, C.R., Mausbach, B.T., Harvey, P.D., 2010. Psychometric properties of performance-based measurements of functional capacity: test-retest
reliability, practice effects, and potential sensitivity to change. Schizophr. Res. 119 (1-3), 246-252.
Mausbach, B.T., Harvey, P.D., Goldman, S.R., Jeste, D.V., Patterson, T.L., 2007. Development of a brief scale of everyday functioning in persons with serious mental illness. Schizophr. Bull. 33 (6), 1364-1372. Mausbach, B.T., Depp, CA, Cardenas, V., Jeste, D.V., Patterson, T.L., 2008. Relationship between functional capacity and community responsibility in patients with schizophrenia: differences between independent and assisted living settings. Community Ment Health J. 44 (5), 385-391. Mausbach, B.T., Harvey, P.D., Pulver, A.E., Depp, CA, Wolyniec, P.S., Thornquist, M.H., et al.,
2010. Relationship of the Brief UCSD Performance-based Skills Assessment (UPSA-B) to multiple indicators of functioning in people with schizophrenia and bipolar disorder. Bipolar Disord. 12 (1), 45-55.
Mausbach, B.T., Depp, C.A., Bowie, C.R., Harvey, P.D., McGrath, J.A., Thronquist, M.H., et al.,
2011. Sensitivity and specificity of the UCSD Performance-based Skills Assessment (UPSA-B) for identifying functional milestones in schizophrenia. Schizophr. Res. 132 (2-3), 165-170.
Mcintosh, B.J., Zhang, X.Y., Kosten, T., Tan, S.P., Xiu, M.H., Rakofsky, J., et al., 2011. Performance-based assessment of functional skills in severe mental illness: results of a large-scale study in China. J. Psychiatr. Res. 45 (8), 1089-1094. Nuechterlein, K.H., Green, M.F., 2006. MATRICS Consensus Cognitive Battery Manual.
MATRICS Assessment Inc., Los Angeles. Nuechterlein, K.H., Green, M.F., Kern, R.S., Baade, L.E., Barch, D.M., Cohen, J.D., et al., 2008. The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am. J. Psychiatry 165 (2), 203-213.
Olsson, A.K., Helldin, L., Hjarthag, F., Norlander, T., 2012. Psychometric properties of a performance-based measurement of functional capacity, the UCSD Performance-based Skills Assessment — Brief Version. Psychiatry Res. 197 (3), 290-294.
Overall, J.E., Gorham, D.R., 1962. The brief psychiatric rating scale. Psychol. Rep. 10, 799-812.
Patterson, T.L., Goldman, S., McKibbin, C.L., Hughs, T., Jeste, D.V., 2001. UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr. Bull. 27 ( 2), 235-245.
Patterson, T.L., Mausbach, B.T., McKibbin, C., Goldman, S., Bucardo, J., Jeste, D.V., 2006. Functional adaptation skills training (FAST): a randomized trial of a psychosocial intervention for middle-aged and older patients with chronic psychotic disorders. Schizophr. Res. 86 (1-3), 291-299.
Sumiyoshi, T., Sumiyoshi, C., Hemmi, C., 2011. UCSD Performance-Based Skills Assessment-Brief (UPSA-B): Japanese version; administration and scoring manual.
Velligan, D.I., Rubin, M., Fredrick M.M., Mintz, J., Nuechterlein, K.H., Schooler, N.R., et al., 2012. The cultural adaptability of intermediate measures of functional outcome in schizophrenia. Schizophr. Bull. 38 (3), 630-641.
Velligan, D.I., Fredrick M., Mintz, J., Li, X., Rubin, M., Dube, S., Deshpande, S.N., Trivedi, J.K., Gautam, S., Avasthi, A., Kern, R.S., Marder, S.R., in press. The reliability and validity of the MATRICS Functional Assessment Battery. Schizophr. Bull.
Vesterager, L., Christensen, T.O., Olsen, B.B., Krarup, G., Melau, M., Forchhammer, H.B., et al., 2012. Cognitive and clinical predictors of functional capacity in patients with first episode schizophrenia. Schizophr. Res. 141 (2-3), 251-256.
Youden, W.J., 1950. Index for rating diagnostic tests. Cancer 3 (1), 32-35.
