Blended internet care for patients with severe mental illnesses: An open label prospective controlled cohort pilot study Academic research paper on "Clinical medicine"

ELSEVIER

Contents lists available at ScienceDirect

Internet Interventions

journal homepage: www.elsevier.com/locate/invent

Internet Interventions

Blended internet care for patients with severe mental illnesses: An open label prospective controlled cohort pilot study

Matthijs Blankers a,b,c,*I Arnold van Emmerikd, Bastian Richters a, Jack Dekkera,e

a Arkin Mental Health Care, Amsterdam, The Netherlands b Trimbos institute, Utrecht, The Netherlands

c Department of Psychiatry, Academic Medical Centre, Amsterdam, The Netherlands d Department of Clinical Psychology, University of Amsterdam, Amsterdam, The Netherlands e Department ofClinical Psychology, VU University, Amsterdam, The Netherlands

ARTICLE INFO

ABSTRACT

Article history:

Received 29 July 2015

Received in revised form 26 July 2016

Accepted 26 July 2016

Available online 28 July 2016

Keywords:

Internet interventions Severe mental illness FACT

Introduction: This paper reports first experiences while providing blended (combined face-to-face and internet-based) flexible assertive community treatment (FACT) to outpatients with severe mental illnesses (SMI). The aim was to compare treatment satisfaction, clinical outcome and quality of life in the short term (3 months) of patients receiving blended FACT with those receiving conventional FACT.

Method: This pilot study was designed as an open label prospective controlled cohort study. 47 SMI patients were found eligible and non-randomly allocated to Blended FACT (n = 25) or to conventional FACT (n = 22). Data were collected at baseline and at a 3-month follow-up. Measures included were the Dutch Mental Health Care Thermometer, Health of the Nation Outcome Scales (HONOS), Manchester Short Assessment of Quality of Life (MANSA), EuroQoL 5 dimensional (EQ5D) and the Mental Health Confidence Scale (MHCS). Results: At a three months follow-up, patients reported slightly improved quality of life (EuroQoL 5 dimensional, Wald x2(1) = 6.80, p = 0.01; MANSA, Wald x2(1) = 4.02, p = 0.05)andself-efficacy beliefs regarding their mental health problems (MHCS, Wald x2(1) = 3.71, p = 0.05). HONOS scores did not change over time, Wald X2(1) = 2.34, p = 0.13. Satisfaction scores were on average between satisfactory - good (BI: M = 7.50, SD = 1.54; CAU: M = 7.53 SD = 0.96; on a 1-10 scale). These results did not differ between the two study groups. Conclusion: It appears acceptable to patients to provide blended FACT with SMI, with outcomes comparable to face-to-face FACT. A future high quality trial is warranted to establish (cost-)effectiveness of blended FACT.

© 2016 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Therapist-led internet-based interventions have been found efficacious for a variety of mental disorders, including depression (van't Hof et al., 2009), anxiety disorders (Cuijpers et al., 2009), alcohol misuse (Riper et al., 2014) and tobacco smoking (Civljak et al., 2013). In addition, patients are generally positive about internet-based interventions (after use, not before, Musiat et al., 2014), as these interventions empower them to manage their own health problems (Barlow et al.,

Abbreviations: BI, Blended intervention; CAU, Care as usual; EQ5D, 5-dimensional EuroQol; FACT, Flexible assertive community treatment; GEE, Generalized Estimating Equations; HONOS, Health of the Nation Outcome Scales; MANSA, Manchester Short Assessment of Quality of Life; MHCS, Mental Health Confidence Scale; SMI, Severe mental illnesses.

* Corresponding author at. Arkin Mental Health Care, Department of Research, PO Box 75848,1070 AV Amsterdam, The Netherlands.

E-mail addresses: matthijs.blankers@arkin.nl (M. Blankers), a.a.p.vanemmerik@uva.nl (A. van Emmerik), bastian.richters@mentrum.nl (B. Richters), jack.dekker@arkin.nl (J. Dekker).

2005; Chou et al., 2012). The number of studies on internet-based interventions for patients with severe or complex mental illnesses is relatively small, but the number is increasing in recent years. Four recent reviews found support for the use of e-mental health in patients with psychotic disorders (Alvarez-Jimenez et al., 2014; Kasckow et al., 2013; van der Krieke et al., 2014; Naslund et al., 2015). Alvarez-Jimenez et al. (2014) included 12 studies examining the usability, acceptability, feasibility, safety or efficacy of internet interventions for patients with schizophrenia spectrum disorders and conclude that these interventions seem to be acceptable and feasible and have the potential to improve clinical and social outcomes. Kasckow et al. (2013) included 18 studies focussing on telepsychiatry interventions for patients with schizophrenia; they concluded that initial results suggest that these intervention modalities may improve patient outcomes but that more high quality research is needed. van der Krieke et al. (2014) draw the same conclusion after reviewing 28 studies on e-mental health self-management interventions for persons with psychotic disorders. Naslund et al. (2015) review contains 46 studies from 12 countries on ehealth or mhealth interventions for patients with severe mental illnesses (SMI); their results support the feasibility and acceptability of

http: //dx.doi.org/10.1016/j.mvent2016.07.004

2214-7829/© 2016 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/40/).

these interventions but they cannot draw conclusions regarding effectiveness due to insufficient high quality studies.

SMI patients tend to report combined psychiatric, somatic and social problems and therefore require complex multidisciplinary care. The majority of them (75%) has a diagnosis of schizophrenia or psychotic disorder. Many SMI patients also report residual symptoms, comorbid alcohol- or drug use and somatic health problems, as well as a need for support in self-care, accommodation, daytime activities and social contacts. For this reason, patients require tailored rehabilitation strategies to have a lasting effect on clinical outcomes (Drukker et al., 2010).

Assertive Community Treatment (ACT) was introduced with this aim by Stein, Test and Marx (Stein and Test, 1978) in the United States, to focus on outreach and persistent and intensive care and treatment for the most severely ill 20% SMI patients. More recently, a Dutch version of ACT (Flexible ACT, or FACT) has been developed and its use has become widespread. In FACT beneficial innovations at the team level, in the daily operation of FACT teams, and in the organisation of community mental health care services for SMI patients have been integrated. A multidisciplinary FACT team of 11-12 employees monitors 200 clients: all individuals with SMI in a catchment area (the 20% for whom ACT is indicated and the other 80%, who require less intensive care). To combine care for these two groups, the FACT team uses a flexible switching system. Patients who require intensive care are is discussed daily by the team; they use a shared caseload approach to be able to provide care at any time or day. For clients who require less intensive care, the team provides individual case management with multidisciplinary treatment and support (van Veldhuizen and Bahler, 2013). In current practise, the management of the 80% who require less intensive care involves frequent home visits by the team to monitor the patient and to help him/her with daily tasks and activities.

For this type of support, a pilot has been started to evaluate whether these activities could be performed using tele-psychiatry. Tele-psychiatry was implemented using an internet platform to support patients with everyday activities using educative videos, information, and an activities board, and by using (video-)chat contacts to amend the home visits of the FACT team. This paper reports on this evaluation of the acceptability of offering (flexible) assertive community treatment (FACT) in a blended (combined internet-based and face-to-face) intervention format (BI) to outpatients with severe mental illnesses (SMI) in Amsterdam, the Netherlands. The web-based care in BI is essentially a tele-psychiatry modality involving live video or chat interactions with psychiatric nurses. In addition, patients were offered access to psycho-educative videos and a leisure activities bulletin board. The comparison group received care as usual (CAU) in the form of face-to-face FACT.

In this pilot phase, the aim was to offer the BI intervention to a selection of patients, and to compare their treatment satisfaction, clinical outcome and quality of life in the short term with patients receiving conventional FACT care. As the amount of care was according to the treatment protocols not different for BI and CAU patients, it was expected that (1) BI and CAU would not lead to significant differences in improvements of psychiatric symptoms or quality of life. It was also expected that (2) the blended form of FACT was acceptable for the patients, as evidenced by comparable scores on treatment satisfaction for the two care modalities. It was however expected that (3) self-efficacy beliefs regarding mental health problems would improve more in BI than in CAU, due to the emphasis on self-management in BI.

2. Material and methods

2.1. inclusion of participants

A convenience sample of fifty-six patients receiving FACT at Mentrum, an SMI treatment centre in Amsterdam, the Netherlands were screened for eligibility. Patients received FACT for (combinations of) psychotic, mood, anxiety, and personality disorders. Inclusion criteria were age > 18 years, permanent housing, and fluency in Dutch.

Exclusion criterion was an unstable psychiatric condition which required hospitalisation. Of the 56 screened patients, 47 were found eligible and non-randomly allocated to BI (n = 25) or to CAU (n = 22). Recruitment, treatment, and data collection took place between November 2012 and April 2013.

22. Interventions

Both BI and CAU were FACT interventions, which is a low intensity, flexible adaptation of ACT (van Veldhuizen and Bahler, 2013). In FACT, teams coordinate treatment following assertive outreach principles to reduce in-patient care admissions and to prevent episodes without care (Drukker et al., 2013; van Veldhuizen and Bahler, 2013). The CAU group received conventional FACT through home visits and other contacts (for example in the community support centre) with a psychiatric nurse. In the BI group, a computer, internet connection and webcam were installed at no costs in participants' homes. They were granted access to the "myMentrum" internet portal, which offered (psycho-)edu-cative videos, a leisure activities bulletin board, an agenda for scheduling appointments with the psychiatric nurse, and a web forum to establish contact with other patients. This internet portal was developed using a platform for the development of guided internet interventions (called mijnTherapie). The content of the platform was selected by employees of the FACT team working for the mental health care organisation that initiated this trial project (Arkin mental health care). In addition to the "myMentrum" internet portal, Skype was installed and made available as well, enabling patients to communicate with the psychiatric nurses using three different channels: face-to-face, as patients were used to, video-chat, and by using text-chat. Video chat contacts and face-to-face contacts were scheduled according to the patients' needs, on average 2-3 times a week. In addition, patients could instantly contact psychiatric nurses during office hours (between 9 am and 5 pm) using text chat functionalities in Skype. Although Skype uses encryption, for security reasons Skype communication was kept separate from the secured patient health record which adheres to the Netherlands norms for health information security (NEN 7510). Patients were trained how to use the platform and Skype, and informed on security aspects of its use.

23. Procedure

This pilot study was designed as an open label prospective controlled cohort study. BI patients were self-referred from ongoing FACT care - BI care and participation in the study was open to all patients in the participating FACT team. In an introduction and recruitment meeting, BI participants were informed about the study procedures and the myMentrum patient portal. CAU participants were recruited from the remaining FACT patients of the same team (those who were not interested in participating in BI) through information leaflets. Data collection took place at baseline and three months after. It consisted of a face-to-face administered questionnaire (which took 20-30 min to complete and which was similar for the BI and CAU condition) and a clinician-administered clinical outcome measure, i.e., the Health of the Nation Outcome Scales (HONOS) (Wing et al., 1998). After filling out the follow-up questionnaire, BI participants were asked to report their experiences (positive and negative) with the BI platform. All participants provided written informed consent and the study procedures were approved by the ethics committee of the University of Amsterdam, registration number 2012-EXT-2441.

2.4. Measures

Clinical outcome was measured using the clinician-administered HONOS, a 12-item instrument that covers clinical problems and social functioning with reasonable adequacy (Wing et al., 1998; Mulder et al., 2004). Higher scores on the HONOS are indicative of worse

(mental) health. Quality of life was measured using the 16-item Manchester Short Assessment of Quality of Life (MANSA) (Priebe et al., 1999) which assesses e.g. number and quality of friendships, leisure activities, and mental health, and the 5-dimensional EuroQol (EQ5D), a 5-item generic measure of health status (e.g. self-care, pain) (EuroQol Group, 1990). For both the MANSA and the EQ5D, higher scores indicate better quality of life. Patient empowerment was measured using the Mental Health Confidence Scale (MHCS). The MHCS reliably assesses self-efficacy beliefs and confidence of patients with mental disorders (Carpinello et al., 2000). Higher scores on the MHCS indicate more self-efficacy beliefs and confidence. Treatment satisfaction was measured using the 11-point rating scale (mark between 0 (lowest) - 10 (highest)) of the Dutch Mental Health Care Thermometer (Kerzman et al., 2003). The estimated number of Skype contacts per month between psychiatric nurses and patients was obtained from care registration forms. Patients in the BI condition were asked to report their subjective experiences with regard to the platform at the end of the interview. In order to guide them to report their experiences, they were asked what they liked most and least about the BI platform and Skype functionality. For all instruments, validated Dutch versions were used.

3. Analyses

All analyses were performed using R 3.0.3 (R Core Team, 2014). Group differences at baseline were tested using t-tests or x2 tests, as appropriate. Analyses of clinical outcome and quality of life were longitudinally modelled using Generalized Estimating Equations (GEE) - hence population-averaged effects were estimated. An identity link function was chosen, assuming Gaussian (normal) data distributions. In addition, an independent working correlation structure was chosen (this is the default setting in the R package geepack which was used for the GEE analyses). Treatment satisfaction was measured at follow-up only and modelled using linear regression. Baseline characteristics of the participants (age, sex, duration of previous treatment, origin of the patient and his or her parents (Netherlands / all other countries), and employment status were included in the models as covariates to level out potential impact of their inter-group variance on the outcome estimations. To assess the influence of missing data, all analyses were repeated after missing data was addressed by single imputation using Amelia 2, assuming missingness at random (Honaker et al., 2011). The current study was found to be sufficiently powered to detect large effect sizes only (Cohen's d > 0.7, assuming a = 0.05 and power (1-(3) = 0.8) based on post-hoc power analysis.

4. Results

4.1. Participants

Patients in the BI condition were on average 44.4 (SD = 9.5) years of age and had received treatment for on average 6.2 years (SD = 4.3). Forty-four percent (11/25) were males, and 96% (24/25) were unemployed. Patients in the CAU condition were on average 48.9 (SD = 10.2) years of age and had received treatment for on average 7.2 years (SD = 4.2). Fifty percent (11/22) were males, and 91% (20/22) were unemployed. The most common primary diagnoses on Axis I were Schizoaffective Disorder (11 patients, 23%) followed by Schizophrenia Paranoid Type (8 patients, 17%). The most common primary diagnoses on Axis II was Borderline personality disorder (5 patients, 11%); most patients (62%) were not diagnosed with a personality disorder. None of the patient characteristics or primary diagnoses differed significantly between the two conditions. (See Table 1.)

The measurement completion rate after three months was 88% for the BI group (22/25) and 73% for the CAU group (16/22), with no significant difference in dropout rates, x2(1) = 0.91, p = 0.34. Of the three patients who dropped out of the BI intervention, one refused to

participate at times of the follow-up, and two were unreachable by phone upon repeated attempts. Of the six patients who dropped out of the BI intervention, one refused to participate at times of the follow-up, one was unreachable due to acute psychiatric hospitalisation, and 4 were unreachable by phone upon repeated attempts. Participants that dropped out were more often males (39.5% vs. 77.8%, x2(1) = 4.29, p = 0.038) and had lower MANSA baseline scores (M = 59.7 (SD = 13.7) vs. M = 47.0 (SD = 14.0) t(43) = 2.47, p = 0.018) than patients that completed the follow-up assessment.

42. Baseline scores

Table 2 presents the means and standard deviations of the HONOS, MANSA, EQ5D utilities (calculated using the UK tariff for preference-based utilities (Dolan, 1997) and the MHCS. HONOS baseline severity scores showed that patients met criteria for outpatient treatment or community care (Nugter et al., 2012). EQ5D baseline scores were approximately one standard deviation below norm scores in the general population for the dominant age group of the patient population (M = 0.85, SD = 0.25, see Kindetal., 1999), indicating that their quality of life was notably affected by their condition.

43. Outcome effects

GEE analyses indicated a main effect of time for the MANSA (Wald = 4.02, p = 0.05), the EQ5D (Wald = 6.80, p = 0.01), and for the MHCS (Wald = 3.71, p = 0.05), indicating that scores improved over time (Table 3). This was however not found for the HONOS (Wald = 2.34, p = 0.13). No significant time by condition interaction was found for the HONOS, MANSA, or EQ5D. This indicates that none of the changes over time differed between the two intervention conditions. Contrary to what was hypothesized, no time by condition interaction was found for the MHCS. Hence, improvements in self-efficacy beliefs over time did not differ between the BI and CAU interventions. Repeating the analyses after missing data were imputed yielded comparable results, although the time by condition interaction for the HONOS was now statistically significant (p = 0.01) and a significant main effect of time was found for the MANSA (p = 0.04), with higher scores over time.

4.4. Patient experiences and adverse events

Unstructured patient reports revealed that most patients were satisfied with the possibilities of the platform. In particular, some mentioned that they liked the possibility to use Skype to establish contact with the therapists and with other patients, friends or relatives, indicating that they used Skype also for other reasons than contact with the nurses. A few even reported having re-established contact with foreign family members with whom they had lost contact years ago. There was a large variation in the extent to which the BI patients made use of the myMentrum platform, Skype, and the internet in general. Based on care registration forms, patients had on average 5.4 Skype contacts per

Table 1

Baseline characteristics.

Measure BI (n = 25) M (SD) / n (%) CAU (n = 22) M (SD) / n (%) t / x2 P

Sex (male) 11(44%) 11 (50%) 0.169 0.68

Years of age 44.4 (9.5) 48.9 (10.2) 1.59 0.12

Born in the Netherlands 15(60%) 13 (59%) 0.004 0.95

Education level 0.733 0.69

Low 6(25%) 8 (36%)

Medium 15 (63%) 12(55%)

High 3 (12%) 2 (9%)

Holds a paid job 1 (4%) 2 (9%) 0.508 0.48

Years in treatment 6.2 (4.3) 7.1 (4.3) 0.780 0.44

Table 2

Scores at pre-treatment and post treatment.

BI(n<25) CAU (n < 22)

Measure Pre M (SD) Post M (SD) Pre M (SD) Post M (SD)

HONOS 8.64 (4.47) 10.43 (5.79) 9.79 (5.83) 8.00 (5.52)

MANSA 54.4 (15.7) 59.4(13.1) 59.9 (13.3) 63.7 (9.4)

EQ5D (MVH-A1) 0.55 (0.36) 0.70 (0.37) 0.55 (0.41) 0.68 (0.38)

MHCS 60.8 (16.8) 65.4 (12.8) 65.0 (13.8) 68.4 (12.0)

Satisfaction NA 7.50 (1.54) NA 7.53 (0.96)

month with psychiatric nurses (SD = 5.8), varying between 0 and 17 Skype contacts per month. For many patients, working with the BI platform constituted their first experience with this form of internet video communication. Four patients had experienced the login procedure to the platform as complicated due to its connection security precautions. No patients reported adverse events during this study. Overall treatment satisfaction scores were on average between satisfactory - good (BI: M = 7.50, SD = 1.54; CAU: M = 7.53 SD = 0.96) and did not differ between the two interventions (p = 0.83).

5. Discussion

5.1. Conclusion

5.2. Limitations

The results of this study should be considered in the light of its limitations. First, the sample size of this study was small, and it was therefore underpowered to detect small or medium-sized effects. Second, the allocation of patients to the two conditions was not random, which may have hampered comparability between the two groups due to self-selection. For example, data on treatment motivation or tech-savviness of the participants has not been collected and is therefore not controlled for. No robust conclusion can therefore be drawn, but results give a first indication of the potential of blended FACT. Third, undesired variation in the amount of intervention exposure may have been introduced by the unstructured nature of the BI intervention and the used communication tools, something we have not controlled for. Fourth, for some patients it was not feasible to collect postintervention results on all instruments, which has led to a measurement incompletion rate of up to 19%. As results based on imputed data and complete cases data differed somewhat for the HONOS and the MANSA, these results should be interpreted with caution. Fifth, the time interval between the pre and post measurement only allowed for reporting short-term effects of the interventions. Sixth, the same FACT team was in contact with both CAU and BI patients, which may have led to spill-over effects.

This open label prospective controlled cohort study evaluated the acceptability of providing FACT in a blended care format (internet-based and face-to-face interaction combined). Thus far, many internet-based interventions have focused on common mental disorders (depression, anxiety disorders, problem drinking, smoking cessation) in a relatively well-functioning population. However, there is rapidly increasing evidence showing that internet interventions are acceptable, feasible, and potentially effective for people with SMI. The current findings are generally in line with the current evidence base that Internet-based interventions for patients with schizophrenia-spectrum disorders seem to be acceptable and feasible and have the potential to improve clinical and social outcomes (Alvarez-Jimenez et al., 2014). First, based on the self-report measures it was found that BI leads to comparable improvements in quality of life outcomes as CAU. Second, it appears acceptable to patients to receive BI, at least to the extent that it is acceptable to them to receive standard FACT, based on their satisfaction scores. Third, SMI patients appeared particularly satisfied with the possibilities of using internet technology for day-to-day contacts with psychiatric nurses - something also noted by de Wit et al. (2015) in their study on online support for people with mild intellectual disabilities or chronic psychiatric disorders. Patients also liked the possibility to establish contact with other patients and their social network through Skype. Given that the involvement of the social network of psychiatric patients is expected to become more important with mental health budget cuts in many countries (Wahlbeck and McDaid, 2012), this is a relevant finding as it indicates that technology can be useful in supporting patients to (re)establish contact with their network.

5.3. Implications

All in all, this first exploration of blended FACT has led to promising results regarding the acceptability of internet applications for SMI patients. Although we have collected clinical outcome variables and compared BI results to CAU results, sample sizes were too small to draw any firm conclusion regarding relative effectiveness. A future study should preferably be designed while taking in account the limitations indicated in this paper. Ideally, a sufficiently powered randomized controlled trial with longer term follow-up would be able to address most of the limitations identified in this paper. In addition, an in-depth evaluation of patient and professionals' needs and experiences while using blended FACT may be worth additional research effort. In addition, treatment uptake and cost data should be collected to evaluate the economic potential of blended FACT. In line with recent reviews (Alvarez-Jimenez et al., 2014; Naslund et al., 2015), more high quality trials are warranted to establish (cost-)effectiveness of internet interventions for SMI patients.

Contributors

MB, AE, BR designed the study, TB, JD facilitated the study, TB managed the implementation of the mijnTherapie platform. BR performed the data collection. MB, AE, BR prepared and preformed the data-analyses. All authors contributed to the preparation of this manuscript, in which MB took the lead. All authors have read and approved the final version of this manuscript.

Effects of Generalized Estimating Equations.

Main effect Time Main effect Condition Time X Condition

Measure Est. (SE) Wald p Est. (SE) Wald/t p Est. (SE) Wald p

HONOS 1.59 (1.04) 2.34 0.13 0.43 (1.40) 0.10 0.76 -2.74 (1.59) 2.98 0.09

MANSA 4.69 (3.34) 4.02 0.05 6.44(4.08) 2.49 0.11 -1.59(3.63) 0.19 0.66

EQ5D (MVH-A1) 0.15 (0.06) 6.80 0.01 0.04(0.10) 0.15 0.70 -0.02(0.10) 0.03 0.87

MHCS 4.30 (2.23) 3.71 0.05 5.57 (4.31) 1.67 0.20 -1.57(3.13) 0.25 0.62

Satisfaction NA NA NA 0.10(0.45) t = 0.22 0.83 NA NA NA

Note. Satisfaction (treatment satisfaction) was only measured post-intervention; post-intervention measurements were performed 3 months after pre-intervention measurements; in the Generalized Estimating Equations analyses, age, sex, duration of treatment in the mental health care treatment facility, origin of the patient and/or his or her parents (Netherlands / all other countries), and employment status were included in the model as covariates.

Acknowledgments

The study was conducted in collaboration with the Mentrum, which is a division of Arkin, an Amsterdam-based mental health and addiction treatment organisation. Arkin supported and funded the study and facilitated its development. The BI intervention was developed by and under supervision of Arkin employees, using the IPPZ mijnTherapie platform. IPPZ was not involved in the preparation, performance or reporting of this study. The authors declare they have no conflict of interest. The authors would like to thank Tom Bos (TB) for his work on the implementation of the mijnTherapie platform.

References

Alvarez-Jimenez, M., Alcazar-Corcoles, M.A., Gonzalez-Blanch, C., Bendall, S., McGorry, P.D., Gleeson, J.F., 2014. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr. Res. 156,96-106.

Barlow, J.H., Ellard, D.R., Hainsworth, J.M., Jones, F.R., Fisher, A., 2005. A review of self-management interventions for panic disorders, phobias and obsessive-compulsive disorders. Acta Psychiatr. Scand. 111, 272-285. Carpinello, S.E., Knight, E.L., Markowitz, F.E., Pease, E.A., 2000. The development of the mental health confidence scale: a measure of self-efficacy in individuals diagnosed with mental disorders. Psychiatr. Rehabil. J. 23, 236-243. Chou, K.R., Shih, Y.W., Chang, C., Chou, Y.Y., Hu, W.H., Cheng, J.S., Yang, C.Y., Hsieh, C.J., 2012. Psychosocial rehabilitation activities, empowerment, and quality of community-based life for people with schizophrenia. Arch. Psychiatr. Nurs. 26,285-294. Civljak, M., Stead, L.F., Hartmann-Boyce, J., Sheikh, A., Car, J., 2013. Internet-based

interventions for smoking cessation. Cochrane Database Syst. Rev., CD007078 Cuijpers, P., Marks, I.M., van Straten, A., Cavanagh, K., Gega, L., Andersson, G., 2009. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn. Behav. Ther. 38, 66-82. de Wit, J., Dozeman, E., Ruwaard, J., Alblas, J., Riper, H., 2015. Web-based support for daily functioning of people with mild intellectual disabilities or chronic psychiatric disorders: a feasibility study in routine practice. Int. Interv. 2,161-168. Dolan, P., 1997. Modeling valuations for EuroQol health states. Med. Care 35,1095-1108. Drukker, M., Bak, M., Campo, J., Driessen, G., Van Os, J., Delespaul, P., 2010. The cumulative needs for care monitor: a unique monitoring system in the south of the Netherlands. Soc. Psychiatry Psychiatr. Epidemiol. 45,475-485. Drukker, M., Visser, E., Sytema, S., van Os, J., 2013. Flexible assertive community treatment, severity of symptoms and psychiatric health service use, a real life observational study. Clinical Pract. Epidemiol. Ment. Health 28,202-209. EuroQol Group, 1990. EuroQol: a new facility for the measurement of health-related quality of life. Health Policy 16,199-208.

van ' t Hof, E., Cuijpers, P., Stein, D.J., 2009. Self-help and internet-guided interventions in depression and anxiety disorders: a systematic review of meta-analyses. CNS Spectr. 14, 34-40.

Honaker, J., King, G., Blackwell, M., 2011. Amelia II: a program for missing data. J. Stat. Softw. 45,1-47.

Kasckow, J., Felmet, K., Appelt, C., Thompson, R., Rotondi, A., Haas, G., 2013. Telepsychiatry in the assessment and treatment of schizophrenia. Clin. Schizophr. Relat. Psychoses 21,1-22.

Kerzman, T., Kok, I., Wijngaarden, B.V., 2003. De GGZ Thermometer Nader Onderzocht: De Evaluatie Van Een Vragenlijst Voor Clientwaardering in de Volwassenenzorg ([Studies on the mental health thermometer: Evaluaiton of an instrument for patient satifaction in adult mental health care]). Trimbos-institute, Utrecht, the Netherlands.

Kind, P., Hardman, G., Macran, S., 1999. UK Population Norms for EQ-SD. York Centre for Health Economics Discussion Paper 172.

Mulder, C.L., Staring, A.B.P., Loos, J., Buwalda, V., Kuijpers, D., Sytema, S., Wierdsma, A.I., 2004. De Health of the Nations Outcome Scales (HoNOS) als instrument voor 'routine outcome assessment ([The Health of the Nations Outcome Scales (HoNOS) as an instrument for routine outcome assessment]). Tijdschr. Psychiatr. 46, 273-285.

Musiat, P., Goldstone, P., Tarrier, N., 2014. Understanding the acceptability of e-mental health-attitudes and expectations towards computerised self-help treatments for mental health problems. BMC Psychiatry 14,109.

Naslund, JA, Marsch, L.A., McHugo, G.J., Bartels, S.J., 2015. Emerging mHealth and eHealth interventions for serious mental illness: a review of the literature. J. Ment. Health 24, 321-332.

Nugter, M.A., Buwalda, V.J.A., Dhondt, A.D.F., Draisma, S., 2012. The use of the HoNOS in the treatment of patients. Tijdschr. Psychiatr. 54, 153-159.

Priebe, S., Huxley, P., Knight, S., Evans, S., 1999. Application and results of the Manchester short assessment of quality of life (MANSA). Int. J. Soc. Psychiatry 45, 7-12.

R Core Team, 2014. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

Riper, H., Blankers, M., Hadiwijaya, H., Cunningham, J., Clarke, S., Wiers, R., Ebert, D., Cuijpers, P., 2014. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One 9, e99912.

Stein, L.I., Test, M.A. (Eds.), 1978. Alternatives to Mental Hospital Treatment. Plenum Press, New York.

van der Krieke, L., Wunderink, L., Emerencia, A.C., de Jonge, P., Sytema, S., 2014. E-mental health self-management for psychotic disorders: state of the art and future perspectives. Psychiatr. Serv. 65, 33-49.

van Veldhuizen, J.R., Bahler, M., 2013. Manual Flexible Assertive Comunity Treatment: Vision, Model, Practice, and Organisation. van Veldhuizen & Bahler, Groningen, the Netherlands.

Wahlbeck, K., McDaid, D., 2012. Actions to alleviate the mental health impact of the economic crisis. World Psychiatry 11,139-145.

Wing, J.K., Beevor, A.S., Curtis, R.H., Park S.B.G., Hadden, S., Burns, A., 1998. Health of the nation outcome scales: research and Development. Br. J. Psychiatry 172,11-18.