Scholarly article on topic 'Conduct problems in children. Characteristics of families recruited for a clinical outcome trial as compared to families in an implementation study'

Conduct problems in children. Characteristics of families recruited for a clinical outcome trial as compared to families in an implementation study Academic research paper on "Psychology"

CC BY-NC-ND
0
0
Share paper
Academic journal
Children and Youth Services Review
OECD Field of science
Keywords
{"Client representativeness" / "Conduct problems" / "Family characteristics" / "Parent characteristics"}

Abstract of research paper on Psychology, author of scientific article — Roar Solholm, Terje Ogden, Reidar Jakobsen

Abstract The implementation of empirically supported treatments (EST) is recommended as a way to transfer knowledge from research to clinical practice and to improve service quality. One area of concern has been client representativeness, that is to which degree participants in EST studies resembles the target group in usual care settings. For children with conduct problems the recommended ESTs have been parent training or parent mediated programs. The aim of this article is to explore and describe central parent and family characteristics of families with conduct disordered children recruited from ordinary clinical practice in connection with the evaluation of the Parent Management Training — Oregon (PMTO) model in Norway, and to see whether the families recruited to a randomized control trial (RCT) differ from families recruited to a large scale implementation study in routine practice. Data from 376 families indicated that there were few differences between the two samples and thus that the parent and family characteristics found in the RCT study were representative of help-seeking families with conduct disordered children in Norway. Perhaps an even better treatment result could be achieved by tailoring PMTO to better suit the characteristics of Norwegian parents and families. Mothers (regardless of marital status) seem to be especially vulnerable to caregiver strain and suggested interventions should take this into consideration.

Academic research paper on topic "Conduct problems in children. Characteristics of families recruited for a clinical outcome trial as compared to families in an implementation study"

Conduct problems in children. Characteristics of families recruited for a clinical outcome trial as compared to families in an implementation study

Roar Solholm a,b* Terje Ogden b, ReidarJakobsenc

a Department of Clinical Psychology, University of Bergen, Norway

b The Norwegian Center for Child Behavioral Development (Atferdssenteret), Norway

c Regional Centre for Child and Adolescent Mental Health and Child Welfare, Uni Health, Uni Research, Norway

CrossMark

ARTICLE INFO

ABSTRACT

Article history:

Received 22 August 2013

Received in revised form 20 March 2014

Accepted 22 March 2014

Available online 31 March 2014

Keywords:

Client representativeness Conduct problems Family characteristics Parent characteristics

The implementation of empirically supported treatments (EST) is recommended as a way to transfer knowledge from research to clinical practice and to improve service quality. One area of concern has been client representativeness, that is to which degree participants in EST studies resembles the target group in usual care settings. For children with conduct problems the recommended ESTs have been parent training or parent mediated programs. The aim of this article is to explore and describe central parent and family characteristics of families with conduct disordered children recruited from ordinary clinical practice in connection with the evaluation of the Parent Management Training — Oregon (PMTO) model in Norway, and to see whether the families recruited to a randomized control trial (RCT) differ from families recruited to a large scale implementation study in routine practice. Data from 376 families indicated that there were few differences between the two samples and thus that the parent and family characteristics found in the RCT study were representative of help-seeking families with conduct disordered children in Norway.

Perhaps an even better treatment result could be achieved by tailoring PMTO to better suit the characteristics of Norwegian parents and families. Mothers (regardless of marital status) seem to be especially vulnerable to care-giver strain and suggested interventions should take this into consideration.

© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.org/licenses/by-nc-nd/3.0/).

1. Introduction

Treatment research has addressed a wide range of social, emotional and behavioral problems in childhood and adolescence and many treatment interventions now meet the criteria for empirically supported (EST) or evidence-based treatment (EBT) (e.g. Hoagwood, Burns, Kiser, Ringeisen, & Schoenwald, 2001; Kazdin, 2008; Kazdin & Weisz, 2010). Based on this research it has been argued that EBTs, either directly or in an adapted form, should be taught or used in ordinary clinical practice, thereby establishing evidence based practice (EBP). The aim of this strategy has been to bridge health research and practice and to ensure that the regular services offered to children and their families are"(...) scientifically proven, state-of-the-art approaches to assessment, treatment and prevention" (Hawley & Weisz, 2002, p.225).

Despite the evidence, most EBTs have not made their way into standard clinical practice and therapist training programs (Nock, Goldman, Wang, & Albano, 2004; Weisz & Gray, 2008). This might reflect challenges in the transfer and implementation process itself (Fixsen, Blase, Duda, Naoom, & Van Dyke, 2010; Hoagwood et al., 2001; La Greca, Silverman, & Lochman, 2009; Schoenwald & Hoagwood, 2001), and also the fact that

* Corresponding author at: Department of Clinical Psychology, UiB, Christiesgt 12,5015 Bergen, Norway. Tel.: +47 55588860.

E-mail address: roar.solholm@psykp.uib.no (R. Solholm).

critics of EBP have questioned the relevance of EBTs to clinical practice (Kazdin, 2008; Weisz & Gray, 2008).

Both critics and advocates ofEBPs have recognized that there are marked differences between key conditions and characteristics in efficacy trials and in ordinary clinical practice (e.g. therapists, clients, treatment settings, context) (Hoagwood et al., 2001; Kazdin, 2008; Schoenwald & Hoagwood, 2001; Weisz & Gray, 2008; Weisz, Jensen-Doss, & Hawley, 2006), and there also seems to be a difference in the aim of psychotherapy under the two conditions, "eliminating symptoms" versus "the process coping with life"(Kazdin, 2008, p.147). However it is not clear how the differences between the two conditions may influence treatment attendance and outcome. There seems to be poorer outcome in community-based effectiveness studies than in research-based efficacy studies in which the researcher has more control over the treatment variables (e.g. Baker-Ericzen, Hurlburt, Brookman-Frazee, Jenkins, & Hough, 2010; Hoagwood et al., 2001). Some studies show that difference in key variables (e.g. case severity, complexity, comorbidity) do not necessarily impede treatment outcome of EBTs (Doss & Weisz, 2006; Kazdin & Whitley, 2006).

One meta-analysis of studies that directly compared EBT with community clinic population or usual care (UC) (Weisz et al., 2006) showed that EBTs systematically outperformed UC. However, the overall effects were modest, and the authors argue that due to heterogeneity of the UC some forms of UC may work better than others and may outperform EBTs for certain target populations. Moreover, studies that showed UC outperforming EBTs did not specify what the effective UC procedures were, what kind of therapists provided them, or to what kind of youths.

One area of special concern is client representativeness, and it has been claimed that EBT populations are less clinically severe and complex when compared to community

http://dx.doi.org/10.1016/j.childyouth.2014.03.019

0190-7409/© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).

clients (Kazdin, 2008; Weisz & Gray, 2008; Weisz & Kazdin, 2010). The reason for this could be that the process of recruitment, selection and enrollment in clinical trials (both in efficacy and effectiveness trials) is quite different from the process leading people to regular clinical services.

EBP is based on the assumption that there is a similarity between the participants in efficacy trials and the children and families met in ordinary clinical practice, but there is only limited evidence for this assumed similarity (Baker-Ericzen et al., 2010). There are few studies that compare the characteristics of samples from efficacy trials directly with the characteristics of community clinical or UC populations (Baker-Ericzen et al., 2010). The studies that have been conducted, do show that there are significant differences between the characteristics of samples in efficacy trials and samples of usual care, for children with anxiety disorder (more child comorbidity, symptom severity and lower family income in the usual care sample) (Southam-Gerow, Weisz, & Kendall, 2003), depression (more child comorbidity and racial/ethnic diversity in the usual care sample) (Weersing & Weisz, 2002), and a range of other disorders (Baker-Ericzen et al., 2010).

In their meta-analysis, Baker-Ericzen et al. (2010) compared data from 34 research trials on five ESTs with one large sample of UC for children with disruptive behavior disorders. They found a large variation in participants' characteristics within and across efficacy studies. They also found that, for most studies, parent and family characteristics were not reported. Comparison of UC and EST samples showed that although child demographics and symptom severity were similar, most parent and family characteristics were different, with higher rates of problems in the UC sample.

Baker-Ericzen et al. (2010) found that parents in the UC sample had lower socioeconomic status, were less educated and were more likely to belong to single parent households than parents in EST samples. A larger proportion of the parents in the UC sample also reported lower levels of social support and experience of increased domestic violence. But according to the authors the comparison was difficult because very few EST studies reported such information. Baker-Ericzen's UC sample did not give information on marital discord but among the parents in the EST samples 50-60% reported that they had experienced marital discord. When it came to psychopathology the findings were mixed. Some EST samples had higher and some lower percentages than the UC samples, but parents in the UC samples reported more parents with depressive symptoms. On the other hand, the parents in the EST samples reported more strain/stress than the UC sample, although the reason for this is not clear.

Baker-Ericzen et al. (2010) concluded that the concern about client representativeness was strengthened for parents of conduct-disordered children. They argued that it is unlikely that implementation of EST in regular practice will be successful if child, parent and family characteristics are not reported or if they turn out to be qualitatively different from the community treatment population. In their opinion, if EST samples differ significantly from UC samples the interventions have to be modified in order to be more effective or they have to be more carefully targeted at families like those treated in efficacy trials. They further argued that this is a particular problem for empirically supported parent-mediated treatments for disruptive behavior disorders, in which child, parent and family factors have been shown to predict and moderate treatment attendance and outcome. The above arguments may also apply when EBTs are moved from effectiveness studies to large scale implementation studies, as demonstrated in the present study.

During the past decade the Norwegian authorities have initiated and funded the national implementation of the Parent Management Training — Oregon Model (PMTO) for young children (4-12 years of age) with conduct problems and their families (Ogden, Forgatch, Askeland, Patterson, & Bullock, 2005). Six generations of Norwegian PMTO-therapists have completed their training during this period. As part of the implementation a randomized treatment effectiveness study was conducted with participants recruited through existing child service agencies (Child Welfare Services and Child and Adolescent Mental Health agencies). A Norwegian clinical trial demonstrated e treatment effectiveness in ordinary clinical practice, although effect sizes were small to moderate (Ogden & Hagen, 2008). These clinical outcomes indicate that PMTO is a relatively robust treatment intervention and that some of the implementation challenges were successfully met.

The participants in the present study came from two different studies on PMTO in Norway: the above mentioned randomized controlled effectiveness study (Ogden & Hagen, 2008) and one large scale implementation study in routine practice (Forgatch & DeGarmo, 2011). In this article these studies will be referred to as the "RCT study" and the "LSI study" respectively. The recruitment of families for the RCT study was restricted to the regular Child and Youth Mental Health Services (CYMHS) and Child Welfare Services (CWS) in the county municipalities while the LSI study also recruited families from various private and primary care services in the municipalities (MPCS). Because both studies were effectiveness studies, one would assume that the samples were more representative of clients in usual care settings than samples from an efficacy study, but the research design and procedures could still influence the selection of participants. In the RCT study, the parents had to accept the randomization procedure which implied that they had a 50% chance of being assigned to treatment as usual. This may have kept reluctant parents and practitioners from volunteering to participate in the study. In the non-randomized study on the other hand, all participants received PMTO. This may have had an effect on the composition of the parent groups participating in the two studies.

It could further be argued that the recruitment of therapists and sites to the RCT study was more controlled than in the LSI study because of a more rigorous selection of

therapists (first and second generations of PMTO therapists only) and because the number oftherapists and number and kinds ofsites were limited. In addition the access to PMTO in Norway was quite limited at the time when the RCT study started and this may have resulted in the recruitment of especially motivated and resourceful parents. On this background the LSI study was more of a "going to scale" study and possibly more representative of the population of regular service users.

In line with the recommendations of Baker-Ericzen et al. (2010), the aim of this article is to explore and describe central parent and family characteristics of families with conduct disordered children recruited from ordinary clinical practice in connection with the implementation of an EBT in Norway.

2. Method

2.1. Participants

The participants were 376 families (children and their parents/caretakers) who were recruited from two different but interconnected effectiveness studies on PMTO in Norway (Ogden et al., 2005). The first was a randomized controlled effectiveness study of PMTO versus regular services (N = 112) (Ogden & Hagen, 2008), and the second was an implementation study investigating treatment adherence over time and across sites (N = 264) (Forgatch & DeGarmo, 2011).

The participating families in both studies came from all five health regions in Norway. The number of families from each health region corresponded to the population distribution, but the two largest regions were slightly underrepresented. The families had contacted or been referred to regular child and adolescent service agencies, either at primary or specialist level, because of their child's behavior problems. Of these, 146 were recruited from the CWS, 176 from the CYMHS and 53 from the MPCS. Inclusion in the studies was not based on formal diagnostic procedures, but rather on the clinical judgment of therapists at the actual site. Children were not included if they (a) were diagnosed with autism, (b) had been exposed to documented sexual assault, (c) were intellectually disabled or (d) had parents with severe psychopathology or who were intellectually disabled.

The children in the RCT study ranged in age from 3.5 to approximately 13 years (M = 8.40, SD = 2.11), and 22 (19.6%) were girls. The age range in the LSI study was from 3 to nearly 13 years (M = 8.64, SD = 2.19) and 74 (28%) were girls. The majority of the children in both studies were Caucasian (RCT study = 95.4%, LSI study = 98%) and ethnic Norwegian (RCT study = 89.9%, LSI study = 97%). There was no significant difference between the two samples on ethnicity. According to Statistics Norway (Dugstad, 2006) 8.3% (defined as both parents being born abroad) or 13.5% (defined as at least one parent born abroad) of the population of Norway have an immigrant background. These numbers show that there was an underrepresen-tation of families from other countries and ethnic groups in this study.

The level of child conduct problems was measured at intake using the externalizing scores of the Child Behavior Checklist (CBCL) and Teacher Report Form (TRF). For girls the mean externalizing raw scores on the CBCL were 24.80 (SD = 13.54) and 22.02 (SD = 7.75) for the RCT study and LSI study respectively. For boys the scores were 26.57 (SD = 11.6) and 23.75 (SD = 9.42). On the TRF the mean externalizing scores for girls in the RCT study were 15.07 (SD = 11.46) and in the LSI study 10.47 (SD= 10.95). The scores for boys were 28.74 (SD = 14.06) and 24.33 (SD = 15.58). An independent t-test showed no significant differences between the girls in the RCT and LSI studies regarding externalizing CBCL scores (t = — 0.77, df = 16.32, p = .455) and externalizing TRF scores (t = —1.40, df = 60, p = .295). The corresponding result for the boys was CBCL (t = —1.92, df = 224, p = .057) and TRF (t = —1.83, df = 185,p = .068).

2.2. Procedure

The recruitment periods for the studies were partly overlapping and lasted from January 2001 to April 2005. Except for the

randomization procedure the intake procedures were identical. Parents were informed of the research projects, invited to participate and asked to give their written consent. Families who agreed to participate filled out various questionnaires regarding the status and functioning of the child and the family. In addition the families were observed and video-filmed during structured interaction tasks. The assessment sessions took place at the local agencies and each lasted for about 2 h. The parents were compensated NOK 300 (about $50) for their time. The two studies were both approved by the Norwegian ethical review board.

2.3. Measures

The participants were asked to fill out questionnaires regarding child behavior, family demographics, family finances, living conditions, family functioning, parent health, and couple relationship. The parent who reported having spent most time with the child filled out the basic questionnaires regarding family demographic, living conditions, financial situation and family characteristics. Questionnaires about health were filled out by the parents separately, but only couples filled out the questionnaire about the marital relationship. Due to this variation in respondents and also due to some missing responses the numbers may differ across variables reported in this study.

The assessment questionnaires were a mixture of well-established instruments and instruments adapted to or specifically designed for the two studies mentioned above. Some of the instruments have been used in previous Norwegian studies and some have been developed and used in previous research at the Oregon Social Learning Center (OSLC). These measures were translated and back translated from English to Norwegian.

2.3.1. Family characteristics

The parents were asked to state their date of birth, marital status, ethnicity and relationship with the child. They were further asked about their highest level of education and current work status. Level of education was recoded into three categories: 1 = completed elementary school, 2 = completed high school, 3 = college education or a university degree. Job status was recoded into four categories (1 = full time work, 2 = part time work, 3 = unemployed, 4 = not part of the work force).

With regard to family economy, the parents were asked about the family's annual salary and how many individuals they supported. In addition, the participants were asked about their gross annual income and filled out the Family Finance Questionnaire (FFQ). FFQis an OSLC developed instrument based on Pearlin and Schooler (1978) that assesses parental perception of the pressure of economic strain. Seven items addressing material needs, specific cutbacks, and inability to afford necessities form a scale of financial stress. In accordance with DeGarmo and Forgatch (2007) the response items ranged from 1 to 5 and a high score indicated high financial stress. The scale with 7 items has shown to be reliable and valid and in the present study the internal consistency of the scale was found to be satisfactory with Cronbach's alpha = .86.

The parents were further asked to describe type of residence and the number of children and adults living at their house. They were also asked to rate how likely it was that they would still live at the current residence in 6 months' time on a 4-point Likert type scale (from 1 =very likely to 4 = extremely unlikely). They also to rated their neighborhood on a five point scale (from 1 = very good to 5 = very bad) with regard to their own and children's personal safety, safe playgrounds, cleanliness, schools, quietness and protection of property.

The Interpersonal Support Evaluation List (ISEL) (Cohen & Hoberman, 1983) was used to assess parents' perception of social support. The ISEL originally consists of 40 items that assess the dimensions

of social support, appraisal, self-esteem, belonging, and tangible. In this study a short version of the ISEL with 16 items was used. Items were rated on a 4-point scale (from 0 = definitely false to 3 = definitely true) with an ISEL score in the range of 0 to 48. Higher scores indicate greater perceived support. Cronbach's alpha for ISEL in the present study was .87.

The Family Adaptability and Cohesion Evaluation Scale (FACES III) was administered as a measure of the parents' perception of family cohesion and adaptability (Olson, Portner, & Lavee, 1985). The FACES III consists of 10 cohesion items (for example, "Family members ask each other for help") and 10 adaptability items (for example, "In solving problems, the children's suggestions are followed"). The items are rated on a 5-point scale (from 1 = almost never to 5 = always). Cronbach's alpha was .87 for the Cohesion scale and .67 for the Adaptability scale in the present study.

232. Parent characteristics

Marital adjustment was measured by the Dyadic Adjustment Scale (Spanier, 1976). It consists of 32 items and assesses four dimensions of the martial relationship. Internal consistency of the DAS was examined and Cronbach's alpha was calculated for both gender groups on each of the scales. Affectional expression (4 items, females = .62, males = .58); Dyadic cohesion (5 items, females = .69, males = .73); Dyadic consensus (13 items, both groups = .89.); Dyadic satisfaction (10 items, females = .87, males = .78) and Total DAS (32 items, both groups = .92). This is in accordance with previous studies, although the reliability scores on Affectional expression and Cohesion were in the lower end of the spectrum. In the present study a total DAS score < 100 was used as a cutoff point.

Separate questions as well as questionnaires were used to assess the participants' perception of somatic and mental health and wellbeing. The participants rated their general health on a 5-point scale (from 1 = very good to 5 = very bad). The participants also rated how they felt about themselves (strong and fit or tired and worn out) on a 7-point scale (from 1 = very strong and fit to 7 = very tired and worn out). The participants were further asked to rate their general level of satisfaction with life on 7-point scale (from 1 = extremely satisfied to 7 = extremely dissatisfied).

Subjective somatic health complaints during the last year were assessed by twelve items and rated on a five point scale (from 1 = not at all to 5 = very much). Only three categories were examined in this study, gastrointestinal diseases, musculoskeletal pain (including headache and migraine) and asthma and allergy.

The Alcohol Use Disorder Identification Test (AUDIT) (Babor, Higgins-Biddle, Saunders, & Monteiro, 2001) was used to assess hazardous alcohol use, dependence symptoms and symptoms of harmful alcohol use. The AUDIT consists of 10 items that are scored on a 5-point Likert scale (0-4) and higher scores indicate more hazardous and harmful drinking. The total scores were classified according to the following risk levels: ZoneI: 1-7 points, ZoneII: 8-15 points, Zone Hi: 16-19points and Zone iV: score > 20.

The Substance Use Questionnaire (Capaldi & Patterson, 1991) was used to assess experiences with a variety of substances, including over-the-counter drugs, marijuana, and hard drugs. Respondents rated their frequency of use of these substances on 5-point Likert type scales (from 1 = never to 5 = daily or almost daily). Medications were recoded into three categories: pain killers (both mild and strong), sleeping pills and tranquilizers and anti-depressant.

Psychological distress was measured by the Symptom Checklist (SCL-5) and General Health Questionnaire (GHQ-28). The SCL-5 is a short version of the Symptom Checklist (SCL-25) and was developed by Tambs and Moum (1993) as a measurement of the level of global mental distress. The five items (feeling fearful, nervousness or shakiness inside, feeling hopeless about the future, feeling blue and worrying too much) related to the previous 14 days and the participants were asked to rate their situation on a four point scale (from 1 = not bothered

and distressed at all to 4 = very much). The sum of the first two items represents the SCL-5 anxiety score and the sum of the last three the SCL-5 depression score. Cronbach's alpha was calculated for both genders on SCL-5 total and subscales: SCL-5 total (5 items, females = .85, males = .83), SLC-5 anxiety (2 items, females = .71, males = .61) and SCL-5 depression (3 items, females = .82, males = .78)

The General Health Questionnaire (GHQ-28) has been found to be a valid instrument for identifying psychological distress (Goldberg & Hillier, 1979). It addresses the last two weeks and identifies chronic stress (Skreden et al., 2008). The version presented to the parents in the present study was slightly modified in that four of the seven items on the depression subscale were omitted from the questionnaire. These four items concerned suicide and suicidal ideation. The total GHQ-28 scores were thus estimated on the basis of the answers to 24 questions. Cronbach's alpha was calculated for both genders on total score (24 items, females = .91, males = .92).

Total GHQ-28 scores based on both the Likert score (items 0,1,2,3) and the case score (items 0, 0,1,1) were calculated with higher scores indicating increased level of distress. Psychological distress was measured by GHQLikert sum score (range 0-84) and clinically important distress was defined as a case score > 6.

2.4. Analytic strategy

The statistical analyses were carried out in SPSS version 18. T-tests for the paired samples were used to compare the mean scores for the two samples and chi-square statistics were used to compare the proportions for categorical variables in the two samples. Data from the questionnaires regarding health and the couple relationship were analyzed for each gender. Probability levels less than .05 were accepted as significant and reported with exact values.

3. Results

In the total sample of 376 families there were 213 couples and 135 single parents; data on marital status of 28 families were missing. The parents consisted of 322 women and 209 men. In 43 cases the gender of the respondent was not reported. The age of the women ranged from 23 to 52 years, M = 36.60, SD = 5.9. The men were slightly older with age ranging from 23 to 62 years (M = 39.9, SD = 7.1). An independent t-test showed that the women in the RCT study were significantly younger than the women in the LSI study (t = 2.01, df = 196, p = .046). In the majority of families at least one of the parents was reported as being a biological parent (89.6%) and

2.2% were foster parents. In 7.8% of cases the information on the family relation of the respondent was missing, but there is reason to believe that at least one of the parents in these families was a biological mother or father.

An overview of the family demographics regarding marital status, number of children in the household, housing situation is given in Table 1, while Table 2 shows parents' level of education and job status. The results indicate that significantly more men in the LSI study had completed high school. Aside from this there were no significant differences between the two samples.

The mean annual salary for the total sample was NOK 407,509, SD = NOK 223,730. The median salary was NOK 400,000. There was no significant difference between the two samples regarding annual salary, but as is shown in Table 3 the participants of the RCT study were experiencing more financial stress.

There were no differences in perceived social support as measured by the ISEL and no differences in family functioning as measured by the cohesion and adaptability subscales on the FACES III.

Of the 324 families who answered the questions regarding their perception of the neighborhood, more than 92% stated that it was likely or very likely that they would remain at their current address for the next six months. Typically more than 90% of the families rated their neighborhood as good or very good on every dimension. There were no significant differences between the two samples on these variables.

Table 4 shows the parents' perception of general health, strength and fitness and life-satisfaction, subjective somatic health complaints, experience and use of alcohol, drugs and medications and clinical psychological distress. As shown in the table, there were no differences between the RCT and LSI samples on these variables.

Regarding the use of alcohol, more than 95% of the women and men in both samples scored at risk levels 1 and 2. Experience with other drugs (amphetamine, cocaine, opiates, designer drugs) was reported in only one or two cases and is therefore not presented. When asked for current drug use (last year), 0.9% of the women and 0.5% of the men reported the use of hashish/marijuana and 0.3% of the women reported the use of opiates. For all other drugs the parents reported no use.

Table 5 shows the couple adjustment measured by the DAS and psychological distress measured by the SCL-5 and total GHQ-28.

There were no differences between the two samples regarding couple adjustment. When a total cutoff score of < 100 was used, 12.6% of the women and 14.3% of the men in the total sample described their families as distressed. There was thus no indication of marital discord being a prominent family characteristic of the participants.

Table 1

Family demographics.

Combined sample RCT study LSI study

N % N % N % x2 df p

149 42.8 45 43.3 104 42.6 0.44 2 ns

64 18.4 17 16.3 47 19.3

135 38.8 42 40.4 93 38.1

76 21.1 23 22.7 53 20.9 1.82 3 ns

177 49 50 46.3 127 50.2

84 23.3 25 23.1 59 23.3

24 6.6 10 9.3 14 5.5

238 73.7 54 70.1 184 74.8 1.69 2 ns

77 23.8 22 28.6 55 22.4

8 2.5 1 1.3 7 2.8

Marital status Married or cohabiting with child's biological parent Married or cohabiting with child's non-biological parent Single parent Number of children Only the referred child Two children Three children Four or more children Housing situation Owns house or apartment Rents own house or apartment Other

Table 2

Level of education and job status.

Combined sample N %

RCT study

LSI study

Level of education Completed elementary school (7-10 years) Completed high school (at least 11 years) College education or a university degree Job status Employed

Unemployed

Not part of the work force

47 16 126 32 67 28

143 71 11

19.6 21.1 52.2 42.1 27.9 36.8

62.4 91.0 4.8 3.8 32.8 5.1

16 1 1

21.4 38.9

16.7 20.0 44.4

58.2 88.9

40.3 5.6

32 9 85 29 53 20

104 55 10

2 48 3

18.8 15.5 50.0 50.0 31.2

64.2 91.7 6.2 3.3

29.6 5.0

3.08 7.56

4.08 0.20

.ns .023

Note: Total N for each variable differs due to varying response rates.

Women in the RCT sample scored significantly higher on both the total scale and the depression subscale of the SCL-5 compared to the LSI sample. However, both genders showed elevated levels of psychological distress compared to Tambs and Moum (1993) original data both on the total SCL-score, and anxiety and depression subscale score.

There were no differences between the two samples on the total GHQ-28 score. The scores were elevated for both genders compared to a sample of parents of preschool children for both total GHQ-score and GHQ-case score (Skreden et al., 2008), thus indicating increased psychological distress. On the question as to whether they had suffered from mental problems in the last 6 months, 81.5% of the women and 96.6% of the men answered "not at all" or "a little". Only about 11% of the women and 6% of the men reported that they had received additional treatment (individual or family therapy) while they were receiving PMTO.

The LSI sample had a higher percentage of referrals from the CWS (42.6% vs. 30.4%) and municipality services (20.2% vs. 0%), while the RCT sample had more referrals from the CYMHS (69.6% vs. 37.3%). X2 (2, N = 375) = 43.14, p < .001.

4. Discussion

The aim of this article was to examine if families recruited to a randomized controlled effectiveness study (the RCT sample) differed from a nonrandomized "going to scale" implementation study (the LSI sample). The result showed few significant differences between the two samples.

In the following discussion we will discuss the similarities and the differences between the samples, and also to some extent the

Table 3

Perceived financial stress, social support and family.

Combined RCT study LSI study sample

M SD M SD M SD t df P

Financial stress index 1.81 0.75 2.01 0.81 1.74 0.72 2.68 274 .008

ISEL 37.46 7.30 36.24 7.52 37.94 7.17 1.90 223 ns

Cohesion 39.46 5.49 39.16 5.60 39.59 5.46 0.64 327 ns

Adaptability 25.55 5.03 26.20 4.41 25.28 5.25 1.49 327 ns

characteristics of the combined sample. This should be done with caution because there is a lack of relevant Norwegian norms and studies reporting norms on the actual variables. In some instances however, the response pattern may indicate whether the characteristics of the sample deviate from what could be considered normative.

Differences between the two parent groups participating in the current study were found in the pattern of referrals, in the mean age of mothers in the educational level of fathers, and in the level of parent financial and psychological stress and depression. The finding that the LSI sample had more referrals from the Child Welfare Services and the municipalities was expected and it probably reflects the process of implementation and diffusion of PMTO in Norway (Ogden, Hagen, Askeland, & Christensen, 2009). As the implementation progressed relatively more therapists were trained in the Child Welfare Services (CWS) and in the municipalities than in the Child and Youth Mental Health Services (CYMHS). The finding that women in the RCT sample were significantly younger than the women in the LSI sample is difficult to explain and may be accidental. And that more men in the LSI sample had completed high school is also difficult to interpret. Given the low number of male respondents in the RCT study (N = 18) this may have happened by chance, but one possible explanation could be that men with higher education were more reluctant to participate in the randomization process. Moreover, the participants in the RCT showed significantly more (although still low levels of) financial stress. This result was somewhat unexpected because more families were recruited from the CWS to the LSI sample, and generally these families are expected to experiences more financial strain than families who are referred to CYMHS. On the other hand, the percentage of single parents and women who were not part of the work force was somewhat higher in the RCT sample. Additionally, women in the RCT sample were characterized by higher levels of total psychological distress and depression (SCL-5), but the reason for this is unclear. The number of differences between the families participating in the RCT and LSI studies was low in relation to the number of comparisons performed, and does not seem to form any consistent pattern. And to the extent that there were differences, they indicate that the parents in the RCT study were more at risk.

The number of similarities between the parents in the two studies far outweighed the differences. On key demographic measures (e.g. ethnic origin, marital status, family size, housing conditions) there were no differences between the samples. The participants' homogeneity of ethnic origin across samples was probably due to features of the recruitment process (e.g. lack of adaptation to groups with less confidence in and use of the treatment services). There is some evidence to suggest

Table 4

Parent characteristics 1.

Combined sample N %

RCT study

LSI study

General health Good and very good

Strong and feeling fit Strong and very strong

Life satisfaction Some degree of satisfaction

Subjective somatic complaints Gastrointestinal disease

Musculoskeletal pain

Allergy and asthma

Alcohol consummation Drinks more than two units a month

Experience with illegal drugs Hashish/marijuana

Use of medication this year Pain killers (mild and strong)

Sleeping pills and tranquilizers

Anti-depressants

Clinical psychological distress GHQ-28 Case score = or >6

Women Men

Women Men

Women Men

Women Men

Women Men

Women Men

320 208

316 208

208 316

207 314 206

320 208

316 206 320 206

322 207

292 195

69.7 85.6

84.8 62.8 39.2

95.9 85.9

10.2 18.0

74 50 76 50 76 50

62.7 84.3

20.5 46.0

41.9 20.0 81.6

46.1 44.0

30.3 42.0

31.6 21.6

97.3 90.0

25.3 14.0

22.4 10.0

34.8 20.0

243 158 240

157 238

243 156

223 150

71.8 86.0

60.2 80.1

85.8 59.2 37.0 34.6

31.4 50.0

22.5 21.0

23.7 9.0

16.7 6.4

30.9 18.0

2.29 0.09

2.32 2.81

3.12 2.81

0.67 0.46 0.81

3.54 1.99 1.43

0.04 0.97

2.55 0.01

0.45 0.91 0.09 1.05 1.28 0.74

0.36 0.09

ns ns ns

ns ns ns ns ns ns

Note: Total N for each variable differs due to varying response rates.

that a change in recruitment procedure lowers the threshold for seeking help among families of first and second generation immigrants (e.g. among families of Pakistani and Somali origin) in Norway (Bjorknes, Jakobsen, & Naerde, 2011). In Norway most of the children in all age

groups live with both their biological parents (Ministry of Children, Equality & Social Inclusion, 2003). Both samples showed an increased percentage of single parent and step-parent families. An increased number of single parent households among parents of conduct-disordered

Table 5

Parent characteristics 2.

Combined sample RCT study LSI study

M SD M SD M SD t df p

Dyadic adjustment scale

Total Women 117.3 17.1 114.8 20.4 118.0 15.9 0.98 149 ns

Men 115.5 16.2 115.6 17.6 115.5 15.9 0.04 138 ns

Consensus Women 51.4 7.9 51.3 7.6 51.6 8.0 0.22 181 ns

Men 51.4 7.6 52.2 7.1 51.1 7.7 0.83 165 ns

Satisfaction Women 38.6 6.9 37.5 7.8 39.0 6.7 1.27 180 ns

Men 37.7 5.5 36.8 6.6 38.0 5.2 1.10 167 ns

Cohesion Women 16.7 4.4 16.8 4.6 16.7 4.3 0.07 189 ns

Men 17.0 4.3 16.5 4.8 17.1 4.1 0.81 170 ns

Affectional expression Women 9.3 2.1 2.2 9.0 9.3 2.0 0.94 190 ns

Men 9.2 2.1 9.1 2.3 9.2 2.0 0.32 171 ns

SCL-5 and GHQ-28

SCL-5 total Women 9.27 3.17 10.0 3.0 9.0 3.2 2.22 313 .027

Men 7.52 2.61 8.0 2.6 7.4 2.6 1.42 199 Ns

SCL-5 anxiety Women 3.33 1.32 3.6 1.3 3.3 1.3 1.78 318 Ns

Men 2.75 1.04 2.8 1.0 2.7 1.1 0.53 201 Ns

SCL-5 depression Women 5.93 2.14 6.4 2.1 5.8 1.4 2.13 315 .034

Men 4.77 1.78 5.2 1.9 4.7 1.8 1.75 201 ns

GHQ-28 Women 25.34 11.39 26.8 11.2 24.9 11.4 1.23 290 ns

Men 22.59 10.24 24.2 11.0 22.1 10.0 1.18 193 ns

children is well documented and is often considered to be a risk factor for the further development of conduct problems (e.g. Waldfogel, Cragie, & Brooks-Gunn, 2010). It is also assumed that the amount of strain and stress associated with single parenthood affect parenting in a negative way (e.g. Forgatch & DeGarmo, 1999; Hinshaw & Lee, 2003). The increased number of step-parent households is perhaps not as easily explained. One could argue that both single-parent households and families with a history of transitions (divorce and/or introduction of a step-parent) are a risk factor for the further development or acceleration of conduct problems (e.g., due to reduced capacity, lack of commitment and skills to manage the child, and more challenging child reactions) (DeGarmo & Forgatch, 2007). On the other hand, the likelihood of both divorce/separation and single parent status could also be increased as the result of the wear and tear from raising children with conduct problems (Forgatch & DeGarmo, 1999). Family size and housing condition is typical for the general population and does not seem to be a problem in either sample.

Most of the children lived with at least one of their biological parents, and the parents in both samples expressed general satisfaction with their own housing conditions and with their neighborhood.

In both samples a large proportion of the women were staying at home (not part of the work force). The reason why so many mothers had chosen to stay at home is not clear, and it could both reflect a perceived need to be available because of the child's problems, the adjustment problems of the mothers themselves or the high number of women with single parent status. It could also be due to the generous Norwegian welfare arrangements or combinations of the reasons mentioned above. Again this is speculation and needs further investigation.

There was no significant difference between the two samples regarding annual salary. In their article on the RCT study, Ogden and Hagen (2008) have described the income level as middle to lower. A reduced annual salary compared to the general population is expected because of the higher proportion of single parent families and the proportion of mothers staying at home. However, the low scores on the financial stress index do indicate that financial conditions in itself is not a serious source of distress among most parents of conduct-disordered children in Norway.

There were no differences between the two samples regarding the perception of social support, family adaptation, couple adjustment, perception of general health and well-being, subjective somatic complaints, use of alcohol and experience and use of drugs. The parents in both samples seem to have experienced social support in the normative range from their family members, friends and neighbors. Moreover, there were no indications of these families having severe problems or of couples having severe marital discord.

Members of the Norwegian population usually give high ratings of perceived general health and wellbeing. In the Hunt population study for instance (Krokstad & Knudsen, 2011) the corresponding positive ratings were over 80% for both genders on identical questions of perceived general health and life satisfaction. Against this background the scores of the women seem a bit low and it is also interesting to note that the women in both samples rate themselves lower than the men on these questions.

The reported use of medications (tranquilizers/sleeping pills, an-tidepressants and pain killers) seems high compared to available population data (Statistics from the Norwegian Prescription Database, 2014). This extensive use could reflect elevated levels of psychological distress (see discussion below). The results do not indicate serious problems with alcohol or drug use in either sample. A larger proportion of women than men reported having tried illegal drugs, which might be an indication of previous externalizing problems. However, this fact does not seem to be reflected in actual use of illegal drugs.

Overall the Norwegian families of children with conduct problems reported relatively few problems on the various parent and

family measures of this study. There were few reports of the expected contextual risk factors associated with conduct disorder (e.g. McMahon, Wells, & Kotler, 2006), the parents did not report serious problems. There was an indication of reduction in perceived general health and well-being, increased psychological distress and extensive use of medications. The women in both samples had higher scores than the men on instruments measuring psychological distress (and also compared to the Norwegian female population). They also have lower scores than men and the female population on ratings of general health, strength and life satisfaction. The use of medication seems a bit excessive compared to the general population (both men and women). It could be that mothers of conduct disordered children have more problems, but it could also be a result of caregiver strain or a combination of the two. The difference between men and women could be the result of sex-roles or different ways of handling stress. These questions need further investigation and international comparisons could also be of interest, using the same procedure and instrument.

There was an increased level of perceived psychological distress in both samples as measured by the SCL-5 and GHQ-28 (total and case score). This may indicate more personal problems, but could also reflect the strain on the caregiver of raising a child with conduct-disorder, or a combination of these. There is no clear explanation for the gender differences in the study. Norway has a clear and explicit equal opportunity policy concerning men and women, and fathers are to a large degree involved in the raising of and caring for their children. Still the differences could represent patterns of traditional gender roles regarding the responsibility for raising and taking care of children and/or possibly gender specific stress reactions to raising a child with conduct problems.

Because there are few systematic differences between the two Norwegian samples one might conclude that the RCT sample was representative for families and parents seeking help for conduct problems in Norway. However, this conclusion could be somewhat premature. Although all the families were recruited from families seeking help through the ordinary services in Norway, the intake criteria in both studies excluded families and parents with more comprehensive and complex problems. However, in our opinion, it is unlikely that the number of excluded families would be large enough to change the parent and family characteristics' profile significantly.

Research from the US indicates that UC samples have a more complex background "with multiple child, parent, and family issues" (Baker-Ericzen et al., 2010, p. 93) and that they have higher rates of problems than EST samples regarding parent and family characteristics. A direct comparison and interpretation of the differences between the sample of the present study and the samples reported in Baker-Ericzen et al. (2010) is not possible because of differences in the measures and variables, child sample characteristics, recruitment criteria and procedures. However, comparisons show that the Norwegian samples seem to have less complex issues and fewer problems than EST and UC samples in the United States (Skogen & Torvik, 2013). Even if Norway and the US have an egalitarian and informal culture in common with emphasis on family and community values there are differences in social and cultural conditions (Ogden et al., 2005). Norway is a social democracy with a welfare state based on social rights and social security, and provides free health care including free public treatment services. Norway also has high living standards and poverty is less of a problem than in the US. In spite of the apparent differences between the Norwegian and US samples, the implementation of the PMTO in Norway has been successful (Ogden et al., 2005) and the treatment has been shown to be reasonably effective (Ogden & Hagen, 2008). The fact that a treatment intervention, which was developed and tested in a different culture and on parents with different characteristics, has proven effective is interesting and attests to the robustness of the intervention.

5. Conclusions

The parent and family characteristics of children with conduct problems are central both to the theoretical analysis and recommended treatment interventions (Forgatch & Patterson, 2010). Because of this it is important that these factors are reported and understood.

The few differences between the RCT and LSI samples in this study indicate that the parent and family characteristics reported in the Norwegian RCT study are fairly representative of the actual target group. The argument has traditionally been that RCT studies have participants with fewer problems than a usual care sample. This study doesn't support the notion that RCT studies are not representative for usual care because the participants have fewer problems. If any, our findings are the opposite and one possible explanation is that this RCT study was done as an effectiveness study. Together with the result from the RCT study (Ogden & Hagen, 2008); this strengthens PMTO as a relevant choice of treatment for clinicians working with children with conduct problems in Norway.

Given the seemingly low problem levels and the profile of Norwegian parent and family characteristics, one would perhaps expect even better treatment results. Perhaps this could be achieved by tailoring PMTO so that it better suits the Norwegian parent and family characteristics. It seems reasonable to focus even more on parents' perception of their own general health and well-being, possible mental health problems, and caregiver strain. The mothers (regardless of marital status) seem to be especially vulnerable to caregiver strain and suggested interventions should take this into consideration.

References

Babor, T. F., Higgins-Biddle, J., Saunders, J. B., & Monteiro, M. (2001). AUDIT. The Alcohol Use Identification Test. Guidelines for use in primary care (2nd ed.). World Health Organization.

Baker-Ericzen, M. J., Hurlburt, M. S., Brookman-Frazee, L., Jenkins, M. M., & Hough, R. L. (2010). Comparing child, parent, and family characteristics in usual care and empirically supported treatment research samples for children with disruptive behavior disorders. Journal of Emotional and Behavioral Disorders, 18(2), 82-99. http://dx.doi. org/10.1177/1063426609336956. Bjorknes, R., Jakobsen, R., & Naerde, A. (2011). Recruiting ethnic minority groups to evidence-based parent training. Who will come and how? Children and Youth Services Review, 33(2), 351-357. http://dx.doi.org/10.1016/jxhildyouth.2010.09.019. Capaldi, D. M., & Patterson, G. R. (1991). Relation of parental transitions to boys adjustment problems. 1. A linear-hypothesis. 2. Mothers at risk for transitions and unskilled parenting. Developmental Psychology, 27(3), 489-504. Cohen, S., & Hoberman, H. M. (1983). Positive events and social supports as buffers of life

change stress. Journal of Applied Social Psychology, 13(2), 99-125. DeGarmo, D. S., & Forgatch, M. S. (2007). Efficacy of parent training for stepfathers: From playful spectator and polite stranger to effective stepfathering. Parenting — Science and Practice, 7(4), 331-355. Doss, A. J., & Weisz, J. R. (2006). Syndrome co-occurrence and treatment outcomes in youth mental health clinics. Journal of Consulting and Clinical Psychology, 74(3), 416-425. http://dx.doi.org/10.1037/0022-006x.743.416. Dugstad, G. (2006). Innvandring og innvandrere 2006 [Immigration and immigrants 2006]. Innvandring og innvandrerelmmigration and immigrants. Oslo: Statistics Norway.

Fixsen, D. L., Blase, K. A., Duda, M. A., Naoom, S. F., & Van Dyke, M. (2010). Implementation of evidence-based treatments for children and adolescents. In J. R Weisz, & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 435-450) (2nd ed.). New York: The Guilford Press. Forgatch, M. S., & Patterson, G. R. (2010). Parent Management Training — Oregon model. An intervention for antisocial behavior in children and adolescents. In J. R Weisz, & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed.). New York: The Guilford Press. Forgatch, M. S., & DeGarmo, D. S. (1999). Parenting through change: An effective prevention program for single mothers. Journal of Consulting and Clinical Psychology, 67(5), 711-724.

Forgatch, M. S., & DeGarmo, D. S. (2011). Sustaining fidelity following the nationwide PMTO (TM) implementation in Norway. Prevention Science, 12(3), 235-246. http://dx.doi.org/10.1007/s11121-011 -0225-6. Goldberg, D. P., & Hillier, V. F. (1979). A scaled version of the General Health Questionnaire. Psychological Medicine, 9(1), 139-145. Hawley, K M., & Weisz, J. R (2002). Increasing the relevance of evidence-based treatment review to practitioners and consumers. Clinical Psychology — Science and Practice, 9(2), 225-230.

Hinshaw, S. P., & Lee, S. S. (2003). Conduct and oppositional defiant disorders. In E. J. Mash, & R. A. Barkley (Eds.), Child psychopathology (pp. 144-198) (2nd ed.). New York: The Guilford Press.

Hoagwood, K., Burns, B.J., Kiser, L., Ringeisen, H., & Schoenwald, S. K. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52(9), 1179-1189.

Kazdin, A. E., & Weisz, J. R. (2010). Introduction. Context, background, and goals. In J. R. Weisz, & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 3-10) (2nd ed.). New York: The Guilford Press.

Kazdin, A. E. (2008). Evidence-based treatment and practice — New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist, 63(3), 146-159. http://dx.doi.org/10.1037/0003-066x.63.3.146.

Kazdin, A. E., & Whitley, M. K. (2006). Comorbidity, case complexity, and effects of evidence-based treatment for children referred for disruptive behavior. Journal of Consulting and Clinical Psychology, 74(3), 455-467. http://dx.doi.org/10.1037/0022-006x.74.3.455.

Krokstad, S., & Knudsen, M. S. (2011). Folkehelse i endring. Helseundersokelsen Nord-Trnndelag. HUNT 1 (1984-86)-HUNT 2 (1995-97)-HUNT 3 (2006-08) [Public health development. The HUNT Study, Norway. HUNT 1 (1984-86)-HUNT 2 (1995-97)-HUNT3 (2006-08)]. Levanger, Norway: HUNT Research Centre, Norwegian University of Science and Technology, 214.

La Greca, A. M., Silverman, W. K., & Lochman, J. E. (2009). Moving beyond efficacy and effectiveness in child and adolescent intervention research. Journal of Consulting and Clinical Psychology, 77(3), 373-382. http://dx.doi.org/10.1037/ A0015954.

McMahon, R J., Wells, K. C., & Kotler,J. S. (2006). Conduct problems. In E.J. Mash, & R. A. Barkley (Eds.), Treatment of childhood disorders (pp. 137-270) (3rd ed.). New York: The Guilford Press.

Ministry of Children, Equality and Social Inclusion (2003). Om familien-forpliktende samliv ogforeldreskap [On the family-binding cohabitation and parenthood]. Oslo: Ministry of Children, Equality and Social Inclusion, 29 (Retrieved from http://www.regjeringen. no/nn/dep/bld/Dokument/proposisjonar-og-meldingar/stortingsmeldingar/2002-2003/stmeld-nr-29-2002-2003-/11/1/2.html?id=471417).

Node, M. K., Goldman, J. L., Wang, Y. P., & Albano, A. M. (2004). From science to practice: The flexible use of evidence-based treatments in clinical settings. Journal of the American Academy of Child and Adolescent Psychiatry, 43(6), 777-780. http://dx.doi. org/10.1097/01.chi.0000120023.14101.58.

Ogden, T., Forgatch, M. S., Askeland, E., Patterson, G. R., & Bullock B. M. (2005). Implementation of parent management training at the national level: The case of Norway. Journal of Social Work Practice, 19(3), 317-329. http://dx.doi.org/10.1080/ 02650530500291518.

Ogden, T., & Hagen, K. A. (2008). Treatment effectiveness of Parent Management Training in Norway: A randomized controlled trial of children with conduct problems. Journal of Consulting and Clinical Psychology, 76(4), 607-621. http://dx.doi.org/10.1037/0022-006x.76.4.607.

Ogden, T., Hagen, K. A., Askeland, E., & Christensen, B. (2009). Implementing and evaluating evidence-based treatments of conduct problems in children and youth in Norway. Research on Social Work Practice, 19(5), 582-591. http://dx.doi.org/10. 1177/1049731509335530.

Olson, D. H., Portner, J., & Lavee, Y. (1985). FACES III. University of Minnesota, Department of Family Social Service.

Pearlin, L. I., & Schooler, C. (1978). Structure of coping. Journal of Health and Social Behavior, 19(1), 2-21.

Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services, 52(9), 1190-1197.

Skogen, J. C., & Torvik A. F. (2013). Atferdsforstyrrelser blant barn ogunge iNorge. Beregnet forekomst og bruk av hjelpetiltak [Conduct disorders among children and adolescents in Norway. Estimated prevalence and the use of helping interventions]. Oslo: Folkehelseinstituttet, 44.

Skreden, M., Skari, H., Bjork M. D., Malt, U. F., Veenstra, M., Faugli, A., et al. (2008). Psychological distress in mothers and fathers of preschool children: A 5-year follow-up study after birth. BJOG, 115(4), 462-471. http://dx.doi.org/10.1111/j.1471-0528. 2007.01631.x ([pii] BJO1631).

Southam-Gerow, M. A., Weisz, J. R., & Kendall, P. C. (2003). Youth with anxiety disorders in research and service clinics: Examining client differences and similarities. Journal of Clinical Child and Adolescent Psychology, 32(3), 375-385. http://dx.doi.org/10.1207/ S15374424JCCP3203_06.

Spanier, G. B. (1976). Measuring dyadic adjustment — New scales for assessing quality of marriage and similar dyads. Journal of Marriage and the Family, 38(1), 15-28.

Statistics from the Norwegian Prescription Database (2014). (Retrieved January 20,2014, from) http://www.norpd.no/Prevalens.aspx

Tambs, K., & Moum, T. (1993). How well can a few questionnaire items indicate anxiety and depression? Acta Psychiatrica Scandinavica, 87(5), 364-367.

Waldfogel, J., Cragie, T. -A., & Brooks-Gunn, J. (2010). Fragile families and child wellbeing. The Future of Children, 20(2), 87-112. http://dx.doi.org/10.1353/ foc.2010.0002.

Weersing, V. R., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70(2), 299-310. http://dx.doi.org/10.1037// 0022-006x.70.2.299.

Weisz, J. R., & Kazdin, A. E. (2010). The present and future of evidence-based psychother-apies for children and adolescents. In J. R. Weisz, & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 557-572) (2nd ed.). New York: The Guilford Press.

Weisz, J. R., & Gray, J. S. (2008). Evidence-based psychotherapy for children and adoles- Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychother-cents: Data from the present and a model for the future. Child and Adolescent apies versus usual clinical care — A meta-analysis of direct comparisons. American

Mental Health, 13(2), 54-65. http://dx.doi.org/10.1111/j.1475-3588.2007.00475.x. Psychologist, 61(7), 671-689. http://dx.doi.org/10.1037/0003-066x.61.7.671.