Scholarly article on topic 'Examining psychometric properties, measurement invariance, and construct validity of a short version of the Test to Measure Intercultural Competence (TMIC-S) in Germany and Brazil'

Examining psychometric properties, measurement invariance, and construct validity of a short version of the Test to Measure Intercultural Competence (TMIC-S) in Germany and Brazil Academic research paper on "Languages and literature"

Share paper
{"Intercultural competence" / "Multimethod test" / "Situational judgment test" / Measurement / invariance / "Cross-cultural generalizability"}

Abstract of research paper on Languages and literature, author of scientific article — Deborah B.L. Schnabel, Augustin Kelava, Fons J.R. van de Vijver, Lena Seifert

Abstract The goal of this study was to develop and validate a short version (TMIC-S) of the Test to Measure Intercultural Competence (TMIC; Schnabel, Kelava, Seifert, & Kuhlbrodt, 2015). TMIC-S measures six malleable abilities that support handling novel or difficult cross-cultural situations. The short TMIC-S version, comprising 25 self-report and six situational judgment items, was administered to 1032 Germans and 769 Brazilians. Confirmatory factor analysis (CFA) showed a good fit of the sixfactor multimethod model in both samples. Measurement invariance was examined by multigroup CFA, which showed metric and scalar invariance of the TMIC-S. An initial construct validation was addressed by computing correlations of the German and Brazilian TMIC-S versions with the Cultural Intelligence Scale (Van Dyne, Ang, & Koh, 2008). Additionally, prior intercultural experience was positively associated with latent TMIC-S means in both samples, highlighting criterion validity. Thus, TMIC-S is a valid instrument, which can be administered efficiently.

Academic research paper on topic "Examining psychometric properties, measurement invariance, and construct validity of a short version of the Test to Measure Intercultural Competence (TMIC-S) in Germany and Brazil"

Contents lists available at ScienceDirect

P9m I International Journal of Intercultural Relations

ELSEVIER journal

Examining psychometric properties, measurement invariance, and construct validity of a short version of the Test to Measure Intercultural Competence (TMIC-S) in Germany and Brazil

Deborah B.L. Schnabel3 *, Augustin Kelava3, Fons J.R. van de Vijver Lena Seiferte

a Hector Research Institute of Education Sciences and Psychology, Eberhard Karls Universitaet Tuebingen, Germany b Department of Cross-Cultural Psychology, Tilburg University, the Netherlands c North-West University, South Africa d University of Queensland, Australia

e Psychiatric Ward, University Medical Center Giessen and Marburg, Germany




Article history:

Received 10 August 2014

Received in revised form 24 August 2015

Accepted 27 August 2015

Available online 21 September 2015


Intercultural competence Multimethod test Situational judgment test Measurement invariance

Cross-cultural generalizability

The goal of this study was to develop and validate a short version (TMIC-S) of the Test to Measure Intercultural Competence (TMIC; Schnabel, Kelava, Seifert, & Kuhlbrodt, 2015). TMIC-S measures six malleable abilities that support handling novel or difficult cross-cultural situations. The short TMIC-S version, comprising 25 self-report and six situational judgment items, was administered to 1032 Germans and 769 Brazilians. Confirmatory factor analysis (CFA) showed a good fit of the sixfactor multimethod model in both samples. Measurement invariance was examined by multigroup CFA, which showed metric and scalar invariance of the TMIC-S. An initial construct validation was addressed by computing correlations of the German and Brazilian TMIC-S versions with the Cultural Intelligence Scale (Van Dyne, Ang, & Koh, 2008). Additionally, prior intercultural experience was positively associated with latent TMIC-S means in both samples, highlighting criterion validity. Thus, TMIC-S is a valid instrument, which can be administered efficiently. © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (

1. Introduction

Intercultural competence (ICC1) has been of scientific interest for decades (cf. Bennett, 1993; Kealey & Ruben, 1983; Sinicrope, Norris, & Watanabe, 2007). However, despite the increasing economic and political relevance of intercultural collaboration (Leung, Ang, &Tan, 2014; Sheahan, 2005), defining and measuring ICC appeared to be elusive (Ang et al., 2007). The present study set out to develop and test a new, theory-based instrument that focuses on intercultural competences in the behavioral domain. The instrument is tested in a cross-cultural framework.

* Corresponding author at: Europastraße 6, 72072 Tuebingen, Germany.

E-mail address: (D.B.L. Schnabel).

1 Although many authors use the term cross-cultural competence, which they abbreviate with 3C, this article uses intercultural competence, abbreviated with ICC, as the instrument discussed is called Test to Measure Intercultural Competence. However, ICC is also an abbreviation for other terms such as interclass correlation. The reader is referred to Schnabel et al. (2015) who pointed to the challenges caused by the variety of terms used in the field.

0147-1767/© 2015 The Authors. Published by Elsevier Ltd. This is an open access article underthe CC BY-NC-ND license (

1.1. The concept of intercultural competence

Researchers in various disciplines have developed ICC definitions (Byram, 1997; Fantini &Tirmizi, 2006; Gudykunst, 1994; Kim, 1992; Lambert, 1994; Thomas, 2003), which differ in focus, extent, and conceptualization (Spitzberg &Changnon, 2009). Bolten (2007a) established a threefold taxonomy for ICC definitions: (a) listing models in which different characteristics of intercultural competence are simply collected (e.g., Brislin, 1981; Ruben, 1976); (b) structure models in which the characteristics of intercultural competence are assigned to affective, behavioral, and cognitive categories (e.g., Dauner, 2011; Gertsen, 1990; Ting-Toomey, 1993); (c) procedural models in which intercultural competence is defined as context-specific competence to act due to its manifold connections with other core competences (e.g., Bolten, 2007b). Spitzberg and Changnon (2009) distinguished five types of competence models: compositional, co-orientational, developmental, adaptational, and causal process. The latter two model types assume that intercultural competence consists of several related components. In causal process models the nature of those relations is defined via correlations tested in empirical research.

Several authors assumed that ICC is a heterogeneous construct involving multiple dimensions that are necessary to interact with people from other cultures adequately and effectively (e.g., Bergemann and Bergemann, 2005; Fantini &Tirmizi, 2006; Hammer, Bennett, & Wiseman, 2003; Müller&Gelbrich, 2004; Spitzberg &Changnon, 2009). Intercultural competence is in theory often defined as ability or skill (Spitzberg & Changnon, 2009). However, prominent measurement approaches operationalize intercultural competence merely as stable personality traits (e.g., Chen & Starosta, 2000; Kelley & Meyers, 1995; Ruben, 1976), cultural intelligence (e.g., Earley & Ang, 2003; Van Dyne, Ang, & Koh, 2008), sensitivity to cultural differences (e.g., Hammer, 2008; Hammer et al., 2003), or a combination of those (e.g., Fantini & Tirmizi, 2006; Koester and Olebe, 1988; Yamazaki and Kayes, 2004). This obviously creates a gap between the conceptualization and the measurement of intercultural competence. In a recent overview of intercultural competence, Leung et al. (2014) reviewed models that are based on (a) traits, (b) attitudes, or (c) capabilities. Whereas several models were described that cover traits and attitudes, only one concept, namely cultural intelligence (CQ; Earley & Ang, 2003), was identified that focuses on malleable abilities. Mixed-models that integrate two or more concepts (e.g., traits and capabilities) in their measurement approach do not distinguish potential predictive or hierarchical relations between traits, attitudes, and capabilities (cf. Leung et al., 2014). Schnabel et al. (2015) developed an onion model of intercultural competence (cf. Schuler and Prochaska, 2000) that places malleable abilities at the core of intercultural competence. Personality traits, knowledge, and attitudes are positioned at the outside layer of the onion model, as they are understood as antecedents that influence the nature of intercultural competences, but are not intercultural competences per se. Locating malleable abilities at the heart of intercultural competence has one major advantage: In contrast to stable characteristics of a person a malleable construct is open to training (Schnabel & Kelava, 2013a,b; Schnabel, Kelava, & Van de Vijver, in press). Although intercultural sensitivity, which falls under the category of attitudes and which is competently assessed with the Intercultural Development Inventory (IDI; Hammer, 2008; Hammer et al., 2003), is a malleable construct too we do not compare it with ability constructs such as the TMIC framework. We thereby follow the suggestions of Hammer et al. (2003) who define intercultural sensitivity as a personal characteristic, which enables an individual to perceive intercultural differences, whereas intercultural competence is the ability to deal with those differences adequately.

The theoretical basis of Schnabel et al.'s (2015) model integrates different features of existing theories, such as the understanding of (intercultural) competence as an ability that can be learned, that helps individuals to master intercultural situations (e.g., Earley & Ang, 2003; Fantini & Tirmizi, 2006; Weinert, 2001), that involves multiple facets, and that triggers an individual's global behavioral orientation (Schuler & Prochaska, 2000). These aspects also serve as the operationalization basis for the Test to Measure Intercultural Competence (TMIC; Schnabel et al., 2015). Specifically, intercultural competence is seen as a global orientation of behavior. Multiple facets are involved, which belong to one of the three following competence groups: (a) "social competence" (e.g., communication competence), (b) "personal competence" (e.g., learning competence), and (c) "methodological competence" (e.g., self-management competence). These competences can be acquired, are directly influencing human behavior, and enable an individual—together or separately—to handle known, unknown, and/or challenging intercultural situations or problems while interacting with people from other cultures. Table 1 gives an overview of the intercultural competence model of Schnabel et al. (2015). It was developed by the help of a phenomenological-expert based strategy as well as by empirical data (for detailed information, see Schnabel, 2015). Fist, a detailed literature research was complemented by nine exploratory expert interviews with intercultural trainers. The primary goal was to develop an instrument that can be applied in training settings. Therefore, the trainers were interviewed about their understanding of intercultural competence, their opinion about which intercultural competence dimensions can be trained, and about which requirements a new instrument should fulfill. Second, on the basis of this information a quantitative pretest was designed and conducted with N =150 employees of an intercultural training company. Some preliminary analyses concerning the psychometric properties yielded a 17-facets model that was then tested in two further studies, which are described in the next section.

1.2. Measuring intercultural competence

Along with the variety of theoretical approaches there are numerous instruments to measure ICC, for example the Behavioral Assessment Scale for Intercultural Communication (BASIC; Koester & Olebe, 1988), the Cross-Cultural Adaptability Inventory (CCAI; Kelley & Meyers, 1995), the Cultural Intelligence Scale (CQS; Van Dyne et al., 2008), the Intercultural Devel-

Table 1

Intercultural competence model of Schnabel et al. (2015).



Social interaction



Creating synergies


Related Facets

In an international context it is particularly important to be responsive to the person you are talking to and to be able to actively direct the conversation. In intercultural communication verbal as well as non-verbal aspects play an important role

Sensitivity in communication, clarity in communication, flexibility in communication, perspective-taking in communication

During cooperation with people from other cultures or during a stay abroad individuals are often faced with unknown situations. This requires the motivation of a person to extend their own knowledge and to perform intercultural important behavioral patterns. Persons are seen as being capable of learning, if they recognize that they have gaps in their knowledge and, as a consequence, invest time in improving their knowledge Willingness to use a foreign language, willingness to learn, information seeking

The building of interpersonal relationships is of great importance, particularly during a stay abroad. Relationships with other people positively influence our own well-being and can reduce or prevent stress and avoid a culture shock. Furthermore, a well-functioning network can support when it comes to achieving aims and satisfying needs

Building professional networks, socializing, integration in groups, building trusting relationships

Actively reflecting and thus understanding of the own cultural identity increases self-knowledge and positively influences the awareness of and also the successful interaction with other cultures

Cultural identity awareness, cultural identity reflection

A stay abroad or cooperation with people from other cultures involves some challenges, which must be dealt with. Problems can arise, which have to be solved. Circumstances for the achievement of aims are more demanding and the new working and living environment can cause stress. The existence of strategies, which make dealing with these challenges easier is therefore of great importance

Goal setting, strategic problem-solving

Different ways of working as well as diverse interests and approaches come together during intercultural cooperation. When a joint aim is to be achieved it is of great importance to have the ability to realize potential

miss-understandings and lead a group towards common solutions

Mediation of different interests, enabling productive collaborations

opment Inventory (IDI; Hammer, 2008; Hammer et al., 2003), and the Intercultural Competence Assessment (INCA; Fantini & Tirmizi, 2006).

Recently, Gabrenya, Moukarzel, Pomerance, Griffith, and Deaton (2011) analyzed 34 instruments that aim to measure intercultural competence. Results concerning face validity, construct validity (convergent and divergent), and criterion validity (congruent and predictive) were taken into consideration. Validation results were then categorized in poor, moderate, and good. Only for seven tests validity findings have been published with five having satisfactory criterion validity (e.g., CQS, Van Dyne et al., 2008; Multicultural Personality Questionnaire, Van Oudenhoven & Van der Zee, 2002).

A large number of those scales incorporate both personality traits and competences or traits alone (Ang et al., 2007). The authors of those scales forgo to provide reasons for the selection of constructs or to analyze potential differences between them. Moreover, the utility of instruments is not addressed, which would require thinking about the application context of ICC instruments (cf. Deardorff, 2004; Fantini, 2009; Rathje, 2007). Most instruments are used in the process of training individuals for meeting intercultural challenges. This requires instruments that assess constructs that are open to training. Thus, the logic behind measuring traits that cannot be trained has to be questioned (Schnabel, 2015).

Additionally, these instruments use exclusively Likert scale based self-report items. However, nowadays many authors (e.g., Deardorff, 2006; Gelfand, Imai, & Fehr, 2008; Leung et al., 2014; Rathje, 2007; Sinicrope et al., 2007) call for multimethod approaches that meet the challenges of the complex ICC construct. The trend goes towards complementing well-established self-report procedures to increase their incremental validity (cf. Bolten, 2007b; Deardorff, 2006; Fantini, 2009; Leung et al., 2014; Rockstuhl, Ang, Ng, Lievens, & Van Dyne, in press). This was also supported by findings of Schnabel et al. (2015), who reported that the combination of a situational judgment test and a self-report scale explained the variance in success-relevant criteria of ICC better than a self-report scale alone. Thus, there is a strong need for performance-based measures, such as situational judgment items that gauge all major aspects of intercultural competence (Leung et al., 2014).

Representing a sophisticated attempt to measure ICC using a multimethod approach that refers to malleable competences rather than stable personality traits, Schnabel et al. (2015) recently published the reliable and valid Test to Measure Intercultural Competence (TMIC), a German instrument. The TMIC is so far the only multimethod ability-based behavior-related measure of ICC, which makes it especially promising for training, coaching, and counseling situations (Schnabel & Kelava, 2013a,b; Schnabel et al., in press). The TMIC assesses 17 competence facets, which are assigned to the following six second-order factors: communication, learning, social interaction, self-knowledge, self-management, and creating synergies. Following the recommendation of using more than one method to assess intercultural competence (Deardorff, 2004; Gabrenya et al., 2011; Leung et al., 2014), the TMIC combines 75 self-report questions (TMIC-SA) with 17 situational judgment items (TMIC-SJT; see Appendix A for a selection of items). Self-report scales aim at measuring the self-concept of a person, whereas situational judgment items uncover behavioral preferences (Bledow & Frese, 2009). A situational judgment test (SJT; McDaniel, Hartman, Whetzel, & Grubb, 2006) consists of critical incident scenarios with a fixed number of behavior alternatives as answering options. Testees must then choose one of those options (McDaniel et al., 2006). Situational judgment tests are known to have the advantages of direct measurements on the one hand and as being particularly practical, economic, valid, and robust against biases on the other hand (e.g., Hooper, Cullen, & Sackett, 2006; Lievens, Peeters, & Schollaert, 2008; McDaniel et al., 2006; McDaniel & Nguyen, 2001; Weekly & Ployhart, 2006).

Schnabel et al. (2015) reported convergent validity of the TMIC as both parts correlated with cultural intelligence (CQS; Van Dyne et al., 2008), r = .67, p < .001 forTMIC-SA and r = .26, p < .001 forTMIC-SJT as well as with global competence (Intercultural Sensitivity Index; Olson & Kroeger, 2001), r = .63, p < .001 for TMIC-SA and r = .26, p < .001 for TMIC-SJT. Criterion validity of TMIC-SA and TMIC-SJT was shown in using four external criteria of intercultural prior knowledge (Bhawuk & Brislin, 1992; Koester & Olebe, 1988; Morris & Robie, 2001). Furthermore, a positive, moderate correlation (r =.49, p<.001) was found between the overall values of the TMIC-SA and the TMIC-SJT, which shows that the two instruments show both common and unique aspects of intercultural competence, with the self-concept more covered in one and behavioral preferences more in the other method (Bledow & Frese, 2009). Moreover, the TMIC was tested in the context of a training of German university students who were preparing for a semester abroad. Schnabel et al. (in press) have designed a collaborative TMIC-based feedback intervention, which aims at increasing intercultural competence, intercultural self-understanding, intercultural self-efficacy, and the motivation to change maladaptive intercultural behavior. An experimental study with a pretest-posttest control group design revealed that students who took part in the intervention scored consistently higher in the TMIC and in other training relevant variables than before the intervention, whereas this was not the case for students in the control and comparison groups. Moreover, students reported that they felt well reflected in the TMIC results (cf. in Schnabel et al., in press). Altogether, these results provide validity support for the TMIC in Germany. Thus, it is worth addressing its cross-cultural applicability. To do so, a short version of the TMIC (TMIC-S) was created in this study. TMIC-S is especially useful for application in practice as it is less time consuming than the original TMIC and avoids cognitive overload of the testee (Eignor, 2013).

Besides the TMIC, the CQS (Van Dyne et al., 2008) is so far the only instrument that aims to measure exclusively malleable abilities. Although it focuses solely on self-appraisal, which raises methodological concerns, its psychometric properties are satisfactory on the whole (cf. Van Dyne et al., 2008). The instrument pictures the four dimensions of CQ. Metacognitive CQ enables a person to gain knowledge about his or her cultural identity and to be aware of others' cultural values and norms in order to choose and adapt behavior accordingly. Cognitive CQ relates to the actual knowledge about specific characteristics of one or more cultures such as norms, traditions, behavior patterns, and so forth. Motivational CQ describes a person's effort

toward learning about another culture. Behavioral CQfosters adequate nonverbal and verbal behavior when interacting with individuals from other cultures (Ang et al., 2007).

i.3. Cross-cultural generalizability of intercultural competence

In contrast to constructs like personality (e.g., Cheung, 2009; Church, 2001; McCrae and Terracciano, 2005) or leadership style (e.g., Brodbeck et al., 2000; House et al., 1999), only little empirical research has been conducted regarding the actual cross-cultural generalizability of intercultural competence. There are different points of view in the literature about universal versus culture-specific aspects of ICC (Arasaratnam & Doerfel, 2005). Deardorff (2004) conducted a Delphi study with 21 intercultural experts to learn more about definitions, components, and assessment of ICC. Her results indicate that definitions and components of ICC are evaluated as rather independent from culture, whereas generalizing assessment instruments across cultures is seen as challenging by intercultural experts.

Deardorff (2004) pointed out the need to apply ICC measures in a cross-cultural context. Few researchers have taken up this challenge so far. Zhong (1998) examined factors influencing the perception of intercultural communication competence with Chinese and American college students and found no differences between the two cultures' perceptions of intercultural communication competence items. Matsumoto et al. (2003) investigated the generalizability of the Intercultural Adjustment Potential Scale (ICAPS) that was originally intended for Japanese sojourners and immigrants by applying it to other cultures. The results indicated that the concept of intercultural adjustment is rather culture-neutral and that international use of the ICAPS is possible. The Intercultural Sensitivity Scale (ISS; Chen & Starosta, 2000), originally developed in the U.S., was applied in Germany by Fritz, Mollenberg, and Chen (2002). These authors were able to replicate the theoretical model and found satisfying results using confirmatory factor analysis (CFA). However, a second replication study of Fritz, Graf, Hentze, Mollenberg, and Chen (2005) produced deviating results that questioned the previously proposed transferability. Tamam (2010) applied the ISS in a non-western context, namely Malaysia, and found that 21 items of the original 24 items were loading on three factors (interaction attentiveness and respect, interaction openness, and interaction confidence) instead of the originally proposed five factors.

The studies reviewed before have one major characteristic in common: They were aiming to attain psychometric equivalence solely by replicating factor structures in a different culture than the original without investigating equivalence of those structures across cultures. Even though some findings are in favor of a universally applicable core set of intercultural competence the basis for deciding whether ICC, as measured by currently employed instruments, can be generalized across cultures is rather shallow. A prerequisite for finding a core set of ICC dimensions would be the availability of instruments in several languages that measure a single ICC construct (Ziegler & Bensch, 2013). This can be achieved by introducing measurement invariance procedures (Meredith, 1993) to cross-cultural research of intercultural competence. A given measurement invariance allows a comparison between scores on the latent construct in different groups in that it assures that the observed items contribute similarly to the latent construct in terms of their factor loadings and intercepts (Meredith, 1993). Measurement invariance is also one major part of the equivalence concept in cross-cultural research in general (Van de Vijver & Leung, 1997). Construct equivalence (also called structural equivalence) is attained when an instrument measures the same construct in different cultures. Van de Vijver and Leung (2009) suggested that a well-validated instrument, which aims to measure intercultural competence, should have good psychometric properties on the one hand and demonstrate measurement equivalence across cultures on the other hand. The next level is measurement unit (or metric) equivalence of two scales. The third level is scalar equivalence and points to identity of both the measurement unit and origin of the measurement scales (Matsumoto & Van de Vijver, 2011; Van de Vijver & Leung, 1997).

i.4. Study aim and hypotheses

The present study takes the multidimensional ability concept as well as the multimethod measurement approach of the Test to Measure Intercultural Competence (Schnabel et al., 2015) as a starting point to develop and validate a short version of the TMIC (TMIC-S), which incorporates 6 of the 17 proposed competence facets. The overall aim of this study is to show that TMIC-S holds six theoretically motivated factors, which are (a) invariant across two cultures and (b) distinct from but correlated with particular dimensions of CQ.

We selected Brazil and Germany for the cross-cultural comparison, because most ICC theories and instruments have an Anglo-American developmental background and are therefore primarily addressed at an English-speaking community (Deardorff, 2009; Martin, 1993). Also, to our knowledge no attention was given so far to cross-cultural comparisons of intercultural competence theories and instruments that involve Latin-American cultures. As Brazil belongs to the five countries in the world (the so-called BRICS countries), which currently experience the fastest and strongest economic growth, this is quite astonishing. Brazil is in itself a multicultural society (Gawora, de Souza Die, & Barbosa, 2011) that becomes even more diverse due to international traffic from and to Brazil. This also creates a strong need for intercultural diagnostic instruments in Brazil as internationalization of the society and the labor market rapidly increases (Muritiba, Muritiba, Campanário, & de Albuquerque, 2010). In assessing measurement invariance (Meredith, 1993) this study also completes a pioneering task in intercultural competence research.

1.4.1. Construct validity

In terms of the onion model of intercultural competence, which defines CQas a prerequisite for developing intercultural competences measured by the TMIC (Schnabel et al., 2015), convergent validity should be shown through overall positive correlations between the TMIC-S and the CQS scales.

Hypothesis 2a. All latent variables of the German and Brazilian TMIC-S show positive correlations with the four dimensions of the CQS (Van Dyne et al., 2008).

Based on a comparison of the definitions of the CQ dimensions with the definitions of the TMIC-S dimensions (see Appendix A), we predict that certain CQ dimensions will be especially related to specific TMIC-S factors.

Hypothesis 2b. Behavioral CQ has a strong positive relation to sensitivity in communication, goal setting, information seeking, mediation of different interests, and socializing.

Hypothesis 2c. Metacognitive CQhas a strong positive relation to sensitivity in communication, goal setting, information seeking, mediation of different interests, and cultural identity reflection.

1.4.2. Criterion validity

To assess criterion validity we selected three different external criteria. First, we hypothesize that individuals who are actively involved in intercultural topics and deal privately or professionally with intercultural topics through an intercultural study, work, or family environment will have an overarching high intercultural competence level (cf. Loboda, 2003; Schacher, 2011). Second, we include previous stays abroad as an external criterion. Being exposed to other cultures has a great impact on the creation of one's cultural identity as awareness about the self increases with reflecting on intercultural differences (Thomas, 2003). Moreover, previous studies showed that experiences abroad tend to foster individual, social, cognitive, and affective intercultural competences (cf. Behrnd & Porzelt, 2012; Morris & Robie, 2001; Williams, 2005). Importantly, the length of the stay abroad mediates the effect of experience abroad on intercultural competence development with longer stays being more effective (Behrnd & Porzelt, 2012). Third, former participation in an intercultural training is defined as another external criterion. Whereas, intercultural involvement and experience abroad are assumed to affect intercultural competence on the whole, intercultural training specifically addresses knowledge and awareness. However, the diversity of intercultural training approaches comes along with a variety of topics such as intercultural communication, relationship orientation, time management and so forth (Landis, Bennett, & Bennett, 2004). This might explain why many studies (for meta-analytic overviews, see Black & Mendenhall, 1990; Deshpande & Viswesvaran, 1992; Morris & Robie, 2001) report significant effects of intercultural training on intercultural competences, which are related to the self, perceptions, and relationships. For example, Black and Mendenhall (1990) identified 19 studies (e.g., Earley, 1987; Gudykunst, Hammer, & Wiseman, 1977; Landis, Brislin, Swanner, Tseng, & Thomas, 1985), which pointed to significant effects of intercultural training on building relationships with natives during a stay abroad. Overall, we infer:

Hypothesis 3a-c. Individuals in the Brazilian and German sample who (a) indicate that they are interculturally involved, (b) report that they had been living abroad for three months or longer,2 and (c) participated in an intercultural training before have higher latent means in all six dimensions of the TMIC-S.

2. Method

2.1. Participants and procedure

2.1.1. German sample

The German sample was mainly recruited from German universities and student organizations with an international focus (e.g., Erasmus Mundus Programme; European Union, 2014) and consisted of 1037 participants: 597 were women (58%), 429 were men (41%), and gender of 11 participants was missing (1%). The average age was 27.96 years (SD = 9.47). In total, 631 individuals indicated that they had stayed abroad for more than 3 months before (61%, 24 missing). Previous participation in an intercultural training program was the case for 687 Germans (66%, 1 missing) and intercultural involvement for 689 (66%, 1 missing).

2.1.2. Brazilian sample

Potential participants were contacted through the Brazilian partner university network of several German universities as well as through international student organizations and communities. Overall, 769 Brazilians took part in the survey. The average age of the 415 women (54%) and 354 men (46%) was 27.38 years (SD = 10.61).

436 (57%) participants reported a previous stay abroad with a duration of more than 3 months. 687 individuals (66%) experienced an intercultural training before and 604 (78%) were privately or professionally involved with other cultures.

2 Although, there is no clear definition of short-term and long-term stays abroad, many academic institutions, such as the German Academic Exchange Service (DAAD), use short-term for a stay abroad, which is only one semester long (approximately 3 months). Thus, extended stays abroad have to be at minimum 3 months long.

2.2. Instruments

2.2.1. Test to Measure Intercultural Competence (TMIC)

The TMIC (Schnabel et al., 2015) includes 75 self-report items (6-point Likert scale from does not apply at all to fully applies) and 17 situational judgment items (McDaniel & Whetzel, 2005). An ESEM-procedure (Exploratory Structural Equation Modeling; Asparouhov & Muthén, 2009) showed a very good model fit, thereby supporting the theoretically driven and empirically grounded 17 competence facets, y2(1636, N =641) = 2579.85, p <.001; y2/df =1.58; RMSEA = .031, 90% CI = [.029, .033]; SRMR= .017; CFI = .955; TLI = .927. Confirmatory factor analysis (CFA) showed an acceptable fit of the six-factor second-order structure that combined self-report and situational judgment items, y2 (3987, N = 641) = 8280.09, p< .001; y2/df = 2.08; RMSEA = .040,90% CI = [.040, .046]; SRMR = .076; CFI = .820; TLI = .810. The internal consistency (Cronbach's a) of the 17 facets ranged between .69 and .90. The overall reliability of the TMIC-SA was found to be excellent (a = .96; Schnabel et al., 2015; Schnabel & Kelava, 2013a,b). When combined with collaborative test feedback, TMIC even served as a brief intervention for students going abroad (Schnabel & Kelava, 2013a,b; Schnabel et al., in press).

The Brazilian test version was developed in two steps. First, we checked for construct and method bias of the instrument (Van de Vijver & Leung, 1997). Specifically, we wanted to ensure that the construct being measured did not depend on specific aspects of one culture and that biases in scores are minimized. Second, the process of translation and back-translation (Brislin, 1970) was used by involving two bilingual translators with extensive knowledge of the source and target language. A third expert reviewed the final version.

2.2.2. Cultural intelligence scale (CQS)

The CQS (Van Dyne et al., 2008) is based on the aforementioned concept of CQ. With a total of 20 self-report items the four dimensions of CQare assessed on a 7-point Likert scale. A German and a Brazilian version of the CQS were created applying the process of translation and back-translation (Brislin, 1970). We selected the CQS for the purpose of validation, as it is the only instrument that aims to measure a malleable aspect of intercultural competence.

2.2.3. External criteria

Previous studies were considered during the formulation of those items (Bhawuk & Brislin, 1992; Koester &Olebe, 1988; Schacher, 2011). Each of the three external criteria was included in the survey with one item using forced choice answer format ("Do you privately or professionally deal with different cultures?", "Have you ever taken part in an intercultural training?", and "What was the longest time that you ever spend abroad at a stretch?").

2.3. Data analysis

The online survey software Unipark EFS Survey (QuestBack GmbH, 1999-2012) was used for collecting data. We conducted the data analysis with the statistical software Mplus for Windows (Version 7.1; Muthén & Muthén, 1998-2012) and IBM SPSS Statistics for Windows (Version 22.0; IBM, 2013). Confirmatory factor analysis, multigroup SEM-comparisons, and correlations were computed with Mplus (Muthén & Muthén, 1998-2012). IBM SPSS Statistics (IBM, 2013) was used to analyze descriptive statistics.

3. Results

The Results section is structured as follows: First, we describe how the TMIC-S was developed. Second, the model fit as well as the psychometric properties of all six TMIC-S dimensions for the German and Brazilian test versions are presented separately. Third, we report results of MG-CFA along with model comparisons that outline measurement invariance of the German and Brazilian TMIC-S. Fourth, we provide evidence for construct and criterion validity of the TMIC-S. The results part closes with exploratory comparisons of the Brazilian and German samples concerning their intercultural competence.

3.i. Development of the TMIC-S

The conceptual starting point for TMIC-S was the six second-order factor model described before. To select facets for the TMIC-S, we followed three main strategies: (a) we chose facets that in our reading of the literature are very influential in intercultural competence, (b) we excluded facets that could have been specific for certain culture dimensions, and (c) we decided on those six competencies that functioned together as an intercultural competence model in both cultures, thus indicating configural invariance. Concerning the first strategy we considered existing research results that found empathy (e.g., Fantini&Tirmizi, 2006; Koester and Olebe, 1988; Ruben, 1976), cultural awareness (e.g., Chen&Starosta, 2000; Thomas, Kammhuber, & Layes, 1997; Triandis, 1977), as well as broad cultural knowledge acquisition (e.g., Deardorff, 2004; Van Dyne et al., 2008) to be crucial characteristics of an individual in the intercultural context. Moreover, we clearly aimed at creating a competence measure that might hold value across cultures. Therefore, the second strategy consisted of excluding competences that have the potential to be driven rather by cultural values than by ability. We considered clarity in communication to be influenced by communication style (Hall & Hall, 1983; Thomas, 1991), building trusting relationships and building professional networks by task- versus relationship-orientation (Bass, 1990; Thomas, 1991), and integration in groups by

individualism versus collectivism (Hofstede & Bond, 1984). Moreover, problem solving is assumed to be affected by cultural standards regarding analytical thinking, solution-orientation, and pragmatism (Thomas, 2010). Joy and Kolb (2009) found that in-group collectivism, institutional collectivism, uncertainty avoidance, future orientation, gender egalitarianism, and assertiveness have an impact on learning styles. A qualitative study by Kong (2014) with Chinese participants about the cross-cultural generalizability of the TMIC yielded similar results. Kong (2014) also found that using a foreign language, which is not perfectly spoken, was evaluated as an incompetent action. However, this phenomenon might be specific for the Chinese culture. In considering these findings, we decided to include 11 TMIC facets in the statistical analysis of a cross-culturally valid TMIC-S version. Because TMIC combines continuous variables (Likert scale) with ordinal variables (SJT) the WLSMV estimator (Muthén, du Toit, & Spisic, 1997) was selected. The WLSMV is a robust weighted least square estimator. It applies a diagonal weight matrix (Muthén & Muthén, 1998-2012). In using the WLSMV estimator factor loadings, structural coefficients, and robust standard errors are computed more accurately when compared to other estimators such as ML and MLR (Li, 2014). Multigroup confirmatory factor analysis (MG-CFA) in the German and Brazilian samples did not lead to a sufficient fit, x2(3670, N = 1496) = 6700.34, p<.001; x2/df =1.83; RMSEA= .033, 90% CI = [.032, .034]; WRMR= 1.801; CFI = .865; TLI = .857. By the help of the third strategy, we therefore further analyzed which of the remaining 11 facets work best together in a six-factor solution, which was theoretically derived from the second-order factor model by Schnabel et al. (2015). Accordingly, we step-wise excluded items with low loadings. We started with four negatively poled items coming from the factors cultural identity awareness, socializing, and willingness to use a foreign language. Two of the four self-appraisal items measuring willingness to use a foreign language were negatively poled. Therefore, we decided to exclude this whole factor. However, the fit was still not satisfactory enough, x2(2878, N = 1496) = 5459.67, p<.001; x2/df=1.90; RMSEA = .035, 90% CI = [.033, .036]; WRMR = 1.768; CFI = .873; TLI = .864. As a consequence, we excluded enabling productive collaborations, flexibility in communication, and cultural identity awareness, which consistently led to higher fit indices.

The final TMIC-S version included the following intercultural competence facets: sensitivity in communication, cross-cultural information seeking, socializing, cultural identity reflection, goal setting, and mediation of interests. The MG-CFA model with 25 self-report items as well as six situational judgment items fitted the data fairly well, x2(838, N =1496) = 1742.01, p<.001; x2/df=2.08; RMSEA=.038, 90% CI = [.035, .040]; WRMR=1.601; CFI = .908; TLI = .898. A short description of the six facets, example self-report items in German, Portuguese, and English as well as all six situational judgment items in English can be found in the Appendix A.

3.2. TMIC-S model fit and descriptive results

To analyze the final TMIC-S factorial model in more detail confirmatory factor analysis (CFA) was used. Satisfactory fit indices were attained in the German sample, x2(419, N =1037) = 824.10, p< .001; x2/df = 1.97; RMSEA = .036, 90% CI = [.033, .040]; WRMR = 1.095; CFI = .913; TLI = .904, as well as in the Brazilian sample, x2(419, N = 769) = 919.35, p <.001; x2/df = 2.19; RMSEA = .039, 90% CI = [.036, .043]; WRMR=1.168; CFI = .902; TLI = .892. Factor loadings for both groups can be found in Table 2. For the self-report items the cut-off value for accepted loadings was set at .40. This was reached in both groups. The lowest loading can be found for the factor mediation of interest (item MI4 = .62) in the German group and for the factor goal setting (item GS3 = .51) in the Brazilian Group. For the situational judgment items lower loadings were allowed when significant. In both samples the item SC-SJT loaded in a weak, yet significant manner on the factor sensitivity in communication.

Internal consistencies ranged from acceptable to good (see Table 2). Cronbach's alpha was overall a bit lower in the Brazilian sample.

3.3. Measurement invariance of the TMIC-S in Germany and Brazil

Measurement invariance (Meredith, 1993) of the TMIC-S was investigated using MG-CFA in Mplus. Three MG-CFAs with varying (nested) parameter restrictions were computed to test for measurement invariance. The WLSMV estimator was used throughout and adjusted chi-square difference tests were applied (Asparouhov and Muthén, 2006). In the configural invariance model all parameters were freely estimated in each group, whereas factor loadings were held equal in both groups for the weak factorial invariance model (metric invariance). To investigate scalar invariance intercepts of the self-report items and thresholds of the situational judgment items were also restricted. The most common procedure of comparing measurement invariance models is to look either at differences in the Comparative Fit Index (CFI) or in the Root Mean Square Error of Approximation (RMSEA; Chen, 2007; Marsh, Nagengast, & Morin, 2013) with the latter one controlling for parsimony (Marsh, 2007). Using the chi-square difference test is also a frequently employed, yet possibly problematic procedure due to its sensitivity to sample size (Bentler&Bonett, 1980;Joreskog, 1993; Milfont& Fischer, 2010). In the present measurement invariance analysis we followed the aforementioned recommendation of using differences in the RMSEA, CFI, and TLI. Hereby, models with a ARMSEA <.015, ACFI <.010, and ATLI <.010 were favored (Chen, 2007). Models were also compared with a chi-square difference test. Due to the large number of comparisons made, a Bonferroni-Holm-correction (Holm, 1979) was applied. An alpha level of .01 was used. As can be seen in Table 3, some Ax2 values were significant, yet difficult to interpret given our large sample sizes. Values of ARMSEA and ATLI pointed clearly to scalar invariance. Finally, values of ACFI showed an unexpected pattern in that the CFI value increased from configural invariance to metric invariance and decreased with the transition to scalar invariance. However, given that the CFI value of the configural and

Table 2

Loadings, descriptive statistics, and reliabilities for the German and Brazilian TMIC-S (CFA Model).

Competence Item Loadings SE M SD a


Sensitivity in communication SC1 .70 .70 .023 .025 4.80 4.44 1.01 1.05 .86 .77

SC2 .74 .61 .021 .025 4.10 3.94 0.96 1.10

SC3 .76 .66 .021 .024 4.53 4.42 0.86 1.02

SC4 .77 .64 .021 .025 4.63 4.40 0.95 1.06

SC5 .85 .71 .020 .022 4.48 4.44 0.96 0.99

SC6 .72 .64 .023 .026 4.43 4.44 0.91 1.07

SC-SJT .20 .10 .045 .051 2.37 2.09 0.89 0.97

Goal setting GS1 .75 .80 .028 .030 5.00 4.56 0.88 1.00 .78 .70

GS2 .76 .69 .030 .029 4.86 4.65 0.92 1.01

GS3 .82 .51 .028 .035 4.92 4.51 0.93 1.05

GS4 .67 .60 .032 .033 4.69 4.59 0.91 1.02

GS-SJT .26 .11 .049 .052 3.36 3.34 0.82 0.86

Information seeking 1S1 .69 .63 .031 .035 5.02 4.71 1.10 1.03 .81 .68

1S2 .82 .79 .028 .029 4.37 4.54 1.21 1.17

1S3 .89 .77 .030 .033 4.97 5.12 1.02 0.99

1S-SJT .53 .29 .044 .059 2.96 2.78 1.21 1.31

Mediation of interests M11 .74 .67 .028 .025 4.41 4.15 0.94 1.18 .72 .65

M12 .71 .63 .027 .027 4.56 4.32 1.00 1.17

M13 .63 .58 .029 .027 4.40 4.27 0.88 1.02

M14 .62 .66 .029 .026 4.63 4.30 0.91 1.08

M1-SJT .23 .14 .061 .054 3.63 3.49 0.53 0.73

Cultural identity reflection C1R1 .83 .82 .025 .027 4.35 4.17 1.32 1.23 .84 .73

C1R2 .65 .77 .033 .029 4.39 4.24 1.27 1.20

C1R3 .68 .71 .028 .031 4.20 4.19 1.23 1.26

C1R4 .72 .62 .029 .034 4.20 4.37 1.34 1.34

C1R-SJT .40 .17 .049 .055 3.10 2.82 1.08 1.23

Socializing SZ1 .81 .79 .027 .026 4.56 4.21 1.05 1.19 .78 .77

SZ2 .85 .66 .024 .033 4.05 3.65 1.19 1.31

SZ3 .84 .65 .026 .030 4.69 4.22 0.98 1.24

SZ4 .85 .80 .024 .027 4.14 3.99 1.16 1.28

SZ-SJT .27 .27 .027 .047 3.45 3.10 0.81 0.86

Note. G = Germany; B = Brazil; SC = Sensitivity in communication; GS = Goal setting; IS »Information seeking; Ml = Mediation of interests; ClR = Cultural identity reflection; SZ = Socializing.

Table 3

Comparing three levels of measurement invariance for the German and Brazilian TMIC-S Version.


Configurai invariance 1742.01* 838 .038 [.035, .040] .908 .898

Metric invariance 1678.93* 863 46.874(25)* .036 [.033, .038] .002 .917 .009 .910 .012

Scaiar invariance 1802.26* 882 241.12(19)* .037 [.035, .040] .001 .906 .011 .901 .009

Note. *p <.01; G = German sample (N = 1037); B = Brazilian sample (N = 769); cut-off values for measurement invariance are: Bonferroni-Holm corrected p <.01 for Ax2, ARMSEA < .015, ACFI < .010, and ATLI < .010 (Chen, 2007; Holm, 1979)

scalar invariance model were close to each other, we interpreted the ACFI values as essentially supporting scalar invariance. All in all, we found considerable support for scalar invariance.

3.4. Divergent validity of the TMIC-S in Germany and Brazil

In the German sample, a CFA model with 10 factors, four CQS factors and six TMIC-S factors, showed a satisfactory fit, x2(1179, N = 313) = 1492.42, p<.001; x2/df=1.27; RMSEA=.029, 90% CI = [.024, .034]; WRMR = 0.858; CF1 = .914; TL1 = .907. In the Brazilian sample, most fit indices reached an acceptable level with CFI and TLI bordering the recommended value of .90 (Bentler & Bonett, 1980), x2(1179, N =769) = 1806.90, p<.001; x2/df =1.53; RMSEA=.026, 90% CI = [.024, .029]; WRMR= 1.036; CFI = .894; TLI = .885. Overall, this supports divergent validity of the TMIC-S in Germany and Brazil.

3.5. Exploratory comparison of the TMIC-S facets in the two samples

As scalar invariance was established for the German and Brazilian TMIC-S versions, differences in latent means on the intercultural competence dimensions could be tested. For the following latent factors higher means were found for the German group in comparison to the Brazilian: Sensitivity in communication, M = 0.17, SEM = 0.05; x2(1) = 15.34, p< .001, d = 0.14, goal setting, M = 0.40, SEM = 0.06; x2(1) = 43.90, p< .001, d = 0.28, mediation of interests, M = 0.36, SEM = 0.07; x2(1) = 30.50, p<.001, d = 0.22, and socializing, M = 0.38, SEM = 0.05; x2(1) = 55.43, p<.001, d = 0.32. The two samples did not differ in

Table 4

Correlation of the Latent TMIC-S Factors with the Four Dimensions of the Cultural Intelligence Scale in the German and Brazilian Samples

1 2 3 4 5 6 7 8 9 10

1. Metacognitive CQ(MC) - .64' .81' .62' .49' .30' .38' .58' .51' .40'

2. Cognitive CQ (COG) .77' - .69' .59' .50' .25' .40' .58' .48' .39'

3. Motivational CQ(MOT) .83' .81' - .76' .56' .42' .43' .60' .64' .51'

4. Behavioral CQ(BEH) .73* .73' .76' - .56' .33' .32' .59' .37' .55'

5. Sensitivity in communication (SC) .57' .31' .43' .34' - .44' .38' .73' .37' .39'

6. Goal setting (GS) .30' .35' .37' .40' .32' - .50' .42' .22' .31'

7. Information seeking (IS) .18' .20' .32' .18' .09 .12 - .34' .35' .32'

8. Mediation of interests (MI) .57' .32' .44' .41' .89' .24' .07 - .39' .62'

9. Cultural identity reflection (CIR) .44' .56' .78' .50' .26' .20' .36' .26' - .43'

10. Socializing (SZ) .34' .31' .37' .51' .25' .24' .14' .31' .31' -

Note. Intercorrelations for the Brazilian participants (n = 769) are presented above the diagonal, and intercorrelations for the German participants (n=313) are presented below the diagonal. * p<.01.

information seeking, M = 0.00, SEM = 0.06; x2(1) = 1.00, p = .32, d = 0.00, and cultural identity reflection, M = 0.05, SEM = 0.05; X2(1) = 1.76, p = .19, d = 0.04. It can be concluded that there were small, yet for some aspects significant cross-cultural differences showing higher scores for German participants.

3.6. Construct validity of the German and Brazilian TMIC-S

We investigated construct validity by the help of the CQS (Van Dyne et al., 2008). First, measurement invariance of the CQS was examined in the German and Brazilian samples. Second, correlations between CQS dimensions and TMIC-S factors were computed. Third, divergent validity was tested via CFA of a model with factors of both instruments.

A multigroup CFA without parameter restrictions (configural invariance) revealed a good model fit of the cultural intelligence model in the German and Brazilian samples, x2(328, N =1082) = 588.26, p<.001; x2/df=1.79; RMSEA=.050, 90% CI = [.044, .057]; SRMR = .049; CFI = .939; TLI = .929. The following fit indices were found for the model in which factor loadings were held invariant between the German and Brazilian samples, x2(344, N = 1082) = 607.16, p<.001; x2/df =1.77; RMSEA = .049, 90% CI = [.043, .056]; SRMR = .056; CFI = .938; TLI = .931. The delta coefficients ARMSEA = .001, ACFI = .001, and ATLI = .002 as well as the nonsignificant chi-square difference test for the configural invariance and the metric invariance models, x2(16) = 18, p = .32, indicated metric invariance of the CQS in this study.

Table 4 shows the intercorrelations of all four CQS factors and all six TMIC-S factors in the German and Brazilian samples. Correlations between the CQS factors and TMIC-S factors were mostly moderate, r =.30 to .64. Only one high correlation between motivational CQ and cultural identity reflection was found in the German sample, r= .78. Lowest correlations were reached for meta-cognitive CQas well as behavioral CQwith information seeking of the German TMIC-S, r = .18, and cognitive CQ with goal setting in the Brazilian TMIC-S, r= .25. To investigate if correlations of the six TMIC-S factors with the four CQS factors are the same in both samples, two further models were computed: In one model all parameters were freely estimated and in the other model the 24 correlation pairs were held equal in both samples. Despite the significant chi-square difference between the models, x2(24) = 45.49, p = .005, values of ARMSEA = .001, ACFI = .007, and ATLI = .008 strongly suggest that correlations between the TMIC-S and the CQS are equal in both samples. Altogether, results support hypothesis 2a.

Table 4 also shows that many relations between single CQS factors and TMIC-S factors were as strong as predicted in hypotheses 2b-c. Supportive evidence was found for goal setting, information seeking, cultural identity reflection, and socializing. As hypothesized, sensitivity in communication was strongly related to motivational CQin both samples. However, behavioral CQwas only most strongly related to sensitivity in communication in the Brazilian sample. In the German sample metacognitive CQ showed the strongest relation to sensitivity in communication. A similar pattern was found for mediation of different interests. As stated in the hypothesis, mediation of different interests correlated strongly with motivational CQ and behavioral CQin the Brazilian sample. Yet, in the German sample mediation of different interests was most strongly related to metacognitive CQ.

3.7. Criterion validity of the German and Brazilian TMIC-S

Table 5 gives an overview about the multigroup SEM comparisons in the German and Brazilian samples. For each external criterion two groups were compared in both samples: (a) involved versus not involved in intercultural matters (hypothesis 3a), (b) experience abroad longer than 3 months versus shorter than 3 months (hypothesis 3b), and (c) previously versus never participated in an intercultural training (hypothesis 3c). In the German sample individuals with more intercultural involvement had a higher sensitivity in communication, were better information seekers and mediators as well as more advanced in cultural identity reflection and socializing. Brazilians who stated that they privately or professionally deal with other cultures outperformed those with less intercultural involvement in all intercultural competences except for goal setting, thereby supporting hypothesis 3a. Germans who stayed more than three months abroad before had higher

Table 5

Multigroup SEM comparisons for all six factors and the external criteria intercultural training participation, intercultural involvement, and experience abroad in the German and Brazilian samples.

AX2 d Mean Standard error

Factor Subgroup G B G B G B G B

SC Training 0.96 9.58* 0.11 0.34 0.09 0.30 0.09 0.10

Involvement 8.23* 10.17* 0.31 0.37 0.25 0.36 0.09 0.11

Abroad >3 months 1.08 15.64* -0.02 0.37 -0.02 0.37 0.08 0.10

GS Training 1.12 1.13 0.04 0.08 0.03 0.04 0.09 0.06

Involvement 2.84 2.25 0.20 0.12 0.16 0.12 0.09 0.11

Abroad > 3 months 0.05 0.20 0.02 -0.04 0.02 -0.02 0.08 0.05

1S Training 4.40* 0.82 0.24 0.10 0.21 0.10 0.10 0.11

Involvement 21.64* 6.50* 0.55 0.26 0.49 0.28 0.10 0.12

Abroad > 3 months 6.34* 1.58 0.25 -0.07 0.25 -0.07 0.10 0.10

M1 Training 4.08* 16.23* 0.21 0.44 0.17 0.39 0.09 0.10

Involvement 7.35* 13.63* 0.39 0.47 0.31 0.43 0.09 0.10

Abroad > 3 months 0.29 17.07* 0.05 0.43 0.04 0.39 0.08 0.09

C1R Training 80.50* 14.33* 1.03 0.42 0.91 0.37 0.10 0.10

Involvement 372.69* 33.32* 1.24 0.68 0.99 0.60 0.09 0.10

Abroad > 3 months 39.96* 15.17* 0.64 0.38 0.58 0.35 0.09 0.09

SZ Training 15.86* 47.47* 0.44 0.76 0.39 0.74 0.10 0.11

Involvement 35.44* 41.95* 0.64 0.72 0.57 0.64 0.10 0.10

Abroad > 3 months 24.62* 30.56* 0.46 0.53 0.46 0.48 0.10 0.09

Note. Subgroups involve: training versus no training, involvement versus no involvement, and experiences abroad more than 3 months versus less than 3 months. Ax2 refers to the delta chi square test with one degree of freedom; d refers to Cohen's d. In the German sample SEM-comparisons were computed without item Ml-SJT, because one of the subgroups did not contain value 1 of the categorical variable. German sample: N=724, training = 237, no training = 487, involvement = 456, no involvement = 268, >3 months = 420, <3 months=304; Brazilian sample: N=769, training = 558, no training = 211, involvement = 604, no involvement = 164, >3 months = 436, <3 months=333, G = German sample; B = Brazilian sample; SC = Sensitivity in communication; GS = Goal setting; IS = Information seeking; Ml = Mediation of interests; ClR = Cultural identity reflection; SZ = Socializing. * p<.05.

Table 6

Comparing three levels of measurement invariance of the TMlC-S for the male and female samples in Germany and Brazil.


Configurai invariance G 1334.95* 838 .034 [.031, .037] .921 .913

B 1354.95* 838 .040 [.036, .044] .890 .878

Metric invariance G 1311.31* 863 26.62(25), p = .38 .032 [.028, .035] .002 .929 .008 .923 .010

B 1333.92* 863 44.24(25), p <.05 .038 [.034, .042] .002 .900 .010 .892 .014

Scalar invariance G 1357.52* 882 83.06(19), p >.001 .032 [.029, .036] .000 .925 .004 .921 .002

B 1372.57* 882 63.78(19), p >.001 .032 [.029, .036] .000 .896 .004 .890 .002

Note. G = German sample (N = 1037); B = Brazilian sample (N = 769); cut-off values for measurement invariance are: Bonferroni-Holm corrected p <.01 for Ax2, ARMSEA< .015, ACFI <.010, and ATLI < .010 (Chen, 2007; Holm, 1979).

means in information seeking, cultural identity reflection, and socializing. Individuals in the Brazilian sample who reported experiences abroad of longer than 3 months were more sensitive in communication, capable of mediating different interests, vigor in reflecting upon their cultural character, and active in building intercultural relationships. Thus, hypothesis 3b is partially supported.

Additionally, German training participants were more sociable and advanced in cultural identity reflection. They also had higher values in information seeking and mediation of different interests. In the Brazilian sample intercultural training participation had a positive influence on the latent means of sensitivity in communication, mediation of interests, cultural identity reflection, and socializing. Therefore, hypothesis 3c is supported at large.

3.8. Exploratory analysis of gender-differences in the TMIC-S factors

To explore gender-differences in the Brazilian and German samples, we first analyzed measurement invariance in the sex groups for each sample separately (see Table 6). In the German sample, all delta coefficients pointed to metric invariance. Scalar invariance was supported by ARMSEA, ACFI, and ATLI. In the Brazilian sample, A/2, ARMSEA, and ACFI showed metric invariance. In terms of scalar invariance the same pattern was found as in the German sample, with all delta coefficients except for A/2 being in favor for invariant factor loadings and intercepts. Altogether, sufficient evidence was found to compare latent means between gender groups.

In the German as well as in the Brazilian sample female study participants outperformed male study participants in (a) sensitivity in communication, A/2s(1) = 12.54 and 23.82, ps < .001; AMs = 0.27 and 0.44, ASEs = 0.08 and 0.10, (b) information seeking, A/2s(1) = 23.31 and 11.02, ps<.001; AMs = 0.43 and 0.32, ASEs = 0.09 and 0.10, (c) cultural identity reflection, A/2s(1) = 30.17 and 8.31, ps<.001 and .01; AMs = 0.40 and 0.24, ASEs = 0.07 and 0.09, and (d) socializing, A/2s(1) = 26.42

and 8.30, ps<.001 and .01; AMs = 0.38 and 0.25, ASEs = 0.07 and 0.09. No significant differences were found for (a) goal setting A/2s(1) = 3.51 and 0.66, ps = .06 and .42; AMs = 0.13 and 0.08, ASEs = 0.07 and 0.09 and (b) mediation of different interests A/2s(1) = 0.24 and 0.10, ps = .62 and .76; AMs = - 0.04 and 0.03, ASEs = 0.08 and 0.09.

4. Discussion

The goal of the study was to validate a short version of the TMIC (TMIC-S) that can be applied in Germany and Brazil and that can serve as a tool to investigate intercultural development. As in Schnabel et al.'s (2015) model, intercultural competence (ICC) is understood as a behavioral orientation that can be acquired. Self-report as well as situational judgment items were integrated in the TMIC-S to follow the multimethod approach of the original TMIC version.

4.1. Summary of important results

For the six-factor model of the German and Brazilian TMIC-S an acceptable fit to the data, good psychometric properties, invariance of the factor structure, loadings, and intercepts/thresholds as well as divergent validity were established. Consequently, we reached our overall aim of this study. As expected most TMIC-S factors positively correlated with the four dimensions of the CQS in Germany and Brazil. Strongest correlations were mostly found for the predicted TMIC-S and CQS factors. Moreover, results suggested that the 24 correlation pairs were equal in both groups. These results underpin hypotheses 2a-c. Concerning the external criteria slightly higher means in most competences of the TMIC-S were found for Brazilians and Germans who attended an intercultural training in the past, who were professionally or privately involved with intercultural matters, and who stayed abroad for more than 3 months before. Together, this partially confirms hypotheses 3a-b.

4.2. Implications

Our study has four major implications. First, in establishing measurement invariance this study substantially adds to research about intercultural competence as it leads the way to a core set of intercultural competence in Germany and Brazil. This finding supports the potential of intercultural competence as a universal concept as measured by the TMIC-S.

Second, factor loadings for some situational judgment items especially in the Brazilian group were below the usual cut-off value of .30 (Kline, 1994). Bledow and Frese (2009) stated that the correlation between the self-report and the situational judgment method could be expected to be no more than moderate. Clearly, both methods should measure the same competence. However, self-report items typically assess the self-concept of a person and situational judgment items refer to behavioral preferences. Thus, both methods are complementary rather than mutually exclusive or in competition. Also, Schnabel et al. (2015) found a moderate correlation between all 75 self-report items and all 17 situational judgment items in the TMIC. The low to moderate relations between both methods are in line with the lower loadings of the situational judgment items on the homogenous factor, because factor loadings can be understood as correlations of an item with a factor (Kline, 1994). As there is a strong need for instruments that integrate two methods to measure ICC (Bolten, 2007b; Deardorff, 2004; Gabrenya et al., 2011; Leung et al., 2014), we decided to keep the situationaljudgment items despite the lower loadings.

Third, motivational CQand cultural identity reflection were strongly related in the German sample. Van Dyne et al. (2008) defined motivational CQ as a special kind of intrinsic motivation and self-efficacy that is directed towards understanding cultural differences. That very cognitive process is required when individuals reflect on their own or other's cultural identity, especially when interacting with people from different cultures or living abroad (Thomas, 2003). In the CQS motivational CQ is assessed with items such as "I enjoy living in cultures that are unfamiliar to me." and "I enjoy interacting with people from different cultures." (Ang et al., 2007), which could explain the relatively high correlation of motivational CQ and cultural identity reflection. Moreover, almost all CQS dimensions were most strongly related to cultural identity reflection, mediation of interests, and sensitivity in communication in both samples. It seems that, in contrast to socializing, information seeking, and goal setting, the abovementioned TMIC-S factors pertain to more than one CQS dimension. Especially mediation of different interests apparently requires metacognitive processes, learning about specific cultural characteristics, openness towards cultural exposure as well as competent performance of nonverbal and verbal communication at the same time.

Fourth, gender differences were consistent in both samples and in one direction, that is women outperformed men in four of six TMIC-S factors. For sensitivity in communication and socializing our results are in line with previous research findings drawn from outside the intercultural competence domain. Women are often found to be more empathic (Eisenberg & Lennon, 1983), better decoders of nonverbal communication (Hall, 1978), more agreeable, and extraverted (Costa, Terracciano, & McCrae, 2001; Feingold, 1994; Schmitt, Realo, Voracek, & Allik, 2008). Surprisingly, in our samples women were also more competent in planning a trip abroad as well as in adequately seeking and evaluating interculturally relevant information from various sources. Additionally, women tended to reflect their cultural identity more often and to be aware of their own values and norms that influence their behavior. However, more research is needed to replicate those findings.

4.3. Limitations

The present study has three specific limitations. First, we included a restricted number of constructs in the present study. TMIC (Schnabel et al., 2015) with its two methods and 92 items has many advantages, but it is also a rather long instrument. To avoid participants' fatigue we decided on a selected number of external criteria as well as related scales. Clearly, there are additional constructs such as personality traits or cognitive abilities, which are worth to investigate in future studies of TMIC-S.

Second, we only included two diverse cultures (Western versus Latin-American). This should be taken as a conceptual starting point for further cross-cultural comparisons, specifically those focused on comparing Western with Asian cultures. Especially, task performance in a situational judgment test seems to differ across cultures, demographic characteristics, and context (Whetzel, McDaniel, & Nguyen, 2008).

Third, so far no results were gained concerning the incremental validity of the TMIC-S when compared to existing instruments, such as the CQS. Through divergent validity we showed that the theoretical concepts of the TMIC-S and CQS differed. Yet, additional validity evidence in new samples is needed to further demonstrate the incremental validity of our measure.

4.4. Future research

As Schnabel et al. (2015) were the first to integrate situational judgment items in a self-report instrument to measure intercultural competence, more research is needed to understand the unique functioning of this method. What we know so far is that an SJT measures a specific aspect of intercultural competence, namely behavioral preferences (Bledow & Frese, 2009; Schnabel et al., 2015). SJTs show how an individual would most probably behave in a situation in which a specific competence is assumed to be central. Self-report scales in the field reflect an individual's intercultural self-efficacy, which is generally known to be the best predictor of behavior (Stajkovic & Luthans, 1998). Yet, they are prone to social desirability (cf. Nederhof, 1985). SJTs are overall less sensitive to faking tendencies (McDaniel et al., 2006). Future studies could focus on how the situational judgment items of the TMIC-S might detect those tendencies when compared with the self-report items. Central could be the question, if individuals most likely behave as competent as they think they do. Thereby, there would be no need to use an additional social desirability scale. Two further questions could be interesting concerning the SJT: (a) Is there a difference in measurement invariance across cultures for self-report measures and SJTs? (b) Which consequences would a manipulation of response instructions (behavior-related versus knowledge-related; McDaniel et al., 2006) have on an individual's TMIC-S result?

Additionally, we see a strong need to focus research more on cross-cultural validation of scales in the field to increase understanding about how intercultural competence and related constructs function across borders. For example, to examine construct validity of different language versions of an instrument related scales have to be available in those languages as well. Cultural intelligence was hypothesized to be related to intercultural competence, but the CQS was neither available in German nor in Portuguese. Thus, we had to translate the CQS. Fit indices showed that CQS worked quite well in different languages. However, we also found factor intercorrelations of r > .80, which queries the proposed latent variables structure. Matsumoto and Hwang (2013) recently argued that there might be a general CQfactor instead of four distinct ones. Clearly, this has to be empirically investigated in the future.

A further aspect concerning the validity, specifically the criterion validity, should be mentioned here. Although it can be argued that intercultural involvement and experience abroad are more related to learning by experience, whereas participation in an intercultural training explicitly fosters knowledge acquisition, no clear pattern was found in terms of which TMIC-S competences are specifically responsive to one of the three external criteria. Intercultural training is a very wide field with lots of different content domains and methods (Landis et al., 2004). However, we did not ask for those differences explicitly. Future studies should differentiate training methods to find out whether training content is tight to certain TMIC-S factors, for example cultural knowledge to cultural identity reflection. This would then offer a further possibility to show divergent validity of the TMIC-S.

Schnabel et al. (in press) proposed the TMIC to support the development processes of students or employees who will spend a longer period abroad or who intensively interact with individuals from other cultures in their home countries. Like in the original version TMIC-S incorporates competences that can be developed through interventions such as training, coaching, or counseling sessions. This raises the potential to create a holistic approach to intercultural competence development. The process might look as follows: (1) Analyzing the status quo with TMIC-S, (2) defining development goals in a (therapeutic) test feedback session, (3) working on intercultural competences in training, coaching, or counseling sessions and (4) analyzing the change in TMIC-S results. Consequently, there would be a need to draft innovative development concepts and to evaluate them by means of longitudinal studies with TMIC-S.

5. Conclusion

The newly developed TMIC-S showed a satisfactory model fit and good psychometric properties in a German and in a Brazilian sample. As factor loadings and intercepts are invariant across groups, TMIC-S can be used to compare Germans and Brazilians concerning their intercultural competence in the future. The TMIC-S deals with the discussion on method effects in the assessment of intercultural competence by integrating two different assessment methods. Thereby, TMIC-S allows

evaluations about the self-concept and the behavioral preferences of an individual in an intercultural context. At the same time, the TMIC-S is less time-consuming than the long version and thus reduces fatigue during the assessment process. The TMIC-S can be well applied during assessment or training settings with individuals who wish to interact in an intercultural context.

Appendix A.

Description of the factors with item examples in German, Portuguese, and English.



Example items (SA)

SJT English with coding (1) = lowest value to (4) = highest value

SC To put oneself in the position ofanother person during communication in order to understand him or her better; high sensibility for verbal and nonverbal communication aspects

6SA + 1 SJT

Purposeful collection of information about a foreign country or another culture

3 SA +1 SJT G

Ich weiß, wie sich andere Personen fühlen, ohne dass sie es mir sagen.

Eu sei como as outras pessoas se sentem, sem que elas tenham que me dizer I know how other people feel without them having to tell me Bei der Planung einer Reise ins Ausland, nutze ich unterschiedliche Informationsquellen

Durante o planejamento de uma viagem ao exterior, eu uso varias fontes de informacao When planning a trip abroad l use various sources of information

You are working together with a foreign delegation on a project. First of all, a meeting takes place in order to discuss the further progression of the project and to set important objectives. You have the impression that the project manager does not directly address which areas they will focus on during the project or talk about which points are most important to them. How are you most likely to behave? l wait to get more information afterthe discussion. (1) l keep quiet until the end of the discussion and then ask the project manager to summarize the most important points again. (2) l listen attentively in order to recognize what the most important points are. (3) l closely observe how the project manager formulates their points and acts during the discussion. (4)

You work for a company and you are going to be sent abroad for six months. How are you most likely to prepare for this? I prepare myself professionally and will get to know the culture when I get there. (1) I read about the basic rules of behavior on the internet. (2) I read a travel guide and look at a map in order to be able to cope when I get there. (3) I stock up on books about the culture, the country, and the language and also do some research on the internet. (4)

SZ Establishing and maintaining 4SA+1SJT G Ich nutze einen großen Teil

contact with people from other meiner Freizeit, um Kontakte

cultures quickly and easily zu pflegen

GS Having clear goals and being able to implement them consistently


Eu uso uma grande parte do meu tempo livre para manter contatos

I use a large part of my free time in order to cultivate contacts

Wenn ich mir etwas vornehme, realisiere ich dies gewöhnlich auch

MI Mediating between parties in 4SA + 1SJT order to achieve the greatest possible benefit from different approaches

P Se planejo algo, normalmente

também costumo realisar isso E When I plan something I

usually then go on to achieve my aim

G Ich bin gut darin, zwischen

Personen mit gegensätzlichen Interessen zu vermitteln

P Eu sou bom na mediacao entre

pessoas com interesses opostos E I am good at mediating

between people with conflicting interests

You move to a new city and do not yet know anyone. How are you most likely to behave in this situation?

I concentrate fully on work. (1) So that I don't feel lonely I have long phone calls with friends or family from my hometown during my free time. (2) I make an effort to be friendly to everyone I meet and therefore signal my interest in getting to know new people. (3) I try to make contact with people through various free time activities. (4)

You have made it your aim to successfully complete the project by the end of the next month. However, after a short time you realize that you have barely made any progress with your project. How are you most likely to behave in such a situation?

I delay completion of the project. (1) I concentrate on the parts of the project that are going well. (2) I work on eliminating the aspects that are responsible for the delay. (3) I define what I must achieve and when I must have achieved it in order to come closer to my aims. (4)

You have been sent by company headquarters to a subsidiary abroad where you are to manage a project. You quickly notice that the company headquarters follow quite different interests to the subsidiary abroad. What are you most likely to do? I recommend that company headquarters give up on the project as soon as possible. (1) I use all of my resources to implement the interests of company headquarters. (2) I analyze which interests have a higher priority and support the most important ones. (3) I put in a lot of effort to mediate between the interests of the subsidiary and company headquarters. (4)

CIR Intensively and constantly reflecting upon ones own cultural character

4SA + 1SJT G

Ich bemühe mich zu verstehen, inwiefern mein Verhalten kulturell geprägt ist

You begin a new position with a company based abroad. You find your new job very interesting and on the whole are satisfied. The only thing which bothers you are the new working hours which are much different to what you are used to in your working life. Your colleagues don't seem to be bothered. How are you most likely to behave? I simply accept the new working hours so that I am not viewed negatively. (1) I look for a way to get as near as possible to the working hours I would like. (2) I consider why the working hours are so different. (3) I think about why the working hours bother me and how I can deal with this in the future. (4)

Tento entender como o meu comportamento é determinado culturalmente

I make an effort to understand to what extent my behavior is shaped by culture

Note. F = Factor; SA = Self-appraisal, SJT = Situational Judgment Test; L=Language; G = German; P = Portuguese; E = English (the English items were only created for a better understanding of the content); SC = Sensitivity in communication; IS = Information seeking; SZ = Socializing; GS = Goal setting; Ml = Mediation of interests; ClR = Cultural identity reflection.

Ang, S., Van Dyne, L., Koh, C., Ng, K. Y., Templer, K.J., Tay, C., et al. (2007). Cultural intelligence: its measure and effects on cultural judgment and decision making, cultural adaptation and task performance. Management and Organization Review, 3,335-371. Arasaratnam, L. A., & Doerfel, M. L. (2005). Intercultural communication competence: identifying key components from multicultural perspectives.

International Journal of Intercultural Relations, 29,137-163. Asparouhov, T., & Muthén, B. (2006). Robust Chi Square Difference Testing with mean and variance adjusted test statistics. Mplus Web Notes: No. 10. May 26, 2006.

Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16,397-438. Bass, B. M. (1990). Bass&Stogdill's handbook of leadership: theory, research & managerial applications (3rd ed.). New York, NY: The Free Press. Behrnd, V., & Porzelt, S. (2012). Intercultural competence and training outcomes of students with experiences abroad. International Journal of Intercultural

Relations, 36, 213-223. Bennett, M.J. (1993). Towards ethnorelativism: a developmental model of intercultural sensitivity. In R. M. Paige (Ed.), Education for the intercultural

experience (2nd ed., pp. 21-77). Yarmouth, ME: Intercultural Press. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88,588-606. Bergemann, B., & Bergemann, N. (2005). Interkulturelle Management-Kompetenz: Anforderungen und Ausbildung [intercultural management competence:

requirements and education!. Heidelberg, Germany: Physica-Verlag. Bhawuk, D., & Brislin, R. (1992). The measurement of intercultural sensitivity using the concepts of individualism and collectivism. International Journal of

Intercultural Relations, 16, 413-436. Black, J. S., & Mendenhall, M. (1990). Cross-cultural training effectiveness: a review and a theoretical framework for future research. Academy of

Management Review, 15,113-136. Bledow, R., & Frese, M. (2009). A situational judgment test of personal initiative and its relationship to performance. Personnel Psychology, 62, 229-258. Bolten, J. (2007a). Was heißt Interkulturelle Kompetenz? Perspektiven für die internationale Personalentwicklung [What does intercultural competence mean? Perspectives for international personnel development]. In V. Künzer, & J. Berninghausen (Eds.), Wirtschaft als interkulturelle Herausforderung [Economy as an intercultural challenge] (pp. 21-42). Frankfurt am Main, Germany: IKO. Bolten, J. (2007b). Interkulturelle Kompetenz [Intercultural competence]. Erfurt, Germany: Thüringer Landeszentrale für politische Bildung. Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1,185-216. Brislin, R. W. (1981). Cross-cultural encounters. Face-to-face encounters. New York, NY: Pergamon.

Brodbeck, F. C., Frese, M., Ackerblom, S., Audia, G., Bakacsi, G., et al. (2000). Cultural variation of leadership prototypes across 22 European countries.

Journal of Occupational and Organizational Psychology, 73,1-29. Byram, M. (1997). Teaching and assessing intercultural communicative competence. Clevedon, United Kingdom: Multilingual Matters. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14,464-504. Chen, G. M., & Starosta, W.J. (2000). The development and validation of the intercultural communication sensitivity scale. Seattle, WA: Paper Presented at the Annual Meeting of the National Communication Association. Retrieved from <>


Cheung, F. M. (2009). The cultural perspective in personality assessment. In J. N. Butcher (Ed.), Oxford handbook of personality assessment (pp. 44-56). New

York, NY: Oxford University Press. Church, A. T. (2001). Personality measurement in cross-cultural perspectives. Journal of Personality, 69,979-1006. Costa, P., Jr., Terracciano, A., & McCrae, R. R. (2001). Gender differences in personality traits across cultures: robust and surprising findings. Journal of

Personality and Social Psychology, 81,322-331. Dauner, A. B. (2011). Die organisierende Funktion von Kompetenz—Entwurf eines zwei-Ebenen-Modells interkultureller Kompetenz [The organizing function of competence—draft of a two-level-model of intercultural competence]. Interculture Journal, ¡0(14), 25-46. Retrieved from <> Deardorff, D. K. (2004). The identification and assessment of intercultural competence as a student outcome of internationalization at institutions of higher

education in the United States (unpublished doctoral dissertation). Raleigh, NC: North Carolina State University. Deardorff, D. K. (2006). Policy Paper zur Interkulturellen Kompetenz [Policy paper about intercultural competence]. Gütersloh, Germany: Bertelsmann Stiftung. Deshpande, S. P., & Viswesvaran, C. (1992). Is cross-cultural training of expatriate managers effective: a meta analysis. International Journal of Intercultural

Relations, ¡6, 295-310. Earley, P. C. (1987). Intercultural training for managers: a comparison of documentary and interpersonal methods. Academy of Management Journal, 30,

685-698. Earley, P. C., & Ang, S. (2003). Cultural intelligence: individual interactions across cultures. Palo Alto, CA: Stanford University Press. Eignor, D. R. (2013). The standards for educational and psychological testing. Washington, DC: American Psychological Association. Eisenberg, N., & Lennon, R. (1983). Sex differences in empathy and related capacities. Psychological Bulletin, 94,100-131. European Union, 2014. Erasmus Mundus Programme. Retrieved from <>.

Fantini, A. E. (2009). Assessing intercultural competence: issues and tools. In D. K. Deardorff (Ed.), The SAGE handbook of intercultural competence (pp.

456-476). Thousand Oaks, CA: Sage. Fantini, A., & Tirmizi, A. (2006). Exploring and assessing intercultural competence (1) World Learning Publications. Retrieved from

<> Feingold, A. (1994). Gender differences in personality: a meta-analysis. Psychological Bulletin, ¡¡6,429-456. Fritz, W., Graf, A., Hentze, J., Möllenberg, A., & Chen, G. M. (2005). An examination

of Chen and Starosta's model of intercultural sensitivity in Germany and United States. Intercultural Communication Studies, ¡4(1), 53-64. Retrieved from



Fritz, W., Möllenberg, A., & Chen, G. M. (2002). Measuring intercultural sensitivity in different cultural contexts. Intercultural Communication Studies, ¡¡(2),

165-176. Retrieved from <> Gertsen, M. C. (1990). Intercultural competence and expatriates. The International Journal of Human Resource Management, ¡, 341-362. Gudykunst, W. B. (1994). Bridging differences: effective intergroup communication (2nd ed.). London, United Kingdom: Sage.

Gudykunst, W. B., Hammer, M. R., & Wiseman, R. L. (1977). An analysis of an integrated approach to cross-cultural training. International Journal of

Intercultural Relations, ¡,99-110. Gabrenya, W. K., Jr., Moukarzel, R. G., Pomerance, M. H., Griffith, H., & Deaton, J. (2011). A validation study of the Defense Language Office Framework for Cross-cultural Competence. Technical report. Defense Equal Opportunity Management Institute. Retrieved from <> Gawora, D., de Souza Ide, M. H., & Barbosa, R. S. (Eds.). (2011). Traditionelle Völker und Gemeinschaften in Brasilien. Lateinamerika-Dokumentationsstelle

[Traditional peoples and communities in Brazil. Latin-America documentation center]. In. Kassel, Germany: Kassel University Press. Gelfand, M. J., Imai, L., & Fehr, R. (2008). Thinking intelligently about cultural intelligence: the road ahead. In S. Ang, & L. Van Dyne (Eds.), Handbook of

cultural intelligence (pp. 375-387). New York, NY: Sharpe. Hall, E. T., & Hall, M. (1983). Hidden differences. Studies in international communication. How to communicate with the Germans. Hamburg, Germany: Gruner & Jahr.

Hall, J. A. (1978). Gender effects in decoding nonverbal cues. Psychological Bulletin, 85, 845-857. Hammer, M. R. (2008). The Intercultural Development Inventory (IDI): an approach for assessing and building intercultural competence. In M. A. Moodian (Ed.), Contemporary leadership and intercultural competence: understanding and utilizing cultural diversity to build successful organizations (pp. 245-259). Thousand Oaks, CA: Sage.

Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). The Intercultural Development Inventory: a measure of intercultural sensitivity. International Journal

of Intercultural Relations, 27, 421-443. Hofstede, G., & Bond, M. H. (1984). Hofstede's Culture Dimensions. An independent validation using Rokeach's value survey. Journal of Cross-Cultural

Psychology, ¡5, 417-433. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6,65-70. Retrieved from

<> Hooper, A. C., Cullen, M. J., & Sackett, P. R. (2006). Operational threats to the use of SJTs: faking, coaching, and retesting issues. In J. A. Weekley, & R. E. Ployhart (Eds.), Situational judgement tests: theory, measurement and application (pp. 205-232). London, United Kingdom: Lawrence Erlbaum Associates.

House, R.J., Hanges, P. J., Ruiz-Quintanilla, S.A., Dorfman, P. W., Javidan, M., Dickson, M. W., & GLOBE. (1999). Cultural influences on leadership: Project

GLOBE. In W. Mobley, J. Gessner, & V. Arnold (Eds.), Advances in global leadership (pp. 171-233). Stamford, CT: JAI Press. IBM. (2013). IBM SPSS Statistics (version 22.0 for Windows) [Computer software!. Armonk, NY: IBM.

Joy, S., & Kolb, D. A. (2009). Are there cultural differences in learning style? International Journal of Intercultural Relations, 33,69-85. Jöreskog, K. G. (1993). Testing structural equation models. In K. A. Bollen, & J. S. Long (Eds.), Testing structural equation models (pp. 294-316). Newbury Park, CA: Sage.

Kelley, C., & Meyers, J. (1995). The Cross-Cultural Adaptability Inventory (manual). Minneapolis, MN: National Computer Systems.

Kealey, D. J., & Ruben, B. D. (1983). Cross-cultural personnel selection: criteria, issues, and methods. In D. Landis, & R. W. Brislin (Eds.), Handbook of

intercultural training (vol. ¡): issues in theory and design (pp. 155-175). New York, NY: Pergamon Press. Kim, Y. Y. (1992). Intercultural communication competence: a systems-thinking view. In W. B. Gudykunst, & Y. Y. Kim (Eds.), Readings on communicating

with strangers: an approach to intercultural communication (pp. 371-381). New York, NY: McGraw-Hill. Kline, P. (1994). An easy guide to factor analysis. London, United Kingdom: Routledge.

Koester, J., & Olebe, M. (1988). The behavioral assessment scale for intercultural communication effectiveness. International Journal of Intercultural Relations, 12, 233-246.

Kong, W. (2014). тШШЛШМШШШ. {Culture-specificity of the Test

to Measure Intercultural Competence. A cross-cultural study concerning the response processes of Chinese and German test participants] (Unpublished master's thesis). Beijing, China: Beijing Foreign Studies University. Lambert, R. D. (Ed.). (1994). Educational exchange and global competence. In. New York, NY: Council on International Educational Exchange.

Landis, D., Bennett, J., & Bennett, M. (Eds.). (2004). Handbook of intercultural training. In. Thousand Oaks, CA: Sage.

Landis, D., Brislin, R., Swanner, G., Tseng, O., & Thomas, J. (1985). Some effects of acculturative training: a field evaluation. International Journal of Group Tensions, 15, 68-91.

Leung, K., Ang, S., & Tan, M. L. (2014). Intercultural competence. Annual Review of Organizational Psychology and Organizational Behavior, 1, 489-519. Li, C.-H. (2014). The performance of MLR, USLMV, and WLSMV estimation in structural regression models with ordinal variables (Unpublished doctoral

dissertation). East Lansing, MI: Michigan State University. Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: a review of recent research. Personnel Review, 37,426-441. Loboda, J. (2003). Entwicklung eines Tests zur Messung Interkultureller Handlungskompetenz (TIHK) [Development of a Test to Measure Intercultural

Competence] (Unpublished diploma thesis). Regensburg, Germany: Universitaet Regensburg. Marsh, H. W. (2007). Application of confirmatory factor analysis and structural equation modeling in sport/exercise psychology. In G. Tenenbaum, & R. C.

Eklund (Eds.), Handbook of sport psychology (3rd ed., 1, pp. 774-798). New York, NY: Wiley. Marsh, H. W., Nagengast, B., & Morin, A.J. S. (2013). Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity,

maturity, and la dolce vita effects. Developmental Psychology, 49,1194-1218. Martin, J. N. (1993). Intercultural communication competence: a review. In R. L. Wiseman, & J. Koester (Eds.), Intercultural communication competence (pp. 16-29). Newbury Park, CA: Sage.

Matsumoto, D., & Hwang, H. C. (2013). Assessing cross-cultural competence: a review of available tests. Journal of Cross-Cultural Psychology, 44, 849-873. Matsumoto, D., LeRoux, J. A., Iwamoto, M., Choi, J., Rogers, D., Tatani, H., & Uchida, H. (2003). The robustness of the Intercultural Adjustment Potential

Scale (ICAPS). International Journal of Intercultural Relations, 27,543-562. Matsumoto, D., & Van de Vijver, F.J. R. (Eds.). (2011). Cross-cultural research methods in psychology. In. New York, NY: Cambridge University Press. McCrae, R. R., Terracciano, A., & 78 members of the Personality Profiles of Cultures Project. (2005). Universal features of personality traits from the

observer's perspective: data from 50 cultures. Journal of Personality and Social Psychology, 88,547-561. McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. (2006). Situational judgment tests, response instructions, and validity: a meta-analysis.

Personnel Psychology, 60,63-91. McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment test: a review of practice and constructs assessed. International Journal of Selection and

Assessment, 9,103-113. McDaniel, M. A., & Whetzel, D. (2005). Situational judgment tests Paper Presented at the 29th Annual IPMAAC. In Conference on Personnel Assessment Orlando, FL,. June.

Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58,525-543. Milfont, R., & Fischer, R. (2010). Testing measurement invariance across groups: applications in cross-cultural research. International Journal of

Psychological Research, 3,112-131. Retrieved from <> Morris, M. A., & Robie, C. (2001). A meta-analysis of the effects of cross-cultural training on expatriate performance and adjustment. International Journal

of Training and Development, 5,112-125. Muritiba, P. M., Muritiba, S. N., Campanario, M., & de Albuquerque, L. G. (2010). International HR strategy in Brazilian technology multinationals.

BAR—Brazilian Administration Review, 7(4), 325-344. Muthen, B., du Toit, S. H. C., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling

with categorical and continuous outcomes. Conditionally Accepted for Publication in Psychometrika. Müller, S., & Gelbrich, K. (2004). Interkulturelles Marketing [Intercultural marketing]. München, Germany: Vahlen. Nederhof, A. J. (1985). Methods of coping with social desirability bias: a review. European Journal of Social Psychology, 15, 263-280. Olson, C. L., & Kroeger, K. R. (2001). Global competency and intercultural sensitivity. Journal of Studies in International Education, 5,116-137. Rathje, S. (2007). Intercultural competence: the status and future of a controversial concept. Language and Intercultural Communication, 7, 254-266.

Rockstuhl, T., Ang, S., Ng, K. Y., Lievens, F., & Van Dyne, L. (2015). Putting judging situations into situational judgment tests: evidence from intercultural multimedia SJTs. Journal ofApplied Psychology (in press), Retrieved from

<> Ruben, B. D. (1976). Assessing communication competency for intercultural adaptation. Group & Organization Studies, 1,334-354. Schacher, C. (2011). Der Perspektivenwechsel in interkulturellen Konfliktsituationen [Change of perspective in intercultural conflict situations] (Unpublished

diploma thesis). Darmstadt, Germany: Technische Universität Darmstadt. Schmitt, D. P., Realo, A., Voracek, M., & Allik, J. (2008). Why can't a man be more like a woman? Sex differences in Big Five personality traits across 55

cultures. Journal of Personality and Social Psychology, 94,168-182. Schnabel, D. (2015). Intercultural competence: development and validation of a theoretical framework, a Cross-Cultural Multimethod Test, and a Collaborative Assessment Intervention (Doctoral dissertation). Tuebingen, Germany: Eberhard Karls Universitaet Tuebingen. Schnabel, D., & Kelava, A. (2013a). Development and validation of the German Test to Measure Intercultural Competence (TMIC): a combined-method

approach. In Paper Presented at the 12th European Conference on Psychological Assessment San Sebastian, Spain,. July. Schnabel, D., & Kelava, A. (2013b). The effects of using therapeutic assessment in counselling students who are going abroad: intercultural competence

and self-related development. In Paper Presented at the 12th World Congress of Psychological Counseling and Guidance Istanbul, Turkey,. September. Schnabel, D., Kelava, A., & Van de Vijver, F.J. R. (2015). The effects of using collaborative assessment with students going abroad: interculturalcompetence

development, self-understanding, self-confidence and stages of change. Journal of College Student Development. (in press). Schnabel, D., Kelava, A., Seifert, L., & Kuhlbrodt, B. (2015). Konstruktion und Validierung eines multimethodalen berufsbezogenen Tests zur Messung interkultureller Kompetenz [Development and validation of ajob-related multimethod Test to Measure Intercultural Competence]. Diagnostica, 61, 3-21. Schuler, H., & Prochaska, M. (2000). Entwicklung und Konstruktvalidierung eines berufsbezogenen Leistungsmotivationstests [Development and

validation of an occupational achievement motivation inventory]. Diagnostica, 46,61-72. Sheahan, P. (2005). Generation Y: thriving and surviving with generation Y at work. Prahran, Australia: Hardie Grant Books.

Sinicrope, C., Norris, J., & Watanabe, Y. (2007). Understanding and assessing intercultural competence: a summary of theory, research, and practice.

Second Language Studies, 26,1-58. Retrieved from <> Spitzberg, B. H., & Changnon, G. (2009). Conceptualizing intercultural competence. In D. K. Deardorff (Ed.), The SAGE handbook of intercultural competence

(pp. 2-52). Los Angeles, CA: Sage Publications. Stajkovic, A. D., & Luthans, F. (1998). Self-efficacy and work-related performance: a meta-analysis. Psychological Bulletin, 124, 240-261. Tamam, E. (2010). Examining Chen and Starosta's Model of intercultural sensitivity in a multiracial collectivistic country. Journal of Intercultural Communication Research, 39,173-183.

Thomas, A. (Ed.). (1991). Kulturstandards in der internationalen Begegnung [Cultural standards in international encounters]. In. Saarbrücken, Germany: Breitenbach.

Thomas, A. (2003). Interkulturelle Kompetenz—Grundlagen, Probleme und Konzepte [Intercultural competence—basics, problems, and concepts]. Erwägen Wissen Ethik, 14(1), 137-221. Retrieved from <>

Thomas, A. (2010). Culture and cultural standards. In A. Thomas, E.-U. Kinast, & S. Schroll-Machl (Eds.), Handbook of intercultural communication and cooperation (2nd ed., pp. 17-27). Goettingen, Germany: Vandenhoeck & Ruprecht.

Thomas, A., Kammhuber, S., & Layes, G. (1997). Interkulturelle Kompetenz. Ein Handbuch für internationale Einsätze der Bundeswehr [Intercultural

competence. A manual for international missions of the German Federal Armed Forces]. Untersuchungen des Psychologischen Dienstes der Bundeswehr, 32, 7-190.

Ting-Toomey, S. (1993). Communicative resourcefulness: an identity negotiation perspective. In R. L. Wiseman, & J. Koester (Eds.), Intercultural communication theory (pp. 72-111). Newbury Park, CA: Sage.

Triandis, H. C. (1977). Interpersonal behavior. Monterey, CA: Brooks/Cole.

QuestBack GmbH. (1999). UniparkEFS Survey [Computer software]. Cologne, Germany: QuestBack GmbH.

Van de Vijver, F.J. R., & Leung, K. (1997). Methods and data analysis for cross-cultural research. Newbury Park, CA: Sage.

Van de Vijver, F.J. R., & Leung, K. (2009). Methodological issues in researching intercultural competence. In D. K. Deardorff (Ed.), The SAGE handbook of intercultural competence (pp. 404-418). Los Angeles, CA: Sage Publications.

Van Dyne, L., Ang, S., & Koh, C. (2008). Development and validation of the CQS. In S. Ang, & L. Van Dyne (Eds.), Handbook of cultural intelligence: theory, measurement and application (pp. 16-38). Armonk, NY: M.E. Sharpe.

Van Oudenhoven, J. P., & Van der Zee, K I. (2002). Predicting multicultural effectiveness of international students: The Multicultural Personality Questionnaire. International Journal of Intercultural Relations, 26,679-694.

Weekly, J. A., & Ployhart, R. E. (2006). Situational Judgment Tests: theory, measurement and application. London, United Kingdom: Lawrence Erlbaum Associates.

Weinert, F. E. (2001). Vergleichende Leistungsmessung in Schulen—eine umstrittene Selbstverständlichkeit [Comparative performance tests in schools—a controversial matter of course]. In F. E. Weinert (Ed.), Leistungsmessungen in Schulen [Performance assessment in schools] (pp. 17-31). Weinheim, Germany: Beltz.

Whetzel, D. L., McDaniel, M. A., & Nguyen, N. T. (2008). Subgroup differences in situational judgment test performance: a meta-analysis. Human Performance, 21, 291-309.

Williams, T. R. (2005). Exploring the impact of study abroad on students' intercultural communication skills: adaptability and sensitivity. Journal of Studies in International Education, 9,356-371.

Yamazaki, Y., & Kayes, D. C. (2004). An experiential approach to cross-cultural learning: a review and integration of competencies for successful expatriate adaptation. Academy of Management Learning and Education, 3,362-379.

Ziegler, M., & Bensch, D. (2013). Lost in translation: thoughts regarding the translation of existing psychological measures into other languages. European Journal of Psychological Assessment, 29, 81-83.

Zhong, M. (1998). Perceived intercultural communication competence in cross-cultural interactions between Chinese and Americans. Critical Studies, 12, 161-179. Retrieved from <>