CC BY-NC-ND
0Available online at www.sciencedirect.com
ScienceDirect
Procedia - Social and Behavioral Sciences 112 (2014) 5 - 16
2013 International Congress on Education and Educational Psychology (ICEEPSY)
College Self-efficacy Subscales: Contrasting associations with background, linguistic, and other psychological variables
Rachel Tanya Fouladi* and Patricia Wallisf
Department of Psychology, Simon Fraser University, 8888 University Drive, Burnaby, BC V5A 1S6, Canada
Abstract
Problem Statement: Although some consider college student self-efficacy a unified construct with recommendations for measurement using questionnaires with total scores with "high reliability and validity", separate analyses of subdomains may be warranted. Research Question: Are there different findings at the subscale level in models of college student self-efficacy (course and social) that consider sex, language fluency and use, academic year, stress preventive resources, and depressive symptomology? Purpose of the Study: This paper considers a contextual psycho-social model of college-student well-being among undergraduates at an English-language university in an ethno-culturally/linguistically diverse city in British Columbia, Canada. It differentiates between patterns of association of key variates from the model with two identified subdomains of college student self-efficacy, i.e., course and social subdomains. Research Method: Participants were undergraduates at a midsized university. Participants completed a battery of questionnaires. Data on focal variables [college self-efficacy (course and social), background variables (sex, immigrant/citizenship generation, instructional language fluency and use, academic year), personality s, stress preventive resources, and depressive symptomology] were included in multivariate general linear model (GLM), hierarchical regression and correlational analyses. Findings: Multivariate GLM analyses of the college self-efficacy subscales yielded omnibus effects (SSType I) for generation, language fluency and home use, academic year, select personality and stress preventive resources variates (ps<.05). Although there were key commonalities, the pattern of statistical significance differed by criterion variable in corresponding univariate GLM (SSType I) and hierarchical regression analyses. Conclusions: Examination of the College Self-Efficacy Inventory and its subscales highlights that although overall scores may be of use, comparative analyses of findings at the subscale level should be considered. Importantly, it is not sufficient to compare/contrast 'statistical significance' profiles, statistical comparisons of corresponding correlations elucidates differential findings and assists developing nomothetic networks and targeted interventions to improve student outcomes.
© 2013 TheAuthors.PublishedbyElsevierLtd.
Selectionandpeer-reviewunderresponsibilityof Cognitive-counselling,research and conference services (c-crcs).
* Corresponding author. E-mail address: rfouladi@sfu.ca
f This research was funded through grants from the Social Sciences and Humanities Research Council to the Corresponding Author.
1877-0428 © 2013 The Authors. Published by Elsevier Ltd.
Selection and peer-review under responsibility of Cognitive-counselling, research and conference services (c-crcs). doi: 10.1016/j.sbspro.2014.01.1134
Keywords: Total scores; Subscales; Nomothetic network.
1. Introduction
1.1. Background
In many disciplines, including psychology, responses to questionnaires are commonly considered as observed variables that are manifestations of underlying latent variables. In the case where the questionnaire is considered to have a single underlying latent variable, scoring of the questionnaire to yield a single composite value for each respondent is commonly advocated (c.f., Thissen, Steinberg, Pyszczynski, & Greenberg, 1983; Barry & Finney, 2009). After appropriately changing the direction of any reverse coded item, the scoring system that is most commonly utilized and advocated by some involves unit-weighting of the items to yield a total sum score or average score based on equal weighting of the items (c.f., Catell & Radcliffe, 1962; Wainer, 1976; Kristoff, 1969; Samejima, 1973; Wilks, 1938); This approach is commonly used even when items have differential latent variable regression and structure coefficients with the assumed underlying latent variable (c.f., Kaiser, 1970; Beauducel & Leue, 2013). Although arguably objectionable (c.f., Barry & Finney, 2009), this approach is also commonly used even with multi-dimensional instruments when there may be a hierarchical structure of a several first order and a single second order latent variables, and there is a differential number of items for each first order latent variable, and differential regression and structure coefficients between the first order and second order latent variable.
General self-efficacy can be conceptualized as a global sentiment of the ability to perform tasks; importantly, however, even though there are these overall conceptualizations of self-efficacy an individual's judgment regarding his/her self-efficacy is considered specific to a given task/situation (Bandura, 1977), as such different self-efficacy domains have been posited. College self-efficacy, more specifically, refers to the ability to function/perform tasks at college/university (c.f. Solberg, O'Brian, Villareal, Kennel, & Davis, 1993). Even though the labeling of college self-efficacy is a step towards specification, insofar as attending college involves the performance of a variety of types of tasks requiring different resources/abilities, heterogeneity in those tasks and an individual's abilities/access to resources and feelings about being able to accomplish those tasks calls for the consideration of more specific domains within college self-efficacy (c.f., Pajares, 1996; Solberg, et al., 1993, Zimmerman, 2000; Zajacova, , Lynch, & Espenshade, 2005).
How one conceptualizes a phenomenon guides how one seeks to measure it. Operationally defining college self-efficacy as a students' "degree of confidence that they could successfully complete a given college related task", Solberg, O'Brian, Villareal, Kennel and Davis (1993) developed a College Self-Efficacy Instrument (CSEI). Starting with an initial item pool based on college self-help manuals, expert judgment reduced the item set to roughly twenty target items (Solberg, et al., 1993). Based on components analysis (principal components analysis followed by oblique transformation) of complete data on a sample of 164, the authors decided on a solution of 3 correlated components (Course/Academic, Social, and Roommate); The percentage of common variance represented by each component was 44.8% for Course/Academic; 11.5% for Roommate; 7.2% for Social -- with each component identified (based on "loadings" of greater than .5) represented by differing numbers of items (Course-7 course items, Social-8 items, Roomate-4 items). The authors reported Cronbach's alpha values for composite scores on the corresponding subscales (0.88 each) as well as an overall/total score scale (0.93). Scoring of the subscale and total scores, were all based on equal-weighting of identified items. The subscale correlations of CSEI-Course/Academic with CSEI-Social was 0.71, with CSEI-Roommate was 0.45; of CSEI-Social with CSEI-Roommate was 0.48. In the sample under consideration, further subscale level analyses yielded patterns of correlations with other measures, which after being subject to a second order principal components analysis, were interpreted as suggesting good convergent and discriminant validity.
In the Solberg, O'Brian, Villareal and colleagues paper, 19 items are reported as having "factor loading" greater than .5 in the 3 correlated component solution (Solberg, et al., 1993). However in a later description in a paper by Solberg,and Villarel, (1997), 20 items are reported as having coefficients of 0.50 or greater (number of items per subscale not specified). Although Solberg and Villareall (1997) provide descriptions of the three subscales of the CSEI: Course/Academic, Social, and Roommate College Self-Efficacy, for the purpose of their 1997 study (N=257), "a total score was used as a single index of college self-efficacy and was generated by averaging the 20 items." (p.190)
A later paper by Solberg, Gusavac, Hamann and colleagues (1998) provided a re-wording of the original CSEI to address the variety of living circumstances of students to a sample of university students (A=388). They found a four component solution to be most "meaningful". They named the four components Course Self-Efficacy (7-items), Social Self-Efficacy (5-items), Coping With Others You Live With (4 items; this corresponds to the CSEI-Roommate in the 1993 paper), and Social integration Self-Efficacy consisting of "three items associated with connection to the institution". The reported Cronbach's alpha coefficient for the total scale based on equal weighting of items was .91, and for the subscales values were 0.86, 0.89, 0.79, and 0.62 respectively. For the remaining analyses in their paper, they restricted their subscale analyses to use of the first three subscales. The subscale correlations of CSEI-Course/Academic with CSEI-Social Revised was 0.67, with CSEI-Roomate/Others was .50; of CSEI-Social Revised with CSEI-Roomate/Others was 0.44.
A variety of researchers have used the CSEI. Most recently, Vuong, Brown-Welty, and Tracz (2010) in their examination of the effects of self-efficacy on academic success of first-generation college sophomore students made use of the three original CSEI subscales (ver. 1993/1997) only.
Zajacova, Lynch, and Espenshade (2005) opted for use of select items from CSEI and another scale which were subjected to factor analysis. Based on their findings, they suggested a hierarchical structure for college self-efficacy. In their analyses, they made use of specific composites based on the suggested 1st order factors.
Gore (2006) and Gore, Leuwerke, and Turley (2006) made use of the full CSEI (ver. 1993/1997). Gore's 2006 analyses described and made use of both subscale and total scores from the CSEI. In particular, Gore, Leuwerke, and Turley (2006, A =257) conducted a psychometric analysis of the CSEI investigating the 3 and 4 factor solutions presented by Solberg and colleagues in the 1993 and 1998, papers; however, they were unable to get convergence on the 4 factor solution, so opted for the 3 factor solution. Their 3 factor solution corresponded with the original Solberg and colleagues item sets. Correlations for the equal-item weighted composites based on the factors were: 0.64 for Course/Academic with Social 0.64, 0.55 for Course/Academic with Roommate, and 0.66 for Social with Roommate. With descriptive statements about differential correlation patterns, they advocated use of the CSEI subscales.
"Additional support for the use of three separate factors can be found in the fact that subscales of the CSEI differentially correlated with students' college expectations (i.e., CSXQ scores), academic performance (GPA), and academic persistence. Our endorsement of the three-factor solution is offered with caution, however, in that we observed moderate to high correlations among the three factors identified in our study. Such findings suggest the possible utility of a higher order academic self-efficacy factor and may justify the use of a CSEI total score in some situations".
Barry and Finney (2009) also conducted a psychometric study of the CSEI to examine validity evidence for the inventory. They examined a variety of factor models (one, three, and four), ultimately they obtained best fit for a three-factor model based on a reduced item set, as well as theoretically consistent correlations of composites based on the suggested factors with scores for other instruments (e.g., social anxiety, test anxiety).
Although the trend has been toward use of CSEI-subscales in analyses, following Solberg and Villareal (1997), some researchers have only used a total score. Barry and Finney (2009) offer a sharp critique:
"It is unclear why a total score would be computed, given that self-efficacy theory dictates that efficacy beliefs are task and domain specific. It would not be appropriate to combine academic,
social, and roommate items unless there is empirical evidence suggesting that a single factor drives responses to all aspects of college self-efficacy or the components/factors are highly correlated." (p. 200)
In addition to the paper by Solberg and Villareal (1997) in which solely the total score was used, other examples include DeWitz (2002) and DeWitz, Woolsey and Walsh (2009) which made use of the CSEI in their examination of college student issues. Although in these two papers different numbers of items are described as on the CSEI (20 and 22 respectively), total score analyses based on equal weighting of the items were used. This approach was used even though items regarding different aspects of college self-efficacy were acknowledged - in the case of the 2009 paper, "thirteen items related to academic work" and "nine items relate to social aspects of college life".
1.2. The current study
A key aspect of understanding a construct and of construct validation is the study of its nomothetic span/network, i.e., how the corresponding test score(s) are related to other variates (c.f.., Whitley, 1983). General self-efficacy and increasingly more specifically college self-efficacy are recognized as integral to the consideration of university/college student well-being, with key relationships with background characteristics, personality, and coping resources (e.g., Chemers, Hu, Garcia, 2001; Dwyer & Cummings, 2001; Salami, 2010; Strobel, Tumasjan, & Sporrle, 2011; Wang & Castaneda-Sound, 2011). Although some researchers of College Self Efficacy have used the global composite of the CSEI, others have used and/or advocated strongly for use of the subscales (e.g., Barry and Finney, 2009). The current paper seeks to examine two correlated components of college self-efficacy (CSEI-Course/Academic and CSEI-Social) as posited by Solberg and colleagues in their original 1993/1997 work with regard to their multivariate and bivariate association with key psycho-social variables (background, personality, stress preventive resources, and depressive symptomology) in a sample of linguistically and culturally diverse university students.
The Roommate component of the CSEI is not examined due to the low rate of students in this sample living outside of a family context. Although several papers have commented on differential correlational results for the subscales with other variables, direct tests of the equality of the correlation patterns have not been conducted. This paper directly compares the associations with direct tests on the correlations (e.g., Steiger, 1980ab; Steiger & Browne, 1984), to evaluate the tenability of a null model where the associations are the same. In this way, this study seeks to a) elucidate the importance of retaining the distinctions between components, even when they are highly correlated, b) highlight use of methodology which can be used to directly compare correlational results.
2. Method
2.1. Participants
Participants for this study were undergraduate students enrolled at a mid-sized university with in a metropolitan area. Participants were recruited using: 1) the Department of Psychology Research Participation System which draws on undergraduate students enrolled in select lower division courses (e.g., introductory psychology and introduction to research methods), 2) advertisements in the student newspaper, and 3) poster advertisements placed on billboards around campus. All participants self-selected to participate in the questionnaire study. This study received approval from and was conducted in compliance with university human subjects review board requirements.
Data for the current paper are from a broad project investigating questionnaire response patterns in university students. Participants are a subset from the broad project, and are participants who were administered and responded to items on the focal College Self-Efficacy Inventory subscales. Data on undergraduate university
students (N=263) who responded to items on sex, their English language fluency/use, and at least 80% of the items for the psychological composites are considered for this multivariate analysis; after screening for outliers, final results on N=259 are reported. Ninety three participants of the final sample were male (36%), 166 were female (64%). Participants had median age of 19 (M=20, SD=2); min=17, max=35. 67% reported being single, 27% committed relationship not living with partner, 4% committed relationship living with partner, 2% married. 80% of participating students indicated living at home with parents or guardians, 2% with siblings, 2% with spouse, 9% in student residence, 5% with roommates, 2% alone. 21% of participants reported being South Asian, 11% middle eastern, 27% Asian (but not South Asian nor Middle eastern), 32% European/Caucasian/other, 9% did not provide ethnicity information. Additional demographics details are provided in the Results sections.
2.2. Procedure and Research Design
Questionnaires were completed by participants in small groups in the principal investigator's research laboratory at the university in 1-2 hour time blocks. The groups constituted of no more than 10 study participants. The research laboratory was set up with tables and chairs; privacy dividers forming cubicles were available for use by participants if desired. Students selected when they would complete the questionnaire study from an array of time slots. Participants arrived at the laboratory in scheduled time blocks, were given an introduction to the questionnaire study project investigating factors related to questionnaire response, provided instructions on procedures, assured of data anonymity, given the opportunity to ask any questions regarding procedures, provided with informed consent forms to complete, and proceeded to respond to questionnaires. Questionnaires were administered on research laboratory lap-top computers. Study participants were logged onto the computers anonymously. Participants were able to skip items or questionnaires as they wished. When participants finished answering questionnaires, participants were thanked by the research staff, given the opportunity to ask further questions about the large-scale project and to not have their data included in the analysis, and given a research participation receipt.
2.3. Measures
The focal measures for the current paper are the College Student Self-Efficacy Inventory Course/Academic and Social subscales, a demographics questionnaire, International Personality Item Pool items, as well as the Stress Preventive Resources Inventory Measure, and the Center for Epidemiologic Studies - Depression scale. For cases with missing data on up to 20% of a given scale, a pro-rated composite score was computed.
Background Variables. The demographics questionnaire was used to determine background characteristics of participants. Example items queried participants regarding their age, gender, immigrant/citizenship generation, self-reported level of instructional language (English) fluency and use, and academic year.
English Language Fluency was obtained from a multiple choice fluency item on the demographics questionnaire. The multiple choice language fluency item was "How fluent are you in English"? Response options on this item were: Very fluent, English is my first language (1), More fluent in English than my first language (2), Same fluency in English as my first language (3), Less fluent in English than my first language (4). Higher scores on the Self-reported English Language Fluency item indicate LESS levels of self-reported proficiency.
Immigration/Citizenship Generation was indicated by responses to "What generation Canadian are you?" Options included: (1) moved to Canada, (2) born in Canada but both neither parent was born in Canada, (3) born in Canada but one parent was not, (4) born in Canada and parents were born in Canada. Higher scores indicate longer history of individual/family presence in Canada.
Language Use - at Home and English Language Use with Friends were assessed on the demographics questionnaire using multiple choice items "What language do you speak at home", and "What language do you
speak with friends". Choices ranged from (1) Only English, (2) Mostly English, (3) About half the time in English, (4) Mostly a non-English language, to (5) Only a non-English language. Thus higher scores on each of these indicate LESS usage of English.
Academic Year reflected student status at the university. Choices included Freshman/1st year, Sophmore/2nd year, Junior/3rd year, Senior/4th year undergraduate student status.
College Self-Efficacy. Course/Academic college self-efficacy and Social college self-efficacy were measured using two subscales from the College Self-Efficacy Inventory developed by Solberg, O'Brien, Villareal, Kennel, and Davis (1993) for which scores have been shown to have high internal consistency, reliability, and validity for use. The original instrument also included a Roommate subscale, however, because few students in the current sample indicate having roommates, the Roommate subscale was not included in this paper. Items on the Course/Academic subscale include self-efficacy regarding ability to: Research a term paper; Do well on exams; Take good notes in class; Keep up-to-date with your schoolwork; Understand your textbooks; Write course papers; Manage time effectively. Items on the Social subscale included self-efficacy regarding ability to: Ask a question in class; Ask a professor a question; Talk to your professors; Talk to university staff; Join a student organization; Make new friends at college; Get a date when you want one. Researchers have used the original CSEI items rating scales with different numbers of anchor points (e.g., Gloria, Kurpius, Hamilton, & Wilson, 1999; Hull-Blanks et al., 2005; Hutchinson, Jenkins-Guarnieri, Murdock, & Wright, in preparation); reliability and validity evidence have been provided for scoring of items with response options from as few as 5 options to as many as 10 options (c.f., Gore, 2006; Solberg, et al., 1993. For the current paper, items were measured on a 7-point rating scale (1-7) with confidence level anchor descriptors at the endpoints, as well as a Not Applicable option.
International Personality Inventory Pool (http://ipip.ori.org; Goldberg 1992, 1999, Goldberg, et al. 2006;) was used for measurement of personality dimensions corresponding to the Big Five model of personality: Surgency/Extraversion, Agreeableness, Conscientiousness, Emotional Stability (considered the opposite pole of Neuroticism), and Intellect/Imagination are each respresented by 20 items on the IPIP100. After appropriate coding of items, a composite sum score is yielded for each dimension, with corresponding coefficient alpha values reported by Golderg (1992) of .88-.91 (http://ipip.ori.org/newBigFive5broadTable.htm). Sample items for each personality dimension include: Surgency/Extraversion - Feel comfortable around people; start conversations; don't talk a lot (reversed); don't like to draw attention to myself (reversed); Agreeableness - Am interested in people; feel other's emotions; feel little concern for others (reversed); am not interested in others' problems (reversed); Conscientiousness - Am always prepared; pay attention to details; neglect my duties (reversed), find it difficult to get down to work (reversed); Emotional Stability - Am relaxed most of the time; seldom feel blue; am not easily bothered by things; get stressed out easily (reversed), worry about things (reversed). Intellect/Imagnination - Have a rich vocabulary, have excellent ideas, am quick to understand things, have difficulty understanding abstract ideas (reversed), am not interested in abstract ideas (reversed), do not have a good imagination (reversed). Varied international studies, e.g., Gow, Whiteman, Pattie, and Deary (2005) have demonstrated correlations with scores from corresponding Big Five personality instruments.
Stress Preventive Resources Inventory. The instrument (PR) asks respondents to indicate their level of agreement with statements about personal habits relating to the prevention of stress. Dimensions on the scale include: Perceived Control -- believing one is able to cope successfully with life's demands and manage situations that could potentially become stressful; Maintaining Perspective - having attitudes and beliefs that are consistent with preventing stressful situations and keeping stress-produced emotions at manageable levels; Social Resourcefulness -- the perceived ability to draw upon a social network of caring others who can act as a buffer against life's demands; Self-Acceptance scale - acceptance/overcoming of shortcomings, imperfections, and limitations in dealing with demanding life situations; Scanning -- one's perceived ability to recognize, anticipate, and plan for demands and potential stressors. Items are scored on a 5-point scale ranging from strongly disagree (1) to strongly agree (5). Higher scores indicate greater perceived use of or access to the stress preventive
resources on each scale. Tbe total score is labeled Preventive Resources, however is not included in the current study. The scale scores have demonstrated good internal consistency, reliability and validity for use in college students (Lambert, McCarthy, Gilbert, Sebree, & Steinley-Bumgamer 2006). Sample items on the each of the scales include: Control - I can handle most things; Maintaining - I am able to avoid causing myself stress by keeping things in perspective; Social - I have mutually supportive relationships; Self-Acceptance - I may not always get what 1 want; Scanning - 1 am good at identifying things that will cause stress in the future.
Center for Epidemiologic Studies - Depression. The CESD is a widely used 20-item self-report questionnaire assessing depressive symptomology (Radloff, 1977, 1991). Participants were instructed to rate each item based on how many times they felt a given way or engaged in a given behaviour during the past week. Response options are scored on a 4-point scale ranging from "Rarely (less than 1 day)" (0) to "Most of the time (5-7 days)" (3). With reverse coding of select items where appropriate, a summated score is computed across the 20 items. Higher scores indicate higher reported levels of depressive symptomology. The total score on the CESD has demonstrated good internal consistency, reliability and validity for use in adolescents and adults (Radloff, 1991). Although the CESD has subscale scoring guidelines, in this paper, only the total score is used.
2.4. Analysis
Descriptive analyses were conducted on demographic background variables. Standard descriptive statistics and were obtained. Descriptive statistics are provided for the overall sample included in the multivariate analyses. Data on focal variables [college self-efficacy (course and social), background variables (sex, immigrant/citizenship generation, instructional language fluency and use, academic year), personality variates, stress preventive resources variates, and depressive symptomology] were included in multivariate general linear model and hierarchical regression. Bivariate correlations were also obtained, and correlation pattern hypotheses tested. SPSS and Multicor (Steiger, 1979) were used for the analyses.
3. Results
Demographics. The university is situated in a linguistically and ethnically diverse community with a large immigrant population. Varied levels of self-reported English language fluency were indicated, however, overall 93% percent of participants indicated they were either native English speakers, or rated themselves as having at least the same level of fluency in English as their first language. On the 4-point ordered response item regarding fluency in English (ELF), 58% indicated they were very fluent in English/it is their first language, 24% indicated they were more fluent in English than their first language, 12% indicated the same level of fluency as their first language, 7% indicated they were less fluent in English than their first language.
With regard to the item "What generation Canadian are you"? 39% indicated they moved to Canada, 26% that they were born in Canada but neither parent was, 9% that they were born in Canada but one parent was not, 26% that they and their parents were born in Canada.
On the 5-point ordered response item regarding language usage at home: 37% indicated they only use English at home, 22% mostly English, 16% about half the time English, 19% mostly a non-English language, 7% only a non-English language. On the 5-point ordered response item regarding language usage with friends: 60% indicated they only use English with friends, 24% mostly English, 12% about half the time English, 3% mostly a non-English language, 2% only a non-English language. 38% were first year students/freshman, 38% were second year students/sophomores, 20% were third year students/juniors, 5% were fourth year students/seniors.
Mulitvariate and Univariate GLM analyses. Multivariate general linear model (GLM) analyses of the college self-efficacy subscales (Course and Social) yielded omnibus effects (SSType I) for generation (4-levels), language fluency (4-levels) and home use (5-levels), academic year (4-levels), select personality and stress preventive resources variates, as well as depressive symptolology (ps<.05); omnibus effects were not obtained for
gender (2-levels) and language use with friends (5-levels) (ps>.05). Although there were key commonalities, the pattern of statistical significance (e.g., sig vs. n.s.) differed by criterion variable in the corresponding univariate GLM (SS Type I) analyses. For example, simply comparing decisions regarding statistical significance at an .05 level, the results in Table 1 for the multivariate and univariate GLM (SSType I) analyses show that although statistical significance was found at the multivariate level for English Language Fluency (n2=.027, p=.047), at the univariate level it was statistically significant at the .05 level for CSEI-Social (n2=.037, p=.034), not for CSEI-Course/Academic (p>.05); similarly, for IPIP-Emotional Stability, although statistical significance was found at the multivariate level for IPIP-Emotional Stability (n2=.058, p=.001), at the univariate level it was only statistically significant at the .05-level for CSEI-Course/Academic (n2=.056, p<.001) , not for CSEI-Social (p>.05). Even simply using a reduced alpha level to adjust for the two univariate analyses corresponding to each multivariate GLM (i.e., .05/2=.025), other patterns emerge; there is no difference between the sig./n.s. decision for English Language Fluency, however instead the decision regarding the tenability of a null hypothesis for PR-Perceived Control now differs for CSEI-Course/Academic (p<.001, sig. at .025) and CSEI-Social (p=.048, n.s. at .025).
Hierarchical Regression Analyses. For the separate hierarchical regression analyses that were conducted for CSEI-Course/Academic and CSEI-Social subscales, background, personality, stress preventive resources, and depressive symptomology were entered sequentially into the corresponding regression equations. The variables for each of the block were as follows: block 1 - sex, block 2 - English language fluency, block 3 - Canadian immigrant/citizen generation, block 4 - language use at home, block 5 - language use with friends, block 6 - the five IPIP personality variates, block 7 - the five Stress Preventive Resources variates, block 8 - CESD. Table 1 summarizes squared multiple correlations at each stage of the hierarchical regression analyses for each of the CSEI subscales after inclusion of each additional block. With the exception of the result at the first block (Sex), the overall proportion of variance explained at each stage of the analysis was statistically significantly different from zero (ps<.001). The overall squared multiple correlation for the final model after inclusion of all the variates was .542 for CSEI-Course/Academic and .512 for CSE-Social. 95% confidence intervals for the population squared multiple correlations can be obtained using R2 (Steiger & Fouladi, 1992) for the final 17 predictor models they were .426-.601 and .384-.565 respectively. As another example, the 95% confidence intervals for the squared multiple correlations after academic year was entered were .066-.225 (R2=.152) and (.026-.159) (R2=.097). Although, the overlap of the intervals can informally be used as an index of lack of statistical difference between the squared multiple correlations, a tenability of a hypothesis of the equal population multiple correlations can be directly tested (c.f., Steiger & Browne, 1984).
A multiple correlation is equal to the bivariate correlation of the observed value of the criterion variable with the predicted value based on the regression model. Predicted values of the criterion variable were saved at each stage of the hierarchical regression analyses. Following Steiger and Browne (1984) and Steiger (1980ab), pairwise comparisons of the multiple correlations based on the regression models of CSEI-Course/Academic and CSEI-Social at each stage of the hierarchical regression analyses were conducted using a Fisher-transform based normal theory chi-square analysis to test the equality of a pair of dependent population correlations where there are no common variables in the two correlations of interest (H0: rho(predicted Y1,Y1)=rho(predicted Y2 ,Y2), and although descriptively there were observed differences in the sample estimates, no statistically significant differences between the multiple correlations at corresponding stages of the hierarchical regression analyses (e.g., the pair of multiple correlations after academic year was entered) were obtained for any of the comparisons (ps>.05). For example even for the multiple correlations after academic year was included (i.e., .152 and .097 for CSEI-Course/Academic and CSEI-Social respectively), the test yielded chi square=2.43, df=1, p= .115.
Table 1 also summarizes, for each stage of the hierarchical regression analyses, the results for the tests of statistical significance for the change in proportion of variance explained in each of the CSEI subscales by inclusion of each additional block. Tests of the significance of the change in the proportion of variance explained in the predictors after inclusion of additional blocks showed the same pattern of decisions (at the .05 level) for all
blocks with the exception of Generation Canadian (Course/Academic />=.040, Social p=.626), preventive resources (Course/Academic p<.001, Social p=.111) and CESD (Course/Academic p<.001, Social p=.083). With the exception of the results for Generation Canadian the pattern of decisions is the same at a reduced alpha level of .05/2=.025, and using step-down Bonferroni (.05/8, .05/7, ...; c.f., Larzelere & Mulaik, 1976) levels controlling Type I error at or below .05 across each of the set of 8 tests of changes in population squared multiple correlation.
Table 1. Partial eta-squareds (rj1 from SSType I multivariate (Pillai) and followup univariate general linear model (GLM) analyses; squared multiple correlations and changes in squared multiple correlations from hierarchical multiple regression analyses (HRA), zero-order Pearson Product moment correlations (ZPP), and results from test of equality of pairs of dependent ZPP correlations (Eq) .
GLM HRA ZPP Eq
Multi. n2 P Course n2 P Social n1 P R 1 Course AR1 P R 1 Social AR2 P Course P Social P P
Overall .570 <.001 .545 <.001 .549 .511
Sex .036 .016 .011 <.001 .007 <.001 .005 .005 .167 .003 .003 .364 .069 .167 -.057 .364
ELF .017 .047 .014 .119 .037 .034 .054 .050 <.001 .043 .040 .001 -.115 <.001 -.198 .001 .591
GenCdn .084 <.001 .159 <.001 .063 .001 .081 .017 .006 .044 .001 .616 .169 <.001 .141 .011 .008
LU-Ho .039 .018 .053 .014 .058 .008 .107 .016 .008 .070 .016 .008 -.310 <.001 -.143 <.001 .096
LU-Fr. .013 .113 .033 .104 .011 .300 .107 .000 .917 .071 .001 .696 -.111 <.001 -.186 <.001 .609
Ac. Yr .068 <.001 .091 <.001 .074 .001 .151 .045 <.001 .097 .016 .007 .141 <.001 .199 .003 .384
IP-S/E .415 <.001 .100 <.001 .411 <.001 .476 .314 <.001 .487 .390 <.001 .360 <.001 .590 .001 <.001
IP-A .003 .717 .001 .550 .003 .446 .133 <.001 .310 <.001 .098
IP-C .170 <.001 .168 <.001 .031 .007 .411 <.001 .183 <.001 .003
IP-ES .058 .001 .056 <.001 .008 .171 .191 <.001 .315 <.001 .466
IP-I .190 <.001 .181 <.001 .101 <.001 .546 <.001 .508 <.001 .354
PR-PC .074 <.001 .074 <.001 .017 .048 .515 .049 <.001 .505 .019 .111 .583 <.001 .549 <.001 .391
PR-MP .008 .417 .000 .999 .006 .155 .458 <.001 .411 <.001 .407
PR-SR .007 .418 .000 .956 .005 .174 .406 <.001 .505 <.001 .010
PR-SA .009 .361 .009 .160 .004 .341 .515 <.001 .493 <.001 .606
PR-S .030 .031 .018 .011 .015 .063 .557 <.001 .437 <.001 .004
CESD .057 .001 .057 <.001 .017 .049 .549 .014 <.001 .511 .006 .083 -.417 <.001 -.380 <.001 .189
Note: N=159. ELF, GenCdn, LU-Ho, Lu-Fr., and Ac. Yr were all treated as categorical variables in GLM analyses; and as continuous covariates in the hierarchical regression analyses. Overall R for Course/Academic .549 (Adj. .517), and Social .511 (Adj. .477). eachp<.001.
Zero-order correlation analyses. CSEI-Course/Academic and CSEI/Social subscales correlated 0.69 (p<.001). Zero-order correlations for the CSEI-Course/Academic and CSEI-Social subscales with the background, personality, stress-preventive resources and depressive symptomology are further reported in Table 1; with the exception of the Sex, all considered variates showed statistically significant correlations with the CSEI subscales.
The 18x18 correlation matrix of the two CSEI subscales and the 16 variables with which they were statistically significantly correlated was used in an omnibus chi-square analysis to test the equivalence of the population correlation vectors (c.f., Steiger, 1979, 1980ab) for each CSEI subscale with the examined background and psychological variables (with the exception of sex which had yielded n.s. correlations with each of the subscales). As such, the omnibus null hypothesis of concern here is that the set of equality statements regarding dependent population correlation pairs where there is one common variable in each correlation pair:
rho(X1,Y1)=rho(X1=Y2); rho(X2,Y1)=rho(X2,Y2), ,...are simultaneously tenable. The chi square analysis, based on the Fisher transforms of correlations, showed there was a statistically significant difference between the two vectors (each with 16 correlations; chi square= 59.5, df=16, /><.001). Followup tests of equivalence of corresponding population correlation pairs were conducted (c.f., Steiger, 1980a, b). Correlations were statistically different for CSEI-Course/Academic and CSEI-Social with Generation Canadian (/=.008), IPIP-Surgency/Extraversion (/<.001) IPIP-Conscientiousness (/=.003), PR-Social Resources (/=.020, and PR-Self acceptance (/=.004). Results were consistent using a step-down Bonferonni strategy with family Type I error controlled at or below .05 (Larzelere & Mulaik, 1976; starting with .05/18, .05/17,...) except for the finding involving IPIP-Conscientiousness.
Summary and Discussion
A key aspect of construct validation is examination of a "the network of relationships of a test score with other variables" (Whitley, 1983). Through use of variates from a contextual psycho-social framework to understand college student function, this paper highlights the differential association of background, personality, and stress preventive resources in two correlated facets of college self-efficacy. This examination of the College Self-Efficacy Inventory and its subscales highlights that although overall scores may be of use, comparative analyses of findings at the subscale level should be considered. The subscales of CSEI-Course/Academic and CSEI-Social were highly correlated, with a correlation value at a level comparable to those found in other studies on undergraduate university students. Despite the relatively high correlation (e.g., 0.69 in this sample), differential results were obtained across a variety of multivariate and correlational analyses conducted.
Multivariate general linear model (GLM) analyses of the college self-efficacy subscales yielded omnibus effects (SSType I) for generation, language fluency and home use, academic year, select personality and stress preventive resources variates, and depressive symptomology (/s<.05). Although there were key commonalities, the pattern of statistical significance differed by criterion variable in the corresponding univariate GLM (SS Type I) analyses. Similarly in the Hierarchical Regression Analyses, differential patterns of statistical significance emerged with regard to the increments in proportion of variance explained by certain blocks of variables.
For the profile of differential patterns from the GLM analyses based on comparisons of sig. vs. n.s. at the nominal level, there are differences between the association of CSEI-Course/Academic and CSEI-Social for English Language Fluency, IPIP-Emotional Stability, and Stress Preventive Resources-Scanning, and at the .05/2 alpha level for IPIP-Emotional Stability and Depressive Symptomology. Based on the hierarchical regression analyses, several of the blocks differ in whether they yield a statistically significant increment in the squared multiple correlation from one step to another (e.g., the change due to the addition of the block with Generation Canadian, the change due to the addition of the block with the Preventive Resources variates, the change due to the addition of the block with CESD) - where in all three instances none of those blocks resulted in a statistically significant increase in the squared multiple correlation for CSEI-Social but did for CSEI-Course/Academic.
Importantly, it is not sufficient to compare/contrast 'statistical significance' profiles, statistical comparisons of corresponding correlations (e.g., Steiger, 1980 a, b.) elucidates differential findings. Even though there was a differential sequence in increments in the squared multiple correlation, tests of the equality of the squared multiple correlations at each step did not indicate a significant difference in the overall proportion of variance in the criterion variables explained by the variates in the model at each step in the hierarchical regression analyses.
With the exception of sex, simple correlation analyses of the variates in the GLM and HRA models showed the association of all the variates with the CSEI-subscales. Tests of equality of bivariate zero-order correlations, however, highlight directly key differences in how the individual variates in the models are associated with CSEI-Course/Academic and CSEI-Social. Results from tests of equality of the bivariate correlations highlight the differential association with CSEI-Course/Academic and CSEI-Social of generation immigrant/citizen, personality surgency/extraversion and conscientiousness, and stress prevention scanning resources. And although
differential correlation patterns with stress prevention social resources were indicated (.406 vs. .505, p=.020), that result did not hold up under reduced alpha levels.
College Self-Efficacy Inventory was developed as multi-dimensional instrument based on the understanding and conceptualization of self-efficacy with regard to specific facets of college/university experience. The set of differential findings in the current paper, further support this conceptualization and this approach to researching College Self-Efficacy, and further assists in developing and understanding nomothetic networks for the different aspects of College Self-Efficacy. Through better understanding of how these aspects of College Self-Efficacy are differentially related to key variates in contextual psycho-social models of college student functioning, there is the potential for improved ability to develop targeted interventions through which to improve student outcomes.
References
Beauducel, A., & Leue, A. (2013). Unit-weighted scales imply models that should be tested! Practical Assessment, Research, and
Evaluation, 18(1). http://pareonline.net/pdf/v18n1.pdf Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. Barry, C.L., & Finney, S.J. (2009). Can we feel confident in how we measure college confidence?: A Psychometric Investigation of
the College Self-Efficacy Inventory. Measurement and Evaluation in Counseling and Development, 43(3). 197-222. Chemers, M.M., Hu, L.T., & Garcia, B.F. (2001). Academic self-efficacy and first year college student performance and adjustment.
Journal of Educational Psychology, 93(1), 55-66. Cattell, R. B., & Radcliffe, J. (1962). Reliability and validity of simple and extended weighted and buffered unifactor scales. British
Journal of Statistical Psychology, 15, 113-128. DeWitz, S.J. & Walsh, W.B. (2002). Self-efficacy and college student satisfaction. Journal of Career Assessment, 10, 315-326 DeWitz, S.J., Woolsey, M.L., & Walsh, W.B.. (2009). College student retention: An exploration of the relationship between self-
efficacy beliefs and purpose in life among college students. Journal of College Student Development, 50(1), 19-34. Dwyer, A.L. & Cummings, A.L (2001). Stress, self-efficacy, social support and coping strategies in university students. Canadian
Journal of Counselling, 35 (3), 208-220. Gow, A.J., Whiteman, M.C., Pattie, A., & Deary, I.J. (2005). Goldberg's 'IPIP' Big Five factor markers: Internal consistency and
concurrent validation in Scotland. Personality and Individual Differences, 39, 315-329. Gloria, A. M., Kurpius, S. E. R., Hamilton, K. D., & Willson, M. S. (1999). African American students' persistence at a predominantly white university: Influence of social support, university comfort, and self-beliefs. Journal of College Student Development, 40, 257-268.
Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4, 26-42. Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe, Vol. 7 (pp. 7-28). Tilburg, The Netherlands: Tilburg University Press. Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. C. (2006). The International
Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96. Gore, P. A., Jr., Leuwerke, W. C., & Turley, S. E. (2006). A psychometric study of the College Self-Efficacy Inventory. Journal of
College Student Retention, 7(3-4), 227-244. Gore, P.A.Jr, (2006). Academic self-efficacy as a predictor of college student outcomes: Two incremental validity studies. Journal of Career Assessment, 14, 92-115.
Hull-Blanks, E., Robinson Kurpius, S. E., Befort, C., Sollenberger, S., Nicpon, M. F., & Huser, L. (2005). Career goals and retention-
related factors among college freshmen. Journal of Career Development, 32, 16-30 Hutchinson, S., Jenkins-Guarnieri, M. A., Murdock, J., & Wright, S. (in preparation). Multiple groups confirmatory factor analysis of
a college self-efficacy scale. Manuscript in preparation. International Personality Item Pool: A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences (http://ipip.ori.org/). Internet Web Site.
Kaiser, H. F. (1970). A second generation little jiffy. Psychometrika, 35, 401-415.
Kristof, W. (1969). Estimates of true score and error variance for tests under various equivalence assumptions. Psychometrika, 34, 489-507.
Lambert, R.G., McCarthy, C.J., Gilbert, T., Sebree, M., & Steinley-Bumgamer, M. (1006). Validity evidence for the use of the Preventive Resources inventory with college students. Measurement & Evaluation in Counseling & Development, 39(2), 66-83.
Larzelere, R.E., & Mulaik, S.A. (1976). Single-sample tests for many correlations. Psychological Bulletin, 84, 557-569.
Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543-578.
Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied PsychologicalMeasurement, 1(3), 385-401.
Radloff, L. S. (1991). The use of the Center for Epidemiologic Studies Depression Scale in adolescents and young adults. Journal of Youth and Adolescence, 20(2), 149-166.
Salami, S.O. (1010). Emotional intelligence, self-efficacy, psychological well-being and students' attitudes: Implications for quality education. European Journal of Educational Studies, 2(3). ISSN 1946-6331. http://ozelacademy.com/EJES_v1n3_8.pdf
Solberg, V. S., & Villarreal, P. (1997). Examination of self-efficacy, social support, and stress as predictors of psychological and physical distress among Hispanic college students. Hispanic Journal of Behavioral Sciences, 19(1), 181-101
Solberg, V. S., Gusavac, N., Hamann, T., Felch, J., Johnson, J., Lamborn, S., & Torres, J. (1998). The Adaptive Success Identity Plan (ASIP): A career intervention for college students [Monograph]. The Career Development Quarterly, 47, 48-95
Solberg, V. S., O'Brien, K., Villareal, P., Kennel, R., & Davis, B. (1993). Self-efficacy and Hispanic college students: Validation of the college self-efficacy instrument. Hispanic Journal of Behavioral Sciences, 15(1), 80-95.
Steiger, J.H. (1979). Multicorr: a computer program for fast, accurate, small-sample tests of correlational pattern hypotheses. Educational and Psychological Measurement, 39, 677-680. (software downloadable from www.statpower.net)
Steiger, J.H. (1980a). Testing pattern hypotheses on correlation matrices: Alternative statistics and some empirical results. Multivariate Behavioral Research, 15, 335-351.
Steiger, J.H. (1980b). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2), 245-251.
Steiger, J.H., & Browne, M.W. (1984). The comparison of interdependent correlations between optimal linear composites. Psychometrika, 49, 11-11.
Steiger, J.H., & Fouladi, R.T. (1991). R2: A Computer Program for Interval Estimation, Power Calculation, and Hypothesis Testing for the Squared Multiple Correlation. Behavior Research Methods, Instruments, and Computers, 4, 581-581. (software downloadable from www.statpower.net)
Strobel, M., Tumasjan, A., & Sporrle, M. (1011). Be yourself, believe in yourself, and be happy: Self-efficacy as a mediator between personality and well-being. Scandinavian Journal of Psychology, 52(1), 43-48.
Thissen, D., Steinberg, L., Pyszczynski, T., Greenberg, J. (1983). An item response theory for personality and attitude scales: Item analysis using restricted factor analysis. Applied Psychological Measurement, 7( 2), 111-116
Vuong, M., Brown-Welty, S., Tracz, S. (1010). The effects of self-efficacy on academic success of first-generation college sophomore students. 51(1), 50-64.
Wainer, H. (1976). Estimating coefficients in linear models: It don't make no nevermind. Psychological Bulletin, 83, 113-117.
Wang, C.-C. D.H., & Castaneda-Sound, C. (1008). The role of generational status, self-esteem, academic self-efficacy, and perceived support in college students' psychological well-being. Journal of College Counseling, 11(2), 101-118.
Whitley, S.E. (1983). Construct validation: Construct representation versus nomothetic span. Psychological Bulletin, 93(1), 179-197.
Wilks, S. S. (1938). Weighting systems for linear functions of correlated variables when there is no dependent variable. Psychometrika, 1938, 3, 13-40.
Wright, S.L., Jenkins-Guarnieri, M.A. & Murdock, J.L. (1013). Career development among first-year college students: College self-efficacy, student persistence, and academic success. Journal of Career Development, 40(4), 191-310.
Zajacova, A., Lynch, M. S., & Espenshade, J. T. (1005). Self-efficacy, stress, and academic success in college. Research in Higher Education, 46(6), 677-706.
Zimmerman, B. J. (1000). Self-efficacy: An essential motive to learn. Contemporary Educational Psychology 25(1), 81-91.
