CC BY-NC-ND
0Education in
Practice
Contents lists available at ScienceDirect
Nurse Education in Practice
journal homepage: www.elsevier.com/nepr
Clinical education
The impact of curricular changes on BSCN students' clinical learning outcomes
CrossMark
Janet Landeen a' *, Donna Carr b, Kirsten Culver a, Lynn Martin a, Nancy Matthew-Maich Charlotte Noesgaard a, Larissa Beney-Gadsby
a School of Nursing, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada b BScN Program, Conestoga College, 299 Doon Valley Drive, Kitchener, Ontario N2G 4M4, Canada
c BScN Program, Mohawk College, Institute of Applied Health Sciences, 1400 Main Street West, Hamilton, Ontario L8S 1C7, Canada
ARTICLE INFO
Article history: Received 9 October 2015 Received in revised form 1 September 2016 Accepted 30 September 2016
Keywords:
Curriculum evaluation Curriculum change Qualitative research
ABSTRACT
Ongoing curricular renewal is a necessary phenomenon in nursing education to align learning with ever-changing professional practice demands. The McMaster Mohawk Conestoga BScN Program in Hamilton, Ontario, Canada recently engaged in a comprehensive curriculum renewal. The purpose of this study was to evaluate the impact of curricular changes on students' deep learning. Faculty perceptions about student learning outcomes during final year clinical placements were gathered through a combination of individual interviews and focus groups using Interpretive Descriptive qualitative research methodology.
Twenty five faculty members who supervised BScN students in clinical placements before and after curriculum renewal shared perceptions of changes in students' overall performance. The chosen clinical learning outcomes were: changes in students' performance related to person-centred care, clinical reasoning and judgment, pathophysiology, and evidence-informed decision-making. Faculty described three major themes in students' performance 1) pulling it all together, 2) seeing the whole person, and 3) finding their nursing voices. This reflected a shift to person-centred care, increasing professional confidence, and improved clinical reasoning and judgment and no changes to integrating pathophysiology or evidence-informed decision-making. In this study curriculum renewal provided an excellent starting point for the scholarship of teaching and learning within nursing education.
© 2016 Published by Elsevier Ltd.
1. Introduction
Nursing education programs aim to facilitate the development of competent, safe, caring novice nurses who can adapt to and influence the ever-changing practice environment. Nursing educators strive to make the students' educational experience engaging and meaningful, using the best available teaching and learning evidence. To do this, ongoing quality improvement initiatives, or at times larger curricular renewal projects, are deemed necessary. These undertakings require large expenditures of time, energy, and
scholarly activity. While different pedagogical innovations may address different aspects of the learning experience, the ultimate goal is to improve the learning outcomes for students.
Once changes are implemented, educators are faced with determining whether the curricular innovations have indeed lived up to their promise; are student learning outcomes any different from those of the previous curricula? This paper describes an initial systematic evaluation of the achievement of learning outcomes following a curricular renewal project in one undergraduate nursing program.
* Corresponding author. McMaster University, 1280 Main Street West, HSC 2J25, Hamilton, Ontario L8S 4K1, Canada.
E-mail addresses: landeen@mcmaster.ca (J. Landeen), Dcarr@conestogac.on.ca (D. Carr), culverk@mcmaster.ca (K. Culver), martl@mcmaster.ca (L. Martin), nancy. maich@mohawkcollege.ca (N. Matthew-Maich), noesgaar@mcmaster.ca (C. Noesgaard), beneyln@gmail.com (L. Beney-Gadsby).
1 Present Address: McMaster Children's Hospital, 1200 Main St W, Hamilton, Ontario, L8N 3Z5, Canada.
http://dx.doi.org/10.1016/j.nepr.2016.09.010 1471-5953/© 2016 Published by Elsevier Ltd.
2. Background
2.1. Literature review
The nursing education and interdisciplinary literature on the scholarship of teaching and learning in higher education (SoTL) are consistent in calling for the evaluation of educational innovations.
Hutchings and Shulman (2010) assert that SoTL's basic premise is that faculty "systematically investigate questions related to student learning" (p.12). Felten (2013) proposes that principles of SoTL are embedded in projects that are "(1) inquiry into student learning, (2) grounded in context, (3) methodologically sound, (4) conducted in partnership with students, and (5) appropriately public" (p. 122). Furthermore, Gurung and Landrum (2014) argue that assessment of student learning is essential in the scholarship of teaching and learning. Thus, the SoTL literature assists in defining the rationale for, and possible approaches to, the evaluation of curriculum renewal. However, much of the focus of the SoTL literature is at the course or individual instructor level (Felten, 2013; Paino et al., 2012).
The focus on assessing student learning outcomes is not new in the nursing education literature and is regarded as a necessary component of nursing curricular development (Billings and Halstead, 2012; Iwasiw and Goldenberg, 2015; Keating, 2014). National accreditation standards for programs of nursing in Canada and the U.S. include expectations that overall program evaluation will be routinely conducted, gathering information from key stakeholders and reporting on student outcomes including program completion and pass rates on licensure examinations. Furthermore, the standards include expectations that results will be used to inform constant quality improvement (CASN, 2014; CNEA, 2015; Keating, 2014). While different models of program evaluation of educational programs have been developed (Aoki, 1991/2005; Stake, 2003; Stufflebeam and Shinkfield, 2007), these models are not designed to measure nuanced differences in student learning outcomes related to curricular innovation.
Higher education in general, and nursing education in particular, have been called upon to facilitate deep knowing, moving beyond the accumulation of knowledge (Barnett, 2009; Benner et al., 2010). Cummings and colleagues (2008) highlight the need for performance assessment that measures higher order problem solving skills to meet current employer requirements of university graduates. Grauerholz and Main (2013) suggest that quantifiable outcome measures, such as tests results and course grades, do not capture the subtleties in differences in deep learning and suggest that other methodologies, including qualitative approaches, are more appropriate to evaluate real differences in student learning outcomes. While longitudinal mixed-method outcome studies of the performance of graduates on targeted areas of curriculum innovation have been found in the nursing literature (Curran et al., 2010; Diefenbeck et al., 2015), it is also important for educators to have more immediate feedback on the changes that they have implemented. Qualitative methods have been used to examine the short-term impact of curriculum on students' attitudes of caring as a first step in longer term evaluation strategies. (Phillips et al., 2015). Consequently, a qualitative approach to the short-term evaluation of the impact of curricular renewal is an appropriate strategy in the scholarship of teaching and learning in nursing education.
2.2. Context
McMaster University has been offering a BScN since 1946, and introduced problem-based, small-group, self-directed learning in 1975, which have remained hallmarks of the curriculum. Details of the evolution of problem-based learning at McMaster University, and discussions of the theoretical constructs, teaching philosophy, and teaching-learning methodology have been documented elsewhere. (See for example: Alderson, 1976; Benson et al., 2002; Lunyk-Child et al., 2001; Majumdar, 1999; and Rideout, 2001). The McMaster Mohawk Conestoga BScN Program is a collaboration which began in 2000 between one university (McMaster
University) and two community colleges (Mohawk College and Conestoga College) in which all students complete the same curriculum (the McMaster University curriculum) and receive a McMaster degree. Faculty from all sites collaborate on developing, delivering, and evaluating all aspects of the educational experience. While it was usual practice of McMaster University to make modifications to the curriculum based on educational philosophy and best evidence (Ciliska, 2005; Ladouceur et al., 2004; Landeen et al., 2013; Roberts and Norman, 1990), it was time for a comprehensive curriculum review.
A fulsome curriculum renewal process undertaken in 2008 culminated in the creation of the Kaleidoscope Curriculum. A recommitment by faculty at all sites was made to the McMaster Philosophy of Nursing and Nursing Education (available from the author). This guided curriculum decisions, along with a review of current nursing and educational literature, focus groups of key student, faculty, and employer stakeholders, and a scan of other nursing education programs in North America. Details of the curriculum plan can be found in the McMaster University Undergraduate Calendar (2014). Key features of the renewal included: 1) a focus on clinical reasoning and judgement, 2) an adaptation of problem-based learning, and 3) the purposeful integration of pathophysiology and evidence-informed decision-making concepts into core nursing courses. Other innovations which have been evaluated separately included the introduction of service learning in the first two years of the four year degree (Schofield et al., 2013), integration of interprofessional education (Salfi et al., 2011), and the introduction of clinical simulation (Landeen et al., 2015). Within the curriculum, all students have one problem-based learning course per semester, and student-centred approaches are integrated into all other courses. Faculty members engaged in an extensive faculty development process, consistent with what has been traditionally offered within the McMaster Mohawk Conestoga consortium (Drummond-Young et al., 2010; Matthew-Maich et al., 2009) to ensure that they were well prepared to deliver curriculum changes.
In the Kaleidoscope Curriculum, Tanner's language and conceptual model of clinical reasoning and judgement, based on evidence of how practicing nurses make clinical decisions (Benner et al., 2009; Tanner, 2006), is reinforced rather than the nursing process model which has historically formed the basis of nursing education programs. In addition to changes to in-class discussions, this language has been systematically integrated into student worksheets and evaluation forms, reinforcing the importance of "noticing, interpreting, responding, and reflecting" within nursing practice (Tanner, 2006, p. 208).
Problem-based learning (PBL) is far from a new educational innovation. Indeed, McMaster University has been a world-wide leader in its use in medical education (Neville and Norman,
2007). Research into PBL has found that while there are mixed results in knowledge acquisition compared to traditional, didactic approaches (Dochy et al., 2003), there are reported benefits in terms of students' ability to think critically (Applin et al., 2011; Kong et al., 2014; Williams et al., 2012), their motivation to learn (Woltering et al., 2009), and their potential to deal with clinical uncertainty and ambiguity (Hodges, 2011). While problem-based learning (PBL) has been proposed as one of the remedies for content-heavy curricula (Forbes and Hickey, 2009), recent analyses suggest that this approach places emphasis on solving the client's clinical problem, focusing on clinical diagnoses and treatment, an approach more consistent with medical practice (Taylor and Miflin,
2008). While Barrows and Tamblyn (1980) originally identified a "learning problem", not a "patient problem" as the basis for PBL, subtle shifts have occurred over time. Thus, PBL was modified within the Kaleidoscope Curriculum to person-based learning
within a problem-based approach, or PBL/PBL as it is currently called. Students remain engaged in learning within small groups, with learning facilitated by a faculty member. Within the new PBL/ PBL, students are introduced to people first through rich multimedia narratives. The power of narratives has been demonstrated in hearing patients' stories (Frank, 1998) and in medical (Charon, 2001) and nursing education (Diekelmann, 2005; Hunter, 2008). While these narratives are designed to engage the students in the learning process, the question is whether this emphasis transfers into their interactions with and discussions about real patients in their clinical learning experiences.
The final area of curriculum renewal was the integration of concepts that are fundamental to nursing practice: pathophysi-ology and evidence-informed decision making into core nursing courses. These two areas had previously been taught in stand-alone courses and there was concern that students were not consistently applying this knowledge in the clinical area. Evidence on the impact of structuring learning in a planned, integrative and reinforced manner (Benner et al., 2010; Hughes and Mighty, 2010) suggested that this knowledge could be better retained and applied if integrated across courses. Thus, explicit care scenarios and planned learning activities were integrated into nursing theory PBL/PBL courses.
Evaluating the outcomes of these shifts in curricular emphasis became the focus for this research project. The specific research question was: does the Kaleidoscope Curriculum make a difference in the clinical learning outcomes of final year students, particularly in relation to: (a) person-based learning within a problem-based approach, and (b) the purposeful integration of clinical reasoning and judgement, pathophysiology, and evidence-informed decision making in core nursing courses?
3. Methods
Consistent with Grauerholz and Main's (2013) suggestion for evaluating differences in deep learning, a qualitative research design was used. Interpretive Descriptive qualitative research is theoretically based and is an appropriate methodology when the research goal is to apply the findings to inform practice, rather than to generate theory or explore the essence of a phenomenon (Thorne, 2008). According to Interpretive Description, the researcher is called upon to systematically address and defend each methodological decision, leading to academic rigour and theoretical congruency.
3.1. Participants
Faculty members who taught clinical courses from the McMaster Mohawk Conestoga BScN Program (full-time, part-time and unpaid clinical faculty) were invited to participate in the study. Unpaid clinical faculty are professional nurses with at least a Master's degree who provide a minimum of 100 hours of teaching to undergraduate students per year. Full and part-time faculty members are paid by the educational institutions, and have expertise in nursing education and in the specific clinical areas. Inclusion criteria were as follows: experience teaching 1) a minimum of two fourth year clinical students prior to implementation of the revised curriculum and 2) a minimum of two fourth year clinical students post-implementation of the Kaleidoscope Curriculum.
Of the three groups of clinical faculty, unpaid clinical faculty had the least involvement in the development of the Kaleidoscope Curriculum, whereas regular full-time and part-time faculty were heavily involved in curriculum revisions. Regardless of their classification, all faculty who taught clinical provided indirect
supervision, meeting with their fourth year students (one-on-one) on a weekly or bi-weekly basis while students were completing clinical courses in their final year of study. Students were supervised in their clinical environments by practicing nurse preceptors who were employed by the clinical agencies. The learning triad (student, clinical faculty, and preceptor) met jointly a minimum of three times over the duration of the 12 week course. During the student-clinical faculty dyad meetings, faculty elicited and evaluated students' knowledge and approaches to clinical practice. As such, faculty teaching clinical courses interacted closely with students, and were in a position to identify differences in patterns of student behaviours. While sample size is difficult to determine a priori in qualitative research, samples of 6—8 focus groups of 4—6 participants each is common in achieving saturation of themes on defined concepts (Emden and Sandelowski, 1999) and 8—10 individual interviews are common in interpretive description (Thorne, 2008). Therefore, our goal was to complete interviews with a minimum of 20 faculty until data saturation was achieved.
3.2. Data collection strategies
This study received ethics approval from the Research Ethics Boards of the three institutions from which participants were recruited. Because the researchers were colleagues of the potential participants, a research assistant (LB-G) with no power relationship with participants was responsible for all recruitment and data collection activities. The researchers were blind to the identity of the participants, and consent forms were kept in a sealed envelope, in a double locked filing cabinet, separate from all data. Only general demographic data (employment type and experience supervising clinical students) were collected to protect the anonymity of the participants. Demographic data was not linked to individual responses to decrease any possibility of the researchers being able to identify the participants.
Perceptions of faculty were gathered through a combination of individual interviews and focus groups. Focus groups were to generate rich data (Lambert and Loiselle, 2008) and empower participants to initiate issues of concern to them, which frequently leads to the collection of important data that may have been missed otherwise (Wilkinson et al., 2007). Individual interviews were also used if the participants were concerned about the confidentiality of their responses, they could not attend any scheduled focus group, or felt that they had unique contributions to make to the study. Individual interviews have been described as the gold standard for data collection in qualitative research (Streubert and Carpenter, 2011). It has been suggested that one strategy to increase the richness of the data is to employ a combination of focus groups and individual interviews, with special attention paid to variations across responses during data analysis (Lambert and Loiselle, 2008). Focus groups and interviews used the same semi-structured questions that are identified in Table 1. The Research Assistant summarized the main points heard throughout the interview or focus group and asked the participants to confirm or further clarify the summary. Each focus group or interview lasted 45—60 min and was digitally recorded. All focus groups were conducted in person and individual interviews were conducted in person or via telephone, based on participant preference.
3.3. Data analysis
Data collection and analysis occurred simultaneously. The recordings were professionally transcribed verbatim, double checked for accuracy by the Research Assistant, and identifying information removed. The primary investigators analyzed the data to determine emerging themes using Thorne's (2008) steps for analysis:
Table 1
Semi-structured questions for focus groups and interviews.
• What, if any, differences have you noticed between students who have taken the Kaleidoscope Curriculum compared to students who were part of the original curriculum?
• Please share examples or stories of how students are achieving course ends-in-view now compared to two or more years ago.
• How do students talk about their clients? Do you notice any difference in their using person-centred language? Do you notice any difference in their approach to being client-centred?
◦ Do they discuss the strengths and assets of the client as well as the health problems? Do they discuss the client's context in their discussion of how they will assist the client?
• Are students any different in their ability to discuss and apply pathophysiological concepts in relation to their clients?
• Are students any different in their ability to bring evidence to their clinical decision making? To their everyday discussions with you?
• Are students any different in their ability to demonstrate clinical reasoning and judgment? Does this approach make any difference in their ability to care for clients?
confirming your bases, expanding on associations, testing relationships, capitalizing on outliers, and engaging the critic. Furthermore, Morse and Field's (1995) four intellectual processes of comprehending, synthesizing, theorizing and re-contextualizing were applied. All transcripts were coded by a minimum of two Investigators and the Principle Investigator coded all transcripts. Researchers achieved consensus on identified themes and examples through multiple team meetings. Patterns of behaviour of Kaleidoscope students were contrasted with those of pre-Kaleidoscope students in the areas of person-centredness and the ability to integrate pathophysiology, evidence informed decision making, and clinical reasoning and judgement into their approaches to clinical practice.
4. Findings
A total of 25 faculty members teaching fourth year clinical courses participated in this study. They shared their perspectives on the impact of the Kaleidoscope Curriculum on student learning outcomes within clinical practicums through individual interviews (n = 20) and focus groups (n = 5). All participants were female, with experience teaching fourth year clinical students ranging from 4to35 + years (M 14.6, SD = 7.56 years). Twelve of the faculty were employed full-time, five part-time, and eight were unpaid clinical faculty. While 11 participants indicated that they had taught students from at least two of the three partner sites, eight participants had taught students exclusively from the McMaster site, four were exclusive to the Mohawk site, and two were exclusive to the Con-estoga site.
Faculty shared stories that powerfully and clearly depicted changes that they observed in the learning outcomes of fourth year students who experienced the Kaleidoscope Curriculum. Findings are represented in Fig. 1; three recurrent themes emerged from the
Pulling it all Together
Being More Prepared for Fourth Year Ramping Up Sooner Having More Confidence
Seeing the Whole Person
Knowing the Story Considering the Context Including the Family
Finding their Nursing Voices
Communicating with IP team Questioning "Usual" Practice Advocating for Selves & Clients
data: pulling it all together, seeing the whole person, and finding their nursing voices. Each is discussed below, supported with participant quotes. It should be noted, however, that many participants prefaced their remarks with comments such as, "I have been blessed, all the students I've had have been very good. I feel like sometimes I've been getting the cream of the crop" [Professor #1]. A few participants indicated that they did not notice any particular changes, and then went on to describe differences that they saw.
4.1. Pulling it all together
Fourth year students completing the Kaleidoscope Curriculum were described as having evolved as nurses with the ability to pull together the many facets of knowing, skill and reasoning that enabled effective nursing practice at a heightened level from the previous curriculum. This facilitated their being more prepared for fourth year expectations, allowed them to 'ramp up' sooner in each placement in the final year, and consequently facilitated greater professional confidence.
[Previously] none of them had that whole picture. They were doing pieces of it very well ... and certainly had a solid foundation for continued growth. But I didn't see that same overall picture of excellence that I am seeing now [Professor #2].
I feel like the students recently seemed to have a better perspective of pulling information together ... It's hard to pin point why that is; but it seems that... it's coming from the way it's been tailored with the Tanner model... It seems that pulls it all together, it helps them guide [their nursing approach]. And when I sit down and talk to them about that they seem to get what that's all about a lot quicker [Professor #3].
I have to say that we are noticing stronger students coming in to the clinical area. As a group, a group of our clinical faculty, we've identified that our students are stronger, they have somewhat of a better awareness of the complexities of healthcare than when I look back to our students 5 or 10 years ago [Professor #4].
4.1.1. Being more prepared for fourth year
Fourth year students were described as being ready with the knowledge, skills, and attitudes needed for the amplified expectations of final year professional practice. This was a change from the previous curriculum where there had been a significant transition between third and fourth years,
From the clinical practice standpoint I think they come better equipped. When they start on [fourth year] now they are better equipped with the information from the previous courses, this isn't a shock to them; they're not struggling so much around managing the client component [Professor #5].
Fig. 1. Differences in final year students' clinical learning outcomes.
4.1.2. Ramping up sooner
It was clearly articulated that the faculty overwhelmingly perceived students to be much more ready to 'ramp up' into practice as they started each of the two fourth year clinical placements and also upon graduation from the BScN program,
Because students seem to be able to get into the culture [of the unit] at lot better. The two preceptors [practicing RN's who directly supervise students in their clinical settings] I had this term in particular commented on the way the students just branched out. They didn't stay with the preceptor as much and learned from other nurses as well. I found that interesting because I haven't heard that before. It usually takes a longer period of time - towards the end [of the semester] they are just getting those wings to fly, whereas by mid-term they were there already [Professor #6].
4.1.3. Having greater professional confidence
Faculty spontaneously shared that students were more confident in their knowledge and skills. In the past, a lack of confidence had been major complaint of fourth year students,
He had the confidence, he had the problem solving, had the critical thinking and he knew the equipment because he was already comfortable with this area ... In previous years it took students a lot longer to sort of get into the real nursing role. So the transition from student to nurse, again it's new, but seems to be quicker for them [Professor #7].
4.2. Seeing the whole person
While faculty considered that the curriculum had always been client-centred, they noticed nuanced differences in the understanding of students who had experienced the Kaleidoscope Curriculum. In the previous curriculum, faculty shared that students would begin student-faculty meetings by discussing the medical diagnosis and related pathophysiology of the clients for whom they were caring. Now, they would begin the meetings by discussing a holistic view of the person and would require a probe from the faculty member to also discuss the pathophysiology. While students were equally versed in the pathophysiology, or the evidence that supported their clinical interventions, this was not the starting point for considering clinical issues.
Stories were repeatedly shared of practice scenarios depicting how the students consistently viewed their clients as unique, whole persons including (1) knowing their clients' 'stories', (2) understanding the contexts in which their clients live and how this impacted health and healing, and (3) considering their clients' families. Faculty considered this development as the accumulation of how students had evolved over their entire undergraduate experience.
My student last semester was very distressed by some discharge planning that was going on that hadn't looked at the literacy level of the patient, the family dynamic. She said it was pretty obvious. And so she had really picked up on the personhood of that individual. And so I do see them focusing more on that. I think it's always been embedded in the Mac [McMaster] program, but I think they have the language now [Professor #4].
It's always hard with a student to really understand that you can influence others by how you deal with the patient. The last two students I've had ... last semester [in Kaleidoscope Curriculum],
they were really committed to holding onto that value of being person-centered. In spite of the demand you know, you need to be more organized, you need to get more done. You know, how can we do this? We're so busy. Because [being person-centred] is so embedded they are not thrown off by this [busy-ness] [Professor #1].
4.2.1. Knowing the 'story' and considering the context
Faculty comments revealed that by actively seeking to learn the client's story and context, students were able to more fully understand the client's needs to plan care holistically. This was pivotal in the students' nursing practice,
When you start off by saying, "Tell me about this patient", I think they more naturally think to mention about that person's story or you know, who they are; their context in their family support as a person along with, well the IV was running at 100 cc's and all the other details that come with it [Professor #8].
The awareness of context and the impact of applying this to nursing practice was pervasive in the students' practice, "they seem to talk to more about context ... to their sense of people have different life courses, often beyond their control and I am, as a student, now more aware of that" [Professor #9]. The sense of context extended to understanding and acting on the need to explore and prepare clients and families for the transition from acute care settings to community.
4.2.2. Considering the family
Faculty described how students in the Kaleidoscope Curriculum understood the importance of family or of not having family, for their clients; "... families definitely more ... they include the family, they are very aware of family and that's often a strength, they often talk about that as a strength for the patient... and certainly a deficit if they don't have family" [Professor #6].
... she [the student] was able to step in and care for a family member who was in distress. Without any qualms whatsoever she utilized all her knowledge of family and how to deal with family and recognized this was an essential part of dealing with children. She took the mother aside and she did some health teaching with her, she provided a comfortable environment for her. And she also recognized the cultural differences and was able to address those [Professor #10].
4.3. Finding their nursing voices
While the importance of client advocacy had always been a component of the previous curriculum, there was evidence of students enacting this advocacy to greater degrees in their clinical practice. Faculty were inspired by the stories students shared about the way they enacted their strong nursing voices professionally and effectively when communicating with interprofessional (IP) teams, questioning 'usual' practice, and advocating for both their clients and themselves.
4.3.1. Communicating with IP team
Faculty shared that students had greater confidence when communicating with the IP team and actively participated in discharge planning discussions with IP teams, voicing their perceptions or concerns of the client's or family's readiness for discharge. This too was new,
One student in ICU identified a significant cultural component to a very challenging ethical issue that was happening with a patient in the ICU, and did that spontaneously. So I think that brings in the patient story and different ways of knowing. And an example in mental health, when the team felt that the patient was ready for discharge the student had spent additional time with that patient and getting to know his story and discovered that his living arrangement was more fragile than the team understood. That [the student sharing this knowledge with the IP team] delayed the discharge by a couple of days so they [the IP team] could get a better discharge plan in place [Professor #11].
4.3.2. Questioning 'usual' practice
Faculty were amazed at the positive, professional, and effective manner students used to question accepted or "usual" practice and at how comfortable they were in doing so, "They challenge, like they're good [at challenging], without creating angst between [people], they will challenge" [Professor #5].
The challenging of the actions that the preceptors are doing, they are actually doing it in a really caring kind of way and not, "oh I know it all and why are you doing this". They are actually approaching the preceptors in a very non-threatening assertive way [Professor #12].
4.3.3. Advocating for clients and self
Many powerful and emotional exemplars were shared of professional practice encounters where the students' active and effective advocacy made a difference.
This student was with a preceptor at a very active medical/ surgical area. The morning that they entered the room to do their care ... I think they had a number anywhere from seven to eight clients that day. The student went into the room, there was a client ready to be discharged and he was, according to her preceptor, 'acting out'. He apparently took his shoe off and heaved it across the room and he indicated that something was wrong with him. Now you recognize he was to be discharged. The student ..., instead of going on with the preceptor to the next room she stayed with this man and recognized that this was real. She then did the first thing, I was so proud of her, she took his vital signs. She positioned him and then she identified that he had not had any urinary output, had him catheterized for 800 ccs of urine. The man then proceeded to tell her that he had tingling sensation; that he was lacking in feeling and so on. She quickly got a physician, they then tested the man for sensory deficits and sure enough they sent him for an MRI and he had a large hematoma on his cervical spine. He went immediately to the operating room where they released the pressure, eventually maybe a week and a half later he was back on that floor and he asked to see the nurse who had been with him. The preceptor went to go in the room and he told the nurse no, it wasn't her, the student that was on that day. She [the student] went in and he said to her, 'you are my rock!' [Professor #10].
5. Discussion
This qualitative study addressed the question: does the Kaleidoscope Curriculum make a difference in the clinical learning outcomes of final year students. While the results demonstrate only
initial impressions of outcomes and longer-term follow-up studies of graduates and their employers are necessary, the early results are very promising. Faculty who were in a position to notice differences in the clinical learning outcomes of fourth year students reported positive changes in relation to students' clinical performance including seeing the whole person, being better prepared for clinical practice, and finding their nursing voice. Faculty did not notice particular differences in relation to students being better able to apply and integrate their knowledge of pathophysiology or evidence-informed decision making.
Current educational literature stresses student engagement and active learning strategies that include flipped classrooms (Presti, 2016), concept-based learning (Fromer, 2013; Lewis, 2014), and values-based learning (McLean, 2012) to name a few. However, the PBL approach, particularly with the inclusion of patient narratives that evoke an emotional engagement (Fredricks et al., 2004), is consistent with the aims of these educational approaches. By including the person at the centre of the PBL learning cycle, the student is more likely to consider the whole person first, and then the problems to be addressed, consistent with strengths-based learning (Gottlieb, 2013). The changes to the process of PBL noted within this study may assist in helping graduates retain a patient-centred approach in addition to the positive professional behaviours of PBL graduates noted in previous studies (Kong et al., 2014; Williams and Day, 2009; Williams et al., 2012).
While it is difficult to separate the impact of different aspects of the curriculum on learning outcomes, the change in focus to Tanner's (2006) language of clinical reasoning appeared to facilitate students' ability to articulate their assessments in actual clinical practice. Students were found to be more articulate in "finding their voice", in advocating for their patients, and in noticing clinical issues beyond the obvious. This is consistent with research outcomes previously reported on clinical reasoning models (Canniford and Fox-Young, 2015; Dillard et al., 2009).
Problem-based learning, in and of itself, can be defined as integrative learning. Findings of this study indicated that more fully integrating pathophysiology and evidence-informed decision making content within PBL classes in the Kaleidoscope Curriculum did not have any noticeable impact. Cognitive load theory may explain these results in that the further integration of these two areas may have been too complex for novice learners (van Merrienboer and Sweller, 2005; Vogel-Walcutt et al., 2011). There were some potential methodological influences on the findings of this study. It was noted that there were differences in the discussions in focus groups as compared to individual interviews. Within focus groups, most participants rarely disagreed with each other which may have been related to any perceived power differentials. While a Research Assistant was used for recruitment and data collection, some participants may have erroneously believed that the researchers could identify them, and they may have modified their comments accordingly. While minimizing power dynamics was accounted for in the study design, the use of focus groups where participants have ongoing work relationships may have an impact on what is shared. In this instance, individual interviews may be the more appropriate data collection method.
This study employed a short-term, interpretive descriptive qualitative approach in evaluating differences in clinical learning outcomes following the implementation of a curriculum renewal. This is consistent with other research on the short-term impact of curriculum change (Phillips et al., 2015) and is particularly helpful to educational programs as they examine their own practices. However, this study should be followed by longer term analyses of teaching and learning approaches. These would include the outcomes typically monitored for accreditation processes such as retention rates, results on national credentialing examinations, and
graduate and employer feedback (CASN, 2014; CNEA, 2015). Other research methodologies such as cross-sectional studies of faculty uptake of the curricular changes (Ide et al., 2014), and longitudinal follow-ups using graduate and employer surveys and focus groups (Diefenbeck et al., 2015) should also be considered. Once the curriculum renewal process has reached steady state, there should be continuous, ongoing monitoring of the process and outcomes to ensure that curriculum drift does not occur (van de Mortel and Bird, 2010).
6. Conclusion
Curriculum renewal can serve as a starting point for educational research. Indeed, all curriculum renewal should be carefully evaluated. A tailored, detailed educational study can provide one aspect of overall program evaluation, leading to a full circle of the scholarship of teaching and learning: using results of the research for constant quality improvement in nursing education.
Conflict of interest statement
All of the authors of this paper declare that they do not have any financial or personal relationships with other people or organisations that could have inappropriately influenced or biased this work.
Acknowledgement
This research was supported by an unrestricted Teaching and Learning Grant, #G21302 from the McMaster Institute for Innovation and Excellence in Teaching and Learning, McMaster University.
References
Alderson, H.J., 1976. Twenty-five Years a growing. X University, City, Province. Aoki, T. T. (1991/2005). Layered understandings of orientation in social studies program evaluation. In Pinar. W.F. & Irwin, R.L (Ed.), (2005). Curriculum in a New Key: the Collected Works of Ted. T. Aoki. Mahwah, NJ: Lawrence Erlbaum. Applin, H., Williams, B., Day, R., Buro, K., 2011. A comparison of competencies between problem-based learning and non-problem-based graduate nurses. Nurse Educ. Today 31,129-134. http://dx.doi.org/10.1016/j.nedt.2010.05.003. Barnett, R., 2009. Knowing and becoming in the higher education curriculum. Stud.
High. Educ. 34 (4), 429-440. http://dx.doi.org/10.1080/03075070902771978. Barrows, H.S., Tamblyn, R., 1980. Problem-based Learning: an Approach to Medical
Education. Springer, NY. Benner, P., Tanner, C.A., Chesla, C.A., 2009. Expertise in Nursing Practice: Caring,
Clinical Judgment & Ethics, second ed. Springer, New York. Benner, P., Sutphen, M., Leonard, V., Day, L., 2010. Educating Nurses: a Call for
Radical Transformation. Jossey-Bass, San Francisco. Benson, G., Brown, B., Coates, A., England-Oxford, V., Fothergill-Bourbonnais, F., Ingram, C., Rideout, E., Ross, M., 2002. A comparison of problem-based education and conventional curricula in nursing education. Adv. Health Sci. Educ. 7 (1), 3-17. http://dx.doi.org/10.1023/A:1014534712178. Billings, D.M., Halstead, J.A. (Eds.), 2012. Teaching in Nursing: a Guide for Faculty,
fourth ed. Elsevier, St. Louis, MO. Canniford, LJ., Fox-Young, S., 2015. Learning and assessing competence in reflective practice: student evaluation of the relative value of aspects of an integrated, interactive reflective practice syllabus. Collegian 22 (3), 291-297. http:// dx.doi.org/10.1016/j.colegn.2014.04.003. CASN Canadian Association of Schools of Nursing, 2014. CASN Accreditation Program Standards June 2014. Version 5. CASN, Ottawa, ON. Retrieved August 31, 2015 from. http://www.casn.ca/wp-content/uploads/2014/12/2014-FINAL-EN-Accred-standards-March-311.pdf. Charon, R., 2001. Narrative medicine: a model for empathy, reflection, profession
and trust. J. Am. Med. Assoc. 286 (15), 1879-1902. Ciliska, D., 2005. Educating for evidence-based practice. J. Prof. Nurs. 21 (6),
345-350. http://dx.doi.org/10.1016Zj.profnurs.2005.10.008. CNEA National League for Nursing Commission for Nursing Education Accreditation, 2015. Standard 5: Culture of Excellence-Program Outcomes. Proposed Standards for Accreditation. Retrieved August 31, 2015 from. http://www.nln. org/docs/default-source/accreditation-services/standard-v--culture-of-excellence-program-outcomes-%28pdf%29.pdf?sfvrsn=2. Curran, V.R., Sharpe, D., Flynn, K., Button, P., 2010. A longitudinal study of the effect of an interprofessional education curriculum on student satisfaction and
attitudes towards interprofessional teamwork and education. J. Interprofessional Care 24 (1), 41-52. http://dx.doi.org/10.3109/ 13561820903011927.
Diefenbeck, C., Herrman, J., Wade, G., Hayes, E., Voelmeck, W., Cowperthwait, A., Norris, S., 2015. Preparedness for clinical: evaluation of the core elements of the clinical immersion curriculum model. J. Prof. Nurs. 31 (2), 124-132. http:// dx.doi.org/10.1016/j.profnurs.2014.08.004.
Diekelmann, N., 2005. Engaging the students and the teacher: Co-creating substantive reform with narrative pedagogy. J. Nurs. Educ. 44 (6), 249-252.
Dillard, N., Sideras, S., Ryan, M., Carlton, K., Lasater, K., Siktberg, L., 2009. A collaborative project to apply and evaluate the clinical judgment model through simulation. Nurs. Educ. Perspect. 30 (2), 99-104.
Dochy, F., Segers, M., Van den Bossche, P., Gijbels, D., 2003. Effects of problem-based learning: a meta-analysis. Learn. Instr. 13, 533-568. http://dx.doi.org/10.1016/ S0959-4752(02)00025-7.
Drummond-Young, M., Brown, B., Noesgaard, C., Lunyk-Child, O., Maich, N.M., Mines, C., Linton, J., 2010. A comprehensive faculty development model for nursing education. J. Prof. Nurs. 26 (3), 152-161. http://dx.doi.org/10.1016/ j.profnurs.2009.04.004.
Emden, C., Sandelowski, M., 1999. The good, the bad and relative, part two: goodness and the Criterion problem in qualitative research. Int. J. Prof. Nurs. Pract. 5 (1), 2-7.
Felten, P., 2013. Principles of good practice in SoTL. Teach. Learn. Inq. 1 (1), 121-125. Available at. http://www.jstor.org/stable/10.2979/teachlearninqu.1.1.121.
Forbes, M.O., Hickey, M.T., 2009. Curriculum reform in baccalaureate nursing education: review of the literature. Int. J. Nurs. Educ. Scholarsh. 6 (1), 1548-1923. http://dx.doi.org/10.2202/1548923X.1797 article 27 Pages -, ISSN (Online).
Frank, A., 1998. Just listening: narrative and deep illness. Fam. Syst. Health 16 (3), 197-212.
Fredricks, J.A., Blumenfeld, P.C., Paris, A.H., 2004. School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74 (1), 59-109. http:// dx.doi.org/10.3102/00346543074001059.
Fromer, R.F., 2013. A Theory-driven Integrative Process/Outcome Evaluation of a Concept-based Nursing Curriculum. Capella University, Ann Arbor, MI. Available at. Doctoral dissertation. www.proquest.com.
Gottlieb, L.N., 2013. Strengths-based Nursing Care: Health and Healing for Person and Family. Springer Publishing, NY.
Grauerholz, L., Main, E., 2013. Fallacies of SoTL: rethinking how we conduct our research. In: McKinney, K. (Ed.), The Scholarship of Teaching and Learning in and across the Disciplines. Indiana University Press, Bloomington, IN, pp. 152-168.
Gurung, R.A.R., Landrum, R.E., 2014. Assessment and the scholarship of teaching and learning. In: Dunn, D.S., Baker, S.C., Mehrotra, C.M., Landrum, R.E., McCarthy, M.A. (Eds.), Assessing Teaching and Learning in Psychology: Current and Future Perspectives. Cengage Learning, pgs, Belmont, CA, pp. 159-171.
Hodges, H., 2011. Preparing new nurses with complexity science and problem-based learning. J. Nurs. Educ. 50 (1), 7-13. http://dx.doi.org/10.3928/ 01484834-20101029-01.
Taking stock: research on teaching and learning in higher education. Montreal, QC and Kinsgston, ON. In: Hughes, C.J., Mighty, J. (Eds.), 2010. Queen's Policy Studies Series. McGill-Queen's University Press.
Hunter, L.A., 2008. Stories as integrated patterns of knowing in nursing education. Int. J. Nurs. Educ. Scholarsh. 5 (1) http://dx.doi.org/10.2202/1548-923X.1630. Article 38.
Hutchings, P., Shulman, L.S., 2010. The scholarship of teaching: new elaborations, new developments. Change Mag. High. Learn. 31 (5), 10-15. http://dx.doi.org/ 10.1080/00091389909604218.
Ide, C.A.C., Arantes, S.L., Mendonca, M.K., da Silva, V.R., de Padua Del Corona, A.R., 2014. Evaluation of the implementation of the integrated curriculum in the nursing undergraduate program. Acta Paul. Enfermagen 27 (4), 340-347. http:// dx.doi.org/10.1590/1982-019420140005 .
Iwasiw, C.L., Goldenberg, D., 2015. Curriculum Development in Nursing Education, (, third ed. Jones & Bartlett Learning, Burlington, MA.
Keating, S.B., 2014. Curriculum Development and Evaluation in Nursing, third ed. Springer, NY.
Kong, L.-N., Qin, B., Zhou, Y-q., Mou, S-y., Gao, H.-M., 2014. The effectiveness of problem-based learning on development of nursing students' critical thinking: a systematic review and meta-analysis. Int. J. Nurs. Stud. 51, 458-469. http:// dx.doi.org/10.1016/j.ijnurstu.2013.06.009.
Ladouceur, M., Rideout, E., Black, M., Crooks, D., O'Mara, L., Schmuck, M., 2004. Development of an instrument to assess individual student performance in small group tutorials. J. Nurs. Educ. 43 (10), 447-455.
Lambert, S.D., Loiselle, C.G., 2008. Combining individual interviews and focus groups to enhance data richness. J. Adv. Nurs. 62 (2), 228-237. http:// dx.doi.org/10.1111/j.1365-2648.2007.04559.x.
Landeen, J., Jewiss, T., Vajoczki, S., Vine, M., 2013. Exploring consistency within a problem-based learning context: perceptions of students and faculty. Nurse Educ. Pract. 13 (4), 277-282. http://dx.doi.org/10.1016/j.nepr.2013.03.013.
Landeen, J., Pierazzo, J., Akhtar-Danesh, N., Baxter, P., van Eijk, S., Evers, C., 2015. Exploring student and faculty perceptions of clinical simulation: a Q-sort study. J. Nurs. Educ. 54 (9), 485-491. http://dx.doi.org/10.3928/01484834-20150814-02.
Lewis, L.S., 2014. Outcomes of a concept-based curriculum. Teach. Learn. Nurs. 9 (2), 75-79. http://dx.doi.org/10.1016/j.teln.2013.12.002.
Lunyk-Child, O., Crooks, D., Ellis, P.J., Ofosu, C., O'Mara, L., Rideout, E., 2001. Self-
directed learning: faculty and student perceptions. J. Nurs. Educ. 40 (3), 116—123. http://dx.doi.org/10.3928/0148-4834-20010301-0.
Majumdar, B., 1999. Empowerment through self-directed learning. Can. Nurse 95 (6), 37—40.
Matthew-Maich, N., Mines, C., Brown, B., Lunyk-Child, O., Carpio, B., Drummond-Young, M., Noesgaard, C., Linton, J., 2009. Evolving as nurse educators in problem-based earning through a community of faculty development. J. Prof. Nurs. 23 (2), 75—82. http://dx.doi.org/10.1016/j.profnurs.2006.07.004.
McLean, C., 2012. The yellow brick road: a values based curriculum model. Nurse Educ. Pract. 12 (3), 159—163. http://dx.doi.org/10.1016/j.nepr.2011.11.002.
McMaster University Undergraduate Calendar. (2014). BScN Program, Hamilton, Ontario, Province: Author. Retrieved August 19, 2016 from http:// academiccalendars.romcmaster.ca/preview_entity.php?catoid=7&ent_ oid=937&returnto=567#Program_Offerings.
Morse, J.M., Field, P., 1995. Qualitative Methods for Health Professionals, second ed. Sage, Thousand Oaks, CA.
Neville, A.J., Norman, G.R., 2007. PBL in the undergraduate MD program at McMaster University: Three iterations in three decades. Acad. Med. 82 (4), 370—374. http://dx.doi.org/10.1097/ACM.0b013e318033385d.
Paino, M., Blankenship, C., Grauerholz, L., Chin, J., 2012. The scholarship of teaching and learning in teaching sociology: 1973-2009. Teach. Sociol. 40 (2), 93—106. Retrieved August 31, 2015 from. http://www.jstor.org/stable/41502734.
Phillips, J., Cooper, K., Rosser, E., Scammell, J., Heaslip, V., White, S., Donaldson, I., Jack, E., Hemingway, A., Harding, A., 2015. An exploration of the perceptions of caring held by students entering nursing programmes in the United Kingdom: a longitudinal qualitative study phase 1. Nurse Educ. Pract. 15 (6), 403—408. http://dx.doi.org/10.1016/j.nepr.2015.05.004.
Presti, C.R., 2016. The flipped learning approach in nursing education: a literature review. J. Nurs. Educ. 55 (5), 252—257. http://dx.doi.org/10.3928/01484834-20160414-03.
Rideout, E. (Ed.), 2001. Transforming Nursing Education through Problem-based Learning. Jones & Bartlett, Mississauga, ON.
Roberts, J., Norman, G., 1990. Reliability and learning from the objective structured clinical examination. Med. Educ. 24 (3), 219—223. http://dx.doi.org/10.1111/ j.1365-2923.1990.tb00004.x.
Salfi, J., Solomon, P., Allen, D., Mohaupt, J., Patterson, C., 2011. Overcoming all obstacles: a framework for embedding interprofessional education into a large, multisite bachelor of science nursing program. J. Nurs. Educ. 51 (2), 106—110. http://dx.doi.org/10.3928/01484834-20111230-05.
Schofield, R., Allan, M., Jewiss, T., Hunter, A., Sinclair, N., Diamond, A., Sidwell, C.,
2013. Knowing self and caring through service learning. Int. J. Nurs. Educ. Scholarsh. 10 (1), 267—274. http://dx.doi.org/10.1515/ijnes-2013-0009.
Stake, R., 2003. Responsive evaluation. In: Kellaghan, T., Stufflebeam, D.L. (Eds.), The International Handbook of Educational Evaluation. Klower Academic Publishers, Dordrecht, the Netherlands.
Streubert, H.J., Carpenter, D.R., 2011. Qualitative Research in Nursing: Advancing the Humanistic Imperative, fifth ed. Wolters Kluwer/Lippincott Williams & Wilkins, Philadelphia, PA.
Stufflebeam, D.L., Shinkfield, A.J., 2007. Evaluation Theory, Models, and Applications. Jossey-Bass, San Francisco.
Tanner, C.A., 2006. Thinking like a nurse: a research-based model of clinical judgment in nursing. J. Nurs. Educ. 45 (6), 204—211.
Taylor, D., Miflin, B., 2008. Problem-based learning: where are we now? Med. Teach. 30 (8), 742—763. http://dx.doi.org/10.1080/01421590802217199.
Thorne, S.E., 2008. Interpretive Description. Left Coast Press, Walnut Creek, CA.
van de Mortel, T.F., Bird, J.L., 2010. Continuous curriculum review in a bachelor of nursing program: preventing curriculum drift and improving quality. J. Nurs. Educ. 49 (10), 592—595. http://dx.doi.org/10.3928/01484834-20100730-05.
van Merrienboer, J.J.G., Sweller, J., 2005. Cognitive load theory and complex learning: recent developments and future directions. Educ. Psychol. Rev. 17 (2), 147—178. http://dx.doi.org/10.1007/s10648-005-3951-0.
Vogel-Walcutt, J.J., Gebrim, J.B., Bowers, C., Carper, T.M., Nicholson, D., 2011. Cognitive load theory vs. constructivist approaches: which best leads to efficient, deep learning? J. Comput. Assisted Learn. 27,133—145. http://dx.doi.org/ 10.1111/j.1365-2729.2010.00381.x.
Wilkinson, C., Rees, C., Knight, L., 2007. "From the heart of my bottom": negotiating humor in focus group discussions. Qual. Health Res. 17 (3), 411—422. http:// dx.doi.org/10.1177/1049732306298375.
Williams, B., Day, R.A., 2009. Employer perceptions of knowledge, competency, and professionalism of baccalaureate nursing graduates from a problem-based program. Int. J. Nurs. Educ. Scholarsh. 6 (1) http://dx.doi.org/10.2202/1548-923X.1646. Article 36.
Williams, B., Spiers, J., Fisk, A., Richards, L., Gibson, B., Kabotoff, W., McIlwraith, D., Sculley, A., 2012. The influence of an undergraduate problem/context based learning program on evolving professional nursing graduate practice. Nurse Educ. Today 32, 417—421. http://dx.doi.org/10.1016/j.nedt.2011.03.002.
Woltering, V., Herrler, A., Spitzer, K., Spreckelsen, C., 2009. Blended learning positively affects students' satisfaction and the role of the tutor in the problem-based learning process: results of a mixed-method evaluation. Adv. Health Sci. Educ. 14, 725—738. http://dx.doi.org/10.1007/s10459-009-9154-6.
