CC BY-NC-ND
0Available online at www.sciencedirect.com
SciVerse ScienceDirect PfOCSCl ¡0
Social and Behavioral Sciences
Procedia - Social and Behavioral Sciences 60 (2012) 103 - 111 —
UKM Teaching and Learning Congress 2011
The Course Outcomes (COs) Evaluation For Civil Engineering
Design II Course
Siti Aminah Osmana,b,\ Othman Jaafara, Wan Hamidon Wan Badaruzzamana,b,
Riza Atiq Abdullah O.K. Rahmata,b
a Centre for Engineering Education Research, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia b Department of Civil and Structural Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia
ELSEVIER
Abstract
Engineering Accreditation Council (EAC) has emphasized that Outcome Based Education (OBE) learning approach is to be adopted for all engineering programs at the Higher Learning Institutions (IPTA) in Malaysia. OBE calls for the evaluation of the Course Outcomes (CO) as specified in the program specification. This study aims to assess the students' achievement on the course outcomes (COs) that has been outlined in the Civil Engineering Design II Course. This assessment was conducted to all 64 final year students in the Department of Civil and Structural Engineering (JKAS), Faculty of Engineering and Built Environment, University Kebangsaan Malaysia. The course was selected as it is a compulsory course for all students in the department and it covers all aspect of civil engineering syllabus from the beginning semester of first year study. Eight COs will be assessed for the course based on course mapping of PO and CO as reported in Engineering Accreditation Council (EAC) self-assessment report. The CO assessment is measured based on students performance on written report of the design project, Bill of Quantities (BQ) report, presentation and also peer assessment. CO1 to CO5 were assessed from design project and presentation, whilst CO6 and CO8 from peer assessment and CO7 from BQ report. From the study it shows that CO6 and CO8 have the highest overall achievement of 100% while CO7 shows the lowest level of achievement of 73% among of all COs. The rest of the COs have a similar percentage around 76% - 78%. This study gives an indication on the achievement of the course and this can be used as a guide to improve the teaching and learning method in the future.
© 2011PublishedbyElsevierLtd. Selection and/orpeer reviewedunderresponsibilityofthe UKMTeachingand LearningCongress 2011
Keywords: Course outcomes; Civil Engineering Design Course; students performance; program outcomes; direct assessment
1. Introduction
The emphasis on the Outcome Based Education (OBE) for all engineering programs at the Higher Learning Institutions (IPTA) in Malaysia has greatly influenced the learning trends of undergraduate students. In fact the assessment and evaluation on the students achievement has shifted and focused based on the learning and program
* Corresponding author. Tel.: +6-03-8921-6221; fax: +6-03-8921-6147. E-mail address: saminah@eng.ukm.my.
1877-0428 © 2011 Published by Elsevier Ltd. Selection and/or peer reviewed under responsibility of the UKM Teaching and Learning Congress 2011 doi:10.1016/j.sbspro.2012.09.354
outcomes. One result of the changes introduced by the new outcome based ABET EC 2000 is that design, communication skills, managerial skills, working in multidisciplinary teams and life-long learning have been given increasingly important treatment in undergraduate engineering curricula (Abdullaal et al. 2011).Through this OBE implementation, students are allow to work more autonomously to construct their own learning and culminates in realistic, student generated product (Maliki and Alizadh, 2006).
In order to assess the students' ability and achievement through this OBE approaches, a systematic assessment must be carried out. Systematic assessment has become the requirement for accreditation by various accrediting organizations (Afida et al. 2011). OBE calls for the evaluation of the CO as specified in each course outline (Rozeha et al. 2007). Normally the evaluation of the COs and POs are largely depend on the students' performance in carrying out tasks such as quizzes, final examination, capstone project and submission of assignments which gives an indication of their learning achievements (Rozeha et al. 2007). Evaluation and measurement on the performance output gives an indication on the achievement of POs and COs for each course and automatically it can be used to guide us in determining the appropriate improvement of the teaching method.
For the purpose of OBE assessment on the students achievement, this paper discusses and presents the POs and COs measurement in Engineering Design II course (KH4253) for session 2010/2011. The measurement is based on the submission of the design project, report, students' presentation and peer assessment. Through this measurement, the individual and overall achievement can be obtained.
2. Methodology
This study was conducted to all 64 final year students who registered for Engineering Design II course (KH4253) in Semester 2 of session 2010/2011. Basically, the civil engineering design course consists of 2 parts in which the first semester students are required to register for Engineering Design I course (KH4013) and in the second semester for Engineering Design II course (KH4253). Students were given the real design project in the first semester and they have to continue with the same project in the second semester. Figure 1 shows the implementation procedure of the course in the department. However for the purpose of this paper, the achievement of COs for the course KH 4253 in the second semester are only presented and discussed here. The course was selected because it is compulsory to all students and it covers the overall aspects of civil engineering courses from first to final year study. Furthermore the achievement of several program outcomes (PO) can be measured in this course as compared to the other courses. POs for the course have been identified and linked to each course outcomes (CO) using course mapping as shown in Table 1. Based on the course mapping of POs and COs, there are eight POs that need to be assessed in this course (EAC 2010). Student's achievement then will be evaluated based on these CO-PO relations using assessment tools such as design project, BQ report, presentation and peer assessment (Hamimi Fadziati et al. 2011).
Figure 1. Flow chart of the course implementation
Siti Aminah Osman et al. /Procedia - Social and Behavioral Sciences 60 (2012) 103 -Table 1 Mapping of POs and COs for KH 4253
O o o o o o o o o o Measurement &
No Course Outcome (CO) h- IJ w 4- Jl c\ -J 8 9 1 o Assessment
Methods
Able to describe project site, identify problems, constraints and propose concepts and solutions
Able to identify and apply appropriate parameters, assumptions and design criteria in consideration of health and safety (example: the use of codes of practice), ethics, economics, environment, sustainability Able to carry out manual design calculations based on the required criteria
Able to carry out design and prepare drawings using relevant computer software (Excel spread sheet, AutoCad and other design software)
Able to produce presentable report containing executive summary, introduction, tasks distribution, concepts, design calculations, drawings for tender documentation, conclusions, etc.
Able to perform tasks individually and be an
effective group member.
Able to prepare bill of quantities and cost
estimation
Able to execute and deliver task with integrity and responsibly
Report & Presentation
Report & Presentation
Report & Presentation
Report & Presentation
Report & Presentation
Peer Assessment
BQ Report Peer Assessment
PO1 - Math, science & engineering knowledge PO2 - Problem solving PO3 - Project design PO4 - Ethics
PO6 - Engineering techniques & tools PO7 - Communication PO8 - Teamwork
P010 - project management & entrepreneurship
2.1. Course Assessment
Using the same design project of civil engineering construction that was given in the first semester, students need to complete the project in group. Each group is required to submit the design project report with drawing and BQ report. At the end of the semester students are required to present their work and will be evaluated by the panel consisting of lecturers and professional engineers who have been specially invited to evaluate the student. Students were assessed individually and in group. Besides that, students also need to evaluate their team members (peer assessment) based on their contribution in completing the project.
In this study, each assessment tool has been prepared with specific rubric and been outlined with specific POs and COs that need to be measured. The assessment for each students (individually) and in group were carried out based on the designed rubric. Then the raw marks obtained for each PO and CO were counted and tabulated.
3. Results and Discussion
In this study, four components that contribute to the full mark i.e. design project report, BQ report, presentation and peer assessment were evaluated to measure student's achievement on related CO. These four components with eight COs were measured and used as direct assessment. The assessments of CO1 to CO5 were measured twice in the design report and presentation assessment while the remaining COs were measured only once.
3.1. Design Project Report
Figure 2 shows the students' achievement against CO1, CO2, CO3, CO4 and CO5 for the design project report. From the figure, it shows that the highest number of students with all COs were in the range of 71% - 80% and CO3 was the highest among all the COs in that range. Based on the results of CO3, nearly 69% from the total number of students (N=44) were able to carry out manual design calculations based on the required criteria when solving the problems in design project. These students managed to solve the design project using formulas, methods, standards and software that have been delivered during the lecture. The maximum percentage of CO was given by 91%-100% which consist of CO4 and CO5 whilst the minimum percentage of 41%-50% was given by CO2, CO3 and CO4.
Figure 2. Students' achievement in design project
3.2. BQ Report
Figure 3 shows the students achievement on CO7 for BQ report assessment. In general, the maximum number of students (N=44 students) were in the range of 71% - 100% for CO7. It shows that most of the students were able to prepare bill of quantities and cost estimation. The highest number of students was located at 71%-80% where 25 students manage to get that score. However, five students got the lowest score which is 0%-10% since they have not submitted their group BQ report.
V» 1— I
s Li° 10 I
o 10 z ■ C07
1 1 1 1
?? Ç? <S> ^ t>>' <o* PERCENTAGE {%) c i-
Figure 3. Students' achievement for CO7 in BQ Report
3.3. Presentation
In this assessment, the same COs was measured as in the design report. However, a detail measurement of CO5 (see Table 1) with several aspect of assessment was carried out to test the students ability in communication. From Figure 4, it shows that more than half of the total students have achieved 71% and above for CO5. This proved that students were able to present and defend their design projects during the interview session and they also manage to produce and submitted the full report of design project. Overall, most students done well in oral presentation except for two weak students with the lowest percentage for CO1 and CO2 around 41%-50%. The weak students can be identified from their response and understanding in solving the design project.
Figure 4. Students' achievement in oral presentation
3.4. Peer Assessment
Analysis on CO6 and CO8 for peer assessment as shown in Figure 5 indicates that all students had achieved the highest percentage which is 91%-100% on both COs. It shows that all students agree that their team members were able to perform tasks individually with integrity and responsibility and can work as an effective group member. In
this assessment, each team member must evaluate the contributions made by the group members and also their own self using the peer assessment rubric form.
Figure 5. Students' achievement for CO6 and CO8 in peer assessment
3.5. Overall
Figure 6 shows the achievement of all COs and POs that have been outlined for the course. From the figure, it shows that all students had scored very high on CO6 and CO8 compared to other COs. This was due to the attribution of CO6 and CO8 where the assessment is assessed only from peer assessment. The lowest percentage that the student achieved was 0%-10% for CO7 and the second lowest percentage was 41%-50% for CO2. None of the students were able to get the mark in between 91%-100% for CO1. From this data, it shows that students still have some difficulty in understanding the design project.
Figure 6. Overall students' achievement for CO and PO in KH4253
Whilst for comparison on the achievements of all COs and POs, CO6 and CO8 shows the highest overall achievement of 100% as depicted in Figure 7. CO7 shows the lowest level of achievement of 73% where the assessments were based from the BQ report. Mean while for CO1 to CO5, the achievements were quite similar with the percentage around 76% - 78%. From the overall achievements of all COs and POs in this particular design project course, we can conclude that the performance of the students was relatively good since most of the percentage were between 70% - 100%. This is an indication that the COs specified in this program are generally achieved.
cai coi to? ccw cos coe co? Cos
IPOi) (POi) (Pt>l> (PKJ6t (P07J (pajol {P04)
PO& CO
Figure 7. Total average achievement for all COs and POs 4. Conclusions
As a conclusion the overall students' achievement in Engineering Design II course (KH 4253) for 2010/2011 session was excellent where all the COs have achieved more than 60% marks. Nearly all students in the class have passed the minimum marks of 50% achievement of the overall COs for the course. However, there were five students failed for CO7 as measured in the BQ report and one student got minimum marks of 50% for CO2 that was based on the design report and presentation assessment. The other COs should also not to be ignored in order to ensure that the achievements of all COs can be reached more than 70% in the next session and hence producing students and graduates with higher quality. Results from this study can be used as guidance for the lecturer in monitoring students' performance in each CO that have been outlined for the course. CO achievement reflects the efficiency of the lecturer's teaching method besides identifying the weak students.
Acknowledgement
We would like to thank UKM for providing the research grant (PTS-2011-017 and UKM-OUP-NBT-28-131/2011). References
Abbdulaal, R.M., Al-Bahi. A.M., Soliman, A.Y. & Iskanderani, F.I. (2011). Design and implementation of a project-based active/cooperative
engineering design course for freshmen. European Journal of Engineering Education, 36(4), (pp. 391-402) Afida, A., Hamimi Fadziati, A.W., Norhana, A., Ahmad Ashrif, A.B., Hafizah, H. (2011). Assessment of Student Program Outcomes through a
Comprehensive Exit Strategy. Procedia Social and Behavioral Sciences 18 (2011) (pp. 33-38). Elsevier Ltd. 2011. Hamimi Fadziati, A.W., Afida, A., Wan Mimi Diyana, W. Z., Hafizah, H., Aini, H., Siti Salasiah, M. (2011). Program Outcomes Measurement
and Assessment Processes. Procedia Social and Behavioral Sciences 18 (2011) (pp. 49-55). Elsevier Ltd. 2011. Maliki, A., & Alizadh, P. (2006). Transforming project- based courses to blended learning environment: Case study architectural foundation courses. Current developments in technology-assisted education. http://www.formatex.org/micte2006/pdf/1918-1921.pdf.
Rozeha, A. R., Azami, Z. & Mohd Saidfudin, M. (2007). Application of Rasch Measurement in Evaluation of Learning Outcomes: A Case Study
in Electrical Engineering. Regional Conference on Engineering Mathematics, Mechanics, Manufacturing & Architecture (EM3ARC) 2007. Self Assessment Report Engineering Accreditation Council (EAC), Department of Civil & Structural Engineering, Faculty of Engineering & Built Environment, UKM, 2010.
Siti Aminah, O., Wan Hamidon, W.B., Kamarudin, A.T., Abdul Khalim, R. & Roszilah, H. (2011). The Program Outcomes Achievement for Basic Reinforced Concrete Design Course between Two Different Sessions. Procedia Social and Behavioral Sciences 18 (2011). (pp. 260265). Elsevier Ltd. 2011
