Scholarly article on topic 'Green Methodology for Learning Assessment'

Green Methodology for Learning Assessment Academic research paper on "Educational sciences"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Technology
OECD Field of science
Keywords
{"mobile learning" / "peer learning" / "green methodology" / "learning assessment" / "vocational education and training."}

Abstract of research paper on Educational sciences, author of scientific article — Liviu Moldovan, Ana-Maria Moldovan

Abstract In this paper a pear learning approach is proposed to help trainees learn from peers and make reflections in vocational education and training (VET) by using mobile devices. Finally it is created a green learning arena by employment of peer assessment, a new methodological approach in 5 steps. It is used the free available software Google Form. To evaluate the effectiveness of the proposed approach, an experiment was conducted in an environmental management course. A total of 48 trainees participating in the experiment were assigned to an experimental group and a control group. The trainees in the experimental group learned with the peer assessment, while those in the control group learned with the conventional approach. It was found from the experimental results that the proposed interactive approach significantly improved the trainees’ learning motivation and learning achievements.

Academic research paper on topic "Green Methodology for Learning Assessment"

ELSEVIER

Available online at www.sciencedirect.com

ScienceDirect

Procedía Technology 22 (2016) 1176 - 1183

Procedía

Technology

9th International Conference Interdisciplinarity in Engineering, INTER-ENG 2015, 8-9 October

2015, Tirgu Mures, Romania

Green Methodology for Learning Assessment

Liviu Moldovana'* Ana-Maria Moldovanb

a"Petru Maior" University of Tirgu Mures, 1 Nicolae Iorga street, Tirgu Mures 540088, Romania bEconomic College "Transilvania " Tirgu Mures, 1 Calimanului street, Tirgu Mures 540074, Romania

Abstract

In this paper a pear learning approach is proposed to help trainees learn from peers and make reflections in vocational education and training (VET) by using mobile devices. Finally it is created a green learning arena by employment of peer assessment, a new methodological approach in 5 steps. It is used the free available software Google Form. To evaluate the effectiveness of the proposed approach, an experiment was conducted in an environmental management course. A total of 48 trainees participating in the experiment were assigned to an experimental group and a control group. The trainees in the experimental group learned with the peer assessment, while those in the control group learned with the conventional approach. It was found from the experimental results that the proposed interactive approach significantly improved the trainees' learning motivation and learning achievements. © 2016 The Authors.Publishedby Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the "Petru Maior" University of Tirgu Mures, Faculty of Engineering Keywords: mobile learning; peer learning; green methodology; learning assessment; vocational education and training.

1. Introduction

Mobile technology becomes more and more popular in various fields like medicine [24], languages, literacy and cultures [5], game-based learning [10], aural skills [6], nursing [15], mathematics and engineering [26], industrial management [7, 8], quality assurance [4, 16], automotive [23] in some situation adopted in a whole college [2]. Anyhow cultural differences influence adoption of mobile learning [3], but recent studies demonstrate that almost half of the number of teachers support mobile phones in the classroom perceived many functions as being useful in

* Corresponding author. Tel.: +4-074-049-8427; fax: +4-026-521-1838. E-mail address: liviu.moldovan@ing.upm.ro

2212-0173 © 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the "Petru Maior" University of Tirgu Mures, Faculty of Engineering doi:10.1016/j.protcy.2016.01.165

the educational process [22] one of the reasons being the online software that is flexible, intuitive, easy and fast to use [25].

In order to employ the new "e" mobile technologies dedicated for evaluation purposes, "Petru Maior" University of Tirgu Mures in partnership with other vocational education training institutions in Europe have developed the eQvet-us project entitled "European Quality Assurance in VET towards new Eco Skills and Environmentally Sustainable Economy" [27]. One of the main purposes of this project is to develop a totally new innovative Peer Learning Evaluation Method, for Eco skills evaluation of trainees participating in VET programs. Taken into account this new approach technology is used to turn the assessment process into a more simplified and learning friendly process [21].

Some results and conclusions from the testing phase of the eQvet-us project are presented in this paper. 2. The green learning arena

Evaluation of teaching may be summative (to provide data for use in making decisions regarding reappointment, tenure, promotion, and merit raises, and for selection of award recipients) or formative (to improve the teaching of the instructor being evaluated).

Development of "e" mobile technology can be done by means of new educational methods that are based on Peer learning processes, where trainees learn from their peers [19, 20]. Peer-assessment is used to help trainees develop assessment criteria learn from viewing peers' work by using mobile device [14].

In eQvet-us project we propose a new evaluation model, where the assessment results from several tests and/or the final exam (summative assessment) in a class can be turned into an active, creative and collaborative peer learning process by the use of immediate feedback.

The system consists of three main parts (Fig. 1): a) Controller interface, used by the trainer; b) Assessment interface, used by trainees or people filling the questionnaire; c) Server, controlled by Gmail administrators, which is a public free service.

/■ N

Assessment interface

Trainee V V

. ~ - _ ' i

V >---s. '

r \ ' '

"I N f /

Server

Gmail v_y

Fig. 1. The two actors in the assessment process, connected via internet.

The time to complete the assessment of a web-based non-optimized questionnaire is significantly different when completed with a smart phone versus a computer [11]. However, there were no significant differences in personality scores.

The peer learning processs for trainees is created by employment of mobile technology and regulate assessment of the course material as a result of the symbiosis between training and assessment, both being represented in figure 2 by the visual design element Taijitu.

In subsidiary, by employing the instant feedback there is created a green learning arena, on which inputs come also from trainer after the evaluation process.

In this way by employment of mobile technology assessments and evaluations can now be done in class or outside of class, using smart phones, tablets or laptops [17, 18]. This is a green initiative that completely eliminates reams of paper used in the past for bubble sheets and comment sheets, and associated printing and scanning.

Control interface

Trainer

If the information from an assessment system to the trainer is correct, dependent upon the results, the teacher can organise teaching in many different ways. With this assessment solution the target is to provide the teacher with instant feedback on the status of the assessment. Teacher analyses what questions did the trainees solve correct and what questions caused more problems.

The trainee might have spent time on parts of the assessment but have just not found or understand the right answer. Dependent upon the nature of the subject being thought, there might be just a hint from a peer student or a teacher that might solve the problem. The entire idea here is to make the assessment a green arena for learning by employment of mobile technology.

3. The peer learning assessment methodology

Very different methods and contexts for the use of peer assessment in classes, supported by different learning software tools: Student Response System [1], Virtual learning environments [12, 13], WebPA Peerwise software tool [9], are reported. The key issue in this approach is to provide a set of tools that the instructor can use and feel free to apply different educational path in the learning activity.

Our development employs free available software Google Forms [28]. The methodological approach of Peer Learning Assessment by employment of Google Forms includes several steps:

• Step 1: Creation of a Gmail account by the trainer and development of a test before the course starts (Fig. 3). The test can be done with questions answered in multiple ways: text, paragraph text, multiple choice, check boxes, choose from a list, scale, grid, date, time.

Page 1 of 1

Untitled form

Form Description

Question Title Untitled Question

Help Text

Question Type Text Paragraph text

! Their answer Multiple choice

Checkboxes

► Advanced settings Choose from a list

Done Scale

Add item ▼ Date Time

Fig. 3. Creation of an account.

• Step 2: Starting a typical assessment session consists in giving/sending the address of the test to the trainees, where from they may access it (Fig. 4).

Fig. 4. Start of an assessment. The duration of the assessment can be limited by using the form limiter function (Fig. 5).

date and time ±

® accepting form responses... on

O June 2015 O

sti mo Tu we Th Fr sa

ll2J a]4J s]

7 8 9 10 11 12 13

| .1.411 151| 161| 17|| ia|| 19|| 20

21 22 23 24 25 26 27 28 29 30

Q Email Form owner when submissions are closed

Fig. 5. Assessment duration limit.

• Step 3: Trainees have to connect to internet address provided by trainer and they may see the test and answer it (Fig. 6). The access can be done from computers in the class, laptops, tablets or touch sensitive mobile phones. At the end of the test they submit it.

Questionnaire 3a-Behaviour evaluation -Trainee evaluation

'Obligatoriu

Send form Lit*, to iliare X

http ,'igoo {(IrtcrmW* o^rtJfiF w у Short Uli ЗИП (ИМ

В Send term via «mail

V Intlkrii 1*1*1 и «Ml

- Cuslomitft ПЧМЗД* and ntbjfttl $ubj«l RiMuili, «vjhfitbon quMlibn. Custom irwu^gt t*fe

СапбИ

Looking to inerte «her «м«в lo tfss fam? и : dlabocuwa

Fig. 6. Trainee interface.

• Step 4: The result consideration phase is after the test submission, when the trainees get a short break.

Instructor uses this phase to obtain a complete overview of the results submitted by the trainees, for the class (Fig. 7) or as individual (Fig. 8).

Fig. 7. Class response result. Questionnaire 3a-Behaviour evaluation - Trainee evaluation (räspunsuri) M

Fiçier Editare Vizualizars InserarB FormatarB Date instrumente Formular Suplimente Ajutor Toate moditicanle salvate Tn Dnve

çj T lei % .0 123 - Arial 10 B I -S- A Ol Œ = |—j- c-3 H [¡¡i¡ Y 2

5 C D E F G

Name 1 work in environmental Gender assurance in the position of: [The course helped me to understand current issues of Environmental related to the work place] [After the course 1 have identified the requirements of ISO 14001 on the organization's Environmental management system and the application in the organization] [After the course I have identified the requirements of ISO 19001 for auditing Environmental management system and the application in the organization]

Adam Alexandru Male a) Environmental professional; Very high High Neutral

Partcu Nie oleta Female a) Environmental manager: High Very high High

Denes Alexandru Male a) Environmental professional: High Very high High

Zaharie Crinel Male a) Environmental auditor Very high Neutral High

Fig. 8. Student response result.

The post-assessment activities can be done by the instructor as effectively as possible due to the software interface that is designed as a tool which ensures that. A question with a high proportion of incorrect answers from trainees is highlighted, so that the instructor may spend more time on it when reviewing the test. The instructor uses this information to select the problematic questions in order to prepare the most important part, from a learning perspective: the post-assessment activities.

• Step 5: In the post-assessment phase, new questions are elaborated, in connection with the problematic questions from the testing phase. They are used by the instructor to provide verification or elaborative feedback in order to enhance new learning activities. In this way, the assessment system is used to reveal the test results and promote and enhance the peer-learning process. The instructor engages the trainees in a process where they learn from the problematic questions they have just spent time trying to solve.

If the information from an assessment system to the instructor is correct, dependent upon the results, the instructor can organise teaching in many different ways. With the eQvet-us assessment solution the target is to provide the instructor with instant feedback on the status of the assessment. Instructors analyses what questions did the trainees solve correct and what questions caused more problems. Than the instructor can apply the following scenarios:

a) Continue as usual and only give the results;

b) Give the trainees verifying feedback and explain what has been misunderstood;

c) Give the trainees a hint of what might be the problem for this question, but not the actual solution;

d) Give the trainees the results, "this is what you voted" and allow the trainees the possibility to discuss the problem;

e) Or to find other ways.

In case of c) and d) the trainees could be allowed to take part of the test again, or could be after a group / peer discussion be able to renegotiate their response.

The main advantage of entering the test subject just after the test is the problem is fresh in mind since the trainee has just been working with it. The trainee might have spent time on parts of the assessment but have just not found or understand the right answer. Dependent upon the nature of the subject being taught, there might be just a hint from a peer trainee or an instructor that might solve the problem. The entire idea here is to make the assessment a green arena for learning, without using papers.

4. Experimental design

To evaluate the performance of the interactive peer-assessment criteria development approach to helping trainees set criteria and perform peer assessment, an empirical experiment was conducted in an environmental management course on a group of 48 trainees.

The measuring tools of the study included the questionnaires for pre-test, post-test, and reaction evaluation.

At the beginning of the learning activity, the trainees took the pre-questionnaires of reaction evaluation and pretest. Than the trainees were provided with an activity based training in quality management. During the peer assessment activity, the trainees in both the experimental group and the control group shared results, provide comments and receive comments from peers. The only difference between the two groups was the assessment approach. The peer learning assessment methodology was used for the trainees in the experimental group. A traditional assessment was used for the trainees in the control group. After the learning activity, all of the trainees completed the questionnaires of reaction evaluation and post-test.

As it was a new experience for the trainees to learn with the interactive peer-assessment criteria development approach in the quality management course, it is interesting to know whether this new approach had increased their cognitive charge during the learning process.

It was deduced from the reaction evaluation pre-questionaires that the two groups of trainees had equivalent learning motivation before participating in the learning activity, while from the post-questionnaire ratings of the two groups are significantly different. It is concluded that the interactive peer-assessment methodology had a significant impact on improving the trainees' learning motivation regarding the environmental management course and the cognitive charge at final was slightly better.

5. Discussion and conclusions

Use of mobile technology doesn't require any reconstruction of classrooms or other infrastructure at the campus, due to the portability and availability of mobile tools (smartphones, tablets and laptops) among the trainees. Due to this, it is possible to apply this evaluation method in any classroom that is connected to the WI-FI network, thus reducing the costs for carrying out advanced and improved evaluations in large scale training environments with thousands of trainees.

In this study, a green learning arena by employment of assessment was proposed for helping trainees develop knowledge and skills in vocational education and learning. An experiment was conducted in the activity based training of an environmental management course to evaluate the performance of the proposed approach. The experimental results show that the proposed approach significantly improved the trainees' learning motivation and learning achievement.

There are many methodologically questions rising in such an arena, regarding: how the teachers will use the system; how the trainees will like to respond electronically; how the immediate feedback will change the view on assessment; the frequency and dimension of the assessments; the focus of the learning culture more on learning than

on assessment; the relevance of continuous evaluation during a course in comparison with the final exam result; the category of trainees that benefit from the change in the assessment, etc.

Acknowledgements

This publication is supported by a grant financed by the European Commission. It reflects the views only of the authors, and the Commission cannot be held responsible for any use, which may be made of the information contained therein.

References

[1] Almeida C, Moldovan L. Mobile learning methodology for European trainers and VET systems quality improvement. Procedia Technology 2014;12:646-653.

[2] Arnesen K, Talmo T, Stav JB. Implementation of Mobile Response Technology and Activity Based Training in a University College. Proceedings of the 7TH International Technology, Education and Development Conference (INTED2013) 2013;4655-4658.

[3] Arpaci I. A comparative study of the effects of cultural differences on the adoption of mobile learning. British Journal of Educational Technology 2015;46(4): 699-712.

[4] Bucur M. A Study on Business Communication on Corporate Social Responsibility in Romania. Procedia Technology 2015;19:996-1003.

[5] Burston J. Mobile Learning: Languages, Literacies, and Cultures. Language Learning & Technology 2015;19(2):44-49.

[6] Chen CWJ. Mobile learning: Using application Auralbook to learn aural skills. International Journal of Music Education 2015;33(2):244-259.

[7] Cioca M, Cioca L-I, Duta L, Web technologies and multi-criterion analysis used in enterprise integration. Studies in Informatics and Control 2011;20(2):129-134.

[8] Cioca L-I, Cioca M, Using distributed programming in production system management. WSEAS Transactions on Information Science and Applications 2007;4(2), 303-308.

[9] Denny P, Luxton-Reilly A, Hamer J. Student Use of PeerWise System. ITiCSE '08 Proceedings of the 13th annual conference on Innovation and technology in computer science education 2008;73-77.

[10] Furio D, Juan MC, Segui I, Vivo R. Mobile learning vs. traditional classroom lessons: a comparative study. Journal of Computer Assisted Learning 2015;31(3):189-201.

[11] Huff KC. The comparison of mobile devices to computers for web-based assessments. Computers in Human Behavior 2015;49:208-212.

[12] Iantovics B. A new intelligent mobile multiagent system. In Proceedings of the IEEE International Workshop on Soft Computing Applications. Szeged-Hungary and Arad-Romania, IEEE Press 2005;153-159.

[13] Iantovics, B. it A Novel Mobile Agent Architecture. In Proceedings of the 4-th International Conference on Theory and Applications in Mathematics and Informatics. Acta Universitatis Apulensis, Alba Iulia 2005;11:295-306.

[14] Lai, Chiu-Lin; Hwang, Gwo-Jen. An interactive peer-assessment criteria development approach to improving students' art design performance using handheld devices 2015;85:149-159.

[15] Lee MK. Effects of mobile phone-based app learning compared to computer-based web learning on nursing students: pilot randomized controlled trial. Healthcare informatics research 2015;21(2):125-133.

[16] Moica S, Radulescu E. Statistical Controls have a Significant Influence on Non Quality Costs. Cases Study in a Company those Manufacturing Aluminum Castings Components. Procedia Technology 2014;12:489-493.

[17] Moldovan L. Innovative Tools and Services for Vocational Education and Training in Quality Assurance. 2nd Review of Management and Economic Engineering International Management Conference. Proceedings of the 2nd Review of Management and Economic Engineering Management Conference: Management of Crisis or Crisis of Management? 2011;138-146.

[18] Moldovan L. New Evaluation Methods by Means of Open Mobile Technology. Proceedings of 3rd Review of Management and Economic Engineering management conference "A New Dilema Between East and West", Technical University of Cluj-Napoca 2012;306-313.

[19] Moldovan L. Innovative method of peer assisted learning by technology and assessment of practical skills. Procedia Technology 2014;12:667-674.

[20] Moldovan L. New Evaluation Model by Means of Mobile Technology. Procedia Technology 2015;1094-1101.

[21] Moldovan L. Sustainability Assessment Framework for VET Organizations, Sustainability 2015;7(6):7156-7174.

[22] O'Bannon BW. Mobile phones in the classroom: Preservice teachers answer the call. Computers & Education 2015;85:110-122.

[23] Pop LD, Nagy E. Improving Product Quality by Implementing ISO / TS 16949. Procedia Technology 2015;19: 1004-1011.

[24] Shaw CM, Tan SA. Integration of Mobile Technology in Educational Materials Improves Participation: Creation of a Novel Smartphone Application for Resident Education, Journal of surgical education 2015;72(4):670-673.

[25] Stav JB, Bjorkli K, Arnesen K. Experiences with Use of Students' Mobile Devices as Immediate Response Tools. Proceedings of the 7TH International Technology, Education and Development Conference (INTED2013) 2013;3305-3309.

[26] Talmo T, Bjorkli K, Stav, JB. Experiences With Use of New and Modern Digital Learning Environments in Mathematics at Engineering Education. Proceedings of the 7TH International Technology, Education and Development Conference (INTED2013) 2013;4622-4629.

[27] *** Project eQvet-us. Online available at: http://eqvet-us.upm.ro/.

[28] *** https://docs.google.com/forms.