Avabodhaka: A System to Analyse and Facilitate Interactive Learning in an ICT Based System for Large Classroom Academic research paper on "Computer and information sciences"

CrossMark

Available online at www.sciencedirect.com

ScienceDirect

Procedía Computer Science 84 (2016) 160 - 168

7th International conference on Intelligent Human Computer Interaction, IHCI 2015

Avabodhaka: A System to analyse and facilitate Interactive Learning in an ICT based system for Large Classroom

Nikumani Choudhury^*, T. Venkatesha, Samit Bhattacharyaa, Sandeep Sarmaa

a Indian Institute of Technology Guwahati,781039, India

Abstract

This paper proposes an ICT based system for enhancing teaching and learning in a large classroom and increasing the interactivity to improve learning. Typically, such a classroom suffers from difficulties in audibility of lectures, seamless teacher-student interaction, monitoring attention of students, collecting attendance and conducting examinations. BYOD (Bring Your Own Device)1 paradigm has been assumed for this system. The paper focuses on a method for improving interaction in the classroom by monitoring students' activities. A report for non-interaction with the system is sent to the teacher after regular intervals, which will assist the teacher in identifying students with low interest in the class lecture. All the activities and interaction of the students in the class are stored as activity records in the system through which the level of interaction of the student can be made. The proposed system solves both the problems associated with teaching and learning in a large classroom as well as helps in prevention of frequent use of smartphones for non-learning purposes. An empirical study was conducted to test the acceptability of the system. The results obtained from the experiences of the participants were found to be encouraging and positive.

Keywords:Classroom interaction; E-Learning; Information and Communication technology; User Experience

1. Introduction

Use of network and human centric computing has allowed in extending the traditional learning methodologies and has laid a strong foundation for experimenting with E-learning. Such systems have the power to transform the performance, knowledge and skills of the users as in a traditional class and even more. Application of Information and Communication Technology (ICT) in E-Learning systems has fetched dramatic changes in higher education. In this perspective, Internet fulfills the emerging demand for advanced study material and supplementary resources.

Use of ICT in E-Learning has brought dramatic changes in the educational sector but teaching and learning in a

* Corresponding author. Tel.: +91-9706943408. E-mail address: nikumani@iitg.ernet.in

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Organizing Committee of IHCI 2015

doi:10.1016/j.procs.2016.04.082

large classroom typically has some unusual challenges. The teacher either uses lecture slides projected on to a screen or a blackboard or uses a personal announcement system for lecturing. Students sitting at a distant from the teacher faces difficulty in viewing the slides from the screen and in hearing the lecture clearly. Although the purpose of using a microphone is to make the lecture audible, it is often observed that poor acoustic design of the classrooms makes the lecture inaudible at many parts of the classroom. A vital factor in classroom teaching and learning is the interaction between students and the teacher4 .In traditional large classroom, effective interaction between students and their teacher has been found to be difficult10. It rarely has a lively discussion and interaction. Generally, a limited number of students in the close proximity of the teacher take active part in class discussions. Hence, teachers have difficulty in assessing the concepts of the students and get very inadequate time for resolving questions put forward by the students11. For instance, when an instructor asks for the comments of the students, the number of people attempting to speak rarely reaches 5% to 10% of the strength. However, when asking the same question and allowing answering anonymously with a clicker, the level of participation raises significantly, which shows positive effect on the process of learning6.

In such a large classroom it gets demanding and difficult for the teacher collect and monitor attendance, interact one-to-one basis, scrutinize the performance of the students on a regular basis. The teacher also fails to monitor whether the students are paying attention in class lectures. Due to these factors, the effectiveness of learning decreases.

BYOD1 (Bring Your Own Device) paradigm has been used to develop a system that aims to enable effective interaction in large classrooms using ICT solutions. Different usability metrics have been used to measure task-success, time-on-task, efficiency and learnability. The method proposed for effective interaction and monitoring attention of the students in the class will increase their concentration levels in the classroom. The activities of the students are recorded as activity records which allow understanding the level of interaction of the student in the class quantitatively. Teachers are facilitated with various graphical charts and statistical information which enables them to monitor the progress of individual students in the class.

2. Literature Review

In a large classroom, it is expected that students will have a lot of queries and comments, and to read and answer all the queries by the teacher during the class hour will be time consuming and inefficient solution. Cheung3 gave an approach to use mobile phone messaging to record the responses of the students in the class. Here, the author demonstrates how mobile phone messages can be used to solve the problems and limitations of pencil-paper and classroom interaction.

Anderson et al.2 presented an approach to improve the interaction and learning experience of a class. They developed a system named "Classroom Presenter system" in the University of Washington. This system worked on the "BYOD" philosophy. Teacher can share slides with the students. Students were allowed to make alterations in the slides like highlight the text, writing any comment. Lindquist et al.5 demonstrated the design and use of mobile phone extension to "Classroom Presenter system". In this system, students were permitted to submit solutions of active learning exercises in the form of text or photo messages. However, in the above works, if the teacher goes through all the slides to choose the best one, then he needs a lot of time to search but if he selects any random slides from the server then it is very much possible that the changes made in the selected slide is not very good for the discussion. As mentioned Lindquist presented an extension of Anderson's work and his work also faces the same problem as Anderson's work. But these works are not able to address challenges relating to interaction in the classroom and doesn't discuss about exam conduction or attendance collection.

The NPTEL8 massive on-line courses program, started since January 2013, is another initiative to address the infrastructure bottlenecks prevalent in the tertiary education sector in India, although it does not address the classroom interactivity. Similarly, attempts to setup virtual classrooms and e-learning portals are not suitable to solve the issues in large classrooms. "Moodle9", free and open-source software learning management system distributed under the GNU General Public License help educators to create online courses with a focus on interaction and collaborative learning content, and it is in continual evolution. However, such systems are unable to deal with issues inside a large classroom with an instructor and hundreds of students.

We found little work been done so far relating to analysis of user experiences in large classrooms. Qi et al.7 , performed a case study in The University of Bolton to measure learning experience from using E-learning tools like Video Lecture, online materials, online presentations, discussion on board, chats, etc. The work does not measure user experience for an integrated system but for tools used in E-Learning systems. Domingo et al.18, in their paper focused on user centered design that helps in measuring usability and learning experience. Their work mainly focuses on a model that facilitates the identification of the elements that help us design better and create positive learner experiences for the students in a virtual e-classroom. It doesn't discuss on development of a system for enhancing teaching and learning in large classroom. The next section introduces the E-learning classroom system and its implementation.

Cadwell12, Martyn13, used the method of clicker to answer questions in MCQ based. By using such devices to respond to questions in the class, it was found that individual students pay attention, participate in a better way in the classroom interaction process and their concentration effectively increases. However, these methods are not suitable for a large classroom; because the students have to purchase the clickers or there is a huge cost in distributing clickers to all students in a large classroom by the college.

Kim et al.14 proposed the use of social network service (SNS) to enable effective interaction and improve Engagement in large classrooms through smartphone response system using Twitter. Judd et. al.15 illustrated a novice method of using SNS to send processing request to a computational engine. To improve class interaction mobile applications like "Poll everywhere"16 and "Socrative"17 were developed. The shortcoming of these methods is that internet connection to Twitter is necessary and only a limited number of students can be aided through this systems.

So far, to the best of our knowledge none of the systems support all the activities of a classroom (lecture deliver, interaction, attendance, conducting tests). In particular, none of the systems work towards solving real-time interaction in class-rooms using ICT-based solutions nor are they friendly for use in large classrooms.

3. Proposed Classroom System

The proposed system has been named as Avabodhaka. It provides bi-directional content delivery across diverse mobile and portable computing devices over a low cost wireless network. The PDF files stored in the server database are converted on the fly and displayed on the instructors' device and all the other devices connected to the classroom WLAN. As the instructor navigates through the slides, the system automatically synchronizes the navigation on all the students' devices. Along with the slides, the voice of the instructor is also streamed live over the wireless network to all the students' devices registered in the system. The audio panel also lets the student play/pause, and record the lecture. The system has provision for students to post their questions to the teacher instantly in the class anonymously. Other students with the same query, instead of reposting, can simply 'like' the query posted by the classmate. Discussion forums will help slow learners and help them grasp concepts at a better rate than before. It enables interaction between the teacher and the students both inside and outside the class. A novice method for improving interaction in the class and efficient use of smartphone for only learning process is implemented. It is observed that students may indulge in non-academic activities in the class hours using their smartphones and tablets. All the activities of the students with the system in the class are stored as activity records and are used to quantitatively measure the interaction of the students. If any student is inactive in the system and not taking part in any discussions like pop-up quizzes, polls, query system, for a defined period of time, a report of the student is sent to the teacher's interface. Such an interactive system helps in keeping the traditional classroom flavour intact and enables classroom learning despite the scale. An automatic attendance collection system is built-in, which could be used by the teacher to monitor the student's attendance in classes. Avabodhaka also facilitates the teacher to create quizzes for each class, and conduct and evaluate the same in a class. Teachers are enabled with graphical and statistical information about individual students and students as a whole about their performance in the various examinations. This helps in understanding the learning outcome from the course.

№de Tasks Logout

/ Atiendan« S**wl

Consistency: Unix Approach (Cont'd.)

P Queries Panel

Which nppro-ar.h is belter asynchronous m 3yrichroric<j5 wnte back policy?*?And why??

■»ano? Mis f! Md G

Average time spent per question, in percentage

Figure 1: a) Student interface; b) Teacher interface; c) Query distribution view; d) Time consumption per question

3.1. Application Architecture

The proposed classroom interaction system is based on a 3-tier client-server architecture. It consists of a top level Client or Presentation Tier, in middle a Logic Tier, and at the bottom a Data Tier. The first tier of the application architecture provides a Graphical User Interface (GUI) to the end users (students and instructor). This tier communicates with other tiers by sending information to the browser and other tiers in the system. The second tier, Logic Tier manages the communication between the end user interface and database. It accepts and analyzes the end user's inputs (e.g. login request, PDF page numbers, queries etc), passes the request to the database and responses back to the end users. The third tier of the architecture, Data Tier manages the data. Here data management typically includes storage and retrieval of data, as well as updating the database.

4. System Evaluation

4.1. Usability Metrics

To measure user behaviour upon using a system, usability metrics are applied. In the paper, to measure such behaviour of the user and system efficiency from users' point of view, the following usability metrics are used19. a) Task Success: It is one of the most common usability metrics used. In the paper, we measure binary success, based on whether participants complete a task successfully or not.

b) Time on Task: This metric is generally used where tasks are performed repeatedly by the user. A lot can be said about a product by the time taken by participants to perform a task. In almost every situation, the faster a participant can complete a tsk, the better the experience.

c) Efficiency: It is basically measured by amount of effort required to complete a task. This can be measured by the number of actions required to perform a task or could be evaluated from the users' view using questionnaire.

d) Learnability: It is the extent to which something can be learned. It can be measured by looking at how much time and effort are required to become proficient with something.

4.2 Experimental Setup

For setting up the system, an intel-based IBM System X3300 M4 series server with Ubuntu 14.04 server OS is used. A dual bandwidth Wi-Fi router (Netgear JR6150) was employed for connecting to the classroom network. Users can connect with their devices like laptops, smart phones, and tablets of varying size, to the classroom network via the Wi-Fi router.

For conducting the class lecture delivery, a topic on binary search was chosen with 12 slides, which was prepared from the online material available at nptel. The document consisted of introduction to the binary search method, illustration of the binary search algorithm with examples, searching elements from a list, understanding the complexity and limitations of the algorithm. The duration of the lecture was of 50 minutes. For experimental purpose, two tasks, Task 1 and Task2, consisting of 10 MCQs and 12 MCQs respectively, were prepared that was distributed to the participants after the end of lecture. The questions were prepared from the nptel material and were based on binary search concepts that were covered in the lecture. The time allotted for completing the tasks was 15 minutes each.

A questionnaire was prepared for the participants that were distributed to them at the end of the lecture and after completion of the tasks.

The following statements were put in the questionnaire: a) PDF is clearly visible on your device. b) Slides movement is synchronized with the teacher's voice. c) I think query module has potential to be a useful addition to a classroom. d) This method made lecture more active and interesting. e) I felt more comfortable responding to the poll anonymously to class with this system than I would have with traditional hand-raising polling. f) Recorded lectures and shared PDF slides will help me more compared to the handwritten notes in the traditional classroom.

Statements in the questionnaire were related to their experiences in the classroom using the system. Likert scale data is being used, where 5, 4, 3, 2, 1 stood for "Strongly agree", "Agree", "Neither agree nor disagree", "Disagree" and "Strongly disagree", respectively". In the study, the number of participants was 29, out of which 20 were male participants and 9 female. The age groups of the participants were between 21-26 years. All the student participants were from non-Computer Science background. The users brought their own devices. 18 users brought their personal laptops, 8 participants had mobile smart phones and three participants used tablets. All the participants were well acquainted with their devices.

4.3 Procedure

The participants were introduced about the system and its various features in 30 minutes. They were demonstrated on how to use different modules of the system so that they could get acquainted with it. The entire process of experiment was explained to them so that they had no doubts about using Avabodhaka. The lecture was initiated by the instructor and then the students were to appear in an exam consisting of two tasks, related to the topic covered in the classroom.

User Experience test was conducted that consisted of two phases: In the first phase participants were informed about the usability testing process and how the data was to be collected. Participants' performances in the two assigned tasks were recorded for analysis purpose. In the second phase, students were asked to fill out a questionnaire.

5. Results and Discussion

Students found Avabodhaka to be very helpful and easy to use. All participants were able to identify the "Attendance", "Exam" and "File Download" links on the interface with ease, as could be claimed from the survey. To make slides visible on small screen devices like smartphones was a challenging task. But in the evaluation procedure students with various screen sizes ranging from 3.7 inch to large 16 inches screens were comfortable with the slides as was visible from ratings. The query module received very positive feedback from the students with ratings of 5. It was easier to put questions to the teacher anonymously compared to hand raising system in the traditional classroom system. Also the idea of 'like' in the queries was popular among the students and received 5 ratings from all users. There was a little difficult with audio transmission, which had an initial lag. The audio quality was found to be good, clear from noises but some latency was present in the transmission.

Table 1: Scores and rating given by the participants after using the system

Q no Average Score No. of 5 ratings No. of 4 ratings No. of 3 ratings No. of 2 ratings No. of 1 ratings

1 4.79 23 6 - - -

2 3.45 - 13 16 - -

3 5 29 - - - -

4 4.75 25 1 3 - -

5 4.62 22 3 4 - -

6 5 29 - - - -

Table 1 describes the ratings obtained from the participants after using the system. Ratings were given out of 5. Statements related to query system-lecture delivery and audio systems were well appreciated by the students. A full rating of 5 was obtained for these two in the questionnaire. Q no. 1 that dealt about the clarity of visibility of the lecture document in their devices got an average rating of 4.79. Taking into consideration that students had various types of computing devices with different screen sizes, it was a challenge to synchronize and fit the lecture slide display into the screens. A full screen option when enabled will use the entire screen to fit the display. Q no. 4 related to interactivity and keeping the classroom alive and not dull received an overall rating of 4.75. This indicates that a majority of the students were satisfied, felt comfortable and found Avabodhaka interesting and were able to indulge actively in classroom discussions. This was one of the major aspects of our objective to make the classroom interactive as E-Learning systems were always challenged from being less active than traditional classroom. Q no. 5 received an average rating of 4.62. This dealt with the query system. The users also appreciated the anonymous query posts and the „like" feature to interact with the teacher in the classroom. Q no. 2 relating to the synchronization of the slide movement and the teacher voice received an average rating of 3.45. This received the least rating in the questionnaire. The slides synchronize in the students' devices instantly whenever the teacher navigates from one page to the next, but there exists an initial latency in the transmission of teachers" voice.

The standard deviation of the average ratings in the questionnaire was found to be 0.58, which is closer to 0, and hence indicates the all the average scores are close enough to the mean of the set. Hence, the deviations among the set members are bare minimum. So, we can claim that the obtained average scores or ratings from the questionnaire having a standard deviation of 0.58, signifies a relatively satisfied and positive users of the system towards all the components.

From table 2, we get the 95 percent confidence interval as just over 0.6. This means that we can have a 95 percent confidence that the population mean is 4.60 ratings plus or minus 0.6 ratings. This implies that we can always have a good rating above 4.0 while considering an entire or larger population.

For calculating intervals for Binary success, binomial confidence interval is calculated using adjusted Wald Method. The data is obtained from table 3 that contains the binary success for the two tasks among the 29 participants. The adjusted Wald interval provides the best coverage for the specified interval when samples are less, as in our study.

Figure 2 a) is a plot of task with success rate percentage (along Y-axis) against the two tasks (X-axis), and the associated confidence interval (error bars). It depicts that nearly 69% participants were able to successfully complete

task 1, with a confidence interval (95%) between 50-86% (approx). Because a moderate sample of 29 participants was taken, we get a wider confidence interval. The error percent is depicted using the error bars as shown in figure 10. Also, above 75% of participants were able to successfully complete task 2, with a confidence interval (95%) between 58-92% (approx). The above results depict that the participants were able to complete assigned tasks using our system with high success percentage. Failure could be due to lack of understanding of the concept taught or due to immediate task evaluation after the lecture.

Table 3: Descriptive statistics of time on task measurements_

Task 1 Task 2 Total Participants 29 29

No of successful tasks completed 20 22

Average (in %)_68.96% 75.86%

Standard Deviation(for binary task 0.46 0.42

success)

Average time (in sec) to complete 805.65s 829.93s

Median(time on task in sec) 821s 856s

Geometric Mean(time on task in sec) 799.85s 827.00s Standard Deviation(time on task in sec) 92.72s 69.44s

'« 6«

j -T--«0 -

T**l Trtl T4H2

Figure 2: a) Confidence interval for Task Success; b) Confidence interval for Time on Task

Table 2: Descriptive statistics of average ratings

Average Score

Mean 4.60

Standard Error 0.24

Median 4.77

Mode 5

Standard Deviation 0.58

Sample Variance 0.34

Range 1.55

Minimum 3.45

Maximum 5

Sum 27.61

Largest 5

Smallest 3.45

Confidence Level (95.0%) 0.61

Next we measure the metric Time-on-task. Here the time required to successfully complete task 1 and task 2 are recorded. Participants who didn't complete task are assumed to take 900seconds, which is the full duration at which the task is to be completed. The data is shown on table 3, which displays the time taken by all the participants to complete task 1 and task 2.

Task completion time is an excellent way to measure the efficiency of the system. The participants were able to solve the given tasks after understanding the concept of "binary search" using the system. This gives a fair understanding that more than 70-75% students were able to understand the concept taught using Avabodhaka and were able to reflect back with successful problem solving (tasks).

From table 3, we get the average time for task 1 to be 805s, with median value at 821s. For task 2, average time is 830s with median value at 856s. Figure 2 b) is produced from table 4 considering the confidence intervals. For task 1, the mean time can be 799s plus or minus 35s with confidence 95%. Similarly the mean time for task 2, can be 827s plus or minus 26s with confidence 95%. This infers that Task 1 can be completed within 840s, and Task 2 within 850s when considering an entire population. The majority of the participants were able to complete the tasks within the allocated time frame which shows that the system is well equipped for conducting tasks.

Efficiency and Learnability metrics are measured from the questionnaire provided to all the participants with the 6 statements. As discussed earlier, the participants' found the learning system very easy to understand and use. Efficiency and learnability can also be claimed with fact that around 70% participants were able to successfully

complete task with ease and without help. The positive feedbacks received from the participants after going through the system, describes it to be an efficient E-Learning system.

6. Conclusion

A classroom system, Avabodhaka that utilizes the BYOD paradigm has been presented in the paper. The system consists of several features that aim to reach the desired goal of improving the overall learning experience in a large classroom using ICT tools. The paper illustrates how different modules of the system can help in effective teaching and learning in a large classroom. Use of attention monitoring method by sending periodic reports to the teacher and pop up quizzes helped in increasing interaction in the classroom. This also lowered use of smartphones and tablets for non-learning purposes in the classroom premises. Thereby, the paper provides a single comprehensive system that addresses the problems associated with a large classroom where, the increasing number of students doesn't hinder the interaction and learning between the students and teacher.

To measure the user experience after using Avabodhaka, an empirical study was conducted among a group of students. The results obtained from the experiment provide a positive justification of the overall acceptability of the system among the students. It has been observed that improvements in minimizing the latency of audio streaming and enhancing interaction using collaborative learning must be achieved.

Acknowledgement

We would like to thank Ankur Chaudhary and Sasupilli Madhuri for their ingenious contribution towards developing Avabodhaka. This work has been supported by the project "ICT-based Framework to Enhance the Teaching and Learning Experience in a Large Classroom" (File No. 11(10)/2013-ELG) of e-Learning Division, Department of Electronics & Information Technology (DeitY), Govt. of India.

References

1. Bring Your Own Device (BYOD): "A Guide for Schools". (2012). http://education.alberta.ca/media/6749210/byodguiderBvised2012-09-05.pdf/. (2015)

2. Anderson, R., Anderson Ruth, Simon, B., Wolfman, S. A., VanDeGrift, T., and Yasuhara, K., "Experiences with a Tablet PC Based Lecture Presentation System in Computer Science Courses", SIGCSE, Norfolk, Virginia, USA (March 2004)

3. Cheung, S.L., "Using Mobile Phone Messaging as a Response Medium in Classroom Experiments", Journal ofEconomic Education, June 2006

4. Erickson, J., and Siau, K., "e-Ducation. Commun." ACM2003, 46, 9 134-140. DOI: http://dx.doi.org/10.1145/903893.903928

5. Lindquist, D., Denning, T., Kelly, M., Malani, R., Griswold, W. G., and Simon, B., "Exploring the Potential of Mobile Phones for Active Learning in the Classroom", SIGCSE707, Covington, Kentucky, USA, March 7-10.

6. Yourstone, A., and Kraye., "Classroom Questioning with Immediate Electronic Response: Do Clickers Improve Learning?", Decision Sciences Journal ofInnovative Education, 2008.

7. Qi, B., Liu, L., and Wang, C., 2009. E-learning Tools to Improve Students' Learning Experience: a case study, I.J.Modern Education and Computer Science, 1, 1-9 Published Online October 2009 in MECS (http://www.mecs-press.org/).

8. National Programme on Technology Enhanced Learning(NPTEL) http://nptel.ac.in/ (Accessed: 21 August 2015)

9. Moodle- Open Source Learning Platform https://moodle.org/ (Accesed: 21 August 2015)

10. E. K. Kim, J. E. Cho, and E. C. Jung, "The study on alternatives for activating communication between instructor and students in large-scale lecture.," J.Korea Soc. Design Forum, vol. 22, pp. 225-234, 2009

11. I. Hwang and S. H. Lee, "An analysis of hindrance factors of students' questioning in the university lecture class," J. Educ. Studies, vol. 42, no. 1, pp. 181-212, Apr. 2011.

12. Caldwell, Jane E. "Clickers in the large classroom: Current research and best-practice tips." CBE-Life Sciences Education 6.1 (2007): 9-20

13. Martyn, Margie. "Clickers in the classroom: An active learning approach." Educause quarterly 30.2 (2007): 71.

14. Kim, Youngjae, et al. "Smartphone Response System Using Twitter to Enable Effective Interaction and Improve Engagement in Large Classrooms." Education, IEEE Transactions, Vol 58, Issue: 2, (2015).

15. B.C.Juddand, C.A.Graves,"CellularSTEM,"in Proc. Int. Symp. Cluster, Cloud Grid Comput. , 2012, pp. 799-804.

16. Kolb, Liz. "Adventures with cell phones." Educational Leadership 68.5 (2011): 39-43.

17. Méndez, David, and Josip Slisko. "Software Socrative and smartphones as tools for implementation of basic processes of active physics learning in classroom: An initial feasibility study with prospective teachers." European Journal of Physics Education 4.2 (2013).

18. Domingo, M. G., and Mor, E., 2007. User Centered Design In E-Learning Environments:From Usability To Learner Experience. http://eprints.uoc.edU/research/bitstream/10363/600/1/EDEN07b_garreta.pdf

19. Tullis, T., and Albert, B., 2008. Measuring the User Experience, Collecting, Analyzing and Presenting Usability Metrics, Elsevier.