Scholarly article on topic 'Usage analysis in the web-based distance learning environment in a foreign language education: Case study'

Usage analysis in the web-based distance learning environment in a foreign language education: Case study Academic research paper on "Economics and business"

CC BY-NC-ND
0
0
Share paper
OECD Field of science
Keywords
{"Usage analysis" / "learners’ behaviour patterns" / "log file" / "data preparation" / evaluation / assessment}

Abstract of research paper on Economics and business, author of scientific article — Daša Klocoková, Michal Munk

Abstract The analysis of behaviour of students in a web-based distance learning environment is one of the most important parts of eeducation optimization. The aim of the paper is usage analysis of course eSyntax, which belongs to compulsory subjects in study program Translation and Interpreting. The authors present a detailed analysis of the user log-on data, on which we can better understand the behaviour of the student in an electronic learning environment. The analysis results showed that the students use the course mostly for communication, handing the assignments and self-testing.

Academic research paper on topic "Usage analysis in the web-based distance learning environment in a foreign language education: Case study"

Available online at www.sciencedirect.com

ScienceDirect

Procedia Social and Behavioral Sciences 15 (2011) 993-997

WCES-2011

Usage analysis in the web-based distance learning environment in a foreign language education: case study

Dasa Klocokovâ a *, Michal Munk a

aConstantine the Philosopher University in Nitra, Tr. A. Hlinku 1, Nitra 949 74, Slovakia

Abstract

The analysis of behaviour of students in a web-based distance learning environment is one of the most important parts of e-education optimization. The aim of the paper is usage analysis of course eSyntax, which belongs to compulsory subjects in study program Translation and Interpreting. The authors present a detailed analysis of the user log-on data, on which we can better understand the behaviour of the student in an electronic learning environment. The analysis results showed that the students use the course mostly for communication, handing the assignments and self-testing. © 2011 Published by Elsevier Ltd.

Keywords: Usage analysis, learners' behaviour patterns, log file, data preparation, evaluation, assessment;

1. Introduction

In the last few years, universities have undergone great changes not only in contents of education but also in using ICT. Using ICT does not only offer the tools used in primary form of learning based on personal presence but also new forms of learning. Distance learning belongs beside present form of education to a still more commonly used form of university education. It results not only from the current change in lifestyle and information technologies oriented society but also from the increasing amount of people with university degree. In present, distance form of education is considered to be the most effective form of learning. The principle is to use as much IT tools and didactic aids under tutorial supervision as possible. This enables students to study independently and using their own pace anywhere and anytime. Therefore it is vital to put emphasis on the creation of didactic aids like electronic learning materials and electronic courses. And for the purpose of the highest efficiency of learning optimize given electronic courses.

The aim of the paper is usage analysis of course eSyntax, which belongs to compulsory subjects in study program Translation and Interpreting. The results of the analysis are important for the further correction and improvement of courses. Results of visit rate analysis of course activities are estimated using association rules analysis and sequence rules which represent the learners' behaviour patterns in a web-based distance learning environment are extracted using sequential rules analysis. The analysis provided us several interesting and surprising results.

* Dasa Klocokovâ. Tel.: +421-905-832-723 ; fax: +421-37-6408-500 . E-mail address: d.klocokova@email.cz .

1877-0428 © 2011 Published by Elsevier Ltd. doi:10.1016/j.sbspro.2011.03.227

2. Electronic course eSyntax

English syntax belongs to basic linguistic disciplines in the study program Interpreting and Translating. It connects all linguistic aspects of a language - phonological, morphological, lexical and semantic ones. It deals with structure of a sentence. It includes the following areas: Simple Sentence and its Sentence Elements, Simple Sentence and its Syntactic Structure, Word Order, Compound Sentence, Complex Sentence, Syntactic and Semantic Functions of Subordinate Clauses.

The course was designed on a university education portal that uses Moodle system for its management of distance learning. Created electronic course has a unified structure. When designing the course we tried to take into account the demand of target group - students of linguistics. The course had to be simple and well organized, not requiring special IT skills. We were motivated by its creation (Verspoor & Sauter, 2000; Hrncir, 2002). It consists of the main part and 13 modules - weeks. The main part contains basic information like the course characteristics, tutor, and participants. Each module consists of electronic book (theoretical study materials), assignments (tasks with open answers with no explicit answer, tutor has to check them), quizzes (exercises with definite answers), glossary (unknown words for particular topic), forum (space for communication among students and tutor and students among themselves) and resources (serves like recommended literature, e.g. for deeper studying).

3. Data source and pre-processing

The automatically saved the user log-on data are our data source. For this reason, this area is also often known as web log mining. In this data we follow the sequences in visiting each course page of user. Each row presents notice about user ID, IP address, the time and the date of the visit, the approaching object etc. We used log file that contains the entities from the e-learning course with 120 students.

With analysis of the user log-on data we can better understand the behaviour of the student in an electronic learning environment. For this purpose the following adjustments (corrections) are made (Munk, Vrabelova & Kapusta, 2010):

a) Data cleaning, data transformation, data integration.

b) Identification of sessions, where the session may be defined as a sequence of the steps, that lead to completing the concrete task (Spiliopoulou & Faulstich, 1999) or as a sequence of the steps, that lead to meeting the concrete target (Chen, Park & Yu, 1996). The simplest method is if we consider the series of clicks in a defined period of time, for example 30 minutes (Berendt & Spiliopoulou, 2000).

c) The reconstruction of activities of a web visitor. Taucher and Greenberg (Taucher & Greenberg, 1997) proved that more than 50% of accesses to web are via backward path. Here comes the problem with the cache of the browser. By the backward path, a query for web server is not running, thus there does not exist a record in the log file. The solution to this problem is path filling. With path filling we add these missing rows into the log file (Cooley, Mobasher & Srivastava, 1999).

By the data preparation we took into account recommendations resulting from series of experiments examining the impact of individual steps of data preprocessing on quantity and quality of extracted rules (Munk, Kapusta & Svec, 2009; Munk, Kapusta & Svec, 2010; Munk, Kapusta, Svec & Turcani, 2010).

The investigated multiple response variables are a variable Course page and a variable Course Activity with categories: assignment, book, course, feedback, forum, glossary, quiz and resource. The sequence/transaction ID variable is a variable Session which identifies sessions based on user ID, IP address and time (30-minute-long time window). In case of sequence analysis the time variable is a variable Unix time which integrates date and time of accesses.

4. Usage analysis of course activities

In the following part we are going to describe the results of association rules analysis, which represents a nonsequential attitude to the data being analysed. We shall not analyse sequences, but transactions, i.e. we shall not include the time variable into the analysis. In our case the transaction represents set of visited course activities by students during one session.

The web graph (Fig. 1 a) visualizes the found association rules; particularly the size of the node represents the support of an element, the line-width - the support of the rule and the brightness of the line - the lift of the rule. We can see from the previous graph, which clearly describes the chosen associations that among the most frequently visited course activities belong: course, quiz, forum, assignment, feedback (support > 80 %), similarly as combinations of pairs of these activities (support > 80 %).

On the other hand (Fig. 1 a), among the lesser frequently visited course activities belong: resource (support = 24 %), glossary (support = 36 %) and book (support = 39 %), similarly as combinations of pairs of these activities and others (support < 40 %).

Similarly we can see (Fig. 1a), that the course activities - book and glossary - occur more frequently jointly in the sets of visited activities of the course than separately (lift = 1.9). The same applies to the course activities - book, resource (lift = 1.6) and resource, glossary (lift = 1.4).

In these cases the highest rate of interestingness was found (lift), which specifies how many times more frequently the visited activities occurred jointly than in case if they were statistically independent. In case that the lift is higher than one, the selected couples occur more frequently jointly than separately in the set of visited course activities by students in each session. However, it is necessary to become aware of the fact that upon characterizing the rate of interestingness - lift, the orientation of the rule makes no odds. In case of the remaining found rules the value of the lift was approximately one.

Association rules 0,917 1,904 Association rules 0,222 1,000

Figure 1. Visualization of the found rules: (a) Web graph; (b) Rule graph

The rule graph (Fig. 1b) visualizes the found association rules; particularly the size of the node represents the support of the rule and the brightness of the node - the confidence of the rule. By the confidence it already depends on the rule's orientation, we should take into account the activities with smallest number of visits.

We can see from the graph (Fig. 1b), that the rule book ==> assignment (confidence = 100 %), has higher confidence than its inverse orientation - rule assignment ==> book (confidence = 42 %). The probability of incidence of activities assignment, quiz, forum, course, feedback under the condition, that the activity book is

present in the set of visited activities, is higher (confidence > 92 %) than in case of conditional probability inversely oriented rules (confidence < 45 %). Similarly, we can judge in case of activities glossary and resource too.

From this fact we can conclude, that there is relatively significant part of students who use the course mostly for handling assignments, self-testing and communication.

5. Usage analysis of course pages

In the following part we are going to describe the results of sequential rules analysis, which can be considered as a valid one, regarding the character of data. Time variable will be included in the analysis, i.e. we shall be aware of the order in which individual students visited individual course pages within the observed period of time. The analysis can bring sequential rules (Table 1), which will be obtained from frequent sequences meeting the minimum support (in our case min support = 0.15).

Let us notice (Table 1) the rule with the highest support first (support = 57.6) and with the fourth highest confidence (confidence = 73.1) - the student, who visited the course page eSyntax, will subsequently visit also the page eSyntax. It does not mean in this case that if the student clicks eSyntax, so he clicks on it again with a 73% probability, but that on the lower level there is a course page with an identical title. The second highest confidence and support (Table 1) of the rules, the presumption of which was formed by the course page eSyntax, showed the rule eSyntax ==> Assignment view all, i.e. if a student accessed the course page eSyntax with more than 53% probability he will access the course page Assignment view all.

Similarly we can see that (Table 1) with more than 23% probability a user shall visit the course pages Practice and Quiz view all, with more than 19% probability he will access the course page Final test, with more than 15% probability he will access the course pages Give an example and Entrance test.

The found rules represent behavioural patterns of the course users.

All identified students" behaviour patterns are connected with handling assignments and self-testing in the course environment.

Table 1. Table of extracted sequential rules

Antecedent ==> Sukcedent Support(%) Confidence^/»)

Entrance test ==> eSyntax 12.121 100.000

Entrance test ==> Entrance test 12.121 100.000

Give an example ==> eSyntax 12.121 80.000

eSyntax ==> eSyntax 57.576 73.077

Quiz view all ==> eSyntax 15.152 71.429

Final test ==> Final test 21.212 70.000

Practice ==> Practice 15.152 62.500

eSyntax ==> Assignment view all 42.424 53.846

Assignment view all ==> eSyntax 24.242 50.000

Assignment view all ==> Practice 12.121 25.000

eSyntax ==> Practice 18.182 23.077

eSyntax ==> Quiz view all 18.182 23.077

eSyntax ==> Final test 15.152 19.231

eSyntax ==> Give an example 12.121 15.385

eSyntax ==> Entrance test 12.121 15.385

6. Conclusions

Students who used electronic course eSyntax were more successful in the final examination. The fact that the course was efficient does not mean that all activities were fully exploited. As showed our research, students used mostly quizzes with immediate feedback where they practised individual topics of English syntax. Another widely used feature was the forum. Through the forum and under tutorial supervision, students clarified problems which they came across during the quiz or assignment. The third most commonly used activity was assignments, which is understandable, because students had to come up with their own examples on particular phenomenon.

Surprisingly, students did not widely use electronic book and resources. The book consists of theoretical learning materials and the activity resource consists of materials or links for deeper study of given topic. We explain this by the fact that these students are studying a non-teaching field of study. These students do not need theoretical knowledge but more practical oriented in their future professions.

The least used activity was glossary. It is also understandable (what we did not realize when designing the course), because these students are focusing on interpreting and translating and their vocabulary is already on high level and furthermore had passed several linguistic disciplines, considering the English syntax is studied in the fourth, or fifth term in the bachelor study.

We can optimize the e-course based on the results of our analysis of each course activity usage and we can make it more effective and bring closer to students' needs. For instance make some less used course activities more attractive for target group - future interpreters and translators. Orient the activity book more on students" needs, whether by implementation of exercises from the professional field or by video and audio recordings. Focus more on authentic or professional materials and connect them with glossary. We will try to direct the sources to more practical skills and professional texts. After implementing the needed changes we can evaluate the impact of the changes on the efficiency of relevant course activities with the similar course analysis.

References

Berendt, B. & Spiliopoulou, M. (2000). Analysis of navigation behaviour in web sites integrating multiple information systems. The VLDB

Journal, Vol. 9, No. 1, pp. 56-75. ISSN 1066-8888. Chen, M., Park, J. S. & Yu, P. S. (1996). Data mining for path traversal patterns in a web environment. ICDCS, pp. 385-392. ISBN 0-8186-73982.

Cooley, R., Mobasher, B. & Srivastava, J. (1999). Data Preparation for Mining World Wide Web Browsing Patterns. Knowledge and

Information System, Springer-Verlag, Vol. 1, ISSN 0219-1377. Hrncir, A. (2002). A Practical English Syntax. Banská Bystrica: UMB, pp. 99. ISBN 80-8055-618-0.

Munk, M., Kapusta, J. & Svec, P. (2009). Data preprocessing dependency for web usage mining based on sequence rule analysis. IADIS

European Conference on Data Mining 2009, ECDM'09, Algarve, pp. 179-181. ISBN 978-972-8924-88-1. Munk, M., Kapusta, J., & Svec, P. (2010). Data Preprocessing Evaluation for Web Log Mining: Reconstruction of Activities of a Web Visitor.

Procedia Computer Science, Vol. 1, No 1, pp. 2267-2274. ISSN 1877-0509. Munk, M., Kapusta, J., Svec, P. & Turcáni, M. (2010). Data Advance Preparation Factors Affecting Results of Sequence Rule Analysis in Web

Log Mining. E+MEconomics and Management, Vol. 13, No. 4, pp. 143-160. ISSN 1212-3609. Munk, M., Vrábelová, M. & Kapusta, J. (2010). Probability Modeling of Accesses to the Web Parts of Portal. Procedia Computer Science, Vol. 2, ISSN 1877-0509.

Spiliopoulou, M. & Faulstich, L. C. (1999). WUM: A Tool for Web Utilization Analysis. Extended version of Proc. EDBT Workshop

WebDB'98, Springer Verlag, pp. 184-203. Taucher, L. & Greenberg, S. (1997). Revisitation patterns in world wide web navigation. Proc. of Int. Conf. CHI'97, Atlanta. Verspoor, M. & Sauter, K. (2000). English Sentence Analysis: An Introductory Course. Amsterdam: John Benjamins Publishing Co., pp. 245. ISBN 90 272 25664.