Identifying foreign language learning profiles in game-based environments by using data mining

Manuel Palomo-DuarteAnke Berns

Andrés Yañez EscolanoJuan Manuel Dodero

Contents

● Introduction● Design and architecture● Case Study

– Settings

– Analysis

● Conclusions

Introduction

● Foreign language courses (ECTS)– few hours of language practice in class

– many hours of independent learning

● We propose using a 3D virtual world to:– Encourage independent learning

– Improve communication skills

– Provide a fun and “natural” learning environment

● But How do students benefit from the experience?– Do all benefit equally?

Design and architecture

● We used the Opensim Virtual world engine– Well-known reliable open source

– Client-server approach

Game design

● Level 1: Memory game– Pre-requisite for level 2

– Single player-mode

– Focus on changing prepositions:

● Combination with the accusative or dative

● And proper gender

– 30+ minutes

Game design

● Level 2: Hidden room– Collaborative game, two players, ~35 minutes

– Reinforce of the previously introduced prepositions

– Game aim: tidying a room using a text-chatStudent1 knows where to collocate the objects

Student2 must collocate the objects in the right place

Case study: settings

● German as a second foreign language undergraduate course, Univ. of Cadiz (Spain)– Beginner level (A1.2 CEFR), 6 ECTS

● 48 hours of face-to-face teaching● 102 hours of independent learning

– Students passed a previous course (A1.1 CEFR)

– 99 students took part in the experiment

Case study: settings

● Experiment steps:

1.Pre-test

2.Playing the game taking one role

3.Post-test 1

4.Playing the game taking the other role

5.Post-test 2

6.Technology Acceptance Model (TAM) questionnaire

Case study: analysis

● Anonymous TAM-based questionnaire [−2,+2]– Students confirm that they benefited from the game

● Interacting with others (average 1.64)● For fostering vocabulary (average 1.42)● Increasing grammatical knowledge (average 1.53)

– Students felt comfortable with the experience● Few issues using technology

Case study: analysis

● Clustering (k-means algorithm)– A well-known data mining technique

– Summarizes global behaviour in groups with similar values in the considered attributes

– Attributes of tests used for clustering:● Number of words written in a text describing the position of

different objects on a picture● Ratio of lexical errors by written word● Ratio of grammatical errors by written word

● Additionally, checked grades in previous A1.1 course

Case study: analysis

● Lexical clustering:– General constant

improvement

– C0 and C1:● Similar pre- level● Very good students● C1 students usually

wrote more words and →? improved more

Case study: analysis

● Lexical clustering:– C2: ~half of students

● Good previous oral skills● Great improvement in

post-test2

– C3: ¼ of students● Wrote few words● Best previous writing

grades● Did not learn so much

with the game?

Case study: analysis

● Lexical clustering:– General constant

improvement

– C0 and C1:● Similar pre- level● Very good students● C1 students usually

wrote more words and →? improved more

Case study: analysis

● Grammatical clustering:– General improvement only after post-test2

– C2: ~half of students, good at participation● Outstanding final results

– C0, C1 and C3: same final level● C0 and C3 had same poor pre-test level

– C3 students wrote always an average number of words and they started improving in post-test1

– C0 students only wrote a reasonable number of words in post-test1 and improved only after post-test2. Needed a second round?

● C1: students between C0 and C1, good previous participation grades →? not comfortable with the game

Case study: analysis

● Lexical and grammatical clustering (5 groups):– C3, 40% of students: very good pre-test results and

the best final results (more words than average)

– The other clusters have similar number of students:● C0 and C4: good at previous tasks

– C0 good at oral tasks and average writing skills in post-test2– C4 made effort to write more, but worst post-test2

● C1 and C2: similar previous performance– C1 poor in vocabulary, good at grammar– C2 the opposite

Conclusions

● Students felt conformable with the game– Although some decreased grades from previous semester

● Particular insights:– Previous oral skills ~ influence in vocabulary

– Good (but not brilliant) student in previous course + good participation got best results in grammar

– Scarce relation to previous grades in mixed analysis

– Some students needed two rounds to improve

● Future work:– Analysis including game-chat logs

– Applying other clustering techniques or association rules

Thank you for your attention!

Questions?Source code available at

https://code.google.com/p/aprendizaje-colaborativo-preposiciones-aleman-en-opensim/