learning analytics: trends and issues of the empirical research of the years 2011-2014

Learning Analytics: Trends and Issues of the Empirical Research of the Years 2011-2014

Nic. Nistor, Michael Derntl, & Ralf Klamma

EC-TEL 2015 Toledo, Spain

1. Introduction

Tasks and methods of Learning Analytics (LA) (Baker & Siemens, 2015) •  predicting learning behaviour and output •  structure discovery: clustering (learner types), social network analysis •  relationship mining •  distillation of data for human judgement: monitoring, visualizing •  discovery with models •  tool development for LA

Ø Open issue…

Nistor, Derntl, & Klamma, EC-TEL2015

2. Purpose

Overview of the empirical LA research 2011-2014 •  educational setting •  data sources •  theoretical framework

from educational perspective •  purpose of data processing •  associated computational methods


3. Methodology

Methodology •  Analysed body of literature: 480 conference papers

–  298 papers from EC-TEL 2011-2014 –  182 papers from LAK 2011-2014

•  197 papers (71 EC-TEL, 126 LAK) were considered relevant


3. Methodology

Methodology •  dynamic topic model using the topic modelling toolkit D-

VITA (http://monet.informatik.rwth-aachen.de/DVita?id=3001)

•  Final manual analysis of 19 most central papers (12 LAK 2014, 7 EC-TEL 2014)


4. Findings

Educational setting •  University/college: 12 cases, [8–10, 12, 13, 15,

17–19, 21, 23, 25] •  School: 6 cases, [11, 14, 16, 20, 22, 24] •  Informal Settings: Total: 1 case, [26]


4. Findings

Data sources – from isolated (scalable) cases to Big Data •  Learning management system log data: 11

cases, [8, 9, 11, 13, 14, 18–20, 23–25] •  Learner-generated data: 4 cases, [10, 12, 21, 26] •  Additionally collected data: 3 cases, [16, 17, 22] •  Available academic administration data: 1 case,

[15]


4. Findings

Theoretical framework from educational perspective •  No Specific Educational Theoretical Framework

Acknowledged: 7 cases, [8, 13, 15, 18, 23–25] •  Linguistic/Logic Approach: 3 cases, [10, 21, 26] •  Situated Learning/Community of Practice: 2 cases, [9, 16] •  Vygotsky’s Zone of Proximal Development: 2 cases, [14, 22] •  Various Approaches: self-regulated learning [12], adaptive

learning environments [11], peer assessment [19], autism [20], or the instructional design concept of learning orchestration [17]


4. Findings

Purpose of data processing •  Learning Trajectory Visualization: 7 cases, [9, 10,

12, 13, 21, 23, 25] •  Predicting Learning Success or Failure/Dropout: 5

cases, [8, 14, 15, 18, 22] •  Diagnosis: 3 cases, [17, 20, 24] •  Assessment: 2 cases, [16, 19] •  Intervention (e.g., automatically adapting or

recommending learning material): 2 cases, [11, 26]


4. Findings

Associated computational methods •  Prediction: 7 cases [10, 15, 16, 18, 21, 22, 24] •  Structure Discovery: 3 cases, clustering [8, 26]

and social network analysis [13] •  Relationship Mining: 6 cases, correlation mining

[12, 17, 19], association rule mining [11] and others with mixed approaches [20, 25]

•  Visualization: 3 cases, [9, 14, 23]


6. Discussion

Summary of findings •  Mainstream:

–  examine log data to visualize learning trajectories and predict the success or failure of university students

•  Innovative studies: –  informal learning in online communities –  video/audio records –  automated student learning assessment –  error/misconception diagnosis


7. Consequences

Recommendation •  Educational and psychological theories urgently

needed for in-depth understanding of educational phenomena, and for significant progress of upcoming LA research


Thank you for your attention! [email protected]


References

LAK 2014 Papers •  6. Pistilli MD, Willis J, Koch D et al. (eds) (2014) Proceedings of Learning Analytics and Knowledge

Conference 2014, LAK ’14. ACM, New York •  8. Bogarín A, Romero C, Cerezo R et al. (2014) Clustering for improving educational process mining. In [6],

pp 11–15 •  9. Clow D (2014) Data wranglers: human interpreters to help close the feedback loop. In [6], pp 49–53 •  10. Coopey E, Shapiro RB, Danahy E (2014) Collaborative spatial classification. In [6], pp 138–142 •  11. Fancsali SE, Ritter S (2014) Context personalization, preferences, and performance in an intelligent tutoring

system for middle school mathematics. In [6], pp 73–77 •  12. Gasevic D, Mirriahi N, Dawson S (2014) Analytics of the effects of video use and instruction to support

reflective learning. In [6], pp 123–132 •  13. Hecking T, Ziebarth S, Hoppe HU (2014) Analysis of dynamic resource access patterns in a blended

learning course. In [6], pp 173–182 •  14. Mendiburo M, Sulcer B, Hasselbring TS (2014) Interaction design for improved analytics. In [6], pp 78–82 •  15. Nam S, Lonn S, Brown T et al. (2014) Customized course advising: investigating engineering student

success with incoming profiles and patterns of concurrent course enrollment. In [6], pp 16–25 •  16. Okada M, Tada M (2014) Formative assessment method of real-world learning by integrating

heterogeneous elements of behavior, knowledge, and the environment. In [6], pp 1–10 •  17. Raca M, Tormey R, Dillenbourg P (2014) Sleepers' lag - study on motion and attention. In [6], pp 36–43 •  18. Santos JL, Klerkx J, Duval E et al. (2014) Success, activity and drop-outs in MOOCs an exploratory study

on the UNED COMA courses. In [6], pp 98–102 •  19. Vozniuk A, Holzer A, Gillet D (2014) Peer assessment based on ratings in a social media course. In [6],

pp 133–137 Nistor, Derntl, & Klamma, EC-TEL2015

References

EC-TEL 2014 Papers •  7. Rensing C, Freitas S de, Ley T et al. (eds) (2014) Open Learning and Teaching in Educational

Communities: 9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings. Lecture Notes in Computer Science, vol 8719. Springer, Berlin

•  20. Cabielles-Hernández D, Pérez Pérez JR, Paule-Ruiz MP et al. (2014) dmTEA: Mobile Learning to Aid in the Diagnosis of Autism Spectrum Disorders. In [7], pp 29–41

•  21. González López S, López-López A (2014) Analysis of Concept Sequencing in Student Drafts. In [7], pp 422–427

•  22. Janning R, Schatten C, Schmidt-Thieme L (2014) Feature Analysis for Affect Recognition Supporting Task Sequencing in Adaptive Intelligent Tutoring Systems. In [7], pp 179–192

•  23. Loboda TD, Guerra J, Hosseini R et al. (2014) Mastery Grids: An Open Source Social Educational Progress Visualization. In [7], pp 235–248

•  24. McTavish TS, Larusson JA (2014) Labeling Mathematical Errors to Reveal Cognitive States. In [7], pp 446–451

•  25. Vahdat M, Oneto L, Ghio A et al. (2014) A Learning Analytics Methodology to Profile Students Behavior and Explore Interactions with a Digital Electronics Simulator. In [7], pp 596–597

•  26. Niemann K, Wolpers M (2014) Usage-Based Clustering of Learning Resources to Improve Recommendations. In [7], pp 317–330
