expert systems with application 36 (2009) chun-hsiung lee, gwo-guang lee, yungho leu

LOGO

Application of Automatically Constructed Concept Map of

Learning to Conceptual Diagnosis of e-learning

Expert Systems with Application 36 (2009)Chun-Hsiung Lee, Gwo-Guang Lee, Yungho Leu

Presenter : Liew Keng Hou

2

Outline

IntroductionPurpose in This StudyResearch ApproachExperiment and Data Analysis- test & AnalysisConclusion and Discussion

3

What is Concept Map?

A B

Epistemological order of concept map

4

Types Concept Map for Learning

Completely manual

Semi-automatic

Automatic

5

Outline

IntroductionPurpose in This StudyResearch ApproachExperiment and Data Analysis- test & AnalysisConclusion and Discussion

6

Purpose in This Study

1. Develop the intelligent Concept Diagnostic System(ICDS) of an automatically constructed concept map of learning by the algorithm of Apriori for Concept Map

2. Teachers were provided with the constructed concept map of learners to diagnose the learning barriers and misconception of learners.

3. Remedial-Instruction Path(RIP) was constructed through the analyst of the concepts and weight in the concept map to offer remedial learning.

4. Statistical methods were used to analyze whether the learning performance of learners can be significantly enhanced after they have been guided by the RIP.

7

Flow Chart of Concept Diagnosis

8

Remedial-Instruction Path

A

B C

Relationships of the epistemological order

D E

Remedial-Instruction Path

9

Outline

IntroductionPurpose in This StudyResearch ApproachExperiment and Data Analysis- test & AnalysisConclusion and Discussion

10

Presetting Conceptual Weight

Question Concept

C1 C2 C3 C4 C5

Q1 1 0 0 0 0Q2 0 1 0 0 0Q3 0.5 0 0.5 0 0

Q4 0.3 0.4 0 0.3 0

Q5 0 0 0 0 1

‘0’: not relevant‘1’: strongly relevant

11

Recording Test Portfolio of Testees

Question Testees

C1 C2 C3 C4 C5 Total

Q1 1 1 1 0 0 3

Q2 1 1 1 1 0 4

Q3 1 1 1 1 1 5

Q4 0 0 1 1 1 3

Q5 0 0 0 1 1 2

‘0’: Student answered correctly the test item‘1’: Student failed to answer correctly the test item

12

Comparison Chart

S1 Q1,Q2,Q3

S2 Q1,Q2,Q3

S3 Q1,Q2,Q3,Q4

S4 Q2, Q3, Q4, Q5

S5 Q3, Q4, Q5

Find Out All Large Item sets

Itemset No. of supports

Q1 3

Q2 4

Q3 5

Q4 3

Q5 2

13

Find Out All Large Item sets(Cont.)

Using association rules of data miningSets Min support(MS) = 0.4

(Depends on teacher)Number of testees = 5Questions with wrong answers given by

testees has to be ≥ MS x N (0.4 x 5 = 2)

14

Find Out All Large Item sets(Cont.)

Itemset No. of support

Q1, Q2 3

Q1, Q3 3

Q1, Q4 1

Q2, Q3 4

Q2, Q4 2

Q3, Q4 3

Q4, Q5 2

Itemset No. of support

Q1, Q2 3

Q1, Q3 3

Q2, Q3 4

Q2, Q4 2

Q3, Q4 3

Q4, Q5 2

MS ≥ 0.40.4 x 5 ≥ 2

15

Ruling the Test Question Association

The confidence level of the test question association rule Q Q is the concept of conditional probability.

It implies that a testee gives a wrong answer to Question Q , there is a probability for the testee to give a wrong answer to Question Q, too

The estimated confidence level formula is

16

Using Association Rules of Data Mining

Question Testees

C1 C2 C3 C4 C5 Total

Q1 1 1 1 0 0 3

Q2 1 1 1 1 0 4

Q3 1 1 1 1 1 5

Q4 0 0 1 1 1 3

Q5 0 0 0 1 1 2

Note: ‘0’: Student answered correctly the test item ‘1’: Student failed to answer correctly the test item

Confidence(Q1 Q2)=P(Q2|Q1) = 100%Confidence(Q1 Q2)=P(Q1|Q2) = 75%

Confidence(Q3 Q2)=P(Q3|Q2) = 80%

17

Using Association Rules of Data Mining

Let the minimum confidence(MC) level be below 70%

Rule 1. Confidence (Q1 Q2) = 100%Rule 2. Confidence (Q1 Q3) = 100%Rule 3. Confidence (Q2 Q1) = 75%Rule 4. Confidence (Q2 Q3) = 100%Rule 5. Confidence (Q3 Q2) = 80%Rule 6. Confidence (Q4 Q3) = 100%Rule 7. Confidence (Q5 Q4) = 100%

18

Relationship Between Concept and Concept

Conversion from “test question association rules” to the effect of “relation between concept and concept”

Q: th test questionC: th conceptRQC: relavance degree between Q and CWCC: relevance degree between Cand X

19

Relationship Between Concept and Concept

1) Q1 Q2 = C1 C2 = Confidence(Q1 Q2) RQ1C1 RQ2C2 = 1 1 1 = 1

2) Q1 Q3 = C1 C3 = Confidence(Q1 Q3) RQ1C1 RQ3C3 = 1 1 0.5 = 0.5

3) Q2 Q1 = C2 C1 = Confidence(Q2 Q1) RQ2C2 RQ1C1 = 0.75 1 1 = 0.75

4) Q2 Q3 = C2 C1 = Confidence(Q2 Q3) RQ2C2 RQ3C1 = 1 1 0.5 = 0.5(0.5 < 0.75(3)) = C2 C3 = Confidence(Q2 Q3) RQ2C2 RQ3C3 = 1 1 0.5 = 0.5

5) Q3 Q2 = C1 C2 = Confidence(Q3 Q2) RQ3C1 RQ2C2 = 0.8 0.5 1 = 0.4(0.4 < 1(1))

= C3 C2 = Confidence(Q3 Q2) RQ3C3 RQ2C2 = 0.8 0.5 1 = 0.4

Question Concept

C1 C2 C3 C4 C5

Q1 1 0 0 0 0

Q2 0 1 0 0 0

Q3 0.5 0 0.5 0 0

Q4 0.3 0.4 0 0.3 0Q5 0 0 0 0 1

Rule 2. Confidence (Q3 Q1) = 100%

Question Concept

C1 C2 C3 C4 C5

Q1 1 0 0 0 0

Q2 0 1 0 0 0

Q3 0.5 0 0.5 0 0

Q4 0.3 0.4 0 0.3 0Q5 0 0 0 0 1

Rule 4. Confidence (Q2 Q3) = 100%

Question Concept

C1 C2 C3 C4 C5

Q1 1 0 0 0 0

Q2 0 1 0 0 0

Q3 0.5 0 0.5 0 0

Q4 0.3 0.4 0 0.3 0Q5 0 0 0 0 1

Question Concept

C1 C2 C3 C4 C5

Q1 1 0 0 0 0

Q2 0 1 0 0 0

Q3 0.5 0 0.5 0 0

Q4 0.3 0.4 0 0.3 0Q5 0 0 0 0 1

20

Relationship Between Concept and Concept

6) Q4 Q3 = C2 C3 = Confidence(Q4 Q3) RQ4C2 RQ3C3 = 1 0.4 0.5 = 0.2(0.2 < 0.5(4))

= C4 C3 = Confidence(Q4 Q3) RQ4C4 RQ3C3 = 1 0.3 0.5 = 0.15

7) Q5 Q4 = C5 C1 = Confidence(Q5 Q4) RQ5C5 RQ4C1 = 1 1 0.3 = 0.3

= C5 C2 = Confidence(Q5 Q4) RQ5C5 RQ4C2 = 1 1 0.4 = 0.4

= C5 C1 = Confidence(Q5 Q4) RQ5C5 RQ4C4 = 1 1 0.3 = 0.3

21

Preliminary Concept Maps(Stage 1)

C1

C2

C3 C4

C5

10.75

0.3

0.3

0.4

0.50.4

0.2

0.5

22

Preliminary Concept Maps (Cont.)

1) Q1 Q2 = C1 C2 = Confidence (Q1 Q2) RQ1C1 RQ2C2 = 1 1 1 = 1

2) Q1 Q3 = C1 C3 = Confidence (Q1 Q3) RQ1C1 RQ3C3 = 1 1 0.5 = 0.5

3) Q2 Q1 = C2 C1 = Confidence (Q2 Q1) RQ2C2 RQ1C1 = 0.75 1 1 = 0.75

4) Q2 Q3 = C2 C1 = Confidence (Q2 Q3) RQ2C2 RQ3C1 = 1 1 0.5 = 0.5 (0.5 < 0.75(3))Q2 Q3 = C2 C3 = Confidence(Q2 Q3) RQ2C2 RQ3C3 = 1 1 0.5 = 0.5

5) Q3 Q2 = C1 C2 = Confidence (Q3 Q2) RQ3C1 RQ2C2 = 0.8 0.5 1 = 0.4 (0.4 < 1(1))Q3 Q2 = C3 C2 = Confidence(Q3 Q2) RQ3C3 RQ2C2 = 0.8 0.5 1 = 0.4

6) Q4 Q3 = C2 C3 = Confidence (Q4 Q3) RQ4C2 RQ3C3 = 1 0.4 0.5 = 0.2 (0.2 < 0.5(4))Q4 Q3 = C4 C3 = Confidence (Q4 Q3) RQ4C4 RQ3C3 = 1 0.3 0.5 = 0.15

7) Q5 Q4 = C5 C1 = Confidence (Q5 Q4) RQ5C5 RQ4C1 = 1 1 0.3 = 0.3Q5 Q4 = C5 C2 = Confidence (Q5 Q4) RQ5C5 RQ4C2 = 1 1 0.4 = 0.4Q5 Q4 = C5 C1 = Confidence (Q5 Q4) RQ5C5 RQ4C4 = 1 1 0.3 = 0.3

23

Preliminary Concept Maps(Cont.)

C1

C2

C3 C4

C5

10.75

0.3

0.3

0.4

0.50.4

0.2

0.5

24

Adjusting Concept Map of Learning(Stage 2)

Child Concept Parent conceptParent C NP

C1 C2 C3 C4 C5Child C1 ― 0 0 0 0.3 1

C2 1 ― 0 0 0.4 2C3 0.5 0.5 ― 0.2 0 3

C4 0 0 0 ― 0.3 1

C5 0 0 0 0 ― 0NC 2 1 0 1 3

NP: Number of father concepts contained in the son conceptNC: Number of son concepts contained in the father concept

25

Complete Concept Map

C5

C1 C2

C3

C4

WC5C1 = 0.3 WC5C2= 0.4

WC5C4 = 0.3

WC1C2 = 1

WC1C3 = 0.5WC2C3 = 0.5 WC4C3 = 0.2

26

Determination of Learning barrier

Calculate the ratio of wrong answers given in the test portfolio:ER(C) =

: weight of the th concept of the th test question which was wrongly answer : weight of the th concept in the whole test paper

27

Question ConceptC1 C2 C3 C4 C5

Q1 1 0 0 0 0Q2 0 1 0 0 0Q3 0.5 0 0.5 0 0

Q4 0.3 0.4 0 0.3 0

Q5 0 0 0 0 1

0.8 0.4 0.5 0.3 0

1.8 1.4 0.5 0.3 1

ER(C) 0.44 0.29 1 1 0

Table of Ratio of Wrong Answer (Failratio)

ER(C1) = = 0.44ER(C2) =

= 0.29

28

Algorithm of Remedial-Instruction Path

010 Void main ()020 Call Find_Remedial-Instruction_Path(k, Fault-Concept)030 End040050 //Cj denotes the FaulConcept, and k denoted the index of a father concept on Cj060 Sub Find_Remedial-Instruction_Path(k,Cj)070 //judge whether the failratio of Concept Cj is greater than the tolerance for the ratio of the giving wrong answers.080 If ER(Cj) failratio then090 Push Cj100 W = Max{WCiCjj1 5 i 5 n}110 While (Cihi RootConcept)do //Not Find to Root-Concept120 push Ci base on W130 Wend140 While Stack is not empty //Find to Root-Concept150 //RIP: Remedial-Instruction_Path160 RIP = Find_Remedial-Instruction_Path(i,Pop())170 Wend180 End if190 End Sub

29

Intelligent concept diagnostic system(ICDS)

30

Intelligent concept diagnostic system(ICDS)

31

Outline

IntroductionPurpose in This StudyResearch ApproachExperiment and Data Analysis- test & AnalysisConclusion and Discussion

32

Design of Experiment and Data Analysis

Target of Study 245 Grade 1 students of a senior high school

Pre-test of “Visual Basic Programming Language”

Table of discrimination index of QuestionsDiscrimination Question

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10

PH 0.93 0.93 0.93 0.93 0.93 0.71 0.86 0.64 0.14 0.86

PL 0.71 0.57 0.36 0.21 0.21 0.14 0.21 0.21 0 0.07

Discrimination Index 0.21 0.36 0.57 0.71 0.71 0.57 0.64 0.43 0.14 0.79

<0.2

33

Flow Chart of Data Analysis

34

Cluster

In order to understand the difference of concept maps produced from the test portfolio of students at different standards

Optimal ratio is 27% for the high-score and low-score clusters

35

Sub-Cluster

1. Experimental group: The RIP in concept map served as the learning guide

2. Control group: Traditional non-guided network learning way was adopted

Group Cluster

Cluster 1(High-score

cluster)

Cluster 2(Medium-

score cluster)

Cluster 3(Low-score

cluster)Experimental group 33 56 33

Control group 33 51 33

Number of students 66 113 66

36

Outline

IntroductionPurpose in This StudyResearch ApproachExperiment and Data Analysis- test & AnalysisConclusion and Discussion

37

- test & Analysis

: significant standardH0 : There is a significant difference

between the mean of experimental group and the mean of control group

If P-value <, then H0 is rejected.

If P-value , then H0 is not rejected

38

-test of Independent Samples of Experimental Group and Control Group of Three Cluster

Cluster and group Item

Mean Standard deviation -value Significance

High-score cluster

Experimental group 72.57 15.83.610 .554

Control group 67.86 12.97

Medium-score cluster

Experimental group 52.25 11.522409 .030*

Control group 40.13 10.23

Low-score cluster

Experimental group 37.29 9.593.695 .003**

Control group 19.29 8.62

* < 0.1* < 0.01

39

Outline

IntroductionPurpose in This StudyResearch ApproachExperiment and Data Analysis- test & AnalysisConclusion and Discussion

40

Conclusion and Discussion

1. Discrimination index of test questions If the test question is too simple or difficult?

2. Attribute of test questions Which type of test question?

3. Learning performance Which cluster(s) has better performance?

LOGO

Thank You for

Your Participating