1

(Jeremy)Behavioral Informatics and Interaction Computation Lab (BIIC)

:

()

2016

2016.07.17

2

3

(BSP)

BSP INGREDIENTS

4

()

: +

I. II.

III. IV.

5

BSP INGREDIENTS

6

. . .

7

:

8

(20133 13 )

(20135 29 )

9

10

200/

:

?

11

62.5 ():" "

89 ():

12

:

13

:

14

:

:

-frame Dense Points Tracking

TRAJ

MBHxy

Each = A Unit-level (66ms) -length Derived Video features

: Dense Trajectory Fisher-

1

2

3

1

2

Acoustic LLDs

Each : = A Unit-level (200ms)-length Dense Acoustic Features

Functionals

1: {1, 1}1

1:1

2:1

:1

1:

: Dense Unit Acoustic Features

2: {1, 2}

3: {1, 3}

4: {1, 4}

K-Means Bag-of-word

15

:

late fusion technique

Support vector regression

Support vector regression

+

16

:

1

2

Spearman correlation()

= .

17

:

2

1

2

2

1

10

= .

= .

= .

18

?

19

20

Word2Vec

Hierarchical Probabilistic

21

Word2Vec

22

...

N-gram K-meansAll Documents

BOWper Document

Word2vec

N

23

()

Average support vector regression

Support vector regression

+

= . .

24

()

25

?

26

multi-task learning

()task

Task 1 - feature

Task 2 - feature

Task 8 - feature

.

.

.

Kernel

Multi-task learning

27

= . .

?

28

- . . .

29

30

31

(Taiwan Triage and Acuity Scale, TTAS)

32

(=)

(~200)

(=)

(=)

(=)

33

:

()

SpeakerDiarization

Raw audio-videorecording

S1

S2

Sk

. . . MFCCPitch

Intensity

1 : [1,1]

2 : [1, 2]

: [1,]

34

:

:

S1

35

:

Support vector classification

Support vector classification

Fusion

36

:

72.3%

51.6%

gold standard

37

(: 0-3, : 4-6, : 7-10)

: :

: :

: :

Poker face Talk with smiling

Trembling voice

38

database

Before After

:

:

:

39

Before After

: :

: :

40

Pilot work ()

- . . .

41

( ~ 2-5s)

Global label ()

3-5 minutes

42

Thin-slice

Naumann et al. : Personality

Ovies et al. : Affect style

Oltmanns et al. : Personality disorders

:

Motion Capture(Avatar)

43

The USC CreativeIT database

44

:

:

: 45: 90

45

(multimodal)

(density-weighted)

(mutualinformation)

thin-slice

46

Activation 0.384 0.722

Dominance 0.675 0.834

Valence 0.571 0.822

(Global)

(Spearman )

47

91%

9%

Act.(10% data remain)

Including

Reduced 98%

2%

Dom.(70% data remain)

Including

Reduced

95%

5%

Val.(20% data remain)

Including

Reduced

thin-slice?

48

49

- :

1. 2.

(10)

Activation:

4.4 ()3.8 ()4.6 ()

0

1

1 3 5 7 9

11

13

15

17

19

21

23

25

27

29

31

33

35

37

39

41

43

45

47

49

51

53

55

57

59

61

63

65

67

69

71

73

75

77

79

81

83

85

87

89

91

93

95

97

99

TIME SEGMENTS

Emotion-Rich behaviors

1

0

1

1 3 5 7 9

11

13

15

17

19

21

23

25

27

29

31

33

35

37

39

41

43

45

47

49

51

53

55

57

59

61

63

65

67

69

71

73

75

77

79

81

83

85

87

89

91

93

95

97

99

TIME SEGMENTS

Emotion-Rich behaviors

2

Valence: 4.0 ()3.7 (4.3 ()

52

Assumption: Gold Standard

53

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Act. Val.

Agreement

Entire Slice

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Act. Val.

Correlation

Entire Slice

thin slice

thin slice

54

55

56

()

Pattern

Contextualize

57

Data

evaluation

Always look for insights

58

59

ASD ADOS

Couple Therapy

Affective Computing

Oral Evaluation

Stroke Prediction

BiiC: BSP

fMRI Analysis

Pain Scale

60

61

:

application domain

62

Challenging the status quoMaking a positive impact

63

BiiC lab @ NTHU EEhttp://biic.ee.nthu.edu.tw