future of voice ux: everywhere and 合流(voice 관련 연구 탐색 및 voice 서비스 통찰...

48
Future of Voice UX: Everywhere and 합류(合流) (Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로) 2017 Billy(최병호)/[email protected] 중앙대학교 교수 홍익대학교 영상대학원(HCI개론 강의)/ 연세대학교 공학대학원(서비스디자인경영 강의)/ 성균관대학교 일반대학원 휴먼ICT융합학과(교수)/ HEDcentric UX미래융합전략연구소(연구소장) InnoUX(대표이사) Research Data: http://www.slideshare.net/BillyChoi/ Blog: http://blog.naver.com/soularchitec Twitter/Facebook: ILOVEHCI

Upload: billy-choi

Post on 14-Apr-2017

325 views

Category:

Design


7 download

TRANSCRIPT

Page 1: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Future of Voice UX:

Everywhere and 합류(合流)(Voice 관련 연구 탐색 및

Voice 서비스 통찰 중심으로)

2017Billy(최병호)/[email protected]

중앙대학교 교수

홍익대학교 영상대학원(HCI개론 강의)/연세대학교 공학대학원(서비스디자인경영 강의)/

성균관대학교 일반대학원 휴먼ICT융합학과(교수)/HEDcentric UX미래융합전략연구소(연구소장)

InnoUX(대표이사)

Research Data: http://www.slideshare.net/BillyChoi/Blog: http://blog.naver.com/soularchitec

Twitter/Facebook: ILOVEHCI

Page 2: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

“넌 세상 바꿔보겠다고 이 짓거리하냐? 난 아닌데!

나는 사람 살려보겠다고 이 짓거리하는 거야.

죽어가는 사람 앞에서 그순간만큼은 내가 마지노선이니까. 내가 물러서면 그 사람은 죽는거고, 내가 포기하지 않고 조금만노력하면 그 사람 사는거고.

낭만!”

Source: 드라마 <낭만닥터 김사부> 20회

Page 3: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Table of Contents

• Voice Fingerprinting & Forensic Science

• Voice Recognition & Smart Healthcare

• Voice Profiling & Digital Interviewing

• Voice Profiling & Call center

2

Page 4: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice Fingerprinting & Forensic Science

Page 5: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: 드라마 <보이스>

Page 6: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

In police and Forensic Scientists, sometimes voice is the only clue available in identifying the criminal.

Source: Gursimarjot Singh Walia, Gurjot Kaur Walia: Level of Asthma: A Numerical Approach based on Voice Profiling (2016)

Page 7: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Sources: • Gursimarjot Singh Walia, Gurjot Kaur Walia: Level of Asthma: A Numerical Approach based on Voice Profiling (2016)• Pragnesh Parmar, Udhayabanu R. “Voice Fingerprinting: A Very Important Tool against Crime”. J Indian Acad Forensic Med. Jan- March 2012, Vol. 34, No.1, ISSN: 0971-0973.

• The voice of each person is differentbecause the anatomy of vocal cavity, oral cavity, nasal cavity, and vocal cords is specific to the individual.

• People in different countries, in fact, people in different parts of the same country, speak with different accents. There are some people who run their words together, and there are others who talk with pauses between their words.

• If a person is having some kind of illness, such as cough, cold, fever etc., or feeling some kind of emotion, such as happiness, sadness, stress, anxiety etc., then their voice would be different from what they sound when they are normal.

비강(鼻腔)

구강(口腔)

성대(聲帶)

Page 8: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: Rita Singh, Joseph Keshet, Eduard Hovy: Profiling Hoax Callers (2016)

Page 9: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

“지금 뉴스에 나오는보이스피싱 사건은 지금인터뷰하는 손자의자작극이에요.

변조로 목소리를 다르게 하려고애를 썼지만 손자의 호흡, 말투, 억양이 일치해요.

사람 목소리는 지문과 같아요.

손자가 자작극을 했을 확률이99.9%에요.”

Source: 드라마 <보이스>

Page 10: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

“대표적인 ‘오원춘사건‘.

피해자가 112에 죽기직전에 전화했는데, 결국 시간이지연되면서 다음날아침 시신으로 발견.

심리분석이 가능한‘보이스프로파일러'가 그전화를 받았다면현장에 돌입할 수있었을 것이다.”

Source: 드라마 <보이스> 홍보영상 https://www.facebook.com/CJTVING/videos/1385055181535656/

Page 11: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: 드라마 <보이스>

Page 12: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

“범죄 현장의 ‘소리'를통해 많은 정보를수집하는 것은 굉장히좋은 수사의 방향이 될수 있다.”

Source: 드라마 <보이스> 홍보영상 https://www.facebook.com/CJTVING/videos/1385055181535656/

Page 13: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

FVC(forensic voice comparison)

• In forensic voice comparison(FVC), Speech recordings from an unknown voice, usually of an offender, are compared with recordings from a

known voice, usually the suspect.

• In the first type, The expert considers their aim to be to say how likely it is, given the evidence, that the suspect said the

incriminating speech.

• In the second type of FVC, The expert’s aim is seen as restricted to estimating the strength of the speech evidence with a

Likelihood Ratio(LR)? In other words, to estimate how much more likely the difference between the suspect and offender

speech samples is, assuming the offender sample has come from the suspect, rather than from another randomly chosen speaker in the relevant population.

• For some time now, The use of a LR has been theoretically recognised as the correct logical framework for the evaluation of

forensic evidence.

Source: Rose, Phil.: Where the Science Ends and the Law Begins: Theory and Reality in Likelihood Ratio-based Forensic Voice Comparison.(2012) References:• Association of Forensic Science Providers: Standards for the formulation of evaluative forensic science expert opinion. Science & Justice 49, 161-164 (2009)• Gonzalez-Rodriguez J., Rose P., Ramos, D., Torre, D. & Ortega-Garcia, J.: Emulating DNA: Rigorous Quantification of Evidential Weight in Transparent and Testable Forensic Speaker Recognition.

IEEE Trans. on Audio Speech and Language Processing 15(7), 2104-2115 (2007)• Balding, D.J.: Weight of Evidence for Forensic DNA Profiles. Wiley, Chichester (2005)• Aitken, C.G.G., Taroni, F.: Statistics and the Evaluation of Evidence for Forensic Scientists. Wiley, Chichester (2004)

12

Page 14: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

LR(Likelihood Ratio) approach

• A crucial desideratum in forensic comparison science? The accuracy (and more recently the precision) of a LR-based FVC(forensic voice

comparison) system are also straightforwardly tested.

• Apart from its correctness, The LR approach has several other important properties. It allows, for example, the combination of evidence of different types, nicely

demonstrated in the testing of both automatic and acoustic-phonetic features in hybrid FVC systems.The LR-based testing of other forensic evidence types is now following:

fingerprints, handwriting and SMS texting.

Source: Rose, Phil.: Where the Science Ends and the Law Begins: Theory and Reality in Likelihood Ratio-based Forensic Voice Comparison.(2012) References:• Morrison, G.S.: Measuring the Validity and Reliability of forensic likelihood-ratio systems. Science & Justice (51) 3, 91-98 (2011)• Morrison: G.S. Forensic voice comparison and the paradigm shift. Science & Justice 49,298-308 (2009)• Gonzalez-Rodriguez J., Drygajlo, A., Ramos-Castro, D., Garcia-Gomar, M., Ortega-Garcia, J.: Robust estimation, interpretation and assessment of likelihood ratios in forensic speaker recognition.

Computer Speech and Language 20, 331-355 (2006)

13

Page 15: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: Rose, Phil.: Where the Science Ends and the Law Begins: Theory and Reality in Likelihood Ratio-based Forensic Voice Comparison.(2012)

• Figure 1 shows the F0 realising the [H.L.LH] intonational pitch of the offender, aligned with its wideband spectrogram.

• F0 on not can be seen to drop from about 200 Hz to 175 Hz; whence it drops further on the nucleus of too to about 125 Hz.

• The F0 shows a small ca. 15 Hz increase from its minimum value of 125 Hz in the /b/ hold, and rises on the nucleus of bad with a slightly convex contour from about 145 Hz to peak at about 185 Hz.

Page 16: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: Rose, Phil.: Where the Science Ends and the Law Begins: Theory and Reality in Likelihood Ratio-based Forensic Voice Comparison.(2012)

• Figure 2 compares the offender F0 with the F0 of the suspect’s 15 not too bad utterances.

• The similarity is considerable, with the offender’s F0 time-course lying completely within, and in some places almost exactly in the middle of, the suspect’s distribution.

• Note too the suspect’s use of both H and L on not.

Page 17: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

• Table shows the parameters that can be extracted using voice analysis, and the information that can be extracted from those voice parameters.

Sources: • Gursimarjot Singh Walia, Gurjot Kaur Walia: Level of Asthma: A Numerical Approach based on Voice Profiling (2016)• Khushboo Batra, Swati Bhasin, Amandeep Singh. “ Acoustic Analysis of voice samples to differentiate Healthy and Asthmatic persons”. International Journal of Engineering and Computer Science,

ISSN 2319-7242, Volume 4, Issue 7, July 2015, Page No. 13161-13164.• Pragnesh Parmar, Udhayabanu R. “Voice Fingerprinting: A Very Important Tool against Crime”. J Indian Acad Forensic Med. Jan- March 2012, Vol. 34, No.1, ISSN: 0971-0973.

※ Extraction of voice parametersThe above parameters were extracted using the MDVP (Model 5105, KayPENTAX) tool of CSL (Model 4500, KayPENTAX) system.

Fo: Average Fundamental FrequencyJitt: Jitter (%)Shim: Shimmer (%)vFo: Coefficient of fundamental frequency variationDUV: Degree of VoicelessDSH: Degree of Sub-HarmonicsSPI: Soft Phonation IndexDVB: Degree of Voice BreaksNHR: Noise-to-Harmonic RatioPPQ: Pitch Period Perturbation Quotient (%)RAP: Relative Average Perturbation (%)To: Average Pitch Period

Page 18: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice 서비스 통찰

1. 공간 구분의 필요성 폐쇄 공간 & 오픈 공간: 지속적으로 상주하는 특정인의 유무 기준으로 구분

2. 폐쇄 공간 내 서비스 예측 및 서비스 디자인 범죄 예방 및 범죄 골든타임 지원 서비스

① 집이나 어린이집 등에서 가까운 관계의 상대에서 가해지는 일시적 또는지속적으로 자행되는 범죄 관련 지원 서비스

② 외부인이 침입 시도 또는 침입하여 발생하는 범죄 관련 지원 서비스

패턴과 Context 기반의 서비스 시나리오 디자인 수요자와 공급자 모두 대상(일시 방문자 및 외부 침입자 포함) 자동 대처와 그 외의 대처 상황 고려 필요 다양한 생태계와 유기적인 공조 서비스 시나리오 디자인 필요 보안 정책 및 사생활 침해 등 윤리 가이드라인 수립 필요

17

Page 19: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice 서비스 통찰(cont.)

3. 폐쇄 공간 내 공급자 제공 기술 예측 범죄 예방 지원 기술

① 지속적으로 상주하는 특정인의 보이스 프로파일링 및 보이스 인증 기술② 일시 방문자의 보이스 프로파일링③ 폐쇄 공간 내 노이즈 프로파일링 및 이상 노이즈 중점 분석 기술④ 행동패턴 누적 및 분석 기술(지속적인 모니터링이 가능한 관제 관련 기술 포함)

범죄 골든타임 지원 기술 영상인식 등 다른 기술과의 연계 지원 기술

18

Page 20: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice Recognition & Smart Healthcare

Page 21: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Sources: • Khushboo Batra, Swati Bhasin, Amandeep Singh: Acoustic Analysis of voice samples to differentiate Healthy and Asthmatic persons(2015)• Rachna, Dinesh Singh, Vikas: FEATURE EXTRACTION FROM ASTHMA PATIENT’S VOICE USING MEL-FREQUENCY CEPSTRAL COEFFICIENTS(2014)• Saloni, R. K. Sharma, and A. K. Gupta. "Disease detection using voice analysis: a review." International Journal of Medical Engineering and Informatics 6.3(2014): 189-209.• Sonu, R. K. Sharma “Disease detection using analysis of voice”, TECHNIA – International Journal of Computing Science and Communication Technologies, VOL.4 NO. 2, January 2012.

• Speech is produced by vocal folds. It involves the interaction of various body parts*. It can hurt the sound quality of the voice.

• Asthma is a lung disease that affects airflow to and fro from lungs. A whistling sound comes when asthmatic patient breathes.

* This includes various components like abdominal, ribcage, lungs, pharynx, oral cavity and nose and each performs its own function in speech production.

Page 22: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Sources: • Gursimarjot Singh Walia, Gurjot Kaur Walia: Level of Asthma: A Numerical Approach based on Voice Profiling (2016)• Khushboo Batra, Swati Bhasin, Amandeep Singh. “ Acoustic Analysis of voice samples to differentiate Healthy and Asthmatic persons”. International Journal of Engineering and Computer Science,

ISSN 2319-7242, Volume 4, Issue 7, July 2015, Page No. 13161-13164.

• There are several voice pathologic disorders related with nasal, neural, respiratory and larynx diseases. (코, 신경, 호흡, 후두관련 질병)

• As a result, analysis and diagnosis of vocal disorders has become an important medical procedure.

Page 23: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Asthma patient’s voice & Voice Recognition

22

Source: Rachna, Dinesh Singh, Vikas: FEATURE EXTRACTION FROM ASTHMA PATIENT’S VOICE USING MEL-FREQUENCY CEPSTRAL COEFFICIENTS(2014)

In the above graphs an analysis of mean of five vowels (a,e,i,o,u) for males and females are presented for various voice parameters like JITTER and SHIMMER of different asthma and healthy persons are compared.

Page 24: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Asthma patient’s voice & Voice Recognition(cont.)

• Asthma has no cure, just it can be controlled. Major risk factors are bedding dust, carpet, furniture dust, also family history or allergy.

• It can be controlled during asthma stages by doing long term meditation daily, regular check up by doctor in case of serious patients, taking some drugs through inhalers when asthma attack came etc.

• Further these extracted coefficients will be analyzed for finding similarities between patients and normal persons.

23

Source: Rachna, Dinesh Singh, Vikas: FEATURE EXTRACTION FROM ASTHMA PATIENT’S VOICE USING MEL-FREQUENCY CEPSTRAL COEFFICIENTS(2014)

Page 25: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice Analysis of Parkinson Disease & Voice Recognition

24

Sources: • Saloni, R. K. Sharma, Anil K. Gupta: Voice Analysis for Telediagnosis of Parkinson Disease Using Artificial Neural Networks and Support Vector Machines (2015)• Max A. Little, P. E. Macsharry, E. J. Hunter, J.Sielman, L. O. Raming, “Suitability Of Dysophonia Measurements for Telemonitoring of Parkinson’s Disease, IEEE Transaction on biomedical engg,Vol.

56,2009, pp 1015-1022.

Page 26: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Acoustic Analysis of voice samples to differentiate Healthy

• A human voice is very closely related to the human health conditions, both physical and mental. Changes in voice quality and pitch occur frequently in hormonal imbalances or deficiencies.

• Through acoustic analysis, factors that affect the production mechanism of human voice leads to the non-invasive diagnosis of diseases.

• The person’s voice suffering from any disease different from the healthy person in some extent. As the various diseases like Parkinson, dysphonia, cardio-vascular, dystharia & respiratory tract infection lay their impression on voice of a person.

• With the help of speech we will extract various information about the speaker, gender, language, emotions health.

25

Sources: • Gursimarjot Singh Walia, Gurjot Kaur Walia: Level of Asthma: A Numerical Approach based on Voice Profiling (2016)• Khushboo Batra, Swati Bhasin, Amandeep Singh: Acoustic Analysis of voice samples to differentiate Healthy and Asthmatic persons(2015)• Cyril R. Pernet et al. “The human voice areas: Spatial organization and inter-individual variability in temporal and extratemporal cortices”. Neuroimage 119(2015) 164-174.• Saloni, R. K. Sharma, and A. K. Gupta. "Disease detection using voice analysis: a review." International Journal of Medical Engineering and Informatics 6.3 (2014):189-209.• Teixeira, João Paulo, and Paula Odete Fernandes. "Jitter, Shimmer and HNR Classification within Gender, Tones and Vowels in Healthy Voices." Procedia Technology 16 (2014): 1228-1237.• Pati, Debadatta and SR Mahadeva Prasanna. "Speaker recognition from excitation source perspective." IETE Technical Review 27.2 (2010): 138-157.• Peng, Ce, et al. "Pathological voice classification based on a single Vowel's acoustic features." Computer and Information Technology, 2007. CIT 2007. 7th IEEE International Conference on. IEEE,

2007.• Friedrich S. Brodnitz. “Hormones and the Human Voice”, Bulletin of the New York Academy of Medicine 03/1971; 47(2):183-91.

Page 27: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice 서비스 통찰

1. 공간 구분의 필요성 폐쇄 공간 & 오픈 공간: 지속적으로 상주하는 특정인의 유무 기준으로 구분

2. 폐쇄 공간 내 공급자 제공 기술 예측 질병 예방 지원 기술

① 지속적으로 상주하는 특정인의 질병 프로파일링 및 보이스 인증 기술② 외부인의 보이스 프로파일링으로 질병 예측 및 감염 여부 분석 기술③ 폐쇄 공간 내/외 환경 분석 기술④ 지속적인 모니터링이 가능한 관제 관련 기술

골든타임 지원 기술 공기 측정, 스마트 드러그 서비스, 영상인식 등 다른 기술과의 연계 지원 기술

26

Page 28: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice 서비스 통찰(cont.)

3. 폐쇄 공간 내 서비스 예측 및 서비스 디자인 질병 예방 및 골든타임 지원 서비스

① 집이나 어린이집 등에서 질병 (감염) 유발 환경 모니터링② 경증, 중증 환자의 경우, 지속적인 건강 상태 모니터링

패턴과 Context 기반의 서비스 시나리오 디자인 수요자와 공급자 모두 대상(외부인 포함) 자동 대처와 그 외의 대처 상황 고려 필요 다양한 생태계와 유기적인 공조 서비스 시나리오 디자인 필요 보안 정책 및 사생활 침해 등 윤리 가이드라인 수립 필요

27

Page 29: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice Profiling & Digital Interviewing

Page 30: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: Dave Winsborough and Tomas Chamorro-Premuzic: Talent Identification in the Digital World: New Talent Signals and the Future of HR Assessment(2016)

Page 31: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

New Talent Signals and the Future of HR Assessment

• Through the addition of innovations, such as text analytics and algorithmic reading of voice-generated emotions, a wider universe of talent signals can be sampled.

• In the case of voice mining, candidates’ speech patterns are compared with an “attractive” exemplar, derived from the voice patterns of high performing employees. Undesirable candidate voices are eliminated from the context, and those who fit move to the next round.

• More recent developments include video-mediated scenario-based questions, images, video, and work samples and automated reading of micro-emotions during the interview.

• For example, Hirevue.com, a leading provider of digital interview technologies, employs coding challenges to screen software engineers for their software writing ability. Likewise, Uber uses similar tools to test and evaluate potential drivers exclusively via their smartphones.

30

Sources: • Tomas Chamorro-Premuzic, Dave Winsborough, Ryne A. Sherman, Robert Hogan: New Talent Signals: Shiny New Objects or a Brave New World?(2016)• Dave Winsborough and Tomas Chamorro-Premuzic: Talent Identification in the Digital World: New Talent Signals and the Future of HR Assessment(2016)

Page 32: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Source: Daniel Chen, Yosh Halberstam, Alan Yu: Covering: Mutable Characteristics and Perceptions of Voice in the U.S. Supreme Court (2016)

Page 33: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

The emotion detection based on speech signal analyses features

1. Pitch

2. Energy (computing Teager Energy Operator – TEO)

3. Energy fluctuation

4. Average level crossing rate (ALCR)

5. Extrema based signal track length (ESTL)

6. Liner prediction cepstrum coefficients (LPCC)

7. Mel frequency cestrum coefficients (MFCC)

8. Formants

9. Consonant vowel transition

32

Source: Batalla, J.M., Mastorakis, G., Mavromoustakis, C.X., Pallis, E.: Beyond the Internet of Things: Everything Interconnected(2017)

Page 34: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

The OCC modelSource: Ortony, A., Clore, G.L., & Collins A. (1990). The Cognitive Structure of Emotions Cambridge Univ. Press

Page 35: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Appendix. Emotional attributions significantly associated with acoustic parameters

34

Source: Klaus R. Scherer: Expression of Emotion in Voice and Music(1995)

Page 36: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Appendix. Microexpressions

• Based on Ekman’s research on emotions (Ekman, 1993), the security sector has developed microexpression detection and analysis technology to enhance the accuracy of interrogation techniques for identifying deception (Ryan, Cohn, & Lucey, 2009). Ekman, P. (1993). Facial expression and emotion. The American Psychologist, 48(4), 384–392. doi:10.1037/0003-066X.48.4.384 Ryan, A., Cohn, J., & Lucey, S. (2009). Automated facial expression recognition system. 43rd Annual 2009 International

Carnahan Conference on Security Technologies, 172–177. doi:10.1109/CCST.2009.5335546

• The recent creation of large databases of microexpressions (Yan, Wang, Liu, Wu, & Fu, 2014) is likely to facilitate the standardization and validation of these methods.Yan, W. J., Wang, S. J., Liu, Y. J., Wu, Q., & Fu, X. (2014). For micro-expression recognition: Database and suggestions.

Neurocomputing, 136, 82–87. doi:10.1016/j.neucom.2014.01.029

• Beyond using automated emotion reading, new research aims to correlate facial features and habitual expression with personality (Kosinski, 2016). Kosinski, M. (2016, January). Mining big data to understand the link between facial features and personality. Paper

presented at the 17th Annual Convention of the Society of Personality and Social Psychology, San Diego, CA.

35

Sources: • Tomas Chamorro-Premuzic, Dave Winsborough, Ryne A. Sherman, Robert Hogan: New Talent Signals: Shiny New Objects or a Brave New World?(2016)• Dave Winsborough and Tomas Chamorro-Premuzic: Talent Identification in the Digital World: New Talent Signals and the Future of HR Assessment(2016)

Page 37: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Voice Profiling & Call center

Page 38: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Reference: Dormehl, Luke (2014-04-03). The Formula: How Algorithms Solve all our Problems … and Create More. Ebury Publishing.

Quantified Self movementSelf-knowledge through numbers

(숫자를 통한 자기 이해)

Based upon speech patterns, the particular words they used, and even details as seemingly trivial as whether they said “um” or “err” – and then utilise these insights to put them through to the agent best suited for dealing with their emotional needs?

(Chicago’s Mattersight Corporation does exactly that. Based on custom algorithms, Mattersight calls its business “predictive behavioral routing”.)

Page 39: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Quantified Self movementSelf-knowledge through numbers

(숫자를 통한 자기 이해)

Based upon speech patterns, the particular words they used, and even details as seemingly trivial as whether they said “um” or“err” – and then utilise these insights to put them through to the agent best suited for dealing with their emotional needs?(Chicago’s Mattersight Corporation does exactly that. Based on custom algorithms, Mattersight calls its business “predictive behavioral routing”.)

The man behind Mattersight’s behavioural models is a clinical psychologist named Dr Taibi Kahler. Kahler is the creator of a type of psychological behavioural profiling called Process Communication.

What Kahler noticed was that certain predictable signs precede particular incidents of distress, and that these distress signs are linked to specific speech patterns. These, in turn, led to him developing profiles on the six different personality types he saw recurring.

Reference: Dormehl, Luke (2014-04-03). The Formula: How Algorithms Solve all our Problems … and Create More. Ebury Publishing.

Page 40: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Quantified Self movementSelf-knowledge through numbers

(숫자를 통한 자기 이해)

Based upon speech patterns, the particular words they used, and even details as seemingly trivial as whether they said “um” or“err” – and then utilise these insights to put them through to the agent best suited for dealing with their emotional needs?(Chicago’s Mattersight Corporation does exactly that. Based on custom algorithms, Mattersight calls its business “predictive behavioral routing”.)

The man behind Mattersight’s behavioural models is a clinical psychologist named Dr Taibi Kahler. Kahler is the creator of a type of psychological behavioural profiling called Process Communication. What Kahler noticed was that certain predictable signs precede particular incidents of distress, and that these distress signs are linked to specific speech patterns. These, in turn, led to him developing profiles on the six different personality types he saw recurring.

A person patched through to an individual with a similar personality type to their own will have an average conversation length of five minutes, with a 92 percent problem-resolution rate. A caller paired up to a conflicting personality type, on the other hand, will see their call length double to ten minutes – while the problem-resolution rate tumbles to 47 percent.

Reference: Dormehl, Luke (2014-04-03). The Formula: How Algorithms Solve all our Problems … and Create More. Ebury Publishing.

Page 41: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Personality type Personality traits How common?

“Thinkers”Thinkers view the world through data. Their primary way of dealing with situations is based upon logical analysis of a situation. They have the potential to become humourless and controlling.

1 in 4 people

“Rebels”Rebels interact with the world based on reactions. They either love things orhate them. Many innovators come from this group. Under pressure they can be negative and blameful.

1 in 5 people

“Persisters”Persisters filter everything through their opinions. Everything is measured upagainst their world view. This describes the majority of politicians.

1 in 10 people

“Harmonisers”Harmonisers deal with everything in terms of emotions and relationships. Tight situations make this group overreactive.

3 in 10 people

“Promoters”Promoters view everything through action. These are the salesmen of the world, always looking to close a deal. They can be irrational and impulsive.

1 in 20 people

“Imaginers”Imaginers deal in unfocused thought and reflection. These people operate in vivid internal worlds and are likely to spot patterns where others cannot.

1 in 10 people

Dr Taibi Kahler’s the six different personality types

Reference: Dormehl, Luke (2014-04-03). The Formula: How Algorithms Solve all our Problems … and Create More. Ebury Publishing.

Page 42: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Similarity-attraction(유사성-매력 전략)(1/4)

• Personality traits(성격 특성); 5살 무렵에 형성• Four Personalities

41

비판형 외향형

내향형 수용형

냉정형 다정형

순응형

지배형

협력

* Source: Clifford Nass & Corina Yen, 2010

Page 43: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Similarity-attraction(유사성-매력 전략)(2/4)

• This is a reproduction of one of the most famous of the Tiffany stained-glass pieces—the colors are absolutely sensational! This first-class, handmade copper-foiled stained-glass shade is over six and one-half inches in diameter and over five inches tall. I am sure that this gorgeous lamp will accent any environment and bring a classic touch of the past to a stylish present. It is guaranteed to be in excellent condition! I very highly recommend it.

• This is a reproduction of a Tiffany stained-glass piece. The colors are quite rich. The handmade copper-foiled stained-glass shade is about six and one-half inches in diameter and five inches tall.

42

* Source: Clifford Nass & Corina Yen, 2010

Page 44: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Similarity-attraction(유사성-매력 전략)(3/4)

• You should definitely select option A instead of option B. There are at least six reasons why this is the right option. I am 90 percent confident of this assessment.

• Perhaps you should select option A instead of option B? It seems like there are reasons why this might be the right choice. I am 40 percent confident of this assessment.

43

* Source: Clifford Nass & Corina Yen, 2010

Page 45: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

© 2017 Billy All rights reserved.Future of Voice UX: Everywhere and 합류(合流)(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

Similarity-attraction(유사성-매력 전략)(4/4)

• Similarity-attraction affects people to such a degree that they feel positive toward not only similar people but also anything associated with those similar people. For example, in the experiment, not only did participants like the sellers who were similar to themselves, they also felt more positive about the items associated with the similar sellers.(유사성-매력 효과는 긍정적 감정 뿐만 아니라 유대감 유발. 심지어성격이 비슷한 판매자가 경매에 올린 제품까지 선호)

• 외향성 음성과 내향성 음성 동일 적용; 음량, 음역, 음성 속도; 성격과 음성의 일관성중요하게 판단함

• When the introduction to a computer-based “Entertainment Guide” matched users’ personalities, users found the recommended music to be significantly better, even though the recommendations themselves were identical.(동일 음악을 추천하여도서비스 도입부가 자신의 성격에 부합하면 선호 발생)

44

* Source: Clifford Nass & Corina Yen, 2010

Page 46: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

“매 순간 정답을 찾을 수는 없지만, 그래도 김사부는 항상 그렇게 말했다.

우리가 왜 사는지,무엇 때문에 사는지에 대한질문하는 것을 포기하지 마라.

그 질문을 포기하는 순간,우리의 낭만도 끝이 나는 거다.

알았냐?

라고 말이다.”

Source: 드라마 <낭만닥터 김사부> 20회

Page 47: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

“거기서는 생명존중이나인간의 존엄 같은 것은 없어.

그냥 하루하루 죽지 않고버티는 거. 그것만 생각해.

같은 시간을 살아가는데, 그런 세상이 존재한다는 거. 믿겨져?”

Source: 드라마 <낭만닥터 김사부:> ‘Appendix. 그 모든 것의 시작’ 편

Page 48: Future of Voice UX: Everywhere and 合流(Voice 관련 연구 탐색 및 Voice 서비스 통찰 중심으로)

경청해주셔서고맙습니다!