haptic feedback and learning

Haptic feedback Learning

&

J e a n n a N i k o l o v - R a m i r e z 2 0 . M a y 2 0 1 5 , U n i v e r s i t y o f V i e n n a

OUTLINE •  What is Haptic

Feedback?

•  Definitions & Method

•  Multimodal Learning

•  Touch and Learning •  Mr. Waterman •  Learning Styles & Models •  Tactile Feedback •  Force Feedback

•  Haptic Disciplines •  Haptic Personality

Types?

•  Haptic Learning in Humans vs. Robots

•  Current Status of Development, Applications

•  Discussion and Further Questions

•  References

2

WHAT IS HAPTIC FEEDBACK?

Haptic technology, haptics, or kinesthetic communication, is tactile feedback technology which recreates the sense of touch by applying forces, vibrations, or motions to the user. Tactile Feedback: Tactile sensing encompasses haptic sensing, but also feelings at the surface level. (textures)

Force Feedback: the simulating of physical attributes such as weight in computer gaming and virtual reality, allowing the user to interact directly with virtual objects using touch.

http://www.instantreality.org/wp-content/uploads/bimanualHapticDevice_automatica2012_hires-300x192.jpg

DEFINITIONS HAPTIC: HAPTIC, TACTILE, TOUCH, SOMESTHESIS OR KINAESTHETIC •  Somesthesis: includes not only cutaneous (skin) sensations

(touch) but also kinesthesis or proprioception (the capability to sense the movement and position of our limbs)

•  Haptic, touch, and somesthetic: haptic refers to the ability to experience the environment through active exploration, typically with our hands, as when palpating an object to gauge its shape and material properties. (active or haptic touch)

“However, the words haptic and haptics are increasingly used to refer to all somesthetic capabilities. This is particularly so within the community that performs research on haptic interfaces, ..” J.C. Craig and G.B. Rollman, “Somesthesis,” Ann. Rev. of Psychology, vol. 50, 1999, pp. 305-231.

DEFINITIONS LEARNING Learning: 1.  Knowledge gained by study, Instruction or Scholarship 2.  The act of gaining knowledge 3.  Any relatively permanent change in behaviour that

occurs as a direct result of experience

Kinaesthetic learning: A learning style, in which knowledge is generated by carrying out an activity, rather than listening to a lecture or observing a demonstration. Fieldwork would be one such method of learning, as would tracing fingers over a tactile map. In a more general sense, it refers to what people learn on an everyday basis as they bodily interact with a place through, for example, walking.

WHEN AND WHAT HAVE YOU LEARNED THROUGH TOUCH?

MY RESEARCH QUESTION? •  How does touch and specifically haptic feedback

technology shape learning? •  Can haptic technology accelerate skill learning? •  What correlations can be drawn between haptic

technology and learning strategies and models, specifically the Multi-Modal Paired Associated Learning Test (MMPALT)?

•  Method: Literature Review

HAPTIC INTERFACE, 1829

http://ejc.net/uploads/magazine/DSC_4288.jpg

MULTIMODAL LEARNING: INTERPLAY OF COGNITION AND AFFECT

2012 Dance your PhD Winner - A super-alloy is born

TOUCH AND LEARNING

Dual-coding theory suggests that there are two main codes or types of information, one nonverbal (imaginal) and the other verbal (Paivio, 1986)

WHAT WOULD A WORLD WITHOUT TOUCH BE LIKE? MR. WATERMAN

BBC Horizon 1998 The Man Who Lost His Body https://www.youtube.com/watch?v=efX6aVmakmw

http://www.thearticulatehand.com/ian.html

THE GAP Investigation of the haptic sense

•  What: Development, perceptual power, complexity, pure basic research, attempt to identify and detail the underlying principles and processes of haptics.

•  How: , sophisticated statistical analyses, and provide sound evidence-based results. •  Who: Developmental and cognitive psychologist •  Environment: “Uncluttered" laboratories with relatively small samples of eager

participants Application or intervention studies that have investigated the efficacy of haptically augmented instruction

•  Who: mainly educators and educational technologists •  What: focus on an in-context technological innovation with attention to issues of

practice and users' experiences with the emerging technology •  How: qualitative-and, to a lesser extent, quantitative-data. •  Environment: unable to control for all of the confounding variables present in today's

classrooms. “As a whole, they have resulted in little empirical evidence for the existence of a cognitive impact of haptic technology.”

Minogue, J., & Jones, M. G. (2006). Haptics in education: exploring an untapped sensory modality. Review of Educational Research, 76(3), 317-348.

LEARNING STYLES AND MODELS

!

Print, aural, interactive, visual, haptic, kinesthetic, and olfactory

HAPTIC FEEDBACK IN HUMANS: MOBILE MUSIC TOUCH GLOVE

http://nextbigfuture.com/2014/06/haptic-feedback-and-augmented-reality.html

HAPTIC FEEDBACK IN ROBOTS: ROBOT HANDSHAKE

http://3.bp.blogspot.com/-GkCtiVuoT9Y/TyldVVNoREI/AAAAAAAAEsw/uosuBm4sYes/s640/robot-handshake.jpg

DISCIPLINES WHERE HAPTIC FEEDBACK IS USED FOR EDUCATIONAL PURPOSE •  Medicine •  Robotics •  Gaming, Educational SW

•  Scientific Visualization

•  Arts and Design

SURGICAL SIMULATION

http://www.chai3d.org/project7.html

AEROSPACE


HAPTICALLY ANNOTATED MOVIES

Gaw, D. Morris, D. Salisbury, K. Haptically Annotated Movies: Reaching Out and Touching the Silver Screen, IEEE Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2006


PEUGEOT HUMAN FACTORS DESIGN

PSA Peugeot Citroën, Team I3D Haptic Interaction for the Automotive Industry, INRIA

SCIENCE VISUALIZATION

http://www.geomagic.com/en/community/videos/university-of-south-floridas-aist-invigorates-education-with-sca/

TOUCHABLE HOLOGRAM

Shinoda Lab, http://phys.org/news168797748.html

Takayuki Iwamoto, Mari Tatezono, and Hiroyuki Shinoda, "Non-Contact Method for Producing Tactile Sensation Using Airborne Ultrasound," Proc. EuroHaptics 2008, LNCS 5024, pp. 504-513, June, 2008.

HAPTIC HEARING

Emoti-Chair: http://www.pbs.org/wgbh/nova/next/body/haptic-hearing/

inFORM: Dynamic Shape Display that can render 3D content physically, so users can interact with digital information in a tangible way

EXOSKELETONS

PENDING QUESTIONS •  Are there real implications for education and lifelong

learning via haptics? •  Can haptic feedback be used to accelerate (skill) learning? •  What is the separating factor between human and robot

touch? •  Are there other relevant learning models?

•  What is intuitive, what is learned or “willed”?

•  Are there reliable sources for personality types that are “kinesthetic”?

•  How do signature moves relate to haptic sensations?

THANK YOU!

Questions?

© Jeanna Nikolov-Ramirez, Slow Art Day Vienna 2015

J e a n n a . n i k o l o v @ g m a i l . c o m