Graphics
cgvr.korea.ac.kr Graphics Lab @ Korea University
Lecture1. Introduction to Virtual Reality
CSE 425 Spring 2002 김 창 헌
Department of Computer Science
Korea University http://cgvr.korea.ac.kr/courses/cse425/
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Contents
What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Computational Reality
Glove CG Machine HMD : This is not VR! Studies on reality - computational reality VR = computational reality
1) Seek for the computational model of reality
2) Apply the model to the VR system
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Analogous to AI lots of hypes AI as a science / engineering
science - computational model for intelligence
engineering - making “intelligent-looking” machines
The essence of science is the search for models. Greek scholars
stars, sky (heavenly bodies) astrology nature (earthly bodies) physics human mind philosophy
Science vs. Engineering
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VR Concept
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
What is VR? Definition -1
Def 1 - Computer generated environment that is ...
Immersive, (like IMAX, dome projection) Interactive, (like computer game) Multi-sensory, Viewer-centered, 3D, and the combination of technologies required to build such
environments.
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
What is VR? Definition -2
Def 2 - Physical participation in the interactive simulation.
Virtual environment A computational model or realization of it by VR techniques.
Cyberspace A computer environment spanning - multiple computers,
- multiple users, - multiple sets of data.
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VR as a computer Technology
Ultimate computer interface : human body and its sensor To turn the whole body into an input device
Technological trends Powerful Supercomputing, parallel computing Smart AI Physical VR
Primary concern - S/WOther important issues - H/W, human factors,
social issues, infrastructure VR computer - interactive, multimodal, immersive
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Interacting with computers
ComputerWorld
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
New Paradigm for HCI
Conventional Computer
VR-based Computer
ComputerWorld
ComputerWorld
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Education/Training
DesignEngineering
Medicine
ScientificVisualization
Entertainment
Communication
ProductsEngineering
MuseumArts VR application
VR Application
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VR technologies
Character CRT
Graphic CRT
Virtual console
Virtual Reality
Tele-conference
TV phone
Telephone
Keyboard
Mouse tablet
3D mouse
Computersimulation/visualization
Tele-Existence
Tele-Operation
Computergraphics
Real timeCG
Video arts
3D CAD
ComputerAided Design
Virtual products
design
All technologies meet together at VR !!
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VR Terminologies
Virtual Reality
Tele-Existence inReal World
Tele-Existence inVirtual World
• Physical World
• Quasi Physical World
• Non Physical World
• Standard Tele-Existence
• Augmented Tele-Existence
• Size• Sensation• Time
VR in Real World vs. Virtual World
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Contents
What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Conceptual Model of VR
• 3D image Large-scale display, Head Mounted Display• Sound field by DSP • Force beedback mechanism• Tactile display
SimulationSystem
Computer
Display system
Sensing system• Non-contact type
magnetic field
supersonic wave
infrared light
• Contact type optical fiber strain gauge potentio-meter
Human
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Contents
What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VR Related areas
1. Training simulation
2. Tele-operation
3. Computer graphics
4. Artificial intelligence
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
1. Training simulation
Example – Flight Simulation
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Training simulation
Differences
Reconfigurable by changing software May include highly unnatural environment Highly interactive and adaptive Use of a wide variety of human sensing modalities an
d sensori-motor systems Highly immersive Near-field is synthetic
Far-field is synthetic
1. Training simulation (cont)
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VR Related areas
1. Training simulation
2. Tele-operation
3. Computer graphics
4. Artificial intelligence
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
2. Tele-operation
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
2. Tele-operation (con’t)
for at least 30 years.
Tele-operator- directly (manually) controlled tele-operator- tele-robot
Tele-operation vs. Virtual reality
Tele-presence vs. Virtual presence
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
3. Computer graphics
Modeling
Motion control (animation)
Rendering
User interface
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
4. Artificial intelligence
Studies on perception and cognition
Tested for AI research
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Contents
What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
History
1’st stage: some visionaries- Morton Heilig : ExperienceTheatre(1962)- Ivan Surtherland : Sketchpad(1963), HMD(1966)- Myron Krueger : Artificial Reality(1972)- William Gibson : “Cyberspace” in Neuromancer(1984)
2’nd stage: technology development for specific purposes- training simulator: Earlier works- space exploration : NASA for astronaut simulation- tele-operation
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
History (cont)
3’rd stage: VR as the general-purpose technology- Jaron Lanier : VPL(1987) – Dataglove, EyePhone,VR system- VR industry : Division Ltd. Sense8, WorldDesign (production
house, W-Industry (game)- VR academia : MIT, UNC, UW,Tokyo U.
Next stage: Toward a scientific discipline- computational reality- a new computing paradigm- a new media- a new art form- representation, creation and operation of virtual worlds
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Contents
What is VR? Conceptual Model of VR VR Related Areas History State of the Art and R&D Issues Some Application Examples
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
State of the Art & Issues
Reference-Virtual Reality: Scientific and Technological Challenges”, pp. 35-
66, National Research Council, National Academic Press, 1995.
Areas of the study1. application domains2. psychological issues3. VR technologies4. evaluation of VR systems
State of the Art and R&D Issues
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
1. Application domains
design, manufacturing & marketing
medicine, health care hazardous operations training entertainment, military experimental psychology education information visualization Tele-communication, Tele-travel
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
2. Psychological issues
human performance characteristics
alteration of sensori-motor loops developing the cognitive model cognitive side-effect
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Gap between the current technology & the required technology(exception - entertainment, tele-operation)
(1) Human-Computer Interface (2) Computer Generation of VE(3) Tele-robotics(4) Network
3. VR technologies
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
State of the Art and R&D Issues (con’t)
1. Application Domains
2. Psycological issues3. VR technologies
(1) HCI (Human-Computer Interface)
(2) Computer Generation of VE
(3) Tele-robotics
(4) Network
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1) Human-Computer Interface (con’t)
visual channel auditory channel haptic channel motion interface position tracking video camera microphone others
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Visual Channel
visual display
- HMD (Head-Mounted Display)
- OHD (Off-Head Display)
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Visual Channel (con’t)
research issues
- ergonomics
- improvement of resolution and FOV (field of view)
- wireless
- integration of visual, auditory, position tracking
- sun glass-like
- see-through option- exploiting FOV and peripheral
vision
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Auditory channel
Current hardware is adequate.
Research issues perceptual issues
similar to the visual channel use for sensory substitution (for visual, haptic)
auditory scene analysis hear-through display
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Auditory channel (con’t)
Traditional Stereo Sound
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Auditory channel (con’t)
3D virtual Sound
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Haptic channel
Force, Pressure, Tactile Feedback
Unique characteristics Haptic interface requires mani
pulation and sensing
Mechanism body-based glove, ex
oskeleton ground-based joystick
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Haptic channel (con’t)
Research issues- haptic science = study on the human h
aptics (bio-mechanical, psychophysical, cognitive)
- tool-hand system (which takes its metaphor from real tools)
- creating the haptic illusion- the interaction effects of haptic and visi
on- texture, temperature devices
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Haptic channel (con’t)
Haptic channel in Tele-operation
Operator module Work side module
Haptic devices
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Haptic channel (con’t)
Interaction of haptic and vision
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Motion interface
motion whole-body motion
passive - e.g., motion platform
active - e.g., locomotion
part-body motion passive active
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Motion interface (con’t)
motion cues
vestibular system - inertial
motor visual auditory proprioceptive / kinesthetic
- muscle tactile
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Motion interface (con’t)
motion interface
inertial system moves the body (e.g., treadmill,
motion platform) non-inertial system
simulates motion
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Position tracking and mapping
Tracking = finding a pointMapping = finding a 3D surface
(e.g., environmental mapping)
Tracking mechanisms Mechanical linkage Magnetic Optical Acoustic Inertial
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Position tracking and mapping (con’t)
eye tracking
research issues tracking mapping
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(1)HCI – Other types of interfaces
olfactory (smell)
gustatory (taste)
heat, wind, humidity
speech
direct physiological sensing and control
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
State of the Art and R&D Issues
1. Application Domains
2. Psycological issues3. VR technologies
(1) human-machine interface
(2) computer generation of VE
(3) tele-robotics
(4) Network
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Conceptual Model of VR
• 3D image Large-scale display, Head Mounted Display• Sound field by DSP • Force beedback mechanism• Tactile display
SimulationSystem
Computer
Display system
Sensing system• Non-contact type
magnetic field
supersonic wave
infrared light
• Contact type optical fiber strain gauge potentio-meter
Human
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Generation of VE(cont’)
The core issue General-purpose VR
system? Trade-off between realism
and interactivity Requirements
frame rate response time scene quality
(2) Computer Generation of VE
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Generation of VE(cont’)
hardware interaction and navigation VE management
-simulation-rendering
modeling autonomous agent hypermedia interaction OS
(2) Computer Generation of VE
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VE management - simulation
Task : simulating everyday world Traditional simulation methods do not work.
requires pre-processing Need : “meta-modeling”
VE management – simulation
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
VE management – Rendering
Issue : load balancing1. partitioning VE2. LOD
Much work has been done on static scene. Research issues
1. dynamic scene2. parallel rendering
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(2) Computer Generation of VE
OS real-time, multi-modal requirements very high-resolution time slicing atomic, transparent distribution of tasks large number of light-weighted processors,
communicating by means of shared memory support for time-critical computing:
negotiated, graceful degradation guaranteed frame rate, lag time
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
State of the Art and R&D Issues
1. Application Domains2. Psycological issues3. VR technologies
(1) human-machine interface(2) computer generation of VE(3) tele-robotics(4) Network
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(3) Tele-robotics
Tele-robotics and VR Hardware Time-delay problem Distributed Tele-robots
Operator- stays in a safe environment - guides the robot intelligently
Robot- works in the hazardous environment
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
(4) Network
The future is here!
Applications distance learning group entertainment distributed training distributed design
current future What is needed
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
International Efforts
U.S.A. defense, space, visualization, medicineU.K. education, training, entertainmentGermanyJapan VR as a logical extension of
robotics, automation, HDTV.
Research Organizations
cgvr.korea.ac.kr
CGVR
Graphics Lab @ Korea University
Academia
HIT Lab, University of WashingtonUniversity of North CarolinaMedia Lab, MITGeorgia Institute of TechnologyNaval Postgraduate SchoolUniversity of PennsylvaniaUniversity of California at BerkeleyUniversity of Illinois - ChicagoColumbia UniversityUniversity of Toronto