Speaker: Shau-Shiang Hung(洪紹祥 )advisor :Shu-Chen Cheng(鄭淑真 )

Date : 2010/4/8

Computer Graphics and Applications, IEEE Publication Date :  March-April 2005 Volume : 25 , Issue:2 On page(s): 13 - 17

Author : Alexandre Gillet, Michel Sanner, Daniel Stoffler, and Arthur Olson

OutlineIntroductionDesign of physical modelsAugmented reality interfaceImplementationExamplesEvaluationConclusions

Introduction 1/2With the prevalence of structural and

genomic data, molecular biology has become a human-guided, computer-assisted endeavorExploring scientific dateTest scientists hypotheses

As databases grow, as structure and process ,and as software methods become more diverseAccess and manipulation of digital information

is increasingly a critical issue

Introduction 2/2 Develop an augmented reality (AR) system

Visual 3D representations Molecular structure 、 properties

With tangible interaction environment User can

intuitively manipulate molecular models and interactions

Easily change the representation shown Access information about molecular properties

Design of physical models 1/2Use Python Molecular Viewer (PMV)

Create virtual objectsDesign tangible modelsSimplifying the integration of the models with the

virtual environment.PMV is software framework

Molecular surfaces Extruded volumes Backbone ribbons Atomic ball-and-stick

Show molecular shape in interaction Atomic details

Design of physical models 2/2PMV includes

Generic 3D visualization component A high level interface to the OpenGL library and its

geometry viewing applicationAdd

all molecular modeling and visualization functionality

Output formats (.stl and .vrml) for the solid printers serve as input

Augmented reality interface 1/2Use computer-based spatial tracking and

rendering method to enhance the physical models’ semantic content and show dynamic properties.

Combines real-world user and physical model presence with computation models and data

Approach is based on the widely used ARToolKitCalculate the real camera position and orientation

relative to physical marker in real time, allowing overlay of virtual objects relative to the physical markers.

Augmented reality interface 2/2Add PyARTK

Integrate ARToolKit with PMV to manage markers Advantage:

Streamline the design and display of models within the same environment.

Assigns the geometries, animations, and masks to specific AR markers or sets of markers

The interface comes with controls for :Computer graphic objectsCamera operationsClippingLighting controls

Marker

Implementation 1/2Integrating the physical models with the

virtual augmentation requires superpositioning the two worlds into the same perceptual space

There are a number of possible ways using a head-mounted display

the user sees video of the real-world scene and the superimposed computer graphic information.

Projecting the computer information onto the physical model.

But expense and imperfect performance

Implementation 2/2A simpler two-view Solution

This configuration has proven to be an effective, inexpensive, and portable solution.

Physical model

Computer –augmented scene

Camera

Examples 1/2 HIV protease

Examples 2/2 Superoxide dismutase (SOD)

Evaluation 1/1The first pilot test involved high school students

from the Biotech Academy program in SeattleThe second pilot study with students from a

college-level biochemistry class at the University of Washington

Produced a number of models for colleagues in our institute for applications ranging from drug design to assembly of large biomolecular complexes Received uniformly positive comments for

enhanced comprehension and communication of structural characteristics.

Conclusions 1/1 In our experience, we found that tangible

molecular models can provide several advantages over computer visualizations alone. They produce a multisensory engagement that includes visual, tactile, and proprioceptive perceptual pathways for learning and memory.

Using PMV along with our system has proven to be a fast and efficient approach to develop and test new ideas.