Hand Gesture Recognition Hand Gesture Recognition Using a Web CamUsing a Web Cam

Chris GledhillChris Gledhill

Project DemoProject Demo

BackgroundBackgroundProject goalsProject goalsImage CaptureImage CaptureImage ProcessingImage ProcessingImage InterpretationImage InterpretationGesture RecognitionGesture RecognitionGesture ModellingGesture ModellingProject ConclusionsProject Conclusions

BackgroundBackground

Current user interfacesCurrent user interfaces– Keyboards, Rodents, Joysticks, TabletsKeyboards, Rodents, Joysticks, Tablets– 2D2D– RangeRange– Require physical contactRequire physical contactHand gesturesHand gestures– 3D3D– IntuitiveIntuitive– Virtual RealityVirtual Reality

Background continued…Background continued…

Gesture recognitionGesture recognition– Mechanical glove based approachMechanical glove based approach– Optical approachOptical approach

Capture the imageCapture the imageProcessing the imageProcessing the imageInterpreting the imageInterpreting the imageGesture recognitionGesture recognition

Project GoalsProject Goals

Research hand gesture recognition Research hand gesture recognition techniquestechniques– AlgorithmsAlgorithms– Present/future usesPresent/future uses– IssuesIssues

Develop a prototype gesture recognition Develop a prototype gesture recognition systemsystem– Optical spotty glove locationOptical spotty glove location

Image CaptureImage Capture

Video Camera, Digital Camera, Web CamVideo Camera, Digital Camera, Web Cam– ResolutionResolution– Frame rateFrame rate– ColourColour

Logitech QuickCam Pro 3000Logitech QuickCam Pro 3000– 640x480640x480– 30 fps30 fps– 24bit colour24bit colour

Image Capture continued…Image Capture continued…

Logitech SDK/Video for Windows/Direct Logitech SDK/Video for Windows/Direct Hardware/DirectXHardware/DirectX

DirectX SDKDirectX SDK– CompatibilityCompatibility– Windows componentWindows component– Useful librariesUseful libraries

Hand gesture recogniser

WindowsUSB

Controller

DirectShow

USB Port

Image ProcessingImage Processing

Tidy up the imageTidy up the image– Resize (640x480 => 70x70)Resize (640x480 => 70x70)– Colour Reduction (16.7 million => 4)Colour Reduction (16.7 million => 4)– Noise ReductionNoise Reduction

Locate spotsLocate spots– Centre of massCentre of mass

Image InterpretationImage Interpretation

Rule based 2DRule based 2D

Interpret Data

Interpret Data

Gesture RecognitionGesture Recognition

3 gestures recognised3 gestures recognisedPalm down resting gesturePalm down resting gesture

2 click gestures2 click gestures

Gesture modellingGesture modelling

3D model based analysis3D model based analysis

Gesture modellingGesture modelling

3D Hand model developed in 3D Studio Max3D Hand model developed in 3D Studio Max

Gesture modelling continued…Gesture modelling continued…

Converted to .X file formatConverted to .X file formatImported into application as Direct3D mesh Imported into application as Direct3D mesh objectobject

.3DS3D Studio Max 5 conv3ds.exe 3D ViewerApplication

textures

.X

Gesture modelling continued…Gesture modelling continued…

Rendered using Rendered using Direct3DDirect3D

Project ConclusionProject Conclusion

Researched hand gesture recognition Researched hand gesture recognition techniquestechniques– AlgorithmsAlgorithms– Present/future usesPresent/future uses– IssuesIssuesDeveloped a prototype gesture recognition Developed a prototype gesture recognition systemsystem– Optical spotty glove locaterOptical spotty glove locater– 3D Hand model3D Hand model

Project Demo SummaryProject Demo SummaryBackgroundBackgroundProject goalsProject goals Image captureImage capture Image processingImage processing Image interpretingImage interpretingGesture recognitionGesture recognitionGesture modellingGesture modellingProject conclusionsProject conclusions

Questions?Questions?