wiimote infrared detection presentation
DESCRIPTION
Apply new hardware to novel problems.TRANSCRIPT
Wiimote Infrared DetectionGreg Peaker
Max Chiswick
Minority Report Video
Project Goal and Setup• Use infrared for gesture recognition
– Works regardless of light and skin tone– Infrared detector: Wiimote– Infrared emitter: LED attached to hand
• Performs gesture recognition• Use gestures to control computer
applications
Wiimote Features
• Built in 1024x768 infrared camera– Processes images using system on
a chip and sends post-processed information to computer (or Wii)
• Connects to PC with Bluetooth• Wii Wiimote Usage:
– Wiimote moves around– Locates two IR LED emitter arrays
in sensor bar• Our Wiimote Usage:
– Wiimote stays still– IR emitter on finger moves around
Approach and Design
• Use off-the-shelf and inexpensive components– Wii remote– Infrared emitter: LED, resistor, battery power– Free, open source software
• Wiimote detects and tracks IR spots• Wiimote connects to computer through Bluetooth
– Sends data in ASCII at 100 Hz• Library parses ASCII to obtain easily readable parameters
– Not used: buttons pressed, acceleration, intensity– Used: (x,y) coordinates
• Code in Visual C#– Interpret user motions/Gesture Recognition– Control Windows Media Player and PowerPoint
How It Works• Software tracks (x,y) coordinates of IR led• Determine start time and coordinates• Create array of all coordinates in gesture
array from start to end– Sampled >100 times per second
• Determine end time and coordinates• Directional movements of finger control
windows applications
How It Works (2): Interpreting Start and Stop of Motion• Start gesture by
– Making infrared signal visible– Holding signal in place for about 2 seconds
• Program examines last 3 samples every second– If samples are within threshold, begin gesture– If samples are not (e.g. random movements), do not
begin gesture• Once gesture begins, program creates array that keeps
track of all (x,y) coordinates• End gesture by holding signal in place for about 2
seconds• Program continues to examine last 3 samples every
second– When it finds they are outside of threshold, continues
gesture– When it finds they are within threshold, ends gesture
How It Works (3): Determining Direction Used• Compare final (x,y) to initial (x,y)
• Obtain variables x_diff and y_diff• If x_diff > threshold, define as East or West• If y_diff > threshold, define as North or South• If neither > threshold, no motion• If one > threshold, basic North, South, East, or West
direction• If both > threshold, see if one direction is significantly
greater than other– Interpret this as only the larger one– Example: 100 units east and 12 units north probably
intended to be only east• If both > threshold and neither significantly greater than
other– Create line between start and end points of gesture array– Count every (x,y) element between start and end and
count them as above or below line– Use this info to determine, for example, whether gesture
is South-East or East-South
User Movements• 12 total possibilities
– Basic 4 directions– Direction combinations (order matters)
• North-East and East-North are different
Applications• Use directional movements for:
– PowerPoint• Next Page (North or East), Previous Page
(South or West)– Media Players
GestureSouth DownNorth UpWest LeftEast Right
GestureSouth – West PreviousNorth - West Volume DownNorth - East Volume UpSouth - East Next
GestureWest – South MuteWest - North MuteEast - North PauseEast - South Stop
Next Steps• Hardware
– Better define and increase range– Smaller IR transmitter/receiver assembly– Use reflection– Two finger or more finger tracking allowing
increased movements and novel applications
• Software– Support applications such as web browsers,
Google Earth, etc.– Use as mouse cursor with clicking ability– Better gesture recognition using machine
learning
IR Computer Vision TodayMicrosoft Surface
iBar
The computer's "vision" is created by a near-infrared, 850-nanometer-wavelength LED light source aimed at the surface. When an object touches the tabletop, the light is reflected to multiple infrared cameras with a net resolution of 1280 x 960, allowing it to sense, and react to items touching the tabletop.
iBar is an interactive, 10 meter long bar-counter using multi-touch technology. iBar has an intelligent tracking system to detect all objects touching the surface. Integrated video-projectors can project any content dynamically with the movements on the counter. The software is flexible and can be adjusted and enhanced. Objects can be illuminated at their position or virtual objects can be “touched” with your fingers. Every glass, cellphone, car key, cup, business card or even fingers are recognized when they touch the bar-counter.
Demo
GestureSouth DownNorth UpWest LeftEast Right
GestureSouth – West PreviousNorth - West Volume DownNorth - East Volume UpSouth - East Next
GestureWest – South MuteWest - North MuteEast - North PauseEast - South Stop