ssip 2004 graz© inst. for computer graphics and vision, 2004 vision guided robotics and...

SSIP 2004 Graz © Inst. For Computer Graphics and Vision, 2004

Vision Guided Robotics

and Applications in Industry and Medicine

Matthias Rüther


Contents

Robotics in General

Industrial Robotics

Medical Robotics

What can Computer Vision do for Robotics?

Vision Sensors

Issues / Problems

Visual Servoing

Application Examples

Summary


Robotics

What is a robot?"A reprogrammable, multifunctional manipulator designed to move

material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks"

Robot Institute of America, 1979

Industrial– Mostly automatic manipulation of rigid parts with well-known shape in a

specially prepared environment.

Medical– Mostly semi-automatic manipulation of deformable objects in a

naturally created, space limited environment.

Field Robotics– Autonomous control and navigation of a mobile vehicle in an arbitrary

environment.


Robot vs Human

Robot Advantages:

– Strength

– Accuracy

– Speed

– Does not tire

– Does repetitive tasks

– Can Measure

Human advantages:

– Intelligence

– Flexibility

– Adaptability

– Skill

– Can Learn

– Can Estimate


Requirements:

– Accuracy– Tool Quality– Robustness– Strength– Speed – Price Production Cost– Maintenance

Industrial Robot

Production Quality


Medical (Surgical) Robot

Requirements

– Safety– Accuracy– Reliability– Tool Quality– Price– Maintenance– Man-Machine Interface


What can Computer Vision do for Robotics?

Accurate Robot-Object Positioning

Keeping Relative Position under Movement

Visualization / Teaching / Telerobotics

Performing measurements

Object Recognition

Registration

Visual Servoing


Vision Sensors

Single Perspective Camera

Multiple Perspective Cameras (e.g. Stereo Camera Pair)

Laser Scanner

Omnidirectional Camera

Structured Light Sensor


Vision Sensors

Single Perspective Camera

XPx x43


Vision Sensors



Vision Sensors


0Fxx'T Fxl'


Vision Sensors

Laser Scanner


Vision Sensors

Omnidirectional Camera


Vision Sensors

Structured Light Sensor

Figures from PRIP, TU Vienna


Issues/Problems of Vision Guided Robotics

Measurement Frequency

Measurement Uncertainty

Occlusion, Camera Positioning

Sensor dimensions


Visual Servoing

Vision System operates in a closed control loop.

Better Accuracy than „Look and Move“ systems

Figures from S.Hutchinson: A Tutorial on Visual Servo Control


Visual Servoing

Example: Maintaining relative Object Position

Figures from P. Wunsch and G. Hirzinger. Real-Time Visual Tracking of 3-D Objects with Dynamic Handling of Occlusion


Visual Servoing

Camera Configurations:

End-Effector Mounted Fixed



Visual Servoing

Servoing Architectures



Visual Servoing

Position-based and Image Based control

– Position based: • Alignment in target coordinate system• The 3D structure of the target is rconstructed• The end-effector is tracked• Sensitive to calibration errors• Sensitive to reconstruction errors

– Image based:• Alignment in image coordinates• No explicit reconstruction necessary• Insensitive to calibration errors• Only special problems solvable• Depends on initial pose• Depends on selected features

target

End-effector

Image of target

Image of end effector


Visual Servoing

EOL and ECL control

– EOL: endpoint open-loop; only the target is observed by the camera

– ECL: endpoint closed-loop; target as well as end-effector are observed by the camera

EOL ECL


Visual Servoing

Position Based Algorithm:1. Estimation of relative pose

2. Computation of error between current pose and target pose

3. Movement of robot

Example: point alignment

p1

p2


Visual Servoing

Position based point alignment

Goal: bring e to 0 by moving p1

e = |p2m – p1m|

u = k*(p2m – p1m)

pxm is subject to the following measurement errors: sensor position, sensor calibration, sensor measurement error

pxm is independent of the following errors: end effector position, target position

p1m p2m

d


Visual Servoing Image based point alignment

Goal: bring e to 0 by moving p1

e = |u1m – v1m| + |u2m – v2m|

uxm, vxm is subject only to sensor measurement error

uxm, vxm is independent of the following measurement errors: sensor position, end effector position, sensor calibration, target position

p1 p2

c1 c2

u1

u2

v1 v2

d1d2


Visual Servoing

Example Laparoscopy

Figures from A.Krupa: Autonomous 3-D Positioning of Surgical Instruments in Robotized Laparoscopic Surgery Using Visual Servoing


Registration

Registration of CAD models to scene features:

Figures from P.Wunsch: Registration of CAD-Models to Images by Iterative Inverse Perspective Matching


Tracking

Instrument tracking in laparoscopy

Figures from Wei: A Real-time Visual Servoing System for Laparoscopic Surgery


Summary

Computer Vision provides accurate and versatile measurements for robotic manipulators

With current general purpose hardware, depth and pose measurements can be performed in real time

In industrial robotics, vision systems are deployed in a fully automated way.

In medicine, computer vision can make more intelligent „surgical assistants“ possible.

ssip 2004 graz© inst. for computer graphics and vision, 2004 vision guided robotics and...

Documents