International Conference on Computer Vision and Graphics, ICCVG ‘2002

Algorithm for Fusion of 3D SceneAlgorithm for Fusion of 3D Sceneby Subgraph Isomorphism by Subgraph Isomorphism with Procrustes Analysiswith Procrustes Analysis

Krzysztof Skabek, Przemysław KowalskiKrzysztof Skabek, Przemysław Kowalski

Instytut Informatyki Teoretycznej i Stosowanej PAN,Instytut Informatyki Teoretycznej i Stosowanej PAN,ul. Bałtycka 5, 44-100 Gliwiceul. Bałtycka 5, 44-100 Gliwicee-mail: e-mail: krzysiek@iitis.gliwice.plkrzysiek@iitis.gliwice.pl

ContentsContents

1. Active vision

2. Stages of 3D fusion

3. Graph representation and algorithms

5. Matching structure graphs

4. Algorithm for 3D Fusion

5. Experiments

Active Vision PlatformActive Vision Platform

Platform movementsThe observed 3D scene

Purpose:Purpose:

Obtaining a complete 3D representation of the scene and relations between components of the scene

basing on a set of 3D frames from multiple viewpoints.

Assumptions:Assumptions:No a-priori information about objects of the scene.

Unknown location of the viewpoint.

We focus on polyhedral objects.

Architecture of Active VisionArchitecture of Active Vision

Act Act

SensingSensing

PlanningPlanning

camera camera

Mision planning

Navigator

Pilot

Controller

Engines

Image preprocessing

Model integration

Location: x,y,z,

3D modelMethods of 3D fusion

3D Fusion of Multipoint Views3D Fusion of Multipoint Views

3D Fusion3D Fusion – integration process of objects in 3D scene on the basis of visual information

from several viewpoints.

Stages of 3D FusionStages of 3D Fusion

Current view

Viewpoint loc.

3D model

Corection

Vision device

Prediction

Navigation to a new viewpoint

Knowledge of the scene

Comparing the view and the model

Exact viewpoint loc.

Updating the model

Hypothesis aboutthe scene objects

Checking the completeness

Preprocessing of Visual InformationPreprocessing of Visual Information

Improvement of image quality, noise reduction Image segmentation, extraction of lines,

segments,vertices: Susan, Hough Methods Stereo matching, depth map: active contours,

hardware support (ranging lasers, depth sensors)

Algorithms prepared for Khoros platform

Viewpoint parametersViewpoint parameters T – vector of translation (3×1),

R – rotation matrix (3×3, orthogonal)

s – scale (scalar)

Relation between coordinates of point P:

Pw – global coordinates,

Pk – coordinate system of the camera

Pk = R(Pw –T)s

Graph representation of 3D sceneGraph representation of 3D scene

0 . 03 3

0 . 0

0 . 0 0 . 0

4 4

4

ContourContourgraphgraph

Face graphFace graph

1

2

3

4

1

2

3

4

5

6

7

Graph IsomorphismGraph IsomorphismGraph IsomorphismGraph Isomorphism

23

32

23

32

Subgraph IsomorphismSubgraph Isomorphism

23

32

23

3

3

3

2

Weak Subgraph IsomorphismWeak Subgraph Isomorphism

23

32

23

3

3

3

2

4

4

Detection of Graph IsomorphismsDetection of Graph Isomorphisms Permutation method Clique detection method Ullman method A* method (error correction) Decision tree method

Algorthm with analysis of 3D structure deformation(decision tree, consistency checking, branch pruning, geometric similarity)

Similarity of Shape – Procrustes analysisSimilarity of Shape – Procrustes analysis

--- - object A

--- - object B

--- - A to B matching

D2(A, B) = || B – S·R·A – T ||2

Translation (T)

Scale (S)

Rotation (R)

Implementation of 3D FusionImplementation of 3D Fusion(matching contour graphs)(matching contour graphs)

Stage I:Stage I:

Generation of groups of vertices Generation of groups of vertices (quadruplets) fulfilling conditions:(quadruplets) fulfilling conditions: Procrustes distance < Procrustes distance < Preserving edge topologyPreserving edge topology

Stage II:Stage II: (for eqch group of vertices from stage I)(for eqch group of vertices from stage I)

Calculation of local transformation Calculation of local transformation ((TTLL R RLL S SLL))

Matching the remaining vertices:Matching the remaining vertices: Local Procrustes distance < Local Procrustes distance < Preserving edge topologyPreserving edge topology

Calculation of exact transformation Calculation of exact transformation ((T R ST R S))

0

2

3

4

1

0

2

3

4

1

5

6

n

ISOMORPHISMS

V

Implementation of 3D FusionImplementation of 3D Fusion(model updating)(model updating)

A

B

C

D

E

5

1

4

3

6

2F

GMi-1 GIi

A

B

C

D

E

F

5

GMi

Implementation of 3D FusionImplementation of 3D Fusion(hypothesis of the scene objects)(hypothesis of the scene objects)

Introducing Introducing unconfirmed unconfirmed elements.elements.

Hypothesis of scene objects:Hypothesis of scene objects: Connecting edgesConnecting edges Closing facesClosing faces Connecting partial facesConnecting partial faces Ground plane detectionGround plane detection Completing vertical facesCompleting vertical faces

Conditions of experimentsConditions of experimentsTotal transformation error consists of:Total transformation error consists of:

rotation, translation, scalerotation, translation, scale

Tolerance of rotation (Tolerance of rotation (RR – matrix of rotation error): – matrix of rotation error):

R = RR = RRR

Estimation of rotation error:Estimation of rotation error:

= ½ [1 – cos(= ½ [1 – cos() ]) ]½½

Assumed value of rotation error:Assumed value of rotation error:

= 0.1 for = 0.1 for 16 16°°

ExperimentsExperiments

Thank you for your attentionThank you for your attention

Graph representation of 3D scene IIGraph representation of 3D scene IIContour graphContour graph: :

Face graphFace graph::

CCCCC EVG ,,,

FFFFF EVG ,,,

CV

CCC VVE

CC VVCC LLV ,:

CECC LV :

- a set of vertices in the scenea set of vertices in the scene- a set of edges between verticesa set of edges between vertices- coordinates coordinates (x,y,z)(x,y,z) of vertices of vertices

- a set of faces in the scenea set of faces in the scene- a set of connections between facesa set of connections between faces- parameters of facesparameters of faces- parameters of connections between parameters of connections between facesfaces

FV

FFF VVE

FF VVFF LLV ,:

FF EEFF LLV ,:

Implementation of 3D FusionImplementation of 3D FusionInput data:Input data:

GIGIii – contour graphs for views (– contour graphs for views (i=1..ni=1..n))

TTii R Rii S Sii – estimated transformation – estimated transformation (from navigation unit)(from navigation unit)

ii – transformation tolerance – transformation tolerance (for navigation unit)(for navigation unit)

ii – observation tolerance – observation tolerance (for optical unit)(for optical unit)

Output data:Output data:

GMGMn n – Contour graphs of model– Contour graphs of model

TTii R Rii S Sii – computed transformation – computed transformation

First step of fusion:First step of fusion:

GMGM11 GI GI11

Experiment IExperiment I