Computer and Robot Vision II Chapter 17 The Consistent Labeling Problem Presented by: & : DC & CV Lab. CSIE NTU 17.1 Introduction N-ary consistent-labeling problem (CLP): 4-tuple DC & CV Lab. CSIE NTU 17.1 Introduction DC & CV Lab. CSIE NTU 17.1 Introduction DC & CV Lab. CSIE NTU 17.2 Examples of Consistent- Labeling Problems consistent-labeling problems arise in: computer vision artificial intelligence science engineering DC & CV Lab. CSIE NTU The N-Queens Problem N-queens problem: given chessboard and N queens queens placed on chessboard: no queen captures any other queen no two queens in same row, same column, or same diagonal of chessboard N-queens problem: modeled as consistent-labeling DC & CV Lab. CSIE NTU The N-Queens Problem unit set set of rows on chessboard label set set of columns on chessboard exactly one queen per row: labeling specifies the column where queen placed unit-constraint relation set unit-label constraint relation: DC & CV Lab. CSIE NTU The N-Queens Problem e.g. pair [(1,1), (2,4)] is in R since two queens do not capture each other. e.g. pair [(1,1), (2,2)] and [(1,1), (3,3)] are not in R consistent labeling: solve N-queens problem with constraints satisfied DC & CV Lab. CSIE NTU The Latin-Square Puzzle Latin-square puzzle: n x n matrix, n 2 objects arranged on matrix, one per square e.g. consider 4 x 4 puzzle for ease of illustration object is one of four colors C = { red, blue, green, yellow } object has one of four shapes S = { circle, square, triangle, octagon } DC & CV Lab. CSIE NTU The Latin-Square Puzzle The problem is to arrange the objects such that each row, each column, and each of the two main diagonals of the matrix contains exactly one object of each color and exactly of each shape. DC & CV Lab. CSIE NTU The Latin-Square Puzzle set of units U = { 1, 2,, 16 }: 16 squares of matrix labels L: objects to be placed on squares, e.g. red square, blue triangle, Cartesian product set L = C x S model T as quaternary constraint: DC & CV Lab. CSIE NTU The Latin-Square Puzzle The unit-label constraint relation R would then consist of quadruples of unit-label pairs of the form DC & CV Lab. CSIE NTU The Edge-Orientation Problem edge-orientation problem: arises in low-level vision local edge operator: applied to determine edge strength and direction due to image noise: edge operator output is also noisy prior knowledge: most edges highly continuous with low curvature DC & CV Lab. CSIE NTU The Edge-Orientation Problem maximum bending angle of any small edge segment: limited to some maximum e.g. DC & CV Lab. CSIE NTU The Edge-Orientation Problem U : set of pixels of the image L: set of possible edge orientations, including special value none E(x): set of possible edge orientations of pixel x Nbd(x): set of neighboring pixels to pixel x DC & CV Lab. CSIE NTU The Edge-Orientation Problem edge orientation of given pixels: constrained only by neighborhood edge pairs of neighboring pixels have compatible labels at least one label none: pairs of neighboring pixels have compatible labels DC & CV Lab. CSIE NTU The Edge-Orientation Problem unit-label constraint relation: DC & CV Lab. CSIE NTU JJJJJoke DC & CV Lab. CSIE NTU The Subgraph- Isomorphism Problem vertices V: set of vertices edges E: nonreflexive, symmetric binary relation over V graph G: pair (V, E) It is often necessary to determine whether two graphs representing two different entities are identical, except for the labels of the vertices, indicating that the two objects have the same structure. DC & CV Lab. CSIE NTU The Subgraph- Isomorphism Problem G=(V, E) isomorphic to G=(V, E): if there is one-to-one, onto mapping f from V to V, satisfying that f: graph isomorphism DC & CV Lab. CSIE NTU The Subgraph- Isomorphism Problem DC & CV Lab. CSIE NTU The Subgraph- Isomorphism Problem unit-set U: set of vertices V of G label-set L: set of vertices V of G unit-constraint relation T: edge set E of G unit-label constraint relation R: DC & CV Lab. CSIE NTU The Subgraph- Isomorphism Problem graph-isomorphism problem: dual consistent-labeling problem f: V V and its inverse must be consistent labelings DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem subgraph-isomorphism: special case of relational-homomorphism problem relational-homomorphism: defined on N-ary relations, instead of binary homomorphism: also structure-preserving mapping, not necessarily one-one DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem relational homomorphism from T to S: mapping f: A B satisfying relational homomorphism applied to N-tuple of T: result is N-tuple of S DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem finding relational homomorphism: sometimes called relational matching relational homomorphism: maps elements of A to B with same relationships relational monomorphism: relational homomorphism that is one-one monomorphism: stronger match than homomorphism DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem relational isomorphism f from N-ary relation T to N- ary relation S: one-one relational homomorphism from T to S f -1 is relational homomorphism from S to T relational isomophism: A, B have same number of elements each primitive in A maps to unique primitive in B each primitive in A mapped to by a primitive of B each tuple in T has corresponding one in S, vice versa A strongest kind of match: symmetric match DC & CV Lab. CSIE NTU The Relational- Homomorphism Problem graph isomorphism: binary-relational isomorphism relational-homomorphism fits consistent-labeling: like graph- isomorphism consistent labeling solution: relational homomorphism from A to B DC & CV Lab. CSIE NTU JJJJoke DC & CV Lab. CSIE NTU 17.3 Search Procedures for Consistent Labeling given a CLP, find all consistent labelings no consistent labeling: returns the empty set method: backtracking tree search speedup: forward checking discrete relaxation DC & CV Lab. CSIE NTU The Backtracking Tree Search backtracking tree search: begin with first unit of U select second unit of U, begins to construct children of first node process continues to level |U| of the tree path from root to any successful nodes at level |U|: consistent labeling DC & CV Lab. CSIE NTU simple digraph-matching problem inout A32 B02 C22 D22 E23 F31 inout DC & CV Lab. CSIE NTU portion of the tree search for solving the graph-matching problem DC & CV Lab. CSIE NTU Backtracking with Forward Checking backtracking tree search: has exponential time complexity forward checking: once a unit-label pair (u,l) is instantiated at a node in the tree, the constraints imposed by the relations cause instantiation of some future unit-label pairs (u,l) to become impossible DC & CV Lab. CSIE NTU if 1=A then 2 can either be D or F, DC & CV Lab. CSIE NTU Backtracking with Forward Checking FTAB: future-error table FTAB(u,l)=1: still possible to instantiate (u,l) FTAB(u,l)=0: (u,l) already been ruled out FTAB(u,l)=X: (u,l) impossible from previous level of recursion one future-error table for each level of recursion in tree search DC & CV Lab. CSIE NTU Backtracking with Discrete Relaxation forward-checking algorithm: prunes search tree of nodes ruled out discrete relaxation: iterative polynomial complexity procedure greatly reduces search for tightly-constrained problems constrains search further for not tightly constrained tree DC & CV Lab. CSIE NTU Ordering the Units better to choose the unit that has the fewest labels left as the next unit DC & CV Lab. CSIE NTU Complexity consistent-labeling problem: NP-complete problem forward checking and look-ahead: drastically reduce number of nodes searched do not change overall complexity DC & CV Lab. CSIE NTU The Inexact Consistent- Labeling Problem extracted line from images: some missing, partially missing, extra, distorted inexact consistent-labeling problem: allows some error for real-life problems DC & CV Lab. CSIE NTU The Inexact Consistent- Labeling Problem tree search: initially called with past_error=0 past_error: never allowed to exceed error threshold DC & CV Lab. CSIE NTU part of the inexact forward checking procedure for digraph matching DC & CV Lab. CSIE NTU 17.4 Continuous Relaxation discrete algorithms: calling a label either possible or impossible continuous procedure: associate probability or certainty that u assigned l DC & CV Lab. CSIE NTU 17.5 Vision Applications- Line Labeling with Discrete Relaxation Assumption: simple blocks world scenes of planar polyhedra no shadows or cracks and trihedral vertices polygonal planar surfaces DC & CV Lab. CSIE NTU Line Labeling with Discrete Relaxation DC & CV Lab. CSIE NTU Line Labeling with Discrete Relaxation DC & CV Lab. CSIE NTU Line Labeling with Discrete Relaxation +: convex interior line segments -: concave interior line segments >, ,