1. APPLICATIONS OF VISION SCIENCE PART I: CONTOURS AND TEXTURESJames ElderCentre for Vision Research, York University

2. 2Applications of Contour and Texture Processing Applications of Vision Science Part I: Contours and Textures Contours Background:Contour extraction Application 1. Interactive contour editing (ICE) Application 2. Demarcating water features in satellite imagery Texture: Enhanced / Synthetic Vision Systems (ESVS) Application1. Optimal Texture Maps for ESVS Application 2. Understanding Texture FusionJ. Elder 3. Applications of Contour Processing 3Applications of Vision Science Part I: Contours and Textures Background: Contour extraction Application 1. Interactive contour editing (ICE) Application 2. Demarcating water features in satellite imageryJ. Elder 4. Applications of Contour Processing 4Applications of Vision Science Part I: Contours and Textures Background: Contour extraction Application 1. Interactive contour editing (ICE) Application 2. Demarcating water features in satellite imageryJ. Elder 5. Edge Detectors and Scale Space 5Applications of Vision Science Part I: Contours and Textures Edge detection algorithms based on Gaussian scale space (Koenderink 84, Young 87, Parent & Zucker 89, Elder & Zucker 98, Lindeberg et al 98, etc.): f ( x, y) x 2 y 3 xe1 [( x / x )2 ( y / y )2 ] 2J. Elder 6. Information Loss 6Applications of Vision Science Part I: Contours and TexturesWhat about all of the information we are throwing away?J. Elder 7. Brightness Filling-In 7Applications of Vision Science Part I: Contours and TexturesCornsweet (1970) J. Elder 8. Reconstruction from Contours 8Applications of Vision Science Part I: Contours and TexturesElder, IJCV 1999J. Elder 9. 9Blurring of Contours in the Natural World Applications of Vision Science Part I: Contours and TexturesJ. Elder 10. 10Incorporating blur in image reconstructionApplications of Vision Science Part I: Contours and TexturesElder, IJCV 1999J. Elder 11. Reconstruction 11Applications of Vision Science Part I: Contours and TexturesJ. Elder 12. 12Applications of Vision Science Part I: Contours and TexturesJ. Elder 13. But now what? 13Applications of Vision Science Part I: Contours and TexturesJ. Elder 14. First-Order Model 14Applications of Vision Science Part I: Contours and TexturesContour Grouping: 1st-Order CuesbdProximity + Good Continuation (Wertheimer 1923)aJ. Elder 15. Markov Chain Model 15Applications of Vision Science Part I: Contours and Textures Model contours as Markov chains: assume long-range statistics completely determined by local statistics.n 1L Li i , i 11p (di i | {ti ,ti } C ) where L ij p (di i | {ti ,ti } C ) 1111J. Elder 16. Tracing Contours using ICE (Elder & Goldberg, PAMI 01) 17. 17Gestalt Cues: Natural Image StatisticsApplications of Vision Science Part I: Contours and TexturesElder & Goldberg, 2002J. Elder 18. Applications of Contour Processing 18Applications of Vision Science Part I: Contours and Textures Background: Contour extraction Application 1. Interactive contour editing (ICE) Application 2. Demarcating water features in satellite imageryJ. Elder 19. User Interface 19Applications of Vision Science Part I: Contours and TexturesElder & Goldberg, PAMI 2001 J. Elder 20. Rapid Interactive Contour Grouping with ICE 21. 21Interactive Correction of Grouping Errors Applications of Vision Science Part I: Contours and TexturesGrouping ErrorCorrected ContourJ. Elder 22. Examples 22Applications of Vision Science Part I: Contours and TexturesJ. Elder 23. Examples 23Applications of Vision Science Part I: Contours and TexturesJ. Elder 24. Examples 24Applications of Vision Science Part I: Contours and TexturesJ. Elder 25. Applications of Contour Processing 25Applications of Vision Science Part I: Contours and Textures Background: Contour extraction Application 1. Interactive contour editing (ICE) Application 2. Demarcating water features in satellite imageryJ. Elder 26. The Challenge: Enhanced/Synthetic Vision for Aviation 26Applications of Vision Science Part I: Contours and TexturesJ. Elder 27. 3D Database Consistency 27Applications of Vision Science Part I: Contours and TexturesJ. Elder 28. Using lakes to index prior models for DEM refinement 28Applications of Vision Science Part I: Contours and TexturesCDEDPrior SPOTPosterior SPOTJ. Elder 29. Fusing GIS & IKONOS data to Compute Accurate Lake Boundaries 29Applications of Vision Science Part I: Contours and TexturesElder et al, PAMI 2003 J. Elder 30. Cues 30Applications of Vision Science Part I: Contours and TexturesGrouping Cues Proximity Good Continuation Luminance SimilarityObject cues: Distance between tangent and model Angle between tangent and model Distance between tangent and nearest neighbouring tangent on dark side Intensity on dark side of tangentJ. Elder 31. Results 31Applications of Vision Science Part I: Contours and TexturesTested on 7 new lakes from IKONOS dataAverage 44% improvement in accuracy over NTDB vector dataAlgorithmHumanJ. Elder 32. Feedback: Prior Knowledge (Elder et al, PAMI 2003) 32Applications of Vision Science Part I: Contours and TexturesJ. Elder 33. Related Application: Skin Region Detection 33Applications of Vision Science Part I: Contours and TexturesJ. Elder 34. 34Application: Finding Major Skin BoundariesApplications of Vision Science Part I: Contours and TexturesJ. Elder 35. Applications of Contour Processing 35Applications of Vision Science Part I: Contours and Textures Background: Contour extraction Application 1. Interactive contour editing (ICE) Application 2. Demarcating water features in satellite imageryJ. Elder 36. 36Applications of Contour and Texture Processing Applications of Vision Science Part I: Contours and Textures Texture: Enhanced / Synthetic Vision Systems (ESVS) Application1. Optimal Texture Maps for ESVS Application 2. Understanding Texture FusionJ. Elder 37. The Challenge: Enhanced/Synthetic Vision for Aviation 37Applications of Vision Science Part I: Contours and TexturesJ. Elder 38. Motivation 38Applications of Vision Science Part I: Contours and TexturesEnhanced Synthetic Vision System (E/SVS) (Ricciardi, 2000)J. Elder 39. 39Applications of Contour and Texture Processing Applications of Vision Science Part I: Contours and Textures Texture: Enhanced / Synthetic Vision Systems (ESVS) Application1. Optimal Texture Maps for ESVS Application 2. Understanding Texture FusionVelisavljevic & Elder 2006 Vision Research. J. Elder 40. Surface Attitude from Texture 40Applications of Vision Science Part I: Contours and TexturesSlant: 45, Tilt: 45Slant: 45,Tilt: 135J. Elder 41. Objectives 41Applications of Vision Science Part I: Contours and TexturesObjective 1: Determine the optimal texture for a range of viewing distances. Objective 2: Determine if texture anisotropy biases tilt judgements.J. Elder 42. Experiment 1 - Textures 42Applications of Vision Science Part I: Contours and TexturesMulti-scale random (2D 1/f)Multi-scale random disksSingle-scale random (2D bandpass)Single-scale random disksMulti-scale random rectilinear (1D 1/f)Single-scale random rectilinear (1D bandpass)Multi-scale regular rectilinearSingle-scale regular rectilinear J. Elder 43. Experiment 1 43Applications of Vision Science Part I: Contours and TexturesMulti-scale random disks Slant: 40, Tilt: 90 Simulated distance: 26 mMulti-scale random disks Slant: 40, Tilt: 90 Simulated distance: 228 mJ. Elder 44. General Procedure 44Applications of Vision Science Part I: Contours and TexturesObservers indicated the perceived surface attitude using a mousecontrolled gauge figure superimposed on a textured plane.Rotation and tilt were randomly selected between -180 deg and 180 deg. Slant was randomly distributed between 40 deg and 60 deg in the first and third experiment but fixed at 60 deg in the second experiment.There were 20 random slant and tilt pairs for each condition.Textures were created from 2048x2048 pixel tiles unless otherwise noted.J. Elder 45. Experiment 1 Results 45Applications of Vision Science Part I: Contours and Textures 20 Multi-scale Single-scaleMean Slant Error (Degrees)10 0 -10 -20 Range: multi-scale-30Range: single-scale-40 -5010 010 2 Simulated Distance (Meters)10 4Multi-scale random rectilinear (1D 1/f) J. Elder 46. Experiment 2 - Procedure 46Applications of Vision Science Part I: Contours and TexturesTo test the effects of anisotropy, we used three textures:Multi-scale disks Multi-scale random rectilinearSingle-scale triangles Isotropic Anisotropic (90) Anisotropic (45)Texture rotation relative to tilt was randomly selected to be between [-45, -30, -15, 0, 15, or 30]. J. Elder 47. Experiment 2 - Procedure 47Applications of Vision Science Part I: Contours and TexturesMulti-scale random rectilinear Slant: 60, Tilt: 0, Rotation: 0 Tilt (0) - Rotation (0) = 0Multi-scale random rectilinear Slant: 60, Tilt: 0, Rotation: 30 Tilt (0) - Rotation (30) = -30 J. Elder 48. 48Bias and Precision in Tilt JudgementsApplications of Vision Science Part I: Contours and TexturesDespite bias induced by rectilinear structure, rectilinear textures yield more accurate tilt judgements.J. Elder 49. Conclusions 49Applications of Vision Science Part I: Contours and TexturesMulti-scale textures support accurate surface attitude judgements over a greater range of viewing distances than single-scale textures.Attitude judgements are best with structured textures. However, textures with anisotropic structure induce a bias in tilt judgements.J. Elder 50. 50Applications of Contour and Texture Processing Applications of Vision Science Part I: Contours and Textures Texture: Enhanced / Synthetic Vision Systems (ESVS) Application1. Optimal Texture Maps for ESVS Application 2. Understanding Texture FusionJ. Elder 51. Data fusion in the natural world 51Applications of Vision Science Part I: Contours and TexturesNatural images may contain mixtures of independent textures. These textures may project from the same surface, or they may project from distinct transparent surfaces.J. Elder 52. The Challenge: Enhanced/Synthetic Vision for Aviation 52Applications of Vision Science Part I: Contours and TexturesJ. Elder 53. Effects of Tilt Differences 54. Example 54Applications of Vision Science Part I: Contours and TexturesSlant = 35 Tilt A = 150 Tilt B2 = 60 Tilt Difference = 90J. Elder 55. When do observers perceive a single surface? 55Applications of Vision Science Part I: Contours and TexturesTexture AP(1 Surface Perceived)Perception of a single surface 1.2 Texture B11 0.8 0.6Texture B20.4 0.2 0 0306090120150180Tilt difference (deg)Texture B1Texture B2Texture B3Texture B3 J. Elder 56. Perception of 2 Distinct Surfaces 56Applications of Vision Science Part I: Contours and TexturesWhen 2 surfaces are perceived, what are their perceived attitudes? Mean Relative Perceived Tilt Relative Tilts for Textures120120909090606030 0 0306090120150180-60 -9030 0 -300306090120150180-60Relative tilt (deg)60 Relative tilt (deg)Relative tilt (deg)A &B3A & B2120-30Relative Tilts for TexturesRelative Tilts for TexturesA & B130 0 -300306090120150180-60 -90-90-120-120Tilt difference (deg) True Tilt of A Perceived Tilt of APerceived Tilt of B1-120Tilt Difference (Deg) Tilt difference (deg)True Tilt of B1Texture B1True Tilt of B2Perceived Tilt of ATexture ATrue Tilt of ATrue Tilt of ATrue Tilt of B3Perceived Tilt of APerceived Tilt of B3Perceived Tilt of B2Texture ATexture B2Texture ATexture B3J. Elder 57. Applications of Vision Science Part I: Contours and TexturesRelative tilt (deg)57Mean Perceived Tilt of Unitary Surface 6030 0306090-30 -60 True Tilt ATilt Difference (Deg) True Tilt B1 Perceived Surface pt. Prob. Model OJ. Elder 58. Modeling Fusion 58Applications of Vision Science Part I: Contours and TexturesThe percept of a unitary surface may arise for 1 of 2 reasons: Selection: each judgement based EITHER on information from Texture A OR Texture B (may change from trial to trial, subject to subject)Fusion: each judgement based on a fusion of information from both texturesJ. Elder 59. Selection or Fusion 59Applications of Vision Science Part I: Contours and TexturesSelection0.02Fusiondenotes true tilt of surface0.030.02DatapSelection A Selection B0.01Data p0.03Fusion 0.0100Relative tilt (deg) Distribution of perceived tilts modeled as a mixture of 2 Gaussians with means A, B equal to the true relative tilts of the 2 surfaces.Relative tilt (deg) Distribution of perceived tilts by a single Gaussian of unknown mean .J. Elder 60. Bayesian Model Selection 60Applications of Vision Science Part I: Contours and TexturesLog(p(fusion)/p(selection))* log 30 0, p